Data conversion device and method

Abstract

The invention provides a data conversion device and method, and belongs to the technical field of data processing, and the device comprises an inverse quantization module which is used for selecting a target quantization mode from a plurality of supported quantization modes and selecting a target quantization format from a plurality of supported quantization formats; the inverse quantization module is further used for obtaining a conversion mapping relation corresponding to a target quantization format in the target quantization mode, the conversion mapping relation is used for indicating a mapping relation between data of a first bit number and data of a second bit number, and the first bit number is smaller than the second bit number; the inverse quantization module is further used for converting the at least one piece of first data into second data based on the conversion mapping relation, the bit number of the first data is the first bit number, and the bit number of the second data is the second bit number. According to the invention, various quantization modes and quantization formats can be flexibly supported, and the evolution of the quantization technology is effectively supported under the condition that the quantization technology is not stable. The method and the device are used for converting the data with low bit number into the data with high bit number.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data conversion device and method. Background Art

[0002] Large models are machine learning models with numerous parameters and complex structures. They can process massive amounts of data and complete complex tasks such as natural language processing, computer vision, and speech recognition. However, the inference process of large models consumes significant amounts of video memory and computing power. To reduce resource consumption during large model inference and improve inference performance, model quantization technology has been proposed.

[0003] Model quantization technology involves converting large model data (including parameter data and activation data) from a high-precision format with a higher bit count to a lower-precision format with a lower bit count for storage and transmission. During inference, the data is restored (dequantized) to a high-precision format for calculation. Different parts of the large model data may have different optimal quantization forms. Therefore, using different data formats to represent different parts of the parameter data can achieve higher quantization accuracy.

[0004] Current quantization technology is evolving towards using different data formats to represent different parts of data. In order to ensure the evolution of quantization technology and improve quantization accuracy, there is an urgent need for an inverse quantization method that can support various quantization forms. Summary of the Invention

[0005] The present application provides a data conversion device and method, which solves the problem in related technologies of the lack of an inverse quantization method that can support various quantization forms. It can flexibly support various quantization modes and quantization formats, and effectively support the evolution of quantization technology when the quantization technology is not yet stable.

[0006] In a first aspect, the present application provides a data conversion device, which includes: an inverse quantization module, used to select a target quantization mode from multiple supported quantization modes, and select a target quantization format from multiple supported quantization formats; the inverse quantization module is also used to obtain a conversion mapping relationship corresponding to the target quantization format under the target conversion mode, and the conversion mapping relationship is used to indicate a mapping relationship between data of a first bit number and data of a second bit number, and the first bit number is less than the second bit number; the inverse quantization module is also used to convert at least one first data into second data based on the conversion mapping relationship, the bit number of the first data is the first bit number, and the bit number of the second data is the second bit number.

[0007] The first data is obtained by quantizing the original data using a target quantization format in a target quantization mode.

[0008] Its beneficial effect is that the inverse quantization process is realized through the hardware of the inverse quantization module, which supports the storage and transmission of data in a low-bit data format, reducing the data transmission bandwidth and storage overhead. The quantization mode and quantization format are configurable, and can flexibly support various quantization modes and quantization formats to achieve the conversion of low-bit data to high-bit data. When the quantization technology is not yet stable, it can effectively support the evolution of quantization technology. In the scenario of large models, it supports the storage and transmission of weights in a low-bit data format, which can reduce the occupancy of HBM and increase the transmission speed of weights, effectively supporting the rapid development of large models.

[0009] The multiple quantization modes include a normal mode and an outlier-victim pair (OVP) mode, and the multiple quantization formats include uniform quantization and non-uniform quantization.

[0010] For example, when the target conversion mode is the normal mode, the conversion mapping relationship may include a normal value conversion mapping relationship. The conversion mapping relationship corresponding to the uniform quantization format in the normal mode includes a normal value uniform quantization mapping relationship, and the conversion mapping relationship corresponding to the non-uniform quantization format in the normal mode includes a normal value non-uniform quantization mapping relationship.

[0011] When the target conversion mode is OVP mode, the conversion mapping relationship may include a normal value conversion mapping relationship and an outlier conversion mapping relationship. The conversion mapping relationship corresponding to the uniform quantization format in OVP mode includes a normal value uniform quantization mapping relationship and an outlier uniform quantization mapping relationship. The conversion mapping relationship corresponding to the non-uniform quantization format in OVP mode includes a normal value non-uniform quantization mapping relationship and an outlier non-uniform quantization mapping relationship.

[0012] In one possible implementation, the target conversion mode is the OVP mode, and the inverse quantization module is further used to: obtain at least one first data from a quantization data matrix, the data in the quantization data matrix includes the following three types: a quantization value of a normal value, a quantization value of an abnormal value, and an abnormal value flag; wherein the abnormal value flag is obtained by replacing the sacrifice value before quantization, and the abnormal value flag is used to indicate that a data in the quantization data matrix is ​​a quantization value of an abnormal value.

[0013] Among them, the normal value is within the preset range, and the abnormal value is not within the preset range.

[0014] In one possible implementation, when the target conversion mode is the OVP mode, the conversion mapping relationship includes a normal value conversion mapping relationship and an abnormal value conversion mapping relationship, and the at least one first data includes two adjacent first data in the quantized data matrix. The dequantization module is specifically configured to: respectively determine whether the two first data are abnormal value flags; based on the determination result of whether the two first data are abnormal value flags, determine the target conversion mapping relationship corresponding to the two first data, respectively, and whether the target conversion mapping relationship is a normal value conversion mapping relationship or an abnormal value conversion mapping relationship; and determine the value mapped by the first data in the target conversion mapping relationship as the second data.

[0015] In one possible implementation, when neither of the two first data is an outlier flag, the normal value conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the two first data respectively; when both of the first data are outlier flags, the outlier conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the two first data respectively; when the two first data are outlier flags and non-outlier flags respectively, the normal value conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the outlier flag, and the outlier conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the non-outlier flag.

[0016] In the normal value conversion mapping relationship, the value mapped to the non-abnormal value flag bit is the normal value, and the value mapped to the abnormal value flag bit is the normal sacrifice value. In the abnormal value conversion mapping relationship, the value mapped to the non-abnormal value flag bit is the abnormal value, and the value mapped to the abnormal value flag bit is the abnormal sacrifice value.

[0017] The beneficial effect is that when the target quantization mode is the OVP mode, the outlier flag is configured as a valid value, which can be the average of the smaller values ​​in the OVP value. Thus, a second bit value can be obtained for the outlier flag based on the conversion mapping relationship, which improves the utilization rate of the conversion mapping relationship compared to related technologies.

[0018] In one possible implementation, the normal sacrifice value in the normal value conversion mapping relationship and the abnormal sacrifice value in the abnormal value conversion mapping relationship can be interchanged. In this way, in the OVP mode, when the two first data are both abnormal value flag bits, since the abnormal sacrifice values ​​to which the two first data should be mapped are located in the normal value conversion mapping relationship, the normal value conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the two first data respectively, that is, the two data can obtain the second data by querying the normal value conversion mapping relationship.

[0019] When the two first data are respectively an outlier flag and a non-outlier flag, since the normal sacrifice value to which the outlier flag should be mapped is located in the outlier conversion mapping relationship, the outlier value to which the non-outlier flag should be mapped is still located in the outlier conversion mapping relationship, the outlier conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the two first data respectively, that is, the second data can be obtained by querying the outlier conversion mapping relationship for both data.

[0020] The beneficial effect thereof is that in the OVP mode, the second data can be obtained only through the normal value conversion mapping relationship or the abnormal value conversion mapping relationship.

[0021] In a possible implementation, the conversion mapping relationship may be in a table format, and the normal value conversion mapping relationship and the abnormal value conversion mapping relationship may be two independent tables; or the normal value conversion mapping relationship and the abnormal value conversion mapping relationship may be two clusters in one table.

[0022] In a possible implementation, the target conversion mode is a normal mode, and the conversion mapping relationship includes a normal value conversion mapping relationship; the dequantization module is specifically configured to determine the value of the first data mapping in the normal value conversion mapping relationship as the second data.

[0023] In one possible implementation, the device also includes: a storage module and a computing module, the inverse quantization module is located on the data path between the storage module and the computing module, or is located at the entrance or inside the computing module; the storage module is used to store at least one first data; the inverse quantization module is also used to obtain at least one first data from the storage module; the inverse quantization module is also used to send second data to the computing module; and the computing module is used to process the second data.

[0024] In a second aspect, the present application provides a data conversion method, which includes: selecting a target quantization mode from multiple supported quantization modes, and selecting a target quantization format from multiple supported quantization formats; obtaining a conversion mapping relationship corresponding to the target quantization format under the target quantization mode, the conversion mapping relationship being used to indicate a mapping relationship between data of a first bit number and data of a second bit number, the first bit number being less than the second bit number; based on the conversion mapping relationship, converting at least one first data into second data, the bit number of the first data being the first bit number, and the bit number of the second data being the second bit number.

[0025] In a possible implementation, the multiple quantization modes include a normal mode and an abnormal-victim pair OVP mode.

[0026] In a possible implementation, the multiple quantization formats include uniform quantization and non-uniform quantization.

[0027] In one possible implementation, the target quantization mode is an OVP mode, and the method further includes: obtaining at least one first data from a quantization data matrix, the data in the quantization data matrix including the following three types: a quantization value of a normal value, a quantization value of an abnormal value, and an abnormal value flag; wherein the abnormal value flag is obtained by replacing the sacrifice value before quantization, and the abnormal value flag is used to indicate that a data in the quantization data matrix is ​​a quantization value of an abnormal value.

[0028] In a possible implementation, the normal value is within a preset range, and the abnormal value is not within the preset range.

[0029] In one possible implementation, the conversion mapping relationship includes a normal value conversion mapping relationship and an outlier conversion mapping relationship, and at least one first data includes two adjacent first data in a quantized data matrix; the process of converting at least one first data into second data based on the conversion mapping relationship includes: respectively judging whether the two first data are outlier flags; based on the judgment result of whether the two first data are outlier flags, determining the target conversion mapping relationship corresponding to the two first data respectively, the target conversion mapping relationship is a normal value conversion mapping relationship or an outlier conversion mapping relationship; and determining the value mapped by the first data in the target conversion mapping relationship as the second data.

[0030] In one possible implementation, based on the judgment result of whether the two first data are outlier flags, the process of determining the target conversion mapping relationship corresponding to the two first data includes: when both the two first data are not outlier flags, the normal value conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the two first data; when both the first data are outlier flags, the outlier conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the two first data; when the two first data are outlier flags and non-outlier flags respectively, the normal value conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the outlier flag, and the outlier conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the non-outlier flag.

[0031] In one possible implementation, based on the judgment result of whether the two first data are outlier flags, the process of determining the target conversion mapping relationship corresponding to the two first data includes: when both the two first data are not outlier flags, the normal value conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the two first data; when both the first data are outlier flags, the normal value conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the two first data; when the two first data are outlier flags and non-outlier flags respectively, the outlier conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the two first data.

[0032] In one possible implementation, in the normal value conversion mapping relationship, the value mapped by the abnormal value flag bit is the normal sacrifice value, and the value mapped by the non-abnormal value flag bit is the normal value; in the abnormal value conversion mapping relationship, the value mapped by the abnormal value flag bit is the abnormal sacrifice value, and the value mapped by the non-abnormal value flag bit is the abnormal value.

[0033] In a possible implementation, the conversion mapping relationship is in a table format, and the normal value conversion mapping relationship and the outlier conversion mapping relationship are two independent tables; or the normal value conversion mapping relationship and the outlier conversion mapping relationship are two clusters in one table.

[0034] In one possible implementation, the target conversion mode is a normal mode, and the conversion mapping relationship includes a normal value conversion mapping relationship; the process of converting at least one first data into second data based on the conversion mapping relationship includes: determining the value of the first data mapping in the normal value conversion mapping relationship as the second data.

[0035] In a third aspect, the present application provides a data conversion device, which includes: a selection module for selecting a target quantization mode from multiple supported quantization modes, and selecting a target quantization format from multiple supported quantization formats; an acquisition module for acquiring a conversion mapping relationship corresponding to the target quantization format under the target quantization mode, the conversion mapping relationship being used to indicate a mapping relationship between data of a first bit number and data of a second bit number, the first bit number being less than the second bit number; a conversion module for converting at least one first data into second data based on the conversion mapping relationship, the bit number of the first data being the first bit number, and the bit number of the second data being the second bit number.

[0036] In a possible implementation, the multiple quantization modes include a normal mode and an abnormal-victim pair OVP mode.

[0037] In a possible implementation, the multiple quantization formats include uniform quantization and non-uniform quantization.

[0038] In one possible implementation, the target quantization mode is the OVP mode, and the acquisition module is further used to obtain at least one first data from the quantization data matrix. The data in the quantization data matrix includes the following three types: the quantization value of the normal value, the quantization value of the abnormal value, and the abnormal value flag; wherein the abnormal value flag is obtained by replacing the sacrifice value before quantization, and the abnormal value flag is used to indicate that a data in the quantization data matrix is ​​the quantization value of the abnormal value.

[0039] In a possible implementation, the normal value is within a preset range, and the abnormal value is not within the preset range.

[0040] In one possible implementation, the conversion mapping relationship includes a normal value conversion mapping relationship and an abnormal value conversion mapping relationship, and at least one first data includes two adjacent first data in a quantized data matrix; the conversion module is specifically used to: respectively determine whether the two first data are abnormal value flags; based on the judgment result of whether the two first data are abnormal value flags, determine the target conversion mapping relationship corresponding to the two first data respectively, the target conversion mapping relationship is a normal value conversion mapping relationship or an abnormal value conversion mapping relationship; determine the value mapped by the first data in the target conversion mapping relationship as the second data.

[0041] In one possible implementation, the conversion module is specifically used to: when both of the first data are not outlier flags, determine the normal value conversion mapping relationship as the target conversion mapping relationship corresponding to the two first data respectively; when both of the first data are outlier flags, determine the outlier conversion mapping relationship as the target conversion mapping relationship corresponding to the two first data respectively; when the two first data are outlier flags and non-outlier flags respectively, determine the normal value conversion mapping relationship as the target conversion mapping relationship corresponding to the outlier flag, and determine the outlier conversion mapping relationship as the target conversion mapping relationship corresponding to the non-outlier flag.

[0042] In one possible implementation, the conversion module is specifically used to: when both of the first data are not outlier flags, determine the normal value conversion mapping relationship as the target conversion mapping relationship corresponding to the two first data respectively; when both of the first data are outlier flags, determine the normal value conversion mapping relationship as the target conversion mapping relationship corresponding to the two first data respectively; when the two first data are outlier flags and non-outlier flags respectively, determine the outlier conversion mapping relationship as the target conversion mapping relationship corresponding to the two first data respectively.

[0043] In one possible implementation, in the normal value conversion mapping relationship, the value mapped by the abnormal value flag bit is the normal sacrifice value, and the value mapped by the non-abnormal value flag bit is the normal value; in the abnormal value conversion mapping relationship, the value mapped by the abnormal value flag bit is the abnormal sacrifice value, and the value mapped by the non-abnormal value flag bit is the abnormal value.

[0044] In a possible implementation, the conversion mapping relationship is in a table format, and the normal value conversion mapping relationship and the outlier conversion mapping relationship are two independent tables; or the normal value conversion mapping relationship and the outlier conversion mapping relationship are two clusters in one table.

[0045] In a possible implementation, the target conversion mode is a normal mode, and the conversion mapping relationship includes a normal value conversion mapping relationship: a conversion module is specifically configured to determine a value mapped by the first data in the normal value conversion mapping relationship as the second data.

[0046] In a fourth aspect, the present application provides a data conversion device, which includes: one or more processors; a memory for storing one or more computer programs or instructions; when the one or more computer programs or instructions are executed by one or more processors, the one or more processors implement a method as described in any one of the second aspects.

[0047] In a fifth aspect, the present application provides a data conversion device, comprising a processor for executing the method as described in any one of the second aspects.

[0048] In a sixth aspect, the present application provides a data conversion device, which includes: a processing circuit and an interface circuit; wherein the interface circuit is used to couple with a memory outside the data conversion device and provide a communication interface for the processing circuit to access the memory; the processing circuit is used to execute program instructions in the memory to implement a method as described in any one of the second aspects.

[0049] In a specific implementation, the data conversion device may be a chip, the input circuit may be an input pin, the output circuit may be an output pin, and the processing circuit may be a transistor, a gate circuit, a trigger, or various logic circuits. The input signal received by the input circuit may be, for example, but not limited to, received and input by a receiver, and the signal output by the output circuit may be, for example, but not limited to, output to and transmitted by a transmitter. The input circuit and the output circuit may be the same circuit, which functions as an input circuit and an output circuit at different times. The embodiments of the present application do not limit the specific implementation of the processor and various circuits.

[0050] In a seventh aspect, the present application provides a computer-readable storage medium, in which program code is stored. When the program code is executed by a processor, the method as described in any one of the second aspects is implemented.

[0051] In an eighth aspect, the present application provides a chip, comprising: at least one processor. The at least one processor is configured to execute the method according to any one of the second aspects.

[0052] Optionally, the chip further includes a memory, and at least one processor is configured to execute code in the memory. When the at least one processor executes the code, the chip implements the method as described in any one of the second aspects.

[0053] Optionally, the chip may also be an integrated circuit.

[0054] In a ninth aspect, the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to implement the method as described in any one of the second aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A schematic diagram of the structure of a chip provided in an embodiment of the present application;

[0056] Figure 2 A schematic diagram of an OVP quantification method provided in an embodiment of the present application;

[0057] Figure 3 A schematic diagram of the structure of a data conversion device provided in an embodiment of the present application;

[0058] Figure 4 A schematic diagram of the structure of another data conversion device provided in an embodiment of the present application;

[0059] Figure 5 A flowchart of a data conversion method provided in an embodiment of the present application;

[0060] Figure 6 A schematic diagram of data conversion in an OVP mode provided in an embodiment of the present application;

[0061] Figure 7 A flowchart of a data conversion method provided in an embodiment of the present application;

[0062] Figure 8 A schematic diagram of a normal value conversion mapping table and an abnormal value conversion mapping table provided in an embodiment of the present application;

[0063] Figure 9 A schematic diagram of a conversion process in an OVP mode provided in an embodiment of the present application;

[0064] Figure 10 A schematic diagram of a conversion process under another OVP mode provided in an embodiment of the present application;

[0065] Figure 11 A schematic diagram of an application scenario provided in an embodiment of the present application;

[0066] Figure 12 A schematic diagram of another application scenario provided by an embodiment of the present application;

[0067] Figure 13 A block diagram of a data conversion device provided in an embodiment of the present application;

[0068] Figure 14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0069] Figure 15 A schematic diagram of the structure of a data conversion device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0071] The terms "first," "second," and the like in the description, embodiments, claims, and drawings of this application are used solely for descriptive purposes and are not to be construed as indicating or implying relative importance or order. Furthermore, the terms "including," "having," and any variations thereof are intended to cover non-exclusive inclusions, such as, for example, inclusion of a series of steps or units. A method, system, product, or apparatus is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0072] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0073] Large models mainly use parameter data and activation data during the inference process. Parameter data includes weights. The parameter data has a large capacity. For example, for a generative pre-trained transformer 3 (GPT3) with 17.5 billion parameters, if the parameter data is formatted as half-precision floating point 16 (FP16), more than 300 gigabytes (GB) of space are required to store the parameter data. Furthermore, the parameter data has a small data range, so the accuracy loss can be controlled using a low-bit quantization compression method. However, the activation data has a large data range, so using a low-bit quantization compression method will result in a significant accuracy loss. Furthermore, for large models that use autoregressive reasoning (such as GPT-like decoding modes), the memory access required for parameter data is much greater than for activation data. Therefore, it is necessary to utilize the different data characteristics and different loads of parameter and activation data to perform model compression while retaining data accuracy and enabling hardware acceleration.

[0074] Model quantization can be combined with high-precision activation data to achieve a compromise between inference accuracy and performance. For example, in asymmetric quantization modes (W4A16 and W4A8), parameter data is stored in a low-precision format with a relatively low number of bits, while activation data is stored in a high-precision format with a relatively high number of bits. During inference, the parameter data is dequantized (reduced) to high-precision data and then combined with the activation data for calculation.

[0075] For example, please refer to Figure 1 , Figure 1 This is a schematic diagram of the structure of a chip provided in an embodiment of the present application. The chip 100 includes a memory module 101, a decoding unit 102, and a computing core 103. The memory module 101 stores activation data, weights (weight, W), and quantization parameters. The weights here are quantized based on the quantization parameters. For example, the weights can be 4 bits and the activation data can be 16 bits. When performing inference, the decoding unit 102 dequantizes the weights into high-bit weights based on the quantization parameters, and then sends them to the computing core 103. The computing core 103 reads the activation data from the memory module 101 and performs inference calculations based on the dequantized weights and activation data.

[0076] Current quantization compression schemes use methods such as uniform and non-uniform quantization to perform offline quantization on data, while also using a small amount of data for calibration to compensate for quantization errors. After quantization, the data is converted from its original high-bit format to a lower-bit format, such as from FP16 to an 8-bit signed integer (int8) or a 4-bit signed integer (int4).

[0077] When the number of data bits after quantization compression is small (for example, 4 bits), the range of data that can be represented is limited (for example, 4 bits can only represent the data range of 0 to 15). If a fixed quantization compression method is used (such as INT4 or post-training optimization tool (POT)), it will cause a large accuracy error. In addition, different parts of the model data may have different optimal quantization formats. For example, most of the parameter data in the parameter data meet a normal distribution, but there are also a small number of outliers with large values. These outliers also contribute greatly to the model inference accuracy. The fixed quantization compression method cannot take into account outliers and normal values ​​(values ​​that meet a normal distribution) well, and its generalization and compatibility are poor. If a non-fixed quantization compression method is used (for example, a clustering algorithm clusters 16 quantization points), there is a problem of too long optimization time.

[0078] To improve inference performance while ensuring inference accuracy, it is possible to consider combining data characteristics and using different quantization and compression methods for different parts of the data. Related technologies have proposed a 4-bit quantization technique called OVP. The OVP quantization method divides data into outliers and normal values ​​according to a preset range. Data within the preset range is considered a normal value, while values ​​outside the preset range are outliers. The adjacent values ​​of each outlier are replaced with an outlier flag to mark the data at a certain position as an outlier. The outlier flag can be considered a victim value. The outliers and normal values ​​are then quantized separately, using either the same or different quantization methods. This reduces the numerical representation range of each interval, improving the quantization compression range and quantization accuracy.

[0079] For example, please refer to Figure 2 , Figure 2 A schematic diagram of an OVP quantification method provided in an embodiment of the present application is shown in FIG. Figure 2 The 4×3 weight matrix is ​​shown, with the 12 weights divided into outliers and normal values. The neighboring values ​​of an outlier are treated as sacrifices, and each sacrifice is replaced with a 4′b1000 flag bit to mark the value at a position as an outlier, thereby distinguishing outliers from normal values. The neighboring values ​​of an outlier include its row neighbors and its column neighbors.

[0080] Figure 2Taking the preset range of [-∞, 16] as an example, based on this preset range, 1.5 and 2.6 in the first two columns of the first row, 7.1 and -6.8 in the last two columns of the second row, and 1.2 and 6.3 in the first two columns of the third row are all classified as normal values. -98 in the first row, 17.6 in the second row, and 30.7 in the third row are all abnormal values. The values ​​adjacent to the left of -98, the values ​​adjacent to the right of 17.6, and the values ​​adjacent to the right of 30.7 are all 4′b1000 flag bits. Among them, the value adjacent to the left of -98 is used to mark -98 as an abnormal value, the value adjacent to the right of 17.6 is used to mark 17.6 as an abnormal value, and the value adjacent to the right of 30.7 is used to mark 30.7 as an abnormal value.

[0081] It should be noted that Figure 2 This is merely an example, and two adjacent values ​​(adjacent rows or adjacent columns) may both be abnormal values, which is not limited in this embodiment of the present application.

[0082] Current quantization technology is not yet stable and is evolving towards using different data formats to represent different parts of the data. This involves a variety of quantization methods. To ensure the evolution of quantization technology and improve quantization accuracy, a dequantization method that supports various quantization formats is urgently needed.

[0083] The embodiment of the present application provides a data conversion device for converting low-bit data into high-bit data. The device can be used for the reasoning process of large models, such as the reasoning structure of large language models (LLM) and generative artificial intelligence (AIGC) models. The product form of the device can be an AI training and reasoning hardware platform, an artificial intelligence (AI) chip, a field programmable gate array (FPGA) chip, a dedicated domain architecture processor (DSA), an application specific integrated circuit (ASIC) chip or a general-purpose processor. The reasoning process can, for example, include the reasoning process involved in AI chips / FPGA chips / DSA / ASIC chips / general-purpose processors. It can also be applied to the compression and transmission process of data, such as compression of interconnected communication between servers, compression transmission of network data, and compression technologies such as image and text.

[0084] For example, the data conversion device may include an inverse quantization module for converting the format of data. Optionally, the device may further include: a storage module and a computing module. The inverse quantization module may be located on a data path between the storage module and the computing module, or may be located at an entrance or within the computing module. The storage module may include a high bandwidth memory (HBM) and a cache. The computing module may include a general matrix multiplication (GEMM) computing module.

[0085] Please refer to Figure 3 , Figure 3 This is a schematic diagram of the structure of a data conversion device provided in an embodiment of the present application. The data conversion device 300 includes an HBM 301, a cache 302, a dequantization module 303, and a calculation module 304. The HBM 301 and cache 302 are storage modules, and the dequantization module 303 is located between the storage module and the calculation module 304. In this structure, the dequantization module can complete data format conversion during data transmission, that is, perform data format conversion before the data is transmitted to the calculation module 304, reducing time overhead.

[0086] Please refer to Figure 4 , Figure 4 This is a schematic diagram of the structure of another data conversion device provided in an embodiment of the present application. The data conversion device 400 includes an HBM 401, a cache 402, a dequantization module 403, and a calculation module 404. HBM 401 and cache 402 are storage modules. Dequantization module 403 is located at the entrance of calculation module 404, completing data format conversion after data is transferred to calculation module 404 and before calculation module 404 performs calculations based on the data.

[0087] It should be noted that the aforementioned Figure 3 and Figure 4 The structure shown is only for illustrative purposes. The embodiments of the present application do not limit the modules and specific structures included in the data conversion device. For example, the data conversion device may only include an inverse quantization module, or the data conversion device may not include a cache, etc.

[0088] The present application provides a data conversion method, which can be performed by the aforementioned data conversion device, for example, Figure 3 or Figure 4 The data conversion device shown is executed. For example, please refer to Figure 5 , Figure 5 A data conversion method according to an embodiment of the present invention may include the following steps:

[0089] 501. Select a target quantization mode from multiple supported quantization modes, and select a target quantization format from multiple supported quantization formats.

[0090] This process can be performed by an inverse quantization module. Among them, the multiple quantization modes supported by the data conversion device may include a normal mode and an OVP mode, and the target quantization mode is a normal mode or an OVP mode. The normal mode means that all data adopt the same quantization method, and the quantization process of each data is independent of each other. As mentioned above, the OVP mode, when quantizing data (such as weights) into 4 bits, will split the data into outliers, normal values ​​and outlier flags, and the quantized data (such as quantization weights) includes the following three types of data: the quantization value of the normal value, the quantization value of the outlier value and the outlier flag. The OVP mode is to quantize the outlier and the normal value separately, and the two adjacent data (two adjacent data in the row or two adjacent data in the column) are coupled together for quantization.

[0091] The multiple quantization formats supported by the data conversion device may include uniform quantization and non-uniform quantization, and the target quantization format is uniform quantization or non-uniform quantization. Uniform quantization involves dividing the data into several equally spaced intervals, quantizing the data in each interval, and obtaining discrete quantized data. In this process, the width of each interval is equal, meaning that the quantization accuracy is uniformly distributed. Non-uniform quantization involves dividing the data into several unequally spaced intervals, quantizing the data in each interval, and obtaining discrete quantized data. In this process, the width of each interval is unequal, meaning that the quantization accuracy is unevenly distributed.

[0092] The target conversion mode and target quantization format refer to the methods used when quantizing data. The inverse quantization module can determine the target conversion mode and target quantization format based on the quantization process, or they can be directly configured to the inverse quantization module by the user. The embodiments of this application do not limit this.

[0093] 502. Obtain a conversion mapping relationship corresponding to a target quantization format in a target conversion mode, where the conversion mapping relationship is used to indicate a mapping relationship between data of a first number of bits and data of a second number of bits, where the first number of bits is smaller than the second number of bits.

[0094] The conversion mapping relationship can express the mapping relationship between the data of the first number of bits and the data of the second number of bits without any format constraints, which can be represented in a tabular form. For example, when the target conversion mode is the normal mode, the conversion mapping relationship may include a normal value conversion mapping relationship (e.g., a normal value conversion mapping table). Specifically, the conversion mapping relationship corresponding to the uniform quantization format under the normal mode includes a normal value uniform quantization mapping relationship, and the conversion mapping relationship corresponding to the non-uniform quantization format under the normal mode includes a normal value non-uniform quantization mapping relationship.

[0095] When the target conversion mode is OVP mode, the conversion mapping relationship may include a normal value conversion mapping relationship and an outlier conversion mapping relationship (e.g., an outlier conversion mapping table). The conversion mapping relationship corresponding to the uniform quantization format under OVP mode includes a normal value uniform quantization mapping relationship and an outlier uniform quantization mapping relationship. The conversion mapping relationship corresponding to the non-uniform quantization format under OVP mode includes a normal value non-uniform quantization mapping relationship and an outlier non-uniform quantization mapping relationship.

[0096] For example, the normal value conversion mapping relationship and the abnormal value conversion mapping relationship can be two independent tables or two clusters in one table. The embodiment of the present application does not limit the form of these two conversion mapping relationships.

[0097] For the conversion mapping relationship corresponding to the target quantization format under the target conversion mode, in one example, all conversion mapping relationships (such as normal value uniform quantization mapping relationship, normal value non-uniform quantization mapping relationship, abnormal value uniform quantization mapping relationship and abnormal value non-uniform quantization mapping relationship) can be pre-stored in a data conversion device (such as a storage module). After determining the target conversion mode and the target quantization format, the inverse quantization module obtains the corresponding conversion mapping relationship from the stored conversion mapping relationship. In another example, the user can directly configure the corresponding conversion mapping relationship to the inverse quantization module, which is not limited in the embodiments of the present application.

[0098] The first bit number is the number of bits of the quantized data, for example, it can be 4 bits or 8 bits, etc. The data format of the first bit number can be int4 (or int8) or other special 4-bit (or 8-bit) formats. The second bit number can be 8, 16 or 32, etc. The data format of the second bit number in the conversion mapping relationship can be an 8-bit integer format (int8), an 8-bit floating-point data format, a 16-bit floating-point data format or a 32-bit floating-point number, etc. 8-bit floating-point data formats may include FP8 and HiF8 (HiFloat8). FP8 may include E4M3 and E5M2, for example, where E indicates the number of bits representing the exponent in FP8, and M indicates the number of bits representing the mantissa in FP8. E4M3 means that FP8 has 4 bits representing the exponent and 3 bits representing the mantissa; E5M2 means that FP8 has 5 bits representing the exponent and 2 bits representing the mantissa. The 16-bit floating point data format may include FP16 and 16-bit brain floating point 16 (BF16), etc. The 32-bit floating point number may include float32.

[0099] The conversion mapping relationship can be a display lookup table (LUT) with Si as the query address, where Si is the complement of the i bit. For example, when the first bit number is 4 and the second bit number is 8, the conversion mapping relationship is an 8-to-1 LUT with S4 (i=4) as the query address. When the first bit number is 4 and the second bit number is 16, the conversion mapping relationship is a 16-to-1 LUT with S4 as the query address. S4 is a 4-bit complement with an encoding range of -8 to 7.

[0100] 503. Convert at least one first data into second data based on the conversion mapping relationship, where the number of bits of the first data is a first number of bits, and the number of bits of the second data is a second number of bits.

[0101] The first data is obtained by quantizing the original data in the target quantization format in the target quantization mode. At least one first data can be stored in the storage module, for example Figure 3 or Figure 4 In the HBM or cache shown. For example, the first data may be data in a quantized data matrix (e.g., a quantized weight matrix), and the data in the quantized data matrix includes the following three types: quantized values ​​of normal values, quantized values ​​of abnormal values, and abnormal value flags. The abnormal value flags are obtained by replacing the sacrificed values ​​before quantization, and each abnormal value flag is used to indicate that a data in the quantized data matrix is ​​a quantized value of an abnormal value.

[0102] This process 503 can be considered the inverse process of quantization (i.e., inverse quantization). When the target quantization mode is the normal mode, there is no correlation between the first data. Therefore, in process 503, each first data can be converted into second data respectively, and at least one first data can include only one first data. As described in the above process 502, the conversion mapping relationship includes a normal value conversion mapping relationship. The inverse quantization module determines the value mapped to the first data in the normal value conversion mapping relationship as the second data.

[0103] When the target quantization mode is the OVP mode, it can be seen from the above description that in the process of quantization using the OVP mode, two adjacent first data (row adjacent or column adjacent) in the data matrix are quantized in an associated manner to obtain a quantized data matrix. Therefore, in process 503, it is necessary to associate and convert the two adjacent first data in the quantized data matrix (i.e., inverse quantization), that is, at least one first data includes two adjacent data in the quantized data matrix. The inverse quantization module first determines whether the two first data are outlier flags. Then, based on the judgment result of whether the two first data are outlier flags, the target conversion mapping relationship corresponding to the two first data is determined, and then the value mapped to the first data in the target conversion mapping relationship is determined as the second data. The target conversion mapping relationship is a normal value conversion mapping relationship or an outlier conversion mapping relationship in the conversion mapping relationship. For the outlier flag, taking the first bit number as 4 bits as an example, the outlier flag can be -8 (b1000).

[0104] When neither of the two first data is an abnormal value flag, the normal value conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the two first data respectively.

[0105] In one example, when both first data have outlier flags, the outlier conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the two first data. When the two first data have outlier flags and non-outlier flags, respectively, the normal value conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the outlier flag, and the outlier conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the non-outlier flag.

[0106] In the normal value conversion mapping relationship, the encoding value of the non-outlier value flag bit mapping is the normal value, and the encoding value of the outlier value flag bit mapping is the normal victim value. The normal victim value can be understood as the normal data sacrificed as the outlier value flag bit in the data matrix of the aforementioned OVP quantization mode. In the outlier value conversion mapping relationship, the encoding value of the non-outlier value flag bit mapping is the outlier value, and the encoding value of the outlier value flag bit mapping is the outlier victim value. The abnormal victim value can be understood as the abnormal data sacrificed as the outlier value flag bit in the data matrix of the aforementioned OVP quantization mode. The difference between the absolute value of the abnormal victim value or the abnormal value and the absolute value of the normal value is large, for example, greater than the first threshold. The difference between the absolute values ​​of any two normal values ​​or the normal value and the normal victim value is small, for example, less than or equal to the first threshold.

[0107] Taking the conversion mapping relationship as a LUT as an example, assuming the outlier flag is represented by F, then in the normal value conversion mapping table, the address !F (non-outlier flag) stores the normal value, and the address F stores the normal sacrifice value. In the outlier conversion mapping table, the address !F stores the outlier value, and the address F stores the outlier sacrifice value.

[0108] In another example, the normal sacrifice value in the normal value conversion mapping relationship can be interchanged with the abnormal sacrifice value in the abnormal value conversion mapping relationship. In this way, in the OVP mode, when the two first data are both abnormal value flag bits, since the abnormal sacrifice values ​​that the two first data should be mapped to are in the normal value conversion mapping relationship, the normal value conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the two first data respectively, that is, the two data can obtain the second data by querying the normal value conversion mapping relationship.

[0109] When the two first data are respectively an outlier flag and a non-outlier flag, since the normal sacrifice value to which the outlier flag should be mapped is located in the outlier conversion mapping relationship, the outlier value to which the non-outlier flag should be mapped is still located in the outlier conversion mapping relationship, the outlier conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the two first data respectively, that is, the second data can be obtained by querying the outlier conversion mapping relationship for both data.

[0110] Please refer to Figure 6 , Figure 6 This is a data conversion diagram under an OVP mode provided in an embodiment of the present application. Figure 6 Taking the conversion mapping relationship as LUT and the first bit number as 4 bits as an example, in the OVP mode, two adjacent first data are required to share a set of table lookup parameters. The LUT table lookup parameter is S4, and a set of table lookup parameters is 8 bits.

[0111] Figure 6Four combinations of two adjacent first data are shown, along with the corresponding table lookup methods for the four combinations. When the combination of two adjacent first data is !F and !F, the conversion mapping relationships corresponding to the two first data are both the normal value conversion mapping table, and the table lookup methods corresponding to the two first data are both N, where N represents a normal value in the normal value conversion mapping table. When the combination of two adjacent first data is F and !F, the target conversion mapping relationship corresponding to F is the normal value conversion mapping table, and the target conversion mapping relationship corresponding to !F is the abnormal value conversion mapping table. The table lookup method corresponding to F is Nv, where Nv represents a normal sacrifice value in the normal value conversion mapping table. The table lookup method corresponding to !F is O, where O represents an abnormal value in the abnormal value conversion mapping table. When the combination of two adjacent first data is F and F, the target conversion mapping relationships corresponding to the two first data are both the abnormal value conversion mapping table, and the table lookup methods corresponding to the two first data are both Ov, where Ov represents an abnormal sacrifice value in the abnormal value conversion mapping table.

[0112] All first data may be in the form of a matrix, and the data conversion device may obtain the first data from the matrix according to a column / group size (grp size). Two adjacent first data in the obtained first data may be two first data adjacent in a row, or two first data adjacent in a column. The first data obtained according to the column / grp size share the same conversion mapping relationship.

[0113] Through this process 503, first data of a first bit number is converted into second data of a second bit number, for example, 4-bit first data is converted into 8-bit, 16-bit or 32-bit second data, or 8-bit first data is converted into 16-bit or 32-bit second data.

[0114] In this process 503, when the target quantization mode is the OVP mode, the value of the abnormal value flag bit mapping is configured as a valid value in the normal value conversion mapping relationship and the abnormal value conversion mapping relationship. The valid value can be the average value of the smaller values ​​of the various encoding values ​​of the conversion mapping relationship. Thus, a second bit value can also be obtained for the abnormal value flag bit based on the conversion mapping relationship, which improves the utilization rate of the conversion mapping relationship compared to the related art. For example, for a 16-choose-1 LUT, the embodiment of the present application can query 16 values, which can achieve a 16 / 16 table lookup utilization rate.

[0115] Please refer to Figure 7 , Figure 7 A flowchart of a data conversion method provided in an embodiment of the present application is provided. Figure 7 Take the conversion mapping relationship in table form as an example for explanation. Figure 7As shown, select the target quantization mode and the target quantization format. Input at least one 4-bit first data and determine whether the target quantization mode is the OVP mode. If it is the OVP mode, perform OVP quantization decoding. The OVP quantization decoding process includes determining the target conversion mapping tables corresponding to two adjacent first data based on the outlier flag. Then query the target conversion mapping table, which is a normal value non-uniform quantization mapping table, a normal value uniform quantization mapping table, an outlier non-uniform quantization mapping table, or an outlier uniform quantization mapping table. Finally, output the 8-bit or 16-bit second data queried from the target conversion mapping table. If it is not the OVP mode, directly query the normal value non-uniform quantization mapping table or the normal value uniform quantization mapping table, and finally output the 8-bit or 16-bit second data obtained by the query.

[0116] Please refer to Figure 8 , Figure 8 A schematic diagram of a normal value conversion mapping table and an abnormal value conversion mapping table provided in an embodiment of the present application, Figure 8 The normal value conversion mapping table and abnormal value conversion mapping table in FIG correspond to 4-bit codes b0000 to b1111 from left to right and from top to bottom, respectively. The normal sacrifice value is -128, and the abnormal sacrifice value is 0.15. Figure 8 The normal value conversion mapping table and abnormal value conversion mapping table shown are only exemplary, and the values ​​and table sizes in the table are not limiting. For example, the normal sacrifice value can also be 0.35, and the abnormal sacrifice value can also be 78.

[0117] Please refer to Figure 9 , Figure 9 A schematic diagram of a conversion process in an OVP mode provided in an embodiment of the present application. Figure 9 An implementation manner in which the normal sacrifice value in the normal value conversion mapping relationship and the abnormal sacrifice value in the abnormal value conversion mapping relationship are not interchanged is described.

[0118] Assume that the abnormal value flag is b1000. When the two adjacent first data input are b0001 and b0010, the two first data are a combination of !F and !F. Figure 9 The normal values ​​in the normal value conversion mapping table are shown as follows: b0001 is mapped to -0.6 in the normal value conversion mapping table, and b0010 is mapped to -0.5 in the normal value conversion mapping table. Therefore, the second data obtained by converting the two first data b0001 and b0010 are -0.6 and -0.5 respectively.

[0119] When the two adjacent first data input are b0001 and b1000, the two first data are a combination of !F and F. For b0001, it is necessary to query Figure 9The abnormal value conversion mapping table shown is abnormal value. For b1000, you need to query Figure 9 The normal sacrifice value in the normal value conversion mapping table is shown as follows: b0001 is mapped to -32 in the abnormal value conversion mapping table, and b1000 is mapped to 0.35 in the normal value conversion mapping table. Therefore, the second data obtained by converting the two first data b0001 and b1000 are -32 and 0.35 respectively.

[0120] When the two adjacent first data input are b1000 and b1000, the two first data are a combination of F and F, and only need to query Figure 9 The abnormal sacrifice value in the abnormal value conversion mapping table shown is shown. The value mapped to b1000 in the abnormal value conversion mapping table is 78, so the second data obtained by converting the two first data b0001 and b0001 are 78 and 78 respectively.

[0121] Please refer to Figure 10 , Figure 10 This is another schematic diagram of the conversion process under the OVP mode provided in the embodiment of the present application. Figure 10 The implementation method of exchanging the normal sacrifice value in the normal value conversion mapping relationship with the abnormal sacrifice value in the abnormal value conversion mapping relationship is described.

[0122] Assume that the abnormal value flag is b1000. When the two adjacent first data inputs are b0001 and b0010, the data conversion process can refer to Figure 9 The related description will not be repeated here in the embodiments of this application.

[0123] When the two adjacent first data input are b0001 and b1000, the two first data are a combination of !F and F. For b0001, it is necessary to query Figure 10 The abnormal value conversion mapping table shown is abnormal value. For b1000, you need to query Figure 10 The normal sacrifice value in the abnormal value conversion mapping table is shown as follows: b0001 is mapped to -32 in the abnormal value conversion mapping table, and b1000 is mapped to 0.35 in the abnormal value conversion mapping table. Therefore, the second data obtained by converting the two first data b0001 and b1000 are -32 and 0.35 respectively.

[0124] When the two adjacent first data input are b1000 and b1000, the two first data are a combination of F and F, and only need to query Figure 10 The abnormal sacrifice value in the normal value conversion mapping table is shown as follows: The value mapped to b1000 in the normal value conversion mapping table is 78, so the second data obtained by converting the two first data b0001 and b0001 are 78 and 78 respectively.

[0125] The data conversion process in normal mode can be referred to Figure 9 or Figure 10 , but the input data is a single one, not two adjacent data, and the embodiment of this application will not be described in detail here.

[0126] The following describes the application of the embodiment of the present invention to convert the weight of the large model into data as an example. Figure 11 and Figure 12 , Figure 11 and Figure 12 These are schematic diagrams of an application scenario provided by an embodiment of the present application. Figure 11 by Figure 3 Taking the data conversion device shown as an example, Figure 12 by Figure 4 The data conversion device shown in FIG. 1 is used as an example. The weights of the large model (i.e., the first data) are stored in a hard disk or memory. The hard disk may include, for example, a solid state disk (SSD) and a hard disk drive (HDD). The memory may include a double data rate synchronous dynamic random-access memory (DDR).

[0127] The weights in the hard drive or memory are transferred to the HBM. The dequantization module retrieves the weights from the HBM, converts them to weights with a second number of bits, and transfers the second number of bits to the computation module. The computation module retrieves activation data from the HBM, performs operations such as matrix multiplication (or convolution) of the weights and activation data, and then writes the calculation results to the HBM. The final calculation results are written to the hard drive or memory, and the activation data is also transferred from the hard drive or memory to the HBM.

[0128] In summary, the data conversion method provided by the embodiment of the present application, the dequantization module selects a target quantization mode from multiple supported quantization modes, and selects a target quantization format from multiple supported quantization formats, and then obtains a conversion mapping relationship corresponding to the target quantization format under the target conversion mode, and then converts at least one first data into a second data based on the conversion mapping relationship, and the conversion mapping relationship is used to indicate the mapping relationship between the data of the first bit number and the data of the second bit number, the first bit number is less than the second bit number, the bit number of the first data is the first bit number, and the bit number of the second data is the second bit number. The dequantization process is implemented by the hardware of the dequantization module, and the data is supported to be stored and transmitted in a low-bit data format, reducing the transmission bandwidth and storage overhead of the data, and the quantization mode and quantization format are configurable, which can flexibly support various quantization modes and quantization formats, and realize the conversion of low-bit data to high-bit data. When the quantization technology is not yet stable, it can effectively support the evolution of quantization technology. In the scenario of large models, it supports the storage and transmission of weights in a low-bit data format, which can reduce the occupancy of HBM and increase the transmission speed of weights, effectively supporting the high-speed development of large models.

[0129] Moreover, when the target quantization mode is the OVP mode, the outlier flag has practical significance. A second bit value can also be obtained for the outlier flag based on the conversion mapping relationship. Compared with the related technology, the conversion range represented by the conversion mapping relationship is increased, and the utilization rate of the conversion mapping relationship is improved.

[0130] The order of the methods provided in the embodiments of the present application can be adjusted appropriately, and the process can be increased or decreased accordingly. Any method that can be easily thought of by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application, and the embodiments of the present application do not limit this.

[0131] The above mainly introduces the data conversion method provided by the embodiment of the present application from the perspective of the device. It is understandable that the data conversion device for executing the above method includes a hardware structure and / or software module corresponding to the execution of each function in order to realize the above functions. Those skilled in the art should easily appreciate that, in conjunction with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0132] The embodiment of the present application can divide the data conversion device into functional modules according to the above method example. For example, each functional module can be divided into corresponding functional modules, or two or more functions can be integrated into a processing subsystem. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. In actual implementation, there may be other division methods.

[0133] Figure 13 This is a block diagram of a data conversion device provided in an embodiment of the present application. When the functional modules are divided according to their functions, the data conversion device 1300 may include: a selection module 1301, an acquisition module 1302, and a conversion module 1303. For example, the data conversion device may be a standalone device, or a chip therein, or other combined device or component having the functions of the aforementioned data conversion device. The functions of the various modules of the device are as follows:

[0134] A selection module 1301 is configured to select a target quantization mode from a plurality of supported quantization modes, and to select a target quantization format from a plurality of supported quantization formats;

[0135] An acquisition module 1302 is configured to acquire a conversion mapping relationship corresponding to a target quantization format in a target quantization mode, where the conversion mapping relationship indicates a mapping relationship between data of a first number of bits and data of a second number of bits, where the first number of bits is smaller than the second number of bits;

[0136] The conversion module 1303 is configured to convert at least one first data into second data based on a conversion mapping relationship, where the number of bits of the first data is a first number of bits, and the number of bits of the second data is a second number of bits.

[0137] In combination with the aforementioned solution, the multiple quantization modes include a normal mode and an abnormal-sacrificial pair OVP mode.

[0138] In combination with the above solution, the multiple quantization formats include uniform quantization and non-uniform quantization.

[0139] In combination with the above-mentioned scheme, the target quantization mode is the OVP mode, and the acquisition module 1302 is also used to obtain at least one first data from the quantization data matrix. The data in the quantization data matrix includes the following three types: the quantization value of the normal value, the quantization value of the abnormal value, and the abnormal value flag; wherein the abnormal value flag is obtained by replacing the sacrifice value before quantization, and the abnormal value flag is used to indicate that a data in the quantization data matrix is ​​the quantization value of the abnormal value.

[0140] In combination with the above scheme, the normal value is within the preset range, and the abnormal value is not within the preset range.

[0141] In combination with the above-mentioned scheme, the conversion mapping relationship includes a normal value conversion mapping relationship and an abnormal value conversion mapping relationship, and at least one first data includes two adjacent first data in the quantized data matrix; the conversion module 1303 is specifically used to: respectively determine whether the two first data are abnormal value flags; based on the judgment result of whether the two first data are abnormal value flags, determine the target conversion mapping relationship corresponding to the two first data respectively, and the target conversion mapping relationship is a normal value conversion mapping relationship or an abnormal value conversion mapping relationship; determine the value mapped by the first data in the target conversion mapping relationship as the second data.

[0142] In combination with the above-mentioned scheme, the conversion module 1303 is specifically used to: when both of the two first data are not outlier flag bits, determine the normal value conversion mapping relationship as the target conversion mapping relationship corresponding to the two first data respectively; when both of the first data are outlier flag bits, determine the outlier conversion mapping relationship as the target conversion mapping relationship corresponding to the two first data respectively; when the two first data are outlier flag bits and non-outlier flag bits respectively, determine the normal value conversion mapping relationship as the target conversion mapping relationship corresponding to the outlier flag bit, and determine the outlier conversion mapping relationship as the target conversion mapping relationship corresponding to the non-outlier flag bit.

[0143] In combination with the above-mentioned scheme, the conversion module 1303 is specifically used to: when both of the two first data are not abnormal value flags, determine the normal value conversion mapping relationship as the target conversion mapping relationship corresponding to the two first data respectively; when both of the two first data are abnormal value flags, determine the normal value conversion mapping relationship as the target conversion mapping relationship corresponding to the two first data respectively; when the two first data are abnormal value flags and non-abnormal value flags respectively, determine the abnormal value conversion mapping relationship as the target conversion mapping relationship corresponding to the two first data respectively.

[0144] Combined with the above-mentioned scheme, in the normal value conversion mapping relationship, the value mapped by the abnormal value flag bit is the normal sacrifice value, and the value mapped by the non-abnormal value flag bit is the normal value; in the abnormal value conversion mapping relationship, the value mapped by the abnormal value flag bit is the abnormal sacrifice value, and the value mapped by the non-abnormal value flag bit is the abnormal value.

[0145] In combination with the above solution, the conversion mapping relationship is in the form of a table, and the normal value conversion mapping relationship and the abnormal value conversion mapping relationship are two independent tables; or the normal value conversion mapping relationship and the abnormal value conversion mapping relationship are two clusters in one table.

[0146] In combination with the above solution, the target conversion mode is the normal mode, and the conversion mapping relationship includes a normal value conversion mapping relationship: the conversion module 1303 is specifically configured to determine the value of the first data mapping in the normal value conversion mapping relationship as the second data.

[0147] Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 1400 may be a chip or functional module in a data conversion device. Figure 14 As shown, the electronic device 1400 includes a processor 1401 , a transceiver 1402 and a communication circuit 1403 .

[0148] The processor 1401 is used to execute the following Figure 5 In any step of the method embodiment shown, and when executing a process such as obtaining the first data, the transceiver 1402 and the communication line 1403 may be selectively called to complete the corresponding operation.

[0149] Furthermore, the electronic device 1400 may further include a memory 1404 . The processor 1401 , the memory 1404 and the transceiver 1402 may be connected via a communication line 1403 .

[0150] Transceiver 1402 is used to communicate with other devices or other communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. Transceiver 1402 can be a module, circuit, transceiver, or any device capable of implementing communication.

[0151] The transceiver 1402 is mainly used for sending and receiving data, etc., and may include a transmitter and a receiver, which respectively send and receive data, etc.; operations other than sending and receiving data, etc. are implemented by the processor, such as converting the first data into the second data based on the conversion mapping relationship.

[0152] The communication line 1403 is used to transmit information between the various components included in the electronic device 1400.

[0153] In one design, the processor can be considered as the logic circuit and the transceiver as the interface circuit.

[0154] The memory 1404 is used to store instructions, where the instructions may be computer programs.

[0155] It should be noted that memory 1404 can exist independently of processor 1401 or can be integrated with processor 1401. Memory 1404 can be used to store instructions, program code, or some data. Memory 1404 can be located within electronic device 1400 or outside of electronic device 1400, without limitation. Processor 1401 is configured to execute instructions stored in memory 1404 to implement the methods provided in the above embodiments of this application.

[0156] In one example, the processor 1401 may include one or more processors, such as Figure 14 Processor 0 and processor 1 in.

[0157] As an optional implementation, the electronic device 1400 includes multiple processors, for example, Figure 14 In addition to the processor 1401, a processor 1407 may also be included.

[0158] As an optional implementation, the electronic device 1400 further includes an output device 1405 and an input device 1406. For example, the input device 1406 is a keyboard, a mouse, a microphone, or a joystick, and the output device 1405 is a display screen, a speaker, or the like.

[0159] It should be noted that the electronic device 1400 can be a chip system or a Figure 14 Devices with similar structures in the chip system. Among them, the chip system can be composed of chips, or it can include chips and other discrete devices. The actions, terms, etc. involved in the various embodiments of this application can refer to each other without limitation. The message names or parameter names in the messages exchanged between the various devices in the embodiments of this application are only examples, and other names can also be used in specific implementations without limitation. In addition, Figure 14 The composition structure shown in the figure does not constitute a limitation on the electronic device 1400, except Figure 14 In addition to the components shown, the electronic device 1400 may include Figure 14 More or fewer components may be shown, or certain components may be combined, or the components may be arranged differently.

[0160] The processor and transceiver described in this application can be implemented on an integrated circuit (IC), an analog IC, a radio frequency integrated circuit, a mixed-signal IC, an application specific integrated circuit (ASIC), a printed circuit board (PCB), an electronic device, etc. The processor and transceiver can also be manufactured using various IC process technologies, such as complementary metal oxide semiconductor (CMOS), n-type metal oxide semiconductor (NMOS), p-type metal oxide semiconductor (positive channel metal oxide semiconductor, PMOS), bipolar junction transistor (BJT), bipolar CMOS (BiCMOS), silicon germanium (SiGe), gallium arsenide (GaAs), etc.

[0161] Figure 15 This is a schematic diagram of the structure of a data conversion device provided in an embodiment of the present application. The data conversion device can be applied to the scenario shown in the above method embodiment. For the convenience of explanation, Figure 15 Only the main components of the data conversion device are shown, including a processor, memory, control circuitry, and input / output devices. The processor is primarily responsible for processing communication protocols and communication data, executing software programs, and processing software program data. The memory is primarily responsible for storing software programs and data. The control circuitry is primarily responsible for power supply and transmission of various electrical signals. The input / output devices are primarily responsible for receiving user input and outputting data to the user.

[0162] When the data conversion device is a hardware device, the control circuit may be a motherboard, the memory may include a hard disk, RAM, ROM, or other media with storage functions, the processor may include a baseband processor and a central processing unit, the baseband processor is mainly used to process the communication protocol and communication data, the central processing unit is mainly used to control the entire data conversion device, execute software programs, and process software program data, and the input and output devices include a display screen, a keyboard, and a mouse. The control circuit may further include or be connected to a transceiver circuit or transceiver, such as a network cable interface, for sending or receiving data or signals, such as for data transmission and communication with other devices. Furthermore, it may also include an antenna for data transmission and reception, for data / request transmission with other devices.

[0163] According to the method provided in the embodiments of the present application, the present application also provides a computer program product, which includes computer program code. When the computer program code runs on a computer, it enables the computer to execute any of the methods described in the embodiments of the present application.

[0164] The present application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be completed by a computer or a device with data conversion capabilities executing a computer program or instruction to control the relevant hardware. The computer program or the group of instructions can be stored in the above computer-readable storage medium. When executed, the computer program or the group of instructions may include the processes of the above method embodiments. The computer-readable storage medium can be an internal storage unit of the data conversion device of any of the above embodiments, such as a hard disk or memory of the data conversion device. The above computer-readable storage medium can also be an external storage device of the above data conversion device, such as a plug-in hard disk equipped on the above data conversion device, a smart memory card (SMC), a secure digital (SD) card, a flash card, etc. Further, the above computer-readable storage medium can also include both the internal storage unit of the above data conversion device and an external storage device. The above computer-readable storage medium is used to store the above computer program or instruction and other programs and data required by the above data conversion device. The above computer-readable storage medium can also be used to temporarily store data that has been output or is to be output.

[0165] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0166] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0167] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0168] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0169] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0170] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0171] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A data conversion device, characterized in that: The device comprises: an inverse quantization module, configured to select a target quantization mode from a plurality of supported quantization modes and a target quantization format from a plurality of supported quantization formats; The inverse quantization module is further configured to obtain a conversion mapping relationship corresponding to the target quantization format in the target quantization mode, the conversion mapping relationship being configured to indicate a mapping relationship between data of a first number of bits and data of a second number of bits, the first number of bits being smaller than the second number of bits; The inverse quantization module is further used to convert at least one first data into second data based on the conversion mapping relationship, where the number of bits of the first data is the first number of bits and the number of bits of the second data is the second number of bits.

2. The device according to claim 1, characterized in that The plurality of quantization modes include a normal mode and an abnormal-victim OVP mode.

3. The device according to claim 1 or 2, characterized in that The plurality of quantization formats include uniform quantization and non-uniform quantization.

4. The device according to claim 2, characterized in that The target quantization mode is the OVP mode, and the inverse quantization module is further configured to: Acquire the at least one first data from a quantized data matrix, where the data in the quantized data matrix includes the following three types: quantized values ​​of normal values, quantized values ​​of abnormal values, and abnormal value flags; The abnormal value flag is obtained by replacing the sacrifice value before quantization, and the abnormal value flag is used to indicate that a data in the quantization data matrix is ​​a quantization value of an abnormal value.

5. The device according to claim 4, characterized in that The normal value is within a preset range, and the abnormal value is not within the preset range.

6. The device according to claim 4 or 5, characterized in that The conversion mapping relationship includes a normal value conversion mapping relationship and an abnormal value conversion mapping relationship, and the at least one first data includes two adjacent first data in the quantized data matrix; The dequantization module is specifically used to: respectively determining whether the two first data are the abnormal value flags; Based on the judgment result of whether the two first data are the abnormal value flag bits, determining the target conversion mapping relationships respectively corresponding to the two first data, the target conversion mapping relationship being the normal value conversion mapping relationship or the abnormal value conversion mapping relationship; The value of the first data mapping in the target conversion mapping relationship is determined as the second data.

7. The device according to claim 6, characterized in that The dequantization module is specifically used for: When neither of the two first data is the abnormal value flag, determining the normal value conversion mapping relationship as the target conversion mapping relationship corresponding to the two first data respectively; When both of the two first data are the abnormal value flags, determining the abnormal value conversion mapping relationship as the target conversion mapping relationship corresponding to the two first data respectively; When the two first data are respectively the abnormal value flag and the non-abnormal value flag, the normal value conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the abnormal value flag, and the abnormal value conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the non-abnormal value flag.

8. The device according to claim 6, characterized in that The dequantization module is specifically used to: When neither of the two first data is the abnormal value flag, determining the normal value conversion mapping relationship as the target conversion mapping relationship corresponding to the two first data respectively; When both of the two first data are the abnormal value flags, determining the normal value conversion mapping relationship as the target conversion mapping relationship corresponding to the two first data respectively; When the two first data are respectively the abnormal value flag bit and the non-abnormal value flag bit, the abnormal value conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the two first data respectively.

9. The device according to claim 7 or 8, characterized in that In the normal value conversion mapping relationship, the value mapped to the abnormal value flag bit is a normal sacrifice value, and the value mapped to the non-abnormal value flag bit is a normal value; In the abnormal value conversion mapping relationship, the value mapped to the abnormal value flag bit is the abnormal sacrifice value, and the value mapped to the non-abnormal value flag bit is the abnormal value.

10. The device according to any one of claims 6 to 9, characterized in that The conversion mapping relationship is in the form of a table, and the normal value conversion mapping relationship and the abnormal value conversion mapping relationship are two independent tables; Or the normal value conversion mapping relationship and the outlier conversion mapping relationship are two clusters in one table.

11. The device according to claim 2, characterized in that The target conversion mode is the normal mode, and the conversion mapping relationship includes a normal value conversion mapping relationship; The dequantization module is specifically configured to determine the value mapped to the first data in the normal value conversion mapping relationship as the second data.

12. The device according to any one of claims 1 to 11, characterized in that The device further comprises: a storage module and a calculation module, wherein the dequantization module is located on a data path between the storage module and the calculation module, or is located at an entrance or inside the calculation module; The storage module is configured to store the at least one first data; The dequantization module is further configured to obtain the at least one first data from the storage module; The dequantization module is further configured to send the second data to the calculation module; The computing module is used to process the second data.

13. A data conversion method, characterized in that: The method comprises: Selecting a target quantization mode from among multiple supported quantization modes, and selecting a target quantization format from among multiple supported quantization formats; Obtaining a conversion mapping relationship corresponding to the target quantization format in the target quantization mode, where the conversion mapping relationship is used to indicate a mapping relationship between data of a first number of bits and data of a second number of bits, where the first number of bits is smaller than the second number of bits; Based on the conversion mapping relationship, at least one first data is converted into second data respectively, the number of bits of the first data is the first number of bits, and the number of bits of the second data is the second number of bits.

14. The method according to claim 13, characterized in that The plurality of quantization modes include a normal mode and an abnormal-victim OVP mode.

15. The method according to claim 13 or 14, characterized in that The plurality of quantization formats include uniform quantization and non-uniform quantization.

16. The method according to claim 14, characterized in that The target quantization mode is the OVP mode, and the method further includes: Acquire the at least one first data from a quantized data matrix, where the data in the quantized data matrix includes the following three types: quantized values ​​of normal values, quantized values ​​of abnormal values, and abnormal value flags; The abnormal value flag is obtained by replacing the sacrifice value before quantization, and the abnormal value flag is used to indicate that a data in the quantization data matrix is ​​a quantization value of an abnormal value.

17. The method according to claim 16, characterized in that The normal value is within a preset range, and the abnormal value is not within the preset range.

18. The method according to claim 16 or 17, characterized in that The conversion mapping relationship includes a normal value conversion mapping relationship and an abnormal value conversion mapping relationship, and the at least one first data includes two adjacent first data in the quantized data matrix; The converting at least one first data into second data based on the conversion mapping relationship includes: respectively determining whether the two first data are the abnormal value flags; Based on the judgment result of whether the two first data are the abnormal value flag bits, determining the target conversion mapping relationships respectively corresponding to the two first data, the target conversion mapping relationship being the normal value conversion mapping relationship or the abnormal value conversion mapping relationship; The value of the first data mapping in the target conversion mapping relationship is determined as the second data.

19. The method according to claim 18, characterized in that The determining, based on the judgment result of whether the two first data are the abnormal value flags, the target conversion mapping relationships respectively corresponding to the two first data includes: When neither of the two first data is the abnormal value flag, determining the normal value conversion mapping relationship as the target conversion mapping relationship corresponding to the two first data respectively; When both of the two first data are the abnormal value flags, determining the abnormal value conversion mapping relationship as the target conversion mapping relationship corresponding to the two first data respectively; When the two first data are respectively the abnormal value flag and the non-abnormal value flag, the normal value conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the abnormal value flag, and the abnormal value conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the non-abnormal value flag.

20. The method according to claim 18, wherein The determining, based on the judgment result of whether the two first data are the abnormal value flags, the target conversion mapping relationships respectively corresponding to the two first data includes: When neither of the two first data is the abnormal value flag, determining the normal value conversion mapping relationship as the target conversion mapping relationship corresponding to the two first data respectively; When both of the two first data are the abnormal value flags, determining the normal value conversion mapping relationship as the target conversion mapping relationship corresponding to the two first data respectively; When the two first data are respectively the abnormal value flag bit and the non-abnormal value flag bit, the abnormal value conversion mapping relationship is determined as the target conversion mapping relationship corresponding to the two first data respectively.

21. The method according to claim 19 or 20, characterized in that In the normal value conversion mapping relationship, the value mapped to the abnormal value flag bit is a normal sacrifice value, and the value mapped to the non-abnormal value flag bit is a normal value; In the abnormal value conversion mapping relationship, the value mapped to the abnormal value flag bit is the abnormal sacrifice value, and the value mapped to the non-abnormal value flag bit is the abnormal value.

22. The method according to any one of claims 18 to 21, characterized in that The conversion mapping relationship is in the form of a table, and the normal value conversion mapping relationship and the abnormal value conversion mapping relationship are two independent tables; Or the normal value conversion mapping relationship and the outlier conversion mapping relationship are two clusters in one table.

23. The method according to claim 14, wherein The target conversion mode is the normal mode, and the conversion mapping relationship includes a normal value conversion mapping relationship; The converting at least one first data into second data based on the conversion mapping relationship includes: The value mapped to the first data in the normal value conversion mapping relationship is determined as the second data.

24. A chip, characterized in that: The chip includes: processing circuits and interface circuits; The interface circuit is used to couple with a memory outside the chip and provide a communication interface for the processing circuit to access the memory; The processing circuit is configured to execute program instructions in the memory to implement the method according to any one of claims 13 to 23.

25. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program code, and when the program code is executed by a processor, the method according to any one of claims 13 to 23 is implemented.

Images