Data set matrix quantization method and device, computer device and storage medium

Abstract

Embodiments of the present application provide a data set matrix quantization method and device, computer equipment and a storage medium. The method comprises: identifying and extracting all outliers in a target data set matrix, and marking the precision categories to which all outliers belong; the sparse structure is composed of the extracted outliers, the positions of the outliers in the target data set matrix, and the precision categories to which the outliers belong; the data in the target data set matrix is quantized to generate a quantized data set matrix; in the quantized data set matrix, the outliers are replaced with specified values at the positions of the outliers to generate a dense matrix; the sparse part is generated according to the dense matrix and the sparse structure; the dense matrix and the sparse part are taken as a target data set matrix quantization result; the data set matrix is divided into a dense matrix and an outlier part; and the outliers are classified more finely. On the premise of guaranteeing performance index requirements such as precision and error, the dense matrix can be represented by fewer bits, and storage space is saved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a method, apparatus, computer device, and storage medium for dataset matrix quantization. Background Technology

[0002] With the continuous development of science and technology, big data and artificial intelligence technologies are widely entering many industrial and daily life fields. Both technologies require the use of algorithm models, CPUs, GPUs, TPUs, or NPUs to process massive amounts of data, which poses a great challenge to related hardware and software. Current large-scale artificial intelligence models often have tens or even trillions of parameters. During training and inference, various important parameters such as weights, activations, key-value caches, and gradients consume massive amounts of computing and storage resources. Quantization technology has found important and widespread applications in the fields of big data, artificial intelligence, and video encoding and decoding. It optimizes the storage and computational efficiency of parameters by reducing their precision. For example, quantization technology can be used to convert high-precision parameters (such as 32-bit floating-point numbers) in large models into low-bit-width numerical representations (such as 8-bit or 4-bit integers). If a 32-bit floating-point number is quantized into a 4-bit integer, the storage will be reduced to 1 / 8 of the original, and the computational load and processing speed will also be significantly reduced. However, quantization inevitably leads to a certain degree of accuracy loss, especially due to outliers (values ​​in the dataset that are significantly different from other data values; significant difference includes being either exceptionally large or exceptionally small compared to other values, or having the greatest impact on performance). How to handle outliers during quantization is a key focus of quantization algorithm research.

[0003] SqueezeLLM (a quantization compression technique for large language models (LLMs)) and SpQR (a mainstream quantization technique used to handle outlier problems in the quantization process of large language models (LLMs)) are two mainstream quantization techniques that employ a "divide and conquer" strategy to deal with the most intractable outliers. This involves decomposing the original dataset matrix into a dense matrix containing the vast majority of normal values ​​and a sparse matrix containing only a few outliers. However, existing quantization techniques with both dense and sparse matrices do not provide more refined classification of outliers, and the sparse matrix representation uses conventional sparse matrix compression algorithms. These factors negatively impact quantization accuracy and storage space consumption. Summary of the Invention

[0004] This application provides a dataset matrix quantization method, apparatus, computer device, and storage medium.

[0005] A first aspect of this application provides a dataset matrix quantization method, including: Identify and extract all outliers in the target dataset matrix, and label the precision categories to which all outliers belong according to different precision requirements. The number of precision categories is an integer greater than or equal to 0. The sparse structure is composed of the extracted outliers, their positions in the target dataset matrix, and their precision categories. Quantize the data in the target dataset matrix to generate a quantized dataset matrix; In the quantized dataset matrix, outliers are replaced with specified values ​​at their locations, while the values ​​at other locations remain unchanged, generating a dense matrix. A sparse part is generated based on the dense matrix and the sparse structure. The dense matrix and the sparse part are used as the matrix quantization result of the target dataset. The sparse part includes a specified value at the position of the outlier in the dense matrix, a correlation value between the outlier in the sparse structure and the specified value at the position of the outlier in the dense matrix, and the precision category to which the outlier belongs. The correlation value is used to reverse generate the original outlier or a value close to the original outlier.

[0006] In an optional embodiment of this application, identifying and extracting all outliers in the target dataset matrix includes: Data that meets any of the following four conditions will be considered outliers in the target dataset matrix: First, extreme data that deviates from the preset range; Second, data that deviates from the median of all data in the target dataset matrix or the median of all data in the quantized group to which it belongs by more than N times; Third, data whose impact on performance is ranked higher than the preset value; Fourth, data that significantly impacts the number of bits required for each quantized data point, while ensuring the preset quantization accuracy requirements are met.

[0007] In an optional embodiment of this application, the step of labeling all outliers to their respective precision categories according to different precision requirements includes: Outliers containing at least one of the following three categories are labeled as low-precision categories: extreme data that deviates from the preset range; data that deviates from the median of all data in the target dataset matrix or all data in the quantization group by more than N times; and data that has a significant impact on the number of bits required for each data after quantization, while ensuring the preset quantization precision requirements. Outliers that have a performance impact higher than the preset value are marked as high-precision categories.

[0008] In an optional embodiment of this application, before quantizing the data in the target dataset matrix, the method further includes: The outlier positions in the target dataset matrix are replaced with a first specified value, which includes one of the following: the zero point value of the quantization algorithm, the median value of the data other than the outlier in the target dataset matrix or the quantization group, and the arithmetic mean of the data other than the outlier. When the zero points of the quantization algorithm are already determined, the positions in the target dataset matrix where outliers exist are the zero points of the quantization algorithm. When the quantization algorithm and quantization parameters are dynamically selected based on the data and distribution in the target dataset matrix, and the target dataset matrix is ​​divided into blocks with different quantization algorithms and quantization parameters selected for each block, the location of outliers in the target dataset matrix is ​​the median or arithmetic mean of the other data in the target dataset matrix, excluding the outliers.

[0009] In an optional embodiment of this application, the specified value is determined by the following steps: Determine the range of values ​​represented by the data type representing the data in the quantized dataset matrix, where the range is the quantized data type range; For each quantization group in the quantization dataset matrix, the set of other data outside the outlier position in the current quantization group is called the other data set. Subtracting the other data set from the quantization data type range gives the set of candidate values ​​for the specified value. Determine whether the number of elements in the candidate set of the specified value is greater than or equal to the number of precision categories. If so, select the value of the number of precision categories from the candidate set of the specified value as the specified value; otherwise, perform the following steps. The difference between the number of precision categories and the number of elements in the candidate set for the specified value is used as the supplementary number for the specified value. Select the values ​​from other datasets that have the same number of replacements as the specified value and the fewest duplicates, and fill them into the candidate set of the specified value; Select the value of the precision category number from the candidate set of filled specified values ​​as the specified value.

[0010] In an optional embodiment of this application, the correlation value between outliers in a sparse structure and a specified value at the location of an outlier in a dense matrix is ​​determined by the following expression: r = f(m) or r = f(m, n) Where r is the correlation value, m is the outlier in the sparse structure, n is the specified value of the outlier position in the dense matrix corresponding to m, and f() is the first preset transformation formula.

[0011] In an optional embodiment of this application, the method further includes: Determine the quantization group to which the location to be restored belongs, and obtain the sparse part corresponding to the quantization group; Obtain the value of the restored position in a dense matrix; Determine whether the value at the restored position is contained within the specified value at the outlier position in the dense matrix of the sparse part. If so, use the second preset transformation formula in the sparse part to reverse generate the original outlier or a value close to the original outlier. Otherwise, proceed to the next step. The original value or a value close to the original value at the restored position is generated using a preset dequantization method.

[0012] A second aspect of this application provides a dataset matrix quantization apparatus, comprising: The identification module is used to identify and extract all outliers in the target dataset matrix, and to label the precision category to which all outliers belong according to different precision requirements. The number of precision categories is an integer greater than or equal to 0, and the module consists of the extracted outliers, their positions in the target dataset matrix and their precision categories, forming a sparse structure. The quantization module is used to quantize the data in the target dataset matrix and generate a quantized dataset matrix. The first generation module is used to replace outliers with specified values ​​at outlier positions in the quantized dataset matrix, while keeping the values ​​at other positions unchanged, to generate a dense matrix. The second generation module is used to generate a sparse part based on the dense matrix and the sparse structure. The dense matrix and the sparse part are used as the matrix quantization result of the target dataset. The sparse part includes a specified value at the position of the outlier in the dense matrix, a correlation value between the outlier in the sparse structure and the specified value at the position of the outlier in the dense matrix, and the precision category to which the outlier belongs. The correlation value is used to reverse generate the original outlier or a value close to the original outlier.

[0013] A third aspect of this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above dataset matrix quantization methods.

[0014] A fourth aspect of the embodiments of this application provides a computer-readable and writable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the dataset matrix quantization method as described above.

[0015] Compared with the prior art, the technical solutions provided in this application have at least some or all of the following advantages: The dataset matrix quantization method described in this application identifies and extracts all outliers in the target dataset matrix, and labels the precision categories to which all outliers belong according to different precision requirements. The number of precision categories is an integer greater than or equal to 0. A sparse structure is formed by the extracted outliers, their positions in the target dataset matrix, and their respective precision categories. The data in the target dataset matrix is ​​quantized to generate a quantized dataset matrix. In the quantized dataset matrix, outliers are replaced with specified values ​​at their positions, while values ​​at other positions remain unchanged, generating a dense matrix. A sparse part is generated based on the dense matrix and the sparse structure. The dense matrix and the sparse part are used as the quantization result of the target dataset matrix, wherein the sparse part includes the outlier positions in the dense matrix. The algorithm assigns a specified value, a correlation value between outliers in the sparse structure and a specified value at the position of the outlier in the dense matrix, and the precision category to which the outlier belongs. The correlation value is used to reverse-generate the original outlier or a value close to the original outlier. This divides the dataset matrix into a dense matrix and an outlier component, and further refines the classification of outliers, making it possible to use a more suitable quantization algorithm. While ensuring performance requirements such as accuracy and error, it can represent the dense matrix with fewer bits, thus saving storage space. Furthermore, this application employs a flexible and variable mapping algorithm from the outlier component to the original outlier, and introduces a sparse matrix representation algorithm that saves more storage space than traditional algorithms such as COO, CSR, and CSC, further saving storage space. This is especially important for GPU memory, which is a scarce and expensive storage resource widely used in large models. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart illustrating a dataset matrix quantization method provided in one embodiment of this application; Figure 2 A schematic diagram of a dataset matrix provided in one embodiment of this application; Figure 3 A flowchart illustrating a method for determining a specified value according to an embodiment of this application; Figure 4 A schematic diagram of a quantized dataset matrix provided in one embodiment of this application; Figure 5 A schematic diagram of a dense matrix provided in one embodiment of this application; Figure 6 This is a schematic diagram of the data structure for the sparse portion of an embodiment of this application; Figure 7A schematic diagram of the data structure of the sparse part when there is only one precision category, as provided in one embodiment of this application; Figure 8 A flowchart illustrating a method for approximating the value at any position in a dataset matrix according to an embodiment of this application; Figure 9 This is a schematic diagram of a dataset matrix quantization device provided in one embodiment of this application; Figure 10 This is a schematic diagram of a computer device structure provided in one embodiment of this application. Detailed Implementation

[0017] To make the technical solutions and advantages of the embodiments of this application clearer, the exemplary embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0018] Please see Figure 1 The dataset matrix quantization method provided in this application includes the following steps S100~S400: S100 identifies and extracts all outliers in the target dataset matrix, and labels the precision categories to which all outliers belong according to different precision requirements. The number of precision categories is an integer greater than or equal to 0. The sparse structure is composed of the extracted outliers, their positions in the target dataset matrix, and their precision categories. S200 quantizes the data in the target dataset matrix to generate a quantized dataset matrix; S300, in the quantized dataset matrix, replaces outliers with specified values ​​at outlier positions while keeping the values ​​at other positions unchanged, generating a dense matrix; S400: Generate a sparse part based on the dense matrix and sparse structure, and use the dense matrix and sparse part as the matrix quantization result of the target dataset. The sparse part includes a specified value at the position of outlier in the dense matrix, a correlation value between the outlier in the sparse structure and the specified value at the position of outlier in the dense matrix, and the precision category to which the outlier belongs. The correlation value is used to reverse generate the original outlier or a value close to the original outlier.

[0019] In an optional embodiment of this application, the target dataset includes, but is not limited to, various types of data such as weights, activations, key-value caches (KV caches), and gradients in a large model. When a computer stores and processes these data, they often exist in the form of a matrix (a type of tensor) of a specified data type, i.e., a dataset matrix.

[0020] In an optional embodiment of this application, in step S100, before identifying and extracting all outliers in the target dataset matrix, the method further includes: The target dataset matrix is ​​grouped to obtain multiple quantized groups.

[0021] In an optional embodiment of this application, step S100, identifying and extracting all outliers in the target dataset matrix, includes: Data that meets any of the following four conditions will be considered outliers in the target dataset matrix: First, extreme data that deviates from the preset range; Second, data that deviates from the median of all data in the target dataset matrix or the median of all data in the quantized group to which it belongs by more than N times; Third, data whose impact on performance is ranked higher than the preset value; Fourth, data that significantly impacts the number of bits required for each quantized data point, while ensuring the preset quantization accuracy requirements are met.

[0022] In an optional embodiment of this application, for ease of interpretation of "extreme data deviating from the preset range," it is assumed that the preset range is from -3700 to 5000. Values ​​outside this range can be considered outliers. For example, the values ​​-80000 and 7000 are both outside this range and can be considered outliers. Such outliers often deviate significantly from most other values.

[0023] In an optional embodiment of this application, "data whose impact on performance ranks higher than a preset value" can be, for example, data that accounts for only 1% of the total data volume in the dataset but causes more than 75% of the total quantization error, and these data need to maintain high accuracy. In this case, "performance" can be quantization error, and "higher than a preset value" can be the top 1% of data that have the greatest impact on quantization error. "Performance" can also refer to quantization accuracy, inference accuracy in large models, etc.

[0024] In another optional embodiment of this application, "data that ranks higher than a preset value in terms of performance impact" can also be equivalent to or similar to outliers in SqueezeLLM or SpQR techniques.

[0025] In an optional embodiment of this application, outliers refer to values ​​in the dataset that are significantly different from other data values. Significant difference means that compared to other values, they are either abnormally large or abnormally small, or data that has the greatest impact on performance. It can also be data that significantly affects the number of bits required for each quantized data point while ensuring quantization accuracy. For example, suppose we need to... Figure 2The dataset matrix shown is quantized, requiring the quantized data to be 4-bit integers while maintaining quantization accuracy. A specified algorithm can be used to identify outliers as abnormally large and small values ​​in the dataset. For example, data deviating more than 10 times the median (when N is 10) can be identified as outliers. Clearly, the data -28215.3 is abnormally small compared to the median, while the data 7633.5 is abnormally large compared to the median; both deviate significantly from the median by more than 10 times. Therefore, both -28215.3 and 7633.5 are identified as outliers.

[0026] Alternatively, a specific algorithm can be used to identify data whose impact on performance ranks higher than a preset value. These outliers often account for only 1% of the total data volume, but can cause more than 75% of the total quantization error, requiring high precision. Figure 2 If the data point 35.7 belongs to this category of outliers, then it might be necessary to represent this value more precisely using 35, 36, or even 35.7, rather than performing quantization operations that would result in a significant loss of precision. The specified algorithm here can be similar to OBC (Optimal Brain Compression) to calculate the sensitivity of the data, thereby identifying data whose impact on performance ranks higher than a preset value.

[0027] Furthermore, the aforementioned "data that affects the number of bits required for each quantized data point while ensuring quantization accuracy" refers to representing the vast majority of data with fewer quantized bits while meeting quantization accuracy requirements. For example, when quantizing the dataset {86.2, 25.1, 103.6, 132.7, 93.6, 385.5, 121.9, 107.8, 115.2}, if all data uses the same quantization algorithm, 5 bits of integer data are needed to represent the quantized data. If the data points 25.1 and 385.5 are removed, only 4 bits of integer data are needed to represent all other quantized data. In this case, the data points 25.1 and 385.5 can be considered "data that affects the number of bits required for each quantized data point while ensuring quantization accuracy," meaning they also belong to outliers.

[0028] In an optional embodiment of this application, step S100, which involves labeling all outliers according to different precision requirements, includes: Outliers containing at least one of the following three categories are labeled as low-precision categories: extreme data that deviates from the preset range; data that deviates from the median of all data in the target dataset matrix or all data in the quantization group by more than N times; and data that has a significant impact on the number of bits required for each data after quantization, while ensuring the preset quantization precision requirements. Outliers that have a performance impact higher than the preset value are marked as high-precision categories.

[0029] In an optional embodiment of this application, if both low-precision and high-precision categories exist, their number can be no less than one.

[0030] In an optional embodiment of this application, "and marking the precision category of outliers according to different precision requirements" means that some outliers need to be represented by high-precision data values, while others only need to be represented by low-precision values, including high-precision categories and low-precision categories. The high-precision category can be the aforementioned "data whose impact on performance ranks higher than a preset value," such as... Figure 2 The value 35.7 might need to be represented more precisely as 35, 36, or even 35.7. Low-precision categories refer to outliers that can be approximated by a close number, such as... Figure 2 The outlier value of 7633.5 can be approximated by 7000.

[0031] In an optional embodiment of this application, to facilitate subsequent specialized processing of outliers, all outliers in the dataset matrix need to be extracted and then stored in a format such as CSR (Compressed Sparse Row). The CSR format consists of three arrays: values ​​(outliers), column_indices (column indices of outliers), and row_pointers (pointers to the beginning of each row). If the outliers are stored in CSR format, a separate array, precision_category, can be used to store the precision category labels to which the outliers belong. Assuming array value 0 represents a low-precision category and 1 represents a high-precision category, under the same array index, the precision category represented by the value in the precision_category array corresponds to the outlier represented by the value in the values ​​array. For example, if the index is 97, then precision_category

[97] =0 means that the outlier stored in values

[97] is the low precision category; and if the index is 213, then precision_category

[213] =1 means that the outlier stored in values

[213] is the high precision category.

[0032] In an optional embodiment of this application, before quantizing the data in the target dataset matrix in step S200, the method further includes: The outlier positions in the target dataset matrix are replaced with a first specified value, which includes one of the following: the zero point value of the quantization algorithm, the median value of the data other than the outlier in the target dataset matrix or the quantization group, and the arithmetic mean of the data other than the outlier. When the zero points of the quantization algorithm are already determined, the positions in the target dataset matrix where outliers exist are the zero points of the quantization algorithm. When the quantization algorithm and quantization parameters are dynamically selected based on the data and distribution in the target dataset matrix, and the target dataset matrix is ​​divided into blocks with different quantization algorithms and quantization parameters selected for each block, the location of outliers in the target dataset matrix is ​​the median or arithmetic mean of the other data in the target dataset matrix, excluding the outliers.

[0033] In an optional embodiment of this application, the structure composed of outliers extracted from the dataset matrix and their respective precision categories is a sparse structure, comprising three parts: outliers (such as the array `values` mentioned above), the positions of the outliers in the dataset matrix (such as the column indices and row pointers for the beginning of each row mentioned above), and the precision category (such as the array `precision_category` mentioned above). Through the sparse structure, the position, specific value, and precision category of each outlier in the dataset matrix can be determined, facilitating subsequent lookup operations.

[0034] In an optional embodiment of this application, the location of outliers in the dataset matrix is ​​replaced by one of the following: the zero point value of the quantization algorithm, the median value of the other values ​​after removing outliers, and the arithmetic mean of the other values ​​after removing outliers.

[0035] In an optional embodiment of this application, outliers are replaced with zero-point values ​​from the quantization algorithm, one purpose of which is to reduce the quantization computation process. For example, in the quantization algorithm shown in the following expression:

[0036] Where f is the original floating-point number, q is the quantized integer, s is the scaling factor, z is the zeropoint, and the round() function performs rounding. By replacing outliers with the zeropoint value (z in the above expression), when quantizing outliers, by using fz=0, the quantized number q can be directly obtained as 0, eliminating the need for division with s and the calculation in the round() function. This reduces the quantization calculation process, which helps to speed up quantization and save on storage consumption during the quantization process.

[0037] In an optional embodiment of this application, when outliers are replaced with the zero-point values ​​of the quantization algorithm, it addresses a scenario where the zero-point of the quantization algorithm is already determined. In practical applications, sometimes a suitable quantization algorithm and quantization parameters are dynamically selected based on the specific data values ​​and their distribution in the dataset. Sometimes, the dataset needs to be divided into blocks, and the quantization algorithm and quantization parameters selected for each block are also different. In this case, replacing outliers with the median value of the other values ​​after removing the outliers, or with the arithmetic mean of the other values ​​after removing the outliers, can significantly reduce the interference of outliers on the selection of suitable quantization algorithms and quantization parameters. If the dataset is {7.2, 10.3, 6.5, 2832.2, 18.5, 26.1, 8.9, 30.8}, and the outlier 2832.2 has been identified, and the median of the remaining numbers after removing this outlier is 10.3, and the arithmetic mean is 15.47, then the dataset after replacing the outlier with the median is {7.2, 10.3, 6.5, 10.3, 18.5, 26.1, 8.9, 30.8}, and the dataset after replacing it with the arithmetic mean is {7.2, 10.3, 6.5, 15.47, 18.5, 26.1, 8.9, 30.8}. The values ​​in bold are the values ​​after the outlier has been replaced.

[0038] In an optional embodiment of this application, in step S200, after removing all outliers from the dataset, the remaining data is relatively stable, with neither particularly large nor particularly small data, nor data that needs to maintain high precision. This avoids interference from outliers on quantization, allowing a suitable algorithm to quantize all other data. Furthermore, while meeting the requirements for quantization accuracy and performance, the quantized data can be represented by fewer bits, reducing the storage space occupied by the data and improving data utilization efficiency.

[0039] In an optional embodiment of this application, in practical applications, the dataset matrix can be divided into multiple groups, each group can use a different quantization algorithm and have its own scaling factor, zero point, and other related quantization parameters. For example, similar to traditional quantization algorithms such as sparse quantization representation SpQR, the dataset matrix can be grouped by rows, columns, or blocks, and quantized by group.

[0040] In an optional embodiment of this application, in order to take advantage of the strengths of modern computer hardware in processing matrix computations and to speed up quantization, the entire dataset matrix containing outlier locations is quantized, rather than bypassing the outlier locations.

[0041] In an optional embodiment of this application, the data at each position in the quantized dataset matrix corresponds to the data in the original dataset matrix. For example, the data in the 7th row and 9th column of the original dataset matrix is ​​also the data in the 7th row and 9th column of the quantized dataset matrix; their row and column positions are the same in both matrices. The main purpose of quantization is to reduce data storage space and improve computational efficiency. For example, a 32-bit floating-point dataset matrix is ​​quantized to generate a 4-bit integer quantized dataset matrix.

[0042] In an optional embodiment of this application, in step S300, the specified value is determined by the following steps: Determine the range of values ​​represented by the data type representing the data in the quantized dataset matrix, where the range is the quantized data type range; For each quantization group in the quantization dataset matrix, the set of other data outside the outlier position in the current quantization group is called the other data set. Subtracting the other data set from the quantization data type range gives the set of candidate values ​​for the specified value. Determine whether the number of elements in the candidate set of the specified value is greater than or equal to the number of precision categories. If so, select the value of the number of precision categories from the candidate set of the specified value as the specified value; otherwise, perform the following steps. The difference between the number of precision categories and the number of elements in the candidate set for the specified value is used as the supplementary number for the specified value. Select the values ​​from other datasets that have the same number of replacements as the specified value and the fewest duplicates, and fill them into the candidate set of the specified value; Select the value of the precision category number from the candidate set of filled specified values ​​as the specified value.

[0043] In an optional embodiment of this application, in step S300, the search and location of outliers in the quantized dataset matrix can be achieved by using the aforementioned sparse structure, because the sparse structure contains information about the location, specific value, and precision category of the outlier in the dataset matrix.

[0044] In an optional embodiment of this application, the specified value can be a reserved value, which is not occupied by other values ​​in the quantized dataset matrix except for the outlier positions; or the specified value can be a dynamically adjusted value based on other values ​​in the quantized dataset matrix except for the outlier positions, the determination of which includes, for example... Figure 3 The steps shown are as follows: Step 10: Determine the range of values ​​represented by the data type representing the values ​​in the quantized dataset matrix, and let this range be the quantized data type range; If the data type representing the values ​​in the quantized dataset matrix is ​​a 4-bit positive integer, then the range of the quantized data type is 16 positive integers from 1 to 16 (inclusive). The range of quantized data types can be viewed as a set.

[0045] Step 20: For each quantization group in the quantization dataset matrix, let the set of values ​​other than the outlier value be called the set of other values. Subtract the set of other values ​​from the range of quantization data types, and let the result be the candidate set of the specified value. Let A be the set of candidate values ​​for a given value, S be the range of quantized data types, and E be the set of other values. Then the relationship between A, S, and E is: A = S - E For example, suppose the quantized data type ranges from 1 to 16 (inclusive), consisting of 16 positive integers, i.e., S = {1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16}, and the other set of values ​​is E = {7,1,7,5,2,5,10,4,5,7,2,8,9,14,10,14,12,14,15}. Then the candidate set for the specified value is A = SE = {3,6,11,13,16}. Therefore, SE is the set of values ​​remaining after removing all values ​​from S that are included in E.

[0046] It should be noted that the other value sets are equivalent to the other data sets mentioned above. When there is only one quantization group, that unique group is the entire quantization dataset matrix.

[0047] Step 30: Determine whether the number of elements in the candidate set of the specified value is greater than or equal to the number of precision categories. If so, select the value of the number of precision categories from the candidate set of the specified value as the specified value; otherwise, proceed to step 40. For example, suppose the candidate set of specified values ​​is A={3,6,11,13,16}, and the number of precision categories is 2. Obviously, the number of elements in A is 5>2, so 2 elements can be selected from A as the specified values.

[0048] Step 40: Subtract the number of elements in the candidate set for the specified value from the number of precision categories, and set the result as the supplementary number for the specified value; For example, suppose the candidate set A = {6} for a specified value has 1 element and the number of precision categories is 2, then the number of supplementary values ​​for the specified value is 2-1=1.

[0049] Step 50: Select the values ​​from other value sets that have the same number of replacements as the specified value and the fewest repetitions, and fill them into the candidate set of the specified value; For example, suppose the specified number of supplementary values ​​is 2, and the set of other values ​​E = {2,15,7,1,4,7,5,2,5,10,4,5,7,2,1,8,9,14,10,14,12,14,15}, where 8, 9, and 12 are repeated only once. Therefore, 2 numbers can be selected from these 3 numbers to fill the candidate set of the specified value.

[0050] Step 60: Select the value of the precision category number from the candidate set of specified values ​​as the specified value.

[0051] In an optional embodiment of this application, matrix quantization in existing large models is often performed in groups, primarily to significantly reduce memory usage and computational overhead while maintaining model inference accuracy. Quantization converts floating-point numbers (e.g., FP32) into low-bit integers (e.g., INT8, INT4), but directly quantizing the entire matrix uniformly leads to a large range of accuracy losses, especially in layers with uneven weight distribution. Group-wise quantization, by dividing the weight matrix into multiple groups and independently calculating quantization parameters (e.g., scaling factors and zeros) for each group, more accurately adapts to local weight distributions. Each quantization group in the quantization dataset matrix only occupies a portion of the entire quantization dataset matrix; therefore, the probability that other value sets in each quantization group cover the range of quantized data types is extremely small, and consequently, the possibility that the set obtained by subtracting other value sets from the range of quantized data types is an empty set is also extremely small.

[0052] In an optional embodiment of this application, such as Figure 4 The quantized dataset matrix shown assumes each row represents a quantization group, with outlier positions indicated by bold text. It further assumes the quantized data type range is 16 positive integers from 1 to 16 (inclusive), i.e., the quantized data type range S = {1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16}. Figure 4As can be seen, some quantization groups have outliers at two positions, such as the first row; some quantization groups have outliers at only one position, such as rows 2 and 4; and some quantization groups have no outliers, such as rows 3 and 5. If we consider the entire quantization dataset matrix, the set of other values ​​E covers all 16 integer values ​​from 1 to 16 (inclusive), in which case the candidate set A for the specified value obtained in step 20 is empty. If we consider each quantization group in the quantization dataset matrix, the range of data covered by the set of other values ​​E will be significantly reduced. Therefore, the probability that the candidate set A for the specified value obtained in step 20 is a non-empty set will be extremely high. For example, in the first row, the set of other values ​​E is {1,3,5,7,8,12,13,15}, then the candidate set A = SE = {2,4,6,9,10,14}, and A contains 6 elements. Therefore, by using each quantization group in the quantization dataset matrix to determine the set of other values ​​and the candidate set for the specified value, the probability of executing step 40 will be extremely small, improving the determination time of the specified value.

[0053] If the specified value is reserved instead of dynamically selected (i.e., first determining the candidate set of the specified value and then selecting from it), the determination time of the specified value will be reduced. However, the reserved value method requires the use of reserved values ​​within the range of the quantized data type, and the reserved value is not occupied by other values ​​in the quantized dataset matrix except for outlier positions. When quantizing other values, it is necessary to avoid generating the reserved value, which is not conducive to the selection of quantization algorithms, nor is it conducive to saving storage space.

[0054] Once the specified value is determined, the outlier positions in the quantized dataset matrix can be replaced with the specified value. Let's take... Figure 4 Taking the quantized dataset matrix shown as an example, each row is still considered as a quantization group. Assuming the number of precision categories is 2, the bold text "6" in the first row belongs to precision category 1, and the bold text "7" belongs to precision category 2. The bold text "6" in rows 2 and 4 both belong to precision category 2. The bold text "6" and "7" in the first row can be replaced by "9" and "10" from the candidate set of specified values, respectively. The bold text "6" in rows 2 and 4 can be replaced by "9" and "8" from the candidate set of specified values ​​belonging to their respective quantization groups. The values ​​in other positions remain unchanged. The resulting matrix is ​​called... Figure 5 The dense matrix shown. Figure 5 The value of the bold text in the text is the same as the value specified in step S300.

[0055] In an optional embodiment of this application, in step S400, the correlation value between outliers in the sparse structure and a specified value at the location of an outlier in the dense matrix is ​​determined by the following expression: r = f(m) or r = f(m, n) Where r is the correlation value, m is the outlier in the sparse structure, n is the specified value of the outlier position in the dense matrix corresponding to m, and f() is the first preset transformation formula.

[0056] In an optional embodiment of this application, the selection principle of the first preset conversion formula is to minimize the number of bits in the associated value while ensuring performance indicators such as accuracy and error.

[0057] In an optional embodiment of this application, each quantization group corresponds to its own independent sparse part, and the data type, first preset conversion formula and number of bits occupied by the associated values ​​in each sparse part can be the same or different.

[0058] In an optional embodiment of this application, the dense matrix remains unchanged during the generation of the sparse portion. The first preset transformation formula is used to establish the connection between the sparse structure and the sparse portion. Outliers in the sparse structure correspond one-to-one with outlier positions in the dense matrix; that is, each outlier in the sparse structure can be located in the dense matrix. The original outliers and the outliers in the sparse structure are the same values ​​as the outliers in the dataset matrix; they are values ​​that have not undergone quantization, transformation, or other processing.

[0059] In an optional embodiment of this application, the correlation value is generated by a first preset transformation formula. The parameters of the first preset transformation formula include outliers in the sparse structure and may also include values ​​at outlier positions in the dense matrix. The original outlier value or a value close to the original outlier value is generated by a second preset transformation formula. The parameters of the second preset transformation formula include the correlation value and may also include values ​​at outlier positions in the dense matrix, i.e.: r = f(m) or r = f(m, n) m ≈ g(r) or m ≈ g(r, n) Where r is the correlation value, m is the outlier in the sparse structure, n is the specified value of the outlier position in the dense matrix corresponding to m, f() is the first preset transformation formula, and g() is the second preset transformation formula. The first preset transformation formula is used to transform the original outlier, and the second preset transformation formula is used to restore the original outlier or generate a value close to the original outlier. Typically, f() and g() are inverse functions of each other.

[0060] In an optional embodiment of this application, again... Figure 5For example, let's take the bolded "9" in the first row as the specified value "n" for the outlier position in the dense matrix. Assume m = 302.7. If the first preset conversion formula f(m, n) = m ÷ n, then: r = m ÷ n = 302.7 ÷ 9 = 33.63, rounded to two decimal places. If r is an integer, f(m, n) = round(m ÷ n), then r = round(m ÷ n) = round(302.7 ÷ 9) = 34, where the round() function performs rounding to the nearest integer. When using the second preset conversion formula to restore the original outlier value, assuming... If a correlation value r=34, and the specified value of the outlier position in the dense matrix corresponding to m is n=9, then the outlier value in the sparse structure is... It is very close to 302.7.

[0061] In an optional embodiment of this application, a suitable first preset conversion formula f() can be selected according to the precision and bit depth requirements of the associated value to transform the associated value into integers or floating-point numbers of different sizes. For example, a fixed scaling factor t can be added to the first preset conversion formula f(m, n) = m÷n to obtain a new first preset conversion formula f(m, n) = m÷ Different quantization groups can use different first and second preset conversion formulas. Even for the same quantization group, different first and second preset conversion formulas can be used depending on the changes in the values ​​of the data items.

[0062] In an optional embodiment of this application, the selection principle of the first preset conversion formula is to minimize the number of bits in the associated value while ensuring performance requirements such as accuracy and error. For example, if the number of bits in the associated value is 8 after using the first preset conversion formula a, and 6 after using the first preset conversion formula b, then the first preset conversion formula b is preferred, as it is more conducive to saving storage space. The entire dense matrix can generate only one sparse part, that is, all quantized groups in the entire dense matrix share the same sparse part. In order to improve the efficiency of computing and storage units, hardware devices such as GPUs and NPUs often split and group the matrix before computing, rather than operating on the entire matrix at once, such as in the training and inference process of large models. Therefore, the method of generating only one sparse part for the entire dense matrix is ​​not conducive to the utilization efficiency of hardware resources.

[0063] In an optional embodiment of this application, each quantization group corresponds to its own independent sparse part. The data type, first preset conversion formula, and number of bits occupied by the associated values ​​in each sparse part can be the same or different. When performing operations such as dequantization using the dense matrix and sparse part, the dense matrix can be split according to the quantization groups, and then each quantization group and its corresponding sparse part can be calculated independently, giving full play to the parallel computing capabilities of the relevant hardware resources. In addition, different first preset conversion formulas can be used according to the data characteristics of each quantization group and the different performance indicators such as accuracy and error of each group to generate associated values ​​with different data types and different numbers of bits. This is beneficial for fine-grained quantization operations, thereby improving quantization accuracy and saving storage space.

[0064] The sparse part can be stored using a data structure such as an array, which stores the precision class to which the outlier belongs (referred to as precision class), the specified value at the outlier's location (referred to as specified value), and the associated value. Assuming there are two precision classes, the data structure of the sparse part would look like this: Figure 6 As shown, Figure 6 In this context, there are two precision categories, t, with values ​​of 1 and 2. Correspondingly, there are two specified values, v: 9 and 28. There are also two sets of associated values, av1 and av2. That is, the specified value v = 9 and the associated value named av1 belong to the precision category of 1, while the specified value v = 28 and the associated value named av2 belong to the precision category of 2. INT8 indicates that the data type of the associated value named av1 is an 8-bit integer, and FP16 indicates that the data type of the associated value named av2 is a 16-bit floating-point number. Figure 6 It can be seen that the number of precision categories, the number of specified values, and the number of associated values ​​(av1 and av2, a total of 2) are all the same, both being 2. Furthermore, the storage order of values ​​in the precision categories corresponds one-to-one with the storage order of the specified values; that is, t=1 corresponds to v=9 (also corresponding to associated value av1), and t=2 corresponds to v=28 (also corresponding to associated value av2). The storage order of each value in the associated values ​​is consistent with the order of the outlier values ​​in their corresponding dense matrix. For example, suppose a dataset matrix is ​​quantized and grouped by column. The specified values ​​of the outlier positions in the k-th group of its corresponding dense matrix are ordered from top to bottom as {28,9,9,28,9,28,28,28,9,...} (let's call it sequence st). The generated sparse part still follows this order. Figure 6For example, the first to nth values ​​in the associated value av2 correspond to the first to nth values ​​of 28 in the sequence st. That is, 62.1 in av2 (the first value in av2) corresponds to the first occurrence of 28 in st, 103.9 in av2 (the second value in av2) corresponds to the second occurrence of 28 in st, 5.3 in av2 (the third value in av2) corresponds to the third occurrence of 28 in st, and so on.

[0065] In an optional embodiment of this application, the sparse part, through the three components of precision category, specified value of outlier position and associated value, and the setting of the above data structure, can easily generate the original outlier value or a value close to the original outlier value by reverse conversion formula.

[0066] In an optional embodiment of this application, when there is only one precision category, the sparse part may contain only two components: the specified value of the outlier location and the associated value. In this case, the specified value of the outlier location has only one value, and the associated value has only one group, such as... Figure 7 As shown. Figure 7 In this model, there is only one specified value for each outlier location and only one set of associated values. It's not necessary to know which precision category they belong to to know they are paired together; that is, the specified value 'v' for an outlier location with a value of 23 must have an associated value of 'av'. Therefore, when there is only one precision category, the sparse part does not need a precision category data structure. Compared to commonly used sparse matrix compression methods such as Coordinate Format (COO), Compressed Sparse Row (CSR), and Compressed Sparse Column (CSC), the data structure and storage method for the sparse part in this application do not need to store the coordinate information of each outlier location. The storage space occupied by the specified values ​​and precision categories of outliers is minimal, thus saving storage resources.

[0067] In an optional embodiment of this application, the method further includes: Determine the quantization group to which the location to be restored belongs, and obtain the sparse part corresponding to the quantization group; Obtain the value of the restored position in a dense matrix; Determine whether the value at the restored position is contained within the specified value of the outlier position in the sparse part. If so, use the second preset transformation formula in the sparse part to reverse generate the original outlier value or a value close to the original outlier value. Otherwise, proceed to the next step. The original value or a value close to the original value at the restored position is generated using a preset dequantization method.

[0068] In an optional embodiment of this application, when applying the quantized structure in practice, a reverse process is needed to restore the quantized structure to a structure that approximates the original dataset; this process is called the quantization inverse process. Figure 8 As shown, based on the dense matrix and the sparse part, the value at any position in the dataset matrix is ​​approximately restored, including the following steps: Step 100: Determine the quantization group to which the location to be restored belongs, and obtain the sparse part corresponding to the quantization group; The location that needs to be restored refers to the coordinates of that location in the original dataset matrix, such as row 719 and column 3921.

[0069] Step 200: Obtain the value of the restored position in the dense matrix. The value of the restored position in the dense matrix is ​​the value of each position in the dense matrix. The restored position in the dense matrix corresponds one-to-one with the position to be restored. The one-to-one correspondence refers to the same coordinates as the restored positions. For example, the position in the 719th row and 3921st column of the original dataset matrix is ​​also the 719th row and 3921st column in the dense matrix.

[0070] Step 300: Determine whether the value at the restored position is contained within the specified value of the outlier position in the dense matrix of the sparse part. If so, use the second preset transformation formula in the sparse part to generate the original outlier or a value close to the original outlier, which is the approximate value of the required restored position. Otherwise, proceed to the next step. Suppose that there are two specified values ​​v for outlier positions in the sparse part: 9 and 28. If the value of the restored position is 28, since v contains 28, it can be considered that the value of the restored position is included within the specified value of the outlier position in the sparse part. If the value of the restored position is 21, but v does not contain 21, then the value of the restored position is not included within the specified value of the outlier position in the sparse part. It can be determined that the value of the restored position cannot be used to reverse the generation of the original outlier value or a value close to the original outlier value using the second preset transformation formula in the sparse part.

[0071] The original outlier or a value close to the original outlier is generated by using the sparse part and the second preset transformation formula. The specific method and examples have been explained above.

[0072] Step 400: Use a preset inverse quantization method to generate the original value of the restored position or a value close to the original value, which is the approximate value of the required restored position.

[0073] If it is not an outlier location, then a standard dequantization method can be used to generate an approximation of the desired location.

[0074] The dataset matrix quantization method described in this invention can solve the technical problem of insufficient computational and storage efficiency in existing "divide and conquer" quantization techniques. It is expected to have considerable practical prospects or reference value in the fields of large model training and inference, big data processing, and even video encoding and decoding.

[0075] It should be understood that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order constraint on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the diagram may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0076] Please see Figure 9 One embodiment of this application provides a dataset matrix quantization device 900, comprising: The identification module 910 is used to identify and extract all outliers in the target dataset matrix, and to label the precision category to which all outliers belong according to different precision requirements. The number of precision categories is an integer greater than or equal to 0, and the sparse structure is composed of the extracted outliers, the position of the outliers in the target dataset matrix and their respective precision categories. The quantization module 920 is used to quantize the data in the target dataset matrix and generate a quantized dataset matrix. The first generation module 930 is used to replace outliers with specified values ​​at outlier positions in the quantized dataset matrix, while keeping the values ​​at other positions unchanged, to generate a dense matrix. The second generation module 940 is used to generate a sparse part based on the dense matrix and the sparse structure, and use the dense matrix and the sparse part as the matrix quantization result of the target dataset. The sparse part includes a specified value at the position of the outlier in the dense matrix, a correlation value between the outlier in the sparse structure and the specified value at the position of the outlier in the dense matrix, and the precision category to which the outlier belongs. The correlation value is used to reverse generate the original outlier or a value close to the original outlier.

[0077] Specific limitations regarding the aforementioned device 900 can be found in the above description of the dataset matrix quantization method, and will not be repeated here. Each module in the aforementioned device 900 can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0078] In one embodiment, a computer device is provided, the internal structure of which can be as follows: Figure 10 As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a dataset matrix quantization method as described above. It includes: memory and a processor; the memory stores a computer program; and the processor executes the computer program to implement any step of the dataset matrix quantization method described above.

[0079] In one embodiment, a computer-readable and writable storage medium is provided having a computer program stored thereon, which, when executed by a processor, can implement any step of the dataset matrix quantization method described above.

[0080] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0081] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0082] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0083] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0084] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0085] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for quantizing a dataset matrix, characterized in that, include: Identify and extract all outliers in the target dataset matrix, and label the precision categories to which all outliers belong according to different precision requirements. The number of precision categories is an integer greater than or equal to 0. The sparse structure is composed of the extracted outliers, their positions in the target dataset matrix, and their precision categories. Quantize the data in the target dataset matrix to generate a quantized dataset matrix; In the quantized dataset matrix, outliers are replaced with specified values ​​at their locations, while the values ​​at other locations remain unchanged, generating a dense matrix. A sparse component is generated based on the dense matrix and the sparse structure. The dense matrix and the sparse component are then used as the matrix quantization result of the target dataset. The sparse component includes a specified value at the location of outliers in the dense matrix, a correlation value between the outliers in the sparse structure and the specified value at the location of the outliers in the dense matrix, and the precision category to which the outliers belong. The correlation value is used to reverse-generate the original outliers or values ​​close to the original outliers. The specified value is determined through the following steps: Determine the range of values ​​represented by the data type representing the data in the quantized dataset matrix, where the range is the quantized data type range; For each quantization group in the quantization dataset matrix, the set of other data outside the outlier position in the current quantization group is called the other data set. Subtracting the other data set from the quantization data type range gives the set of candidate values ​​for the specified value. Determine whether the number of elements in the candidate set of the specified value is greater than or equal to the number of precision categories. If so, select the value of the number of precision categories from the candidate set of the specified value as the specified value; otherwise, perform the following steps. The difference between the number of precision categories and the number of elements in the candidate set for the specified value is used as the supplementary number for the specified value. Select the values ​​from other datasets that have the same number of replacements as the specified value and the fewest duplicates, and fill them into the candidate set of the specified value; Select the value of the precision category number from the candidate set of filled specified values ​​as the specified value.

2. The method according to claim 1, characterized in that, The process of identifying and extracting all outliers from the target dataset matrix includes: Data that meets any of the following four conditions will be considered outliers in the target dataset matrix: First, extreme data that deviates from the preset range; Second, data that deviates from the median of all data in the target dataset matrix or the median of all data in the quantized group to which it belongs by more than N times; Third, data whose impact on performance is ranked higher than the preset value; Fourth, data that significantly impacts the number of bits required for each quantized data point, while ensuring the preset quantization accuracy requirements are met.

3. The method according to claim 2, characterized in that, The method involves labeling all outliers according to different precision requirements, including: Outliers containing at least one of the following three categories are labeled as low-precision categories: extreme data that deviates from the preset range; data that deviates from the median of all data in the target dataset matrix or all data in the quantization group by more than N times; and data that has a significant impact on the number of bits required for each data after quantization, while ensuring the preset quantization precision requirements. Outliers that have a performance impact higher than the preset value are marked as high-precision categories.

4. The method according to claim 1, characterized in that, Before quantizing the data in the target dataset matrix, the method further includes: The outlier positions in the target dataset matrix are replaced with a first specified value, which includes one of the following: the zero point value of the quantization algorithm, the median value of the data other than the outlier in the target dataset matrix or the quantization group, and the arithmetic mean of the data other than the outlier. When the zero points of the quantization algorithm are already determined, the positions in the target dataset matrix where outliers exist are the zero points of the quantization algorithm. When the quantization algorithm and quantization parameters are dynamically selected based on the data and distribution in the target dataset matrix, and the target dataset matrix is ​​divided into blocks with different quantization algorithms and quantization parameters selected for each block, the location of outliers in the target dataset matrix is ​​the median or arithmetic mean of the other data in the target dataset matrix, excluding the outliers.

5. The method according to claim 1, characterized in that, The following expression determines the correlation between outliers in a sparse structure and a specified value at the location of an outlier in a dense matrix: r = f(m) or r = f(m, n) Where r is the correlation value, m is the outlier in the sparse structure, n is the specified value of the outlier position in the dense matrix corresponding to m, and f() is the first preset transformation formula.

6. The method according to claim 1, characterized in that, The method further includes: Determine the quantization group to which the location to be restored belongs, and obtain the sparse part corresponding to the quantization group; Obtain the value of the restored position in a dense matrix; Determine whether the value at the restored position is contained within the specified value at the outlier position in the dense matrix of the sparse part. If so, use the second preset transformation formula in the sparse part to reverse generate the original outlier or a value close to the original outlier. Otherwise, proceed to the next step. The original value or a value close to the original value at the restored position is generated using a preset dequantization method.

7. A dataset matrix quantization device, characterized in that, include: The identification module is used to identify and extract all outliers in the target dataset matrix, and to label the precision category to which all outliers belong according to different precision requirements. The number of precision categories is an integer greater than or equal to 0, and the module consists of the extracted outliers, their positions in the target dataset matrix and their precision categories, forming a sparse structure. The quantization module is used to quantize the data in the target dataset matrix and generate a quantized dataset matrix. The first generation module is used to replace outliers with specified values ​​at outlier positions in the quantized dataset matrix, while keeping the values ​​at other positions unchanged, to generate a dense matrix. The second generation module is used to generate a sparse part based on the dense matrix and the sparse structure. The dense matrix and the sparse part are then used as the matrix quantization result of the target dataset. The sparse part includes specified values ​​at the outlier positions in the dense matrix, correlation values ​​between the outliers in the sparse structure and the specified values ​​at the outlier positions in the dense matrix, and the precision category to which the outliers belong. The correlation values ​​are used to reverse-generate the original outliers or values ​​close to the original outliers. The specified value is determined through the following steps: Determine the range of values ​​represented by the data type representing the data in the quantized dataset matrix, where the range is the quantized data type range; For each quantization group in the quantization dataset matrix, the set of other data outside the outlier position in the current quantization group is called the other data set. Subtracting the other data set from the quantization data type range gives the set of candidate values ​​for the specified value. Determine whether the number of elements in the candidate set of the specified value is greater than or equal to the number of precision categories. If so, select the value of the number of precision categories from the candidate set of the specified value as the specified value; otherwise, perform the following steps. The difference between the number of precision categories and the number of elements in the candidate set for the specified value is used as the supplementary number for the specified value. Select the values ​​from other datasets that have the same number of replacements as the specified value and the fewest duplicates, and fill them into the candidate set of the specified value; Select the value of the precision category number from the candidate set of filled specified values ​​as the specified value.

8. A computer device, comprising: A memory and a processor, the memory storing a computer program, characterized in that the processor executes the computer program to implement the steps of the dataset matrix quantization method according to any one of claims 1 to 6.

9. A computer-readable and writable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the dataset matrix quantization method according to any one of claims 1 to 6.

Images