Model compression method based on low-bit floating-point number and quantization after model training

Abstract

The invention discloses a model compression method based on a low-bit floating-point number and quantization after model training. The model compression is suitable for a convolutional network and a linear network. And a user is supported to compress a high-precision model into a model represented by a 4-bit or 8-bit low-bit floating-point number. According to the method, model compression is carried out through a bit width set by a user, in a model quantization module layer, a configuration search algorithm is used for finding an optimal low-bit floating-point number quantization configuration and supporting a mixed quantization scheme with the same bit width, and floating-point numbers with the same bit width have various combinations of exponent bits and mantissa bits; under the scene that data calibration exists between model quantization module layers, a joint calibration optimization process is used, output of an original model serves as a data calibration object of a current quantized model, output errors between the quantized model and the original model are reduced to the maximum extent, and therefore the precision of the compressed model is maintained. The model compression method for performing PTQ by using the low-bit floating-point number is provided, a new scheme is provided for PTQ, the method can be widely applied to quantization tasks after training of various model compression, and the precision after model compression is maintained.

Description

Technical Field

[0001] The present invention relates to PTQ optimization in the field of model compression, and more specifically, to a method for performing post-model training quantization on trained model weights using a low-bit floating point data format. Background Art

[0002] Post-Training Quantization (PTQ) is a model compression technology that is used to convert model parameters (such as weights and activation values) from high-precision data formats (such as 32-bit floating-point numbers) to low-precision data formats (such as 4-bit and 8-bit integers) after model training is completed, so as to reduce the storage requirements and computational load of the model while maintaining the accuracy and performance of the compressed model as much as possible.

[0003] Floating point numbers are a data type used by computers to represent real numbers. They consist of three parts: a sign bit, an exponent, and a mantissa. The sign bit is used to indicate the positive or negative value. The exponent is used to control the range of the value. The mantissa is used to control the precision of the value. 32-bit floating point numbers are usually used to represent values. They can represent a wider range of values ​​and are suitable for calculations that require high precision and dynamic range. Low-bit floating point numbers refer to floating point numbers that are represented by a smaller bit width (4 bits or 8 bits). Compared with 32-bit floating point numbers, they can reduce the resources consumed by data storage and calculation.

[0004] The model in Model Compression refers to an image or language model based on a convolutional network and a linear network. The image or language model is widely used in many tasks, including image classification, generative language dialogue, etc. As the complexity of deep learning models continues to increase, the scale of model parameters and the amount of computation have also increased exponentially, especially for large models in recent years. The parameters of the model have expanded rapidly, reaching hundreds of millions or even hundreds of billions of parameters, which requires a large amount of video memory and computing resources. When deploying and implementing deep learning models, severe storage and computing overhead will be encountered.

[0005] Current model compression mostly uses low-bit integer PTQ methods (especially 8-bit integer PTQ methods), which have been widely used in the industry, but it still has some shortcomings and limitations.

[0006] First of all, outliers in integer quantization are difficult to handle. Integer quantization evenly divides a range of data. If abnormally large values ​​appear in the tensor, the scale factor in integer quantization will be affected by these abnormal values. When the maximum value of the tensor corresponds to the maximum value represented by the integer, most of the remaining values ​​will be mapped to a smaller range, and some values ​​will be mapped to the same integer value, resulting in serious compression loss of model accuracy. This phenomenon is especially present in complex large models.

[0007] Secondly, the dynamic range of integer quantization is limited. An 8-bit integer can only represent integers from -128 to +127. This limited dynamic range cannot effectively capture the differences in data distribution in different module layers when processing weights or activation values ​​in different module layers. In particular, the distribution of some weights and activation values ​​is relatively wide, which can easily lead to loss of accuracy after model compression.

[0008] Finally, the accuracy of integer quantization is insufficient. Although 8-bit integer quantization can maintain accuracy close to the original model (the model before the application of model compression technology is called the original model) and meet the needs of most scenarios, for tasks with higher precision requirements, such as the GPT model in natural language processing, there will often be a significant decrease in accuracy after quantization, affecting the performance of the quantized model (the model after the application of model compression technology is called the quantized model). Summary of the invention

[0009] In order to solve the technical problems of difficulty in handling outliers, limited dynamic range and reduced model accuracy when using low-bit integers for model post-training quantization (PTQ) in model compression, the present invention proposes a model compression method using low-bit floating-point numbers for PTQ. Low-bit floating-point quantization uses 4-bit or 8-bit floating-point representations, and uses different combinations of exponent bits and mantissa bits of floating-point numbers to provide a larger dynamic range and numerical flexibility under the same bit width, which can effectively reduce the loss of model compression accuracy caused by outliers or uneven data distribution. Compared with traditional low-bit integer quantization methods, low-bit floating-point numbers can better represent the complex weight and activation value distribution of the model in model compression, especially in models with larger parameter quantities and deeper hierarchical networks. The method of the present invention can reduce storage overhead and accelerate calculations while maintaining the accuracy of the quantized model.

[0010] The present invention provides a model compression method based on low-bit floating point numbers and quantization after model training, wherein the following steps are performed in the tuning between model layers:

[0011] Step 1: quantize and initialize the model;

[0012] Step 1-1, model loading;

[0013] Step 1-2, the user sets the PTQ parameters;

[0014] Steps 1-3, initialize the loading model;

[0015] Steps 1-4, get the iterator of the model;

[0016] Steps 1-5, determine whether the current network layer needs to be quantized;

[0017] Steps 1-6, retain the original network layer for the network layer that does not need to be quantized;

[0018] Step 2: quantize the network layer by layer;

[0019] Step 2-1, determine whether there is a calibration data set for the first network layer that needs to be quantized and quantize it;

[0020] Step 2-2, for the first network layer that does not need to be quantized, determine whether there is a calibration data set and quantize it;

[0021] Step 2-3: for any network layer that needs to be quantized after the first network layer, determine whether there is a calibration data set and perform calculation processing;

[0022] Steps 2-4: For any network layer after the first network layer that does not require quantization, determine whether there is a calibration data set and perform calculation processing;

[0023] Steps 2-5, complete model quantization and save quantization configuration;

[0024] Step 3: Export the weights of the quantized model by weight compression;

[0025] Step 3-1, converting the weight of the quantization module;

[0026] Step 3-2, extracting low-bit floating point numbers for compression;

[0027] Step 3-3, replace the weights and export the model.

[0028] The present invention uses low-bit floating point numbers to perform PTQ model compression, which is divided into two parts:

[0029] The first part is the tuning between model layers, which is between the model quantization module layers. In the scenario with data calibration, an optimization method of joint calibration is proposed, which allows the quantized model to use the full-process output of the original model for quantitative calibration, making the output of the quantized model closer to the original model.

[0030] The second part is the quantization within the model quantization layer. In the model quantization module layer, a hybrid low-bit floating-point quantization search algorithm is proposed for low-bit floating-point quantization. The optimal low-bit floating-point format configuration and tensor maximum value are obtained through search.

[0031] The advantages of the model compression method based on low-bit floating point numbers and PTQ of the present invention are:

[0032] ① Introducing low-bit floating-point numbers into the post-training quantization of the model, introducing a combination of exponent bits and mantissa bits, and providing a wider dynamic range than traditional integer quantization by mixing the data format combination of low-bit floating-point numbers. This can better adapt to the widely distributed weights and activation values ​​in complex models, especially the quantization of outliers;

[0033] ② A hybrid low-bit floating-point quantization search method is implemented, so that different quantization modules in a model use different low-bit floating-point quantization formats. Different combinations of exponent bits and mantissa bits under the same bit width can dynamically adjust the floating-point format according to the numerical distribution characteristics of different layers or weights of the model. Some layers use more exponent bits to represent a wider data range, and some layers use more mantissa bits to represent higher data accuracy, thereby ensuring that the extreme values ​​and accuracy of different modules in complex models can be well represented;

[0034] ③ The joint calibration optimization method in the small batch data calibration scenario can minimize the output error between the quantized model and the original model, especially in the scenario of mixed low-bit floating-point number configuration. Different modules may use different floating-point number configurations, which makes it more likely that cross-layer quantization errors will occur, resulting in a decrease in model accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart of constructing the quantization layer of the QuantizedConv2d network of the present invention.

[0036] Figure 2 It is a flow chart of constructing the QuantizedLinear network quantization layer of the present invention.

[0037] Figure 3 It is a flow chart of model compression based on quantization after model training of low-bit floating point numbers in the present invention.

[0038] Figure 4 This is a weight compression structure diagram of the present invention. DETAILED DESCRIPTION

[0039] In order to more clearly illustrate the technical solution and content of the present invention, the present invention is further described in detail below with reference to the accompanying drawings.

[0040] In the process of model compression, there are two main types of networks for model compression: the first is a convolutional network, mainly a Conv2d convolutional network layer; the second is a linear network, namely a Linear linear network. In the present invention, the PTQ of low-bit floating-point numbers also performs low-bit floating-point quantization compression processing on these two types of networks.

[0041] See also Figure 1 , Figure 2 , Figure 3 As shown, the present invention is a model compression method based on post-training quantization (PTQ) of a low-bit floating-point model. In the model compression process, on the one hand, it is model inter-layer scheduling based on layer-by-layer joint calibration executed sequentially by the network (referred to as inter-model layer tuning), and on the other hand, it is model intra-layer quantization based on low-bit floating-point number configuration search (referred to as intra-model layer quantization).

[0042] 1. Constructing the QuantizedConv2d network quantization layer

[0043] In the present invention, the QuantizedConv2d network quantization layer is used for the low-bit floating point PTQ quantization of the Conv2d network layer, and a weight quantizer is added to the Conv2d network layer. The weight quantizer includes a quantization granularity MP[PT, PC], a low-bit floating point bit width BW[4,8], a tensor maximum value W_MAX, and a low-bit floating point format LFF. Through the intra-layer search quantization of the QuantizedConv2d network quantization layer, the PTQ model compression of the low-bit floating point of the Conv2d network layer is realized.

[0044] The quantization granularity is divided into per-tensor and per-channel levels. For the quantization granularity, please refer to the arXiv preprint arXiv:2106.08295 published on June 15, 2021, "A White Paper on NeuralNetwork Quantization", Section 2.2.3 Quantization granularity, author Markus Nagel et al.

[0045] In the present invention, the quantization granularity is recorded as MP[PT, PC], where PT represents the tensor-level quantization granularity and PC represents the channel-level quantization granularity.

[0046] See also Figure 1 As shown, in the model compression process based on low-bit floating point numbers and quantization after model training in the present invention, the following steps are involved in constructing the QuantizedConv2d network quantization layer:

[0047] Step A, initializing the search space of low-bit floating point format of weights;

[0048] Initialize the low-bit floating-point format search space C of the weight according to the bit width BW[4,8] set by the user.

[0049] The low-bit floating-point number bit width BW[4,8] is selected from two options, one is 4 bits and the other is 8 bits.

[0050] Taking an 8-bit floating point number as an example, the search space is initialized to C = {E6M1, E5M2, E4M3, E3M4, E2M5, E1M6}, where E represents the number of exponent bits and M represents the format of the mantissa bits.

[0051] In binary representation, unlike integer representation, the binary representation of floating-point numbers can be divided into three parts: sign, exponent, and mantissa. A floating-point number corresponding to a binary number can be expressed as Where X FP Represents a floating point number, s represents the sign bit, and p represents the range [0,2 e -1], e represents the exponent value, b represents the offset, and the default value is 2 E-1 -1, E represents the number of exponent bits, d m Indicates the value of the mth bit of the mantissa, which can be 0 or 1, where m represents the number of bits in the mantissa.

[0052] For floating-point numbers with the same bit width, there are many different combinations of exponent bits and mantissa bits. Taking 8-bit floating-point numbers as an example, after removing the 1-bit sign bit, the sum of the exponent bits and mantissa bits is 7, so there are six combinations of exponent bits and mantissa bits: {E6M1, E5M2, E4M3, E3M4, E2M5, E1M6}. If there are more exponent bits, the current floating-point format can represent a larger dynamic range of data, and can more effectively deal with the problem of abnormal values, but the accuracy of the data that can be represented is lower; if there are more mantissa bits, the current floating-point format can represent data with higher accuracy and more precision, but it will affect the representation range of the data and make it difficult to deal with the problem of abnormal values.

[0053] Step B, initializing the weight maximum search space;

[0054] In the present invention, the quantization granularity is recorded as MP[PT, PC], where PT represents the tensor-level quantization granularity and PC represents the channel-level quantization granularity.

[0055] Step b-1, determining the quantization granularity of the weight quantizer;

[0056] If it is PT, go to step b-2;

[0057] If it is PC, go to step b-3;

[0058] Step b-2, initializing the search space ST of the maximum weight according to the PT;

[0059] Get the maximum value of the current entire weight tensor and record it as W max , configure the adjustable parameters lower bound percentage point β1, upper bound percentage point β2 and the number of equally divided search shares k, the present invention sets the search space ST of the maximum weight to [β1W max ,β2W max ], and is evenly divided into k search points, the search space of the maximum weight at the tensor level granularity Go to step C.

[0060] Step b-3, initializing the search space SC of the maximum weight according to the PC;

[0061] The maximum value of each channel of the weight tensor is obtained at the channel level granularity and recorded as j represents the jth channel, and the adjustable parameters lower bound percentage point β1, upper bound percentage point β2 and the number of equally divided search shares k are configured. The present invention sets the search space SC of the maximum weight according to the channel, and the search space range of the jth channel is And divided into k search points, the search space with the maximum value of the jth channel weight Go to step C.

[0062] In the present invention, the operations of the tensor-level weight maximum search space ST and the channel-level weight maximum search space SC in the search process are the same, and are collectively referred to as the weight maximum search space S[ST, SC].

[0063] Step C, calculating the quantization errors of all low-bit floating point formats and weight maximum value combinations;

[0064] Step c-1, the first loop traverses the low-bit floating point number configuration search space C;

[0065] The floating point format in the weight quantizer is set, that is, the floating point format in the weight quantizer is set to an element in a low-bit floating point configuration search space C, and the elements in the low-bit floating point configuration search space are traversed in order, and the identifier of the first layer loop is denoted by c.

[0066] Taking 8-bit floating-point numbers as an example, the search space is C = {E6M1, E5M2, E4M3, E3M4, E2M5, E1M6}. In the first round of loop, the floating-point number format in the weight quantizer is set to E6M1. The weight quantizer will quantize the weight according to the exponent bit as 6 bits and the median bit as 1 bit.

[0067] Step c-2, the second loop traverses the maximum weight search space S[ST,SC];

[0068] Set the maximum value in the weight quantizer, that is, set it to the search object in the maximum value search space S[ST,SC], traverse the elements in the maximum value search space in order, and the identifier of the second layer loop is denoted by s.

[0069] Taking tensor-level granularity as an example, the search space for the maximum weight of tensor-level granularity is In the first cycle, set the maximum value of the tensor in the weight quantizer to The weight tensor quantizer will constrain the weight range according to the set maximum value of the tensor.

[0070] Step c-3, determine whether there are calibration data sets CDS_Q and CDS_O, and perform quantitative calibration;

[0071] The calibration data set used by the quantized module is recorded as CDS_Q, and the calibration data set used by the original module is recorded as CDS_O.

[0072] The CDS_Q and CDS_O may be 32, 64 or 128 samples, the number of samples of the two may be the same, and the two may exist at the same time or not exist at the same time.

[0073] If there are no calibration data sets CDS_Q and CDS_O, proceed to step c-4;

[0074] If there are calibration data sets CDS_Q and CDS_O, proceed to step c-5.

[0075] In PTQ, MSE (Mean Squared Error) is used to measure the difference between the model before and after quantization. The basic formula is: n represents the number of elements in the tensor, i represents the identification number of the element in the tensor, and X i represents the original value of the i-th element, Represents the predicted value of the i-th element.

[0076] The low-bit floating point quantization calculation method refers to arXiv preprint arXiv:2208.09225 published on August 19, 2022, FP8 Quantization:The Power of the Exponent, Section 4.1, 2022 NIPS. The present invention can simulate the result of the current tensor calculated after quantization by low-bit floating point according to the exponent bit and mantissa bit configuration of the low-bit floating point number and the set tensor maximum value.

[0077] Step c-4, in the scenario without calibration data, quantify the error record;

[0078] In a scenario without calibration data, the optimization goal of quantization is to minimize the error between the tensor itself before and after quantization, calculate the tensor quantization error under the current configuration, and record it, and then execute step c-6.

[0079] If the quantization granularity is PT, use the tensor-level weighted mean square error formula To calculate the error size of the model tensor before and after quantization, ntt represents the number of elements in the tensor without calibration data, and W represents the weight tensor of the model before quantization. Represents the weight tensor of the quantized model. The tensor-level weight mean square error mse_pt_t under the current low-bit floating-point configuration and the tensor maximum configuration is calculated and recorded in the intermediate result PT_T_ERROR[c][s]. PT_T_ERROR is a two-dimensional matrix that records the tensor-level weight errors under each low-bit floating-point configuration and each tensor maximum configuration. c is the identifier of the first-layer loop, which corresponds to the current low-bit floating-point configuration, and s is the identifier of the second-layer loop, which corresponds to the current tensor maximum configuration.

[0080] When the quantization granularity is PC, the channel-level weighted mean square error formula is used To calculate the error size of the model tensor channel j before and after quantization, where nct j represents the number of elements in the output tensor channel j in the scenario without calibration data, W j represents the weight tensor of the model before quantization, Represents the weight tensor of the quantized model. Calculate the channel-level weight mean square error mse_pc_t under the current low-bit floating-point configuration and tensor maximum configuration, and record it in the intermediate result PC_T_ERROR[c][s][j]. PC_T_ERROR is a three-dimensional matrix that records each low-bit floating-point configuration, each tensor maximum configuration, and the channel-level weight error under each channel. j represents the channel of the tensor.

[0081] Step c-5, in the scenario with calibration data, quantify the error record;

[0082] In the scenario with calibration data, the optimization goal of quantization is to minimize the output error before and after quantization. The output error of the quantized tensor under the current configuration is calculated and recorded, and then step c-6 is executed.

[0083] If the quantization granularity is PT, use the tensor-level output mean square error formula To calculate the error size of the module output before and after quantization, nto represents the number of elements in the output tensor in the scenario with calibration data, CDS_O represents the calibration data set used by the original module, CDS_Q represents the calibration data set used by the module after quantization, and W represents the weight tensor value of the model before quantization. Indicates the weight tensor value of the quantized model, func indicates the function torch.nn.functional.conv2d corresponding to the Conv2d module in pytorch. Calculate the tensor-level output mean square error mse_pt_o under the current low-bit floating-point configuration and the tensor maximum configuration, and record it in the intermediate result PT_O_ERROR[c][s]. PT_O_ERROR is a two-dimensional matrix that records the tensor-level output errors under each low-bit floating-point configuration and each tensor maximum configuration.

[0084] If the quantization granularity is PC, use the tensor-level output mean square error formula To calculate the error size of the module output before and after quantization, where nco j Indicates the number of elements in the output tensor channel j, CDS_O j Indicates the data corresponding to channel j of the calibration data set used by the original module, CDS_Q j Represents the data corresponding to channel j in the calibration data set used by the quantized module, W j represents the weight tensor of the model before quantization, Represents the weight tensor of the quantized model, and func represents the function torch.nn.functional.conv2d corresponding to the Conv2d module in pytorch. The channel-level output mean square error mse_pc_o under the current low-bit floating-point configuration and tensor maximum configuration is calculated and recorded in the intermediate result PC_O_ERROR[c][s][j]. PC_O_ERROR is a three-dimensional matrix that records each low-bit floating-point configuration, each tensor maximum configuration, and the channel-level output error under each channel.

[0085] Step c-6, determining whether the maximum value search space has been traversed;

[0086] If the elements in the S[ST,SC] space are traversed, execute step c-7;

[0087] If the elements in the S[ST,SC] space have not been traversed, execute step c-2.

[0088] Step c-7, determining whether the low-bit floating point number configuration search space has been traversed;

[0089] If the elements in space C are traversed, execute step D;

[0090] If the elements in space C have not been traversed, execute step c-1.

[0091] In the present invention, step c-6 and step c-7 are for finding the best maximum value and low-bit floating point format configuration from the S[ST, SC] space and the C space, so as to minimize the error between the current quantized tensor and the original tensor.

[0092] Step D, obtaining an optimal low-bit floating point format and weight maximum value configuration;

[0093] Step d-1, determining whether the quantization granularity of the weight quantizer is at the tensor level;

[0094] If the quantization granularity of the weight quantizer is PT, execute step d-2;

[0095] If the quantization granularity of the weight quantizer is PC, execute step d-3.

[0096] Step d-2, updating the configuration in the weight quantizer according to the tensor level;

[0097] In the present invention, the operations on the tensor level weight mean square error PT_T_ERROR[c][s] and the tensor level output mean square error PT_O_ERROR[c][s] in value taking are the same, and are collectively referred to as the tensor level mean square error PT_ERROR[c][s].

[0098] Find the minimum tensor-level mean square error PT_ERROR[c min ][s min ], c min is the low-bit floating-point search space subscript corresponding to the minimum tensor-level mean square error, s min It is the search space subscript of the maximum weight corresponding to the minimum tensor-level mean square error.

[0099] Update the corresponding low-bit floating-point format LFF in the weight quantizer to the corresponding low-bit floating-point search space C min The element corresponding to the subscript, that is, LFF = C[c min ]; Update the corresponding maximum weight value W_MAX in the weight quantizer to the corresponding maximum weight value in the search space ST s min The element corresponding to the subscript, that is, W_MAX = ST[s min ], execute step d-4;

[0100] Step d-3, updating the configuration in the weight quantizer according to the channel level;

[0101] In the present invention, the operations on the channel-level weighted mean square error PC_T_ERROR[c][s][j] and the channel-level output mean square error PC_O_ERROR[c][s][j] in value selection are the same, collectively referred to as the channel-level mean square error PC_ERROR[c][s][j].

[0102] Find the minimum channel-level mean square error in the three-dimensional matrix corresponding to each channel channel by channel is the low-bit floating-point search space subscript corresponding to the minimum mean square error of channel j, is the search space subscript of the maximum weight corresponding to the minimum mean square error of channel j.

[0103] Select The c with the highest number of occurrences min As the corresponding low-bit floating-point configuration subscript in the weight quantizer, the corresponding low-bit floating-point format LFF in the weight quantizer is updated to the corresponding low-bit floating-point search space C in c min The element corresponding to the subscript, that is, LFF = C[c min ].

[0104] After determining the elements of the low-bit floating-point search space in the weight quantizer, find the minimum channel-level mean square error in the three-dimensional matrix corresponding to each channel channel by channel For min The search space subscript of the maximum weight corresponding to the minimum mean square error in channel j, and the corresponding maximum weight W_MAX in the weight quantizer is calculated channel by channel j , updated to the corresponding search space SC with the maximum weight j middle The element corresponding to the subscript, that is Execute step d-4.

[0105] Step d-4, determine whether there are calibration data sets CDS_Q and CDS_O, and if so, calculate the input of the next network layer;

[0106] The calibration data set used by the quantized module is recorded as CDS_Q, and the calibration data set used by the original module is recorded as CDS_O.

[0107] If there are no calibration data sets CDS_Q and CDS_O, the execution is completed;

[0108] If there are calibration data sets CDS_Q and CDS_O, use the weight quantizer after updating the configuration to calculate the quantized weights According to the input calibration data set CDS_Q, the output of the quantization module is recorded as CDS_QC_O. Assign it to CDS_QC_O, that is According to the input calibration data set CDS_O, the original module output is recorded as CDS_OC_O, and func(CDS_O,W) is assigned to CDS_OC_O, that is, CDS_OC_O = func(CDS_O,W), where func represents the function torch.nn.functional.conv2d corresponding to the Conv2d module in pytorch. CDS_QC_O and CDS_OC_O are used as the input of the next layer of the network.

[0109] 2. Constructing the Quantized Linear Network Quantization Layer

[0110] In the present invention, the method for constructing the quantized layer of the QuantizedLinear network is different from the method for constructing the quantized layer of the QuantizedConv2d network in the applied functions. The function func used by QuantizedConv2d is torch.nn.functional.conv2d. The function func used by QuantizedLinear is torch.nn.functional.linear.

[0111] In the present invention, the QuantizedLinear network quantization layer is used for the low-bit floating point PTQ quantization of the Linear network layer, and a weight quantizer is added to the Linear network layer. The weight quantizer includes a quantization granularity MP[PT, PC], a low-bit floating point bit width BW[4,8], a tensor maximum value W_MAX, and a low-bit floating point format LFF. Through the intra-layer search quantization of the QuantizedLinear network quantization layer, the PTQ model compression of the low-bit floating point of the Linear network layer is realized.

[0112] See also Figure 2 As shown, in the model compression process based on low-bit floating point numbers and quantization after model training of the present invention, the following steps are involved in constructing the QuantizedLinear network quantization layer:

[0113] Step I, initializing the search space of low-bit floating point format of weights;

[0114] Initialize the low-bit floating-point format search space C of the weight according to the bit width BW[4,8] set by the user.

[0115] The low-bit floating-point number bit width BW[4,8] is selected from two options, one is 4 bits and the other is 8 bits.

[0116] Taking an 8-bit floating point number as an example, the search space is initialized to C = {E6M1, E5M2, E4M3, E3M4, E2M5, E1M6}, where E represents the number of exponent bits and M represents the format of the mantissa bits.

[0117] In binary representation, unlike integer representation, the binary representation of floating point numbers can be divided into three parts: sign, exponent, and mantissa.

[0118] The corresponding floating point number can be expressed as Where X FP Represents a floating point number, s represents the sign bit, and p represents the range [0,2 e -1], e represents the exponent value, b represents the offset, and the default value is 2 E-1 -1, E represents the number of exponent bits, d m Indicates the value of the mth bit of the mantissa, which can be 0 or 1, where m represents the number of bits in the mantissa.

[0119] For floating-point numbers with the same bit width, there are many different combinations of exponent bits and mantissa bits. Taking 8-bit floating-point numbers as an example, after removing the 1-bit sign bit, the sum of the exponent bits and mantissa bits is 7, so there are six combinations of exponent bits and mantissa bits: {E6M1, E5M2, E4M3, E3M4, E2M5, E1M6}. If there are more exponent bits, the current floating-point format can represent a larger dynamic range of data, and can more effectively deal with the problem of abnormal values, but the accuracy of the data that can be represented is lower; if there are more mantissa bits, the current floating-point format can represent data with higher accuracy and more precision, but it will affect the representation range of the data and make it difficult to deal with the problem of abnormal values.

[0120] Step II, initialize the weight maximum search space;

[0121] In the present invention, the quantization granularity is recorded as MP[PT, PC], where PT represents the tensor-level quantization granularity and PC represents the channel-level quantization granularity.

[0122] Step II-1, determine the quantization granularity of the weight quantizer;

[0123] If it is PT, proceed to step II-2;

[0124] If it is PC, proceed to step II-3;

[0125] Step II-2, initializing the search space ST of the maximum weight according to the PT;

[0126] Get the maximum value of the current entire weight tensor and record it as W max, configure the adjustable parameters lower bound percentage point β1, upper bound percentage point β2 and the number of equally divided search shares k, the present invention sets the search space ST of the maximum weight to [β1W max ,β2W max ], and is evenly divided into k search points, the search space of the maximum weight at the tensor level granularity Execute step III.

[0127] Step II-3, initializing the search space SC of the maximum weight according to the PC;

[0128] The maximum value of each channel of the weight tensor is obtained at the channel level granularity and recorded as j represents the jth channel, and the adjustable parameters lower bound percentage point β1, upper bound percentage point β2 and the number of equally divided search shares k are configured. The present invention sets the search space SC of the maximum weight according to the channel, and the search space range of the jth channel is And divided into k search points, the search space with the maximum value of the jth channel weight Execute step III.

[0129] In the present invention, the operations of the tensor-level weight maximum search space ST and the channel-level weight maximum search space SC in the search process are the same, and are collectively referred to as the weight maximum search space S[ST, SC].

[0130] Step III, calculating the quantization errors of all low-bit floating point formats and weight maximum value combinations;

[0131] Step III-1, the first loop traverses the low-bit floating-point configuration search space C;

[0132] The floating point format in the weight quantizer is set, that is, the floating point format in the weight quantizer is set to an element in a low-bit floating point configuration search space C, and the elements in the low-bit floating point configuration search space are traversed in order, and the identifier of the first layer loop is denoted by c.

[0133] Taking 8-bit floating-point numbers as an example, the search space is C = {E6M1, E5M2, E4M3, E3M4, E2M5, E1M6}. In the first round of loop, the floating-point number format in the weight quantizer is set to E6M1. The weight quantizer will quantize the weight according to the exponent bit as 6 bits and the median bit as 1 bit.

[0134] Step III-2, the second loop traverses the maximum weight search space S[ST,SC];

[0135] Set the maximum value in the weight quantizer, that is, set it to the search object in the maximum value search space S[ST,SC], traverse the elements in the maximum value search space in order, and the identifier of the second layer loop is denoted by s.

[0136] Taking tensor-level granularity as an example, the search space for the maximum weight of tensor-level granularity is In the first cycle, set the maximum value of the tensor in the weight quantizer to The weight tensor quantizer will constrain the weight range according to the set maximum value of the tensor.

[0137] Step III-3, determine whether there are calibration data sets CDS_Q and CDS_O, and perform quantitative calibration;

[0138] The calibration data set used by the quantized module is recorded as CDS_Q, and the calibration data set used by the original module is recorded as CDS_O.

[0139] The CDS_Q and CDS_O may be 32, 64 or 128 samples, the number of samples of the two may be the same, and the two may exist at the same time or not exist at the same time.

[0140] If there is no calibration data set CDS_Q and CDS_O, proceed to step III-4;

[0141] If there are calibration data sets CDS_Q and CDS_O, proceed to step III-5.

[0142] In PTQ, MSE (Mean Squared Error) is used to measure the difference between the model before and after quantization. The basic formula is: n represents the number of elements in the tensor, i represents the identification number of the element in the tensor, and X i represents the original value of the i-th element, Represents the predicted value of the i-th element.

[0143] The low-bit floating point quantization calculation method refers to arXiv preprint arXiv:2208.09225 published on August 19, 2022, FP8 Quantization:The Power of the Exponent, Section 4.1, 2022 NIPS. The present invention can simulate the result of the current tensor calculated after quantization by low-bit floating point according to the exponent bit and mantissa bit configuration of the low-bit floating point number and the set tensor maximum value.

[0144] Step III-4, quantify the error records in the scenario without calibration data;

[0145] In a scenario without calibration data, the optimization goal of quantization is to minimize the error between the tensor before and after quantization. The tensor quantization error under the current configuration is calculated and recorded, and then steps III-6 are executed.

[0146] If the quantization granularity is PT, use the tensor-level weighted mean square error formula To calculate the error size of the model tensor before and after quantization, ntt represents the number of elements in the tensor without calibration data, and W represents the weight tensor of the model before quantization. Represents the weight tensor of the quantized model. The tensor-level weight mean square error mse_pt_t under the current low-bit floating-point configuration and the tensor maximum configuration is calculated and recorded in the intermediate result PT_T_ERROR[c][s]. PT_T_ERROR is a two-dimensional matrix that records the tensor-level weight errors under each low-bit floating-point configuration and each tensor maximum configuration. c is the identifier of the first-layer loop, which corresponds to the current low-bit floating-point configuration, and s is the identifier of the second-layer loop, which corresponds to the current tensor maximum configuration.

[0147] When the quantization granularity is PC, the channel-level weighted mean square error formula is used To calculate the error size of the model tensor channel j before and after quantization, where nct j represents the number of elements in the output tensor channel j in the scenario without calibration data, W j represents the weight tensor of the model before quantization, Represents the weight tensor of the quantized model. Calculate the channel-level weight mean square error mse_pc_t under the current low-bit floating-point configuration and tensor maximum configuration, and record it in the intermediate result PC_T_ERROR[c][s][j]. PC_T_ERROR is a three-dimensional matrix that records each low-bit floating-point configuration, each tensor maximum configuration, and the channel-level weight error under each channel. j represents the channel of the tensor.

[0148] Step III-5, in the scenario with calibration data, quantify the error records;

[0149] In the scenario with calibration data, the optimization goal of quantization is to minimize the output error before and after quantization. The output error of the quantized tensor under the current configuration is calculated and recorded, and then step III-6 is executed.

[0150] If the quantization granularity is PT, use the tensor-level output mean square error formula To calculate the error size of the module output before and after quantization, nto represents the number of elements in the output tensor in the scenario with calibration data, CDS_O represents the calibration data set used by the original module, CDS_Q represents the calibration data set used by the module after quantization, and W represents the weight tensor value of the model before quantization. Indicates the weight tensor value of the quantized model, func indicates the function torch.nn.functional.linear corresponding to the Linear module in pytorch. Calculate the tensor-level output mean square error mse_pt_o under the current low-bit floating point configuration and the tensor maximum configuration, and record it in the intermediate result PT_O_ERROR[c][s]. PT_O_ERROR is a two-dimensional matrix that records the tensor-level output errors under each low-bit floating point configuration and each tensor maximum configuration.

[0151] If the quantization granularity is PC, use the tensor-level output mean square error formula To calculate the error size of the module output before and after quantization, where nco j Indicates the number of elements in the output tensor channel j, CDS_O j Indicates the data corresponding to channel j of the calibration data set used by the original module, CDS_Q j Represents the data corresponding to channel j in the calibration data set used by the quantized module, W j represents the weight tensor of the model before quantization, Represents the weight tensor of the quantized model, and func represents the function torch.nn.functional.linear corresponding to the Linear module in pytorch. Calculate the channel-level output mean square error mse_pc_o under the current low-bit floating point configuration and tensor maximum configuration, and record it in the intermediate result PC_O_ERROR[c][s][j]. PC_O_ERROR is a three-dimensional matrix that records each low-bit floating point configuration, each tensor maximum configuration, and the channel-level output error under each channel.

[0152] Step III-6, determine whether the maximum value search space has been traversed;

[0153] If the elements in the S[ST,SC] space are traversed, execute step III-7;

[0154] If the elements in the S[ST,SC] space have not been traversed, execute step III-2.

[0155] Step III-7, determining whether the low-bit floating point number configuration search space has been traversed;

[0156] If the elements in space C are traversed, execute step IV;

[0157] If the elements in space C have not been traversed, execute step III-1.

[0158] In the present invention, step III-6 and step III-7 are for finding the best maximum value and low-bit floating point format configuration from the S[ST, SC] space and the C space, so as to minimize the error between the current quantized tensor and the original tensor.

[0159] Step IV, obtaining the best low-bit floating point format and weight maximum value configuration;

[0160] Step IV-1, determine whether the quantization granularity of the weight quantizer is at the tensor level;

[0161] If the quantization granularity of the weight quantizer is PT, execute step IV-2;

[0162] If the quantization granularity of the weight quantizer is PC, execute step IV-3.

[0163] Step IV-2, update the configuration in the weight quantizer according to the tensor level;

[0164] In the present invention, the operations on the tensor level weight mean square error PT_T_ERROR[c][s] and the tensor level output mean square error PT_O_ERROR[c][s] in value taking are the same, and are collectively referred to as the tensor level mean square error PT_ERROR[c][s].

[0165] Find the minimum tensor-level mean square error PT_ERROR[c min ][s min ], c min is the low-bit floating-point search space subscript corresponding to the minimum tensor-level mean square error, s min It is the search space subscript of the maximum weight corresponding to the minimum tensor-level mean square error.

[0166] Update the corresponding low-bit floating-point format LFF in the weight quantizer to the corresponding low-bit floating-point search space C min The element corresponding to the subscript, that is, LFF = C[c min ]; Update the corresponding maximum weight value W_MAX in the weight quantizer to the corresponding maximum weight value in the search space ST s min The element corresponding to the subscript, that is, W_MAX = ST[s min ], execute step IV-4;

[0167] Step IV-3, updating the configuration in the weight quantizer according to the channel level;

[0168] In the present invention, the operations on the channel-level weighted mean square error PC_T_ERROR[c][s][j] and the channel-level output mean square error PC_O_ERROR[c][s][j] in value selection are the same, collectively referred to as the channel-level mean square error PC_ERROR[c][s][j].

[0169] Find the minimum channel-level mean square error in the three-dimensional matrix corresponding to each channel channel by channel is the low-bit floating-point search space subscript corresponding to the minimum mean square error of channel j, is the search space subscript of the maximum weight corresponding to the minimum mean square error of channel j.

[0170] Select The c with the highest number of occurrences min As the corresponding low-bit floating-point configuration subscript in the weight quantizer, the corresponding low-bit floating-point format LFF in the weight quantizer is updated to the corresponding low-bit floating-point search space C in c min The element corresponding to the subscript, that is, LFF = C[c min ].

[0171] After determining the elements of the low-bit floating-point search space in the weight quantizer, find the minimum channel-level mean square error in the three-dimensional matrix corresponding to each channel channel by channel For min The search space subscript of the maximum weight corresponding to the minimum mean square error in channel j, and the corresponding maximum weight W_MAX in the weight quantizer is calculated channel by channel j , updated to the corresponding search space SC with the maximum weight j middle The element corresponding to the subscript, that is Execute step IV-4.

[0172] Step IV-4, determine whether there are calibration data sets CDS_Q and CDS_O, and if so, calculate the input of the next network layer;

[0173] The calibration data set used by the quantized module is recorded as CDS_Q, and the calibration data set used by the original module is recorded as CDS_O.

[0174] If there are no calibration data sets CDS_Q and CDS_O, the execution is completed;

[0175] If there are calibration data sets CDS_Q and CDS_O, use the weight quantizer after updating the configuration to calculate the quantized weights According to the input calibration data set CDS_Q, the output of the quantization module is recorded as CDS_QL_O. Assign a value to CDS_QL_O, that is According to the input calibration data set CDS_O, the original module output is recorded as CDS_OL_O, and func(CDS_O,W) is assigned to CDS_OL_O, that is, CDS_OL_O = func(CDS_O,W), where func represents the function torch.nn.functional.linear corresponding to the Linear module in pytorch. CDS_QL_O and CDS_OL_O are used as the input of the next layer of the network.

[0176] See also Figure 3 As shown, in the present invention, a model compression method based on low-bit floating point numbers and quantization after model training is performed, wherein the tuning between model layers performs the following steps:

[0177] Step 1: quantize and initialize the model;

[0178] Step 1-1, model loading;

[0179] Load the model based on pytorch framework.

[0180] The network layer of the pytorch architecture model is represented as a network layer set OM in set form, and OM = {om1, om2, ..., om a-1 ,om a ,om a+1 ,…,om b ,om b+1 ,…,om A-1 ,om A}, the subscript a represents the identification number of the network layer, and the subscript A represents the total number of network layer layers.

[0181] om1 represents the first network layer of the pytorch architecture model. The model weight of the network layer om1 is recorded as W1.

[0182] om2 represents the second network layer of the pytorch architecture model. The model weight of the network layer om2 is recorded as W2.

[0183] om a Represents the ath network layer of the pytorch architecture model. Network layer om a The model weight is recorded as W a .

[0184] om A Represents the last network layer of the pytorch architecture model. Network layer om A The model weight is denoted as W A .

[0185] For the convenience of explanation, the om aAlso known as the current network layer of the pytorch architecture model. Located in the network layer om a The previous one is recorded as the previous network layer om a-1 , located at the network layer a The following is recorded as the next network layer om a+1 .

[0186] In the present invention, OM={om1,om2,…,om a-1 ,om a ,om a+1 ,…,om b ,om b+1 ,…,om A-1 ,om A The network layers in} are arranged sequentially. The quantization of the network layers is also completed sequentially.

[0187] Step 1-2, the user sets the PTQ parameters;

[0188] The PTQ parameters set by the user include: quantization calibration data set CDS, quantization granularity MP[PT, PC], and low-bit floating point bit width BW[4,8].

[0189] The calibration data set CDS is a calibration data set used by the module, which may be 32, 64 or 128 samples. The CDS is initially an empty set and is also empty when the user does not set calibration data.

[0190] In the quantization granularity MP[PT, PC], PT represents tensor-level quantization, and PC represents channel-level quantization. The MP[PT, PC] is a choice between the two.

[0191] The low-bit floating-point number bit width BW[4,8] is selected from two options, one is a 4-digit number and the other is an 8-digit number.

[0192] In the present invention, the quantization operation of the quantization granularity MP[PT, PC] refers to arXiv preprint arXiv:2106.08295 published on June 15, 2021, Section 2.2.3 Quantization granularity of “A White Paper on Neural Network Quantization”, author Markus Nagel et al.

[0193] Steps 1-3, initialize the loading model;

[0194] Use the PTQ parameters set in step 1-2 to initialize the pytorch architecture model in step 1-1.

[0195] Steps 1-4, get the iterator of the model;

[0196] Through the named_children() interface of the pytorch framework model, the iterator of the model is obtained.

[0197] The model iterator is used to perform OM={om1, om2, ..., om a-1 ,om a ,om a+1 ,…,om b ,om b+1 ,…,om A-1 ,om A} for access.

[0198] Steps 1-5, determine whether the current network layer needs to be quantized;

[0199] The quantitative condition is recorded as Conv2d is a two-dimensional convolutional network module, and Linear is a linear network module.

[0200] If the current network layer om a It belongs to the Conv2d network module, and replaces the Conv2d network layer with the QuantizedConv2d network quantization layer. The QuantizedConv2d network quantization layer is a quantization module that adds a weight quantizer to the Conv2d network layer. The quantization module includes the quantization granularity MP[PT,PC], the bit width BW[4,8] of the low-bit floating point number, the maximum value of the tensor and low-bit floating point formats

[0201] If the current network layer om a It belongs to the Linear network module, and replaces the Linear network layer with the QuantizedLinear network quantization layer. The QuantizedLinear network quantization layer is a quantization module that adds a weight quantizer to the Linear network layer. The quantization module includes the quantization granularity MP[PT,PC], the bit width BW[4,8] of the low-bit floating point number, the maximum value of the tensor and low-bit floating point formats

[0202] For OM={om1,om2,…,om a-1 ,om a ,om a+1 ,…,om b ,om b+1 ,…,om A-1 ,om ARepeat steps 1-5 for all network layers in} to obtain the set of network layers that need to be quantized QM = {qm1,…,qm a ,qm a+1 ,…,qm A-1}.

[0203] qm1 represents the replacement of the network layer om1 by the network quantization layer. The qm1 retains the model weight W1 of the network layer om1.

[0204] qm a Indicates the network layer om a The quantization layer of the network is replaced. a The network layer is retained a The model weight W a .

[0205] qm a+1 Indicates the network layer om a+1 The quantization layer of the network is replaced. a+1 The network layer is retained a+1 The model weight W a+1 .

[0206] qm A-1 Indicates the network layer om A-1 The quantization layer of the network is replaced. A-1 The network layer is retained A-1 The model weight W A-1 .

[0207] In the present invention, the constructed QuantizedConv2d network quantization layer and QuantizedLinear network quantization layer are applied to perform quantization within the model layer, that is, based on the quantization module parameter settings and the maximum value settings, a search is performed to search for the format of the exponent and mantissa bits of low-bit floating-point numbers and the optimal tensor maximum value.

[0208] Steps 1-6, retain the original network layer for the network layer that does not need to be quantized;

[0209] From OM={om1,om2,…,om a-1 ,om a ,om a+1 ,…,om b ,om b+1 ,…,om A-1 ,om A} exclude QM={qm1,…,qm a ,qm a+1 ,…,qm A-1}, is recorded as the network layer set that does not need to be quantized NM = {nm2,…,nm a-1,…,nm b ,nm b+1 ,…,nm A}.

[0210] nm2 indicates that the network layer om2 is reserved.

[0211] nm a-1 Indicates that the network layer is retained a-1 .

[0212] nm b Indicates that the network layer is retained b .

[0213] nm b+1 Indicates that the network layer is retained b+1 .

[0214] nm A Indicates that the network layer is retained A .

[0215] In the present invention, the computer language sets the current network layer om a For non The original network layer is retained, that is, NM is formed.

[0216] In the present invention, the network layer set MM loaded by the model is the union of QM and NM.

[0217] For the convenience of explanation, the network layer set OM = {om1, om2, ..., om a-1 ,om a ,om a+1 ,…,om b ,om b+1 ,…,om A-1 ,om A After steps 1-5 and 1-6, the obtained network layer set after orderly arrangement and replacement is recorded as MM, and MM = {qm1,nm2,…,nm a-1 ,qm a ,qm a+1 ,…,nm b ,nm b+1 ,…,qm A-1 ,nm A}.

[0218] Step 2: quantize the network layer by layer;

[0219] From MM = {qm1,nm2,…,nm a-1 ,qm a ,qm a+1 ,…,nm b ,nm b+1 ,…,qmA-1 ,nm A The first network layer selected from} is recorded as MM 首 For example, MM 首 =[qm1], then execute step 2-1.

[0220] Following MM 首 The following is recorded as MM 次 For example, MM 次 ={nm2,…,nm a-1 ,qm a ,qm a+1 ,…,nm b ,nm b+1 ,…,qm A-1 ,nm A According to the PTQ setting and quantization conditions, the network layer to be quantized is recorded as The network layer that does not need to be quantized is denoted as

[0221] In the present invention, if MM 首 The elements are NM={nm2,…,nm a-1 ,…,nm b ,nm b+1 ,…,nm A}, namely MM 首 =[nm2], then execute step 2-2. That is, from MM = {nm2, qm1, ..., nm a-1 ,qm a ,qm a+1 ,…,nm b ,nm b+1 ,…,qm A-1 ,nm A The first network layer selected from} is recorded as MM 首 , namely MM 首 =[nm2].

[0222] Step 2-1, determine whether there is a calibration data set for the first network layer that needs to be quantized and quantize it;

[0223] The first network layer MM 首 =[qm1].

[0224] If there is a calibration data set in qm1 And belongs to QuantizedConv2d, Assign value to quantization module input Will Assign a copy of the original module input Will and As the input of qm1, qm1 is processed by the QuantizedConv2d network quantization layer, and the output is and Will Assign value to the next layer of quantization module input Will Assign value to the next layer of original module input

[0225] If there is no calibration data set in qm1 And it belongs to QuantizedConv2d, and directly performs QuantizedConv2d network quantization layer calculation on qm1.

[0226] If there is a calibration data set in qm1 And belongs to QuantizedLinear, Assign value to quantization module input Will Assign a copy of the original module input Will and As the input of qm1, qm1 is processed by the QuantizedLinear network quantization layer and the output is and Will Assign value to the next layer of quantization module input Will Assign value to the next layer of original module input

[0227] If there is no calibration data set in qm1 And it belongs to QuantizedLinear, and directly performs QuantizedLinear network quantization layer calculation processing on qm1.

[0228] After completing step 2-1, the first network layer GMM that has completed quantization is obtained 首 , and GMM 首 =[gqm1].

[0229] Step 2-2, for the first network layer that does not need to be quantized, determine whether there is a calibration data set and quantize it;

[0230] The first network layer MM 首 =[nm2].

[0231] If there is a calibration dataset in nm2 Will Assign value to quantization module input Will Assign a copy of the original module input Will and As input to nm2, output and Will Assign value to the next layer of quantization module input Will Assign value to the next layer of original module input

[0232]

[0233] If there is no calibration data set in nm2 No additional processing is performed.

[0234] After completing step 2-2, the first network layer GMM that does not require quantization is obtained 首 , and GMM 首 =[nm2], no need to quantize nm2.

[0235] Step 2-3: for any network layer that needs to be quantized after the first network layer, determine whether there is a calibration data set and perform calculation processing;

[0236] from Select qm a ,qm a It is the network layer that needs to be quantized.

[0237] If qm a There is a calibration dataset and And belongs to QuantizedConv2d, and As qm a The input of qm a Perform QuantizedConv2d network quantization layer calculation and output and Will Assign value to the next layer of quantization module input Will Assign value to the next layer of original module input

[0238] If qm a No calibration dataset and And belongs to QuantizedConv2d, directly to qm a Perform QuantizedConv2d network quantization layer calculation processing.

[0239] If qm a There is a calibration dataset and And belongs to QuantizedLinear, and As qm a The input of qm a Perform QuantizedLinear network quantization layer calculation and output and Will Assign value to the next layer of quantization module input Will Assign value to the next layer of original module input

[0240] If qm a No calibration dataset and And it belongs to QuantizedLinear, directly to qm a Perform QuantizedLinear network quantization layer calculation processing.

[0241] According to MM 次 ={nm2,…,nm a-1 ,qm a ,qm a+1 ,…,nm b ,nm b+1 ,…,qm A-1 ,nm A}, and the network layers that need to be quantized are sorted one by one. For the remaining network layers, execute steps 2-3.

[0242] After completing steps 2-3, the non-first network layer after quantization is obtained

[0243]

[0244] Steps 2-4: For any network layer after the first network layer that does not require quantization, determine whether there is a calibration data set and perform calculation processing;

[0245] from Select nm b , nm b It is a network layer that does not need to be quantized.

[0246] If nm b There is a calibration dataset and Will and As nm b Input, output and Will Assign value to the next layer of quantization module input Will Assign value to the next layer of original module input

[0247] If nm b No calibration dataset and No additional processing is performed.

[0248] According to MM 次 ={nm2,…,nm a-1 ,qm a ,qm a+1 ,…,nm b ,nm b+1 ,…,qm A-1 ,nm A}, and the network layers that do not need to be quantized are sorted one by one. For the remaining network layers in , execute steps 2-4.

[0249] After completing steps 2-4, the non-first network layer that does not need to be quantized is obtained

[0250]

[0251] Steps 2-5, complete model quantization and save quantization configuration;

[0252] GMM 首 =[gqm1] and All the quantized network modules are configured and saved.

[0253] If the current network layer gqm a Belongs to the QuantizedConv2d network module. The QuantizedConv2d network quantization layer needs to save the quantization granularity MP[PT,PC], the bit width BW[4,8] of the low-bit floating point number, and the maximum value of the tensor and low-bit floating point formats

[0254] If the current network layer gqm a Belongs to the QuantizedLinear network module. The QuantizedLinear network quantization layer needs to save the quantization granularity MP[PT,PC], the bit width BW[4,8] of the low-bit floating point number, and the maximum value of the tensor and low-bit floating point formats

[0255] Step 3: Export the weights of the quantized model by weight compression;

[0256] Step 3-1, converting the weight of the quantization module;

[0257] GMM 首 =[gqm1] and All network modules that have completed quantization undergo weight conversion.

[0258] For low-bit floating-point quantization calculation methods, refer to arXiv preprint arXiv:2208.09225 published on August 19, 2022, FP8 Quantization: The Power of the Exponent, Section 4.1, 2022 NIPS.

[0259] The present invention can simulate the result of calculating the current tensor after quantization by a low-bit floating point number and the corresponding scaling value according to the exponent bit and mantissa bit configuration of the low-bit floating point number and the set tensor maximum value, that is, use a high-bit floating point number to represent the current low-bit floating point number.

[0260] The present invention needs to convert the low-bit floating point number represented by the high-bit floating point number into a corresponding representation and perform bit compression.

[0261] If the current network layer gqm a Belongs to the QuantizedConv2d network module, using the maximum value of the tensor in the quantized layer of the QuantizedConv2d network and low-bit floating point formats To simulate the calculation to get the weight The result after low-bit floating point quantization and scaling value Assume that the current weight floating point format is float32 (E8M23), and the low-bit floating point format to be converted is float8 (E4M3). For tensor Any element w_qc in can be represented in float32(E8M23) as Where s represents the sign bit, and p represents the e -1], e represents the exponent value, the offset is 127, d m Indicates the value of the mth bit of the mantissa, which can be 0 or 1. m indicates the number of bits in the mantissa. The corresponding float8(E4M3) is represented as The offset is 7, and the sign bit of w_qc8 here is w_qc 32 The sign bit of w_qc8 is w_qc 32 The exponent value of -127+7 is the last 4 bits, and the mantissa of w_qc8 is w_qc 32The first 3 bits of the digit. The high bit represents the weight The weight after low bit conversion is recorded as Any element in w_lc is an 8-bit floating point number.

[0262] If the current network layer gqm a Belongs to the QuantizedLinear network module, using the maximum value of the tensor in the QuantizedLinear network quantization layer and low-bit floating point formats To simulate the calculation to get the weight The result after low-bit floating point quantization and scaling value Assume that the current weight floating point format is float32 (E8M23), and the low-bit floating point format to be converted is float8 (E4M3). For tensor Any element w_ql in can be represented in float32(E8M23) as Where s represents the sign bit, and p represents the e -1], e represents the exponent value, the offset is 127, d m Indicates the value of the mth bit of the mantissa, which can be 0 or 1. m indicates the number of bits in the mantissa. The corresponding float8(E4M3) is represented as The offset is 7, and the sign bit of w_ql8 here is w_ql 32 The sign bit of w_ql8 is w_ql 32 The exponent value of -127+7 is the last 4 bits, and the mantissa of w_ql8 is w_ql 32 The first 3 bits of the digit. The high bit represents the weight The weight after low bit conversion is recorded as Any element w_ll in is an 8-bit floating point number.

[0263] The network module that completes the weight conversion after step 3-1 is recorded as GLMM 首 =[glqm1] and

[0264]

[0265] Step 3-2, extracting low-bit floating point numbers for compression;

[0266] For GLMM 首 =[glqm1] and Compressed storage.

[0267] If the current network layer is glqm a Belongs to the QuantizedConv2d network module, and its converted low-bit floating-point number is represented as There are n elements in the weight tensor. Assuming that the current weight floating point format is float8 (E4M3), the weight is stored in a 32-bit tensor. Each 32-bit bit can store four 8-bit weight values. The compressed tensor size is 32 bits, and fill the low-bit floating-point tensor into the compressed tensor in order (such as Figure 4 ), and the compressed tensor is recorded as

[0268]

[0269] If the current network layer is glqm a Belongs to the QuantizedLinear network module, and its converted low-bit floating-point number is represented as There are n elements in the weight tensor. Assuming that the current weight floating point format is float8 (E4M3), the weight is stored in a 32-bit tensor. Each 32-bit bit can store four 8-bit weight values. The compressed tensor size is 32 bits, and fill the low-bit floating-point tensor into the compressed tensor in order (such as Figure 4 ), and the compressed tensor is recorded as

[0270]

[0271] The network module that completes weight compression after step 3-2 is recorded as GLFMM 首 =[glfqm1] and

[0272]

[0273] Step 3-3, replace the weights and export the model;

[0274] GLFMM 首 =[glfqm1] and Export the model after weight replacement; Export the model.

[0275] If the current network layer is glfqm a Belongs to the QuantizedConv2d network module, using the compressed weight tensor and scaling value Replace the original weight tensor Export the network layer. With the is the same value. With the is the same value. With the are the same value.

[0276] If the current network layer is glfqm a Belongs to the QuantizedLinear network module, using compressed weight tensors and scaling value Replace the original weight tensor Export the network layer. With the is the same value. With the is the same value. With the are the same value.

[0277] If the current network layer is nm b , and export the network layer.

[0278] At this point, model compression of low-bit floating-point numbers and quantization after model training has been completed.

[0279] Example 1

[0280] The method of the present invention is tested on a visual model and a language model based on the pytorch architecture.

[0281] (I) PTQ experimental results of visual models in the ImageNet dataset

[0282] In terms of visual model testing, the imagenet dataset of the visual classification task was used for simulation. The simulation results represent the accuracy of the model in the classification task, and common visual classification models were selected for testing. In the test data table, Origin represents the classification accuracy of the uncompressed model, INT4 represents the classification accuracy after the model is compressed using the traditional integer PTQ method, and FP4 represents the classification accuracy after the model is compressed using the low-bit floating point PTQ method of the present invention. The quantization granularity is PC. The closer the classification accuracy is to the uncompressed model, the better the compression effect.

[0283] Table 1 Visual model on 4-bit PTQ without data calibration

[0284] Model Origin INT4 FP4 MobileNetV2 72.15 2.47 15.17 MobileNetV3 Small 67.67 5.66 5.48 MobileNetV3 Large 75.27 6.53 8.71 VGG19 72.38 66.79 70.33 EfficientNet B0 77.69 7.34 15.05 EfficientNet B7 84.12 81.46 82.65 ResNet18 69.76 53.75 60.52 ResNet50 80.86 19.54 45.81 ViT 80.32 79.54 80.04

[0285] The MobileNetV2 model has 3.4 million parameters and introduces deep separable convolution and inverted residual results, which effectively reduce the computational complexity. At the same time, it enhances the feature transfer capability through linear bottleneck design, making it suitable for efficient image processing tasks in resource-constrained environments.

[0286] The MobileNetV3 Small model has 2.5 million parameters. It combines automated neural structure search and hardware-friendly design, optimizes the depthwise separable convolution and attention modules, and further reduces the model size to meet performance requirements in extremely low-resource scenarios.

[0287] The MobileNetV3 Large model has 5.5 million parameters. Based on the automatic search and optimization of the network architecture, it adds an attention mechanism and an improved activation function, which significantly improves the ability to balance accuracy and efficiency in high-performance tasks.

[0288] The VGG 19 model has 143.7 million parameters and uses a simple but deep convolutional structure. It builds a feature hierarchy through continuous 3×3 convolution and maximum pooling operations, and improves the performance of large-scale image classification with deep feature representation capabilities.

[0289] The EfficientNet B0 model has 5.3 million parameters and uses a multi-dimensional optimization method based on compound scaling to find the optimal balance between depth, width, and resolution. It improves parameter utilization and inference speed through a carefully designed inverted residual module.

[0290] The EfficientNet B7 model has 66.3 million parameters. By extending the compound scaling factor of the EfficientNet series, the network structure is further deepened and widened, providing high-precision classification while significantly increasing the demand for hardware resources.

[0291] The ResNet 18 model has 11.7 million parameters. By introducing a residual module with skip connections, it solves the bottleneck of gradient vanishing and degradation problems in deep neural networks and achieves efficient feature learning capabilities with a shallower structure.

[0292] The ResNet50 model has 25.6 million parameters and adopts a deep residual network design. It significantly increases the model depth by stacking multiple bottleneck residual blocks, while using batch normalization and ReLU activation layers to enhance the learning ability of complex features.

[0293] The ViT model has 86.6 million parameters and is based on the Transformer architecture. It captures global dependencies by dividing images into fixed-size tiles and introducing a self-attention mechanism, while improving the overall expressiveness of visual tasks through large-scale pre-training.

[0294] The data in Table 1 are for the case where no dataset is used for calibration. After PTQ model compression, the models MobileNetV2, MobileNetV3Small, MobileNetV3 Large, EfficientNet B0, and ResNet50 all show serious model performance degradation, which is far behind the performance of the Origin uncompressed models. Neither INT4 nor FP4 are suitable for PTQ model compression without dataset calibration.

[0295] It can be seen from Table 1 that in the VGG 19 model, the FP4 PTQ model compression method of the present invention is 5.30% higher than the traditional INT4 PTQ model compression method; in the EfficientNet B7 model, the FP4 PTQ model compression method of the present invention is 1.46% higher than the traditional INT4 PTQ model compression method; in the ResNet 18 model, the FP4 PTQ model compression method of the present invention is 12.60% higher than the traditional INT4 PTQ model compression method; in the ViT model, the FP4 PTQ model compression method of the present invention is 0.63% higher than the traditional INT4 PTQ model compression method.

[0296] Table 2 Visual model on 4-bit PTQ with data calibration

[0297] Model Origin INT4 FP4 MobileNetV2 72.15 63.36 69.51 MobileNetV3 Small 67.67 49.05 61.32 MobileNetV3 Large 75.27 65.67 71.10 VGG19 72.38 71.19 70.61 EfficientNet B0 77.69 71.68 75.50 EfficientNet B7 84.12 83.47 83.83 ResNet18 69.76 64.81 68.15 ResNet50 80.86 66.78 68.78 ViT 80.32 79.51 79.72

[0298] Table 2 shows the compression of the PTQ model using the dataset for calibration. Except for the VGG 19 network, the compression effect of the FP4PTQ model compression method of the present invention on the visual classification task is better than that of the traditional INT4 PTQ model compression method. At the same time, after the dataset calibration, the compression effect of all models can be close to that of the uncompressed Origin model.

[0299] It can be seen from Table 2 that in the MobileNetV2 model, the FP4 PTQ model compression method of the present invention is improved by 9.71% compared with the traditional INT4 PTQ model compression method; in the MobileNetV3Small model, the FP4PTQ model compression method of the present invention is improved by 25.02% compared with the traditional INT4 PTQ model compression method; in the MobileNetV3Large model, the FP4 PTQ model compression method of the present invention is improved by 8.27% compared with the traditional INT4 PTQ model compression method; in the EfficientNet B0 model, the FP4 PTQ model compression method of the present invention is improved by 5.33% compared with the traditional INT4PTQ model compression method; in the EfficientNet B7 model, the FP4 PTQ model compression method of the present invention is improved by 0.43% compared with the traditional INT4 PTQ model compression method; in the ResNet 18 model, the FP4PTQ model compression method of the present invention is improved by 5.15% compared with the traditional INT4 PTQ model compression method; in the ResNet50 model, the FP4 The PTQ model compression method is improved by 2.99% compared with the traditional INT4 PTQ model compression method; in the ViT model, the FP4PTQ model compression method of the present invention is improved by 0.26% compared with the traditional INT4 PTQ model compression method.

[0300] (II) PTQ Experimental Results of the Language Model in the Wikitext2 Dataset

[0301] In terms of language model testing, the WikiText-2 data set of the language modeling task was used for simulation. The simulation results represent the perplexity (PPL) of the model in the generation task. The current mainstream LLaMA2-7B model was selected for testing. PPL is an important indicator for measuring the quality of language model generation. The lower its value, the more accurate the model's prediction of the test data. In the test data table, Origin represents the perplexity of the uncompressed model, INT4 represents the perplexity after the model is compressed using the traditional integer PTQ method, and FP4 represents the perplexity after the model is compressed using the low-bit floating point PTQ method used in the present invention. All quantized granularities are PC. The closer the perplexity is to the uncompressed model, the better the compression effect.

[0302] Table 3 Language model on 4-bit PTQ without data calibration

[0303] Model Origin INT4 FP4 LLaMA2-7B 5.472456 28.906588 7.397842

[0304] The LLaMA2 model has 7 billion parameters and adopts an optimized Transformer architecture. It significantly improves language understanding and generation capabilities by introducing a more efficient attention mechanism and improved training strategies. It also has strong scalability and is suitable for large-scale model application scenarios for a variety of natural language processing tasks.

[0305] It can be seen from Table 3 that in the LLaMA2-7B model, the FP4 PTQ model compression method of the present invention is improved by 74.41% compared with the traditional INT4 PTQ model compression method.

[0306] Table 4 Language model on 4-bit PTQ with data calibration

[0307] Model Origin INT4 FP4 LLaMA2-7B 5.472456 6.04249 5.871451

[0308] It can be seen from Table 4 that in the LLaMA2-7B model, the FP4 PTQ model compression method of the present invention is improved by 2.83% compared with the traditional INT4 PTQ model compression method, reaching 93.2% of the effect of the original model.

[0309] (III) Model compression ratio

[0310] Table 5 Model compression effect

[0311]

[0312] illustrate:

[0313] 1.M indicates the quantity is in million.

[0314] 2.*For most models, FP32 means using 32-bit floating point numbers to represent the model.

[0315] 3.**For LLaMA2-7B, the model uses FP 16, that is, 16-bit floating point numbers are used to represent the model.

[0316] It can be seen from Table 5 that after the model is compressed by the PTQ model, the model space represented by 32-bit floating point numbers is compressed to about 1 / 8 of the original size. MobileNetV2, MobileNetV3 Small, MobileNetV3Large, VGG 19, EfficientNet B0, EfficientNet B7, ResNet 18, ResNet50 and ViT are compressed to 13.91%, 13.57%, 13.36%, 12.52%, 14.00%, 13.37%, 12.64%, 12.79% and 12.89% of the original models respectively; the model space represented by 16-bit floating point numbers is compressed to about 1 / 4 of the original size, and LLaMA2-7B is compressed to 26.50% of the original model.

Claims

1. A model compression method based on low-bit floating point numbers and quantization after model training, characterized in that The following steps are performed to fine-tune the model layers: Step 1: quantize and initialize the model; Step 1-1, model loading; Load the model based on pytorch framework; The network layer of the pytorch architecture model is represented as a network layer set OM in set form, and OM = {om1, om2, ..., om a-1 ,om a ,om a+1 ,…,om b ,om b+1 ,…,om A-1 ,om A }, the subscript a represents the identification number of the network layer, and the subscript A represents the total number of network layers; om1 represents the first network layer of the pytorch architecture model; the model weight of the network layer om1 is recorded as W1; om2 represents the second network layer of the pytorch architecture model; the model weight of the network layer om2 is recorded as W2; om a Represents the ath network layer of the pytorch architecture model; network layer om a The model weight is denoted as W a ; om A Represents the last network layer of the pytorch architecture model; network layer om A The model weight is denoted as W A ; Step 1-2, the user sets the PTQ parameters; The PTQ parameters set by the user include: quantization calibration data set CDS, quantization granularity MP[PT, PC], and low-bit floating point bit width BW[4,8]; The calibration data set CDS is a calibration data set used by the module, which is 32, 64 or 128 samples; the CDS is initially an empty set and is also empty when the user does not set calibration data; In the quantization granularity MP[PT, PC], PT represents tensor-level quantization, and PC represents channel-level quantization; MP[PT, PC] is a choice between the two; The low-bit floating-point number bit width BW[4,8] is either a 4-digit number or an 8-digit number. Steps 1-3, initialize the loading model; Use the PTQ parameters set in step 1-2 to initialize the pytorch architecture model in step 1-1; Steps 1-4, get the iterator of the model; Get the model iterator through the named_children() interface of the pytorch framework model; The model iterator is used to perform OM={om1, om2, ..., om a-1 ,om a ,om a+1 ,…,om b ,om b+1 ,…,om A-1 ,om A } to access; Steps 1-5, determine whether the current network layer needs to be quantized; The quantitative condition is recorded as Conv2d is a two-dimensional convolutional network module, and Linear is a linear network module; If the current network layer om a It belongs to the Conv2d network module, and replaces the Conv2d network layer with the QuantizedConv2d network quantization layer; the QuantizedConv2d network quantization layer is a quantization module that adds a weight quantizer to the Conv2d network layer; the quantization module includes the quantization granularity MP[PT,PC], the bit width BW[4,8] of the low-bit floating point number, and the maximum value of the tensor and low-bit floating point formats If the current network layer om a It belongs to the Linear network module, and replaces the Linear network layer with the QuantizedLinear network quantization layer; the QuantizedLinear network quantization layer is a quantization module that adds a weight quantizer to the Linear network layer; the quantization module includes the quantization granularity MP[PT,PC], the bit width BW[4,8] of the low-bit floating point number, and the maximum value of the tensor and low-bit floating point formats For OM={om1,om2,…,om a-1 ,om a ,om a+1 ,…,om b ,om b+1 ,…,om A-1 ,om A Repeat steps 1-5 for all network layers in} to obtain the set of network layers that need to be quantized QM = {qm1,…,qm a ,qm a+1 ,…,qm A-1 }; qm1 represents the replacement of the network layer om1 by the network quantization layer; the qm1 retains the model weight W1 of the network layer om1; qm a Indicates the network layer om a Replacement of the network quantization layer; the qm a The network layer is retained a The model weight W a ; qm a+1 Indicates the network layer om a+1 Replacement of the network quantization layer; the qm a+1 The network layer is retained a+1 The model weight W a+1 ; qm A-1 Indicates the network layer om A-1 Replacement of the network quantization layer; the qm A-1 The network layer is retained A-1 The model weight W A-1 ; Steps 1-6, retain the original network layer for the network layer that does not need to be quantized; From OM={om1,om2,…,om a-1 ,om a ,om a+1 ,…,om b ,om b+1 ,…,om A-1 ,om A } exclude QM={qm1,…,qm a ,qm a+1 ,…,qm A-1 }, is recorded as the network layer set that does not need to be quantized NM = {nm2,…,nm a-1 ,…,nm b ,nm b+1 ,…,nm A }; nm2 means retaining the network layer om2; nm a-1 Indicates that the network layer is retained a-1 ; nm b Indicates that the network layer is retained b ; nm b+1 Indicates that the network layer is retained b+1 ; nm A Indicates that the network layer is retained A ; Computer language settings current network layer om a For non The original network layer is retained, that is, NM is formed; The network layer set MM loaded by the model is the union of QM and NM; Network layer set OM = {om1, om2, ..., om a-1 ,om a ,om a+1 ,…,om b ,om b+1 ,…,om A-1 ,om A After steps 1-5 and 1-6, the obtained network layer set after orderly arrangement and replacement is recorded as MM, and MM = {qm1,nm2,…,nm a-1 ,qm a ,qm a+1 ,…,nm b ,nm b+1 ,…,qm A-1 ,nm A }; Step 2: quantize the network layer by layer; From MM = {qm1,nm2,…,nm a-1 ,qm a ,qm a+1 ,…,nm b ,nm b+1 ,…,qm A-1 ,nm A The first network layer selected from} is recorded as MM 首 ; For example, MM 首 =[qm1], then execute step 2-1; Following MM 首 The following is recorded as MM 次 ; For example, MM 次 ={nm2,…,nm a-1 ,qm a ,qm a+1 ,…,nm b ,nm b+1 ,…,qm A-1 ,nm A }; According to the PTQ setting and quantization conditions, the network layer to be quantized is recorded as The network layer that does not need to be quantized is denoted as If MM 首 The elements are NM={nm2,…,nm a-1 ,…,nm b ,nm b+1 ,…,nm A }, namely MM 首 =[nm2], then execute step 2-2; that is, from MM = {nm2, qm1, …, nm a-1 ,qm a ,qm a+1 ,…,nm b ,nm b+1 ,…,qm A-1 ,nm A The first network layer selected from} is recorded as MM 首 , namely MM 首 =[nm2]; Step 2-1, determine whether there is a calibration data set for the first network layer that needs to be quantized and quantize it; The first network layer MM 首 =[qm1]; If there is a calibration data set in qm1 And belongs to QuantizedConv2d, Assign value to quantization module input Will Assign a copy of the original module input Will and As the input of qm1, qm1 is processed by the QuantizedConv2d network quantization layer, and the output is and Will Assign value to the next layer of quantization module input Will Assign value to the next layer of original module input If there is no calibration data set in qm1 And it belongs to QuantizedConv2d, directly perform QuantizedConv2d network quantization layer calculation on qm1; If there is a calibration data set in qm1 And belongs to QuantizedLinear, Assign value to quantization module input Will Assign a copy of the original module input Will and As the input of qm1, qm1 is processed by the QuantizedLinear network quantization layer and the output is and Will Assign value to the next layer of quantization module input Will Assign value to the next layer of original module input If there is no calibration data set in qm1 And it belongs to QuantizedLinear, directly perform QuantizedLinear network quantization layer calculation processing on qm1; After completing step 2-1, the first network layer GMM that has completed quantization is obtained 首 , and GMM 首 =[gqm1]; Step 2-2, for the first network layer that does not need to be quantized, determine whether there is a calibration data set and quantize it; The first network layer MM 首 =[nm2]; If there is a calibration dataset in nm2 Will Assign value to quantization module input Will Assign a copy of the original module input Will and As input to nm2, output and Will Assign value to the next layer of quantization module input Will Assign value to the next layer of original module input If there is no calibration data set in nm2 No additional processing is performed; After completing step 2-2, the first network layer GMM that does not require quantization is obtained 首 , and GMM 首 =[nm2], no need to quantize nm2; Step 2-3: for any network layer that needs to be quantized after the first network layer, determine whether there is a calibration data set and perform calculation processing; from Select qm a ,qm a It is the network layer that needs to be quantized; If qm a There is a calibration dataset and And belongs to QuantizedConv2d, and As qm a The input of qm a Perform QuantizedConv2d network quantization layer calculation and output and Will Assign value to the next layer of quantization module input Will Assign value to the next layer of original module input If qm a No calibration dataset and And belongs to QuantizedConv2d, directly to qm a Perform calculation and processing of the quantized layer of the QuantizedConv2d network; If qm a There is a calibration dataset and And belongs to QuantizedLinear, and As qm a The input of qm a Perform QuantizedLinear network quantization layer calculation and output and Will Assign value to the next layer of quantization module input Will Assign value to the next layer of original module input If qm a No calibration dataset and And it belongs to QuantizedLinear, directly to qm a Perform QuantizedLinear network quantization layer calculation processing; According to MM 次 ={nm2,…,nm a-1 ,qm a ,qm a+1 ,…,nm b ,nm b+1 ,…,qm A-1 ,nm A }, and the network layers that need to be quantized are sorted one by one. For the remaining network layers, execute steps 2-3; After completing steps 2-3, the non-first network layer after quantization is obtained Steps 2-4: For any network layer after the first network layer that does not require quantization, determine whether there is a calibration data set and perform calculation processing; from Select nm b , nm b It is a network layer that does not need to be quantized; If nm b There is a calibration dataset and Will and As nm b Input, output and Will Assign value to the next layer of quantization module input Will Assign value to the next layer of original module input If nm b No calibration dataset and No additional processing is performed; According to MM 次 ={nm2,…,nm a-1 ,qm a ,qm a+1 ,…,nm b ,nm b+1 ,…,qm A-1 ,nm A }, and the network layers that do not need to be quantized are sorted one by one. For the remaining network layers, execute steps 2-4; After completing steps 2-4, the non-first network layer that does not need to be quantized is obtained Steps 2-5, complete model quantization and save quantization configuration; GMM 首 =[gqm1] and All the quantized network modules are configured and saved; If the current network layer gqm a Belongs to the QuantizedConv2d network module. The QuantizedConv2d network quantization layer needs to save the quantization granularity MP[PT,PC], the bit width BW[4,8] of the low-bit floating point number, and the maximum value of the tensor and low-bit floating point formats If the current network layer gqm a Belongs to the QuantizedLinear network module. The QuantizedLinear network quantization layer needs to save the quantization granularity MP[PT,PC], the bit width BW[4,8] of the low-bit floating point number, and the maximum value of the tensor and low-bit floating point formats Step 3: Export the weights of the quantized model by weight compression; Step 3-1, converting the weight of the quantization module; GMM 首 =[gqm1] and All network modules that have completed quantization undergo weight conversion; If the current network layer gqm a Belongs to the QuantizedConv2d network module, using the maximum value of the tensor in the quantized layer of the QuantizedConv2d network and low-bit floating point formats To simulate the calculation to get the weight The result after low-bit floating point quantization and scaling value Assume that the current weight floating point format is float32 (E8M23), and the low-bit floating point format to be converted is float8 (E4M3). For tensor Any element w_qc in can be represented in float32(E8M23) as Where s represents the sign bit, and p represents the e -1], e represents the exponent value, the offset is 127, d m Indicates the value of the mth bit of the mantissa, which can be 0 or 1. m indicates the number of bits in the mantissa. The corresponding float8(E4M3) is represented as The offset is 7, and the sign bit of w_qc8 here is w_qc 32 The sign bit of w_qc8 is w_qc 32 The exponent value of -127+7 is the last 4 bits, and the mantissa of w_qc8 is w_qc 32 The first 3 bits of the digit; the high bit represents the weight The weight after low bit conversion is recorded as Any element in w_lc is an 8-bit floating point number; If the current network layer gqm a Belongs to the QuantizedLinear network module, using the maximum value of the tensor in the QuantizedLinear network quantization layer and low-bit floating point formats To simulate the calculation to get the weight The result after low-bit floating point quantization and scaling value Assume that the current weight floating point format is float32 (E8M23), and the low-bit floating point format to be converted is float8 (E4M3). For tensor Any element w_ql in can be represented in float32(E8M23) as Where s represents the sign bit, and p represents the e -1], e represents the exponent value, the offset is 127, d m Indicates the value of the mth bit of the mantissa, which can be 0 or 1. m indicates the number of bits in the mantissa. The corresponding float8(E4M3) is represented as The offset is 7, and the sign bit of w_ql8 here is w_ql 32 The sign bit of w_ql8 is w_ql 32 The exponent value of -127+7 is the last 4 bits, and the mantissa of w_ql8 is w_ql 32 The first 3 bits of the digit; the high bit represents the weight The weight after low bit conversion is recorded as Any element w_ll in is an 8-bit floating point number; The network module that completes the weight conversion after step 3-1 is recorded as GLMM 首 =[glqm1] and Step 3-2, extracting low-bit floating point numbers for compression; For GLMM 首 =[glqm1] and Compressed storage; If the current network layer is glqm a Belongs to the QuantizedConv2d network module, and its converted low-bit floating-point number is represented as There are n elements in the weight tensor; assuming that the current weight floating point format is float8 (E4M3), the weight is stored in a 32-bit tensor. Each 32-bit bit can store four 8-bit weight values. The compressed tensor size is 32 bits, fill the low-bit floating-point tensor into the compressed tensor in order (as shown in Figure 4), and record the compressed tensor as If the current network layer is glqm a Belongs to the QuantizedLinear network module, and its converted low-bit floating-point number is represented as There are n elements in the weight tensor; assuming that the current weight floating point format is float8 (E4M3), the weight is stored in a 32-bit tensor. Each 32-bit bit can store four 8-bit weight values. The compressed tensor size is 32 bits, fill the low-bit floating-point tensor into the compressed tensor in order, and record the compressed tensor as The network module that completes weight compression after step 3-2 is recorded as GLFMM 首 =[glfqm1] and Step 3-3, replace the weights and export the model; GLFMM 首 =[glfqm1] and Export the model after weight replacement; Export the model; If the current network layer is glfqm a Belongs to the QuantizedConv2d network module, using the compressed weight tensor and scaling value Replace the original weight tensor Export the network layer; With the is the same value; With the is the same value; With the are the same value; If the current network layer is glfqm a Belongs to the QuantizedLinear network module, using compressed weight tensors and scaling value Replace the original weight tensor Export the network layer; With the is the same value; With the is the same value; With the are the same value; If the current network layer is nm b , export the network layer; At this point, model compression of low-bit floating-point numbers and quantization after model training has been completed.

2. The model compression method based on low-bit floating point numbers and quantization after model training according to claim 1 is characterized in that There are the following steps to construct the QuantizedConv2d network quantization layer: Step A, initializing the search space of low-bit floating point format of weights; Initialize the weighted low-bit floating-point search space C according to the bit width BW[4,8] set by the user; The low-bit floating-point number bit width BW[4,8] is selected from two options, one is 4 bits and the other is 8 bits; Step B, initializing the weight maximum search space; The quantization granularity is recorded as MP[PT,PC], where PT represents the tensor-level quantization granularity and PC represents the channel-level quantization granularity; Step b-1, determining the quantization granularity of the weight quantizer; If it is PT, go to step b-2; If it is PC, go to step b-3; Step b-2, initializing the search space ST of the maximum weight according to the PT; Get the maximum value of the current entire weight tensor and record it as W max , configure the adjustable parameters lower bound percentile β1, upper bound percentile β2 and the number of equally divided search shares k, and set the search space ST with the maximum weight to [β1W max ,β2W max ], and is evenly divided into k search points, the search space of the maximum weight at the tensor level granularity Execute step C; Step b-3, initializing the search space SC of the maximum weight according to the PC; The maximum value of each channel of the weight tensor is obtained at the channel level granularity and recorded as j represents the jth channel, and the adjustable parameters lower bound percentile β1, upper bound percentile β2 and the number of equally divided search shares k are configured. The search space SC of the maximum weight is set according to the channel. The search space range of the jth channel is And divided into k search points, the search space with the maximum value of the jth channel weight Execute step C; Step C, calculating the quantization errors of all low-bit floating point formats and weight maximum value combinations; Step c-1, the first loop traverses the low-bit floating point number configuration search space C; Set the floating point format in the weight quantizer, that is, set the floating point format in the weight quantizer to the elements in the low-bit floating point configuration search space C, traverse the elements in the low-bit floating point configuration search space in order, and the identifier of the first layer loop is recorded as c; Taking 8-bit floating-point numbers as an example, the search space is C = {E6M1, E5M2, E4M3, E3M4, E2M5, E1M6}. In the first cycle, the floating-point format in the weight quantizer is set to E6M1. The weight quantizer will quantize the weight according to the exponent bit as 6 bits and the median bit as 1 bit. Step c-2, the second loop traverses the maximum weight search space S[ST,SC]; Set the maximum value in the weight quantizer, that is, set it as the search object in the maximum value search space S[ST,SC], traverse the elements in the maximum value search space in order, and the identifier of the second layer loop is denoted by s; Taking tensor-level granularity as an example, the search space for the maximum weight of tensor-level granularity is In the first cycle, set the maximum value of the tensor in the weight quantizer to The weight tensor quantizer will constrain the weight range according to the set maximum value of the tensor; Step c-3, determine whether there are calibration data sets CDS_Q and CDS_O, and perform quantitative calibration; The calibration data set used by the quantized module is recorded as CDS_Q, and the calibration data set used by the original module is recorded as CDS_O ； The CDS_Q and CDS_O can be 32, 64 or 128 samples, the number of samples is the same, and both exist or do not exist at the same time; If there are no calibration data sets CDS_Q and CDS_O, proceed to step c-4; If there are calibration data sets CDS_Q and CDS_O, execute step c-5; Step c-4, in the scenario without calibration data, quantify the error record; In the scenario where there is no calibration data, the optimization goal of quantization is to minimize the error of the tensor itself before and after quantization. The tensor quantization error under the current configuration is calculated and recorded, and then step c-6 is executed; If the quantization granularity is PT, use the tensor-level weighted mean square error formula To calculate the error size of the model tensor before and after quantization, ntt represents the number of elements in the tensor without calibration data, and W represents the weight tensor of the model before quantization. Represents the weight tensor of the quantized model; calculates the tensor-level weight mean square error mse_pt_t under the current low-bit floating-point configuration and the tensor maximum configuration, and records it in the intermediate result PT_T_ERROR[c][s]. PT_T_ERROR is a two-dimensional matrix that records the tensor-level weight errors under each low-bit floating-point configuration and each tensor maximum configuration. c is the identifier of the first-layer loop, which corresponds to the current low-bit floating-point configuration, and s is the identifier of the second-layer loop, which corresponds to the current tensor maximum configuration. When the quantization granularity is PC, the channel-level weighted mean square error formula is used To calculate the error size of the model tensor channel j before and after quantization, where nct j represents the number of elements in the output tensor channel j in the scenario without calibration data, W j represents the weight tensor of the model before quantization, Represents the weight tensor of the quantized model; calculates the channel-level weight mean square error mse_pc_t under the current low-bit floating-point configuration and tensor maximum configuration, and records it in the intermediate result PC_T_ERROR[c][s][j]. PC_T_ERROR is a three-dimensional matrix that records each low-bit floating-point configuration, each tensor maximum configuration, and the channel-level weight error under each channel. j represents the channel of the tensor. Step c-5, in the scenario with calibration data, quantify the error record; In the scenario with calibration data, the optimization goal of quantization is to minimize the output error before and after quantization. The output error of the quantized tensor under the current configuration is calculated and recorded, and then step c-6 is executed; If the quantization granularity is PT, use the tensor-level output mean square error formula To calculate the error size of the module output before and after quantization, nto represents the number of elements in the output tensor in the scenario with calibration data, CDS_O represents the calibration data set used by the original module, CDS_Q represents the calibration data set used by the module after quantization, and W represents the weight tensor value of the model before quantization. Represents the weight tensor value of the quantized model, func represents the function torch.nn.functional.conv2d corresponding to the Conv2d module in pytorch; calculates the tensor-level output mean square error mse_pt_o under the current low-bit floating-point configuration and the tensor maximum configuration, and records it in the intermediate result PT_O_ERROR[c][s]. PT_O_ERROR is a two-dimensional matrix that records the tensor-level output errors under each low-bit floating-point configuration and each tensor maximum configuration; If the quantization granularity is PC, use the tensor-level output mean square error formula To calculate the error size of the module output before and after quantization, where nco j Indicates the number of elements in the output tensor channel j, CDS_O j Indicates the data corresponding to channel j of the calibration data set used by the original module, CDS_Q j Represents the data corresponding to channel j in the calibration data set used by the quantized module, W j represents the weight tensor of the model before quantization, Represents the weight tensor of the quantized model, func represents the function torch.nn.functional.conv2d corresponding to the Conv2d module in pytorch; calculates the channel-level output mean square error mse_pc_o under the current low-bit floating-point configuration and tensor maximum configuration, and records it in the intermediate result PC_O_ERROR[c][s][j]. PC_O_ERROR is a three-dimensional matrix that records each low-bit floating-point configuration, each tensor maximum configuration, and the channel-level output error under each channel; Step c-6, determining whether the maximum value search space has been traversed; If the elements in the S[ST,SC] space are traversed, execute step c-7; If the elements in the S[ST,SC] space have not been traversed, execute step c-2; Step c-7, determining whether the low-bit floating point number configuration search space has been traversed; If the elements in space C are traversed, execute step D; If the elements in space C have not been traversed, execute step c-1; Step D, obtaining an optimal low-bit floating point format and weight maximum value configuration; Step d-1, determining whether the quantization granularity of the weight quantizer is at the tensor level; If the quantization granularity of the weight quantizer is PT, execute step d-2; If the quantization granularity of the weight quantizer is PC, execute step d-3; Step d-2, updating the configuration in the weight quantizer according to the tensor level; The operations on the tensor-level weight mean square error PT_T_ERROR[c][s] and the tensor-level output mean square error PT_O_ERROR[c][s] in value selection are the same, collectively referred to as the tensor-level mean square error PT_ERROR[c][s]; Find the minimum tensor-level mean square error PT_ERROR[c min ][s min ], c min is the low-bit floating-point search space subscript corresponding to the minimum tensor-level mean square error, s min is the search space subscript for the maximum weight corresponding to the minimum tensor-level mean square error; Update the corresponding low-bit floating-point format LFF in the weight quantizer to the corresponding low-bit floating-point search space C min The element corresponding to the subscript, that is, LFF = C[c min ]; Update the corresponding maximum weight value W_MAX in the weight quantizer to the corresponding maximum weight value search space ST s min The element corresponding to the subscript, that is, W_MAX = ST[s min ], execute step d-4; Step d-3, updating the configuration in the weight quantizer according to the channel level; The channel-level weight mean square error PC_T_ERROR[c][s][j] and the channel-level output mean square error PC_O_ERROR[c][s][j] have the same operation in value selection, collectively referred to as the channel-level mean square error PC_ERROR[c][s][j]; Find the minimum channel-level mean square error in the three-dimensional matrix corresponding to each channel channel by channel is the low-bit floating-point search space subscript corresponding to the minimum mean square error of channel j, is the search space subscript of the maximum weight corresponding to the minimum mean square error of channel j; Select The c with the highest number of occurrences min As the corresponding low-bit floating-point configuration subscript in the weight quantizer, the corresponding low-bit floating-point format LFF in the weight quantizer is updated to the corresponding low-bit floating-point search space C in c min The element corresponding to the subscript, that is, LFF = C[c min ]; After determining the elements of the low-bit floating-point search space in the weight quantizer, find the minimum channel-level mean square error in the three-dimensional matrix corresponding to each channel channel by channel For min The search space subscript of the maximum weight corresponding to the minimum mean square error in channel j, and the corresponding maximum weight W_MAX in the weight quantizer is calculated channel by channel j , updated to the corresponding search space SC with the maximum weight j middle The element corresponding to the subscript, that is Execute step d-4; Step d-4, determine whether there are calibration data sets CDS_Q and CDS_O, and if so, calculate the input of the next network layer; The calibration data set used by the quantized module is recorded as CDS_Q, and the calibration data set used by the original module is recorded as CDS_O; If there are no calibration data sets CDS_Q and CDS_O, the execution is completed; If there are calibration data sets CDS_Q and CDS_O; Use the weight quantizer after updating the configuration to calculate the quantized weight According to the input calibration data set CDS_Q, the output of the quantization module is recorded as CDS_QC_O. Assign it to CDS_QC_O, that is According to the input calibration data set CDS_O, the original module output is recorded as CDS_OC_O, and func(CDS_O,W) is assigned to CDS_OC_O, that is, CDS_OC_O = func(CDS_O,W), where func represents the function torch.nn.functional.conv2d corresponding to the Conv2d module in pytorch; CDS_QC_O and CDS_OC_O are used as the input of the next layer of the network.

3. The model compression method based on low-bit floating point numbers and quantization after model training according to claim 1 is characterized in that There are the following steps to construct the QuantizedLinear network quantization layer: Step I, initializing the search space of low-bit floating point format of weights; Initialize the weighted low-bit floating-point search space C according to the bit width BW[4,8] set by the user; The low-bit floating-point number bit width BW[4,8] is selected from two options, one is 4 bits and the other is 8 bits; Step II, initialize the weight maximum search space; The quantization granularity is recorded as MP[PT,PC], where PT represents the tensor-level quantization granularity and PC represents the channel-level quantization granularity; Step II-1, determine the quantization granularity of the weight quantizer; If it is PT, go to step II-2; If it is PC, proceed to step II-3; Step II-2, initializing the search space ST of the maximum weight according to the PT; Get the maximum value of the current entire weight tensor and record it as W max , configure the adjustable parameters lower bound percentile β1, upper bound percentile β2 and the number of equally divided search shares k, and set the search space ST with the maximum weight to [β1W max ,β2W max ], and is evenly divided into k search points, the search space of the maximum weight at the tensor level granularity Execute step III; Step II-3, initializing the search space SC of the maximum weight according to the PC; The maximum value of each channel of the weight tensor is obtained at the channel level granularity and recorded as j represents the jth channel, and the adjustable parameters lower bound percentile β1, upper bound percentile β2 and the number of equally divided search shares k are configured. The search space SC of the maximum weight is set according to the channel. The search space range of the jth channel is And divided into k search points, the search space with the maximum value of the jth channel weight Execute step III; Step III, calculating the quantization errors of all low-bit floating point formats and weight maximum value combinations; Step III-1, the first loop traverses the low-bit floating-point configuration search space C; Set the floating point format in the weight quantizer, that is, set the floating point format in the weight quantizer to the elements in the low-bit floating point configuration search space C, traverse the elements in the low-bit floating point configuration search space in order, and the identifier of the first layer loop is recorded as c; Taking 8-bit floating-point numbers as an example, the search space is C = {E6M1, E5M2, E4M3, E3M4, E2M5, E1M6}. In the first cycle, the floating-point format in the weight quantizer is set to E6M1. The weight quantizer will quantize the weight according to the exponent bit as 6 bits and the median bit as 1 bit. Step III-2, the second loop traverses the maximum weight search space S[ST,SC]; Set the maximum value in the weight quantizer, that is, set it as the search object in the maximum value search space S[ST,SC], traverse the elements in the maximum value search space in order, and the identifier of the second layer loop is denoted by s; Taking tensor-level granularity as an example, the search space for the maximum weight of tensor-level granularity is In the first cycle, set the maximum value of the tensor in the weight quantizer to The weight tensor quantizer will constrain the weight range according to the set maximum value of the tensor; Step III-3, determine whether there are calibration data sets CDS_Q and CDS_O, and perform quantitative calibration; The calibration data set used by the quantized module is recorded as CDS_Q, and the calibration data set used by the original module is recorded as CDS_O; The CDS_Q and CDS_O can be 32, 64 or 128 samples, the number of samples is the same, and both exist or do not exist at the same time; If there is no calibration data set CDS_Q and CDS_O, proceed to step III-4; If there are calibration data sets CDS_Q and CDS_O, proceed to step III-5; Step III-4, quantify the error records in the scenario without calibration data; In the scenario where there is no calibration data, the optimization goal of quantization is to minimize the error of the tensor itself before and after quantization. The tensor quantization error under the current configuration is calculated and recorded, and then step III-6 is executed; If the quantization granularity is PT, use the tensor-level weighted mean square error formula To calculate the error size of the model tensor before and after quantization, ntt represents the number of elements in the tensor without calibration data, and W represents the weight tensor of the model before quantization. Represents the weight tensor of the quantized model; calculates the tensor-level weight mean square error mse_pt_t under the current low-bit floating-point configuration and the tensor maximum configuration, and records it in the intermediate result PT_T_ERROR[c][s]. PT_T_ERROR is a two-dimensional matrix that records the tensor-level weight errors under each low-bit floating-point configuration and each tensor maximum configuration. c is the identifier of the first-layer loop, which corresponds to the current low-bit floating-point configuration, and s is the identifier of the second-layer loop, which corresponds to the current tensor maximum configuration. When the quantization granularity is PC, the channel-level weighted mean square error formula is used To calculate the error size of the model tensor channel j before and after quantization, where nct j represents the number of elements in the output tensor channel j in the scenario without calibration data, W j represents the weight tensor of the model before quantization, Represents the weight tensor of the quantized model; calculates the channel-level weight mean square error mse_pc_t under the current low-bit floating-point configuration and tensor maximum configuration, and records it in the intermediate result PC_T_ERROR[c][s][j]. PC_T_ERROR is a three-dimensional matrix that records each low-bit floating-point configuration, each tensor maximum configuration, and the channel-level weight error under each channel. j represents the channel of the tensor. Step III-5, in the scenario with calibration data, quantify the error records; In the scenario with calibration data, the optimization goal of quantization is to minimize the output error before and after quantization. The output error of the quantized tensor under the current configuration is calculated and recorded, and then step III-6 is executed; If the quantization granularity is PT, use the tensor-level output mean square error formula To calculate the error size of the module output before and after quantization, nto represents the number of elements in the output tensor in the scenario with calibration data, CDS_O represents the calibration data set used by the original module, CDS_Q represents the calibration data set used by the module after quantization, and W represents the weight tensor value of the model before quantization. Represents the weight tensor value of the quantized model, func represents the function torch.nn.functional.linear corresponding to the Linear module in pytorch; calculates the tensor-level output mean square error mse_pt_o under the current low-bit floating-point configuration and the tensor maximum configuration, and records it in the intermediate result PT_O_ERROR[c][s]. PT_O_ERROR is a two-dimensional matrix that records the tensor-level output errors under each low-bit floating-point configuration and each tensor maximum configuration; If the quantization granularity is PC, use the tensor-level output mean square error formula To calculate the error size of the module output before and after quantization, where nco j Indicates the number of elements in the output tensor channel j, CDS_O j Indicates the data corresponding to channel j of the calibration data set used by the original module, CDS_Q j Represents the data corresponding to channel j in the calibration data set used by the quantized module, W j represents the weight tensor of the model before quantization, Represents the weight tensor of the quantized model, func represents the function torch.nn.functional.linear corresponding to the Linear module in pytorch; calculates the channel-level output mean square error mse_pc_o under the current low-bit floating-point configuration and tensor maximum configuration, and records it in the intermediate result PC_O_ERROR[c][s][j]. PC_O_ERROR is a three-dimensional matrix that records each low-bit floating-point configuration, each tensor maximum configuration, and the channel-level output error under each channel; Step III-6, determine whether the maximum value search space has been traversed; If the elements in the S[ST,SC] space are traversed, execute step III-7; If the elements in the S[ST,SC] space have not been traversed, execute step III-2; Step III-7, determining whether the low-bit floating point number configuration search space has been traversed; If the elements in space C are traversed, execute step IV; If the elements in space C have not been traversed, execute step III-1; Step IV, obtaining the best low-bit floating point format and weight maximum value configuration; Step IV-1, determine whether the quantization granularity of the weight quantizer is at the tensor level; If the quantization granularity of the weight quantizer is PT, execute step IV-2; If the quantization granularity of the weight quantizer is PC, execute step IV-3; Step IV-2, update the configuration in the weight quantizer according to the tensor level; The operations on the tensor-level weight mean square error PT_T_ERROR[c][s] and the tensor-level output mean square error PT_O_ERROR[c][s] in value selection are the same, collectively referred to as the tensor-level mean square error PT_ERROR[c][s]; Find the minimum tensor-level mean square error PT_ERROR[c min ][s min ], c min is the low-bit floating-point search space subscript corresponding to the minimum tensor-level mean square error, s min is the search space subscript for the maximum weight corresponding to the minimum tensor-level mean square error; Update the corresponding low-bit floating-point format LFF in the weight quantizer to the corresponding low-bit floating-point search space C min The element corresponding to the subscript, that is, LFF = C[c min ]; Update the corresponding maximum weight value W_MAX in the weight quantizer to the corresponding maximum weight value search space ST s min The element corresponding to the subscript, that is, W_MAX = ST[s min ], execute step IV-4; Step IV-3, updating the configuration in the weight quantizer according to the channel level; The channel-level weight mean square error PC_T_ERROR[c][s][j] and the channel-level output mean square error PC_O_ERROR[c][s][j] have the same operation in value selection, collectively referred to as the channel-level mean square error PC_ERROR[c][s][j]; Find the minimum channel-level mean square error in the three-dimensional matrix corresponding to each channel channel by channel is the low-bit floating-point search space subscript corresponding to the minimum mean square error of channel j, is the search space subscript of the maximum weight corresponding to the minimum mean square error of channel j; Select The c with the highest number of occurrences min As the corresponding low-bit floating-point configuration subscript in the weight quantizer, the corresponding low-bit floating-point format LFF in the weight quantizer is updated to the corresponding low-bit floating-point search space C in c min The element corresponding to the subscript, that is, LFF = C[c min ]; After determining the elements of the low-bit floating-point search space in the weight quantizer, find the minimum channel-level mean square error in the three-dimensional matrix corresponding to each channel channel by channel For min The search space subscript of the maximum weight corresponding to the minimum mean square error in channel j, and the corresponding maximum weight W_MAX in the weight quantizer is calculated channel by channel j , updated to the corresponding search space SC with the maximum weight j middle The element corresponding to the subscript, that is Execute step IV-4; Step IV-4, determine whether there are calibration data sets CDS_Q and CDS_O, and if so, calculate the input of the next network layer; The calibration data set used by the quantized module is recorded as CDS_Q, and the calibration data set used by the original module is recorded as CDS_O; If there are no calibration data sets CDS_Q and CDS_O, the execution is completed; If there are calibration data sets CDS_Q and CDS_O; Use the weight quantizer after updating the configuration to calculate the quantized weight According to the input calibration data set CDS_Q, the output of the quantization module is recorded as CDS_QL_O. Assign a value to CDS_QL_O, that is According to the input calibration data set CDS_O, the original module output is recorded as CDS_OL_O, and func(CDS_O,W) is assigned to CDS_OL_O, that is, CDS_OL_O = func(CDS_O,W), where func represents the function torch.nn.functional.linear corresponding to the Linear module in pytorch; CDS_QL_O and CDS_OL_O are used as the input of the next layer of the network.

4. The model compression method based on low-bit floating point numbers and quantization after model training according to claim 1, characterized in that: OM={om1,om2,…,om a-1 ,om a ,om a+1 ,…,om b ,om b+1 ,…,om A-1 ,om A The network layers in} are arranged sequentially; the quantization of the network layers is also completed sequentially.

5. The model compression method based on low-bit floating point numbers and quantization after model training according to claim 1, characterized in that: Taking an 8-bit floating point number as an example, the search space is initialized to C = {E6M1, E5M2, E4M3, E3M4, E2M5, E1M6}, where E represents the number of exponent bits and M represents the format of the mantissa bits; a binary floating point number can be expressed as Where X FP Represents a floating point number, s represents the sign bit, and p represents the range [0,2 e -1], e represents the exponent value, b represents the offset, and the default value is 2 E-1 -1, E represents the number of exponent bits, d m Indicates the value of the mth bit of the mantissa, which can be 0 or 1, where m represents the number of bits in the mantissa.

Images