Model quantization method and apparatus

Abstract

Embodiments of the present specification provide a model quantization method and device, wherein the model quantization method comprises: receiving a model quantization request submitted for an initial task processing model, splitting a first tensor expressed in a number of bits into different components according to the number of bits, to obtain at least two parts of the first tensor, according to an initial data precision corresponding to the initial task processing model, the first tensor comprising a first tensor of training data and a first tensor of model parameters, respectively adjusting quantization of the at least two parts according to a quantization precision contained in the model quantization request, to generate a second tensor conforming to a number of bits corresponding to the quantization precision, wherein the quantization adjustment comprises: reducing the value of at least one of the parts and correspondingly amplifying the value of at least another of the parts, so that the second tensor satisfies the initial data precision, and generating a target task processing model using the second tensor.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the technical field of computer technology, and particularly relate to a model quantization method. BACKGROUND

[0002] Artificial intelligence (AI) is the use of digital computers or digital computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is the design principle and implementation method of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making. The research in the field of artificial intelligence includes robots, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, basic theory, etc.

[0003] Neural network models are widely used. By quantizing the operators in the neural network model, i.e., quantizing the parameters of the operators and quantizing the input data, the floating-point operation can be converted into the operation on fixed-point numbers, and the model size, inference speed and power consumption and other aspects can be improved. According to the data range of the operator, the quantization parameters of the operator are determined, which can improve the accuracy of the data processing result of the quantized operator. However, the current processing method often leads to poor overall processing performance. SUMMARY

[0004] Therefore, the embodiments of the present specification provide a model quantization method. One or more embodiments of the present specification also relate to a model quantization device, a computing device, a computer-readable storage medium and a computer program to solve the technical defects in the prior art.

[0005] According to a first aspect of the embodiments of the present specification, a model quantization method is provided, comprising:

[0006] receiving a model quantization request submitted for an initial task processing model;

[0007] According to the initial data precision corresponding to the initial task processing model, a first tensor expressed in the number of bits is split according to different components of the number of bits to obtain at least two parts of the first tensor, and the first tensor includes a first tensor of training data and a first tensor of model parameters;

[0008] quantize the at least two parts according to a quantization precision contained in the model quantization request, to generate a second tensor conforming to a bit number corresponding to the quantization precision, wherein the quantization adjustment comprises: reducing values of at least one of the parts and correspondingly amplifying values of at least another one of the parts so that the second tensor satisfies the initial data precision;

[0009] generate the target task processing model by using the second tensor.

[0010] According to a second aspect of the embodiments of the present specification, a model quantization apparatus is provided, comprising:

[0011] a receiving module configured to receive a model quantization request submitted for an initial task processing model;

[0012] a splitting module configured to split, according to an initial data precision corresponding to the initial task processing model, a first tensor expressed in a bit number into different components of the bit number, to obtain at least two parts of the first tensor, the first tensor comprising a first tensor of training data and a first tensor of model parameters;

[0013] an adjusting module configured to quantize the at least two parts according to a quantization precision contained in the model quantization request, to generate a second tensor conforming to a bit number corresponding to the quantization precision, wherein the quantization adjustment comprises: reducing values of at least one of the parts and correspondingly amplifying values of at least another one of the parts so that the second tensor satisfies the initial data precision;

[0014] a generating module configured to generate the target task processing model by using the second tensor.

[0015] According to a third aspect of the embodiments of the present specification, a computing device is provided, comprising:

[0016] a memory and a processor;

[0017] the memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions to implement steps of any one of the model quantization methods.

[0018] According to a fourth aspect of the embodiments of the present specification, a computer readable storage medium is provided, which stores computer executable instructions, and the instructions are executed by a processor to implement steps of any one of the model quantization methods.

[0019] According to a fifth aspect of the embodiments of the present specification, a computer program is provided, and when the computer program is executed in a computer, the computer is caused to execute steps of the model quantization method.

[0020] One embodiment of the specification splits a first tensor expressed in a number of bits into at least two parts according to different components of the number of bits based on an initial data precision corresponding to an initial task processing model, the first tensor including a first tensor of training data and a first tensor of model parameters, adjusts the at least two parts respectively according to a quantization precision included in a model quantization request submitted for the initial task processing model, and generates a second tensor conforming to a number of bits corresponding to the quantization precision, wherein the adjustment includes reducing the value of at least one of the parts and correspondingly increasing the value of at least another of the parts so that the second tensor satisfies the initial data precision, and generates a target task processing model using the second tensor.

[0021] An embodiment of the specification adjusts the training data and the model parameters of the initial data precision according to the quantization precision, and trains the initial task processing model using the quantization result corresponding to the training data, so as to adjust the model parameters, that is, by using the training data of low precision to train the model, the accuracy of the training result can be ensured while the task processing performance of the task processing model is improved. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is a schematic diagram of a model quantization process provided by one embodiment of the specification;

[0023] Figure 2 is a flowchart of a model quantization method provided by one embodiment of the specification;

[0024] Figure 3a is a schematic diagram of a model quantization method provided by one embodiment of the specification;

[0025] Figure 3b is a schematic diagram of a specific implementation process of quantization provided by one embodiment of the specification;

[0026] Figure 4 is a process flowchart of a model quantization method provided by one embodiment of the specification;

[0027] Figure 5 is a structural schematic diagram of a model quantization device provided by one embodiment of the specification;

[0028] Figure 6 is a structural block diagram of a computing device provided by one embodiment of the specification. DETAILED DESCRIPTION

[0029] In the following description, numerous specific details are set forth to provide a thorough understanding of the specification. However, the specification can be practiced without the specific details. In other instances, well-known methods, procedures, components, and networks have not been described in detail so as not to unnecessarily obscure the specification. The specification should not be construed as limited to the embodiments set forth herein and thus include other embodiments and modifications that are apparent to those skilled in the art.

[0030] The terminology used in this description of one or more embodiments should not be interpreted as limiting the scope of the one or more embodiments of the specification. The singular forms "a," "an," and "the" used in this description should not be construed as limiting the specification to a single embodiment and thus should be read to also cover the plural forms, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this description, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0031] It will be understood that, although the terms first, second, etc. can be used herein to describe various information, these terms are not intended to denote a temporal or chronological order. Rather, these terms are used only to distinguish one from another. For example, without departing from the scope of the one or more embodiments of the specification, first can be termed second and, similarly, second can be termed first. The term "if' as used herein can be interpreted as meaning "when" or "in response to determining" depending on the context.

[0032] First, the noun terms related to the one or more embodiments of the specification are explained.

[0033] FP8: 8-bit floating point number, where 1 bit is a sign bit and the remaining 7 bits are mantissa (fraction bit) + exponent (exponent bit).

[0034] FP16: 16-bit floating point number.

[0035] FP32: 32-bit floating point number.

[0036] QAT: quantization aware training, static quantization-aware training, weight quantization, activation quantization, quantization values in the training process are modeled.

[0037] INT8: 8-bit signed integer.

[0038] BIT (Binary digit): a computer term, is a unit of information, is translated from English BIT. It is also a bit in binary digits, a unit of measurement of information, in binary number system, each 0 or 1 is a bit (bit), therefore, the number of bits is the number of binary digits.

[0039] Tensor: is the generalization of the concept of vector and matrix, scalar is 0 order tensor, vector is 1 order tensor, matrix is two order tensor, and a new array can be obtained by combining multiple matrices into a three order tensor, which can be intuitively understood as a cubic matrix composed of numbers.

[0040] In the present specification, a model quantization method is provided, the present specification also relates to a model quantization device, a computing device, a computer readable storage medium, and a computer program, which are described in detail one by one in the following embodiments.

[0041] Figure 1 A schematic diagram of a model quantization method according to an embodiment of the present specification is shown, Figure 1 In the present specification, a model quantization method is provided, the present specification also relates to a model quantization device, a computing device, a computer readable storage medium, and a computer program, which are described in detail one by one in the following embodiments. In the present specification, a model quantization method is provided, the present specification also relates to a model quantization device, a computing device, a computer readable storage medium, and a computer program, which are described in detail one by one in the following embodiments.

[0042] The embodiment of the present specification adjusts the training data and the model parameters of the initial data precision by quantizing the precision, and trains the initial task processing model by using the quantization result corresponding to the training data, so as to realize the adjustment of the model parameters, that is, by using the low-precision training data to train the model, the accuracy of the training result can be ensured, and the task processing performance of the task processing model can be improved.

[0043] Figure 2 A flowchart of a model quantization method according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0044] Step 202, receiving a model quantization request submitted for an initial task processing model.

[0045] With the development of technology, deep neural networks have made great progress in computer vision tasks such as image classification, object detection, image segmentation, natural language processing, or video classification. However, deep neural networks often contain a large number of model parameters, which require a large amount of device resources (such as storage space and computing power, etc.), and are difficult to run efficiently on terminal devices. Therefore, it is necessary to compress the neural network to reduce the device resources occupied by the neural network.

[0046] Model quantization technology is an effective method for compressing neural networks. This technology can convert the parameters of the neural network from high-bit (for example, 32 bits) values to low-bit (for example, 4 bits) values for representation, thereby significantly reducing the resources occupied by the parameters of the neural network.

[0047] Based on this, before training the initial task processing model, the embodiment of the present specification can first receive a model quantization request submitted by a user for the initial task processing model, and obtain the training data of the initial task processing model, and then quantize the model parameters of the initial task processing model. The training data used for model training can also be quantized to realize model training based on the quantization result.

[0048] In specific implementation, receiving a model quantization request submitted for an initial task processing model includes:

[0049] Receiving a model quantization request of an initial task processing model submitted by a user by calling a model quantization interface;

[0050] In response to the model quantization request, traversing the network layers contained in the initial task processing model;

[0051] Determining the network layer to be quantized according to the traversal result, and adding a corresponding quantization node to the network layer to be quantized.

[0052] Specifically, the model quantization method provided in the embodiments of the present specification is applied to a server. The server can provide a model quantization interface for a user. The user can submit a model quantization request for an initial task processing model to the server by calling the model quantization interface. The server responds to the model quantization request, traverses a network layer included in the initial task processing model to determine a to-be-quantized network layer in the initial task processing model, and then can add a corresponding quantization node to the to-be-quantized network layer. The quantization node can be used to identify the network layer as the to-be-quantized network layer, so as to quantize network parameters, i.e., model parameters, of the network layer when performing model quantization.

[0053] In actual application, the initial task processing model can be traversed by using a recursive algorithm. The specific determination can be made according to actual requirements, which is not limited herein.

[0054] The embodiments of the present specification provide a model quantization interface for a user. For the user, only one interface needs to be called to quantize the model, that is, only one line of code needs to be modified on the user side to enable model quantization, without the need for other modification operations. This is conducive to reducing the processing complexity, thereby improving the user experience.

[0055] In step 204, at least two parts of the first tensor are obtained by splitting the first tensor expressed by a number of bits into different components of the number of bits according to an initial data precision corresponding to the initial task processing model. The first tensor includes a first tensor of training data and a first tensor of model parameters.

[0056] Specifically, the initial data precision corresponding to the initial task processing model can be FP32, i.e., 32-bit floating point, single precision, and the number of bits of the initial data precision is 32 bits. Alternatively, the initial data precision can be FP16 or FP64 or other precisions. The specific determination can be made according to actual requirements, which is not limited herein.

[0057] In the embodiments of the present specification, the model quantization request can include quantization precision, and the quantization precision can be FP8, i.e., 8-bit floating point. The number of bits corresponding to the quantization precision is 8 bits.

[0058] In the quantization adjustment of the first tensor with the initial data precision of FP32 to adjust it to the second tensor with the precision of FP8, due to the tensors with the precision of FP8 and FP32, the difference lies in the different components or the different numerical ranges of the same component, therefore, in quantizing the tensor with the precision of FP32 to the tensor with the precision of FP8, specifically, the tensor with the precision of FP32 needs to be compressed according to the different components of the tensor with the precision of FP8, in this case, the different components of the first tensor need to be split according to the bit number of the initial data precision (FP32) corresponding to the initial task processing model, to generate at least two parts of the first tensor.

[0059] In a specific implementation, the initial data precision and the data type corresponding to the quantization precision are the same, and the bit number corresponding to the initial data precision is greater than the quantization bit number corresponding to the quantization precision.

[0060] Correspondingly, the splitting of the first tensor expressed in bit number according to the different components of the bit number corresponding to the initial data precision of the initial task processing model to obtain at least two parts of the first tensor includes:

[0061] According to the initial data precision corresponding to the initial task processing model, the data structure corresponding to the first tensor expressed in bit number is determined, wherein the data structure includes at least two components.

[0062] According to the data structure, the at least two components of the first tensor are split to generate at least two parts of the first tensor.

[0063] The data structure is composed of a sign bit, a mantissa bit, a base number and an exponent bit.

[0064] Correspondingly, the splitting of the first tensor according to the data structure to generate at least two parts of the first tensor includes:

[0065] The data corresponding to the exponent bit in the first tensor is split into a first part, and the data corresponding to the mantissa bit in the first tensor is split into a second part.

[0066] Specifically, as described above, the initial data precision corresponding to the initial task processing model can be FP32, and the quantization precision in the model quantization request can be FP8, that is, the data types corresponding to the initial data precision and the quantization precision are the same, both are floating point type, and the bit number corresponding to the initial data precision is greater than the quantization bit number corresponding to the quantization precision.

[0067] According to the bit number of the initial data precision corresponding to the initial task processing model, different components of the first tensor are split to generate at least two parts of the first tensor, specifically, the data structure corresponding to the first tensor is determined according to the bit number of the initial data precision corresponding to the initial task processing model, and at least two components of the first tensor are split according to the data structure to generate at least two parts.

[0068] Further, in the case that the data types corresponding to the initial data precision and the quantization precision are both floating point types, the data structure of the first tensor of the training data and the first tensor of the model parameters can be composed of a sign bit, a mantissa bit, a base, and an exponent bit.

[0069] For example, the training data or the model parameter is z = (-1) n *x*2 y where n is a sign bit, x is a mantissa bit, 2 is a base, and y is an exponent bit, which together constitute the first tensor.

[0070] Therefore, when splitting, the data corresponding to the exponent bit in the first tensor can be split into a first part, and the data corresponding to the mantissa bit in the first tensor can be split into a second part.

[0071] The embodiments of the present specification split different components of the first tensor to compress the split results according to the bit number corresponding to the quantization precision in the subsequent processing process, and train the initial task processing model using the compressed results corresponding to the training data, so as to adjust the model parameters, that is, by using low-precision training data for model training, the accuracy of the training results can be guaranteed while the task processing performance of the task processing model is improved.

[0072] In addition, after traversing the network layers in the initial task processing model, determining the network layer to be quantized according to the traversal result, and adding the quantization node of the network layer to be quantized, in the case that the initial task processing model needs to be trained, it can be detected whether each network layer in the initial task processing model has a quantization node, if yes, the network layer can be determined as the network layer to be quantized, that is, in the case that the network layer to be quantized has a quantization node, the network parameters of the network layer to be quantized are determined, and the first tensor of the network parameters is determined, wherein the first tensor is expressed by the bit number, and then the first tensor of the network parameters is split according to the bit number of different components.

[0073] In step 206, according to the quantization precision contained in the model quantization request, the at least two parts are respectively quantized and adjusted to generate a second tensor that meets the bit number corresponding to the quantization precision, wherein the quantization adjustment includes: reducing the value of at least one of the parts and correspondingly amplifying the value of at least another part to make the second tensor meet the initial data precision.

[0074] Specifically, as mentioned earlier, the initial data precision corresponding to the initial task processing model can be FP32, and the quantization precision contained in the model quantization request can be FP8.

[0075] When the first tensor with an initial data precision of FP32 is quantized and adjusted to a second tensor with a precision of FP8, since the tensors with precisions of FP8 and FP32 differ in component parts, or in other words, the same component part has a different corresponding numerical range, when quantizing the tensor with a precision of FP32 to a tensor with a precision of FP8, it is necessary to compress the tensor with a precision of FP32 according to the different component parts of the tensor with a precision of FP8.

[0076] Since the first tensor with a precision of FP32 and the second tensor with a precision of FP8 are both floating-point data, their corresponding data structures are the same, i.e., they can both be composed of a sign bit, a mantissa bit, a base, and an exponent bit.

[0077] Following the above example, the training data can be z1 = (-1) n1 *x1*2 y1 where n1 is a sign bit, x1 is a mantissa bit, 2 is a base, and y1 is an exponent bit, which together constitute the first tensor. By splitting the different component parts of the first tensor, the mantissa bit x1 can be split into a first part and the exponent bit y1 can be split into a second part, and then x1 and y1 can be compressed to quantize and adjust them to generate the corresponding quantization result.

[0078] Similarly, the model parameter can be z2 = (-1) n2 *x2*2 y2 where n2 is a sign bit, x2 is a mantissa bit, 2 is a base, and y2 is an exponent bit, which together constitute the first tensor. By splitting the different component parts of the first tensor, the mantissa bit x2 can be split into a first part and the exponent bit y2 can be split into a second part, and then x2 and y2 can be compressed to quantize and adjust them to generate the corresponding quantization result.

[0079] In actual application, since the first tensor with the precision of FP32 has the sum of the data bit numbers corresponding to the sign bit, the mantissa bit and the exponent bit equal to 32, and the second tensor with the precision of FP8 has the sum of the data bit numbers corresponding to the sign bit, the mantissa bit and the exponent bit equal to 8, in the case that the data bit number of the sign bit is all 1, the mantissa bit and the exponent bit of the first tensor with the precision of FP32 are different from those of the second tensor with the precision of FP8, that is, the corresponding numerical ranges are different. In this case, the at least two parts of the first tensor are quantized and adjusted according to the bit number corresponding to the quantization precision, specifically, the values corresponding to the exponent bit and / or the mantissa bit of the first tensor are compressed according to the bit number corresponding to the quantization precision.

[0080] In specific implementation, in the case that the quantization precision is FP8, that is, the first tensor needs to be quantized into the second tensor with the precision of FP8, the generated second tensor is the tensor with the precision of FP8, and therefore, the data structure of the second tensor is composed of the sign bit, the mantissa bit, the base and the exponent bit.

[0081] Correspondingly, the quantization adjustment of the at least two parts of the first tensor respectively according to the quantization precision contained in the model quantization request to generate the second tensor with the bit number corresponding to the quantization precision comprises:

[0082] determining a preset data threshold corresponding to the exponent bit in the second tensor according to the bit number corresponding to the quantization precision;

[0083] judging whether the value corresponding to the first part is greater than the preset data threshold;

[0084] if yes, taking the preset data threshold as the data corresponding to the exponent bit in the second tensor;

[0085] performing quantization processing on the second part according to the first part and the preset data threshold to generate the data corresponding to the mantissa bit in the second tensor;

[0086] if no, taking the first part as the data corresponding to the exponent bit in the second tensor, and taking the second part as the data corresponding to the mantissa bit in the second tensor, to generate the second tensor based on the first part and the second part.

[0087] Specifically, the data structure of the second tensor is composed of a sign bit, a mantissa bit, a base, and an exponent bit, and in the case where the data precision thereof is FP8, the data bit number of the mantissa bit is 3 bits, the data bit number of the exponent bit is 4 bits, the value range corresponding to the exponent bit is 0000 to 1111, after removing the two special meaning numbers 0000 and 1111, the value range corresponding to the exponent bit is actually 0001 to 1110, if converted into a decimal number, the value range corresponding to the exponent bit is 1 to 14, and the exponent offset of the 8-bit floating point number is 7, in combination with the exponent offset to map the aforementioned value range, the value range of the exponent bit is actually -6 to 7, and therefore, the preset data threshold corresponding to the exponent bit in the second tensor is 7.

[0088] In addition, the data structure of the first tensor is also composed of a sign bit, a mantissa bit, a base, and an exponent bit, but since the data precision of the first tensor is FP32, the data bit number of the mantissa bit is 23 bits, the data bit number of the exponent bit is 8 bits, the value range corresponding to the exponent bit is 00000001 to 11111110, if converted into a decimal number, the value range of the mantissa bit is -126 to 127, and it can be seen that the value range of the exponent bit and the mantissa bit of the first tensor with the data precision of FP32 is different from that of the second tensor with the data precision of FP8.

[0089] In this case, if the first tensor with the data precision of FP32 needs to be quantized into the second tensor with the data precision of FP8, it is necessary to first determine whether the first part of the first tensor (the value of the exponent bit of the first tensor with the data precision of FP32) is greater than the preset data threshold 7 corresponding to the exponent bit in the second tensor, if not, it indicates that the value of the exponent bit of the first tensor does not exceed the value range of the exponent bit in the second tensor, in this case, the first part of the first tensor can be taken as the data corresponding to the exponent bit in the second tensor, and the second part of the first tensor can be taken as the data corresponding to the mantissa bit in the second tensor, so as to generate the second tensor based on the first part and the second part of the first tensor.

[0090] Among them, since the value range of the mantissa bit of the first tensor with the data precision of FP32 is also different from that of the second tensor with the data precision of FP8, the data bit number of the mantissa bit of the first tensor with the data precision of FP32 is 23 bits, while the data bit number of the mantissa bit of the second tensor with the data precision of FP8 is only 3 bits, therefore, when the second part of the first tensor is taken as the data corresponding to the mantissa bit in the second tensor, the rescale operation can be performed on the second part of the first tensor by rounding, so as to scale it from 23 bits to 3 bits, and the scaling result is taken as the data corresponding to the mantissa bit in the second tensor.

[0091] For example, if the training data is 100, which is converted into 32-bit numbers, the conversion result is z1= (-1) 2 *1.100100*2 6 The mantissa in z1 contains 6 bits (6 bits after the decimal point), but if it is converted into 8-bit numbers, the data bit number of the mantissa is only 3 bits. In this case, 1.100100 can be scaled by rounding to scale 100100 from 6 bits to 3 bits. Since the value of the fourth bit in 100100 is 1, the third bit is incremented to 1 when scaled to 3 bits. Therefore, the scaling result corresponding to 100100 is 101, and the scaling result corresponding to 1.100100 is 1.101. The conversion result obtained is z1= (-1) 2 *1.101*2 6 If the conversion result is z1= (-1) 2 *1.101000*2 6 The mantissa is 101000, and the value of the fourth bit is 0. When 101000 is scaled to 3 bits, the fourth to sixth bits can be directly discarded, and the scaling result obtained is 101. The scaling result corresponding to 1.101000 is 1.101, and the final 8-bit conversion result is z1= (-1) 2 *1.101*2 6 .

[0092] When it is determined that the first part of the first tensor (the first tensor with a data precision of FP32, the value of the exponent bit) is greater than the preset data threshold corresponding to the exponent bit in the second tensor 7, it indicates that the value of the exponent bit of the first tensor exceeds the value range of the exponent bit in the second tensor. In this case, the first part of the first tensor needs to be truncated. Specifically, 7 is directly taken as the data corresponding to the exponent bit in the second tensor, and the part exceeding the exponent bit can be filled by adjusting the data of the mantissa. Specifically, it can be implemented in the following way:

[0093] Determine the difference between the value corresponding to the first part and the preset data threshold, and quantize the second part based on the difference and the base number, wherein the base number corresponding to the initial data precision is equal to the base number corresponding to the quantization precision.

[0094] Further, the quantization processing of the part based on the difference and the base number comprises:

[0095] Taking the base number as the base and the difference as the exponent, the base number and the difference are operated to generate a first operation result;

[0096] The second part corresponding value is multiplied with the first operation result to generate a second operation result, and the second operation result is taken as data corresponding to the tail number bit in the second tensor.

[0097] Specifically, as described above, the training data can be z1=(-1) n1 *x1*2 y1 The quantized training data can be z3=(-1) n3 *x3*2 y3 In z1 and z3, n1 and n3 are equal, so quantizing z1 to z3 is essentially quantizing x1 to x3 and quantizing y1 to y3, wherein, since the value range of y1 is-126 to 127 and the value range of y3 is-6 to 7, if the value of y1 is greater than 7, the value of y3 can be determined as 7, and the part of y1 exceeding 7 is filled by adjusting x1. For example, the value of y1 is equal to 10, which is greater than 7, so after determining the value of y3 as 7, z3=(-1) n3 *x3*2 7 In order to make the quantized z3 equal or substantially equal to z1, x3 can be adjusted to x1*2 3 , that is, z3=(-1) n3 *(x1*2 3 )*2 7 In z3, the tail number bit is x1*2 3 , and the exponent bit is 7.

[0098] In addition, the value range of the tail number bit of the first tensor with data precision of FP32 is different from that of the second tensor with data precision of FP8, and the data bit number of the tail number bit of the first tensor with data precision of FP32 is 23, while the data bit number of the tail number bit of the second tensor with data precision of FP8 is only 3, so when x1*2 3 is taken as z3, x1*2 3 can be rescaled by rounding, x1*2 3 is reduced from 23 bits to 3 bits, and the scaling result is taken as data corresponding to the tail number bit in z3, so as to generate the second tensor based on the first part and the second part of the first tensor.

[0099] The embodiments of the present specification adjust the different components of the initial data precision first tensor, so as to train the initial task processing model based on the quantized result corresponding to the training data in subsequent training, thereby adjusting the model parameters, that is, training the model by using low-precision training data, which can ensure the accuracy of the training result while improving the task processing performance of the task processing model.

[0100] In step 208, a target task processing model is generated by using the second tensor.

[0101] Specifically, after the first tensor is quantized to generate the second tensor, the target task processing model can be generated according to the second tensor. Specifically, the first tensor of the model parameter in the first tensor can be replaced by the second tensor of the model parameter in the second tensor, the initial task processing model is trained based on the second tensor of the training data in the second tensor, and the second tensor of the model parameter of the initial task processing model is adjusted through forward or backward propagation during the training process to generate the target task processing model.

[0102] In actual application, in the forward propagation process, the weight matrix of the network layer needs to meet the FP8 constraint, and in the backward propagation process, the data is directly transmitted and needs to meet the FP8 constraint.

[0103] The flowchart of the model quantization method provided by the embodiments of the present specification is shown in Figure 3a Figure 3a The schematic diagram of the specific implementation process of quantization is shown in Figure 3b

[0104] Among them, the additional operation introduced by the embodiments of the present specification is divided into three layers: network (network model traversal layer) -> module (model processing layer) -> operator (quantization operation layer). For users, only the model quantization interface of the network layer is needed to quantize (convert) the model.

[0105] In the network model traversal layer, a recursive algorithm is used to traverse the network model to determine the to-be-quantized network layer that needs to insert a quantization node.

[0106] In the model processing layer, a quantization node is inserted in the to-be-quantized network layer that needs to be quantized. Because FP8 is of FP32 type after common multiply-accumulate operation, dequant operator (dequantization operator) is not needed here.

[0107] In the quantization operation layer, as shown in Figure 3a For the tensor (tensor) that needs to be quantized, such as the tensor (input tensor) input into the to-be-quantized network layer and the tensor corresponding to the network parameter (weight) of the to-be-quantized network layer, a quantization operation can be performed on it to obtain a corresponding quantization result. For the to-be-quantized network layer to which a quantization node is added, the network parameter of the to-be-quantized network layer can be replaced by the quantization result, and the output result (output tensor) of the model can be obtained based on the replacement result.

[0108] As shown in Figure 3b For the tensor (tensor) that needs to be quantized, the specific quantization operation includes:​​

[0109] Step 1, split the tensor input into the network layer to be quantized or the tensor corresponding to the network parameter of the network layer to be quantized into a mantissa part (a decimal part) and an exponent part;

[0110] Step 2, for the split data, according to the range of the FP8 data type, the mantissa part is rescaled, and the exponent part is truncated;

[0111] Step 3, round the mantissa part data after scaling;

[0112] Step 4, merge the mantissa part and the exponent part;

[0113] Step 5, output the re-merged data for subsequent model training process;

[0114] Step 6, for the above operation, the gradient is transmitted during back propagation.

[0115] In addition, for the network parameter of the network layer to be quantized, the result obtained by re-merging the mantissa part and the exponent part is the quantization result of the network parameter. After obtaining the result, the network parameter of the network layer to be quantized is replaced based on the result, that is, the network parameter of the network layer to be quantized is replaced with the quantization result. For users, only one interface needs to be called to convert the model. The foregoing three-layer logic is encapsulated in this interface, and users do not need to make other modifications.

[0116] During model training, the relevant data (tensor) is converted into FP8 precision data (tensor) for processing through the foregoing process.

[0117] The QAT scheme proposed in the embodiments of the present specification only needs to make one line of code modification on the user side to implement model quantization.

[0118] In practical applications, the inference demand of neural network models on public clouds is increasing, and there is a demand to optimize the performance of inference tasks. The current methods for customers to optimize inference on public clouds include: running the model on FP16 data precision. However, the nominal computing power of FP8 is twice that of FP16, and directly quantizing the model parameters to FP8 precision results in unacceptable precision loss, and users have low acceptance of complex QAT schemes. Therefore, a simple QAT scheme needs to be provided for users. Based on this, for neural network models that users have acceleration requirements, if the use of FP16 precision acceleration does not meet the requirements, the QAT scheme for FP8 is used to support customers to use FP8 data type to improve inference performance.

[0119] In addition, as mentioned above, the target task processing model obtained through training can be used to process image classification, object detection, image segmentation, natural language processing (speech recognition, character recognition, etc.), or video classification, etc. tasks, and when there are multiple different processing requirements for tasks to be processed, a corresponding target task processing model can be trained for each task to be processed, and the training data used in the process of training the initial task processing model can be data related to each task to be processed. For example, if the task to be processed is an image classification task or an image segmentation task, the training data used to train the initial task processing model can be image data; if the task to be processed is a video classification task, the training data used to train the initial task processing model can be video data; if the task to be processed is a natural language processing task, the training data used to train the initial task processing model can be natural language data, such as audio or text, etc.

[0120] Taking an image classification task as an example, the initial image classification model can be trained first to generate a target image classification model, and a target image to be classified can be input into the target image classification model to process the target image using the target image classification model to obtain a corresponding image classification result.

[0121] Specifically, after receiving a model quantization request submitted by a user for the initial image classification model, the first tensor expressed in the number of bits can be split according to different components of the number of bits according to the initial data precision corresponding to the initial image classification model, to obtain at least two parts of the first tensor, wherein the first tensor includes a first tensor of training data and a first tensor of model parameters, and the training data includes image data; then the at least two parts can be quantized and adjusted according to the quantization precision included in the model quantization request to generate a second tensor corresponding to the number of bits of the quantization precision, wherein the quantization adjustment includes: reducing the value of at least one part and correspondingly increasing the value of at least another part to make the generated second tensor satisfy the initial data precision, i.e., to make the size of the quantized training data equal or substantially equal to the size of the unquantized training data; after the corresponding second tensor is generated through quantization, the second tensor also includes a second tensor of training data and a second tensor of model parameters, and thus the initial image classification model can be trained using the second tensor of training data to adjust the second tensor of model parameters to generate a target image classification model. Then the target image to be classified can be input into the target image classification model to process the target image using the target image classification model to obtain a corresponding image classification result.

[0122] The embodiments of the present specification only take the image classification task as an example to illustrate the specific processing process (the training process and the application process of the model) of the image classification task, and the specific implementation process of other types of to-be-processed tasks is similar to the processing process of the image classification task, which will not be repeated here.

[0123] An embodiment of the present specification splits a first tensor expressed in a number of bits into at least two parts according to different components of the number of bits according to an initial data precision corresponding to an initial task processing model by receiving a model quantization request submitted for the initial task processing model, the first tensor including a first tensor of training data and a first tensor of model parameters, and adjusts the at least two parts respectively according to a quantization precision contained in the model quantization request to generate a second tensor conforming to a number of bits corresponding to the quantization precision, wherein the quantization adjustment includes: reducing the value of at least one of the parts and correspondingly amplifying the value of at least another part to make the second tensor satisfy the initial data precision, and generating a target task processing model by using the second tensor.

[0124] An embodiment of the present specification adjusts the training data and the model parameters of the initial data precision according to the quantization precision, and trains the initial task processing model by using the quantization result corresponding to the training data, so as to realize the adjustment of the model parameters, that is, by using the training data with low precision to train the model, the accuracy of the training result can be guaranteed while the task processing performance of the task processing model is improved.

[0125] The following describes the embodiments of the present specification in conjunction with the accompanying Figure 4 The model quantization method provided in the present specification is taken as an example in the application of the video classification scene to further illustrate the model quantization method. Among them, Figure 4 A processing process flowchart of a model quantization method provided by an embodiment of the present specification is shown, which specifically includes the following steps.

[0126] Step 402, receiving a model quantization request of an initial video classification model submitted by a user by calling a model quantization interface.

[0127] Step 404, determining a to-be-quantized network layer in the initial video classification model, and adding a quantization node of the to-be-quantized network layer.

[0128] Step 406, traversing the network layer contained in the initial video classification model.

[0129] Step 408, in the case where it is detected that the network layer exists the quantization node, splitting different components of the network parameters of the to-be-quantized network layer according to the number of bits of the initial data precision corresponding to the initial video classification model to generate at least two model sub-parameters.

[0130] In step 410, a data structure corresponding to the training data is determined according to the number of bits of the initial data precision corresponding to the initial video classification model, where the data structure is composed of a symbol bit, a mantissa bit, a base, and an exponent bit, and the training data includes video data.

[0131] In step 412, the data corresponding to the exponent bit in the training data is split into first training sub-data, and the data corresponding to the mantissa bit in the training data is split into second training sub-data.

[0132] In step 414, the quantization precision included in the model quantization request is determined, and a preset data threshold corresponding to the exponent bit in the first quantization result is determined according to the number of bits corresponding to the quantization precision.

[0133] In step 416, in a case where the first training sub-data is greater than the preset data threshold, the preset data threshold is determined as the data corresponding to the exponent bit in the first quantization result, a difference between the first training sub-data and the preset data threshold is determined, the base is determined as the base, and the difference is determined as the exponent, and the base and the difference are subjected to power operation to generate a first operation result.

[0134] In step 418, the second training sub-data and the first operation result are subjected to product operation to generate a second operation result, and the second operation result is determined as the data corresponding to the mantissa bit in the first quantization result.

[0135] In step 420, the second training sub-data is subjected to quantization processing according to the first training sub-data and the preset data threshold to generate the data corresponding to the mantissa bit in the first quantization result, and the first quantization result is generated based on the first training sub-data and the second training sub-data.

[0136] In step 422, the initial task processing model is trained according to the first quantization result to adjust the second quantization result, and a target video classification model is generated.

[0137] The embodiments of the present specification adjust the training data and the model parameters of the initial data precision according to the quantization precision, and train the initial video classification model by using the quantization result corresponding to the training data, so as to realize the adjustment of the model parameters, that is, by using the training data with low precision to train the model, the accuracy of the training result can be ensured, and the processing performance of the video classification model is improved.

[0138] Corresponding to the method embodiments described above, the present specification also provides model quantization device embodiments, Figure 5 A structural schematic diagram of a model quantization device provided by an embodiment of the present specification is shown. As shown in the figure, Figure 5 The device includes:

[0139] The receiving module 502 is configured to receive a model quantization request submitted for an initial task processing model;

[0140] The splitting module 504 is configured to split, according to an initial data precision corresponding to the initial task processing model, a first tensor expressed in a number of bits into different components of the number of bits, to obtain at least two parts of the first tensor, the first tensor including a first tensor of training data and a first tensor of model parameters;

[0141] The adjusting module 506 is configured to quantitatively adjust the at least two parts respectively according to a quantization precision contained in the model quantization request, to generate a second tensor conforming to a number of bits corresponding to the quantization precision, wherein the quantitative adjustment includes reducing the value of at least one of the parts and correspondingly amplifying the value of at least another of the parts so that the second tensor satisfies the initial data precision.

[0142] The generating module 508 is configured to generate a target task processing model by using the second tensor.

[0143] Optionally, the initial data precision and the quantization precision correspond to the same data type, and the number of bits corresponding to the initial data precision is greater than a quantization number of bits corresponding to the quantization precision.

[0144] Correspondingly, the splitting module 504 is further configured to:

[0145] determine a data structure corresponding to a first tensor expressed in a number of bits according to an initial data precision corresponding to the initial task processing model, wherein the data structure includes at least two components;

[0146] split the at least two components of the first tensor according to the data structure, to generate at least two parts of the first tensor.

[0147] Optionally, the data structure is composed of a sign bit, a mantissa bit, a base number and an exponent bit.

[0148] Correspondingly, the splitting module 504 is further configured to:

[0149] split data corresponding to the exponent bit in the first tensor into a first part, and split data corresponding to the mantissa bit in the first tensor into a second part.

[0150] Optionally, a data structure of the second tensor is composed of a sign bit, a mantissa bit, a base number and an exponent bit.

[0151] Correspondingly, the adjusting module 506 is further configured to:

[0152] determining a preset data threshold corresponding to the index bits in the second tensor according to the bit number corresponding to the quantization precision;

[0153] determining whether the value corresponding to the first part is greater than the preset data threshold;

[0154] if yes, taking the preset data threshold as the data corresponding to the index bits in the second tensor;

[0155] performing quantization processing on the second part according to the first part and the preset data threshold, to generate the data corresponding to the mantissa bits in the second tensor.

[0156] Optionally, the adjusting module 506 is further configured to:

[0157] if the value corresponding to the first part is less than or equal to the preset data threshold, taking the first part as the data corresponding to the index bits in the second tensor, and taking the second part as the data corresponding to the mantissa bits in the second tensor, to generate the second tensor based on the first part and the second part.

[0158] Optionally, the adjusting module 506 is further configured to:

[0159] determining a difference between the value corresponding to the first part and the preset data threshold, and performing quantization processing on the second part based on the difference and the base, wherein the base corresponding to the initial data precision is equal to the base corresponding to the quantization precision.

[0160] Optionally, the adjusting module 506 is further configured to:

[0161] performing power operation on the base and the difference, taking the base as the base number and the difference as the index, to generate a first operation result;

[0162] performing product operation on the value corresponding to the second part and the first operation result, to generate a second operation result, and taking the second operation result as the data corresponding to the mantissa bits in the second tensor.

[0163] Optionally, the generating module 508 is further configured to:

[0164] training the initial task processing model according to the second tensor of the training data, to adjust the second tensor of the model parameter, and generate a target task processing model.

[0165] Optionally, the receiving module 502 is further configured to:

[0166] receiving a model quantization request of the initial task processing model submitted by a user through calling a model quantization interface;

[0167] Correspondingly, the apparatus further comprises a processing module configured to:

[0168] traverse network layers contained in the initial task processing model in response to the model quantization request;

[0169] determine a network layer to be quantized according to the traversal result, and add a corresponding quantization node to the network layer to be quantized.

[0170] Optionally, the splitting module 504 is further configured to:

[0171] in a case where it is detected that the network layer to be quantized has a quantization node, determine a network parameter of the network layer to be quantized, and determine a first tensor of the network parameter, wherein the first tensor is expressed in terms of a number of bits;

[0172] split the first tensor of the network parameter according to different components of the number of bits according to an initial data precision corresponding to the initial task processing model.

[0173] Optionally, the data type corresponding to the initial data precision and the quantization precision is floating point type, the number of bits corresponding to the initial data precision is equal to 32, and the quantization number of bits corresponding to the quantization precision is equal to 8.

[0174] One embodiment of the present specification receives a model quantization request submitted for an initial task processing model through a receiving module, splits a first tensor expressed in terms of a number of bits according to different components of the number of bits according to an initial data precision corresponding to the initial task processing model through a splitting module, to obtain at least two parts of the first tensor, the first tensor including a first tensor of training data and a first tensor of model parameters, adjusts the at least two parts according to a quantization precision contained in the model quantization request through an adjusting module to generate a second tensor conforming to a number of bits corresponding to the quantization precision, wherein the quantization adjustment includes reducing the value of at least one of the parts and correspondingly increasing the value of at least another part to make the second tensor satisfy the initial data precision, and generates a target task processing model using the second tensor through a generating module.

[0175] The embodiment of the present specification adjusts the first tensor of the initial data precision according to the quantization precision, and trains the initial task processing model using the quantization result corresponding to the training data, so as to realize the adjustment of the model parameters, that is, by using low-precision training data to train the model, the accuracy of the training result can be ensured while the task processing performance of the task processing model is improved.

[0176] The above is a schematic scheme of the model quantization apparatus of the embodiment. It should be noted that the technical scheme of the model quantization apparatus and the technical scheme of the model quantization method described above belong to the same concept, and the details of the technical scheme of the model quantization apparatus that are not described in detail can be referred to the description of the technical scheme of the model quantization method.

[0177] Figure 6 A structural block diagram of a computing device 600 is shown, according to one embodiment of the present specification. The components of the computing device 600 include, but are not limited to, a memory 610 and a processor 620. The processor 620 is connected to the memory 610 through a bus 630, and a database 650 is used to save data.

[0178] The computing device 600 also includes an access device 640, which enables the computing device 600 to communicate via one or more networks 660. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 640 can include one or more of any type of network interface (e.g., network interface card (NIC)), wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0179] In one embodiment of the present specification, the above-mentioned components of the computing device 600 and other components not shown in the Figure 6 may be connected to each other, for example, through a bus. It should be understood that Figure 6 The structural block diagram of the computing device shown is only for the purpose of example, and is not a limitation on the scope of the present specification. Other components can be added or replaced as needed by those skilled in the art.

[0180] The computing device 600 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a PC. The computing device 600 can also be a mobile or stationary server.

[0181] The processor 620 is configured to execute computer-executable instructions, which, when executed by the processor, implement the steps of the model quantization method described above.

[0182] The above is a schematic scheme of the computing device of the embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the model quantization method described above belong to the same concept, and the details of the technical scheme of the computing device that are not described in detail can be referred to the description of the technical scheme of the model quantization method.

[0183] An embodiment of the present specification further provides a computer readable storage medium, which stores computer executable instructions. The computer executable instructions are executed by a processor to implement the steps of the model quantization method.

[0184] The above is a schematic scheme of the computer readable storage medium of the embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the model quantization method described above belong to the same concept, and the details of the technical scheme of the storage medium that are not described in detail can be referred to the description of the technical scheme of the model quantization method.

[0185] An embodiment of the present specification further provides a computer program, which causes a computer to execute the steps of the model quantization method when the computer program is executed in the computer.

[0186] The above is a schematic scheme of the computer program of the embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the model quantization method described above belong to the same concept, and the details of the technical scheme of the computer program that are not described in detail can be referred to the description of the technical scheme of the model quantization method.

[0187] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims can be performed in a different order than the order described in the embodiments and still achieve desirable results. In addition, the processes depicted in the figures do not necessarily require the particular order shown or sequential order in order to achieve the desired results. In some implementations, multitasking and parallel processing can be advantageous or necessary.

[0188] The computer readable medium can include any entity or apparatus capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, software distribution medium, etc. It should be noted that the computer readable medium can include appropriate additions or subtractions according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0189] It should be noted that for the foregoing method embodiments, the descriptions are each simply a combination of a series of acts for the sake of brevity, but those skilled in the art should know that the present application is not limited by the order of the acts described, because some steps can be performed in other orders or at the same time in accordance with the present application. In addition, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the acts and modules involved are not necessarily essential to the present application.

[0190] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0191] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The alternative embodiments do not describe all the details and limit the present application to the specific embodiments described. Obviously, according to the content of the present application, many modifications and changes can be made. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can well understand and use the present application. The present application is limited only by the claims and their full scope and equivalents.

Claims

1. A model quantization method applied to a server, comprising: receiving a model quantization request submitted for an initial task processing model, wherein the model quantization request is sent by a user to the server through a model quantization interface of the server, and the initial task processing model comprises an initial image processing model; splitting a first tensor expressed in a number of bits into at least two parts according to different components of the number of bits based on initial data precision corresponding to the initial task processing model, wherein the first tensor comprises a first tensor of training data and a first tensor of model parameters, and the training data comprises image data; adjusting the at least two parts respectively according to a quantization precision contained in the model quantization request to generate a second tensor conforming to a number of bits corresponding to the quantization precision, wherein the adjusting comprises: determining a preset data threshold corresponding to an exponent bit in the second tensor according to the number of bits corresponding to the quantization precision, and reducing the value of at least one of the parts and correspondingly amplifying the value of at least another part according to the preset data threshold so that the second tensor satisfies the initial data precision; generating a target task processing model using the second tensor, wherein the target task processing model comprises a target image processing model.

2. The model quantization method of claim 1, wherein the initial data precision and the quantization precision correspond to the same data type, and the number of bits corresponding to the initial data precision is greater than the quantization number of bits corresponding to the quantization precision; correspondingly, the splitting the first tensor expressed in the number of bits into the at least two parts according to the different components of the number of bits based on the initial data precision corresponding to the initial task processing model comprises: determining a data structure corresponding to the first tensor expressed in the number of bits based on the initial data precision corresponding to the initial task processing model, wherein the data structure comprises at least two components; splitting the at least two components of the first tensor according to the data structure to generate the at least two parts of the first tensor.

3. The model quantization method of claim 2, wherein the data structure is composed of a sign bit, a mantissa bit, a base, and an exponent bit; correspondingly, the splitting the at least two components of the first tensor according to the data structure to generate the at least two parts of the first tensor comprises: splitting data corresponding to the exponent bit in the first tensor into a first part, and splitting data corresponding to the mantissa bit in the first tensor into a second part.

4. The model quantization method of claim 3, wherein a data structure of the second tensor is composed of a sign bit, a mantissa bit, a base, and an exponent bit; correspondingly, the adjusting the at least two parts of the first tensor respectively according to the quantization precision contained in the model quantization request to generate the second tensor conforming to the number of bits corresponding to the quantization precision comprises: determining a preset data threshold corresponding to the exponent bit in the second tensor according to the number of bits corresponding to the quantization precision. determining whether the value corresponding to the first part is greater than the preset data threshold value; if yes, taking the preset data threshold value as data corresponding to an index bit in the second tensor; quantizing the second part according to the first part and the preset data threshold value to generate data corresponding to a mantissa bit in the second tensor.

5. The model quantization method of claim 4, if the value corresponding to the first part is less than or equal to the preset data threshold value, the method further comprises: taking the first part as data corresponding to an index bit in the second tensor and taking the second part as data corresponding to a mantissa bit in the second tensor to generate the second tensor based on the first part and the second part.

6. The model quantization method of claim 5, the quantizing the second part according to the first part and the preset data threshold value comprises: determining a difference between the value corresponding to the first part and the preset data threshold value, and quantizing the second part based on the difference and the base, wherein the base corresponding to the initial data precision is equal to the base corresponding to the quantization precision.

7. The model quantization method of claim 6, the quantizing the part based on the difference and the base comprises: performing power operation on the base and the difference by taking the base as a base number and taking the difference as an index to generate a first operation result; performing product operation on the value corresponding to the second part and the first operation result to generate a second operation result, and taking the second operation result as data corresponding to a mantissa bit in the second tensor.

8. The model quantization method of claim 1, the generating a target task processing model by using the second tensor comprises: training the initial task processing model according to the second tensor of the training data to adjust the second tensor of the model parameter and generate the target task processing model.

9. The model quantization method of claim 1, the receiving a model quantization request submitted for an initial task processing model comprises: receiving a model quantization request of an initial task processing model submitted by a user by calling a model quantization interface; correspondingly, the method further comprises: traversing network layers contained in the initial task processing model in response to the model quantization request; determining a network layer to be quantized according to a traversal result, and adding a corresponding quantization node to the network layer to be quantized.

10. The model quantization method of claim 9, the splitting a first tensor expressed in a number of bits into different component parts according to an initial data precision corresponding to the initial task processing model comprises: in a case where it is detected that the network layer to be quantized has a quantization node, determining a network parameter of the network layer to be quantized and determining a first tensor of the network parameter, wherein the first tensor is expressed in the number of bits; splitting the first tensor of the network parameter into different component parts according to the initial data precision corresponding to the initial task processing model.

11. The model quantization method of claim 1 or 2, wherein the initial data precision corresponds to a floating point type, and the initial data precision corresponds to a bit number equal to 32, and the quantization precision corresponds to a quantization bit number equal to 8.

12. A model quantization apparatus applied to a server, comprising: a receiving module configured to receive a model quantization request submitted for an initial task processing model, wherein the model quantization request is submitted by a user through a client to the server by invoking a model quantization interface of the server, and the initial task processing model comprises an initial image processing model; a splitting module configured to split a first tensor expressed in a bit number into different components according to an initial data precision corresponding to the initial task processing model, to obtain at least two parts of the first tensor, wherein the first tensor comprises a first tensor of training data and a first tensor of model parameters, and the training data comprises image data; an adjusting module configured to quantitatively adjust the at least two parts according to a quantization precision contained in the model quantization request, to generate a second tensor conforming to a bit number corresponding to the quantization precision, wherein the quantitatively adjusting comprises: determining a preset data threshold corresponding to an index bit in the second tensor according to the bit number corresponding to the quantization precision, and reducing a value of at least one of the parts and correspondingly amplifying a value of at least another one of the parts according to the preset data threshold, so that the second tensor satisfies the initial data precision; a generating module configured to generate a target task processing model by using the second tensor, wherein the target task processing model comprises a target image processing model.

13. A computing device, comprising: a memory and a processor; the memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions, and the computer executable instructions, when executed by the processor, implement steps of the model quantization method of any one of claims 1 to 11.

14. A computer readable storage medium storing computer executable instructions, and the computer executable instructions, when executed by a processor, implement steps of the model quantization method of any one of claims 1 to 11.

15. A computer program product, characterised in that, computer instructions, and the computer instructions, when executed by a processor, implement steps of the model quantization method of any one of claims 1 to 11.

Images