Model quantitative adjustment method and device, computing equipment and storage medium

Abstract

The invention discloses a model quantitative adjustment method and device, computer equipment and a storage medium, and belongs to the technical field of computers. In the method, on the basis of training data of a target model, a pseudo-quantization model constructed based on features and hyper-parameters of the target model is subjected to multiple times of iterative training, and for each iteration, if the number of weights capable of being updated by any quantization layer in the pseudo-quantization model based on a first learning rate is smaller than a preset value, the pseudo-quantization model is subjected to iterative training. And if not, adjusting the first learning rate of the quantization layer until the number of the weights updated by the adjusted learning rate is greater than the preset value. According to the method, adaptive gradient compensation is realized by performing adaptive scaling on the first learning rate, the problems of gradient mismatch and gradient disappearance caused by quantization are solved, and a certain number of weights can be ensured to be updated in a fine adjustment stage with a relatively low learning rate, so that the quantization precision loss is reduced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model quantization adjustment method, device, computing equipment and storage medium. Background Art

[0002] In recent years, neural network models have gradually shown a trend of structural complexity, resulting in an increase in the weight, calculation amount and size of neural network models, which in turn leads to an increase in reasoning time. In scenarios such as intelligent robots or autonomous driving that require lightweight models and are sensitive to latency, it is particularly important to reduce the memory usage of neural network models and shorten the model reasoning time. For the above scenarios, quantizing the model is an effective way to reduce the model size and reduce the reasoning time. Model quantization can convert floating-point models into fixed-point integer calculation models, thereby achieving quantization compression and reasoning acceleration of the model. However, model quantization will reduce the reasoning accuracy of the model. Therefore, it is necessary to use the training quantization method to fine-tune the quantized model to restore the accuracy. Specifically, a pseudo-quantized model is constructed based on the target model, the pseudo-quantized model is trained to obtain a trained pseudo-quantized model, and finally the trained pseudo-quantized model is deployed on the hardware to achieve true quantization reasoning of the model, thereby effectively reducing the reasoning accuracy loss caused by quantization.

[0003] In the related technology, the quantization layer in the target model is identified, and a pseudo-quantization operator is inserted before the quantization layer to construct an initial pseudo-quantization model. The initial pseudo-quantization model is fine-tuned and trained. During the fine-tuning training, the rounding calculation process of the deployed hardware is simulated, and a back-propagation gradient calculation with differentiable characteristics is constructed to learn the precision loss caused by quantization rounding. Based on the precision loss of the pseudo-quantization model during the training process, the weights and quantization parameters of the pseudo-quantization model are fine-tuned to obtain the trained pseudo-quantization model.

[0004] In the above method, the pseudo-quantization operator performs rounding operations on the input and weights of the quantization layer during the training process, resulting in non-differentiable error back propagation during the training phase. The use of approximate gradient calculation will lead to gradient mismatch or gradient vanishing, making the training process of the pseudo-quantization model have poor convergence, and the weights of the pseudo-quantization model cannot be effectively updated, thereby failing to effectively reduce the accuracy loss caused by quantization. Summary of the invention

[0005] The embodiments of the present application provide a model quantization adjustment method, apparatus, computing device and storage medium, which can alleviate gradient mismatch and gradient vanishing, effectively update the weights of the pseudo-quantization model, and thus effectively reduce the accuracy loss caused by quantization. The technical solution is as follows.

[0006] In a first aspect, a model quantization adjustment method is provided, the method comprising:

[0007] A pseudo-quantization model is constructed according to the characteristics and hyperparameters of the target model, and the training data of the target model is input into the pseudo-quantization model; the pseudo-quantization model is iteratively trained for multiple times to obtain a trained pseudo-quantization model, wherein the first training of the first quantization layer among the multiple quantization layers in the pseudo-quantization model includes: determining the number of weights that can be updated based on a first learning rate of the first quantization layer; when the number of weights that can be updated based on the first learning rate is less than a preset value, adjusting the first learning rate and continuing to determine the number of weights that can be updated until the determined number of weights that can be updated is greater than or equal to the preset value; updating the weights based on the adjusted first learning rate; wherein I is an integer greater than or equal to 1; and outputting the trained pseudo-quantization model as a quantization model.

[0008] Among them, the target model is an unquantized floating-point model, that is, a pre-trained model obtained by pre-training based on training data. The characteristics of the target model refer to the characteristics of the target model's occupation of computing resources and memory resources, for example, the characteristics of the target model are computationally intensive, or the characteristics of the target model are memory access intensive, etc. The hyperparameter indicates the quantization method corresponding to the pseudo-quantization model, and the hyperparameter includes a quantization mode and a quantization bit width, etc., wherein the quantization mode includes symmetric quantization or asymmetric quantization, etc. The pseudo-quantization model includes multiple quantization layers; the first quantization layer is any one of the multiple quantization layers, and the data used for training the first quantization layer is data processed according to the quantization parameters corresponding to the first quantization layer. The quantization layer includes a convolutional layer and a linear layer (that is, a fully connected layer). Outputting the trained pseudo-quantization model as a quantization model means: removing the quantization parameters in the trained pseudo-quantization model, and the resulting model is the quantization model.

[0009] In the above method, adaptive gradient compensation is achieved by adaptively scaling the learning rate to alleviate the gradient mismatch and gradient vanishing problems caused by quantization. In the fine-tuning stage with a small learning rate, a certain number of weights can be updated, thereby reducing the model accuracy loss caused by quantization.

[0010] Optionally, after adjusting the first learning rate, continuing to determine the number of weights that can be updated until the number of updated weights is greater than or equal to a preset value includes:

[0011] Adjust the first learning rate of the first quantization layer to the second learning rate; determine the number of weights that can be updated based on the second learning rate, and when the number of weights that can be updated based on the second learning rate is less than a preset value, continue to adjust the second learning rate until the number of weights that can be updated based on the adjusted Nth learning rate is greater than or equal to the preset value, where N is an integer greater than 1.

[0012] The first learning rate is the initial learning rate of the first quantization layer, that is, the learning rate that has not been amplified; the Nth learning rate refers to the learning rate obtained after adjusting the first learning rate N-1 times.

[0013] Optionally, the method further comprises:

[0014] A target learning rate is determined according to the Nth learning rate, where the target learning rate is the learning rate of the first quantization layer in the Ith training.

[0015] Among them, the target learning rate is the learning rate used by the first quantization layer to actually update the weights during the I training.

[0016] Optionally, determining the target learning rate according to the Nth learning rate includes:

[0017] The target learning rate is determined according to the Nth learning rate and the N-1th learning rate, where the N-1th learning rate is a previous learning rate of the Nth learning rate, and the target learning rate is the optimal learning rate between the Nth learning rate and the N-1th learning rate.

[0018] The optimal learning rate means that the number of weights that can be updated based on the target learning rate is the largest among all learning rates between the Nth learning rate and the N-1th learning rate.

[0019] Optionally, the method further comprises:

[0020] When the sum of the number of weights and quantization parameters that can be updated based on the first learning rate is less than a preset value, the sum of the number of weights and quantization parameters that can be updated continues to be determined after adjusting the first learning rate until the sum of the number of weights and quantization parameters that can be updated is greater than or equal to the preset value.

[0021] The quantization parameter is used to adjust the accuracy of mapping the data for training the first quantization layer to a preset quantization bit width, and the preset quantization bit width is determined based on the hyperparameter. The quantization parameter performs pseudo-quantization processing on the data for training the first quantization layer, and the pseudo-quantization process includes: first quantizing the weights (weight) and inputs (input) of the first quantization layer from floating point to integer (float→int), and then dequantizing the quantized weights and inputs from integer to floating point (int→float).

[0022] Optionally, a pseudo-quantization model is constructed according to the characteristics and hyperparameters of the target model, including:

[0023] According to the characteristics of the target model, the target model is approximatedly transformed, and the approximate transformation refers to transforming a subgraph or operator in the target model; the quantization layer in the target model after the approximate transformation is identified; a pseudo-quantization operator corresponding to a hyperparameter is inserted before each quantization layer to obtain a pseudo-quantization model, and the pseudo-quantization operator includes a quantization parameter.

[0024] The characteristics of the target model refer to the characteristics of the target model's occupation of computing resources and memory resources, such as the characteristics of the target model being computationally intensive, or the characteristics of the target model being memory-access intensive, etc. According to the characteristics of the target model, approximate transformation of the target model includes: replacing memory-access intensive operators and computationally intensive operators in the target model, and pruning the target model. Memory-access intensive operators refer to operators with small computational complexity but a large number of data transfer operations, such as input / output (I / O) intensive operators and binary operators with low computational complexity. Computationally intensive operators refer to operators with large and concentrated computational and logical judgments, such as convolutional layers.

[0025] In the above method, the target model is approximately transformed to prune the model or replace part of the subgraph or operator in the model with a subgraph or operator with higher computational efficiency, which can improve the reasoning speed and achieve reasoning acceleration.

[0026] In the above method, according to the characteristics of the target model, the target model is approximately transformed to prune the model or replace part of the subgraphs or operators in the model with subgraphs or operators with higher computational efficiency. This can reasonably consider the characteristics of the target model's occupancy of computing resources and memory resources, thereby helping to improve the reasoning speed and achieve reasoning acceleration.

[0027] Optionally, performing an approximate transformation on the target model includes at least one of the following: pruning the target model; replacing an activation layer in the target model; replacing an operator in the target model.

[0028] In a second aspect, a model quantization adjustment device is provided, which includes at least one functional module, and the at least one functional module is used to execute the model quantization adjustment method provided by the first aspect or any possible implementation method of the first aspect.

[0029] In a third aspect, a computing device is provided, the computing device comprising a processor and a memory;

[0030] The processor is used to execute instructions stored in the memory so that the computing device executes the model quantization adjustment method provided by the aforementioned first aspect or any possible implementation of the first aspect.

[0031] In a fourth aspect, a computer program product comprising instructions is provided. When the instructions are executed by a computing device, the computing device executes the model quantization adjustment method provided in the aforementioned first aspect or any possible implementation of the first aspect.

[0032] In a fifth aspect, a computer-readable storage medium is provided, which includes computer program instructions. When the computer program instructions are executed by a computing device, the computing device executes the model quantization adjustment method provided in the first aspect or any possible implementation of the first aspect.

[0033] Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of an implementation environment of a model quantization adjustment method provided in an embodiment of the present application;

[0035] Figure 2 A schematic diagram of a system architecture for applying a model quantization adjustment method provided in an embodiment of the present application;

[0036] Figure 3 It is a flowchart of a method for constructing a pseudo-quantization model provided in an embodiment of the present application;

[0037] Figure 4 is an example diagram of an approximate transformation method provided in an embodiment of the present application;

[0038] Figure 5 A schematic diagram of a pseudo quantization process provided in an embodiment of the present application;

[0039] Figure 6 It is a flowchart of a method for constructing a pseudo-quantization model provided in an embodiment of the present application;

[0040] Figure 7 It is a flowchart of a training method of a pseudo-quantization model provided in an embodiment of the present application;

[0041] Figure 8 It is a flow chart of a gradient compensation strategy provided in an embodiment of the present application;

[0042] Fig. 9 is a schematic diagram of a gradient compensation effect provided by an embodiment of the present application;

[0043] Fig.10 is a schematic diagram of a model quantization adjustment method provided in an embodiment of the present application;

[0044] Fig.11It is a structural schematic diagram of a training device for a pseudo-quantization model provided in an embodiment of the present application;

[0045] Fig.12 It is a structural diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0047] In order to facilitate understanding of the contents of the embodiments of the present application, the relevant technical terms involved in the embodiments of the present application are first explained below.

[0048] Model quantization: Convert a floating-point model to a model that uses fixed-point integer calculations. A floating-point model refers to a model whose weights or outputs are in single-precision floating-point format (float32, FP32) or half-precision floating-point format (float16, FP16). FP32 occupies 32 bits of memory, and FP16 occupies 16 bits of memory. A fixed-point integer calculation model refers to a model whose weights or outputs are in fixed-point integer format, such as 8-bit integers and 4-bit integers. Since the fixed-point integer format occupies less memory space and numerical representation accuracy than FP32 and FP16, model quantization can achieve model compression, but it will also cause a certain loss of accuracy.

[0049] Linear quantization: A method of model quantization, which refers to converting floating-point data into fixed-point data in the form of linear imaging. The formula for linear quantization is: Among them, R represents the input floating-point data, Q represents the quantized fixed-point data, Z represents the offset, and S represents the scaling factor. Z and S are also called quantization parameters. The solution of S is: Among them, R max and R min Respectively represent the maximum and minimum values ​​of the input floating-point data, Q max and Q min They represent the maximum and minimum values ​​of the quantized fixed-point data respectively. The solution for z is: z = Q max -R min ÷S. Linear quantization can be divided into symmetric quantization and asymmetric quantization.

[0050] Symmetric quantization: A type of linear quantization, it refers to mapping floating-point data to the range of [-128, 127]. Symmetric quantization needs to ensure that the zero point in the floating-point data still corresponds to the zero point in the interval [-128, 127] after mapping, that is, when Z=0 and Q=0, R=0.

[0051] Asymmetric quantization: A type of linear quantization, which refers to mapping floating-point data to the range of [0, 255].

[0052] Quantization aware training: also known as training aware quantization (quantization aware training, QAT) The process of quantization aware training includes: identifying the quantization layer in the target model, inserting pseudo quantization operators before each quantization layer to construct a pseudo quantization model, and then training the pseudo quantization model so that the trained pseudo quantization model can adapt to the quantization behavior of the model on the hardware to be deployed, wherein the pseudo quantization model can be trained in a fine-tuning manner, and the learning rate used in the training process is relatively small. After completing the quantization aware training, that is, after obtaining the trained pseudo quantization model, the trained pseudo quantization model can be deployed on the hardware (that is, the inference platform) for execution, thereby achieving quantization compression and inference acceleration of the model.

[0053] Approximate transformation: For subgraphs or operators in the model, use subgraphs or operators with higher computational efficiency for approximate replacement to achieve computational acceleration. Approximate transformation of the model will result in a certain degree of accuracy loss.

[0054] Gradient mismatch: After the model is quantized, due to discrete operations such as rounding and clipping, the back propagation in the training phase is not differentiable. Using approximate gradient back propagation may deviate from the direction of steepest descent, which may cause quantization-aware training to fail to converge;

[0055] Gradient disappearance: After the model is quantized, due to discrete operations such as rounding and clipping, the back propagation in the training phase is not differentiable. Usually the change of floating point value is not enough to cross the threshold of integer value change, which can easily cause the deep gradient to be zero.

[0056] The implementation environment of the embodiments of the present application is introduced below.

[0057] Figure 1 Schematic diagram of the implementation environment of a model quantization adjustment method provided in the embodiment of the present application. Figure 1 As shown, the implementation environment includes a computing device 101 and an embedded device 102. The computing device 101 and the embedded device 102 are communicatively connected via a wired network or a wireless network.

[0058] The computing device 101 can be at least one of a desktop computer, a laptop, a wireless terminal, and a laptop portable computer, etc. It can be an independent physical server, or a server cluster composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, distribution networks (content delivery networks, CDN), and basic cloud computing services such as big data and artificial intelligence platforms. Exemplarily, the computing device 101 provides a target model acquisition function, a pseudo-quantization model construction function, a pseudo-quantization model training function, and a model deployment function. Among them, the pseudo-quantization model construction function refers to the computing device 101 being able to identify the quantization layer in the target model, inserting a pseudo-quantization operator before each quantization layer to construct a pseudo-quantization model. In some embodiments, the computing device 101 first performs an approximate transformation on the target model based on the characteristics of the target model, and then constructs a pseudo-quantization model based on the model after the approximate transformation. The training function of the pseudo-quantization model refers to the computing device 101 being able to perform multiple iterations of training on the constructed pseudo-quantization model based on the training data set to obtain a trained pseudo-quantization model. The model deployment function means that the computing device 101 can output the trained pseudo-quantization model as a quantization model, and the quantization model can be deployed on the embedded device 102 corresponding to the business. In some embodiments, the computing device 101 deploys the quantization model based on the output of the trained pseudo-quantization model on the embedded device 102 corresponding to the business by accessing the model deployment service provided by the inference platform.

[0059] Figure 2 A schematic diagram of a system architecture for a model quantization adjustment method application provided in an embodiment of the present application, such as Figure 2 As shown, the system architecture includes four functional modules, wherein the first functional module is used to obtain the target model; the second functional module is used to construct the pseudo-quantization model; the third functional module is used to train the pseudo-quantization model; and the fourth functional module is used to output and deploy the quantization model. The first functional module is used to obtain the training data set and training script of the target model; the second functional module is used to perform approximate transformation and insert pseudo-quantization operators; and the third functional module is used to perform gradient compensation. The above four functional modules correspond to the four functions of the computing device 101, respectively.

[0060] It should be noted that the computing device 101 may generally refer to one of multiple computing devices. Those skilled in the art may know that the computing device 101 may be a single computing device 101 or a computing device cluster composed of multiple computing devices 101. The embodiments of the present application do not limit this.

[0061] The embedded device 102 refers to a computer system applied to a specific task. Compared with a general computer processing system, the storage space of the embedded device is small and cannot realize a large-capacity storage function. The embedded device can be an embedded microprocessor, an embedded microcontroller, an embedded processor or an embedded system on chip, etc. The embedded device 102 can be embedded in other devices, such as: intelligent robots, autonomous driving vehicles, virtual reality terminals (virtualreality, VR), augmented reality terminals (augmented reality, AR), mobile phones and smart home appliances. Exemplarily, the embedded device 102 can deploy a quantitative model, so that the reasoning task corresponding to the quantitative model can be performed on the embedded device 102, wherein the embedded device can deploy a lightweight model and is sensitive to latency. For example, in the application scenario of intelligent robots, intelligent robots such as inspection robots and interactive robots have high requirements for model accuracy and authentication feedback latency; in the scenarios of autonomous driving and mobile robots, the model needs to be able to identify multiple targets with small memory, high accuracy and low latency.

[0062] In some embodiments, the wireless network or wired network uses standard communication technology and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or any combination of a virtual private network. In some embodiments, peer-to-peer (P2P) communication is implemented between the computing device 101 and the embedded device 102 based on a remote procedure call protocol (RPC). In some embodiments, the computing device 101 and the embedded device 102 use technologies and / or formats including hypertext markup language (HTML), extensible markup language (XML), etc. to represent data blocks exchanged through the network. In addition, conventional encryption technologies such as secure socket layer (SSL), transport layer security (TLS), virtual private network (VPN), Internet protocol security (IPsec) can be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can be used to replace or supplement the above data communication technologies.

[0063] The embodiment of the present application provides a model quantization adjustment method. In this method, based on the training data of the target model, a pseudo-quantization model constructed based on the characteristics and hyperparameters of the target model is iteratively trained multiple times, wherein, for each iteration, if the number of weights that can be updated by any quantization layer in the pseudo-quantization model based on the first learning rate is less than a preset value, the first learning rate of the quantization layer is adjusted until the number of weights updated by the adjusted learning rate is greater than the preset value. In the above method, adaptive gradient compensation is achieved by adaptively scaling the first learning rate to solve the gradient mismatch and gradient vanishing problems caused by quantization. In the fine-tuning stage with a small learning rate, a certain number of weights can be guaranteed to be updated, thereby reducing the loss of quantization accuracy.

[0064] The following first introduces a method for constructing a pseudo-quantization model provided in an embodiment of the present application. Figure 3 is a flow chart of a method for constructing a pseudo-quantization model provided in an embodiment of the present application, such as Figure 3 As shown, taking the method executed by a computing device as an example, the process includes the following steps 301 to 303.

[0065] 301. The computing device performs an approximate transformation on the target model based on the characteristics of the target model. The approximate transformation refers to transforming a subgraph or operator in the target model that complies with the preset approximate transformation rule.

[0066] Among them, the target model is an unquantized floating-point model, that is, a pre-trained model obtained by pre-training based on the training data set.

[0067] The characteristics of the target model refer to the characteristics of the target model's occupation of computing resources and memory resources, such as the characteristics of the target model being computationally intensive or memory access intensive, etc. The hyperparameter indicates the quantization method corresponding to the pseudo-quantization model, and the hyperparameter includes a quantization mode and a quantization bit width, etc., wherein the quantization mode includes symmetric quantization or asymmetric quantization, etc.

[0068] Among them, the computing device performs approximate transformation of the target model according to the characteristics of the target model, including: replacing memory-intensive operators and computation-intensive operators in the target model, and pruning the target model. Among them, memory-intensive operators refer to operators with small computational complexity but a large number of data transmission operations, such as input / output (I / O) intensive operators and binary operators with low computational complexity. Computation-intensive operators refer to operators with large and concentrated computational and logical judgment, such as convolutional layers.

[0069] In some embodiments, the target model is also approximatedly transformed based on the hardware on which the target model is to be deployed, wherein the computing device replaces a subgraph or operator in the target model with a subgraph or operator with higher computational efficiency on the hardware on which the target model is to be deployed. For example, when the hardware is a neural network processing unit (NPU), the activation layer in the target model is replaced from a Gaussian error linear unit (Gelu) to a fast Gaussian error linear unit (FastGelu) to increase the computing speed.

[0070] The computing device performs approximate transformation on the target model in at least one of the following ways: pruning the target model; replacing the activation layer in the target model; replacing the operator in the target model. Figure 4 Taking as an example, the method of approximate transformation is illustrated. Figure 4 is an example diagram of an approximate transformation method provided in an embodiment of the present application. Figure 4 It includes three sub-example diagrams (a), (b) and (c), and a box in each sub-example diagram represents a sub-graph or operator of the model. Among them, (a) shows that the convolution layer 401 of the model is pruned to reduce the number of weights of the convolution layer, wherein the specific method and process of pruning will be introduced in the subsequent step 303 and will not be repeated here; (b) shows that the activation layer of the model is replaced by a rectified linear unit (rectified linear unit (Relu6) with an upper bound of 6); (c) shows that the activation layer in the model is replaced by Gelu with FastGelu.

[0071] It should be noted that the above Figure 4 The approximate transformation method illustrated is only exemplary, and the approximate transformation method can be determined according to actual needs, model structure or hardware performance, and the present application embodiment does not limit this. Figure 4 In the process, the input of the convolutional layer is first quantized (quantized), and then the output of the convolutional layer is dequantized (dequantized). This process is the hardware quantization process of the model, that is, the actual quantization process. Figure 5 The figure shows the simulation of the quantization process by the computing device, which is also a pseudo quantization process. Figure 5 A schematic diagram of a pseudo quantization process provided in an embodiment of the present application is shown in FIG. Figure 5 As shown in the figure, the pseudo-quantization operator first quantizes the weights and inputs of the convolution layer from floating point to integer (float→int), and then dequantizes the quantized weights and inputs from integer to floating point (int→float). This process is also called pseudo-quantization. The convolution layer processes the pseudo-quantized weights and inputs, and the output is floating point.

[0072] It should be noted that step 301 is an optional step. In some embodiments, step 301 is not performed, and the following steps 302 and 303 are directly performed.

[0073] 302. The computing device identifies a quantization layer in the target model after the approximate transformation.

[0074] Among them, the quantization layer in the target model includes a convolutional layer and a linear layer (that is, a fully connected layer). The computing device identifies the convolutional layer and the linear layer in the target model based on the training script of the target model in the pre-training stage. Among them, the training script includes the processing method of the data set, the model structure definition, the loss function definition, the optimizer selection, the training process setting and the verification process setting, etc., wherein the training process setting includes the training cycle setting and the learning rate setting, etc. In some embodiments, the learning rates corresponding to different layers in the target model are the same, and in other embodiments, the learning rates corresponding to different layers in the target model are different, which is not limited in the embodiments of the present application.

[0075] 303. The computing device inserts a pseudo quantization operator corresponding to the hyperparameter before each quantization layer to obtain a pseudo quantization model, where the pseudo quantization operator includes the quantization parameter.

[0076] Each quantization layer corresponds to a pseudo-quantization operator, which is used to perform pseudo-quantization operations on the input and weight of the quantization layer corresponding to the pseudo-quantization operator, wherein the pseudo-quantization operation refers to first quantizing the weight and input of the quantization layer from floating point to integer, and then dequantizing the quantized weight and input from integer to floating point. The pseudo-quantization operator can simulate the error caused by the hardware quantization process of the model by performing pseudo-quantization operations on the input and weight of the quantization layer, and perform forward operations on the model with the error, so that the model can adapt to the hardware quantization process in the inference stage during the training stage.

[0077] Among them, the hyperparameter indicates the quantization method corresponding to the pseudo-quantization model, and the hyperparameter includes a quantization mode and a quantization bit width, etc., wherein the quantization mode includes symmetric quantization or asymmetric quantization, etc. The type of the pseudo-quantization operator is determined based on the hyperparameter, and the accuracy of the pseudo-quantization operation of the pseudo-quantization operator is determined based on the quantization parameter, and the quantization parameter is used to adjust the accuracy of mapping the data used for training the first quantization layer to the preset quantization bit width, and the preset quantization bit width is determined based on the hyperparameter. Among them, the quantization parameter includes an offset value and a scaling factor, etc. The initial values ​​of the hyperparameter and the quantization parameter are both preset values, but the quantization parameter can be updated during the subsequent training of the pseudo-quantization model.

[0078] Among them, the types of pseudo-quantization operators corresponding to different quantization layers are the same, but the values ​​of quantization parameters in the pseudo-quantization operators may be different. For example, the pseudo-quantization operators corresponding to different quantization layers are all symmetric quantization operators or asymmetric quantization operators. It should be noted that the type of pseudo-quantization operator and the value of the quantization parameter corresponding to each quantization layer can be set according to actual needs (such as quantization bit width) or hardware type, and the embodiments of the present application do not limit this.

[0079] It should be noted that the above steps 301 to 303 are explained by taking the case where the approximate transformation method adopted is independent of the pseudo-quantization operator as an example. In this case, step 301 is executed first, and then step 302 and step 303 are executed, that is, the target model is first approximated by transformation, and then the quantization layer in the target model after the approximate transformation is identified to insert the pseudo-quantization operator. In some embodiments, the approximate transformation method adopted is related to the pseudo-quantization operator. In this case, step 302 and step 303 are executed first, and then step 301 is executed, that is, the quantization layer in the target model is first identified to insert the pseudo-quantization operator, and then the target model after the pseudo-quantization operator is inserted is approximated by transformation. The embodiments of the present application do not limit this.

[0080] Below through Figure 6 The above steps 301 to 303 are described with examples. Figure 6 is a flow chart of a method for constructing a pseudo-quantization model provided in an embodiment of the present application, such as Figure 6 As shown, the method includes obtaining a target model (that is, the original model), performing an approximate transformation on the target model (corresponding to the above step 301), identifying the quantization layer in the target model after the approximate transformation (corresponding to the above step 302), inserting a pseudo quantization operator before each quantization layer (corresponding to the above step 303), and obtaining the constructed pseudo quantization model.

[0081] In the above method, the target model is approximately transformed according to the characteristics of the model and the computing characteristics of the hardware to be deployed, so as to prune the model or replace part of the subgraphs or operators in the model with subgraphs or operators with higher computational efficiency. This can reasonably consider the computing characteristics of the hardware and improve the affinity of the model to the hardware, which is conducive to fully exploring and giving play to the performance advantages of the hardware, giving full play to the computing power of the hardware in practical applications, thereby improving the reasoning speed and achieving reasoning acceleration.

[0082] The above describes a method for constructing a pseudo-quantization model. The following describes a method for training a pseudo-quantization model provided in an embodiment of the present application. Taking the method executed by a computing device as an example, in the method, the computing device performs multiple iterations of training on a pseudo-quantization model constructed based on the characteristics and hyperparameters of the target model based on the training data of the target model. The pseudo-quantization model includes multiple quantization layers, and the hyperparameters indicate the quantization method corresponding to the pseudo-quantization model. The pseudo-quantization model can be based on the above Figure 3 The pseudo-quantization model constructed in the embodiment shown is a pseudo-quantization model that has undergone an approximate transformation during the construction process, or it may be a pseudo-quantization model that has not undergone an approximate transformation during the construction process, and the embodiments of the present application do not limit this.

[0083] The above training process is introduced below by taking the I-th iteration training of the first quantization layer among multiple quantization layers as an example. Figure 7 is a flowchart of a training method for a pseudo-quantization model provided in an embodiment of the present application, such as Figure 7 As shown, the method includes the following steps 701 to 704.

[0084] 701. A computing device determines the number of weights that can be updated based on a first learning rate of a first quantization layer, where the first quantization layer is any one of a plurality of quantization layers, and data used for training the first quantization layer is data processed according to a quantization parameter corresponding to the first quantization layer.

[0085] Among them, the first learning rate is the initial learning rate of the quantization layer, that is, the learning rate of the quantization layer preset in the training script of the target model. For different quantization layers, the first learning rate may be the same or different, and the embodiment of the present application does not limit this.

[0086] Among them, the process of processing the training data with the quantization parameters corresponding to the first quantization layer is also a pseudo-quantization process, which includes: first quantizing the weights (weight) and inputs (input) of the first quantization layer from floating point to integer (float→int), and then dequantizing the quantized weights and inputs from integer to floating point (int→float).

[0087] Among them, if the floating-point weight corresponding to the first iteration of the first quantization layer is recorded as θ (I) , the gradient is denoted as g( I) , the first learning rate is recorded as η, then the process of the computing device determining the number of weights that can be updated based on the first learning rate includes: determining if based on g (I) and η for weight θ (I) Update the updated floating-point weight θ (I+1 ) and quantization parameters; the floating-point weight θ (I) Quantized to integer weight int(θ (I) ), the floating point weight θ (I) Quantized to integer weight int(θ (I) ), the floating point weight θ (I+1 ) quantized to integer weight int(θ (I+1) ); based on int(θ (I) ) and int(θ (I+1 )) to determine the number of weights that can be updated, the number of weights that can be updated is the number of non-zero components in the difference; wherein, based on g (I) and η for weight θ (I) The updating process can be expressed by the following formula (1). The updating process of the quantization parameter is the same as that of formula (1), and will not be described in detail.

[0088] θ (I+1) =θ (I) -η×g (I) (1)

[0089] 702. When the number of weights that can be updated based on the first learning rate is less than a preset value, the computing device adjusts the first learning rate to the second learning rate, determines the number of weights that can be updated based on the second learning rate, and when the number of weights that can be updated determined based on the second learning rate is less than the preset value, continues to adjust the second learning rate until the number of weights that can be updated determined based on the adjusted Nth learning rate is greater than or equal to the preset value, where N is an integer greater than 1.

[0090] Among them, the number of weights that can be updated based on the first learning rate is less than or equal to the preset value, which means that the number of weights that can be updated based on the first learning rate is less than expected, and the update effect of the weights is poor, which is not conducive to improving the accuracy of the model. Therefore, gradient compensation is required to ensure the effective update of weights and quantization parameters. In some embodiments, the adjustment of the first learning rate by the computing device refers to amplifying the first learning rate, which is not limited in the embodiments of the present application.

[0091] The second learning rate refers to a learning rate obtained by adjusting the first learning rate once, and the Nth learning rate refers to a learning rate obtained by adjusting the first learning rate N-1 times.

[0092] Among them, during the Nth adjustment, the first learning rate is enlarged to the original L N times, after amplifying the first learning rate for the N-1th time, an Nth learning rate is obtained, and the number of weights that can be updated based on the Nth learning rate is greater than the preset value, and L is the gradient compensation strength, and L is greater than 0. The gradient compensation strength can be set according to actual needs, for example, the gradient compensation strength is set to 2 or 3, etc., and the embodiment of the present application is not limited to this.

[0093] 703. The computing device determines a target learning rate according to the Nth learning rate, where the target learning rate is the learning rate of the first quantization layer in the Ith training.

[0094] The target learning rate is the learning rate used by the first quantization layer to actually update the weights during the Ith training. The optimal learning rate means that the number of weights that can be updated based on the target learning rate is the largest among all the learning rates between the Nth learning rate and the N-1th learning rate.

[0095] The process of determining the target learning rate by the computing device includes: determining the target learning rate according to the Nth learning rate and the N-1th learning rate, the N-1th learning rate being the previous learning rate of the Nth learning rate, and the target learning rate being the optimal learning rate between the Nth learning rate and the N-1th learning rate. In some embodiments, the computing device determines the optimal learning rate between the Nth learning rate and the N-1th learning rate based on a dichotomy, which is not limited in the embodiments of the present application.

[0096] In the gradient compensation strategy shown in the above steps 702 and 703, the first learning rate is first amplified according to a fixed gradient compensation strength to achieve gradient compensation. Since the next amplification is performed on the basis of the learning rate obtained by the previous amplification each time, the later the amplification, the stronger the effect. That is, the learning rate obtained by each amplification is exponentially increasing, which is convenient for quickly determining the learning rate at which the number of updated weights can reach a preset value, and can improve the efficiency of gradient compensation. Moreover, since the N-1th amplification is the one with the strongest amplification effect among the previous N-1th amplifications, the Nth learning rate obtained by amplification is also the largest. However, the largest learning rate does not mean that the weights that can be updated based on the maximum learning rate are the largest, that is, the effect of gradient compensation is not necessarily the best. Therefore, a binary search method is used to determine the optimal learning rate, that is, the target learning rate, between the N-1th learning rate obtained by the N-2th amplification and the Nth learning rate obtained by the N-1th amplification, so as to avoid a decrease in the gradient compensation effect due to excessive amplification of the learning rate.

[0097] It should be noted that the description of the adjustment method of the first learning rate as amplification in the above steps 702 to 703 is only exemplary. In some embodiments, the first learning rate is also adjusted based on other methods so that the adjusted learning rate meets the requirements of weight updating. The embodiments of the present application are not limited to this.

[0098] It should be noted that the above steps 702 to 703 are an implementation method of continuing to determine the number of weights that can be updated after adjusting the first learning rate when the number of weights that can be updated based on the first learning rate is less than the preset value until the determined number of weights that can be updated is greater than or equal to the preset value. In some embodiments, this step is also implemented based on other methods, which is not limited to the embodiments of the present application.

[0099] 704. The computing device updates the weight of the first quantization layer based on the target learning rate.

[0100] The process of updating the weights of the first quantization layer based on the target learning rate includes: determining the updated floating-point weights in the same manner as the above formula (1); and quantizing the updated floating-point weights to obtain updated integer weights.

[0101] It should be noted that the above steps 701 to 704 are explained by taking the case that the number of weights that can be updated based on the first learning rate of the first quantization layer is less than the preset value as an example. In some embodiments, the number of weights that can be updated based on the first learning rate of the first quantization layer is greater than or equal to the preset value, and the computing device directly updates the weights of the first quantization layer based on the first learning rate. Among them, the number of weights that can be updated is greater than or equal to the preset value, indicating that the number of weights that can be updated based on the first learning rate meets expectations, and the update effect of the weights is good, which is conducive to improving the accuracy of the model. Therefore, there is no need to perform gradient compensation to ensure the effective update of weights and quantization parameters. Among them, the process of the computing device updating the weights of the first quantization layer based on the first learning rate is the same as the above step 704, and will not be repeated.

[0102] It should be noted that the above steps 701 to 704 are described by taking the updating of only the weights of the first quantization layer as an example. In some embodiments, the quantization parameters in the pseudo-quantization operator corresponding to the first quantization layer are also updated. For the first training of the first quantization layer, it includes: determining the sum of the weights and quantization parameters that can be updated based on the first learning rate of the first quantization layer; when the sum of the weights and quantization parameters that can be updated based on the first learning rate is less than a preset value, adjusting the first learning rate and continuing to determine the sum of the weights and quantization parameters that can be updated until the determined sum of the weights and quantization parameters that can be updated is greater than or equal to the preset value; updating the weights and the quantization parameters in the corresponding pseudo-quantization operator based on the adjusted first learning rate. The following is an example of a method for updating the weights and quantization parameters of the first quantization layer. Figure 8 Take as an example to illustrate the process.

[0103] Figure 8 is a flow chart of a gradient compensation strategy provided in an embodiment of the present application, such as Figure 8 As shown, for the first iteration of the first quantization layer, it is determined that if based on g (I) and η for weight θ (I) Update the updated floating-point weight θ (I+1) and quantization parameters; the floating-point weight θ (I) Quantized to integer weight int(θ (I) ), the floating point weight θ (I+1 ) quantized to integer weight int(θ (I+1) ); based on int(θ (I) ) and int(θ (I+1)), determine the number of weights that can be updated, and the number of weights that can be updated is the number of non-zero components in the difference; determine the number of quantization parameters that can be updated based on the quantization parameters before and after the update; determine the sum of the weights that can be updated and the quantization parameters that can be updated; if the sum is less than or equal to the preset value, amplify the first learning rate by 2 times, and redetermine the sum based on the gradient and the amplified learning rate until the sum that can be determined based on the amplified learning rate is greater than the preset value. For the case where gradient compensation has been performed before, at the last amplification, the optimal learning rate is determined between (η / 2, η) by using the binary search method, that is, the second learning rate; if the sum determined based on the first learning rate is greater than the preset value, it means that there is no need to perform gradient compensation, and the weights and quantization parameters are directly updated based on the first learning rate.

[0104] In the above method, in addition to updating the weights, the quantization parameters are also updated, which can improve the convergence effect of the training process, and realize adaptive gradient compensation by adaptively scaling the learning rate, so as to alleviate the gradient mismatch and gradient vanishing problems caused by quantization. In the fine-tuning stage with a small learning rate, it can ensure that a certain number of weights and quantization parameters can be updated, thereby reducing the loss of quantization accuracy; further, when the update effect of the weights and quantization parameters based on the first learning rate is poor, gradient compensation is performed; in addition, a pseudo-quantization model with lossless precision can be obtained, and for the pseudo-quantization model combined with approximate transformation, the precision loss caused by the approximate transformation can be compensated, and the quantization and approximate transformation are effectively combined. The convergence effect is better than that of conventional training, and the inference performance can be accelerated by 10% to 20%, thereby realizing the acceleration of quantization inference with lossless precision. Fig. 9 is a schematic diagram of a gradient compensation effect provided by an embodiment of the present application, such as Fig. 9 As shown in the left figure, before gradient compensation, when the floating-point data g1 is updated to g2 by the gradient and learning rate in the current state, the corresponding integer weight is n, so the weight is not effectively updated and the gradient needs to be compensated. After gradient compensation, g1 is updated to Fig. 9 In the right figure, at g2, the weight changes from n to n+1, indicating that the weight has been effectively updated.

[0105] In some embodiments, the computing device outputs the trained pseudo-quantization model as a quantization model that can be inferred on a hardware inference platform. Due to the approximate transformation step, the quantization model can obtain higher inference performance than the quantization model that has not undergone the approximate transformation step; the computing device deploys the output quantization model on the hardware inference platform for verification to obtain the accuracy and inference information of the quantization model.

[0106] Below Fig.10 Taking as an example, the process of the embodiment of the present application is illustrated. Fig.10is a schematic diagram of a model quantization adjustment method provided in an embodiment of the present application, such as Fig.10 As shown, the method includes obtaining a training script and a training data set of a target model, performing quantization configuration on the target model, building a pseudo-quantization model, training the pseudo-quantization model, and outputting the model and verifying the inference platform.

[0107] The quantized model obtained by training a model quantization adjustment method provided in an embodiment of the present application can be deployed on intelligent robots (such as inspection robots and interactive robots, etc.) to optimize the model reasoning performance while meeting the target accuracy; it can also be deployed on autonomous driving vehicles to optimize the recognition speed of target detection in autonomous driving scenarios. In addition, since this method uses a fine-tuning method based on a pre-trained model to perform quantitative perception training, the training cost is low and can be controlled within 5 epochs in most scenarios; and the method can quantize floating-point data to 8 bits without loss of accuracy, and can be quantized to 4 bits if hardware support is provided, further optimizing the reasoning performance and compressing the model size.

[0108] Fig.11 It is a structural diagram of a model quantization adjustment device provided in an embodiment of the present application, which includes a construction module 1101, an input module 1102, a training module 1103 and an output module 1104.

[0109] A construction module 1101 is used to construct a pseudo quantization model according to the characteristics and hyperparameters of the target model, wherein the pseudo quantization model includes multiple quantization layers;

[0110] An input module 1102, configured to input the training data of the target model into the pseudo quantization model;

[0111] The training module 1103 is used to perform multiple iterative training on the pseudo-quantization model to obtain a trained pseudo-quantization model. For the I-th training of the first quantization layer among the multiple quantization layers, it includes: determining the number of weights that can be updated based on the first learning rate of the first quantization layer, and the first quantization layer is any quantization layer among the multiple quantization layers; when the number of weights that can be updated based on the first learning rate is less than a preset value, adjusting the first learning rate and continuing to determine the number of weights that can be updated until the determined number of weights that can be updated is greater than or equal to the preset value; updating the weights based on the adjusted first learning rate; wherein I is an integer greater than or equal to 1;

[0112] The output module 1104 is used to output the trained pseudo quantization model as a quantization model.

[0113] Optionally, the training module 1103 includes an adjustment unit, configured to:

[0114] Adjust the first learning rate of the first quantization layer to a second learning rate;

[0115] The number of weights that can be updated is determined based on the second learning rate. When the number of weights that can be updated based on the second learning rate is less than the preset value, the second learning rate is continuously adjusted until the number of weights that can be updated based on the adjusted Nth learning rate is greater than or equal to the preset value, where N is an integer greater than 1.

[0116] Optionally, the training module 1103 further includes:

[0117] A determination unit is used to determine a target learning rate according to the Nth learning rate, where the target learning rate is the learning rate of the first quantization layer in the Ith training.

[0118] Optionally, the determining unit is used to:

[0119] The target learning rate is determined according to the Nth learning rate and the N-1th learning rate, the N-1th learning rate is a previous learning rate of the Nth learning rate, and the target learning rate is the optimal learning rate between the Nth learning rate and the N-1th learning rate.

[0120] Optionally, the training module 1103 is further used for:

[0121] When the sum of the number of weights and quantization parameters that can be updated based on the first learning rate is less than a preset value, the first learning rate is adjusted and the sum of the number of weights and quantization parameters that can be updated continues to be determined until the determined sum of the number of weights and quantization parameters that can be updated is greater than or equal to the preset value. The quantization parameter is used to adjust the accuracy of mapping the data used for training the first quantization layer to a preset quantization bit width, and the preset quantization bit width is determined based on the hyperparameter.

[0122] Optionally, the construction module 1101 includes:

[0123] An approximate transformation unit, used for performing an approximate transformation on the target model according to the characteristics of the target model, wherein the approximate transformation refers to transforming a subgraph or an operator in the target model;

[0124] An identification unit, used for identifying a quantization layer in the target model after the approximate transformation;

[0125] An inserting unit is used to insert a pseudo quantization operator corresponding to the hyperparameter before each quantization layer to obtain the pseudo quantization model, wherein the pseudo quantization operator includes the quantization parameter.

[0126] Optionally, the approximate transformation unit is used for at least one of the following:

[0127] Pruning the target model;

[0128] replacing activation layers in the target model;

[0129] Replace the operator in the target model.

[0130] Among them, the construction module 1101, the input module 1102, the training module 1103 and the output module 1104 can all be implemented by software, or can be implemented by hardware. Exemplarily, the implementation of the construction module 1101 is introduced below by taking the construction module 1101 as an example. Similarly, the implementation of the input module 1102, the training module 1103 and the output module 1104 can refer to the implementation of the construction module 1101.

[0131] As an example of a software functional unit, the construction module 1101 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the construction module 1101 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code can be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with similar geographical locations. Among them, usually a region can include multiple AZs.

[0132] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0133] As an example of a hardware functional unit, the building block 1101 may include at least one computing device, such as a server, etc. Alternatively, the building block 1101 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0134] The multiple computing devices included in the construction module 1101 can be distributed in the same region or in different regions. The multiple computing devices included in the construction module 1101 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the construction module 1101 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0135] It should be noted that, in other embodiments, the steps that the above modules are responsible for implementing can be specified as needed, and all the functions of the above device can be realized by respectively implementing different steps in the above model quantization adjustment method through the above modules. That is, the model quantization adjustment device provided in the above embodiment only uses the division of the above functional modules as an example when implementing the model quantization adjustment method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device provided in the above embodiment belongs to the same concept as the corresponding method embodiment. The specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0136] The present application implements a computing device such as Fig.12 As shown, Fig.12 1 is a schematic diagram of a computing device provided in an embodiment of the present application. The computing device 1200 may be a host, a server, or a personal computer, etc. The computing device 1200 may be implemented by a general bus architecture.

[0137] The computing device 1200 includes at least one processor 1201 , a communication bus 1202 , a memory 1203 , and at least one communication interface 1204 .

[0138] The processor 1201 is, for example, a general-purpose central processing unit (CPU), a network processor (NP), a graphics processing unit (GPU), a neural-network processing unit (NPU), a data processing unit (DPU), a microprocessor, or one or more integrated circuits for implementing the solution of the present application. For example, the processor 1201 includes an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD is, for example, a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0139] The communication bus 1202 is used to transmit information between the above components. The communication bus 1202 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.12 The fact that only one line is used in the diagram does not mean that there is only one bus or only one type of bus.

[0140] The memory 1203 is, for example, a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, or a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1203 is, for example, independent and connected to the processor 1201 via the communication bus 1202. The memory 1203 can also be integrated with the processor 1201.

[0141] The communication interface 1204 uses any transceiver-like device for communicating with other devices or communication networks. The communication interface 1204 includes a wired communication interface and may also include a wireless communication interface. Among them, the wired communication interface may be, for example, an Ethernet interface. The Ethernet interface may be an optical interface, an electrical interface, or a combination thereof. The wireless communication interface may be a wireless local area network (WLAN) interface, a cellular network communication interface, or a combination thereof, etc.

[0142] As an embodiment, the processor 1201 may include one or more CPUs.

[0143] As an embodiment, the computing device 1200 may include multiple processors, each of which may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0144] As an embodiment, the computing device 1200 may also include an output device and an input device. The output device communicates with the processor 1201 and can display information in a variety of ways. For example, the output device may be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. The input device communicates with the processor 1201 and can receive user input in a variety of ways. For example, the input device may be a mouse, a keyboard, a touch screen device, or a sensor device.

[0145] In some embodiments, the memory 1203 is used to store program codes for executing the solution of the present application, and the processor 1201 can execute the program codes stored in the memory 1203. That is, the computing device 1200 can implement the model quantization adjustment method provided in the method embodiment through the processor 12012 and the program codes in the memory 1203.

[0146] The present application implements a computer program product including instructions, which may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device implements the model quantization adjustment method provided in the above embodiment.

[0147] The embodiment of the present application provides a computer-readable storage medium, which can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions, and when the instructions are executed by a computing device cluster, the computing device cluster executes the model quantization adjustment method provided in the above embodiment.

[0148] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the target models and training data involved in this application are obtained with full authorization.

[0149] Those of ordinary skill in the art will appreciate that the various method steps and units described in the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the steps and components of each embodiment have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those of ordinary skill in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0150] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0151] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the unit is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0152] The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment of the present application.

[0153] In addition, each unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or software unit.

[0154] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computing device (which can be a personal computer, a server, or a computing device, etc.) to execute all or part of the steps of the method in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0155] In this application, the terms "first", "second", etc. are used to distinguish the same or similar items with basically the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is the quantity and execution order limited. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of various examples, the first learning rate can be referred to as the second learning rate, and similarly, the second learning rate can be referred to as the first learning rate. Both the first learning rate and the second learning rate can be learning rates, and in some cases, can be separate and different learning rates.

[0156] The term "at least one" in this application means one or more, and the term "multiple" in this application means two or more, for example, multiple quantization layers means two or more quantization layers. The terms "system" and "network" are often used interchangeably herein.

[0157] It should also be understood that the term "if" may be interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting." Similarly, the phrase "if it is determined that ..." or "if [a stated condition or event] is detected" may be interpreted to mean "upon determining that ..." or "in response to determining that ..." or "upon detecting [a stated condition or event]" or "in response to detecting [a stated condition or event]," depending on the context.

[0158] The above description is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0159] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer program instructions. When the computer program instructions are loaded and executed on a computer, the process or function in accordance with the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0160] The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital video disc (DVD), or a semiconductor medium (e.g., a solid-state drive)), etc.

[0161] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0162] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A model quantization adjustment method, characterized in that: The method comprises: Constructing a pseudo-quantization model according to the characteristics and hyperparameters of the target model, wherein the pseudo-quantization model includes multiple quantization layers; Inputting the training data of the target model into the pseudo quantization model; The pseudo-quantization model is iteratively trained for multiple times to obtain a trained pseudo-quantization model, and for the I-th training of the first quantization layer among the multiple quantization layers, the training includes: determining the number of weights that can be updated based on a first learning rate of the first quantization layer, where the first quantization layer is any one of the multiple quantization layers; when the number of weights that can be updated based on the first learning rate is less than a preset value, adjusting the first learning rate and continuing to determine the number of weights that can be updated until the determined number of weights that can be updated is greater than or equal to the preset value; updating the weights based on the adjusted first learning rate; wherein I is an integer greater than or equal to 1; The trained pseudo quantization model is output as a quantization model.

2. The method according to claim 1, characterized in that The step of continuing to determine the number of weights that can be updated after adjusting the first learning rate until the number of updated weights is greater than or equal to the preset value comprises: Adjust the first learning rate of the first quantization layer to a second learning rate; The number of weights that can be updated is determined based on the second learning rate. When the number of weights that can be updated based on the second learning rate is less than the preset value, the second learning rate is continuously adjusted until the number of weights that can be updated based on the adjusted Nth learning rate is greater than or equal to the preset value, where N is an integer greater than 1.

3. The method according to claim 2, characterized in that The method further comprises: A target learning rate is determined according to the Nth learning rate, where the target learning rate is the learning rate of the first quantization layer in the Ith training.

4. The method according to claim 3, characterized in that Determining the target learning rate according to the Nth learning rate includes: The target learning rate is determined according to the Nth learning rate and the N-1th learning rate, the N-1th learning rate is a previous learning rate of the Nth learning rate, and the target learning rate is the optimal learning rate between the Nth learning rate and the N-1th learning rate.

5. The method according to claim 1, characterized in that The method further comprises: When the sum of the number of weights and quantization parameters that can be updated based on the first learning rate is less than the preset value, the first learning rate is adjusted and the sum of the number of weights and quantization parameters that can be updated continues to be determined until the determined sum of the number of weights and quantization parameters that can be updated is greater than or equal to the preset value, and the quantization parameter is used to adjust the accuracy of mapping the data used for training the first quantization layer to a preset quantization bit width, and the preset quantization bit width is determined based on the hyperparameter.

6. The method according to any one of claims 1 to 5, characterized in that The pseudo-quantization model is constructed according to the characteristics and hyperparameters of the target model, including: According to the characteristics of the target model, the target model is approximated and transformed, wherein the approximate transformation refers to transforming a subgraph or an operator in the target model; Identifying a quantization layer in the target model after the approximate transformation; A pseudo quantization operator corresponding to the hyperparameter is inserted before each quantization layer to obtain the pseudo quantization model, wherein the pseudo quantization operator includes the quantization parameter.

7. The method according to claim 6, characterized in that The approximate transformation of the target model comprises at least one of the following: Pruning the target model; replacing activation layers in the target model; Replace the operator in the target model.

8. A model quantization adjustment device, characterized in that: The device comprises: A construction module, used to construct a pseudo-quantization model according to the characteristics and hyperparameters of the target model, wherein the pseudo-quantization model includes multiple quantization layers; An input module, used for inputting the training data of the target model into the pseudo quantization model; A training module, used for performing multiple iterative training on the pseudo-quantization model to obtain a trained pseudo-quantization model, and for the I-th training of the first quantization layer among the multiple quantization layers, comprising: determining the number of weights that can be updated based on a first learning rate of the first quantization layer, the first quantization layer being any one of the multiple quantization layers; when the number of weights that can be updated based on the first learning rate is less than a preset value, adjusting the first learning rate and continuing to determine the number of weights that can be updated until the determined number of weights that can be updated is greater than or equal to the preset value; updating the weights based on the adjusted first learning rate; wherein I is an integer greater than or equal to 1; An output module is used to output the trained pseudo quantization model as a quantization model.

9. The device according to claim 8, characterized in that The training module comprises an adjustment unit, which is used to: Adjust the first learning rate of the first quantization layer to a second learning rate; The number of weights that can be updated is determined based on the second learning rate. When the number of weights that can be updated based on the second learning rate is less than the preset value, the second learning rate is continuously adjusted until the number of weights that can be updated based on the adjusted Nth learning rate is greater than or equal to the preset value, where N is an integer greater than 1.

10. The device according to claim 9, characterized in that The training module also includes: A determination unit is used to determine a target learning rate according to the Nth learning rate, where the target learning rate is the learning rate of the first quantization layer in the Ith training.

11. The device according to claim 8, characterized in that The determining unit is used for: The target learning rate is determined according to the Nth learning rate and the N-1th learning rate, the N-1th learning rate is a previous learning rate of the Nth learning rate, and the target learning rate is the optimal learning rate between the Nth learning rate and the N-1th learning rate.

12. The device according to claim 8, characterized in that The training module is also used to: When the sum of the number of weights and quantization parameters that can be updated based on the first learning rate is less than a preset value, the first learning rate is adjusted and the sum of the number of weights and quantization parameters that can be updated continues to be determined until the determined sum of the number of weights and quantization parameters that can be updated is greater than or equal to the preset value, and the quantization parameter is used to adjust the accuracy of mapping the data used for training the first quantization layer to a preset quantization bit width, and the preset quantization bit width is determined based on the hyperparameter.

13. The device according to any one of claims 8 to 12, characterized in that The building blocks include: An approximate transformation unit, used for performing an approximate transformation on the target model according to the characteristics of the target model, wherein the approximate transformation refers to transforming a subgraph or an operator in the target model; An identification unit, used for identifying a quantization layer in the target model after the approximate transformation; The inserting unit is used to insert a pseudo quantization operator corresponding to the hyperparameter before each quantization layer to obtain the pseudo quantization model, wherein the pseudo quantization operator includes the quantization parameter.

14. The device according to claim 12, characterized in that The approximate transformation unit is used for at least one of the following: Pruning the target model; replacing activation layers in the target model; Replace the operator in the target model.

15. A computing device, characterized in that: The computing device comprises a processor, and the processor is used to execute program code, so that the computing device performs the model quantization adjustment method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that: At least one program code is stored in the storage medium, and the at least one program code is read by the processor to enable the computing device to execute the model quantization adjustment method according to any one of claims 1 to 7.

Images