Model quantification method, apparatus, computer device, and storage medium

Abstract

Embodiments of the present application disclose a model quantization method and device, computer equipment and a storage medium, belonging to the technical field of computers. The method comprises: obtaining a first model, the first model comprising a plurality of first operators, a splicing layer and a second operator; determining an input range of each operator in the plurality of first operators and the second operator; updating a quantization parameter of each operator based on the input range of each operator and a target output range, so that the quantization parameter of each operator converges to a quantization parameter corresponding to the target output range; and performing quantization processing on network parameters in the splicing layer based on the quantization parameter corresponding to the target output range, so as to ensure that the plurality of input ranges of the splicing layer and the range of the network parameters are the same, thereby completing quantization of the splicing layer.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a model quantization method, apparatus, computer device, and storage medium. Background Technology

[0002] With the continuous development of artificial intelligence technology, the application of various neural network models is becoming increasingly widespread. In order to improve the processing efficiency of neural network models, the network parameters of various network layers in the neural network model are usually quantized, so that the quantized network layers can directly process integer data.

[0003] In related technologies, considering that multiple data inputs to the concatenation layer need to have the same range, while the range of integer data output by each network layer after quantization may be different, the network parameters of the concatenation layer are not quantized. Instead, the multiple integer data inputs to the concatenation layer are first converted into floating-point data, then the multiple floating-point data are concatenated based on the concatenation layer, and finally the concatenated data is converted from floating-point data back to integer data. However, it still cannot achieve quantization of the network parameters of the concatenation layer. Therefore, there is an urgent need to provide a method for quantizing the network parameters of the concatenation layer in the model. Summary of the Invention

[0004] This application provides a model quantization method, apparatus, computer device, and storage medium, capable of quantizing network parameters of the splicing layer in a model. The technical solution is as follows:

[0005] On the one hand, a model quantization method is provided, the method comprising:

[0006] Obtain a first model, which includes multiple first operators, a splicing layer, and a second operator. The output of the first operator is the input of the splicing layer, and the output of the splicing layer is the input of the second operator.

[0007] The input range of each of the multiple first operators and second operators is determined. Based on the input range of each operator and the target output range, the quantization parameter of each operator is updated so that the quantization parameter of each operator converges to the quantization parameter corresponding to the target output range. The quantization parameter corresponding to the target output range is used to quantize any data to the target output range.

[0008] Based on the quantization parameters corresponding to the target output range, the network parameters in the splicing layer are quantized.

[0009] Optionally, updating the quantization parameters of the second operator based on the input range of the second operator in the current round, the input range of the previous round, and the target output range includes:

[0010] The second operator is weighted by the input range of the current round and the input range of the previous round to obtain the second input range of the second operator;

[0011] Based on the second input range of the second operator and the target output range, the quantization parameters of the second operator are updated so that the updated quantization parameters can quantize the data in the second input range to the target output range.

[0012] Optionally, determining the input range of each of the plurality of first operators and second operators includes:

[0013] Obtain the association information of the splicing layer, wherein the association information indicates the first operator located before the splicing layer and a plurality of second operators located after the splicing layer;

[0014] Based on the association information, a plurality of the first operator and the second operator are determined;

[0015] Determine the input range of multiple first operators and the input range of the second operator.

[0016] Optionally, obtaining the first model includes:

[0017] Obtain a second model, the second model comprising a plurality of network layers connected in sequence, the plurality of network layers comprising at least the splicing layer and a plurality of first network layers, wherein the first network layer is a network layer connected to the splicing layer and located before the splicing layer;

[0018] The first operator is inserted between the splicing layer and each first network layer, and the second operator is inserted after the splicing layer to obtain the first model.

[0019] On the other hand, a model quantization apparatus is provided, the apparatus comprising:

[0020] The model acquisition module is used to acquire a first model, which includes multiple first operators, a splicing layer, and a second operator. The output of the first operator is the input of the splicing layer, and the output of the splicing layer is the input of the second operator.

[0021] The parameter update module is used to determine the input range of each of the multiple first operators and second operators, and update the quantization parameter of each operator based on the input range of each operator and the target output range, so that the quantization parameter of each operator converges to the quantization parameter corresponding to the target output range, wherein the quantization parameter corresponding to the target output range is used to quantize any data to the target output range;

[0022] The quantization processing module is used to quantize the network parameters in the splicing layer based on the quantization parameters corresponding to the target output range.

[0023] Optionally, the parameter update module includes:

[0024] The first parameter update unit is used to update the quantization parameters of each first operator based on the input range of each first operator, the input range of the second operator, and the target output range.

[0025] The second parameter update unit is used to update the quantization parameters of the second operator based on the input range of the second operator and the target output range, so that the quantization parameters of each of the first operator and the second operator converge to the quantization parameters corresponding to the target output range.

[0026] Optionally, the parameter update module includes:

[0027] A range determination unit is used to determine the input range of each operator in each of a plurality of rounds in that round;

[0028] The third parameter update unit is used to update the quantization parameters of each operator based on the input range of each operator in the current round, the input range of the previous round, and the target output range, so that the quantization parameters of each operator converge to the quantization parameters corresponding to the target output range.

[0029] Optionally, the range determination unit is used to process multiple test data corresponding to the round by calling the first model to determine the input range of each operator in the round, wherein the multiple test data corresponding to each round are different.

[0030] Optionally, the third parameter update unit is used for:

[0031] The quantization parameters of each first operator are updated based on the input range of each first operator in the current round and the input range of the previous round, the input range of the second operator in the current round and the input range of the previous round, and the target output range.

[0032] Based on the input range of the second operator in the current round, the input range of the previous round, and the target output range, the quantization parameters of the second operator are updated until the quantization parameters of each of the first and second operators converge to the quantization parameters corresponding to the target output range.

[0033] Optionally, the third parameter update unit is used for:

[0034] The input range of each first operator in the current round and the input range of the previous round are weighted to obtain the first input range of each first operator;

[0035] The second operator is weighted by the input range of the current round and the input range of the previous round to obtain the second input range of the second operator;

[0036] The quantization parameters of each first operator are updated based on the first input range, the second input range, and the target output range of each first operator.

[0037] Optionally, the third parameter update unit is used for:

[0038] Determine a first weight parameter corresponding to each of the first operators and a second weight parameter corresponding to each of the second operators, wherein the first weight parameter is negatively correlated with the number of times the quantization parameter is updated, and the second weight parameter is positively correlated with the number of times the quantization parameter is updated;

[0039] Based on the first weight parameter and the second weight parameter, the first input range and the second input range of each first operator are weighted to obtain the target input range of each first operator;

[0040] Based on the target input range and the target output range of each first operator, the quantization parameters of each first operator are updated so that the updated quantization parameters can quantize the data in the target input range to the target output range.

[0041] Optionally, the third parameter update unit is used for:

[0042] For each first operator, based on the first weight parameter and the second weight parameter of the first operator, the minimum value in the first input range and the minimum value in the second input range of the first operator are weighted to obtain the target minimum value;

[0043] Based on the first weight parameter and the second weight parameter, the maximum value in the first input range and the maximum value in the second input range are weighted to obtain the target maximum value;

[0044] The range between the target minimum value and the target maximum value is determined as the target input range of the first operator.

[0045] Optionally, the first operator includes a first quantization parameter, and the third parameter update unit is used for:

[0046] For each of the first operators, determine a first difference between the maximum and minimum values ​​in the target input range of the first operator;

[0047] Determine the second difference between the maximum and minimum values ​​within the target output range;

[0048] The ratio between the first difference and the second difference is determined as the updated first quantization parameter of the first operator, whereby the first quantization parameter indicates the scaling factor of the data.

[0049] Optionally, the first operator further includes a second quantization parameter, and the third parameter update unit is used for:

[0050] The reference minimum value is obtained by rounding down the ratio between the minimum value in the target input range and the updated first quantization parameter.

[0051] The difference between the minimum value in the target output range and the reference minimum value is determined as the updated second quantization parameter of the first operator, and the second quantization parameter indicates the displacement magnitude of the data.

[0052] Optionally, the third parameter update unit is used for:

[0053] The second operator is weighted by the input range of the current round and the input range of the previous round to obtain the second input range of the second operator;

[0054] Based on the second input range of the second operator and the target output range, the quantization parameters of the second operator are updated so that the updated quantization parameters can quantize the data in the second input range to the target output range.

[0055] Optionally, the parameter update module includes:

[0056] The range determination unit is used to turn off each of the first operators and the second operators in the first model, determine the multiple input ranges of the splicing layer as the third input ranges of the corresponding first operators, and determine the output range of the splicing layer as the third input range of the second operators;

[0057] The third parameter update unit is used to update the quantization parameters of each first operator and the quantization parameters of each second operator based on the third input range of each first operator, the third input range of the second operator, and the target output range.

[0058] The range determination unit is further configured to open each of the first operators and the second operators in the first model and determine the fourth input range of each of the first operators and the second operators;

[0059] The third parameter update unit is used to continue updating the quantization parameters of each first operator and the second operator based on the fourth input range of each first operator, the fourth input range of the second operator, and the target output range, so that the quantization parameters of each first operator and the second operator converge to the quantization parameters corresponding to the target output range.

[0060] Optionally, the first operator includes a first quantization operator and a first inverse quantization operator, and the apparatus further includes:

[0061] The operator adjustment module is used to delete the first inverse quantization operator in each of the first operators and to delete the second quantization operator in each of the second operators.

[0062] Optionally, the parameter update module includes:

[0063] A range determination unit is used to obtain the association information of the splicing layer, wherein the association information indicates the first operator located before the splicing layer and a plurality of second operators located after the splicing layer;

[0064] The range determination unit is used to determine multiple first operators and second operators based on the association information;

[0065] The range determination unit is used to determine the input range of multiple first operators and the input range of the second operator.

[0066] Optionally, the model acquisition module includes:

[0067] A model acquisition unit is used to acquire a second model, the second model including multiple network layers connected in sequence, the multiple network layers including at least the splicing layer and multiple first network layers, the first network layers being network layers connected to the splicing layer and located before the splicing layer;

[0068] An operator insertion unit is used to insert the first operator between the splicing layer and each first network layer, and to insert the second operator after the splicing layer to obtain the first model.

[0069] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to perform the operations performed in the model quantization method as described above.

[0070] On the other hand, a computer-readable storage medium is provided that stores at least one computer program, which is loaded and executed by a processor to perform the operations performed in the model quantization method as described above.

[0071] On the other hand, a computer program product or computer program is provided, the computer program product or computer program including computer program code stored in a computer-readable storage medium, a processor of a computer device reading the computer program code from the computer-readable storage medium, the processor executing the computer program code, causing the computer device to perform the operations performed in the model quantization method as described above.

[0072] The method, apparatus, computer device, and storage medium provided in this application update the quantization parameters of each operator by using the input range and target output range of each operator, so that the quantization parameters of each operator converge to the quantization parameters corresponding to the target output range. In this way, the output range of multiple operators is constrained to be consistent by using the target output range, so that the output range of each operator is consistent. This is equivalent to ensuring that the multiple input ranges and network parameter ranges of the splicing layer are the same, thereby completing the quantization of the splicing layer. Attached Figure Description

[0073] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0074] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application.

[0075] Figure 2 This is a flowchart of a model quantization method provided in an embodiment of this application.

[0076] Figure 3 This is a flowchart of a model quantization method provided in an embodiment of this application.

[0077] Figure 4 This is a schematic diagram of an operator insertion method provided in an embodiment of this application.

[0078] Figure 5 This is a schematic diagram of another operator insertion method provided in the embodiments of this application.

[0079] Figure 6 This is a flowchart of a model quantization method provided in an embodiment of this application.

[0080] Figure 7 This is a flowchart of another model quantization method provided in the embodiments of this application.

[0081] Figure 8 This is a flowchart of another model quantization method provided in the embodiments of this application.

[0082] Figure 9 This is a schematic diagram of the structure of a model quantization device provided in an embodiment of this application.

[0083] Figure 10 This is a schematic diagram of another model quantization device provided in an embodiment of this application.

[0084] Figure 11 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application.

[0085] Figure 12 This is a schematic diagram of the structure of a server provided in an embodiment of this application. Detailed Implementation

[0086] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0087] It is understood that the terms "first," "second," etc., used in this application may be used to describe various concepts herein, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of this application, a first operator may be referred to as a second operator, and similarly, a second operator may be referred to as a first operator.

[0088] "At least one" refers to one or more operators. For example, at least one operator can be one operator, two operators, three operators, or any integer number of operators greater than or equal to one. "Multiple" refers to two or more operators. For example, multiple operators can be two operators, three operators, or any integer number of operators greater than or equal to two. "Each" refers to each of the at least one operators. For example, each operator refers to each of the multiple operators. If the multiple operators are three operators, then each operator refers to each of the three operators.

[0089] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0090] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0091] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instruction-based learning.

[0092] The following will describe the model quantization method provided in the embodiments of this application based on artificial intelligence technology.

[0093] This application provides a model quantization method, executed by a computer device. The computer device can quantize the stitching layer in the first model, enabling the stitching layer to directly process data of the same range. Optionally, the computer device is a terminal, such as a mobile phone, desktop computer, laptop computer, tablet computer, smart TV, smart speaker, vehicle terminal, smart robot, or smartwatch. Optionally, the computer device is a server, which can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0094] In one possible implementation, the computer program involved in the embodiments of this application may be deployed and executed on a computer device, or executed on multiple computer devices located in one location, or executed on multiple computer devices distributed in multiple locations and interconnected through a communication network. Multiple computer devices distributed in multiple locations and interconnected through a communication network can form a blockchain system.

[0095] In one possible implementation, the computer device in this application embodiment is a node in a blockchain system. The node is able to store the quantized first model in the blockchain, and then the node or other nodes in the blockchain can retrieve the quantized first model from the blockchain.

[0096] In one possible implementation, such as Figure 1 As shown, the implementation environment of this application embodiment includes a server 101 and a terminal 102, which establish a communication connection via wireless or wired means. The server 101 trains a first model, quantizing the stitching layer and other network layers in the first model to obtain a quantized first model. Since quantizing the first model reduces its size and lowers the configuration requirements of the device deploying it, the server 101 can send the quantized first model to the terminal 102, which can then deploy and process the quantized first model. For example, if the first model is an image segmentation model, the terminal 102 can use the quantized first model to segment any image and obtain the segmentation result.

[0097] Figure 2This is a flowchart illustrating a model quantization method provided in an embodiment of this application. The execution entity of this embodiment is a computer device; optionally, the computer device is the one described above. Figure 1 The terminal or server in the system. See also Figure 2 The method includes:

[0098] 201. Computer equipment obtains the first model.

[0099] The first model includes multiple first operators, a concatenation layer, and a second operator. The output of the first operator is the input of the concatenation layer, and the output of the concatenation layer is the input of the second operator. That is, in this first model, the concatenation layer is connected to multiple first operators before it and to a second operator after it. The concatenation layer is used to concatenate multiple input data.

[0100] 202. The computer device determines the input range of each of the multiple first operators and second operators, and updates the quantization parameters of each operator based on the input range of each operator and the target output range, so that the quantization parameters of each operator converge to the quantization parameters corresponding to the target output range.

[0101] The input range of an operator refers to the numerical range of the input data of the operator. In this embodiment, the target output range is the numerical range of the expected output data of the operator. Since the output of the first operator is the input of the splicing layer, the target output range can also be understood as the numerical range of the expected input data of the splicing layer. The quantization parameter corresponding to the target output range is used to quantize any data to the target output range, that is, to quantize any data based on the quantization parameter, and the resulting data belongs to the target output range.

[0102] In this embodiment, after the computer device determines the input range and the target output range of each operator, it updates the quantization parameters of each operator based on these parameters, so that the quantization parameters of each operator converge to the quantization parameters corresponding to the target output range. Since the quantization parameters corresponding to the output range are used to quantize any data to the target output range, the convergence of the quantization parameters of each operator to the quantization parameters corresponding to the target output range is equivalent to updating the quantization parameters of each operator so that the output data of each operator belongs to the target output range, which means that the multiple input data of the splicing layer all belong to the target output range.

[0103] 203. The computer equipment performs quantization processing on the network parameters in the splicing layer based on the quantization parameters corresponding to the target output range.

[0104] The computer device updates the quantization parameters of each operator until the quantization parameters of each operator converge to the quantization parameters corresponding to the target output range. Based on the quantization parameters corresponding to the target output range, the network parameters in the splicing layer are quantized, that is, the network parameters of the splicing layer are quantized to the target output range based on the quantization parameters.

[0105] Since performing step 202 above enables multiple input data of the stitching layer to fall within the target output range, performing step 203 enables the network parameters in the stitching layer to be quantized to the target output range. Therefore, the data processed by the stitching layer is all within the target output range, thereby achieving quantization of the stitching layer.

[0106] The method provided in this application updates the quantization parameters of each operator by using the input range and target output range of each operator, so that the quantization parameters of each operator converge to the quantization parameters corresponding to the target output range. In this way, the output range of multiple operators is constrained to be consistent by using the target output range, so that the output range of each operator is consistent. This is equivalent to ensuring that the multiple input ranges of the splicing layer and the range of network parameters are the same, thereby completing the quantization of the splicing layer.

[0107] Figure 3 This is a flowchart illustrating a model quantization method provided in an embodiment of this application. The execution entity of this embodiment is a computer device; optionally, the computer device is the one described above. Figure 1 The terminal or server in the system. See also Figure 3 The method includes:

[0108] 301. Computer equipment obtains the first model.

[0109] The first model includes multiple first operators, a concatenation layer, and a second operator. The output of the first operator is the input of the concatenation layer, and the output of the concatenation layer is the input of the second operator. The concatenation layer is used to concatenate multiple input data. Optionally, the concatenation layer is a Concat operator. The first model can be any type of neural network model, as long as it includes a concatenation layer. For example, the first model can be a speech recognition model, an image segmentation model, an image classification model, or a face recognition model, etc., and this application embodiment does not limit this.

[0110] In one possible implementation, a computer device obtains a second model comprising a plurality of sequentially connected network layers, the plurality of network layers including at least a splicing layer and a plurality of first network layers, the first network layers being network layers connected to and preceding the splicing layer. The computer device inserts a first operator between the splicing layer and each of the first network layers, and inserts a second operator after the splicing layer to obtain the first model. The first network layers can be of any type, such as convolutional layers or pooling layers.

[0111] After the computer device adds a corresponding first operator to the splicing layer, input data that could previously be directly input to the splicing layer now needs to be processed by the first operator before it can be input to the splicing layer. After the computer device adds a corresponding second operator to the splicing layer, the output data that could previously be directly processed by the splicing layer now needs to be processed by the second operator before further processing can proceed.

[0112] Taking the first and second operators as pseudo-quantization operators as an example, Figure 4 This is a schematic diagram of an operator insertion method provided in an embodiment of this application. See also... Figure 4 A pseudo-quantization operator 402 is inserted between the first network layer 401 and the splicing layer 403, and a pseudo-quantization operator 404 is inserted after the splicing layer. Each pseudo-quantization operator includes a quantization operator and an inverse quantization operator. The quantization operator is used for quantization processing, and the inverse quantization operator is used for inverse quantization processing. Each pseudo-quantization operator also has a corresponding range statistics function, which is used to determine the input range of the pseudo-quantization operator.

[0113] Figure 5 This is a schematic diagram of another operator insertion method provided in an embodiment of this application. See also... Figure 5 The second model includes convolutional layers 501, 502, and 503, and a splicing layer 504, with the splicing layer 504 connected to each of the convolutional layers 501, 502, and 503. The computer device inserts a pseudo-quantization operator 505 between the convolutional layers 501 and 504, a pseudo-quantization operator 506 between the convolutional layers 502 and 504, and a pseudo-quantization operator 507 between the convolutional layers 503 and 504. A pseudo-quantization operator 508 is then connected after the splicing layer 504 to obtain the first model. Pseudo-quantization operators 505, 506, and 507 are the first operators shown in this embodiment, and pseudo-quantization operator 508 is the second operator shown in this embodiment.

[0114] 302. For each of multiple rounds, the computer device determines the input range of each of multiple first operators and second operators.

[0115] In this embodiment, the computer device executes multiple rounds to update the quantization parameters of the operator. For each of these rounds, the computer device determines the input range of each operator. The input range of an operator refers to the numerical range to which the input data of the operator belongs.

[0116] In one possible implementation, the computer device processes multiple test data points corresponding to a given round by calling a first model to determine the input range of each operator in that round. The multiple test data points differ for each round. The type of these multiple test data points is related to the type of the first model; for example, if the first model is an image segmentation model, then the multiple test data points can be multiple images.

[0117] The computer device acquires multiple test data points corresponding to the current round and calls the first model to process these test data points respectively. For each operator, other network layers are connected in front of it. During the processing of a certain test data point by the first model, the output data of these other network layers is input into the operator. The first model processes the multiple test data points, resulting in multiple output data points input into the operator. The computer device counts the values ​​of these multiple output data points and determines the input range of the operator based on these values. For example, the computer device determines the input range of the operator as the range between the minimum and maximum values ​​of the multiple output data points.

[0118] In another possible implementation, the computer device obtains the association information of the splicing layer, which indicates a first operator located before the splicing layer and multiple second operators located after the splicing layer. Based on the association information, the multiple first operators and second operators are determined, and then the computer device determines the input range of the multiple first operators and the input range of the second operators.

[0119] To facilitate determining which operators are connected in the splicing layer, when inserting the first and second operators corresponding to the splicing layer, the computer generates association information for the splicing layer. This association information indicates the first operators preceding the splicing layer and multiple second operators following the splicing layer. Therefore, by querying the association information of the splicing layer, the computer can determine the multiple first operators preceding the splicing layer and the multiple second operators following the splicing layer, thereby determining the input range of the multiple first operators and the input range of the multiple second operators.

[0120] The computer device processes multiple test data by calling the first model to determine the input range of each operator in the first model and stores the input range of each operator. Then, based on the association information, the computer device queries the first and second operators connected to the splicing layer, and can obtain the input range of each first operator and the input range of each second operator from the determined input range of multiple operators.

[0121] Optionally, the associated information includes an operator identifier and multiple operator pointers, where the operator identifier indicates a second operator located after the splicing layer, and the operator pointers indicate a first operator located before the splicing layer. Therefore, the computer device can determine the second operator based on the operator identifier and determine multiple first operators based on the operator pointers.

[0122] 303. The computer device updates the quantization parameters of each first operator based on the input range of each first operator in the current round and the input range of the previous round, the input range of the second operator in the current round and the input range of the previous round, and the target output range.

[0123] In addition to determining the input range of each operator in this round, the computer device also obtains the input range of each operator in the previous round, and updates the quantization parameters of each first operator based on the input range of each first operator in this round and the input range of the previous round, the input range of the second operator in this round and the input range of the previous round, and the target output range.

[0124] Since the test data used in each round is different, the input range determined in each round is also different. In each round, the computer device combines the input range of the current round with the input range of the previous round to update the quantization parameters of the first operator. This is equivalent to considering the test data in two rounds, which increases the amount of information referenced in updating the quantization parameters of the first operator. This is beneficial to improving the accuracy of updating the quantization parameters and also to improving the convergence speed of the quantization parameters.

[0125] Furthermore, by updating the quantization parameters of the first operator based on the input range and target output range of each first and second operator, it is equivalent to broadcasting the input range of the second operator to each first operator. Thus, the input range of the second operator is used to constrain the quantization parameters of the first operator, so that the quantization parameters of the first and second operators converge to the same value as soon as possible, thereby further improving the convergence speed of the quantization parameters.

[0126] The target output range refers to the numerical range of the output data of the desired operator. Since the output of the first operator is the input of the splicing layer, the target output range can also be understood as the numerical range of the input data of the desired splicing layer. This target output range is also called the quantization range. Optionally, the target output range is preset by the computer device or provided to the computer device by the developer; this embodiment does not limit this.

[0127] In one possible implementation, the operator's output data is integer data, which is stored in binary format. The computer device determines the target storage space for the integer data and, based on the storage space, determines the target output range. Optionally, the computer device determines the minimum and maximum values ​​that the storage space can store, and defines the range between the minimum and maximum values ​​as the target output range. For example, in an 8-bit storage space, the minimum value that can be stored is -128, and the maximum value that can be stored is 127; therefore, the target output range is [-128, 127]. Optionally, the size of the target storage space is preset by the computer device or provided to the computer device by the developer; this embodiment does not limit this.

[0128] In one possible implementation, the computer device performs weighted processing on the input range of each first operator in the current round and the input range of the previous round to obtain a first input range for each first operator. Similarly, it performs weighted processing on the input range of the second operator in the current round and the input range of the previous round to obtain a second input range for the second operator. Therefore, the first input range contains information about the input range of the first operator in the current round and the previous round, and the second input range contains information about the input range of the second operator in the current round and the previous round. The computer device then updates the quantization parameters of each first operator based on its first input range, second input range, and target output range, thereby increasing the amount of information referenced when updating the quantization parameters of the first operators.

[0129] For example, the computer device performs weighted processing on the minimum value of each first operator in the current round's input range and the minimum value in the previous round's input range to obtain a weighted minimum value. Similarly, the computer device performs weighted processing on the maximum value of each first operator in the current round's input range and the maximum value in the previous round's input range to obtain a weighted maximum value. The range between the weighted minimum value and the weighted maximum value is defined as the first input range. The second input range is determined in the same way as the first input range.

[0130] For example, the computer device uses the third weight parameter corresponding to the current round and the fourth weight parameter corresponding to the previous round to weight the input range of each first operator in the current round and the input range of the previous round, thus obtaining the first input range of each first operator. The computer device uses the same third and fourth weight parameters to weight the input range of the second operator in the current round and the input range of the previous round, thus obtaining the second input range of the second operator. Optionally, the sum of the third and fourth weight parameters is 1, and both the third and fourth weight parameters are values ​​between 0 and 1. For example, both the third and fourth weight parameters are 0.5.

[0131] In another possible implementation, the computer device updates the quantization parameters of each first operator by: determining a first weight parameter corresponding to each first operator and a second weight parameter corresponding to a second operator; weighting the first input range and the second input range of each first operator based on the first weight parameter and the second weight parameter to obtain the target input range of each first operator; and updating the quantization parameters of each first operator based on the target input range and the target output range of each first operator so that the updated quantization parameters can quantize the data in the target input range to the target output range.

[0132] The first weight parameter is negatively correlated with the number of times the quantization parameters are updated, while the second weight parameter is positively correlated with the number of times the quantization parameters are updated. The fewer the number of times the quantization parameters are updated, the larger the first weight parameter and the smaller the second weight parameter. Conversely, the more the number of times the quantization parameters are updated, the smaller the first weight parameter and the larger the second weight parameter. Optionally, the sum of the first and second weight parameters is 1, and both the first and second weight parameters are values ​​between 0 and 1. For example, in the first update of the quantization parameters, the first weight parameter is 1 and the second weight parameter is 0; in the last update, the first weight parameter is 0 and the second weight parameter is 1. That is, in the initial stage of updating the quantization parameters, when determining the target input range, the first input range of the first operator has a larger weight, and the second input range of the second operator has a smaller weight. As the quantization parameters are continuously updated, the weight of the first input range of the first operator gradually decreases, and the weight of the second input range of the second operator gradually increases, so that under the constraint of the input range of the second operator, the quantization parameters of the first and second operators gradually converge to the same value.

[0133] In another possible implementation, the computer device obtains the target input range for each first operator by: for each first operator, weighting the minimum value in the first input range and the minimum value in the second input range based on the first weight parameter and the second weight parameter of the first operator to obtain a target minimum value; weighting the maximum value in the first input range and the maximum value in the second input range based on the first weight parameter and the second weight parameter to obtain a target maximum value; and determining the range between the target minimum value and the target maximum value as the target input range of the first operator.

[0134] For example, computer equipment uses the following formula to determine the target minimum and target maximum values:

[0135]

[0136]

[0137] in, xmin4 represents the minimum target value corresponding to the first operator, xmin4 represents the minimum value in the second input range of the second operator, and xmin1 represents the minimum value in the first input range of the first operator. Let xmax4 represent the target maximum value corresponding to the first operator, xmax4 represent the maximum value in the second input range of the second operator, and xmax1 represent the maximum value in the first input range of the first operator. α represents the second weight parameter, and 1-α represents the first weight parameter. α increases with the number of times the quantization parameters are updated. The minimum value of α is 0, and the maximum value of α is 1.

[0138] In another possible implementation, the first operator includes a first quantization parameter, and the computer device updates the quantization parameter of each first operator by: for each first operator, determining a first difference between the maximum and minimum values ​​in the target input range of the first operator, determining a second difference between the maximum and minimum values ​​in the target output range, and determining the ratio between the first difference and the second difference as the updated first quantization parameter of the first operator.

[0139] The first quantization parameter indicates the scaling factor of the data. In this embodiment, the first operator includes a quantization operator and an inverse quantization operator. The first operator can quantize and inverse quantize any data. Quantizing any data includes shrinking the data, thereby mapping the data from one numerical range to another. Inverse quantization of any data includes amplifying the quantized data, thereby mapping the quantized data from the other numerical range back to the original numerical range. The first quantization parameter is also the scaling factor for the data.

[0140] For example, computer devices use the following formula to determine the first quantization parameter:

[0141]

[0142] Where scale represents the first quantization parameter, max_val represents the maximum value in the target input range, min_val represents the minimum value in the target input range, qmax represents the maximum value in the target output range, and qmin represents the minimum value in the target output range.

[0143] Optionally, the first operator further includes a second quantization parameter. The computer device updates the quantization parameter of each first operator by: taking the integer part of the ratio between the minimum value in the target input range and the updated first quantization parameter to obtain a reference minimum value, and determining the difference between the minimum value in the target output range and the reference minimum value as the updated second quantization parameter of the first operator.

[0144] The second quantization parameter indicates the magnitude of the data shift. In this embodiment, quantizing and dequantizing any data further includes shifting the data, thereby mapping it from one numerical range to another, that is, offsetting the data from one numerical range to another. The second quantization parameter is the magnitude of the data shift.

[0145] For example, computer devices use the following formula to determine the second quantization parameter:

[0146]

[0147] Where zero_point represents the second quantization parameter, qmin represents the minimum value in the target output range, min_val represents the minimum value in the target input range, scale represents the first quantization parameter, and round(·) represents the rounding function.

[0148] The aforementioned first and second quantization parameters can quantize data from the target input range to the target output range. In one possible implementation, the first and second operators are pseudo-quantization operators, which include quantization operators and inverse quantization operators. The first and second quantization parameters are the parameters of the quantization operator and the inverse quantization parameter, respectively. The expression formula for the quantization operator is as follows:

[0149]

[0150] Among them, X F X represents the input data of the quantization operator. QThis represents the output data of the quantization operator. `scale` represents the first quantization parameter, `round(·)` represents the rounding function, `zero_point` represents the second quantization parameter, `qmin` represents the minimum value in the target output range, and `qmax` represents the maximum value in the target output range. `clamp(·)` represents the interval constraint function used to quantize X... Q Limited to between qmin and qmax. Optionally, the X... F For floating-point data, X Q The data is integer.

[0151] The formula for the inverse quantization operator is as follows:

[0152] X F1 =(X Q -zero_point)*scale;

[0153] Among them, X Q X represents the input data of the dequantization operator. F1 X represents the output data of the inverse quantization operator. F1 With X F There is an error caused by quantization. `scale` represents the first quantization parameter, and `zero_point` represents the second quantization parameter.

[0154] 304. The computer device updates the quantization parameters of the second operator based on the input range of the second operator in the current round, the input range of the previous round, and the target output range, until the quantization parameters of each first operator and second operator converge to the quantization parameters corresponding to the target output range.

[0155] In addition to determining the input range of each operator in this round, the computer device also obtains the input range of each operator in the previous round, and updates the quantization parameters of the second operator based on the input range of the second operator in this round, the input range of the previous round, and the target output range.

[0156] It should be noted that steps 302-304 above only illustrate the process of updating the quantization parameters of the first and second operators using only one round as an example. However, in this embodiment, multiple rounds are executed to update the quantization parameters of each operator multiple times until the quantization parameters of each first and second operator converge to the quantization parameters corresponding to the target output range. The quantization parameters corresponding to the target output range are used to quantize any data to the target output range.

[0157] In one possible implementation, the computer device updates the quantization parameters of the second operator by: weighting the input range of the second operator in the current round and the input range of the previous round to obtain a second input range of the second operator; and updating the quantization parameters of the second operator based on the second input range and the target output range, so that the updated quantization parameters can quantize the data in the second input range to the target output range.

[0158] The process by which the computer device updates the quantization parameters of the second operator based on the second input range and the target output range is the same as the process in step 303 above that updates the quantization parameters of the first operator based on the target input range and the target output range, and will not be described again here.

[0159] By executing steps 303-304 above, the computer device updates the quantization parameters of each operator based on the input range of each operator in the current round, the input range of the previous round, and the target output range, so that the quantization parameters of each operator converge to the quantization parameters corresponding to the target output range. Alternatively, the computer device can update the quantization parameters in a different manner than steps 303-304 above. For example, the computer device can determine a target first operator among multiple first operators and constrain the quantization parameters of other first and second operators based on the input range of this target first operator.

[0160] 305. The computer equipment performs quantization processing on the network parameters in the splicing layer based on the quantization parameters corresponding to the target output range.

[0161] The computer device quantizes the network parameters of the splicing layer to the target output range based on the quantization parameters corresponding to the target output range.

[0162] In one possible implementation, the quantization parameters include the first quantization parameter and the second quantization parameter shown in step 303 above. The formula for the computer device to quantize the network parameters in the splicing layer is as follows:

[0163]

[0164] Among them, X C X represents the network parameters of the splicing layer. L This represents the quantized network parameters. `scale` represents the first quantization parameter, `round(·)` represents the rounding function, `zero_point` represents the second quantization parameter, `qmin` represents the minimum value in the target output range, and `qmax` represents the maximum value in the target output range. `clamp(·)` represents the interval constraint function used to quantize X... L It is limited to between qmin and qmax.

[0165] By executing steps 301-305 above, multiple input data of the stitching layer can be made to fall within the target output range. By executing step 306, the network parameters in the stitching layer can also be quantized to the target output range. Therefore, the data processed by the stitching layer is all within the target output range, thereby realizing the quantization of the stitching layer.

[0166] It should be noted that, in another embodiment, both the first operator and the second operator are pseudo-quantization operators. The first operator includes a first quantization operator and a first inverse quantization operator, and the second operator includes a second quantization operator and a second inverse quantization operator. The quantization parameters of the updated first and second operators are the quantization parameters corresponding to the target output range. After performing step 304 above, the computer device deletes the first inverse quantization operator in each first operator and deletes the second quantization operator in each second operator, retaining the first quantization operator in the first operator and the second inverse quantization operator in the second operator.

[0167] Since the updated quantization parameters of the first quantization operator can quantize the input data to the target output range, and the output data of the first quantization operator is the input data of the concatenation layer, all multiple input data of the concatenation layer belong to the target output range. Because the concatenation layer concatenates quantized data, after the concatenation layer concatenates the data, the second inverse quantization operator inverse-quantizes the quantized data. This allows subsequent network layers to process the inverse-quantized data, thereby reducing errors caused by quantization processing.

[0168] It should be noted that the computer device quantizes the network parameters in the stitching layer to obtain the quantized first model. The computer device then uses this quantized first model to process the data. Since multiple input data points in the stitching layer of the quantized first model all fall within the target output range, the stitching layer can directly stitch together multiple input data points belonging to the same range. Therefore, it is unnecessary to first convert multiple input data points into floating-point data, simplifying the operation process and improving the model's processing efficiency.

[0169] The quantized first model can be of any type, and different types of first models can perform different tasks.

[0170] For example, the first quantized model is an image segmentation model used to segment the sky region in an image. A computer device takes a sky image including the sky, calls the image segmentation model to perform image segmentation on the sky image, and obtains the sky region in the sky image.

[0171] For example, the quantized first model is a face recognition model, which is used to identify whether a face in an image is the target face. A computer device acquires a face image, calls the face recognition model, performs face recognition on the face image, and obtains a face recognition result, which indicates whether the face in the face image is the target face.

[0172] For example, the first quantized model is an image classification model used to classify landscape images and portrait images. A computer device acquires the image to be detected from an image library, calls the image classification model, classifies the image, and obtains a classification result indicating whether the image to be detected is a landscape image or a portrait image.

[0173] The method provided in this application updates the quantization parameters of each operator by using the input range and target output range of each operator, so that the quantization parameters of each operator converge to the quantization parameters corresponding to the target output range. In this way, the output range of multiple operators is constrained to be consistent by using the target output range, so that the output range of each operator is consistent. This is equivalent to ensuring that the multiple input ranges of the splicing layer and the range of network parameters are the same, thereby completing the quantization of the splicing layer.

[0174] Furthermore, since the test data used in each round is different, the input range determined in each round is also different. In each round, the computer device combines the input range of the current round with the input range of the previous round to update the quantization parameters of the operator. This is equivalent to considering the test data in two rounds, which increases the amount of information referenced when updating the quantization parameters of the operator. This is beneficial to improving the accuracy of updating the quantization parameters and also to improving the convergence speed of the quantization parameters.

[0175] Furthermore, updating the quantization parameters of the first operator based on the input range of the first and second operators and the target output range is equivalent to broadcasting the input range of the second operator to each first operator. Thus, the input range of the second operator is used to constrain the quantization parameters of the first operator, so that the quantization parameters of the first and second operators converge to the same value as soon as possible, thereby further improving the convergence speed of the quantization parameters.

[0176] Furthermore, as the quantization parameters are continuously updated, the proportion of the first input range of the first operator gradually increases when determining the target input range, while the proportion of the second input range of the second operator gradually decreases. This results in the quantization parameters of the first and second operators gradually converging to the same value under the constraint of the input range of the second operator.

[0177] Figure 3The embodiments illustrate quantizing the stitching layer of the first model by updating the quantization parameters of the operator through multiple rounds. Alternatively, the computer device can quantize the stitching layer of the first model by performing a single operation to update the quantization parameters, thus eliminating the need to reference the input range of other rounds. For example, the computer device will... Figure 3 In the embodiments, the test data used in multiple rounds are all used in a single update of the quantization parameters to complete the quantization. See also... Figure 6 , Figure 6 This is a flowchart of a model quantization method provided in an embodiment of this application. This embodiment uses a first pseudo-quantization operator as the first operator and a second pseudo-quantization operator as an example for illustration. The execution subject of this embodiment is a computer device, and the method includes:

[0178] 601. Computer equipment obtains the first model.

[0179] 602. The computer device determines the input range of each pseudo-quantization operator among a plurality of first pseudo-quantization operators and second pseudo-quantization operators.

[0180] The processes of steps 601-602 above are the same as those of steps 301-302 above, and will not be repeated here.

[0181] 603. The computer device updates the quantization parameters of each first pseudo-quantization operator based on the input range of each first pseudo-quantization operator, the input range of the second pseudo-quantization operator, and the target output range.

[0182] The computer device uses the input range of the second pseudo-quantization operator to constrain the quantization parameters of the first pseudo-quantization operator, so that the quantization parameters of the first pseudo-quantization operator and the second pseudo-quantization operator converge to the same value.

[0183] In one possible implementation, a first weight parameter corresponding to each first pseudo-quantization operator and a second weight parameter corresponding to each second pseudo-quantization operator are determined. Based on the first and second weight parameters, the input ranges of each first pseudo-quantization operator and the second pseudo-quantization operator are weighted to obtain the target input range of each first pseudo-quantization operator. Based on the target input range and target output range of each first pseudo-quantization operator, the quantization parameters of each first pseudo-quantization operator are updated so that the updated quantization parameters can quantize the data in the target input range to the target output range. The process of determining the target input range based on the input ranges of the first and second pseudo-quantization operators is the same as the process of determining the target input range based on the first and second input ranges in step 303 above, and will not be repeated here. The process of updating the quantization parameters based on the target input range and target output range is the same as the process of updating the quantization parameters based on the target input range and target output range in step 303 above, and will not be repeated here.

[0184] 604. The computer device updates the quantization parameters of the second pseudo-quantization operator based on the input range and the target output range of the second pseudo-quantization operator, so that the quantization parameters of each first pseudo-quantization operator and the second pseudo-quantization operator converge to the quantization parameters corresponding to the target output range.

[0185] The process of updating the quantization parameters of the second pseudo-quantization operator in step 604 based on the input range and target output range is the same as the process of updating the quantization parameters of the second pseudo-quantization operator based on the second input range and target output range in step 304 above, and will not be described again here.

[0186] By executing steps 603-604 above, the computer device updates the quantization parameters of each operator based on the input range and target output range of each operator, so that the quantization parameters of each operator converge to the quantization parameters corresponding to the target output range. Alternatively, the computer device can update the quantization parameters in a different manner than steps 603-604. For example, the computer device can determine a target first pseudo-quantization operator among multiple first pseudo-quantization operators, and constrain the quantization parameters of other first pseudo-quantization operators and second pseudo-quantization operators based on the input range of this target first pseudo-quantization operator.

[0187] 605. The computer equipment deletes the first inverse quantization operator in each first pseudo-quantization operator and deletes the second quantization operator in the second pseudo-quantization operator.

[0188] 606. The computer equipment performs quantization processing on the network parameters in the splicing layer based on the quantization parameters corresponding to the target output range.

[0189] The process of steps 605-606 above is the same as that of step 305 above, and will not be repeated here.

[0190] The method provided in this application updates the quantization parameters of each operator by using the input range and target output range of each operator, so that the quantization parameters of each operator converge to the quantization parameters corresponding to the target output range. In this way, the output range of multiple operators is constrained to be consistent by using the target output range, so that the output range of each operator is consistent. This is equivalent to ensuring that the multiple input ranges of the splicing layer and the range of network parameters are the same, thereby completing the quantization of the splicing layer.

[0191] In the above Figure 3 and Figure 6 In one embodiment, with the first and second operators in the first model enabled, the quantization parameters of the first and second operators are updated. In another embodiment, to ensure the accuracy of the defined input range, the first and second operators can be disabled first, an update operation performed, and then the first and second operators enabled again for another update operation, such as... Figure 7 As shown, Figure 7 This is a flowchart of a model quantization method provided in an embodiment of this application. The execution subject of this embodiment is a computer device, and the method includes:

[0192] 701. Computer equipment acquires the first model.

[0193] The process of step 701 is the same as that of step 301 above, and will not be repeated here.

[0194] 702. The computer device shuts down each first operator and second operator in the first model, determines the multiple input ranges of the splicing layer as the third input range of the corresponding first operator, and determines the output range of the splicing layer as the third input range of the second operator.

[0195] The computer device disables each first and second operator in the first model. When the first model is invoked to process data, the output data of the network layer preceding the first operator is directly processed by the concatenation layer. The first operator does not process the output data of the network layer preceding it, and the second operator does not process the output data of the concatenation layer. Instead, the network layer following the second operator continues to process the output data of the concatenation layer. For example, in Figure 5 In the first model shown, after turning off each first operator and second operator, that is, after turning off each pseudo-quantization operator, the output data of convolutional layer 501, convolutional layer 502 and convolutional layer 503 are directly input into the splicing layer 504.

[0196] After the computer device closes the first and second operators, it determines the multiple input ranges and output range of the splicing layer. Since the splicing layer corresponds to multiple input data, the numerical ranges of these input data constitute the multiple input ranges of the splicing layer. These multiple input data correspond one-to-one with multiple first operators; therefore, the multiple input ranges of the splicing layer can be respectively determined as the third input range of the corresponding first operator. Since the output data of the splicing layer should be the input data of the second operator, the output range of the splicing layer can be determined as the third input range of the second operator. For example, in... Figure 5 In the first model shown, the output data of each convolutional layer is the input data of the splicing layer 504. The splicing layer 504 has three input data. The output data of the convolutional layer 501 corresponds to the pseudo-quantization operator 505, the output data of the convolutional layer 502 corresponds to the pseudo-quantization operator 506, and the output data of the convolutional layer 503 corresponds to the pseudo-quantization operator 507.

[0197] In one possible implementation, the computer device acquires multiple test data sets, processes these test data sets by calling a first model that disables the first and second operators, and determines multiple input ranges and the output range of the splicing layer. This step is similar to the process of determining the input range of each operator in step 302 above, and will not be repeated here.

[0198] 703. The computer device updates the quantization parameters of each first operator and the second operator based on the third input range of each first operator, the third input range of the second operator, and the target output range.

[0199] The process of step 703 is the same as that of steps 603-604 above, and will not be repeated here.

[0200] 704. The computer device opens each first operator and second operator in the first model and determines the fourth input range of each first operator and second operator.

[0201] After closing the first and second operators, the computer device updates the quantization parameters of the first and second operators, and then reopens each of the first and second operators in the first model. When the first model is called to process the data, the output data of the network layer before the first operator is first processed by the splicing layer, and then the splicing layer processes the output data of the first operator. The output data of the splicing layer is first processed by the second operator, and then the network layer after the second operator continues to process the output data of the second operator. After the computer device reopens each first and second operator, it redetermines the fourth input range of each first and second operator. This step is the same as the process of determining the input range of each operator in step 302 above, and will not be repeated here.

[0202] 705. The computer device continues to update the quantization parameters of each first operator and the second operator based on the fourth input range of each first operator, the fourth input range of the second operator, and the target output range, so that the quantization parameters of each first operator and the second operator converge to the quantization parameters corresponding to the target output range.

[0203] The quantization parameter corresponding to the target output range is used to quantize any data to the target output range. This step is the same as the process of steps 603-604 above, and will not be repeated here.

[0204] In this embodiment, when the first and second operators are closed, they do not process the data. Therefore, the third input range determined by the first and second operators in this case is not affected by the first and second operators themselves, which reduces the error caused by the range of the data by the first and second operators. As a result, the third input range determined by the computer device is more accurate, and the quantization parameters updated based on the third input range are also more accurate. Then, the first and second operators with updated quantization parameters are turned on, and the data is processed using more accurate quantization parameters to obtain a new input range, thereby continuing to update the quantization parameters. This helps to improve the accuracy of the entire quantization parameter update process. For example, in the embodiments of this application, the first operator and the second operator are pseudo-quantization operators. Pseudo-quantization operators include quantization operators and dequantization operators. Therefore, both the first operator and the second operator need to quantize the data first and then dequantize it. In this process, errors caused by quantization will occur, resulting in an inaccurate input range. Therefore, the embodiments of this application adopt the method of turning off the first operator and the second operator first to determine the input range of the first operator and the second operator, which can avoid the errors caused by quantization and make the update process of quantization parameters more accurate.

[0205] It should be noted that this embodiment only illustrates one round. In another embodiment, the computer device repeatedly executes steps 702-705 to perform multiple rounds, updating the quantization parameters of the first and second operators multiple times so that the quantization parameters of each first and second operator converge to the quantization parameters corresponding to the target output range. Furthermore, when updating the quantization parameters based on the input range in each round, the input range of the current round and the input range of the previous round can be comprehensively considered. The detailed process is the same as described above. Figure 3 The process of updating based on the input range of two rounds in the embodiment is similar and will not be described again here.

[0206] 706. The computer equipment performs quantization processing on the network parameters in the splicing layer based on the quantization parameters corresponding to the target output range.

[0207] The process of step 706 above is the same as that of step 305 above, and will not be repeated here.

[0208] The method provided in this application updates the quantization parameters of each operator by using the input range and target output range of each operator, so that the quantization parameters of each operator converge to the quantization parameters corresponding to the target output range. In this way, the output range of multiple operators is constrained to be consistent by using the target output range, so that the output range of each operator is consistent. This is equivalent to ensuring that the multiple input ranges of the splicing layer and the range of network parameters are the same, thereby completing the quantization of the splicing layer.

[0209] The above embodiments can be applied to quantize any type of first model including a stitching layer. For example, the first model is an image segmentation model, which is used to segment an image. Figure 8 This is a flowchart of a model quantization method provided in an embodiment of this application. See also... Figure 8 The method includes:

[0210] 801. The computer device acquires the trained image segmentation model.

[0211] 802. The computer device inserts a first pseudo-quantization operator before the stitching layer of the image segmentation model and a second pseudo-quantization operator after the stitching layer.

[0212] 803. For each of multiple rounds, the computer device determines the input range of each pseudo-quantization operator among multiple first pseudo-quantization operators and second pseudo-quantization operators.

[0213] The computer device processes multiple test data sets by calling an image segmentation model to determine the input range of each pseudo-quantization operator. For example, the multiple test data sets are multiple images captured by a camera, each image including a human figure region and a background region. The image segmentation model segments the images, separating the human figure region and the background region from the image.

[0214] 804. The computer device updates the quantization parameters of each first pseudo-quantization operator based on the input range of each first pseudo-quantization operator in the current round and the input range of the previous round, the input range of the second pseudo-quantization operator in the current round and the input range of the previous round, and the target output range.

[0215] 805. The computer device updates the quantization parameters of the second pseudo-quantization operator based on the input range of the second pseudo-quantization operator in the current round, the input range of the previous round, and the target output range, until the quantization parameters of each first pseudo-quantization operator and the second pseudo-quantization operator converge to the quantization parameters corresponding to the target output range.

[0216] 806. The computer equipment deletes the inverse quantization operator in each first pseudo-quantization operator, deletes the quantization operator in each second pseudo-quantization operator, and retains the quantization operator in each first pseudo-quantization operator and the inverse quantization operator in each second pseudo-quantization operator.

[0217] 807. The computer device quantizes the network parameters in the stitching layer based on the quantization parameters corresponding to the target output range, and quantizes the network parameters to the target output range to obtain the quantized image segmentation model.

[0218] 808. The computer device calls the quantized image segmentation model to perform the image segmentation task.

[0219] For example, when an image segmentation model is used to segment the human figure region in an image, the computer device uses a camera to capture the original image containing the human figure, calls the quantized image segmentation model, and segments the original image to obtain a target image containing only the human figure region, thereby achieving image segmentation.

[0220] In this embodiment, to improve the running efficiency of the stitching layer in the image segmentation model, pseudo-quantization operators are inserted before and after the stitching layer for quantization-aware training. The input range of the second pseudo-quantization operator after the stitching layer is broadcast to the first pseudo-quantization operator before the stitching layer. Thus, the input range of the second pseudo-quantization operator is used to constrain the input range of each first pseudo-quantization operator, making the quantization parameters of each first pseudo-quantization operator and the second pseudo-quantization operator the same. As a result, multiple integer data input to the stitching layer all belong to the same range, eliminating the need to convert integer data to floating-point data, which helps improve the processing efficiency of the model.

[0221] In addition, the first model can also be any type of neural network model such as a face recognition model, an image classification model, a speech recognition model, or an image denoising model. This application does not limit this type of model.

[0222] To verify the feasibility of the embodiments of this application, a floating-point model for image segmentation was obtained. The floating-point model includes at least one stitching layer. The method of the embodiments of this application was used to quantize the floating-point model to obtain an 8-bit quantized model. The floating-point model and the 8-bit quantized model were then used to process the same batch of data. The processing results are shown in Table 1 below.

[0223] Table 1

[0224] Model Outline boundary IOU Overall IOU Floating-point model 0.5857 0.8336 8-bit quantization model 0.5861 0.8365

[0225] Among them, the 8-bit quantization model has a higher processing speed than the floating-point model, and IOU (Intersection over Union) is a standard for measuring accuracy. As can be seen from Table 1, the accuracy of the 8-bit quantization model is higher than that of the floating-point model.

[0226] Figure 9 This is a schematic diagram of a model quantization device provided in an embodiment of this application. See also... Figure 9 The device includes:

[0227] The model acquisition module 901 is used to acquire a first model, which includes multiple first operators, a splicing layer, and a second operator. The output of the first operator is the input of the splicing layer, and the output of the splicing layer is the input of the second operator.

[0228] The parameter update module 902 is used to determine the input range of each of the multiple first operators and second operators, and update the quantization parameter of each operator based on the input range of each operator and the target output range, so that the quantization parameter of each operator converges to the quantization parameter corresponding to the target output range. The quantization parameter corresponding to the target output range is used to quantize any data to the target output range.

[0229] The quantization processing module 903 is used to quantize the network parameters in the splicing layer based on the quantization parameters corresponding to the target output range.

[0230] The model quantization device provided in this application update the quantization parameters of each operator by using the input range and target output range of each operator, so that the quantization parameters of each operator converge to the quantization parameters corresponding to the target output range. In this way, the output range of multiple operators is constrained by the target output range, so that the output range of each operator is consistent. This is equivalent to ensuring that the multiple input ranges of the splicing layer and the range of network parameters are the same, thereby completing the quantization of the splicing layer.

[0231] Optionally, see Figure 10 The parameter update module 902 includes:

[0232] The first parameter update unit 912 is used to update the quantization parameters of each first operator based on the input range of each first operator, the input range of the second operator, and the target output range.

[0233] The second parameter update unit 922 is used to update the quantization parameters of the second operator based on the input range and the target output range of the second operator, so that the quantization parameters of each first operator and second operator converge to the quantization parameters corresponding to the target output range.

[0234] Optionally, see Figure 10The parameter update module 902 includes:

[0235] The range determination unit 932 is used to determine the input range of each operator in each of the multiple rounds.

[0236] The third parameter update unit 942 is used to update the quantization parameters of each operator based on the input range of each operator in the round, the input range of the previous round, and the target output range, so that the quantization parameters of each operator converge to the quantization parameters corresponding to the target output range.

[0237] Optionally, see Figure 10 The range determination unit 932 is used to process multiple test data corresponding to each round by calling the first model to determine the input range of each operator in each round. The multiple test data corresponding to each round are different.

[0238] Optionally, see Figure 10 The third parameter update unit 942 is used for:

[0239] Based on the input range of each first operator in the current round and the input range of the previous round, the input range of the second operator in the current round and the input range of the previous round, and the target output range, update the quantization parameters of each first operator;

[0240] Based on the input range of the second operator in the current round, the input range of the previous round, and the target output range, update the quantization parameters of the second operator until the quantization parameters of each first operator and second operator converge to the quantization parameters corresponding to the target output range.

[0241] Optionally, see Figure 10 The third parameter update unit 942 is used for:

[0242] The input range of each first operator in the current round and the input range of the previous round are weighted to obtain the first input range of each first operator;

[0243] The second input range of the second operator is obtained by weighting the input range of the second operator in the current round and the input range of the previous round.

[0244] The quantization parameters of each first operator are updated based on the first input range, the second input range, and the target output range of each first operator.

[0245] Optionally, see Figure 10 The third parameter update unit 942 is used for:

[0246] Determine the first weight parameter corresponding to each first operator and the second weight parameter corresponding to each second operator. The first weight parameter is negatively correlated with the number of times the quantization parameter is updated, and the second weight parameter is positively correlated with the number of times.

[0247] Based on the first weight parameter and the second weight parameter, the first input range and the second input range of each first operator are weighted to obtain the target input range of each first operator;

[0248] Based on the target input range and target output range of each first operator, the quantization parameters of each first operator are updated so that the updated quantization parameters can quantize the data in the target input range to the target output range.

[0249] Optionally, see Figure 10 The third parameter update unit 942 is used for:

[0250] For each first operator, based on the first weight parameter and the second weight parameter of the first operator, the minimum value in the first input range and the minimum value in the second input range of the first operator are weighted to obtain the target minimum value;

[0251] Based on the first weight parameter and the second weight parameter, the maximum value in the first input range and the maximum value in the second input range are weighted to obtain the target maximum value;

[0252] The range between the target minimum and the target maximum is defined as the target input range of the first operator.

[0253] Optionally, see Figure 10 The first operator includes a first quantization parameter, and the third parameter update unit 942 is used for:

[0254] For each first operator, determine the first difference between the maximum and minimum values ​​in the target input range of the first operator;

[0255] Determine the second difference between the maximum and minimum values ​​within the target output range;

[0256] The ratio between the first difference and the second difference is determined as the updated first quantization parameter of the first operator, which indicates the scaling factor of the data.

[0257] Optionally, see Figure 10 The first operator also includes a second quantization parameter and a third parameter update unit 942, used for:

[0258] The reference minimum value is obtained by rounding down the ratio between the minimum value in the target input range and the updated first quantization parameter.

[0259] The difference between the minimum value in the target output range and the reference minimum value is determined as the updated second quantization parameter of the first operator. The second quantization parameter indicates the displacement magnitude of the data.

[0260] Optionally, see Figure 10 The third parameter update unit 942 is used for:

[0261] The second input range of the second operator is obtained by weighting the input range of the second operator in the current round and the input range of the previous round.

[0262] Based on the second input range and the target output range of the second operator, the quantization parameters of the second operator are updated so that the updated quantization parameters can quantize the data in the second input range to the target output range.

[0263] Optionally, see Figure 10 The parameter update module 902 includes:

[0264] The range determination unit 932 is used to turn off each first operator and second operator in the first model, determine the multiple input ranges of the splicing layer as the third input range of the corresponding first operator, and determine the output range of the splicing layer as the third input range of the second operator.

[0265] The third parameter update unit 942 is used to update the quantization parameters of each first operator and the second operator based on the third input range of each first operator, the third input range of the second operator, and the target output range.

[0266] The range determination unit 932 is also used to open each first operator and second operator in the first model and determine the fourth input range of each first operator and second operator;

[0267] The third parameter update unit 942 is used to continue updating the quantization parameters of each first operator and the second operator based on the fourth input range of each first operator, the fourth input range of the second operator, and the target output range, so that the quantization parameters of each first operator and the second operator converge to the quantization parameters corresponding to the target output range.

[0268] Optionally, see Figure 10 The first operator includes a first quantization operator and a first inverse quantization operator, and the device further includes:

[0269] The operator adjustment module 904 is used to delete the first inverse quantization operator in each first operator and delete the second quantization operator in the second operator.

[0270] Optionally, see Figure 10 The parameter update module 902 includes:

[0271] The range determination unit 932 is used to obtain the association information of the splicing layer, the association information indicating the first operator located before the splicing layer and multiple second operators located after the splicing layer;

[0272] The range determination unit 932 is also used to determine multiple first operators and second operators based on the associated information;

[0273] The range determination unit 932 is also used to determine the input range of multiple first operators and the input range of second operators.

[0274] Optionally, see Figure 10 The model acquisition module 901 includes:

[0275] The model acquisition unit 911 is used to acquire a second model, which includes multiple network layers connected in sequence. The multiple network layers include at least a splicing layer and multiple first network layers. The first network layers are network layers connected to the splicing layer and located before the splicing layer.

[0276] Operator insertion unit 921 is used to insert a first operator between the splicing layer and each first network layer, and to insert a second operator after the splicing layer to obtain the first model.

[0277] It should be noted that the model quantization device provided in the above embodiments is only illustrated by the division of the above functional modules when quantizing the model. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the model quantization device and the model quantization method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0278] This application also provides a computer device, which includes a processor and a memory. The memory stores at least one computer program, which is loaded and executed by the processor to implement the operations performed in the model quantization method of the above embodiments.

[0279] Optionally, the computer device is provided as a terminal. Figure 11 A schematic diagram of the structure of a terminal 1100 provided in an exemplary embodiment of this application is shown.

[0280] Terminal 1100 includes a processor 1101 and a memory 1102.

[0281] Processor 1101 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1101 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1101 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1101 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1101 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0282] The memory 1102 may include one or more computer-readable storage media, which may be non-transitory. The memory 1102 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1102 are used to store at least one computer program, which is used by the processor 1101 to implement the model quantization method provided in the method embodiments of this application.

[0283] In some embodiments, the terminal 1100 may also optionally include: a peripheral device interface 1103 and at least one peripheral device. The processor 1101, memory 1102, and peripheral device interface 1103 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1103 via a bus, signal line, or circuit board. Optionally, the peripheral device includes at least one of: a radio frequency circuit 1104, a display screen 1105, a camera assembly 1106, and an audio circuit 1107.

[0284] Peripheral device interface 1103 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 1101 and memory 1102. In some embodiments, processor 1101, memory 1102 and peripheral device interface 1103 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 1101, memory 1102 and peripheral device interface 1103 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0285] The radio frequency (RF) circuit 1104 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1104 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1104 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 1104 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 1104 can communicate with other devices via at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: metropolitan area networks (MANs), various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks (WLANs), and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1104 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0286] Display screen 1105 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 1105 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 1101 for processing. In this case, display screen 1105 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 1105, disposed on the front panel of terminal 1100; in other embodiments, there may be at least two display screens, disposed on different surfaces of terminal 1100 or in a folded design; in still other embodiments, display screen 1105 may be a flexible display screen, disposed on a curved or folded surface of terminal 1100. Furthermore, display screen 1105 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. The display screen 1105 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0287] The camera assembly 1106 is used to acquire images or videos. Optionally, the camera assembly 1106 includes a front-facing camera and a rear-facing camera. The front-facing camera is disposed on the front panel of the terminal 1100, and the rear-facing camera is disposed on the back of the terminal 1100. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 1106 may also include a flash. The flash may be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cool light flash, which can be used for light compensation at different color temperatures.

[0288] The audio circuit 1107 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 1101 for processing, or input to the radio frequency circuit 1104 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each positioned at a different location on the terminal 1100. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 1101 or the radio frequency circuit 1104 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 1107 may also include a headphone jack.

[0289] Those skilled in the art will understand that Figure 11 The structure shown does not constitute a limitation on terminal 1100 and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0290] Optionally, the computer device is provided as a server. Figure 12 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 1200 can vary significantly due to different configurations or performance. It may include one or more Central Processing Units (CPUs) 1201 and one or more memories 1202. The memories 1202 store at least one computer program, which is loaded and executed by the processor 1201 to implement the methods provided in the above-described method embodiments. Of course, the server may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server may also include other components for implementing device functions, which will not be elaborated upon here.

[0291] This application also provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to implement the operations performed in the model quantization method of the above embodiments.

[0292] This application also provides a computer program product or computer program, which includes computer program code stored in a computer-readable storage medium. A processor of a computer device reads the computer program code from the computer-readable storage medium and executes the computer program code, causing the computer device to perform the operations performed in the model quantization method of the above embodiments. In some embodiments, the computer program involved in this application can be deployed and executed on a single computer device, or on multiple computer devices located in one location, or on multiple computer devices distributed across multiple locations and interconnected via a communication network. These multiple computer devices distributed across multiple locations and interconnected via a communication network can constitute a blockchain system.

[0293] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0294] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present application should be included within the protection scope of the present application.

Claims

1. A model quantization method, characterized in that, The method includes: A first model is obtained, which includes multiple first operators, a concatenation layer, and a second operator. The output of the first operator is the input of the concatenation layer, and the output of the concatenation layer is the input of the second operator. The first model is any one of a speech recognition model, an image segmentation model, an image classification model, or a face recognition model. Determine the input range for each of the plurality of first operators and second operators; Based on the input range of each first operator, the input range of the second operator, and the target output range, the quantization parameters of each first operator are updated, and based on the input range of the second operator and the target output range, the quantization parameters of the second operator are updated, so that the quantization parameters of each first operator and the second operator converge to the quantization parameters corresponding to the target output range, wherein the quantization parameters corresponding to the target output range are used to quantize any data to the target output range; Based on the quantization parameters corresponding to the target output range, the network parameters in the splicing layer are quantized.

2. The method according to claim 1, characterized in that, The method further includes: For each of the multiple rounds, determine the input range of each operator in that round; Based on the input range of each operator in the current round, the input range of the previous round, and the target output range, the quantization parameters of each operator are updated so that the quantization parameters of each operator converge to the quantization parameters corresponding to the target output range.

3. The method according to claim 2, characterized in that, Determining the input range of each operator in the round includes: By calling the first model to process multiple test data corresponding to the round, the input range of each operator in the round is determined, and the multiple test data corresponding to each round are different.

4. The method according to claim 2, characterized in that, The step of updating the quantization parameters of each operator based on the input range of each operator in the current round, the input range of the previous round, and the target output range, so that the quantization parameters of each operator converge to the quantization parameters corresponding to the target output range, includes: The quantization parameters of each first operator are updated based on the input range of each first operator in the current round and the input range of the previous round, the input range of the second operator in the current round and the input range of the previous round, and the target output range. Based on the input range of the second operator in the current round, the input range of the previous round, and the target output range, the quantization parameters of the second operator are updated until the quantization parameters of each of the first and second operators converge to the quantization parameters corresponding to the target output range.

5. The method according to claim 4, characterized in that, The step of updating the quantization parameters of each first operator based on the input range of each first operator in the current round and the input range of the previous round, the input range of the second operator in the current round and the input range of the previous round, and the target output range includes: The input range of each first operator in the current round and the input range of the previous round are weighted to obtain the first input range of each first operator; The second operator is weighted by the input range of the current round and the input range of the previous round to obtain the second input range of the second operator; The quantization parameters of each first operator are updated based on the first input range, the second input range, and the target output range of each first operator.

6. The method according to claim 5, characterized in that, The step of updating the quantization parameters of each first operator based on the first input range, the second input range, and the target output range includes: Determine a first weight parameter corresponding to each of the first operators and a second weight parameter corresponding to each of the second operators, wherein the first weight parameter is negatively correlated with the number of times the quantization parameter is updated, and the second weight parameter is positively correlated with the number of times the quantization parameter is updated; Based on the first weight parameter and the second weight parameter, the first input range and the second input range of each first operator are weighted to obtain the target input range of each first operator; Based on the target input range and the target output range of each first operator, the quantization parameters of each first operator are updated so that the updated quantization parameters can quantize the data in the target input range to the target output range.

7. The method according to claim 6, characterized in that, The step of weighting the first input range and the second input range of each first operator based on the first weight parameter and the second weight parameter to obtain the target input range of each first operator includes: For each first operator, based on the first weight parameter and the second weight parameter of the first operator, the minimum value in the first input range and the minimum value in the second input range of the first operator are weighted to obtain the target minimum value; Based on the first weight parameter and the second weight parameter, the maximum value in the first input range and the maximum value in the second input range are weighted to obtain the target maximum value; The range between the target minimum value and the target maximum value is determined as the target input range of the first operator.

8. The method according to claim 6, characterized in that, The first operator includes a first quantization parameter. The step of updating the quantization parameter of each first operator based on the target input range and the target output range, so that the updated quantization parameter can quantize data from the target input range to the target output range, includes: For each of the first operators, determine a first difference between the maximum and minimum values ​​in the target input range of the first operator; Determine the second difference between the maximum and minimum values ​​within the target output range; The ratio between the first difference and the second difference is determined as the updated first quantization parameter of the first operator, whereby the first quantization parameter indicates the scaling factor of the data.

9. The method according to claim 8, characterized in that, The first operator further includes a second quantization parameter. The step of updating the quantization parameter of each first operator based on the target input range and the target output range, so that the updated quantization parameter can quantize data from the target input range to the target output range, includes: The reference minimum value is obtained by rounding down the ratio between the minimum value in the target input range and the updated first quantization parameter. The difference between the minimum value in the target output range and the reference minimum value is determined as the updated second quantization parameter of the first operator, and the second quantization parameter indicates the displacement magnitude of the data.

10. The method according to claim 1, characterized in that, The method further includes: Turn off each of the first operator and the second operator in the first model, determine the multiple input ranges of the splicing layer as the third input range of the corresponding first operator, and determine the output range of the splicing layer as the third input range of the second operator; Based on the third input range of each first operator, the third input range of each second operator, and the target output range, update the quantization parameters of each first operator and the quantization parameters of each second operator; Open each of the first and second operators in the first model, and determine the fourth input range for each of the first and second operators; Based on the fourth input range of each first operator, the fourth input range of the second operator, and the target output range, the quantization parameters of each first operator and the second operator are continuously updated so that the quantization parameters of each first operator and the second operator converge to the quantization parameters corresponding to the target output range.

11. The method according to claim 1, characterized in that, The first operator includes a first quantization operator and a first inverse quantization operator, and the second operator includes a second quantization operator and a second inverse quantization operator; the method further includes: Delete the first inverse quantization operator in each of the first operators, and delete the second quantization operator in the second operators.

12. A model quantization device, characterized in that, The device includes: The model acquisition module is used to acquire a first model, which includes multiple first operators, a concatenation layer, and a second operator. The output of the first operator is the input of the concatenation layer, and the output of the concatenation layer is the input of the second operator. The first model is any one of a speech recognition model, an image segmentation model, an image classification model, or a face recognition model. A parameter update module is used to determine the input range of each of the multiple first operators and second operators; The parameter update module is further configured to update the quantization parameter of each first operator based on the input range of each first operator, the input range of the second operator, and the target output range, and update the quantization parameter of the second operator based on the input range of the second operator and the target output range, so that the quantization parameters of each first operator and the second operator converge to the quantization parameter corresponding to the target output range, wherein the quantization parameter corresponding to the target output range is used to quantize any data to the target output range; The quantization processing module is used to quantize the network parameters in the splicing layer based on the quantization parameters corresponding to the target output range.

13. The apparatus according to claim 12, characterized in that, The parameter update module includes: A range determination unit is used to determine the input range of each operator in each of a plurality of rounds in that round; The third parameter update unit is used to update the quantization parameters of each operator based on the input range of each operator in the current round, the input range of the previous round, and the target output range, so that the quantization parameters of each operator converge to the quantization parameters corresponding to the target output range.

14. The apparatus according to claim 13, characterized in that, The range determination unit is used to process multiple test data corresponding to the round by calling the first model to determine the input range of each operator in the round, wherein the multiple test data corresponding to each round are different.

15. The apparatus according to claim 13, characterized in that, The third parameter update unit is used for: The quantization parameters of each first operator are updated based on the input range of each first operator in the current round and the input range of the previous round, the input range of the second operator in the current round and the input range of the previous round, and the target output range. Based on the input range of the second operator in the current round, the input range of the previous round, and the target output range, the quantization parameters of the second operator are updated until the quantization parameters of each of the first and second operators converge to the quantization parameters corresponding to the target output range.

16. The apparatus according to claim 15, characterized in that, The third parameter update unit is used for: The input range of each first operator in the current round and the input range of the previous round are weighted to obtain the first input range of each first operator; The second operator is weighted by the input range of the current round and the input range of the previous round to obtain the second input range of the second operator; The quantization parameters of each first operator are updated based on the first input range, the second input range, and the target output range of each first operator.

17. The apparatus according to claim 16, characterized in that, The third parameter update unit is used for: Determine a first weight parameter corresponding to each of the first operators and a second weight parameter corresponding to each of the second operators, wherein the first weight parameter is negatively correlated with the number of times the quantization parameter is updated, and the second weight parameter is positively correlated with the number of times the quantization parameter is updated; Based on the first weight parameter and the second weight parameter, the first input range and the second input range of each first operator are weighted to obtain the target input range of each first operator; Based on the target input range and the target output range of each first operator, the quantization parameters of each first operator are updated so that the updated quantization parameters can quantize the data in the target input range to the target output range.

18. The apparatus according to claim 17, characterized in that, The third parameter update unit is used for: For each first operator, based on the first weight parameter and the second weight parameter of the first operator, the minimum value in the first input range and the minimum value in the second input range of the first operator are weighted to obtain the target minimum value; Based on the first weight parameter and the second weight parameter, the maximum value in the first input range and the maximum value in the second input range are weighted to obtain the target maximum value; The range between the target minimum value and the target maximum value is determined as the target input range of the first operator.

19. The apparatus according to claim 17, characterized in that, The first operator includes a first quantization parameter, and the third parameter update unit is used for: For each of the first operators, determine a first difference between the maximum and minimum values ​​in the target input range of the first operator; Determine the second difference between the maximum and minimum values ​​within the target output range; The ratio between the first difference and the second difference is determined as the updated first quantization parameter of the first operator, whereby the first quantization parameter indicates the scaling factor of the data.

20. The apparatus according to claim 19, characterized in that, The first operator further includes a second quantization parameter, and the third parameter update unit is used for: The reference minimum value is obtained by rounding down the ratio between the minimum value in the target input range and the updated first quantization parameter. The difference between the minimum value in the target output range and the reference minimum value is determined as the updated second quantization parameter of the first operator, and the second quantization parameter indicates the displacement magnitude of the data.

21. The apparatus according to claim 12, characterized in that, The parameter update module includes: The range determination unit is used to turn off each of the first operators and the second operators in the first model, determine the multiple input ranges of the splicing layer as the third input ranges of the corresponding first operators, and determine the output range of the splicing layer as the third input range of the second operators; The third parameter update unit is used to update the quantization parameters of each first operator and the quantization parameters of each second operator based on the third input range of each first operator, the third input range of the second operator, and the target output range. The range determination unit is further configured to open each of the first operators and the second operators in the first model and determine the fourth input range of each of the first operators and the second operators; The third parameter update unit is further configured to continue updating the quantization parameters of each first operator and the second operator based on the fourth input range of each first operator, the fourth input range of the second operator, and the target output range, so that the quantization parameters of each first operator and the second operator converge to the quantization parameters corresponding to the target output range.

22. The apparatus according to claim 12, characterized in that, The first operator includes a first quantization operator and a first inverse quantization operator, and the second operator includes a second quantization operator and a second inverse quantization operator; the device further includes: The operator adjustment module is used to delete the first inverse quantization operator in each of the first operators and to delete the second quantization operator in each of the second operators.

23. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one computer program, which is loaded and executed by the processor to perform the operations performed in the model quantization method as described in any one of claims 1 to 11.

24. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to perform the operations performed in the model quantization method as described in any one of claims 1 to 11.

25. A computer program product, characterized in that, The computer program product includes computer program code stored in a computer-readable storage medium, a processor of a computer device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code to cause the computer device to perform the operations performed in the model quantization method as described in any one of claims 1 to 11.

Images