Model quantification method and device, equipment and storage medium

Abstract

The invention relates to the technical field of model quantification, and discloses a model quantification method and device, equipment and a storage medium, and the method comprises the steps: carrying out the model quantification of a target model according to a quantification configuration parameter, and obtaining a first quantification model; obtaining an overall quantization loss corresponding to the first quantization model and a network quantization loss corresponding to each layer of network in the first quantization model; if the overall quantization loss is greater than a preset loss threshold, selecting a quantization loss area according to the network quantization loss; carrying out operator splitting on a network corresponding to the quantitative loss region in the target model to obtain an adjusted target model; performing model quantization on the adjusted target model based on the quantization configuration parameter to obtain a second quantization model; and if the overall quantization loss corresponding to the second quantization model is less than or equal to a preset loss threshold, taking the second quantization model as a quantization completed model. Due to the fact that the operator is split and then quantized, the quantization effect close to channel-by-channel quantization is achieved, and it is guaranteed that the overall quantization loss of the model can be reduced.

Description

Technical Field

[0001] The present application relates to the field of model quantization technology, and in particular to a model quantization method, device, equipment and storage medium. Background Art

[0002] To ensure the model effect, the parameters in the model generally use higher-precision data types and bit numbers to represent the model parameters (such as using 32-bit floating-point data FP32 to represent model parameters). Models that represent model parameters with high precision occupy a large amount of memory space during operation and have high computing performance requirements for the device. Due to the performance limitations of the device side (such as the user side), if today's models are to be applied on the device side, they generally need to be quantized to reduce the space occupied by the model and lower the model's requirements for device performance to ensure that the model can be used smoothly on the device side.

[0003] During the model quantization process, the model accuracy will decrease. Although some methods have solved the quantization loss problem through cross-layer weight balancing and bias absorption, since bias absorption is not an equivalent change, quantization-aware training needs to be introduced to ensure model accuracy. The weights in the network need to be adjusted. This process will cause the accuracy of the floating-point model to decrease, resulting in further loss of quantization accuracy, and the actual use effect is difficult to guarantee. Summary of the invention

[0004] The main purpose of this application is to provide a model quantization method, device, equipment and storage medium, aiming to solve the technical problem that the related technology is difficult to solve the quantization loss problem of model quantization.

[0005] To achieve the above objectives, the present application proposes a model quantization method, which includes:

[0006] Performing model quantization on the target model according to the quantization configuration parameters to obtain a first quantized model, wherein the target model is a model that has not been model quantized and represents model parameters with high precision;

[0007] Obtaining an overall quantization loss corresponding to the first quantization model and a network quantization loss corresponding to each layer of the network in the first quantization model;

[0008] If the overall quantization loss is greater than a preset loss threshold, a quantization loss region is selected according to the network quantization loss, and the quantization loss region is a layer of the network with the largest corresponding network quantization loss in the first quantization model;

[0009] Performing operator splitting on the network corresponding to the quantization loss region in the target model to obtain an adjusted target model, wherein the operator splitting is to split the network corresponding to the quantization loss region in the target model into multiple operators based on the output channel, and the adjusted target model is the target model after the operator splitting;

[0010] Performing model quantization on the adjusted target model based on the quantization configuration parameters to obtain a second quantization model;

[0011] If the overall quantization loss corresponding to the second quantization model is less than or equal to the preset loss threshold, the second quantization model is used as the quantization completed model.

[0012] In a possible implementation of the present application, performing operator splitting on the network corresponding to the quantization loss area in the target model to obtain an adjusted target model includes:

[0013] Marking the network corresponding to the quantization loss region in the target model as an operator to be split;

[0014] Taking the output channel as a reference, splitting the operator to be split into multiple sub-operators, wherein the sub-operators are operators at the output channel level;

[0015] Dividing the multiple sub-operators into at least one operator set, wherein a difference in quantization precision of the sub-operators in the operator set is less than a preset difference threshold;

[0016] The sub-operators in each operator set are fused respectively to obtain multiple fused operators;

[0017] The network corresponding to the quantization loss area in the target model is replaced with the multiple fusion operators to obtain an adjusted target model.

[0018] In a possible implementation of the present application, dividing the multiple sub-operators into at least one operator set includes:

[0019] Obtain the quantization coefficient corresponding to each sub-operator;

[0020] Traversing the multiple sub-operators to obtain a current sub-operator, where the current sub-operator is a sub-operator that has not been traversed and has not been divided into an operator set;

[0021] A sub-operator among the multiple sub-operators, the difference between the corresponding quantization coefficient and the quantization coefficient corresponding to the current sub-operator is less than or equal to a preset difference threshold, is used as a target operator;

[0022] Dividing the current sub-operator and the target operator into an operator set;

[0023] Detect whether there is a sub-operator among the multiple sub-operators that has not been traversed and has not been divided into the operator set;

[0024] If not, the traversal of the multiple sub-operators is terminated to obtain at least one operator set.

[0025] In a possible implementation of the present application, replacing the network corresponding to the quantization loss area in the target model with the multiple fusion operators to obtain the adjusted target model includes:

[0026] Marking a cascade network in the target model according to the quantization loss region, wherein the cascade network is a network cascaded with output layers of networks corresponding to the quantization loss region in the target model;

[0027] Detecting whether the operator splitting of the cascade network has splitting benefits, wherein the splitting benefits are that after the operator splitting of the cascade network, the model is quantized based on the split network structure, and the overall quantization loss of the model is reduced;

[0028] If there is a splitting benefit, the cascade network is split to obtain a plurality of cascade fusion operators, and the cascade fusion operators correspond one to one to the fusion operators;

[0029] The network corresponding to the quantization loss area in the target model is replaced with the multiple fusion operators, and the cascade network in the target model is replaced with the multiple cascade fusion operators to obtain an adjusted target model.

[0030] In a possible implementation of the present application, after performing model quantization on the adjusted target model based on the quantization configuration parameters to obtain the second quantization model, the method further includes:

[0031] If the overall quantization loss corresponding to the second quantization model is greater than the preset loss threshold, determining the highest loss region according to the network quantization loss corresponding to each layer of the network in the second quantization model, wherein the highest loss region is the network with the largest corresponding network quantization loss in the second quantization model;

[0032] Detecting whether the highest loss area is in a split area, where the split area is a network area that has been split by an operator;

[0033] If it is not in the split area, the area with the highest loss is used as a new quantization loss area, and the step of performing operator splitting on the network corresponding to the quantization loss area in the target model is returned to obtain the adjusted target model.

[0034] In a possible implementation of the present application, after detecting whether the highest loss area is in the split area, the method further includes:

[0035] If it is in the split area, the fusion operator corresponding to the highest loss area in the target model is set to high-precision implementation to obtain the set target model;

[0036] Performing model quantization on the set target model based on the quantization configuration parameters to obtain a third quantization model;

[0037] If the overall quantization loss corresponding to the third quantization model is less than or equal to the preset loss threshold, the third quantization model is used as the quantization completed model.

[0038] In a possible implementation of the present application, after performing model quantization on the set target model based on the quantization configuration parameters to obtain the third quantization model, the method further includes:

[0039] If the overall quantization loss corresponding to the third quantization model is greater than the preset loss threshold, determining the area to be adjusted according to the network quantization loss corresponding to each layer of the network in the third quantization model, the area to be adjusted being the network with the largest network quantization loss in the third quantization model;

[0040] The region to be adjusted is used as a new quantization loss region, and the step of performing operator splitting on the network corresponding to the quantization loss region in the target model is returned to obtain an adjusted target model.

[0041] In addition, to achieve the above purpose, the present application also proposes a model quantization device, the device comprising:

[0042] A first quantization module, configured to perform model quantization on a target model according to quantization configuration parameters to obtain a first quantization model, wherein the target model is a model that has not been model quantized and represents model parameters with high precision;

[0043] A loss acquisition module, used to acquire the overall quantization loss corresponding to the first quantization model and the network quantization loss corresponding to each layer network in the first quantization model;

[0044] A region selection module, configured to select a quantization loss region according to the network quantization loss if the overall quantization loss is greater than a preset loss threshold, wherein the quantization loss region is a layer of the network corresponding to the largest network quantization loss in the first quantization model;

[0045] An operator splitting module is used to perform operator splitting on the network corresponding to the quantization loss area in the target model to obtain an adjusted target model, wherein the operator splitting is to split the network corresponding to the quantization loss area in the target model into multiple operators based on the output channel, and the adjusted target model is the target model after the operator splitting;

[0046] A second quantization module, configured to perform model quantization on the adjusted target model based on the quantization configuration parameters to obtain a second quantization model;

[0047] The quantization determination module is used to use the second quantization model as a quantization-completed model if the overall quantization loss corresponding to the second quantization model is less than or equal to the preset loss threshold.

[0048] In addition, to achieve the above-mentioned purpose, the present application also proposes a model quantization device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the model quantization method as described above.

[0049] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the model quantization method as described above are implemented.

[0050] In addition, to achieve the above-mentioned purpose, the present application also proposes a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the model quantization method described above are implemented.

[0051] One or more technical solutions proposed in this application have at least the following technical effects:

[0052] Since the operator is split and then quantized, and the operator has better quantitative expression ability, it can have a quantization effect close to channel-by-channel quantization, thereby improving the quantization effect of the model and ensuring that the overall quantization loss of the model can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0055] Figure 1 A schematic diagram of a flow chart provided for the first embodiment of the model quantization method of the present application;

[0056] Figure 2 A schematic diagram of the process flow provided for the second embodiment of the model quantization method of the present application;

[0057] Figure 3 This is a schematic diagram of operator splitting and fusion in one embodiment of the model quantization method of the present application;

[0058] Figure 4 Schematic diagram of the process provided for the third embodiment of the model quantification method of this application

[0059] Figure 5 This is a schematic diagram of the overall implementation process of model quantization of an embodiment of the model quantization method of the present application;

[0060] Figure 6 This is a schematic diagram of the module structure of the model quantization device according to an embodiment of the present application;

[0061] Figure 7 Schematic diagram of the device structure of the hardware operating environment involved in the model quantization method in the embodiment of the present application.

[0062] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0063] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0064] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0065] The present application embodiment provides a model quantization method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the model quantization method of the present application.

[0066] In this embodiment, the model quantization method includes steps S10 to S60:

[0067] Step S10: quantizing the target model according to the quantization configuration parameters to obtain a first quantization model.

[0068] It should be noted that the executor of this embodiment can be the model quantization device, and the model quantization device can be an end-side device, such as a personal computer, a tablet computer, a smart phone and other devices, or a server-side device, such as a server, a cloud server, etc. Of course, it can also be other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment and the following embodiments, the model quantization method of this embodiment and the following embodiments is described with a model quantization device.

[0069] It should be noted that the quantization configuration parameters may be parameters required for quantizing the model, and may include parameters such as quantization coefficients and quantization regions. The quantization configuration parameters may be pre-set by the administrator of the model quantization device. The target model may be a model that is specified to be quantized, but has not yet been quantized, and the model parameters are represented by high-precision data types and bit numbers. The target model may be specified by the administrator of the model quantization device.

[0070] In actual use, the model parameters represented by high-precision implementation in the target model can be quantized according to the quantization configuration parameters and converted into model parameters represented by low-precision implementation. For example, the model parameters represented by single-precision floating-point data, double-precision floating-point data or higher-precision floating-point data in the target model can be quantized according to the quantization configuration parameters and converted into model parameters identified by 8-bit integer data or integer data of other bits.

[0071] Step S20: Obtain the overall quantization loss corresponding to the first quantization model and the network quantization loss corresponding to each layer of the network in the first quantization model.

[0072] It should be noted that the overall quantization loss corresponding to the first quantization model can be the overall accuracy loss of the model obtained by comparing the first quantization model generated after all model parameters in the target model are quantized with the target model.

[0073] The network quantization loss corresponding to the network in the first quantization model can be the accuracy loss of the model obtained by comparing the quantization model obtained after only quantizing the network of this layer in the target model with the target model. The network quantization loss corresponding to each layer of the network in the first quantization model can be obtained by processing each layer of the network in the target model in turn.

[0074] Step S30: If the overall quantization loss is greater than a preset loss threshold, a quantization loss region is selected according to the network quantization loss.

[0075] It should be noted that the quantization loss region may be a layer of the network with the largest corresponding network quantization loss in the first quantization model.

[0076] In actual use, if the overall quantization loss corresponding to the first quantization model is greater than the preset loss threshold, it means that the target quantization model is directly quantized according to the quantization configuration parameters, and the accuracy loss of the model is higher than the actual demand standard. The accuracy loss is high and needs further processing to reduce the quantization loss. At this time, in order to quickly reduce the quantization loss, the part with higher quantization loss can be processed first. Therefore, the quantization loss area can be selected according to the network quantization loss.

[0077] Step S40: performing operator splitting on the network corresponding to the quantization loss region in the target model to obtain an adjusted target model.

[0078] It should be noted that the operator splitting is to split the network corresponding to the quantization loss area in the target model into multiple operators based on the output channel, and the adjusted target model is the target model after the operator splitting.

[0079] In actual use, when quantizing a network, the overall quantization coefficient of the network is generally used to quantize its parameters. In fact, the quantization coefficients of each part of the network may be different. Using the overall quantization coefficient for quantization may have a poor quantization effect, resulting in a large quantization loss for the network. By splitting the quantization loss area by operators, the network corresponding to the quantization loss area can be further divided into multiple parts. When the model is quantized later, each split part can be quantized separately and specifically, reducing the quantization granularity of the model quantization, thereby improving the quantization effect and reducing the quantization loss.

[0080] Step S50: performing model quantization on the adjusted target model based on the quantization configuration parameters to obtain a second quantization model.

[0081] Step S60: If the overall quantization loss corresponding to the second quantization model is less than or equal to the preset loss threshold, the second quantization model is used as a quantization-completed model.

[0082] It should be noted that the network corresponding to the quantization loss area in the adjusted target model is divided into multiple parts. At this time, the adjusted target model is quantized based on the quantization configuration parameters. The quantization effect of the quantization loss area will be improved, thereby reducing the quantization loss of the network corresponding to the quantization loss area. The overall quantization loss corresponding to the second quantization model thus obtained should actually be smaller than the overall quantization loss of the first quantization model. However, although the overall quantization loss is reduced, it is difficult to determine whether it meets the actual demand standards. Therefore, the overall quantization loss corresponding to the second quantization model can be compared with the preset loss threshold.

[0083] In actual use, if the overall quantization loss corresponding to the second quantization model is less than or equal to the preset loss threshold, it means that after the operator splitting processing, the network corresponding to the quantization loss area is split into multiple parts for model quantization respectively, and the overall quantization loss of the obtained model has reached the standard of actual needs. Therefore, the second quantization model can be used as a quantized model.

[0084] This embodiment provides a model quantization method, which performs model quantization on a target model according to quantization configuration parameters to obtain a first quantization model; obtains the overall quantization loss corresponding to the first quantization model and the network quantization loss corresponding to each layer of the network in the first quantization model; if the overall quantization loss is greater than a preset loss threshold, selects a quantization loss region according to the network quantization loss; performs operator splitting on the network corresponding to the quantization loss region in the target model to obtain an adjusted target model; performs model quantization on the adjusted target model based on the quantization configuration parameters to obtain a second quantization model; if the overall quantization loss corresponding to the second quantization model is less than or equal to the preset loss threshold, the second quantization model is used as the quantized model. Since the operator is split and then quantized, and the operator has a better quantization expression capability, it can have a quantization effect close to channel-by-channel quantization, thereby improving the quantization effect of model quantization and ensuring that the overall quantization loss of the model can be reduced.

[0085] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 , step S40 of the model quantization method comprises steps S401 to S405:

[0086] Step S401: Mark the network corresponding to the quantization loss area in the target model as an operator to be split.

[0087] Step S402: Split the operator to be split into multiple sub-operators based on the output channel.

[0088] It should be noted that the sub-operator is an operator at the output channel level, and its type can be a convolution operator, a fully connected operator, an inverse sub-operator, etc. Each sub-operator corresponds to one or more output channels in the operator to be split (i.e., the network corresponding to the quantization loss area).

[0089] Among them, the fewer output channels corresponding to the sub-operator, the smaller the granularity of the model quantization of the network corresponding to the quantization loss area in subsequent processing, and the better the quantization effect. In order to ensure the quantization effect, each sub-operator can correspond to an output channel in the operator to be split, that is, the operator to be split can be split into multiple sub-operators based on a single output channel.

[0090] Step S403: Divide the multiple sub-operators into at least one operator set.

[0091] Step S404: fuse the sub-operators in each operator set respectively to obtain multiple fused operators.

[0092] It should be noted that the quantization precision of the sub-operators in the operator set is similar, that is, the difference in quantization precision corresponding to each sub-operator in the operator set is less than a preset precision threshold, where the preset precision threshold can be set in advance by the administrator of the model quantization device.

[0093] In actual use, parts of multiple sub-operators with similar quantization precision can be divided into the same operator set. In the division process, clustering or preset precision interval matching can be used for division, which is not limited in this embodiment.

[0094] For example: cluster the quantization precision of each of the multiple sub-operators to obtain multiple cluster centers, and divide the sub-operators belonging to the same cluster center into an operator set; or set multiple preset precision intervals, and divide the corresponding sub-operators whose quantization precision is in the same preset precision interval into an operator set.

[0095] It should be noted that if too many sub-operators are divided, more targets will need to be processed during the subsequent model quantization, which will cause the overall process of model quantization to take too long. In order to avoid the overall time-consuming due to too many operators, the divided sub-operators can be reasonably fused. When fusion is performed, in order to minimize the impact of fusion on quantization loss, some operators with similar quantization accuracy can be fused. Therefore, the sub-operators in each operator set can be fused separately to obtain multiple fused operators. Among them, the fused operator corresponds to the operator set, and the number of the two is the same.

[0096] In a specific implementation, fusing the sub-operators in the operator set may be to merge the output channels corresponding to the sub-operators in the operator set, thereby achieving fusion.

[0097] In a possible implementation of this embodiment, in order to quickly perform the division, step S403 of this embodiment may include:

[0098] Obtain the quantization coefficient corresponding to each sub-operator;

[0099] Traversing the multiple sub-operators to obtain a current sub-operator;

[0100] A sub-operator among the multiple sub-operators, the difference between the corresponding quantization coefficient and the quantization coefficient corresponding to the current sub-operator is less than or equal to a preset difference threshold, is used as a target operator;

[0101] Dividing the current sub-operator and the target operator into an operator set;

[0102] Detect whether there is a sub-operator among the multiple sub-operators that has not been traversed and has not been divided into the operator set;

[0103] If not, the traversal of the multiple sub-operators is terminated to obtain at least one operator set.

[0104] It should be noted that the quantization coefficient can be the coefficient used to quantize parameters such as weights, biases, activation layers, etc. in the sub-operator. The quantization coefficient can reflect the quantization accuracy. If the quantization coefficients corresponding to two sub-operators are similar or the same, it means that the quantization accuracy corresponding to the two sub-operators is similar.

[0105] In order to quickly divide the operator set, multiple sub-operators can be traversed, and the traversed sub-operators can be used as the current sub-operator. In order to avoid repeated division, the current sub-operator can be a sub-operator that has not been traversed and has not been divided into the operator set;

[0106] After determining the current sub-operator, the quantization coefficient corresponding to the current sub-operator can be used as a reference to search for a sub-operator among multiple sub-operators whose corresponding quantization coefficient is less than or equal to a preset difference threshold value and the quantization coefficient corresponding to the current sub-operator, and use such a sub-operator as a target operator. At this time, the quantization accuracy of the target operator is similar to that of the current sub-operator. Therefore, the current sub-operator and the target operator can be divided into an operator set;

[0107] Afterwards, it can be detected whether there are sub-operators among the multiple sub-operators that have not been traversed and have not been divided into operator sets. If so, it means that the division of the sub-operators has not been completed at this time, and the traversal of the multiple sub-operators can continue at this time; if not, it means that the division of the sub-operators has been completed at this time, and the traversal of the multiple sub-operators can be ended. At this time, the operator set generated during the traversal process is obtained, and at least one operator set formed by the division of multiple sub-operators can be obtained.

[0108] Step S405: replacing the network corresponding to the quantization loss region in the target model with the multiple fusion operators to obtain an adjusted target model.

[0109] It should be noted that after determining multiple fusion operators, the specific method of splitting the operators of the network corresponding to the quantization loss area is determined. At this time, the network corresponding to the quantization loss area in the target model can be replaced with multiple fusion operators, and the target model after the replacement is used as the adjusted target model.

[0110] In a possible implementation of this embodiment, in order to complete the model quantization as quickly as possible, step S405 of this embodiment may include:

[0111] Marking the cascade network in the target model according to the quantized loss region;

[0112] Detecting whether splitting the cascade network operators has splitting benefits;

[0113] If there is a splitting benefit, the cascade network is split to obtain a plurality of cascade fusion operators, and the cascade fusion operators correspond one to one to the fusion operators;

[0114] The network corresponding to the quantization loss area in the target model is replaced with the multiple fusion operators, and the cascade network in the target model is replaced with the multiple cascade fusion operators to obtain an adjusted target model.

[0115] It should be noted that the cascade network may be a network cascaded with the output layer of the network corresponding to the quantization loss region in the target model, that is, another network directly connected to the output layer of the network corresponding to the quantization loss region.

[0116] In actual use, there will be a certain correlation between directly connected networks. When splitting the network corresponding to the quantization loss area, the corresponding cascade network is also split at the same time, which may further reduce the overall quantization loss. Therefore, in order to ensure that the model quantization can be completed quickly, after determining the operator splitting method of the network corresponding to the quantization loss area, it is possible to test whether the operator splitting of the cascade network in the same way has splitting benefits;

[0117] If there is no splitting benefit, it means that it is difficult to further reduce the overall quantization loss by splitting the operators of the cascade network at this time. Therefore, the subsequent steps can be omitted, and the network corresponding to the quantization loss area in the target model can be directly replaced with multiple fusion operators;

[0118] If there is a split benefit, in order to quickly complete the model quantization, the same splitting method can be used to split the operators of the cascade network, thereby obtaining multiple cascade fusion operators. After that, the network corresponding to the quantization loss area in the target model is replaced with multiple fusion operators, and the cascade network in the target model is replaced with multiple cascade fusion operators to obtain the adjusted target model. Among them, since the splitting method used is the same, the cascade fusion operator corresponds to the fusion operator one by one.

[0119] In practical applications, splitting benefits mean that after the cascade network is split by operators, the model is quantized based on the split network structure, and the overall quantization loss of the model is reduced (or the reduction in the overall quantization loss is greater than a certain amount). Otherwise, it is judged that there is no splitting benefit.

[0120] For ease of understanding, now combined Figure 3 This is for illustration only, but not for limitation. Figure 3 This is a schematic diagram of operator splitting and fusion in this embodiment.

[0121] like Figure 3 As shown, the network structure of the target model is as follows Figure 3As shown in a, it includes three layers of networks: conv1, conv2 and conv4. At this time, conv2 is the operator to be split (that is, conv2 is the network corresponding to the quantization loss area), and the data processed by conv2 is "W<32x32x1x1>B <32> ", where the first 32 in W is the number of output channels, and B is the offset of each channel. Taking a single output channel as the reference, split it channel by channel to obtain the following: Figure 3 As shown in b, there are 32 sub-operators (Conv2_1, ...Conv2_17, ...Conv2_32), and the data processed by each sub-operator is "W<1x32x1x1>B <1> ", in this case, the set storing each sub-operator can be recorded as SetOp;

[0122] At this time, the sub-operators in SetOp can be re-quantized based on the original floating-point parameters (i.e., model parameters) of the floating-point model (i.e., the target model that uses floating-point data to represent model parameters with high precision) and the input and output data of conv2 saved in the quantized model. At this time, the weights, biases, and quantization coefficients of the activation layers corresponding to each sub-operator in SetOp will be generated, which can be recorded as SetW, SetB, and SetA respectively;

[0123] After that, the channels with similar quantization coefficients of the weight, bias and activation layers can be merged to fuse the subdivided operators. At this time, the quantization coefficients of Conv2_1-Conv2_16 are similar and merged into Conv2_1, the quantization coefficients of Conv2_17-Conv2_24 are similar and merged into Conv2_17, and the quantization coefficients of Conv2_25-Conv2_32 are similar and merged into Conv2_32, and multiple fusion operators (Conv2_1, Conv2_17 and Conv2_32) are obtained.

[0124] At this point, we can try to split the operators of the cascade network Conv4 of Conv2 and evaluate whether there is any benefit in splitting the operators of Conv4. If there is no benefit, replace Conv2 in the target model with multiple fusion operators (Conv2_1, Conv2_17 and Conv2_32). The final split result (i.e. the adjusted target model) network structure is as follows: Figure 3 As shown in c;

[0125] If there is a benefit, the cascade network Conv4 is split into operators to obtain multiple cascade fusion operators (Conv4_1, Conv4_2, and Conv4_3), and Conv2 in the target model is replaced with multiple fusion operators (Conv2_1, Conv2_17, and Conv2_32), and Conv4 in the target model is replaced with multiple cascade fusion operators (Conv4_1, Conv4_2, and Conv4_3). At this time, the final split result (i.e., the adjusted target model) network structure is as follows: Figure 3 As shown in d.

[0126] This embodiment provides a model quantization method. After the operator is split at the channel level, the parts of the split sub-operators with similar quantization accuracy are merged, thereby avoiding the low model quantization efficiency caused by too many sub-operators and ensuring the quantization efficiency of the model as high as possible.

[0127] Based on the first embodiment and / or the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the above embodiments can be referred to the above introduction, and will not be described in detail later. Figure 4 After step S50, the model quantization method further includes steps S70 to S90:

[0128] Step S70: If the overall quantization loss corresponding to the second quantization model is greater than the preset loss threshold, the highest loss area is determined according to the network quantization loss corresponding to each layer network in the second quantization model.

[0129] It should be noted that the area with the highest loss is the network with the largest network quantization loss in the second quantization model. The overall quantization loss of the second quantization model and the network quantization loss corresponding to each layer of the network in the second quantization model are obtained in the same way as the first quantization model. The specific implementation can refer to the above description and will not be repeated here.

[0130] If the overall quantization loss corresponding to the second quantization model is greater than the preset loss threshold, it means that after the operator splitting processing, the network corresponding to the quantization loss area is split into multiple parts for model quantization respectively, and the overall quantization loss of the obtained model still does not meet the actual demand standard. At this time, further adjustment is needed. In order to make quick adjustments, the part with the largest network quantization loss in the model needs to be used as the adjustment target. Therefore, the area with the highest loss can be determined according to the network quantization loss corresponding to each layer of the network in the second quantization model.

[0131] Step S80: Detect whether the area with the highest loss is in the split area.

[0132] It should be noted that the split area is a network area that has been split by operators, including areas where operator splitting is performed directly and areas where operator splitting is performed in cascade.

[0133] For example: Figure 3 c as an example, the split regions now include Conv2_1, Conv2_17 and Conv2_32; and taking the above Figure 3 d as an example, the split regions now include Conv2_1, Conv2_17 and Conv2_32, as well as Conv4_1, Conv4_2 and Conv4_3.

[0134] In actual use, whether the area with the highest detection loss is in the split area can be to detect whether the network (or fusion operator) corresponding to the area with the highest detection loss is the network (or fusion operator) included in the split area.

[0135] Step S90: If it is not in the split area, the area with the highest loss is used as a new quantized loss area.

[0136] It can be understood that if the area with the highest loss is not in the split area, it means that the network corresponding to the area with the highest loss has not yet been tried for processing through operator splitting. Therefore, at this time, you can try to reduce the quantization loss of the network corresponding to the area with the highest loss through operator splitting. At this time, the area with the highest loss can be used as a new quantization loss area, and return to execute step S40.

[0137] In a possible implementation of this embodiment, after step S80 described in this embodiment, the following may also be included:

[0138] If it is in the split area, the fusion operator corresponding to the highest loss area in the target model is set to high-precision implementation to obtain the set target model;

[0139] Performing model quantization on the set target model based on the quantization configuration parameters to obtain a third quantization model;

[0140] If the overall quantization loss corresponding to the third quantization model is less than or equal to the preset loss threshold, the third quantization model is used as the quantization completed model.

[0141] It should be noted that if the area with the highest loss is in the split area, it means that the network (or fusion operator) corresponding to the area with the highest loss has been processed by operator splitting, but its quantization loss is still high, which means that the quantization loss of the model quantization for this part of the network (or fusion operator) is too high and difficult to reduce by operator splitting. At this time, the accuracy of the model is prioritized. The fusion operator corresponding to the area with the highest loss in the target model can be set to high-precision implementation to obtain the target model after setting.

[0142] Among them, high-precision implementation means using a quantization configuration with higher precision than the original quantization configuration for model quantization. For example, in the original quantization configuration, 8-bit integer INT8 is used for model quantization, and when a network or fusion operator is set to high-precision implementation, a quantization configuration with higher precision than INT8, such as 16-bit integer INT16, 16-bit floating point number FP16, 32-bit floating point number, etc., is used to quantize the network or fusion operator;

[0143] In actual use, if the network or fusion operator in the model is set to high-precision implementation, and the accuracy of the quantization configuration is the same as the representation accuracy of the original parameters of the target model, then when the model is quantized later, this part of the network or fusion operator will not be quantized; if the accuracy of the quantization configuration is different from the guaranteed accuracy of the original parameters of the target model, quantization is still required.

[0144] For example: if the original parameters of the target model are guaranteed by FP32, and the network or fusion operator in the model is set to use the quantization configuration FP32 for high-precision implementation, then when the model is quantized later, this part of the network or fusion operator will not be quantized; and if the network or fusion operator in the model is set to use the quantization configuration INT16 or FP16 for high-precision implementation, then when the model is quantized later, this part of the network or fusion operator still needs to be quantized.

[0145] In actual use, after completing the settings, it is difficult to determine whether the model quantization after the settings meets the actual needs. At this time, in order to facilitate verification, the target model after the settings can be quantized based on the quantization configuration parameters to obtain a third quantization model.

[0146] Among them, the overall quantization loss of the third quantization model and the network quantization loss corresponding to each layer of the network in the third quantization model are obtained in the same way as the first quantization model. The specific implementation can refer to the above description and will not be repeated here.

[0147] It can be understood that if the overall quantization loss corresponding to the third quantization model is less than or equal to the preset loss threshold, it means that after the operator splitting processing and the model quantization after setting, the overall quantization loss of the obtained model has reached the standard of actual needs. Therefore, the third quantization model can be used as a quantized model.

[0148] In a specific implementation, if the overall quantization loss corresponding to the third quantization model is greater than the preset loss threshold, it means that after the operator splitting process and the model quantization after the setting, the overall quantization loss of the obtained model still cannot meet the standard of actual demand, and further processing is required at this time. Therefore, after the step of performing model quantization on the set target model based on the quantization configuration parameters to obtain the third quantization model in this embodiment, it may also include:

[0149] If the overall quantization loss corresponding to the third quantization model is greater than the preset loss threshold, determining the area to be adjusted according to the network quantization loss corresponding to each layer network in the third quantization model;

[0150] The region to be adjusted is used as a new quantization loss region, and the step of performing operator splitting on the network corresponding to the quantization loss region in the target model is returned to obtain an adjusted target model.

[0151] It should be noted that the area to be adjusted may be the network with the largest corresponding network quantization loss in the third quantization model.

[0152] It can be understood that if the overall quantization loss corresponding to the third quantization model is greater than the preset loss threshold, it means that after the operator splitting processing and the model quantization is performed after setting, the overall quantization loss of the model still cannot meet the actual demand standards. At this time, further processing is needed. In order to speed up the processing speed of model quantization, the part with the largest quantization loss can be given priority. Therefore, the area to be adjusted can be used as a new quantization loss area, and the step of performing operator splitting on the network corresponding to the quantization loss area in the target model is returned to obtain the adjusted target model.

[0153] For ease of understanding, now combined Figure 5 This is for explanation only, but does not limit the present solution. Figure 5 The figure is a schematic diagram of the overall implementation process of the model quantization of this embodiment.

[0154] like Figure 5 As shown, the segmentation algorithm for the overall implementation of model quantization may include the following steps:

[0155] 1. Generate a basic quantitative model M0 according to the quantitative configuration parameters of the terminal platform to be deployed (i.e., perform model quantization on the target model according to the quantitative configuration parameters to obtain a first quantitative model);

[0156] 2. Calculate the layer-by-layer quantization loss of the basic quantization model (i.e., the network quantization loss corresponding to each layer of the network) and the overall quantization loss of the model (i.e., the overall quantization loss);

[0157] 3. If the overall quantization loss of the model is greater than the threshold, the quantization loss area is split and merged.

[0158] The split-fusion process is as follows: according to the layer-by-layer quantization loss, select the operator with the highest current quantization loss and mark it as the operator to be split; group the adjacent channels with similar weights and activation layer data distribution between channels and determine the split granularity; determine the operator splitting strategy according to the type of the operator to be split and the upper and lower associated network structure; split the operator according to the splitting strategy and splitting granularity; mark the operator splitting scope as the split region R;

[0159] 4. The adjusted network structure is quantized again to obtain a quantization model M1 (i.e., a second quantization model);

[0160] 5. According to the quantization model M1, recalculate the layer-by-layer quantization loss and overall quantization loss of the updated model;

[0161] 6. If the overall quantization loss of the adjusted network (i.e. the adjusted target model) is less than or equal to the threshold, the second quantization model is used as the quantized model; if the overall quantization loss is still greater than the threshold, it is determined whether the area with the highest current quantization loss (i.e. the area with the highest loss) is located in the split area R. If it is located in the split area, the branch corresponding to the area with the highest loss (i.e. the fusion operator) is set to high-precision implementation. If it is not located in the split area, return to step 3;

[0162] 7. The set network structure (i.e., the set target model) is quantized again to obtain the quantization model Mn (i.e., the third quantization model), and the layer-by-layer quantization loss and the overall network quantization loss are calculated;

[0163] 8. If the overall quantization loss is still greater than the threshold, return to step 3; if the overall quantization loss is less than or equal to the threshold, the third quantization model is used as the quantized model.

[0164] In a specific implementation, the model quantization method provided in this embodiment and the above embodiments can be applied to the intelligent recognition field, the intelligent question answering field, or other fields where models need to be deployed on the terminal side. For ease of understanding, the intelligent recognition field and the intelligent question answering field are used as examples for illustration:

[0165] In the field of intelligent identification, such as detecting whether the actions of specific people are standardized, whether they are wearing corresponding accessories or corresponding clothes, whether the trash can is overflowing, etc., the model is usually applied in the corresponding detection equipment (such as image acquisition equipment, etc.), and the equipment performance of the detection equipment is relatively low, and it is difficult to provide the computing power when the unquantized model is running. At this time, the above-mentioned model quantization method can be used first, and the model quantization method can be introduced to process the model to be used first to obtain the quantized model, and then the quantized model is deployed in the detection equipment. Since the demand for computing power and resources for the quantized model will be greatly reduced, such deployment can ensure that even if the equipment performance of the detection equipment is poor, the quantized model can still be operated normally, and the model can be used for intelligent identification;

[0166] In the field of intelligent question answering, in some cases, it is necessary to set the intelligent question answering model in a mobile device (such as a user's smartphone, tablet computer, etc.) to ensure that offline intelligent question answering services can be provided to users. However, the computing power of such devices is generally low, and it is difficult to provide computing power for the operation of an unquantized model. In this case, the above-mentioned model quantization method can be adopted first, and the model quantization method can be introduced to process the model to be used first to obtain a quantized model. The quantized model is then deployed in a mobile device to ensure that the mobile device can normally run the quantized model and use the model for intelligent question answering.

[0167] This embodiment provides a model quantization method. It detects whether the area with the highest loss is in the split area, and determines whether the network corresponding to the area with the highest loss can reduce the quantization loss through operator splitting. When it is determined that the quantization loss cannot be reduced through operator splitting, the network or fusion operator corresponding to the area with the highest loss will be set to high-precision implementation, ensuring that the quantized model can be calculated using mixed precision. Since only some channels of the split operator need to introduce mixed precision, the problem of rapid time growth caused by mixed precision inference can also be reduced.

[0168] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the model quantization method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0169] This application also provides a model quantization device, please refer to Figure 6 , the model quantization device comprises:

[0170] A first quantization module 10 is used to perform model quantization on a target model according to quantization configuration parameters to obtain a first quantization model, wherein the target model is a model that has not been model quantized and represents model parameters with high precision;

[0171] A loss acquisition module 20, used to acquire the overall quantization loss corresponding to the first quantization model and the network quantization loss corresponding to each layer network in the first quantization model;

[0172] A region selection module 30, configured to select a quantization loss region according to the network quantization loss if the overall quantization loss is greater than a preset loss threshold, wherein the quantization loss region is a layer of the network corresponding to the largest network quantization loss in the first quantization model;

[0173] An operator splitting module 40 is used to perform operator splitting on the network corresponding to the quantization loss area in the target model to obtain an adjusted target model, wherein the operator splitting is to split the network corresponding to the quantization loss area in the target model into multiple operators based on the output channel, and the adjusted target model is the target model after the operator splitting;

[0174] A second quantization module 50, configured to perform model quantization on the adjusted target model based on the quantization configuration parameters to obtain a second quantization model;

[0175] The quantization determination module 60 is configured to use the second quantization model as a quantized model if the overall quantization loss corresponding to the second quantization model is less than or equal to the preset loss threshold.

[0176] The model quantization device provided by the present application adopts the model quantization method in the above embodiment, which can solve the technical problem that the related technology is difficult to solve the quantization loss problem of model quantization. Compared with the related technology, the beneficial effects of the model quantization device provided by the present application are the same as the beneficial effects of the model quantization method provided by the above embodiment, and the other technical features in the model quantization device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0177] The present application provides a model quantization device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the model quantization method in the above-mentioned embodiment 1.

[0178] Reference below Figure 7, which shows a schematic diagram of the structure of a model quantization device suitable for implementing the embodiment of the present application. The model quantization device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The model quantization device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0179] like Figure 7 As shown, the model quantization device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the model quantization device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the model quantization device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a model quantization device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0180] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0181] The model quantization device provided by the present application adopts the model quantization method in the above embodiment, which can solve the technical problem that the related technology is difficult to solve the quantization loss problem of model quantization. Compared with the related technology, the beneficial effects of the model quantization device provided by the present application are the same as the beneficial effects of the model quantization method provided by the above embodiment, and the other technical features in the model quantization device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0182] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0183] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0184] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the model quantization method in the above-mentioned embodiment.

[0185] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0186] The computer-readable storage medium may be included in the model quantization device; or may exist independently without being assembled into the model quantization device.

[0187] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the model quantization device, the model quantization device: performs model quantization on the target model according to the quantization configuration parameters to obtain a first quantization model, wherein the target model is a model that has not been quantized and represents the model parameters with high precision; obtains the overall quantization loss corresponding to the first quantization model and the network quantization loss corresponding to each layer of the network in the first quantization model; if the overall quantization loss is greater than a preset loss threshold, selects a quantization loss region according to the network quantization loss, wherein the quantization loss region is a layer of network with the largest corresponding network quantization loss in the first quantization model; performs operator splitting on the network corresponding to the quantization loss region in the target model to obtain an adjusted target model, wherein the operator splitting is to split the network corresponding to the quantization loss region in the target model into multiple operators based on the output channel, and the adjusted target model is a target model that has been operator split; performs model quantization on the adjusted target model based on the quantization configuration parameters to obtain a second quantization model; if the overall quantization loss corresponding to the second quantization model is less than or equal to the preset loss threshold, the second quantization model is used as the quantized model.

[0188] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0189] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0190] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0191] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned model quantization method, and can solve the technical problem that the related technology is difficult to solve the quantization loss problem of model quantization. Compared with the related technology, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the model quantization method provided in the above-mentioned embodiment, and will not be repeated here.

[0192] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned model quantization method when executed by a processor.

[0193] The computer program product provided by this application can solve the technical problem that the related technology is difficult to solve the quantization loss problem of model quantization. Compared with the related technology, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the model quantization method provided by the above embodiment, which will not be repeated here.

[0194] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A model quantization method, characterized in that: The method comprises: Performing model quantization on the target model according to the quantization configuration parameters to obtain a first quantized model, wherein the target model is a model that has not been model quantized and represents model parameters with high precision; Obtaining an overall quantization loss corresponding to the first quantization model and a network quantization loss corresponding to each layer of the network in the first quantization model; If the overall quantization loss is greater than a preset loss threshold, a quantization loss region is selected according to the network quantization loss, and the quantization loss region is a layer of the network with the largest corresponding network quantization loss in the first quantization model; Performing operator splitting on the network corresponding to the quantization loss region in the target model to obtain an adjusted target model, wherein the operator splitting is to split the network corresponding to the quantization loss region in the target model into multiple operators based on the output channel, and the adjusted target model is the target model after the operator splitting; Performing model quantization on the adjusted target model based on the quantization configuration parameters to obtain a second quantization model; If the overall quantization loss corresponding to the second quantization model is less than or equal to the preset loss threshold, the second quantization model is used as the quantization completed model.

2. The model quantization method according to claim 1, characterized in that: The performing operator splitting on the network corresponding to the quantization loss region in the target model to obtain an adjusted target model includes: Marking the network corresponding to the quantization loss region in the target model as an operator to be split; Taking the output channel as a reference, splitting the operator to be split into multiple sub-operators, wherein the sub-operators are operators at the output channel level; Dividing the multiple sub-operators into at least one operator set, wherein a difference in quantization precision of the sub-operators in the operator set is less than a preset difference threshold; The sub-operators in each operator set are fused respectively to obtain multiple fused operators; The network corresponding to the quantization loss area in the target model is replaced with the multiple fusion operators to obtain an adjusted target model.

3. The model quantization method according to claim 2, characterized in that: The dividing the multiple sub-operators into at least one operator set comprises: Obtain the quantization coefficient corresponding to each sub-operator; Traversing the multiple sub-operators to obtain a current sub-operator, where the current sub-operator is a sub-operator that has not been traversed and has not been divided into an operator set; A sub-operator among the multiple sub-operators, the difference between the corresponding quantization coefficient and the quantization coefficient corresponding to the current sub-operator is less than or equal to a preset difference threshold, is used as a target operator; Dividing the current sub-operator and the target operator into an operator set; Detect whether there is a sub-operator among the multiple sub-operators that has not been traversed and has not been divided into the operator set; If not, the traversal of the multiple sub-operators is terminated to obtain at least one operator set.

4. The model quantization method according to claim 2, characterized in that: The replacing the network corresponding to the quantization loss region in the target model with the multiple fusion operators to obtain the adjusted target model includes: Marking a cascade network in the target model according to the quantization loss region, wherein the cascade network is a network cascaded with output layers of networks corresponding to the quantization loss region in the target model; Detecting whether the operator splitting of the cascade network has splitting benefits, wherein the splitting benefits are that after the operator splitting of the cascade network, the model is quantized based on the split network structure, and the overall quantization loss of the model is reduced; If there is a splitting benefit, the cascade network is split to obtain a plurality of cascade fusion operators, and the cascade fusion operators correspond one to one to the fusion operators; The network corresponding to the quantization loss area in the target model is replaced with the multiple fusion operators, and the cascade network in the target model is replaced with the multiple cascade fusion operators to obtain an adjusted target model.

5. The model quantization method according to claim 1, characterized in that: After performing model quantization on the adjusted target model based on the quantization configuration parameters to obtain a second quantization model, the method further includes: If the overall quantization loss corresponding to the second quantization model is greater than the preset loss threshold, determining the highest loss region according to the network quantization loss corresponding to each layer of the network in the second quantization model, wherein the highest loss region is the network with the largest corresponding network quantization loss in the second quantization model; Detecting whether the highest loss area is in a split area, where the split area is a network area that has been split by an operator; If it is not in the split area, the area with the highest loss is used as a new quantization loss area, and the step of performing operator splitting on the network corresponding to the quantization loss area in the target model is returned to obtain the adjusted target model.

6. The model quantization method according to claim 5, characterized in that: After detecting whether the highest loss area is in the split area, the method further includes: If it is in the split area, the fusion operator corresponding to the highest loss area in the target model is set to high-precision implementation to obtain the set target model; Performing model quantization on the set target model based on the quantization configuration parameters to obtain a third quantization model; If the overall quantization loss corresponding to the third quantization model is less than or equal to the preset loss threshold, the third quantization model is used as the quantization completed model.

7. The model quantization method according to claim 6, characterized in that: After the set target model is quantized based on the quantization configuration parameters to obtain a third quantization model, the method further includes: If the overall quantization loss corresponding to the third quantization model is greater than the preset loss threshold, determining the area to be adjusted according to the network quantization loss corresponding to each layer of the network in the third quantization model, the area to be adjusted being the network with the largest network quantization loss in the third quantization model; The region to be adjusted is used as a new quantization loss region, and the step of performing operator splitting on the network corresponding to the quantization loss region in the target model is returned to obtain an adjusted target model.

8. A model quantization device, characterized in that: The device comprises: The first quantization module is used to quantize the target model according to the quantization configuration parameters to obtain A first quantized model, wherein the target model is a model that has not been quantized and represents model parameters with high precision; A loss acquisition module, used to acquire the overall quantization loss corresponding to the first quantization model and the network quantization loss corresponding to each layer network in the first quantization model; A region selection module, configured to select a quantization loss region according to the network quantization loss if the overall quantization loss is greater than a preset loss threshold, wherein the quantization loss region is a layer of the network corresponding to the largest network quantization loss in the first quantization model; An operator splitting module is used to perform operator splitting on the network corresponding to the quantization loss area in the target model to obtain an adjusted target model, wherein the operator splitting is to split the network corresponding to the quantization loss area in the target model into multiple operators based on the output channel, and the adjusted target model is the target model after the operator splitting; A second quantization module, configured to perform model quantization on the adjusted target model based on the quantization configuration parameters to obtain a second quantization model; The quantization determination module is used to use the second quantization model as a quantization-completed model if the overall quantization loss corresponding to the second quantization model is less than or equal to the preset loss threshold.

9. A model quantization device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the model quantization method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the model quantization method according to any one of claims 1 to 7 are implemented.

Images