Weight segmentation quantification method and device based on convolutional neural network

Abstract

The invention relates to the technical field of convolutional neural networks, and discloses a weight segmentation quantification method and device based on a convolutional neural network, and the method comprises the steps: obtaining the weight of each calculation layer in a plurality of calculation layers of the convolutional neural network, enabling the calculation layers to be layers with parameters of the convolutional neural network, and obtaining the weight of each calculation layer according to the weights of all calculation layers, screening out a target calculation layer to be segmented and quantized from all calculation layers, segmenting the weight of the target calculation layer according to all the weights of the target calculation layer to obtain a weight segmentation result of the target calculation layer, and carrying out segmented quantization on the first weight group and the second weight group of the target calculation layer to obtain a weight segmentation result of the target calculation layer; and obtaining the quantization bit width of the target calculation layer. It can be seen that by implementing the method, the target calculation layer needing segmented quantization can be accurately screened out, each group of weights can be accurately quantized, the quantization precision of each layer of weight in the convolutional neural network can be improved, and the calculation amount can be reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of convolutional neural networks, and in particular to a weight segmentation quantization method and device based on a convolutional neural network. Background Art

[0002] Currently, deep learning applications are becoming more and more widespread, but as its convolutional neural network deepens and the number of parameters increases, there is a huge memory and computing power overhead in its calculation and reasoning process. For efficiency, bandwidth, and power consumption considerations, most of the current edge-side SoC NPUs only support integer operations, so weight quantization will be an inevitable choice for deep learning deployment on the edge.

[0003] However, practice has found that due to the characteristics of quantization using low-bit (quantization bit width) integer operations instead of high-bit floating-point operations, there is a problem of loss of calculation accuracy. The more common solution in the industry is to quantize the floating-point 32-bit network into a fixed-point 8-bit integer network calculation. Since the network learning process has a certain degree of randomness, the required information capacity of each layer is different. When some layers are calculated with 8-bit quantization, it is easy for the results to be far from the precision of the floating point. Therefore, it is particularly important to propose a new weight quantization scheme to improve the quantization accuracy. Summary of the invention

[0004] The present invention provides a weight segmentation quantization method and device based on convolutional neural network, which can improve the quantization accuracy of weights of each layer in the convolutional neural network.

[0005] In order to solve the above technical problems, the first aspect of the present invention discloses a weight segmentation quantization method based on a convolutional neural network, the method comprising:

[0006] Obtain all weights of each of the plurality of computing layers of the convolutional neural network, wherein the computing layer is a parameterized layer of the convolutional neural network;

[0007] According to the weights of all the computing layers, a target computing layer to be segmented and quantized is selected from all the computing layers;

[0008] For each of the target computing layers, segmenting the weights of the target computing layer according to the ranges to which all weights of the target computing layer belong, to obtain a weight segmentation result of the target computing layer, wherein the weight segmentation result includes a first weight group and a second weight group;

[0009] The first weight group and the second weight group of the target computing layer are piecewise quantized to obtain a quantization bit width of the target computing layer.

[0010] As an optional implementation manner, in the first aspect of the present invention, the step of selecting a target computing layer to be segmented and quantized from all the computing layers according to the weights of all the computing layers comprises:

[0011] According to all the weights of each of the computing layers, a weight histogram of each of the computing layers is constructed, wherein all the square areas contained in the weight histogram are used to represent the weight distribution of the corresponding computing layer, and each of the square areas is an area composed of a weight as a height and a group distance mapped to the weight in the weight histogram as a length;

[0012] For each of the calculation layers, judging whether the weight distribution of the calculation layer satisfies a preset tailing distribution condition according to the weight histogram of the calculation layer;

[0013] When it is determined that the weight distribution of the calculation layer satisfies the tailing distribution condition, the calculation layer is determined as a target calculation layer to be segmented and quantized;

[0014] When it is determined that the weight distribution of the computing layer does not satisfy the tailing distribution condition, a loss calculation is performed on the computing layer according to all weights of the computing layer to obtain a quantization information loss of the computing layer, and it is determined whether the quantization information loss of the computing layer is greater than a preset loss value;

[0015] When it is determined that the quantization information loss of the computing layer is greater than the preset loss value, the computing layer is determined as a target computing layer to be segmented quantized;

[0016] When it is determined that the weight distribution of the computing layer does not satisfy the tailing distribution condition and the quantization information loss of the computing layer is less than or equal to the preset loss value, the operation of determining whether the weight distribution of the computing layer satisfies the preset tailing distribution condition is re-executed for each computing layer according to the weight histogram of the computing layer until all the computing layers are screened.

[0017] As an optional implementation manner, in the first aspect of the present invention, for each of the computing layers, performing loss calculation on the computing layer according to all weights of the computing layer to obtain the quantized information loss of the computing layer includes:

[0018] According to all weights of the calculation layer, floating point calculations are performed on all weights of the calculation layer to obtain floating point results of the calculation layer;

[0019] According to all weights of the calculation layer, all weights of the calculation layer are quantized to obtain a quantization result of the calculation layer;

[0020] According to the floating-point result and the quantization result of the calculation layer, the loss of the calculation layer is calculated to obtain the quantization information loss of the calculation layer.

[0021] As an optional implementation manner, in the first aspect of the present invention, for each of the computing layers, judging whether the weight distribution of the computing layer satisfies a preset tailing distribution condition according to the weight histogram of the computing layer includes:

[0022] According to the weighted histogram of the calculation layer, a target area satisfying a preset area condition is determined from the weighted histogram, wherein the number of square areas in the target area is less than the number of square areas in the weighted histogram;

[0023] Calculating the sum of the length values ​​of all square areas in the target area to obtain the area length of the target area, and determining whether the area length of the target area is less than a preset length value;

[0024] When it is determined that the area length of the target area is less than the preset length value, determining that the weight distribution of the calculation layer satisfies a preset tailing distribution condition;

[0025] When it is determined that the region length of the target region is greater than or equal to the preset length value, it is determined that the weight distribution of the calculation layer does not satisfy the preset tailing distribution condition.

[0026] As an optional implementation manner, in the first aspect of the present invention, for each of the calculation layers, determining the target area that meets the preset area condition from the weighted histogram according to the weighted histogram of the calculation layer includes:

[0027] According to the weighted histogram of the calculation layer, a square area whose height value is the maximum height value is selected from all the square areas of the weighted histogram as a reference area;

[0028] A new area consisting of the reference area and all square areas adjacent to the reference area is used as a candidate area;

[0029] Calculate the area of ​​the candidate region according to the areas corresponding to all square regions in the candidate region;

[0030] Determine whether the area of ​​the candidate area is greater than or equal to the area of ​​the preset area;

[0031] When it is determined that the area of ​​the candidate area is smaller than the area of ​​the preset area, the candidate area is determined as a new reference area, and the operation of using the reference area and all the square areas adjacent to the reference area as a new area of ​​the candidate area is triggered;

[0032] When it is determined that the area of ​​the candidate region is greater than or equal to the area of ​​the preset region, the candidate region is determined as a target region that meets the preset area condition.

[0033] As an optional implementation, in the first aspect of the present invention, for each of the target computing layers, according to the range to which all weights of the target computing layer belong, the weight of the target computing layer is segmented to obtain the weight segmentation result of the target computing layer, including:

[0034] For each of the target computing layers, according to the region length in the target region of the target computing layer determined based on the weight histogram of the target computing layer, determining the maximum value and the minimum value of the endpoints of the region length in the abscissa of the weight histogram;

[0035] For each of the weights in each of the target computing layers, determining whether the weight is greater than the endpoint minimum value and less than the endpoint maximum value;

[0036] When it is determined that the weight is greater than the minimum endpoint value and less than the maximum endpoint value, the weight is classified into the first weight group;

[0037] When it is determined that the weight is less than or equal to the minimum value of the endpoint, or greater than or equal to the maximum value of the endpoint, the weight is classified into the second weight group.

[0038] As an optional implementation manner, in the first aspect of the present invention, for each of the target computing layers, the step of performing piecewise quantization on the first weight group and the second weight group of the target computing layer to obtain a quantization bit width of the target computing layer includes:

[0039] Performing quantization calculation on all weights in the first weight group of the target calculation layer to obtain a first weight quantization result of the target calculation layer;

[0040] Performing quantization calculation on all weights in the second weight group of the target calculation layer to obtain a second weight quantization result of the target calculation layer;

[0041] The first weighted quantization result and the second weighted quantization result of the target calculation layer are accumulated to obtain the quantization accumulated result of the target calculation layer, and a preset bias value is added to the quantization accumulated result of the target calculation layer to obtain the quantization bit width of the target calculation layer.

[0042] The second aspect of the present invention discloses a weight segmentation quantization device based on a convolutional neural network, the device comprising:

[0043] An acquisition module, used to acquire all weights of each of the multiple computing layers of the convolutional neural network, wherein the computing layer is a parameterized layer of the convolutional neural network;

[0044] A screening module, used for screening out a target computing layer to be segmented and quantized from all the computing layers according to the weights of all the computing layers;

[0045] A segmentation module is used for segmenting the weight of each target computing layer according to the range to which all weights of the target computing layer belong, to obtain a weight segmentation result of the target computing layer, wherein the weight segmentation result includes a first weight group and a second weight group;

[0046] The quantization module is used to perform piecewise quantization on the first weight group and the second weight group of the target computing layer to obtain the quantization bit width of the target computing layer.

[0047] As an optional implementation, in the second aspect of the present invention, the screening module screens out the target computing layer to be segmented and quantized from all the computing layers according to the weights of all the computing layers, specifically including:

[0048] According to all the weights of each of the computing layers, a weight histogram of each of the computing layers is constructed, wherein all the square areas contained in the weight histogram are used to represent the weight distribution of the corresponding computing layer, and each of the square areas is an area composed of a weight as a height and a group distance mapped to the weight in the weight histogram as a length;

[0049] For each of the calculation layers, judging whether the weight distribution of the calculation layer satisfies a preset tailing distribution condition according to the weight histogram of the calculation layer;

[0050] When it is determined that the weight distribution of the calculation layer satisfies the tailing distribution condition, the calculation layer is determined as a target calculation layer to be segmented and quantized;

[0051] When it is determined that the weight distribution of the computing layer does not satisfy the tailing distribution condition, a loss calculation is performed on the computing layer according to all weights of the computing layer to obtain a quantization information loss of the computing layer, and it is determined whether the quantization information loss of the computing layer is greater than a preset loss value;

[0052] When it is determined that the quantization information loss of the computing layer is greater than the preset loss value, the computing layer is determined as a target computing layer to be segmented quantized;

[0053] When it is determined that the weight distribution of the computing layer does not satisfy the tailing distribution condition and the quantization information loss of the computing layer is less than or equal to the preset loss value, the operation of determining whether the weight distribution of the computing layer satisfies the preset tailing distribution condition is re-executed for each computing layer according to the weight histogram of the computing layer until all the computing layers are screened.

[0054] As an optional implementation, in the second aspect of the present invention, for each of the computing layers, the screening module performs loss calculation on the computing layer according to all weights of the computing layer, and the manner of obtaining the quantitative information loss of the computing layer specifically includes:

[0055] According to all weights of the calculation layer, floating point calculations are performed on all weights of the calculation layer to obtain floating point results of the calculation layer;

[0056] According to all weights of the calculation layer, all weights of the calculation layer are quantized to obtain a quantization result of the calculation layer;

[0057] According to the floating-point result and the quantization result of the calculation layer, the loss of the calculation layer is calculated to obtain the quantization information loss of the calculation layer.

[0058] As an optional implementation, in the second aspect of the present invention, for each of the calculation layers, the screening module determines whether the weight distribution of the calculation layer satisfies the preset tailing distribution condition according to the weight histogram of the calculation layer, specifically including:

[0059] According to the weighted histogram of the calculation layer, a target area satisfying a preset area condition is determined from the weighted histogram, wherein the number of square areas in the target area is less than the number of square areas in the weighted histogram;

[0060] Calculating the sum of the length values ​​of all square areas in the target area to obtain the area length of the target area, and determining whether the area length of the target area is less than a preset length value;

[0061] When it is determined that the area length of the target area is less than the preset length value, determining that the weight distribution of the calculation layer satisfies a preset tailing distribution condition;

[0062] When it is determined that the region length of the target region is greater than or equal to the preset length value, it is determined that the weight distribution of the calculation layer does not satisfy the preset tailing distribution condition.

[0063] As an optional implementation, in the second aspect of the present invention, for each of the calculation layers, the screening module determines the target area that meets the preset area condition from the weighted histogram according to the weighted histogram of the calculation layer, specifically including:

[0064] According to the weighted histogram of the calculation layer, a square area whose height value is the maximum height value is selected from all the square areas of the weighted histogram as a reference area;

[0065] A new area consisting of the reference area and all square areas adjacent to the reference area is used as a candidate area;

[0066] Calculate the area of ​​the candidate region according to the areas corresponding to all square regions in the candidate region;

[0067] Determine whether the area of ​​the candidate area is greater than or equal to the area of ​​the preset area;

[0068] When it is determined that the area of ​​the candidate area is smaller than the area of ​​the preset area, the candidate area is determined as a new reference area, and the operation of using the reference area and all the square areas adjacent to the reference area as a new area of ​​the candidate area is triggered;

[0069] When it is determined that the area of ​​the candidate region is greater than or equal to the area of ​​the preset region, the candidate region is determined as a target region that meets the preset area condition.

[0070] As an optional implementation, in the second aspect of the present invention, the segmentation module segments the weight of each target computing layer according to the range to which all weights of the target computing layer belong, and the method of obtaining the weight segmentation result of the target computing layer specifically includes:

[0071] For each of the target computing layers, according to the region length in the target region of the target computing layer determined based on the weight histogram of the target computing layer, determining the maximum value and the minimum value of the endpoints of the region length in the abscissa of the weight histogram;

[0072] For each of the weights in each of the target computing layers, determining whether the weight is greater than the endpoint minimum value and less than the endpoint maximum value;

[0073] When it is determined that the weight is greater than the minimum endpoint value and less than the maximum endpoint value, the weight is classified into the first weight group;

[0074] When it is determined that the weight is less than or equal to the minimum value of the endpoint, or greater than or equal to the maximum value of the endpoint, the weight is classified into the second weight group.

[0075] As an optional implementation, in the second aspect of the present invention, for each of the target computing layers, the quantization module performs piecewise quantization on the first weight group and the second weight group of the target computing layer, and a method of obtaining the quantization bit width of the target computing layer specifically includes:

[0076] Performing quantization calculation on all weights in the first weight group of the target calculation layer to obtain a first weight quantization result of the target calculation layer;

[0077] Performing quantization calculation on all weights in the second weight group of the target calculation layer to obtain a second weight quantization result of the target calculation layer;

[0078] The first weighted quantization result and the second weighted quantization result of the target calculation layer are accumulated to obtain the quantization accumulated result of the target calculation layer, and a preset bias value is added to the quantization accumulated result of the target calculation layer to obtain the quantization bit width of the target calculation layer.

[0079] The third aspect of the present invention discloses another weight segmentation quantization device based on a convolutional neural network, the device comprising:

[0080] A memory storing executable program code;

[0081] a processor coupled to the memory;

[0082] The processor calls the executable program code stored in the memory to execute the weight segmentation quantization method based on convolutional neural network disclosed in the first aspect of the present invention.

[0083] The fourth aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute the weight segmentation quantization method based on convolutional neural network disclosed in the first aspect of the present invention.

[0084] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0085] In an embodiment of the present invention, all weights of each computing layer in multiple computing layers of a convolutional neural network are obtained, and the computing layer is a parameterized layer of the convolutional neural network. According to the weights of all computing layers, a target computing layer to be segmented and quantized is screened out from all computing layers. For each target computing layer, according to all weights of the target computing layer, the weight of the target computing layer is segmented to obtain a weight segmentation result of the target computing layer, and the weight segmentation result includes a first weight group and a second weight group. The first weight group and the second weight group of the target computing layer are segmented and quantized to obtain the quantization bit width of the target computing layer. It can be seen that the implementation of the present invention can obtain all weights of each computing layer in multiple computing layers of the convolutional neural network, and according to the weights of all computing layers, the target computing layer to be segmented and quantized is screened out from all computing layers. Then, for each target computing layer, according to the range of all weights of the target computing layer, the weight of the target computing layer is segmented to obtain the weight segmentation result of the target computing layer (the weight segmentation result includes the first weight group and the second weight group), and then the first weight group and the second weight group of the target computing layer are segmented and quantized to obtain the quantization bit width of the target computing layer. The target computing layer that needs to be segmented and quantized can be accurately screened out through the weights of all computing layers of the convolutional neural network, which is conducive to improving the accuracy and efficiency of the target computing layer obtained by screening, and the segmentation accuracy and segmentation efficiency of the target computing layer weight can be improved through the range of the weight of the target computing layer, and each group of weights can be accurately quantized through the weight segmentation result obtained by accurate segmentation, which is conducive to improving the quantization accuracy of the weights of each layer in the convolutional neural network, thereby improving the network accuracy, and the way of segmenting the weights is compared with the mixed precision quantization way adopted in the prior art. The way is conducive to making the increased amount of calculation as small as possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0087] Figure 1 It is a flowchart of a weight piecewise quantization method based on a convolutional neural network disclosed in an embodiment of the present invention;

[0088] Figure 2 It is a flowchart of another weight segmentation quantization method based on a convolutional neural network disclosed in an embodiment of the present invention;

[0089] Figure 3 is a schematic diagram of a weight histogram disclosed in an embodiment of the present invention;

[0090] Figure 4 It is a weight segmentation schematic diagram disclosed in an embodiment of the present invention;

[0091] Figure 5 It is a structural schematic diagram of a weight segmentation quantization device based on a convolutional neural network disclosed in an embodiment of the present invention;

[0092] Figure 6 It is a structural schematic diagram of another weight segmentation quantization device based on a convolutional neural network disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0093] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0094] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0095] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0096] The present invention discloses a weight segmentation quantization method and device based on a convolutional neural network, which can obtain all weights of each computing layer in multiple computing layers of a convolutional neural network, and select a target computing layer to be segmented and quantized from all computing layers according to the weights of all computing layers, and then for each target computing layer, according to the range to which all weights of the target computing layer belong, the weight of the target computing layer is segmented to obtain the weight segmentation result of the target computing layer (the weight segmentation result includes a first weight group and a second weight group), and then the first weight group and the second weight group of the target computing layer are segmented and quantized to obtain the quantization bit width of the target computing layer. The target computing layer that needs to be quantized by segmentation can be accurately screened out through the weights of all computing layers of the convolutional neural network, which is beneficial to improving the accuracy and efficiency of the target computing layer screened, and the segmentation accuracy and efficiency of the target computing layer weight can be improved through the weight range of the target computing layer, and each group of weights can be accurately quantized through the weight segmentation results obtained by accurate segmentation, which is beneficial to improving the quantization accuracy of the weights of each layer in the convolutional neural network, thereby improving the network accuracy, and the way of quantizing the weights by segmentation is beneficial to making the increased amount of calculation as small as possible compared to the mixed precision quantization method used in the prior art. The following are detailed descriptions.

[0097] Embodiment 1

[0098] See also Figure 1 , Figure 1 is a flow chart of a weight segmentation quantization method based on a convolutional neural network disclosed in an embodiment of the present invention. Figure 1 The described weight segmentation quantization method based on convolutional neural network can be applied to a weight segmentation quantization device based on convolutional neural network, wherein the device may include a quantization device or a quantization server, wherein the quantization server may include a cloud server or a local server, which is not limited in the embodiment of the present invention. Figure 1 As shown, the weight segmentation quantization method based on convolutional neural network may include the following operations:

[0099] 101. Obtain all weights of each of the multiple computing layers of the convolutional neural network.

[0100] In the embodiment of the present invention, the computing layer is a parameterized layer of a convolutional neural network. Optionally, the parameterized layer may be a convolutional layer, a linear layer, or any other layer containing weight parameters, which is not limited in the embodiment of the present invention.

[0101] 102. According to the weights of all the computing layers, a target computing layer to be segmented and quantized is selected from all the computing layers.

[0102] In an embodiment of the present invention, optionally, the weight distribution of each computing layer can be analyzed according to the weight of each computing layer, and according to the weight distribution of all computing layers, the computing layers whose weight distribution meets the preset tailing distribution condition can be screened out from all computing layers as the target computing layers to be segmented quantized; the quantization information loss of each computing layer can also be analyzed according to the weight of each computing layer, and according to the quantization information loss of all computing layers, the computing layers whose quantization information loss is greater than the preset loss value can be screened out from all computing layers as the target computing layers to be segmented quantized. Further optionally, the computing layers screened by quantization information loss can be all the above-mentioned computing layers, or can be the remaining computing layers whose weight distribution does not meet the tailing distribution condition, and for the remaining computing layers whose weight distribution does not meet the tailing distribution condition and whose quantization information loss is less than or equal to the preset loss value, the parameters of the tailing distribution condition and / or the parameters of the preset loss value can be adjusted first and then the above-mentioned screening process can be re-executed, which is not limited in the embodiment of the present invention.

[0103] 103. For each target computing layer, the weights of the target computing layer are segmented according to the ranges to which all weights of the target computing layer belong, to obtain a weight segmentation result of the target computing layer.

[0104] In an embodiment of the present invention, the weight segmentation result includes a first weight group and a second weight group. Among them, all weights included in the first weight group do not overlap with all weights included in the second weight group, and the sum of the number of weights in the first weight group and the number of weights in the second weight group is equal to the total number of weights in the target calculation layer. Specifically, it can be determined whether the range to which each weight of the target calculation layer belongs is the determined target range. If the judgment result is yes, the weight is classified into the first weight group; if the judgment result is no, the weight is classified into the second weight group.

[0105] 104. Perform piecewise quantization on the first weight group and the second weight group of the target computing layer to obtain a quantization bit width of the target computing layer.

[0106] It can be seen that implementation Figure 1The described weight piecewise quantization method based on convolutional neural network can obtain all weights of each computing layer in multiple computing layers of convolutional neural network, and screen out the target computing layer to be piecewise quantized from all computing layers according to the weights of all computing layers, and then for each target computing layer, the weight of the target computing layer is segmented according to the range to which all weights of the target computing layer belong, and the weight segmentation result of the target computing layer is obtained (the weight segmentation result includes the first weight group and the second weight group), and then the first weight group and the second weight group of the target computing layer are piecewise quantized to obtain the quantization bit width of the target computing layer, which can be obtained by The weights of all computing layers of the convolutional neural network accurately screen out the target computing layers that need to be piecewise quantized, which is beneficial to improving the accuracy and efficiency of screening the target computing layers, and can improve the segmentation accuracy and efficiency of the target computing layer weights through the weight range of the target computing layer, and can accurately quantize each group of weights through the weight segmentation results obtained by accurate segmentation, which is beneficial to improving the quantization accuracy of the weights of each layer in the convolutional neural network, thereby helping to improve the network accuracy, and the way of piecewise quantizing weights is beneficial to minimizing the increased amount of calculation as much as possible compared to the mixed precision quantization way used in the prior art.

[0107] In an optional embodiment, for each target computing layer, performing piecewise quantization on the first weight group and the second weight group of the target computing layer in step 104 to obtain the quantization bit width of the target computing layer may include:

[0108] Performing quantization calculation on all weights in the first weight group of the target calculation layer to obtain a first weight quantization result of the target calculation layer;

[0109] Performing quantization calculation on all weights in the second weight group of the target calculation layer to obtain a second weight quantization result of the target calculation layer;

[0110] The first weighted quantization result and the second weighted quantization result of the target calculation layer are accumulated to obtain the quantization accumulated result of the target calculation layer, and a preset bias value is added to the quantization accumulated result of the target calculation layer to obtain the quantization bit width of the target calculation layer.

[0111] In an embodiment of the present invention, specifically, for each target computing layer, all weights in the first weight group of the target computing layer are quantized to obtain a first weight quantization result of the target computing layer, which may include: according to all weights in the first weight group of the target computing layer and the input Tensor (i.e., tensor) of the target computing layer, calculating a first quantization bit width that matches all weights in the first weight group as the first weight quantization result of the target computing layer.

[0112] Specifically, for each target computing layer, all weights in the second weight group of the target computing layer are quantized to obtain the second weight quantization result of the target computing layer, which may include: according to all weights in the second weight group of the target computing layer and the input Tensor of the target computing layer, calculating the second quantization bit width that matches all weights in the second weight group as the second weight quantization result of the target computing layer.

[0113] Specifically, for each target computing layer, a first quantization bit width that matches all weights in the first weight group and a second quantization bit width that matches all weights in the second weight group in the target computing layer are accumulated to obtain a quantization accumulation result of the target computing layer, and a preset bias value (bias) is added to the quantization accumulation result to obtain the quantization bit width of the target computing layer.

[0114] Specifically, the quantization bit width of each target calculation layer can be calculated as follows:

[0115] Aout=conv(w,ain)+bias=conv(w0,ain)+conv(w1,ain)+bias;

[0116] Among them, Aout is the quantization bit width of the target computing layer, w is any weight of the target computing layer, ain is the input Tensor of the target computing layer, conv(w,ain) is the quantization bit width calculated by the target computing layer based on the weight and the input Tensor, bias is the preset bias, w0 is any weight in the first weight group of the target computing layer, conv(w0,ain) is the first quantization bit width calculated by the target computing layer based on the first weight group and the input Tensor, w is any weight in the second weight group of the target computing layer, conv(w1,ain) is the second quantization bit width calculated by the target computing layer based on the second weight group and the input Tensor.

[0117] It can be seen that this optional embodiment can perform quantization calculation on all weights in the first weight group of each target calculation layer to obtain the first weight quantization result of the target calculation layer, and perform quantization calculation on all weights in the second weight group of the target calculation layer to obtain the second weight quantization result of the target calculation layer, and then accumulate the first weight quantization result and the second weight quantization result of the target calculation layer to obtain the quantization accumulation result of the target calculation layer, and then add the preset bias value to the quantization accumulation result of the target calculation layer to obtain the quantization bit width of the target calculation layer, and can accurately calculate the quantization bit width of the target calculation layer by performing segmented quantization on the first weight group and the second weight group, which is beneficial to improving the accuracy and reliability of the calculated quantization bit width, and compared with the mixed precision quantization method in the prior art, this scheme can reduce the mixed quantization bit width requirements for hardware devices through the unified quantization bit width used through segmented quantization, which is beneficial to saving costs and improving computing efficiency.

[0118] Embodiment 2

[0119] See also Figure 2 , Figure 2 is a flow chart of a weight segmentation quantization method based on a convolutional neural network disclosed in an embodiment of the present invention. Figure 2 The described weight segmentation quantization method based on convolutional neural network can be applied to a weight segmentation quantization device based on convolutional neural network, wherein the device may include a quantization device or a quantization server, wherein the quantization server may include a cloud server or a local server, which is not limited in the embodiment of the present invention. Figure 2 As shown, the weight segmentation quantization method based on convolutional neural network may include the following operations:

[0120] 201. Obtain all weights of each computing layer in multiple computing layers of the convolutional neural network.

[0121] 202. Construct a weight histogram of each computing layer based on all weights of each computing layer.

[0122] In the embodiment of the present invention, the weight histogram may include a plurality of square regions, and each square region is a region composed of a weight as a height and a group distance mapped to the weight in the weight histogram as a length.

[0123] 203. For each calculation layer, determine whether the weight distribution of the calculation layer meets a preset tailing distribution condition according to the weight histogram of the calculation layer.

[0124] In an embodiment of the present invention, optionally, the tailing distribution condition can be used to represent the area range to which the area of ​​the target area in the weight histogram of the corresponding calculation layer belongs, or can be used to represent the length range to which the area length of the target area in the weight histogram of the corresponding calculation layer belongs, and the embodiment of the present invention is not limited thereto.

[0125] In an embodiment of the present invention, when the judgment result of step 203 is no, when it is judged that the weight distribution of the calculation layer does not meet the tailing distribution condition, step 204 is triggered to be executed; when the judgment result of step 203 is yes, that is, when it is judged that the weight distribution of the calculation layer meets the tailing distribution condition, step 206 is triggered to be executed.

[0126] 204. Perform loss calculation on the calculation layer according to all weights of the calculation layer to obtain quantized information loss of the calculation layer.

[0127] 205. Determine whether the quantization information loss of the calculation layer is greater than a preset loss value.

[0128] In an embodiment of the present invention, when the judgment result of step 205 is yes, that is, when it is judged that the quantization information loss of the calculation layer is greater than the preset loss value, step 206 is triggered to be executed; when the judgment result of step 205 is no, that is, when it is judged that the weight distribution of the calculation layer does not meet the tailing distribution condition and the quantization information loss of the calculation layer is less than or equal to the preset loss value, the operation in step 203 is re-executed until all calculation layers are screened.

[0129] In an embodiment of the present invention, optionally, after determining that the weight distribution of the computing layer does not satisfy the tailing distribution condition and the quantization information loss of the computing layer is less than or equal to the preset loss value, and before re-executing the operation in step 203, the judgment parameters and / or the preset loss values ​​included in the tailing distribution condition can be adjusted to obtain the adjusted parameters, and then trigger the execution of the operation in step 203.

[0130] 206. Determine the calculation layer as a target calculation layer to be segmented and quantized.

[0131] 207. For each target computing layer, segment the weights of the target computing layer according to the ranges to which all weights of the target computing layer belong, and obtain a weight segmentation result of the target computing layer.

[0132] 208. Perform piecewise quantization on the first weight group and the second weight group of the target computing layer to obtain a quantization bit width of the target computing layer.

[0133] In the embodiment of the present invention, for other descriptions of step 201, step 207 and step 208, please refer to the detailed description of step 101, step 103 and step 104 in the first embodiment, which is not limited in the embodiment of the present invention.

[0134] It can be seen that implementation Figure 2The described weight piecewise quantization method based on convolutional neural network can obtain all weights of each computing layer in multiple computing layers of convolutional neural network, and screen out the target computing layer to be piecewise quantized from all computing layers according to the weights of all computing layers, and then for each target computing layer, the weight of the target computing layer is segmented according to the range to which all weights of the target computing layer belong, and the weight segmentation result of the target computing layer is obtained (the weight segmentation result includes the first weight group and the second weight group), and then the first weight group and the second weight group of the target computing layer are piecewise quantized to obtain the quantization bit width of the target computing layer, which can be obtained by The weights of all computing layers of the convolutional neural network accurately screen out the target computing layers that need to be piecewise quantized, which is beneficial to improving the accuracy and efficiency of screening the target computing layers, and can improve the segmentation accuracy and efficiency of the target computing layer weights through the weight range of the target computing layer, and can accurately quantize each group of weights through the weight segmentation results obtained by accurate segmentation, which is beneficial to improving the quantization accuracy of the weights of each layer in the convolutional neural network, thereby helping to improve the network accuracy, and the way of piecewise quantizing weights is beneficial to minimizing the increased amount of calculation as much as possible compared to the mixed precision quantization way used in the prior art.In addition, it is also possible to construct a weight histogram of each computing layer based on all the weights of each computing layer, and for each computing layer, determine whether the weight distribution of the computing layer meets the preset tailing distribution condition based on the weight histogram of the computing layer. When it is determined that the weight distribution of the computing layer meets the tailing distribution condition, the computing layer is determined as the target computing layer to be segmented and quantized. When it is determined that the weight distribution of the computing layer does not meet the tailing distribution condition, the computing layer is subjected to loss calculation based on all the weights of the computing layer to obtain the quantization information loss of the computing layer, and determine whether the quantization information loss of the computing layer is greater than the preset loss value. When it is determined that the quantization information loss of the computing layer is greater than the preset loss value, the computing layer is determined as the target computing layer to be segmented and quantized. When it is determined that the weight distribution of the computing layer does not meet the tailing distribution condition and the quantization information loss of the computing layer is less than or equal to the preset loss value, re-execute for each computing layer, based on According to the weight histogram of the calculation layer, it is judged whether the weight distribution of the calculation layer meets the preset tailing distribution condition, until all the calculation layers are screened, the weight histogram of each calculation layer can be accurately constructed through all the weights of the calculation layer, and it is possible to accurately judge whether the weight distribution of the calculation layer meets the preset tailing distribution condition through the weight histogram of each calculation layer, and when it is judged that the tailing distribution condition is met, the calculation layer is accurately determined as the target calculation layer that needs to be piecewise quantized, and when it is judged that the tailing distribution condition is not met, it is further judged whether the quantization information loss of the calculation layer calculated based on all the weights of the calculation layer is greater than the preset loss value, and when it is judged that it is greater than the preset loss value, the calculation layer is accurately determined as the target calculation layer that needs to be piecewise quantized, which is conducive to improving the diversity and flexibility of the screening method of the target calculation layer, and is conducive to improving the accuracy and reliability of the screened target calculation layer.

[0135] In an optional embodiment, for each computing layer, performing loss calculation on the computing layer according to all weights of the computing layer in step 205 to obtain the quantized information loss of the computing layer may include:

[0136] According to all weights of the calculation layer, floating point calculations are performed on all weights of the calculation layer to obtain floating point results of the calculation layer;

[0137] According to all weights of the calculation layer, all weights of the calculation layer are quantized to obtain a quantization result of the calculation layer;

[0138] According to the floating-point result and the quantization result of the calculation layer, the loss of the calculation layer is calculated to obtain the quantization information loss of the calculation layer.

[0139] In the embodiment of the present invention, the calculation formula of the quantization information loss of any computing layer is as follows:

[0140] S = (fq) / f;

[0141] Among them, S is the quantization information loss of the calculation layer, f is the floating point result of the calculation layer, and q is the quantization result of the calculation layer.

[0142] It can be seen that this optional embodiment can perform floating-point calculations on all weights of each calculation layer according to all weights of the calculation layer to obtain the floating-point results of the calculation layer, and quantize all weights of the calculation layer according to all weights of the calculation layer to obtain the quantization results of the calculation layer, and perform loss calculation on the calculation layer according to the floating-point results and quantization results of the calculation layer to obtain the quantization information loss of the calculation layer. The floating-point results and quantization results of the calculation layer can be accurately calculated through all weights of each calculation layer, and the quantization information loss of the calculation layer can be accurately calculated through the floating-point results and quantization results of the calculation layer, which is beneficial to improving the accuracy and reliability of the calculated quantization information loss.

[0143] In another optional embodiment, for each computing layer, judging whether the weight distribution of the computing layer satisfies a preset tailing distribution condition according to the weight histogram of the computing layer in step 203 may include:

[0144] According to the weighted histogram of the calculation layer, a target area satisfying a preset area condition is determined from the weighted histogram;

[0145] Calculate the sum of the length values ​​of all square areas in the target area to obtain the area length of the target area, and determine whether the area length of the target area is less than a preset length value;

[0146] When it is determined that the region length of the target region is less than a preset length value, determining that the weight distribution of the calculation layer satisfies a preset tailing distribution condition;

[0147] When it is determined that the region length of the target region is greater than or equal to the preset length value, it is determined that the weight distribution of the calculation layer does not satisfy the preset tailing distribution condition.

[0148] In an embodiment of the present invention, the preset area condition can be used to represent the condition of the area range to which the area of ​​the target area in the weight histogram of the corresponding calculation layer belongs, and the number of square areas in the target area is less than the number of square areas in the weight histogram. Optionally, the preset length value can be a value obtained by multiplying the first preset coefficient (such as 0.25) by the total length of the horizontal axis of the above-mentioned weight histogram, or it can be any other pre-set value, which is not limited in the embodiment of the present invention.

[0149] It can be seen that this optional embodiment can determine the target area that meets the preset area condition from the weight histogram of each calculation layer, and calculate the sum of the length values ​​of all square areas in the target area to obtain the area length of the target area, and judge whether the area length of the target area is less than the preset length value. When it is judged that the area length of the target area is less than the preset length value, it is determined that the weight distribution of the calculation layer meets the preset tailing distribution condition. When it is judged that the area length of the target area is greater than or equal to the preset length value, it is determined that the weight distribution of the calculation layer does not meet the preset tailing distribution condition. The weight histogram of each calculation layer can be used to accurately obtain the target area that meets the preset area condition. Determining the target area that meets the preset area condition in the weight histogram is beneficial to improving the accuracy and efficiency of determining the target area, and can accurately determine that the weight distribution of the calculation layer meets the preset tailing distribution condition when it is judged that the area length of the calculated target area is less than the preset length value, which is beneficial to improving the accuracy and reliability of the calculation layer whose weight distribution meets the tailing distribution condition, and can accurately determine that the weight distribution of the calculation layer does not meet the preset tailing distribution condition when it is judged that the area length is greater than or equal to the preset length value, which is beneficial to improving the accuracy and reliability of the calculation layer whose weight distribution does not meet the tailing distribution condition.

[0150] In this optional embodiment, as an optional implementation manner, for each calculation layer, according to the weight histogram of the calculation layer, determining the target area that meets the preset area condition from the weight histogram may include:

[0151] According to the weight histogram of the calculation layer, a square area with a maximum height value is selected from all square areas of the weight histogram as a reference area;

[0152] A new area consisting of the base area and all the square areas adjacent to the base area is used as a candidate area;

[0153] Calculate the area of ​​the candidate region according to the areas corresponding to all square regions in the candidate region;

[0154] Determine whether the area of ​​the candidate area is greater than or equal to the area of ​​the preset area;

[0155] When it is determined that the area of ​​the candidate region is smaller than the area of ​​the preset region, the candidate region is determined as a new reference region, and an operation of using the reference region and all square regions adjacent to the reference region as a new region is triggered;

[0156] When it is determined that the area of ​​the candidate region is greater than or equal to the area of ​​the preset region, the candidate region is determined as the target region that meets the preset area condition.

[0157] In an embodiment of the present invention, the area of ​​the preset region may be the area obtained by multiplying the second preset coefficient (such as 0.7) by the total area of ​​all square regions in the above-mentioned weight histogram. Specifically, for each calculation layer, the area of ​​the candidate region of the calculation layer is recorded as S, and the total area of ​​the weight histogram of the calculation layer is recorded as S total. When S<S total*0.7, the search range is gradually expanded until S≥Stotal*0.7, the candidate region is determined as the target region that meets the preset area condition, and the region length of the target region is recorded as length_x=right-left, wherein right is the rightmost horizontal coordinate value of the rightmost square region in the target region, and left is the leftmost horizontal coordinate value of the leftmost square region in the target region. When length_x<0.25*Length_total, it is determined that the weight distribution of the calculation layer meets the tailing distribution condition. When length_x≥0.25*Length_total, it is determined that the weight distribution of the calculation layer does not meet the tailing distribution condition. Exemplarily, such as Figure 3 As shown, Figure 3 is a schematic diagram of a weight histogram disclosed in an embodiment of the present invention, Figure 3 The area composed of all square areas between left and right in is the above target area.

[0158] It can be seen that this optional implementation method can, according to the weight histogram of each calculation layer, screen out the square area with the maximum height value from all the square areas of the weight histogram as the reference area, and use the reference area and a new area composed of all the square areas adjacent to the reference area as the candidate area, and calculate the area of ​​the candidate area according to the areas corresponding to all the square areas within the candidate area, and then judge whether the area of ​​the candidate area is greater than or equal to the preset area. When it is judged that the area of ​​the candidate area is less than the preset area, the candidate area is determined as the new reference area, and the execution of the operation of using the reference area and a new area composed of all the square areas adjacent to the reference area as the candidate area is triggered. When it is judged that the area of ​​the candidate area is less than the preset area, the candidate area is determined as the new reference area. When the area of ​​the candidate region is greater than or equal to the area of ​​the preset region, the candidate region is determined as the target region that meets the preset area condition, and the weighted histogram of each calculation layer can be used to accurately select the reference region with the maximum height value from the square region of the weighted histogram, which is beneficial to improve the accuracy and efficiency of screening the reference region based on the weighted histogram, and can accurately determine the candidate region as the target region that meets the preset area condition when the area of ​​the candidate region is greater than or equal to the area of ​​the preset region through the area of ​​the candidate region composed of the reference region and its adjacent square regions and combined with the area of ​​the preset region, and gradually expand the search range when the area of ​​the region is less than the area of ​​the preset region, so as to accurately determine the target region that meets the preset area condition.

[0159] In yet another optional embodiment, for each target computing layer in step 208, segmenting the weight of the target computing layer according to the range to which all weights of the target computing layer belong, and obtaining the weight segmentation result of the target computing layer may include:

[0160] For each target computing layer, according to the region length in the target region of the target computing layer determined based on the weight histogram of the target computing layer, determine the maximum value and the minimum value of the endpoint of the region length in the abscissa of the weight histogram;

[0161] For each weight in each target calculation layer, determine whether the weight is greater than the minimum endpoint value and less than the maximum endpoint value;

[0162] When it is determined that the weight is greater than the minimum endpoint value and less than the maximum endpoint value, the weight is classified into the first weight group;

[0163] When it is determined that the weight is less than or equal to the minimum endpoint value, or greater than or equal to the maximum endpoint value, the weight is classified into the second weight group.

[0164] In the embodiment of the present invention, Figure 3 As shown, the area length in the target area is Figure 3The maximum value of the endpoint in the horizontal axis of the weight histogram shown is recorded as vmax=right, and the length of the region is Figure 3 The minimum value of the endpoint in the abscissa of the weight histogram is recorded as vmin=left, and the segmentation methods of the first weight group and the second weight group can be expressed as follows:

[0165] W0=w[if vmin <wi<vmax,return wi,else return 0];

[0166] W1=w[if vmin≤wi≤vmax, return 0, else return wi];

[0167] Where W0 represents the first weight group, W1 represents the second weight group, wi represents the i-th weight of the corresponding calculation layer, i=1,2,..,n (n is the number of weights in the calculation layer), and return represents the returned result. For example, for the first weight group, assume that a certain weight is less than the maximum value of the endpoint and greater than the minimum value of the endpoint. For example, Figure 4 As shown, Figure 4 is a weight segmentation schematic diagram disclosed in an embodiment of the present invention, wherein: Figure 4 Table 1 records all the weights of a certain computing layer (such as Figure 4 0.023, -0.85, -0.03, 0.045, 1.20, 0.65, -0.75, -0.018 and 1.03), Table 2 records all the weights of the first weight group of the calculation layer, and Table 3 records all the weights of the second weight group of the calculation layer.

[0168] It can be seen that this optional embodiment can determine the endpoint maximum and endpoint minimum of the region length in the horizontal axis of the weight histogram for each target calculation layer according to the region length in the target area of ​​the target calculation layer determined based on the weight histogram of the target calculation layer, and for each weight in each target calculation layer, judge whether the weight is greater than the endpoint minimum and less than the endpoint maximum. When it is judged that the weight is greater than the endpoint minimum and less than the endpoint maximum, the weight is classified into the first weight group; when it is judged that the weight is less than or equal to the endpoint minimum, or greater than or equal to the endpoint maximum, the weight is classified into the second weight group. The endpoint maximum and endpoint minimum can be accurately determined through the region length of the target area of ​​each target calculation layer, and weights of different sizes can be accurately classified into corresponding weight groups through the size relationship between the weight and the endpoint maximum and endpoint minimum, which is beneficial to improving the accuracy and reliability of weight segmentation.

[0169] Embodiment 3

[0170] See also Figure 5, Figure 5 : is a structural diagram of a weight segmentation quantization device based on a convolutional neural network disclosed in an embodiment of the present invention. Figure 5 The described weight segmentation quantization device based on convolutional neural network may include a quantization device or a quantization server, wherein the quantization server may include a cloud server or a local server, which is not limited in the embodiment of the present invention. Figure 5 As shown, the weight segmentation quantization device based on convolutional neural network may include:

[0171] An acquisition module 301 is used to acquire all weights of each computing layer in a plurality of computing layers of a convolutional neural network, where the computing layer is a parameterized layer of the convolutional neural network;

[0172] A screening module 302 is used to screen out a target computing layer to be segmented and quantized from all computing layers according to the weights of all computing layers;

[0173] The segmentation module 303 is used to segment the weights of each target computing layer according to the ranges to which all weights of the target computing layer belong, and obtain the weight segmentation result of the target computing layer, wherein the weight segmentation result includes a first weight group and a second weight group;

[0174] The quantization module 304 is used to perform piecewise quantization on the first weight group and the second weight group of the target computing layer to obtain a quantization bit width of the target computing layer.

[0175] It can be seen that implementation Figure 5 The described weight segmentation quantization device based on convolutional neural network can obtain all weights of each computing layer in multiple computing layers of the convolutional neural network, and screen out the target computing layer to be segmented and quantized from all computing layers according to the weights of all computing layers. Then, for each target computing layer, the weight of the target computing layer is segmented according to the range to which all weights of the target computing layer belong, and the weight segmentation result of the target computing layer is obtained (the weight segmentation result includes the first weight group and the second weight group), and then the first weight group and the second weight group of the target computing layer are segmented and quantized to obtain the quantization bit width of the target computing layer, which can be obtained through The weights of all computing layers of the convolutional neural network accurately screen out the target computing layers that need to be piecewise quantized, which is beneficial to improving the accuracy and efficiency of screening the target computing layers, and can improve the segmentation accuracy and efficiency of the target computing layer weights through the weight range of the target computing layer, and can accurately quantize each group of weights through the weight segmentation results obtained by accurate segmentation, which is beneficial to improving the quantization accuracy of the weights of each layer in the convolutional neural network, thereby helping to improve the network accuracy, and the way of piecewise quantizing weights is beneficial to minimizing the increased amount of calculation as much as possible compared to the mixed precision quantization way used in the prior art.

[0176] In an optional embodiment, the method in which the screening module 302 screens out the target computing layer to be segmented and quantized from all computing layers according to the weights of all computing layers may specifically include:

[0177] According to all the weights of each computing layer, a weight histogram of each computing layer is constructed. All square areas contained in the weight histogram are used to represent the weight distribution of the corresponding computing layer, and each square area is an area composed of a weight as a height and a group distance mapped to the weight in the weight histogram as a length;

[0178] For each computing layer, judging whether the weight distribution of the computing layer satisfies the preset tailing distribution condition according to the weight histogram of the computing layer;

[0179] When it is determined that the weight distribution of the calculation layer satisfies the tailing distribution condition, the calculation layer is determined as the target calculation layer to be segmented and quantized;

[0180] When it is determined that the weight distribution of the computing layer does not satisfy the tailing distribution condition, a loss calculation is performed on the computing layer according to all weights of the computing layer to obtain the quantization information loss of the computing layer, and it is determined whether the quantization information loss of the computing layer is greater than a preset loss value;

[0181] When it is determined that the quantization information loss of the calculation layer is greater than a preset loss value, the calculation layer is determined as a target calculation layer to be segmented quantized;

[0182] When it is determined that the weight distribution of the calculation layer does not meet the tailing distribution condition and the quantization information loss of the calculation layer is less than or equal to the preset loss value, the operation of re-executing for each calculation layer, judging whether the weight distribution of the calculation layer meets the preset tailing distribution condition according to the weight histogram of the calculation layer, until all calculation layers have been screened.

[0183] It can be seen that this optional embodiment can construct a weight histogram of each computing layer based on all the weights of each computing layer, and for each computing layer, determine whether the weight distribution of the computing layer meets the preset tailing distribution condition based on the weight histogram of the computing layer. When it is determined that the weight distribution of the computing layer meets the tailing distribution condition, the computing layer is determined as the target computing layer to be segmented quantized. When it is determined that the weight distribution of the computing layer does not meet the tailing distribution condition, the computing layer is subjected to loss calculation based on all the weights of the computing layer to obtain the quantization information loss of the computing layer, and determine whether the quantization information loss of the computing layer is greater than the preset loss value. When it is determined that the quantization information loss of the computing layer is greater than the preset loss value, the computing layer is determined as the target computing layer to be segmented quantized. When it is determined that the weight distribution of the computing layer does not meet the tailing distribution condition and the quantization information loss of the computing layer is less than or equal to the preset loss value, the step of re-executing the weight distribution of each computing layer is repeated. The method comprises the following steps: a) determining a weight distribution of a computing layer based on a weight histogram of the computing layer and determining whether the weight distribution of the computing layer satisfies a preset tailing distribution condition, until all computing layers are screened, and accurately constructing a weight histogram of the computing layer through all weights of the computing layer, and accurately determining whether the weight distribution of the computing layer satisfies a preset tailing distribution condition through the weight histogram of each computing layer, and accurately determining the computing layer as a target computing layer requiring piecewise quantization when it is determined that the tailing distribution condition is satisfied, and further determining whether the quantization information loss of the computing layer calculated based on all weights of the computing layer is greater than a preset loss value when it is determined that the quantization information loss of the computing layer is greater than the preset loss value when it is determined that the quantization information loss of the computing layer is greater than the preset loss value, and accurately determining the computing layer as a target computing layer requiring piecewise quantization when it is determined that the quantization information loss of the computing layer is greater than the preset loss value, which is beneficial to improving the diversity and flexibility of the screening method of the target computing layer, and is beneficial to improving the accuracy and reliability of the screened target computing layer.

[0184] In this optional embodiment, as an optional implementation, for each computing layer, the screening module 302 performs loss calculation on the computing layer according to all weights of the computing layer, and the manner of obtaining the quantitative information loss of the computing layer may specifically include:

[0185] According to all weights of the calculation layer, floating point calculations are performed on all weights of the calculation layer to obtain floating point results of the calculation layer;

[0186] According to all weights of the calculation layer, all weights of the calculation layer are quantized to obtain a quantization result of the calculation layer;

[0187] According to the floating-point result and the quantization result of the calculation layer, the loss of the calculation layer is calculated to obtain the quantization information loss of the calculation layer.

[0188] It can be seen that this optional implementation can perform floating-point calculations on all weights of each calculation layer according to all weights of the calculation layer to obtain the floating-point results of the calculation layer, and quantize all weights of the calculation layer according to all weights of the calculation layer to obtain the quantization results of the calculation layer, and perform loss calculation on the calculation layer according to the floating-point results and quantization results of the calculation layer to obtain the quantization information loss of the calculation layer. The floating-point results and quantization results of the calculation layer can be accurately calculated through all weights of each calculation layer, and the quantization information loss of the calculation layer can be accurately calculated through the floating-point results and quantization results of the calculation layer, which is beneficial to improving the accuracy and reliability of the calculated quantization information loss.

[0189] In this optional embodiment, as another optional implementation, for each calculation layer, the screening module 302 determines whether the weight distribution of the calculation layer satisfies the preset tailing distribution condition according to the weight histogram of the calculation layer, and the method may specifically include:

[0190] According to the weighted histogram of the calculation layer, a target area satisfying a preset area condition is determined from the weighted histogram, and the number of square areas in the target area is less than the number of square areas in the weighted histogram;

[0191] Calculate the sum of the length values ​​of all square areas in the target area to obtain the area length of the target area, and determine whether the area length of the target area is less than a preset length value;

[0192] When it is determined that the region length of the target region is less than a preset length value, determining that the weight distribution of the calculation layer satisfies a preset tailing distribution condition;

[0193] When it is determined that the region length of the target region is greater than or equal to the preset length value, it is determined that the weight distribution of the calculation layer does not satisfy the preset tailing distribution condition.

[0194] It can be seen that this optional implementation can determine the target area that meets the preset area condition from the weight histogram of each calculation layer, and calculate the sum of the length values ​​of all square areas in the target area to obtain the area length of the target area, and judge whether the area length of the target area is less than the preset length value. When it is judged that the area length of the target area is less than the preset length value, it is determined that the weight distribution of the calculation layer meets the preset tailing distribution condition. When it is judged that the area length of the target area is greater than or equal to the preset length value, it is determined that the weight distribution of the calculation layer does not meet the preset tailing distribution condition. The weight histogram of each calculation layer can be used to accurately Determining the target area that meets the preset area condition from the weight histogram is beneficial to improving the accuracy and efficiency of determining the target area, and can accurately determine that the weight distribution of the calculation layer meets the preset tailing distribution condition when it is judged that the area length of the calculated target area is less than the preset length value, which is beneficial to improving the accuracy and reliability of determining that the weight distribution of the calculation layer meets the tailing distribution condition, and can accurately determine that the weight distribution of the calculation layer does not meet the preset tailing distribution condition when it is judged that the area length is greater than or equal to the preset length value, which is beneficial to improving the accuracy and reliability of determining that the weight distribution of the calculation layer does not meet the tailing distribution condition.

[0195] In this optional implementation, optionally, for each calculation layer, the screening module 302 determines the target area that meets the preset area condition from the weighted histogram according to the weighted histogram of the calculation layer, and the method may specifically include:

[0196] According to the weight histogram of the calculation layer, a square area with a maximum height value is selected from all square areas of the weight histogram as a reference area;

[0197] A new area consisting of the base area and all the square areas adjacent to the base area is used as a candidate area;

[0198] Calculate the area of ​​the candidate region according to the areas corresponding to all square regions in the candidate region;

[0199] Determine whether the area of ​​the candidate area is greater than or equal to the area of ​​the preset area;

[0200] When it is determined that the area of ​​the candidate region is smaller than the area of ​​the preset region, the candidate region is determined as a new reference region, and an operation of using the reference region and all square regions adjacent to the reference region as a new region is triggered;

[0201] When it is determined that the area of ​​the candidate region is greater than or equal to the area of ​​the preset region, the candidate region is determined as the target region that meets the preset area condition.

[0202] It can be seen that this optional implementation method can also filter out the square area with the maximum height value from all the square areas of the weight histogram according to the weight histogram of each calculation layer as the reference area, and use the reference area and a new area composed of all the square areas adjacent to the reference area as the candidate area, and calculate the area of ​​the candidate area according to the areas corresponding to all the square areas in the candidate area, and then judge whether the area of ​​the candidate area is greater than or equal to the preset area. When it is judged that the area of ​​the candidate area is less than the preset area, the candidate area is determined as the new reference area, and the execution of the operation of using the reference area and a new area composed of all the square areas adjacent to the reference area as the candidate area is triggered. When When it is determined that the area of ​​the candidate area is greater than or equal to the area of ​​the preset area, the candidate area is determined as the target area that meets the preset area condition, and the weighted histogram of each calculation layer can be used to accurately select the reference area with the maximum height value from the square area of ​​the weighted histogram, which is beneficial to improve the accuracy and efficiency of screening the reference area based on the weighted histogram, and can accurately determine the candidate area as the target area that meets the preset area condition when the area is greater than or equal to the preset area through the area area of ​​the candidate area composed of the reference area and its adjacent square areas and combined with the preset area, and gradually expand the search range when the area is less than the preset area, so as to accurately determine the target area that meets the preset area condition.

[0203] In this optional embodiment, as another optional implementation, the segmentation module 303 segments the weight of each target computing layer according to the range to which all weights of the target computing layer belong, and the method of obtaining the weight segmentation result of the target computing layer may specifically include:

[0204] For each target computing layer, according to the region length in the target region of the target computing layer determined based on the weight histogram of the target computing layer, determine the maximum value and the minimum value of the endpoint of the region length in the abscissa of the weight histogram;

[0205] For each weight in each target calculation layer, determine whether the weight is greater than the minimum endpoint value and less than the maximum endpoint value;

[0206] When it is determined that the weight is greater than the minimum endpoint value and less than the maximum endpoint value, the weight is classified into the first weight group;

[0207] When it is determined that the weight is less than or equal to the minimum endpoint value, or greater than or equal to the maximum endpoint value, the weight is classified into the second weight group.

[0208] It can be seen that this optional implementation method can determine the endpoint maximum and endpoint minimum of the area length in the horizontal axis of the weight histogram for each target calculation layer according to the area length in the target area of ​​the target calculation layer determined based on the weight histogram of the target calculation layer, and for each weight in each target calculation layer, judge whether the weight is greater than the endpoint minimum and less than the endpoint maximum. When it is judged that the weight is greater than the endpoint minimum and less than the endpoint maximum, the weight is classified into the first weight group; when it is judged that the weight is less than or equal to the endpoint minimum, or greater than or equal to the endpoint maximum, the weight is classified into the second weight group. The endpoint maximum and endpoint minimum can be accurately determined through the area length of the target area of ​​each target calculation layer, and weights of different sizes can be accurately classified into corresponding weight groups through the size relationship between the weight and the endpoint maximum and endpoint minimum, which is beneficial to improving the accuracy and reliability of weight segmentation.

[0209] In another optional embodiment, for each target computing layer, the quantization module 304 performs piecewise quantization on the first weight group and the second weight group of the target computing layer, and the method of obtaining the quantization bit width of the target computing layer may specifically include:

[0210] Performing quantization calculation on all weights in the first weight group of the target calculation layer to obtain a first weight quantization result of the target calculation layer;

[0211] Performing quantization calculation on all weights in the second weight group of the target calculation layer to obtain a second weight quantization result of the target calculation layer;

[0212] The first weighted quantization result and the second weighted quantization result of the target calculation layer are accumulated to obtain the quantization accumulated result of the target calculation layer, and a preset bias value is added to the quantization accumulated result of the target calculation layer to obtain the quantization bit width of the target calculation layer.

[0213] It can be seen that this optional embodiment can perform quantization calculation on all weights in the first weight group of each target calculation layer to obtain the first weight quantization result of the target calculation layer, and perform quantization calculation on all weights in the second weight group of the target calculation layer to obtain the second weight quantization result of the target calculation layer, and then accumulate the first weight quantization result and the second weight quantization result of the target calculation layer to obtain the quantization accumulation result of the target calculation layer, and then add the preset bias value to the quantization accumulation result of the target calculation layer to obtain the quantization bit width of the target calculation layer, and can accurately calculate the quantization bit width of the target calculation layer by performing segmented quantization on the first weight group and the second weight group, which is beneficial to improving the accuracy and reliability of the calculated quantization bit width, and compared with the mixed precision quantization method in the prior art, this scheme can reduce the mixed quantization bit width requirements for hardware devices through the unified quantization bit width used through segmented quantization, which is beneficial to saving costs and improving computing efficiency.

[0214] Embodiment 4

[0215] See also Figure 6 , Figure 6 FIG. 1 is a schematic diagram of a weight segmentation quantization device based on a convolutional neural network disclosed in an embodiment of the present invention. Figure 6 As shown, the weight segmentation quantization device based on convolutional neural network may include:

[0216] A memory 401 storing executable program codes;

[0217] a processor 402 coupled to the memory 401;

[0218] The processor 402 calls the executable program code stored in the memory 401 to execute the steps in the weight segmentation quantization method based on the convolutional neural network described in the first embodiment of the present invention or the second embodiment of the present invention.

[0219] Embodiment 5

[0220] An embodiment of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute the steps in the weight segmentation quantization method based on a convolutional neural network described in Embodiment 1 or Embodiment 2 of the present invention.

[0221] Embodiment 6

[0222] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps in the weight segmentation quantization method based on a convolutional neural network described in Example 1 or Example 2.

[0223] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0224] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0225] Finally, it should be noted that the weight segmentation quantization method and device based on convolutional neural network disclosed in the embodiment of the present invention only discloses the preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A weight segmentation quantization method based on convolutional neural network, characterized in that: The method comprises: Obtain all weights of each of the plurality of computing layers of the convolutional neural network, wherein the computing layer is a parameterized layer of the convolutional neural network; According to the weights of all the computing layers, a target computing layer to be segmented and quantized is selected from all the computing layers; For each of the target computing layers, segmenting the weights of the target computing layer according to the ranges to which all weights of the target computing layer belong, to obtain a weight segmentation result of the target computing layer, wherein the weight segmentation result includes a first weight group and a second weight group; The first weight group and the second weight group of the target computing layer are piecewise quantized to obtain a quantization bit width of the target computing layer.

2. The weight segmentation quantization method based on convolutional neural network according to claim 1, characterized in that: The step of selecting a target computing layer to be segmented and quantized from all the computing layers according to the weights of all the computing layers comprises: According to all the weights of each of the computing layers, a weight histogram of each of the computing layers is constructed, wherein all the square areas contained in the weight histogram are used to represent the weight distribution of the corresponding computing layer, and each of the square areas is an area composed of a weight as a height and a group distance mapped to the weight in the weight histogram as a length; For each of the calculation layers, judging whether the weight distribution of the calculation layer satisfies a preset tailing distribution condition according to the weight histogram of the calculation layer; When it is determined that the weight distribution of the calculation layer satisfies the tailing distribution condition, the calculation layer is determined as a target calculation layer to be segmented and quantized; When it is determined that the weight distribution of the computing layer does not satisfy the tailing distribution condition, a loss calculation is performed on the computing layer according to all weights of the computing layer to obtain a quantization information loss of the computing layer, and it is determined whether the quantization information loss of the computing layer is greater than a preset loss value; When it is determined that the quantization information loss of the computing layer is greater than the preset loss value, the computing layer is determined as a target computing layer to be segmented quantized; When it is determined that the weight distribution of the computing layer does not satisfy the tailing distribution condition and the quantization information loss of the computing layer is less than or equal to the preset loss value, the operation of determining whether the weight distribution of the computing layer satisfies the preset tailing distribution condition is re-executed for each computing layer according to the weight histogram of the computing layer until all the computing layers are screened.

3. The weight segmentation quantization method based on convolutional neural network according to claim 2, characterized in that: For each of the computing layers, performing loss calculation on the computing layer according to all weights of the computing layer to obtain the quantized information loss of the computing layer includes: According to all weights of the calculation layer, floating point calculations are performed on all weights of the calculation layer to obtain floating point results of the calculation layer; According to all weights of the calculation layer, all weights of the calculation layer are quantized to obtain a quantization result of the calculation layer; According to the floating-point result and the quantization result of the calculation layer, the loss of the calculation layer is calculated to obtain the quantization information loss of the calculation layer.

4. The weight piecewise quantization method based on convolutional neural network according to claim 2 or 3, characterized in that: For each of the computing layers, judging whether the weight distribution of the computing layer satisfies a preset tailing distribution condition according to the weight histogram of the computing layer includes: According to the weighted histogram of the calculation layer, a target area satisfying a preset area condition is determined from the weighted histogram, wherein the number of square areas in the target area is less than the number of square areas in the weighted histogram; Calculating the sum of the length values ​​of all square areas in the target area to obtain the area length of the target area, and determining whether the area length of the target area is less than a preset length value; When it is determined that the area length of the target area is less than the preset length value, determining that the weight distribution of the calculation layer satisfies a preset tailing distribution condition; When it is determined that the region length of the target region is greater than or equal to the preset length value, it is determined that the weight distribution of the calculation layer does not satisfy the preset tailing distribution condition.

5. The weight segmentation quantization method based on convolutional neural network according to claim 4, characterized in that: For each of the calculation layers, determining a target area that meets a preset area condition from the weighted histogram according to the weighted histogram of the calculation layer includes: According to the weighted histogram of the calculation layer, a square area whose height value is the maximum height value is selected from all the square areas of the weighted histogram as a reference area; A new area consisting of the reference area and all square areas adjacent to the reference area is used as a candidate area; Calculate the area of ​​the candidate region according to the areas corresponding to all square regions in the candidate region; Determine whether the area of ​​the candidate area is greater than or equal to the area of ​​the preset area; When it is determined that the area of ​​the candidate area is smaller than the area of ​​the preset area, the candidate area is determined as a new reference area, and the operation of using the reference area and all the square areas adjacent to the reference area as a new area of ​​the candidate area is triggered; When it is determined that the area of ​​the candidate region is greater than or equal to the area of ​​the preset region, the candidate region is determined as a target region that meets the preset area condition.

6. The weight piecewise quantization method based on a convolutional neural network according to any one of claims 2, 3 and 5, characterized in that: For each of the target computing layers, segmenting the weights of the target computing layer according to the range to which all weights of the target computing layer belong, to obtain the weight segmentation result of the target computing layer, including: For each of the target computing layers, according to the region length in the target region of the target computing layer determined based on the weight histogram of the target computing layer, determining the maximum endpoint value and the minimum endpoint value of the region length in the abscissa of the weight histogram; For each of the weights in each of the target computing layers, determining whether the weight is greater than the endpoint minimum value and less than the endpoint maximum value; When it is determined that the weight is greater than the minimum endpoint value and less than the maximum endpoint value, the weight is classified into the first weight group; When it is determined that the weight is less than or equal to the minimum value of the endpoint, or greater than or equal to the maximum value of the endpoint, the weight is classified into the second weight group.

7. The weight piecewise quantization method based on a convolutional neural network according to any one of claims 1, 2, 3 and 5, characterized in that: For each of the target computing layers, the step of performing piecewise quantization on the first weight group and the second weight group of the target computing layer to obtain the quantization bit width of the target computing layer includes: Performing quantization calculation on all weights in the first weight group of the target calculation layer to obtain a first weight quantization result of the target calculation layer; Performing quantization calculation on all weights in the second weight group of the target calculation layer to obtain a second weight quantization result of the target calculation layer; The first weighted quantization result and the second weighted quantization result of the target calculation layer are accumulated to obtain the quantization accumulated result of the target calculation layer, and a preset bias value is added to the quantization accumulated result of the target calculation layer to obtain the quantization bit width of the target calculation layer.

8. A weight segmentation quantization device based on convolutional neural network, characterized in that: The device comprises: An acquisition module, used to acquire all weights of each of the multiple computing layers of the convolutional neural network, wherein the computing layer is a parameterized layer of the convolutional neural network; A screening module, used for screening out a target computing layer to be segmented and quantized from all the computing layers according to the weights of all the computing layers; A segmentation module is used for segmenting the weight of each target computing layer according to the range to which all weights of the target computing layer belong, to obtain a weight segmentation result of the target computing layer, wherein the weight segmentation result includes a first weight group and a second weight group; The quantization module is used to perform piecewise quantization on the first weight group and the second weight group of the target computing layer to obtain the quantization bit width of the target computing layer.

9. A weight segmentation quantization device based on convolutional neural network, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the weight segmentation quantization method based on a convolutional neural network as described in any one of claims 1-7.

10. A computer storage medium, characterized in that: The computer storage medium stores computer instructions, which, when called, are used to execute the weight piecewise quantization method based on a convolutional neural network as described in any one of claims 1 to 7.

Images