A method and device for implementing deep convolutional neural network processing

Abstract

The present application discloses a method and device for implementing deep convolutional neural network processing, including: dividing the quantized parameters into blocks, dividing each convolution kernel of each convolution layer of the deep convolutional neural network into multiple sub-convolution kernels; counting the number of categories of all parameters in each sub-convolution kernel. The present application provides a guarantee for better acceleration of deep convolutional neural networks. Furthermore, the present application also performs clustering processing for each sub-convolution kernel according to the number of parameters in each category. In this way, the acceleration of deep convolutional neural networks is better achieved.

Description

Technical Field

[0001] The present application relates to but is not limited to the field of computer technology, and in particular to a method and device for implementing deep convolutional neural network processing. Background Art

[0002] Deep convolutional neural networks (CNNs) have achieved remarkable results in computer vision, speech recognition, natural language processing, and bioinformatics, significantly improving the performance of tasks such as image classification, target detection and tracking, semantic segmentation, face recognition, and video analysis. CNNs have also been widely used in industries such as autonomous driving, smart healthcare, security, and the Internet.

[0003] Most deep convolutional neural networks have millions of parameters and operations. For example, the AlexNet network has 60M parameters and 720M multiplication and addition operations; another example is the VGG16 network has 138M parameters and 15300M multiplication and addition operations. Therefore, to achieve real-time reasoning of deep convolutional neural networks, a large amount of storage and computing resources are required, and it is necessary to rely on high-performance computer clusters equipped with acceleration devices such as graphics processing units (GPUs).

[0004] However, for mobile devices such as mobile phones, wearable devices, smart homes, cars, robots, etc., their storage and computing resources are very limited due to factors such as size and power consumption. Especially in application scenarios with high real-time requirements such as autonomous driving, video surveillance, and mobile phone photography, common deep convolutional neural networks are difficult to play a role. Therefore, accelerating deep convolutional neural networks is crucial.

[0005] In order to solve the acceleration problem of deep convolutional neural networks, the algorithms proposed in related technologies usually use pruning and quantization methods to achieve acceleration, and the network needs to be retrained to ensure that the prediction accuracy loss is within the allowable range. However, the pruned and quantized deep convolutional neural network still has the potential for acceleration. The current algorithm does not consider the acceleration problem after pruning and quantization, so how to accelerate the pruned and quantized network remains to be studied. Summary of the invention

[0006] The present application provides a method and device for implementing deep convolutional neural network processing, which can provide a guarantee for better acceleration of deep convolutional neural network.

[0007] The present application provides a method for implementing deep convolutional neural network processing, including:

[0008] The quantized parameters are divided into blocks, and each convolution kernel of each convolution layer of the deep convolutional neural network is divided into multiple sub-convolution kernels;

[0009] Count the number of categories N of all parameters in each sub-convolution kernel.

[0010] In an exemplary embodiment, the method further comprises:

[0011] The deep convolutional neural network is quantized to obtain quantized parameters.

[0012] In an exemplary embodiment, quantizing a deep convolutional neural network includes:

[0013] Uniform quantization or non-uniform quantization is used to quantize the parameters of the deep convolutional neural network, and the parameters of the deep convolutional neural network are converted from floating point type to integer type.

[0014] In an exemplary embodiment, the quantizing the parameters of the deep convolutional neural network by uniform quantization includes:

[0015] For each convolutional layer, count the maximum absolute value of the convolutional layer input X max , and the maximum value Y of the absolute value of output Y max ;

[0016] The maximum absolute value of the statistical weight W max and the maximum absolute value of the bias b max ;

[0017] Calculate input quantization coefficients Output quantization coefficients Weight quantization coefficient Biased quantization coefficient k b =k X +k W ;

[0018] The integer value of floating-point input X after quantization is: q =round(X*k X ), the quantized result of weight W is W q =round(W*k W ), the quantization result of bias b is b q =round(b*k b );

[0019] Use the quantized input, quantized weights, and quantized biases for forward calculations, and multiply the result by the quantization coefficient The final product is the output result of the convolutional layer.

[0020] In an exemplary embodiment, the method further comprises:

[0021] Reduce the input quantization coefficient k X and the weight quantization coefficient k W The value of .

[0022] In an exemplary embodiment, obtaining the quantized parameters includes:

[0023] The parameters of the deep convolutional neural network are obtained using a deep learning framework tool, or through a script or manually.

[0024] In an exemplary embodiment, dividing the quantized parameters into blocks includes:

[0025] For each convolution kernel of each convolution layer of the deep convolutional neural network, the height and width of the convolution kernel are kept unchanged, and the convolution kernel is divided into multiple sub-convolution kernels in the channel dimension, wherein the number of channels of each sub-convolution kernel is greater than zero and less than or equal to the number of channels of the convolution kernel.

[0026] In an exemplary embodiment, counting the number of categories N of all parameters in each sub-convolution kernel includes:

[0027] For each subconvolution kernel, delete the parameters with a value of 0; take the absolute value of the negative parameters to get a positive value;

[0028] For each parameter in each sub-convolution kernel, divide the parameter by 2 until the remainder d is not equal to 0, where d is the category of the parameter. According to the number of d, the number N of categories of all parameters in each sub-convolution kernel is counted.

[0029] In an exemplary embodiment, for each subconvolution kernel, if the number of categories N of all parameters is greater than the number of categories K after clustering, the method further includes:

[0030] According to the number of parameters in each category, clustering is performed on the (NK) class parameters containing a small number of parameters, so that the sub-convolution kernel includes K class parameters.

[0031] In an exemplary embodiment, clustering the (NK) class parameters containing a small number of parameters includes:

[0032] For each subconvolution kernel, count the number of parameters of each category {l1,l2...,l N}, and sort the N categories by size;

[0033] The parameters of the first K categories are kept unchanged, and the parameters of the reciprocal (NK) category are adjusted to ensure that the number of parameters that need to be modified is minimized.

[0034] In an exemplary embodiment, the adjusting the parameters of the reciprocal (NK) class includes:

[0035] For the parameter w of the reciprocal (NK) class after the sorting, calculate the parameter w′ with the smallest L1Norm distance from the parameter w among the first K class parameters, and replace the parameter w with the parameter w′.

[0036] In an exemplary embodiment, the calculating of the parameter w′ having the smallest L1Norm distance with the parameter w among the K types of parameters before, and replacing the parameter w with the parameter w′, comprises:

[0037] Calculate all parameters w′ of the same type within the quantization bit width range of the first K types of parameters;

[0038] For the parameter w of the reciprocal (NK) class, calculate w′ that has the smallest L1Norm, and replace the parameter w with the calculated w′;

[0039] The above process is repeated until all parameters of the reciprocal (NK) class are classified into the top K classes.

[0040] In an exemplary embodiment, the method further comprises:

[0041] Perform forward calculation on the clustered deep convolutional neural network.

[0042] In an exemplary embodiment, the forward calculation of the clustered deep convolutional neural network includes:

[0043] For each sub-convolution kernel, the convolution results are calculated according to different categories, and the convolution results calculated by different categories are summed to obtain the convolution result of the sub-convolution kernel;

[0044] For each convolution kernel, the convolution results of all sub-convolution kernels in the same convolution kernel are summed, and the sum plus the bias is used to obtain the convolution result of the convolution kernel.

[0045] For each convolution layer, the convolution results of all convolution kernels in the convolution layer are divided by the quantization coefficient output by the convolution layer as the output result of the convolution layer.

[0046] In an exemplary embodiment, for each sub-convolution kernel, respectively calculating the convolution result according to different categories includes:

[0047] For a certain category, Calculate, where q represents the number of parameters of this category, m represents the maximum odd factor of this category, and k i Indicates the positive and negative sign of the parameter, n i Indicates the number of shifts of the parameter, p iIndicates the index of the input x in the feature map corresponding to the parameter.

[0048] The present application also provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute any of the above methods for implementing deep convolutional neural network processing.

[0049] The present application also provides a device for implementing deep convolutional neural network processing, including a processor and a memory; wherein the memory stores a computer program that can be run on the processor: used to execute the steps of any of the above-mentioned methods for implementing deep convolutional neural network processing.

[0050] The present application includes: dividing the quantized parameters into blocks, dividing each convolution kernel of each convolution layer of the deep convolutional neural network into multiple sub-convolution kernels; and counting the number of categories N of all parameters in each sub-convolution kernel. The present application provides a guarantee for better acceleration of the deep convolutional neural network.

[0051] In an exemplary embodiment, the present application further includes: for each sub-convolution kernel, according to the number of parameters in each category, clustering the (NK) class parameters containing a small number of parameters, so that the sub-convolution kernel includes K class parameters, where K is the number of clustered categories. In this way, the acceleration of the deep convolutional neural network is better achieved.

[0052] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are used to provide further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0054] Figure 1 A flowchart of a method for implementing deep convolutional neural network processing for this application;

[0055] Figure 2 This is a schematic diagram of the convolution kernel block of this application;

[0056] Figure 3 A schematic diagram of the composition structure of the device for implementing deep convolutional neural network processing in this application. DETAILED DESCRIPTION

[0057] In a typical configuration of the present application, a computing device includes one or more processors (CPU), an input / output interface, a network interface, and a memory.

[0058] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0059] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0060] In order to make the purpose, technical solution and advantages of the present application more clear, the embodiments of the present application will be described in detail below with reference to the accompanying drawings. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other arbitrarily without conflict.

[0061] Figure 1 A flowchart of a method for implementing deep convolutional neural network processing for this application, such as Figure 1 As shown, including at least:

[0062] Step 100: Divide the quantized parameters into blocks, and divide each convolution kernel of each convolution layer of the deep convolutional neural network into multiple sub-convolution kernels.

[0063] In an exemplary embodiment, the size of the sub-convolution kernel can be selected to a suitable value according to software resources and / or hardware resources. If the software resources and / or hardware resources are limited, a smaller sub-convolution kernel can be selected, otherwise a larger sub-convolution kernel can be selected.

[0064] Assume that the dimension of the convolution kernel is (C, H, W), where C represents the number of channels of the convolution kernel, H represents the height of the convolution kernel, and W represents the width of the convolution kernel.

[0065] In an exemplary embodiment, for each convolution kernel of each convolution layer of the deep convolutional neural network, the quantized parameters are divided into blocks, including:

[0066] Figure 2 This is a schematic diagram of the convolution kernel block for this application, such as Figure 2 As shown in the figure, the height and width of the convolution kernel are kept unchanged, and the convolution kernel is divided into multiple sub-convolution kernels in the channel dimension. The number of channels of each sub-convolution kernel is greater than zero and less than or equal to the number of channels of the convolution kernel. In other words, the dimension of each sub-convolution kernel is (C′, H, W); where 0<C′≤C.

[0067] In an exemplary embodiment, if C is not divisible by C′, then the channel dimension of the last block is padded with zeros, resulting in Blocks.

[0068] In an exemplary embodiment, before step 100, the following steps are further included:

[0069] Quantize a deep convolutional neural network; obtain quantized parameters.

[0070] In an exemplary embodiment, the parameters of the deep convolutional neural network can be quantized by using methods such as uniform quantization or non-uniform quantization, and the parameters of the deep convolutional neural network can be converted from floating point types to integer types.

[0071] In an exemplary embodiment, taking an 8-bit uniform quantization method as an example, the process of quantizing a convolutional layer includes:

[0072] Count the maximum absolute value X of the current convolutional layer input X max , and the maximum value Y of the absolute value of output Y max ;

[0073] The maximum absolute value of the statistical weight W max and the maximum absolute value of the bias b max ;

[0074] Calculate input quantization coefficients Output quantization coefficients Weight quantization coefficient Biased quantization coefficient k b =k X +k W ;

[0075] The integer value of floating-point input X after quantization is: q =round(X*k X ), the quantized result of weight W is W q =round(W*k W), the quantization result of bias b is b q =round(b*k b );

[0076] Use the quantized input, quantized weights, and quantized biases for forward calculations, and multiply the result by the quantization coefficient The final product is the output result of the current convolutional layer.

[0077] In this application, the quantization coefficient 2 can be adjusted q Right now To ensure the quantization bit width requirements.

[0078] Input quantization coefficient k X and weight quantization coefficient k W The larger the quantized convolution layer input X and weight W are, the larger the convolution result is, and the more computing and storage resources are required. In an exemplary embodiment, in order to save resources, the application also includes:

[0079] Reduce the input quantization coefficient k X and the weight quantization coefficient k W , in order to limit the value of the intermediate result.

[0080] In an exemplary embodiment, the input quantization coefficient k can be reduced by subtracting a preset step size (such as 1). X and the weight quantization coefficient k W If the requirement is still not met, you can continue to reduce it.

[0081] However, if it is reduced too much, the network accuracy will be reduced. Therefore, in actual processing, this application needs to strike a balance between saving resources and ensuring accuracy to ensure network accuracy.

[0082] In an exemplary embodiment, the parameters of the deep convolutional neural network can be obtained by using deep learning framework tools such as PyTorch, TensorFlow, etc., or by scripting or manually. For any convolution layer in the network, its dimension is (N, C, H, W), where N is the number of convolution kernels, C is the number of channels of the convolution kernel, H is the height of the convolution kernel, and W is the width of the convolution kernel. The dimension of each convolution kernel is (C, H, W).

[0083] Step 101: Count the number of categories N of all parameters in each sub-convolution kernel.

[0084] Any integer parameter can be expressed as m*2 n In the form of, for example -16 = (-1) * 2 4 , 31=31*2 0, where m represents the largest odd factor. In the embodiment of the present application, this principle is used to classify the parameters with the same absolute value of m in each sub-convolution kernel into one category, for example, 1, 16, -32, 64, and -128 belong to the same category with an absolute value of m of 1.

[0085] In an exemplary embodiment, this step may include:

[0086] For each subconvolution kernel, preprocess the parameters: delete the parameters with a value of 0; take the absolute value of the negative parameters to get a positive value;

[0087] Each parameter in each subconvolution kernel is processed as follows: divide the parameter by 2 until the remainder d is not equal to 0, then d is the category of the parameter. In this way, according to the number of d, the categories of all parameters in each subconvolution kernel {d1, d2, ..., d N}, a total of N categories.

[0088] In the embodiment of the present application, the multiplication operation is converted into a shift operation by extracting the largest odd factor m of the same type of parameters. For example, if the network parameters are w1=3, w2=-12, w3=96, and the inputs are x1, x2, and x3, then w1*x1+w2*x2+w3*x3=3*x1-12*x2+96*x3=3*(2 0 *x1-2 2 *x2+2 5 *x3). It can be seen that the multiplication operation in the brackets can be converted into a shift operation, and the multiplication operation is reduced from 3 times to 1 time, that is, only one multiplication operation is required for the parameters of the same category, thus greatly reducing the number of multiplication operations, thereby achieving the acceleration of deep convolutional neural networks.

[0089] In an exemplary embodiment, further, step 101 of the present application further includes:

[0090] The more parameter categories there are, the more computing resources are required. In order to make better use of limited resources, the present application also includes: for each sub-convolution kernel, if the number of categories N of all parameters is greater than the required number of categories K after clustering, the parameters of the sub-convolution kernel need to be clustered. That is to say, for each sub-convolution kernel, assuming that the categories of all parameters are required to be less than or equal to K categories, if N≤K, then the current block meets the classification conditions and the process ends; if N>K, continue to step 102. Among them, K is pre-set to the number of categories after clustering.

[0091] Step 102: For each sub-convolution kernel, cluster the (NK) class parameters containing a small number of parameters according to the number of parameters in each category, so that the sub-convolution kernel includes K class parameters, where K is the number of categories after clustering.

[0092] In an exemplary embodiment, assuming that C′*H*W parameters in each sub-convolution kernel need to be clustered into K categories, this step may include:

[0093] For each subconvolution kernel, count the number of parameters of each category {l1,l2...,l N}, and sort the N categories by size;

[0094] Keep the first K class parameters unchanged and only adjust the parameters of the last (NK) class to ensure that the number of parameters that need to be modified is minimized.

[0095] In an exemplary embodiment, adjusting the parameters of the reciprocal (NK) class may include:

[0096] For the parameter w of the reciprocal (NK) class after sorting, calculate the parameter w′ with the smallest L1Norm distance from the parameter w in the first K classes, and replace the original parameter w with the parameter w′ to ensure that the difference before and after the parameter modification is minimized. Among them, L1Norm refers to the sum of the absolute values ​​of the differences between the corresponding elements of the n-dimensional vectors p and q, which can be understood as the distance between p and q. Here, it means finding a parameter w′ in the first K classes, the absolute value of the difference between the parameter w′ and the parameter w is the smallest, and both w′ and w are scalars.

[0097] In an exemplary embodiment, calculating the parameter w′ with the smallest L1Norm among the first K types of parameters and replacing the original parameter w with the parameter w′ may include:

[0098] Calculate all parameters w′ of the same type in the quantization bit width range (e.g., 8-bit) of the first K parameters. For example, all parameters of category 3 in the 8-bit quantization bit width range include: (-96, -48, -24, -12, -6, -3, 3, 6, 12, 24, 48, 96).

[0099] For the parameter w of the reciprocal (NK) class, calculate w′ with the smallest L1Norm, that is, Replace the original w with the calculated w′, so that the parameter w is classified into the first K categories of parameters;

[0100] The above process is repeated until all parameters of the reciprocal (NK) class are classified into the first K classes, and the clustering process of this step is completed.

[0101] The clustering process in the embodiment of the present application ensures that the impact on prediction accuracy is as low as possible by limiting the number of parameters that need to be modified and the difference between the parameters before and after modification.

[0102] This application provides a guarantee for better acceleration of deep convolutional neural networks through the above-mentioned processing of deep convolutional neural network parameters.

[0103] In an exemplary embodiment, the method of the present application may further include:

[0104] Perform forward calculation on the clustered deep convolutional neural network.

[0105] In an exemplary embodiment, forward calculation is performed on the clustered deep convolutional neural network, including:

[0106] For each sub-convolution kernel, the convolution results are calculated according to different categories, and the convolution results calculated by different categories are summed to obtain the convolution result of the sub-convolution kernel;

[0107] For each convolution kernel, the convolution results of all sub-convolution kernels in the same convolution kernel are summed, and the sum plus the bias is used to obtain the convolution result of the convolution kernel.

[0108] For each convolution layer, the convolution results of all convolution kernels in the convolution layer are divided by the quantization coefficient output by the convolution layer to obtain the output result of the convolution layer.

[0109] In an exemplary embodiment, for each sub-convolution kernel, calculating the convolution results according to different categories may include:

[0110] For a certain category, Calculate, where q represents the number of parameters of this category, m represents the maximum odd factor of this category, and k i Indicates the positive and negative sign of the parameter, n i Indicates the number of shifts of the parameter, p i Indicates the index of the input x in the feature map corresponding to the parameter.

[0111] This application achieves better acceleration of the deep convolutional neural network through the above processing of the deep convolutional neural network parameters.

[0112] It should be noted that the parameters in the embodiments of the present application refer to the weights of the convolutional layer. The convolutional layer generally includes two parts, weight and bias, both of which are quantized. However, in the present application, only the weights are processed by block division, classification, clustering, etc.

[0113] An embodiment of the present invention also provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute any of the above methods for implementing deep convolutional neural network processing.

[0114] An embodiment of the present invention also provides a device for implementing deep convolutional neural network processing, including a processor and a memory; wherein the memory stores a computer program that can be run on the processor: used to execute the steps of any of the above-mentioned methods for implementing deep convolutional neural network processing.

[0115] Figure 3 A schematic diagram of the structure of the device for implementing deep convolutional neural network processing in this application is shown in FIG. Figure 3 As shown, it at least includes: a block module and a classification module; wherein,

[0116] The block module is used to block the quantized parameters and divide each convolution kernel of each convolution layer of the deep convolutional neural network into multiple sub-convolution kernels;

[0117] The classification module is used to count the number of categories N of all parameters in each sub-convolution kernel.

[0118] In an exemplary embodiment, the classification module is specifically used to:

[0119] For each subconvolution kernel, preprocess the parameters: delete the parameters with a value of 0; take the absolute value of all negative parameters to get positive numbers;

[0120] For each parameter in each subconvolution kernel, the following processing is performed: divide the parameter by 2 until the remainder d is not equal to 0, then d is the category of the parameter. In this way, the categories of all parameters in each subconvolution kernel {d1, d2, ..., d N}, a total of N categories.

[0121] In an exemplary embodiment, the apparatus for implementing deep convolutional neural network processing in the present application may further include: a quantization module and an acquisition module; wherein:

[0122] Quantization module, used to quantize deep convolutional neural networks;

[0123] The acquisition module is used to obtain the quantized parameters.

[0124] In an exemplary embodiment, the classification module is further used for: for each sub-convolution kernel, if the number of categories N of all parameters is greater than the required number of categories K after clustering, clustering processing needs to be performed on the parameters of the sub-convolution kernel; accordingly,

[0125] The apparatus for implementing deep convolutional neural network processing in the present application may also include a clustering module for:

[0126] For each sub-convolution kernel, clustering is performed on the (NK) class parameters containing a small number of parameters according to the number of parameters in each category, so that the sub-convolution kernel includes K class parameters, where K is the number of categories after aggregation.

[0127] In an exemplary embodiment, the clustering module is specifically used to:

[0128] For each subconvolution kernel, count the number of parameters of each category {l1,l2...,lN}, and sort the N categories by size;

[0129] Keep the first K class parameters unchanged and only adjust the parameters of the last (NK) class to ensure that the number of parameters that need to be modified is minimized.

[0130] In an exemplary embodiment, the parameters of the reciprocal (NK) class in the clustering module are adjusted, including:

[0131] For the parameter w of the reciprocal (NK) class after sorting, calculate the parameter w′ with the smallest L1Norm distance among the first K class parameters, and replace the original parameter w with the parameter w′ to ensure that the difference before and after the parameter modification is minimized.

[0132] In an exemplary embodiment, the apparatus for implementing deep convolutional neural network processing in the present application further includes: a computing module to complete forward computing of the clustered deep convolutional neural network. The computing module can be implemented using a programmable logic device, or can be implemented as a computer program software or a program module.

[0133] In an exemplary embodiment, the computing module is specifically configured to:

[0134] For each sub-convolution kernel, the convolution results are calculated according to different categories, and the convolution results calculated by different categories are summed to obtain the convolution result of the sub-convolution kernel;

[0135] For each convolution kernel, the convolution results of all sub-convolution kernels in the same convolution kernel are summed, and the sum plus the bias is used to obtain the convolution result of the convolution kernel.

[0136] For each convolution layer, the convolution results of all convolution kernels in the convolution layer are divided by the quantization coefficient output by the convolution layer to obtain the output result of the convolution layer.

[0137] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for implementing deep convolutional neural network processing, comprising: The quantized parameters are divided into blocks, and each convolution kernel of each convolution layer of the deep convolutional neural network is divided into multiple sub-convolution kernels; The size of the subconvolution kernel is determined according to hardware resources; Count the number of categories N of all parameters in each sub-convolution kernel; For each subconvolution kernel, if the number of categories N of all parameters is greater than the number of categories after clustering, the method further includes: According to the number of parameters in each category, cluster the (NK) class parameters containing a small number of parameters, so that the subconvolution kernel includes K class parameters; The method further comprises: For each convolutional layer, uniform quantization is used to quantize the parameters of the deep convolutional neural network to obtain a quantized input, a quantized weight, and a quantized bias; Perform forward calculation using the quantized input, the quantized weight, and the quantized bias, and multiply the calculation result by the quantization coefficient The product obtained after is the output result of the current convolutional layer; Among them, k X Represents the input quantization coefficient, k Y Represents the output quantization coefficient, k W represents the weight quantization coefficient; the quantized bias b q is the result of quantizing the bias b, expressed as b q =round(b*k b ), k b Represents the bias quantization coefficient k b =k X +k W .

2. The method according to claim 1, further comprising: quantizing the deep convolutional neural network; Get the quantized parameters.

3. The method according to claim 2, wherein: The quantizing of the deep convolutional neural network includes: Uniform quantization or non-uniform quantization is used to quantize the parameters of the deep convolutional neural network, and the parameters of the deep convolutional neural network are converted from floating point type to integer type.

4. The method according to claim 1, further comprising: Reduce the input quantization coefficient k X and the weight quantization coefficient k W The value of .

5. The method according to claim 2, wherein: The obtaining of the quantized parameters includes: The parameters of the deep convolutional neural network are obtained using a deep learning framework tool, or through a script or manually.

6. The method according to claim 1 or 2, wherein: The step of dividing the quantized parameters into blocks includes: For each convolution kernel of each convolution layer of the deep convolutional neural network, the height and width of the convolution kernel are kept unchanged, and the convolution kernel is divided into multiple sub-convolution kernels in the channel dimension, wherein the number of channels of each sub-convolution kernel is greater than zero and less than or equal to the number of channels of the convolution kernel.

7. The method according to claim 1 or 2, wherein: The number of categories N of all parameters in each sub-convolution kernel is counted, including: For each subconvolution kernel, delete the parameters with a value of 0; take the absolute value of the negative parameters to get a positive value; For each parameter in each sub-convolution kernel, divide the parameter by 2 until the remainder d is not equal to 0, where d is the category of the parameter. According to the number of d, the number N of categories of all parameters in each sub-convolution kernel is counted.

8. The method according to claim 7, wherein: The clustering process of the (NK) class parameters containing a small number of parameters includes: For each subconvolution kernel, count the number of parameters of each category {l1,l2...,l N }, and sort the N categories by size; The parameters of the first K categories are kept unchanged, and the parameters of the reciprocal (NK) category are adjusted to ensure that the number of parameters that need to be modified is minimized.

9. The method according to claim 8, wherein: The adjustment of the parameters of the reciprocal (NK) class includes: For the parameter w of the reciprocal (NK) class after the sorting, calculate the parameter w′ with the smallest L1 Norm distance from the parameter w among the first K class parameters, and replace the parameter w with the parameter w′.

10. The method according to claim 9, wherein: The calculating of the parameter w′ having the smallest L1 Norm distance with the parameter w among the K types of parameters before the parameter w, and replacing the parameter w with the parameter w′, comprises: Calculate all similar parameters w′ of the first K types of parameters within the quantization bit width range; For the parameter w of the reciprocal (NK) class, calculate w′ that has the smallest L1 Norm, and replace the parameter w with the calculated w′; The above process is repeated until all parameters of the reciprocal (NK) class are classified into the top K classes.

11. The method according to claim 7, further comprising: Perform forward calculation on the clustered deep convolutional neural network.

12. The method according to claim 11, wherein the forward calculation of the clustered deep convolutional neural network comprises: For each sub-convolution kernel, the convolution results are calculated according to different categories, and the convolution results calculated by different categories are summed to obtain the convolution result of the sub-convolution kernel; For each convolution kernel, the convolution results of all sub-convolution kernels in the same convolution kernel are summed, and the sum plus the bias is used to obtain the convolution result of the convolution kernel. For each convolution layer, the convolution results of all convolution kernels in the convolution layer are divided by the quantization coefficient output by the convolution layer as the output result of the convolution layer.

13. The method according to claim 12, wherein for each sub-convolution kernel, the convolution results are calculated according to different categories, comprising: For a certain category, Calculate, where q represents the number of parameters of this category, m represents the maximum odd factor of this category, and k i Indicates the positive and negative signs of the parameters, n i Indicates the number of shifts of the parameter, p i Indicates the index of the input x in the feature map corresponding to the parameter.

14. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the method for implementing deep convolutional neural network processing as described in any one of claims 1 to claim 13.

15. A device for implementing deep convolutional neural network processing, comprising a processor and a memory; wherein: The memory stores a computer program that can be run on the processor: used to execute the steps of the method for implementing deep convolutional neural network processing as described in any one of claims 1 to 13.

Images