Large-size sparse convolution module design method

Abstract

The invention discloses a large-size sparse convolution module design method, which comprises the following steps of: decomposing a convolution kernel to be compressed into a plurality of factor matrixes, performing Kronecker product operation and stacking on every two factor matrixes to obtain three sparse block matrixes, further reconstructing into a three-layer convolution structure, and applying a large-size convolution kernel to the structure to obtain a large-size sparse convolution module. Therefore, the compression and acceleration of the convolution kernel are completed while the model effect is guaranteed. The method can be widely applied to various models, the parameter quantity and the calculation quantity are effectively reduced, and the calculation efficiency of the model is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning of machine learning, and in particular relates to a method for designing a large-size sparse convolution module. Background Art

[0002] As one of the representative algorithms of deep learning, convolutional neural networks are good at handling various visual tasks such as image classification and target detection. However, the stronger the visual understanding ability of convolutional neural networks, the more parameters and computational complexity they have, which poses a challenge to deploying convolutional neural network models in resource-constrained hardware environments. To solve this problem, a large number of convolutional neural network compression methods have been proposed, including tensor decomposition methods with high compression ratios and direct trainability. However, existing tensor decomposition methods are difficult to balance time complexity and space complexity, and are not convenient for compressing typical convolutional neural network models that already have bottleneck structures. Therefore, a large-size sparse convolution module design method came into being, which can effectively reduce the number of parameters and computational complexity of the model while taking into account the model effect. Summary of the invention

[0003] The purpose of the present invention is to provide a large-size sparse convolution module design method, which decomposes the convolution kernel to be compressed into several factor matrices, and performs Kronecker product operations on the factor matrices in pairs, stacks them to obtain three sparse block matrices, and further transforms them into a three-layer convolution structure. The large convolution kernel is applied to the convolution structure, thereby achieving compression and acceleration of the convolution kernel while ensuring the model effect.

[0004] In order to solve the above-mentioned technical problems, the present invention provides a large-size sparse convolution module design method, which comprises the following steps:

[0005] Step 1: For the h×w convolution kernel P∈R S×C×h×w , which can be reconstructed into a 3rd-order tensor P∈R S×C×hw And decomposed into:

[0006]

[0007] In the formula, represents the Kronecker product, K is called the KT rank, A k and B k Perform CP decomposition:

[0008]

[0009]

[0010] In the formula, and is the CP factor matrix, and is the superdiagonal tensor, and A k and B k CP rank;

[0011] Step 2: Replace the A k With B k The CP factor matrices are Kronecker products of two by two and stacked into sparse block matrices F 3 ∈R hw×T , the specific form is as follows:

[0012]

[0013]

[0014]

[0015] Step 3: Based on the inverse transformation of the convolution expansion process, Convert to F 1 ∈R S×T×1×1 , we get a 1×1 convolution kernel with input channel T and output channel S. Similarly, Convert to F 2 ∈R T ×C×1×1 , we get a 1×1 convolution kernel with input channel C and output channel T, and transform F 3 ∈R hw×T Convert to F 3 ∈R T×h×w , we get a Dwise convolution kernel with input and output channels of T and size of h×w, and then implement the following convolution operations based on the existing convolution programming module:

[0016] Y=((X*F 2 )*F 3 )*F 1

[0017] Where Y represents the output result, X represents the input feature map, and * represents the convolution operation;

[0018] After completing step 3, if there are still convolution kernels that need to be compressed, jump to step 1 and repeat the above steps for the convolution kernels to be compressed until all convolution kernels are compressed, and then go to step 4;

[0019] Step 4: Use the error back propagation algorithm to train the compressed convolution module to adjust it to the best performance.

[0020] The beneficial effects of the present invention are as follows:

[0021] The present invention decomposes the convolution kernel to be compressed into multiple factor matrices, and obtains a three-layer convolution structure through the Kronecker product operation stacking transformation of the matrices, thereby completing the compression and acceleration of the convolution kernel. While taking into account the model effect, it reduces the number of model parameters and the amount of calculation, thereby improving the model operation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A flowchart of a large-size sparse convolution module design method of the present invention;

[0023] Figure 2 It is a schematic diagram of decomposing the convolution kernel to be compressed into several factor matrices according to the present invention;

[0024] Figure 3 It is a schematic diagram of the factor matrix reorganization of the present invention;

[0025] Figure 4 It is a schematic diagram of reconstructing the matrix of the present invention into a three-layer convolution structure. DETAILED DESCRIPTION

[0026] The present invention is further described in detail below with reference to the accompanying drawings and specific implementation examples, but the implementation methods of the present invention are not limited thereto.

[0027] like Figure 1 As shown, a large-size sparse convolution module design method includes the following steps:

[0028] Step 1: Figure 2 As shown, the convolution kernel P∈R S×C×h×w Reconstructed into a 3rd-order tensor P∈R S×C×hw And decomposed into:

[0029]

[0030] right Perform CP decomposition:

[0031]

[0032]

[0033] In the formula, the CP factor matrix and

[0034] Step 2: If Figure 3 As shown, replace step 1 in A k With B k The CP factor matrices are Kronecker products of two by two and stacked into sparse block matrices F 3 ∈R hw×T , the specific form is as follows:

[0035]

[0036]

[0037]

[0038] Step 3: If Figure 4 As shown, replace step 2 Convert to convolution kernel form F 1 ∈R S×T×1×1 ,Will Convert to convolution kernel form F 2 ∈R T×C×1×1 , F 3 ∈R hw×T Convert to a large-size Dwise convolution kernel F 3 ∈R T×h×w , and relying on the existing convolutional programming module, perform the following convolution operation on the input feature map X:

[0039] Y=((X*F 2 )*F 3 )*F 1

[0040] After completing step 3, if there are still convolution kernels that need to be compressed, jump to step 1 and repeat the above steps for the convolution kernels to be compressed until all convolution kernels are compressed, and then go to step 4;

[0041] Step 4: Use the error back propagation algorithm to train the compressed convolution module to achieve the optimal effect.

[0042] In order to better illustrate the beneficial effects of the present invention, the present invention is experimentally verified on the image classification dataset CIFAR-10 based on the VGG classification model. The experimental results show that the large-size sparse convolution module design method provided by the present invention can compress the number of parameters to 13.30% of the baseline model and the amount of calculation to 77.50% of the baseline model with a loss of 1.35% in accuracy, thereby effectively achieving model compression and acceleration.

[0043] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A design method for large-scale sparse convolution modules, It is characterized in that It includes the following steps: Step 1: For the h×w convolution kernel P∈R S×C×h×w , which can be reconstructed into a 3rd-order tensor P∈R S×C×hw And decomposed into: In the formula, represents the Kronecker product, K is called the KT rank, And S = S 1 S 2 , C=C 1 C 2 ; for A k and B k Perform CP decomposition: In the formula, and is the CP factor matrix, and is the superdiagonal tensor, and A k and B k CP rank; Step 2: Replace the A k With B k The CP factor matrices are Kronecker products of two by two and stacked into sparse block matrices F 3 ∈R hw×T , the specific form is as follows: In the formula, Step 3: Based on the inverse transformation of the convolution expansion process, Convert to F 1 ∈R S×T×1×1 , we get a 1×1 convolution kernel with input channel T and output channel S. Similarly, Convert to F 2 ∈R T ×C×1×1 , we get a 1×1 convolution kernel with input channel C and output channel T, and transform F 3 ∈R hw×T Convert to F 3 ∈R T×h×w , we get a large size h×w Dwise convolution kernel with input and output channels of T, and then implement the following convolution operations based on the existing convolution programming module: Y=((X*F 2 )*F 3 )*F 1 Among them, Y represents the output result, X represents the input feature map, and * represents the convolution operation; After completing step 3, if there are still convolution kernels that need to be compressed, jump to step 1 and repeat the above steps for the convolution kernels to be compressed until all convolution kernels are compressed, and then go to step 4; Step 4: Use the error back propagation algorithm to train the compressed convolution module to adjust it to the best performance.

Images