Methods, apparatus, devices and storage media for compression and porting of convolutional neural networks

Abstract

This application discloses a method, apparatus, device, and storage medium for compressing and porting convolutional neural networks. The method for compressing a convolutional neural network includes: obtaining an initial convolutional neural network to be compressed; adding a sparse convolutional layer after each convolutional layer to be compressed in the initial convolutional neural network; obtaining the weight parameters of each sparse convolutional layer; pruning and decomposing each convolutional layer to be compressed based on the weight parameters to obtain a sparsed convolutional neural network; training the sparsed convolutional neural network; and using the trained convolutional neural network as the compressed convolutional neural network. This approach can effectively reduce model parameters.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, and storage medium for compressing and porting convolutional neural networks. Background Technology

[0002] Over the past few years, convolutional neural networks have achieved state-of-the-art performance in a variety of computer vision tasks. However, the millions of parameters and heavy computational burden that are essential for new advances in this field are impractical for deploying neural network solutions on edge devices and mobile devices.

[0003] Therefore, a solution is needed that can compress neural network models, facilitating deployment and portability on mobile devices, to reduce the original model size without sacrificing accuracy. While both pruning-based and decomposition-based methods can compress models and maintain accuracy, each has its limitations. Pruning only affects the feature output channels, not preventing kernel parameters from participating in convolution. Furthermore, pruning prevents some key features from propagating downwards, impacting network accuracy. Kernel decomposition introduces another 1x1 convolutional layer, which increases GPU computation. Summary of the Invention

[0004] The main technical problem addressed in this application is to provide a method, apparatus, device, and storage medium for compressing and porting convolutional neural networks, which can effectively reduce model parameters.

[0005] To address the aforementioned problems, a first aspect of this application provides a method for compressing a convolutional neural network. The method includes: obtaining an initial convolutional neural network to be compressed; adding a sparse convolutional layer after each convolutional layer to be compressed in the initial convolutional neural network; obtaining weight parameters for each sparse convolutional layer; performing pruning and decomposition processing on each convolutional layer to be compressed based on the weight parameters to obtain a sparsed convolutional neural network; training the sparsed convolutional neural network; and using the trained convolutional neural network as the compressed convolutional neural network.

[0006] To address the aforementioned problems, a second aspect of this application provides a method for porting a convolutional neural network. The method includes: obtaining an initial convolutional neural network in a first device; compressing the initial convolutional neural network model using a convolutional neural network compression method to obtain a compressed convolutional neural network; and porting the compressed convolutional neural network to a second device; wherein the convolutional neural network compression method includes the convolutional neural network compression method described in the first aspect.

[0007] To address the aforementioned problems, a third aspect of this application provides a compression device for a convolutional neural network. The compression device comprises: an acquisition module, configured to acquire an initial convolutional neural network to be compressed, and add a sparse convolutional layer after each convolutional layer to be compressed in the initial convolutional neural network; a processing module, configured to acquire the weight parameters of each sparse convolutional layer, and perform pruning and decomposition processing on each convolutional layer to be compressed based on the weight parameters to obtain a sparsified convolutional neural network; and a training module, configured to train the sparsified convolutional neural network, and use the trained convolutional neural network as the compressed convolutional neural network.

[0008] To address the aforementioned problems, a fourth aspect of this application provides an electronic device, wherein the sound source location positioning electronic device includes a processor and a memory interconnected; the memory is used to store program instructions, and the processor is used to execute the program instructions to implement the convolutional neural network compression method of the first aspect above, or the convolutional neural network porting method of the second aspect above.

[0009] To address the aforementioned problems, a fifth aspect of this application provides a computer-readable storage medium storing program instructions thereon, which, when executed by a processor, implement the compression method of the convolutional neural network of the first aspect or the porting method of the convolutional neural network of the second aspect.

[0010] The beneficial effects of this invention are as follows: Unlike existing technologies, this application adds a sparse convolutional layer after each convolutional layer to be compressed in the convolutional neural network. Based on the weight parameters of each sparse convolutional layer, pruning and decomposition are performed on each convolutional layer to be compressed, resulting in a sparsified convolutional neural network. By fusing convolutional kernel pruning and convolutional kernel decomposition through sparse convolutional layers, pruning and decomposition are performed simultaneously, which can compress the model, reduce model parameters, and accelerate model computation. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating an embodiment of the compression method for convolutional neural networks in this application;

[0012] Figure 2 yes Figure 1 A flowchart illustrating an embodiment of step S12;

[0013] Figure 3 This is a schematic diagram illustrating the integrated convolution kernel pruning and decomposition in the convolutional neural network compression method of this application;

[0014] Figure 4 This is a schematic diagram of the workflow of a sparse group matrix in one application scenario of this application;

[0015] Figure 5 This is a schematic diagram of kernel sorting and selection in an application scenario of this application;

[0016] Figure 6 This is a flowchart illustrating an embodiment of the convolutional neural network porting method of this application;

[0017] Figure 7 This is a schematic diagram of the structure of an embodiment of the compression device for the convolutional neural network of this application;

[0018] Figure 8 This is a schematic diagram of the structure of an embodiment of the electronic device of this application;

[0019] Figure 9 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0020] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0021] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0022] In this paper, the terms "system" and "network" are often used interchangeably. The term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this paper means two or more.

[0023] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the convolutional neural network compression method of this application. The convolutional neural network compression method in this embodiment includes the following steps:

[0024] S11: Obtain the initial convolutional neural network to be compressed, and add a sparse convolutional layer after each convolutional layer to be compressed in the initial convolutional neural network.

[0025] In this application, for each convolutional layer to be compressed, the convolutional kernel in the convolutional layer needs to be pruned and decomposed. By adding a sparse convolutional layer after each convolutional layer to be compressed, the convolutional layer to be compressed is sparsified while ensuring output accuracy.

[0026] S12: Obtain the weight parameters of each sparse convolutional layer, and perform pruning and decomposition processing on each convolutional layer to be compressed based on the weight parameters to obtain a sparse convolutional neural network.

[0027] Pruning removes redundant parameters from a network, reducing the number of parameters and unnecessary computation. Specifically, pruning can target a single convolutional kernel, reducing the number of channels in each convolutional layer of the neural network. Decomposition breaks down an original matrix into multiple simpler, smaller matrices, using the parameters of these smaller matrices to replace the original matrix, thereby reducing storage space and computational load. For example, a one-dimensional image input x∈R m×1 Meanwhile, a convolution kernel is W = {w1, ..., w} n}∈R m×n Pruning-based methods can reduce the number of convolution kernels, using x T C is used as the output of the convolution to replace the original x. T W, where C∈R m×k Let A be the number of convolution kernels after pruning, and let k be the number of output channels. Based on the convolution kernel decomposition method, let convolution kernel A∈R... m×k and B∈R m×k This replaces the original convolution kernel W, while ensuring that the rank of the convolution kernel matrix A*B equals the rank of W. As can be seen from the above assumptions, the methods of convolution kernel pruning and decomposition are complementary. By combining these two methods, the shortcomings of a single method can be addressed, thereby improving the compression efficiency of convolutional neural networks.

[0028] Please combine Figure 2 , Figure 2 yes Figure 1 A flowchart illustrating one embodiment of step S12. In one embodiment, step S12 specifically includes:

[0029] S121: Based on the weight parameters of each sparse convolutional layer, obtain the group sparse matrix corresponding to each sparse convolutional layer.

[0030] S122: For a certain convolution kernel, multiply the convolution kernel with the row simplification matrix of the group sparse matrix to obtain a new convolution kernel, and convert a single heavyweight convolution kernel into a lightweight convolution kernel and a 1*1 convolution kernel through the new convolution kernel and the column simplification matrix of the group sparse matrix to obtain the sparsified convolutional neural network.

[0031] Please combine Figure 3 , Figure 3 This is a schematic diagram illustrating the integrated convolutional kernel pruning and decomposition in the convolutional neural network compression method of this application. For example... Figure 3In the convolution operation shown, the key to pruning and decomposition is finding a group sparse matrix. By leveraging the sparsity of the rows and columns of this group sparse matrix, the convolution kernel matrix can be pruned and decomposed simultaneously. Specifically, the convolution kernel W is compared with the row-simplified matrix A of the group sparse matrix A. c Multiplying them together yields a new convolution kernel W. c This process reduces the output dimension and parameters of the convolution kernel; this is a pruning operation. Simultaneously, if group sparsity constraints are applied to the rows of A, the internal channels of the matrix product W*A—that is, the output channels of W and the input channels of A—can be reduced. Furthermore, to save computation, the already simplified A... r Matrix and W r The matrix transforms a single heavyweight convolutional kernel W into a lightweight convolutional kernel and a 1x1 convolutional kernel; this process is called kernel decomposition. In the diagram, the convolutional kernel is a two-dimensional matrix W∈R. features×outputs The sparse matrix of the group is A n ×n By using a group sparse matrix to increase the sparsity of the convolutional kernel, and by applying group sparsity constraints to the columns of the group sparse matrix, the optimal solution of the group sparse matrix can be obtained during model training. Based on the optimal solution of the group sparse matrix, the sparsified convolutional neural network can be obtained.

[0032] Specifically, the sparse convolutional layer is a 1*1 convolutional layer. During the initial training cycle, the number of output channels of the sparse convolutional layer is the same as the number of output channels of its corresponding convolutional layer to be compressed. During training, by applying group sparsity constraints to the matrix of the sparse convolutional layer, the number of output channels of the sparse convolutional layer is made less than the number of output channels of its corresponding convolutional layer to be compressed. Through continuous iterative training, the number of output channels of the convolutional layer can be reduced, thus achieving sparsification of the convolutional layer.

[0033] S13: Train the sparsed convolutional neural network and use the trained convolutional neural network as the compressed convolutional neural network.

[0034] After obtaining the sparsed convolutional neural network, the network is continuously trained to ensure that the final compressed convolutional neural network reduces the number of convolutional kernel parameters while maintaining output accuracy.

[0035] The above scheme adds a sparse convolutional layer after each convolutional layer to be compressed in the convolutional neural network. Based on the weight parameters of each sparse convolutional layer, pruning and decomposing the convolutional layers to be compressed are performed, resulting in a sparsified convolutional neural network. By fusing convolutional kernel pruning and decomposition through sparse convolutional layers, pruning and decomposition can be performed simultaneously, compressing the model, reducing model parameters, and accelerating model computation.

[0036] Specifically, let {x, y} represent the network's input and output. Without losing feature values, let x represent the input feature of the convolutional layer, z represent the output feature after convolutional layer a, and W represent the convolution kernel. Therefore, the convolution between the input feature x and the kernel W can be converted into matrix multiplication, i.e.:

[0037] Z = X * W (1)

[0038] Where, X∈R N×cwh , W∈R cwh×N Z∈R N×n , representing the input features, convolution kernel, and output features of the convolutional layer, respectively. The parameters c, n, w, h, and N represent the number of input channels, the number of convolution kernels (or output channels), the width and height of the input feature map, and the number of input images, respectively. Please refer to... Figure 4 , Figure 4 This is a schematic diagram illustrating the workflow of a group sparse matrix in an application scenario of this application. As shown in the figure, if the columns of the group sparse matrix A are regularized, invalid convolutional kernels and corresponding output features will be deleted. If the rows of the group sparse matrix A are regularized, the weights of some rows in the convolutional kernels will be cleared to zero, resulting in the deletion of the convolutional kernels and feature maps of the previous layer. The convolutional neural network compression method of this application not only selects the convolutional kernels in one convolutional layer for compression, but also performs linear combination of the convolutional kernels to minimize the error between the original convolutional kernels and the decomposed convolutional kernels. On the other hand, when optimizing the sparsity problem, it is not necessary to change the weights of the original convolutional kernel W of the convolutional neural network too much to ensure the accuracy of the original network model.

[0039] In one embodiment, step S13 may specifically include: adding relevant regularization terms to the loss function of the convolutional neural network, and updating the parameters of the convolutional neural network according to the loss function of the convolutional neural network.

[0040] Understandably, during the training of a convolutional neural network (CNN), the parameters can be updated based on the CNN's loss function. When overfitting occurs during training, the model's training error is small, but the testing error is large. This means the model is so complex that it can fit all training data, but performs poorly when predicting new data. Therefore, a regularization term needs to be added to the CNN's loss function to prevent overfitting.

[0041] In one embodiment, the relevant regularization term includes a weight decay regularization term. The weight decay regularization term is L2 norm regularization, which reduces the learned model parameter values ​​by adding an L2 norm regularization term to the loss function. The L2 norm regularization term refers to the product of the sum of the squares of each element of the model weight parameters and a positive constant.

[0042] In one embodiment, the relevant regularization term includes a sparse regularization term. During the training phase, the parameters of the group sparse matrix are constrained, with unimportant parameters constrained to 0, resulting in a constrained group sparse matrix. This constrained group sparse matrix can then be used to constrain the convolutional kernel parameters of the previous convolutional layer, thereby achieving structured pruning of the convolutional kernel. It can be understood that the loss function of a convolutional neural network can be expressed as:

[0043] Loss Total =L(y,f(x,W))+αD(W)+βR(W) (2)

[0044] Where D(W) and R(W) are weight decay and sparsity regularization terms, and α and β are regularization factors. This application introduces a group sparse matrix A∈R n×n To add group sparsity constraints, the rows and columns of the group sparse matrix A are respectively represented by A i and A j This means that the matrix is ​​transformed into a 1*1 convolution kernel after the original convolutional layer. Therefore, the convolution in the original formula (1) can be transformed into Z = X*(W*A). By performing group sparsity regularization on the group sparse matrix A, the loss function in formula (2) becomes:

[0045] loss1=L(y,f(x,W,A))+αD(W)+βR(A) (3)

[0046] During training, the sparse group matrix A is optimized. The sparse group matrix A is used to perform pruning and decomposition operations on the convolution kernel, thereby reducing the model parameters of the resulting sparse convolutional neural network and compressing the model parameters.

[0047] In one embodiment, the relevant regularization term further includes a kernel sorting regularization term. Specifically, the step of adding relevant regularization terms to the loss function of the convolutional neural network and updating the parameters of the convolutional neural network according to the loss function of the convolutional neural network may include: calculating the matrix 2 norm of each convolutional kernel in each convolutional layer, and sorting the convolutional kernels according to the value of the matrix 2 norm; obtaining a kernel filtering threshold based on the kernel sorting relationship of all convolutional layers; and removing convolutional kernels in each convolutional layer according to the kernel filtering threshold to obtain the updated convolutional neural network.

[0048] By adding a kernel sorting regularization term to the loss function of formula (3), the kernel sorting regularization term is used to represent the influence of each kernel on the convolutional neural network. By sorting the kernels according to the value of the matrix 2 norm, and then deleting the kernels that have little influence on the result of the convolutional neural network, the network parameters can be reduced and a compressed model can be achieved.

[0049] Please combine Figure 5 , Figure 5 This is a schematic diagram illustrating the kernel sorting and selection in an application scenario of this application. The correlation of convolutional kernels in a convolutional layer is represented using the matrix 2-norm. The matrix 2-norm is calculated for each convolutional kernel in each convolutional layer, and the kernels are sorted according to the magnitude of the matrix 2-norm value. While ensuring the output accuracy of the sparsed convolutional neural network, the selection criteria for the current layer's convolutional kernel can be determined, i.e., the convolutional kernel selection threshold I is determined.

[0050]

[0051] Where i represents the i-th convolutional layer, convolutional kernels whose 2-norm values ​​are less than the kernel selection threshold I can be removed to obtain the updated convolutional neural network.

[0052] Furthermore, the steps of adding relevant regularization terms to the loss function of the convolutional neural network and updating the parameters of the convolutional neural network according to the loss function of the convolutional neural network may specifically include: adding weight decay regularization terms, sparsity regularization terms, and convolutional kernel sorting regularization terms to the loss function of the convolutional neural network to obtain a total loss function; training based on the total loss function to determine the optimal solution of the weight parameters of each sparse convolutional layer, so as to obtain the sparsified convolutional neural network.

[0053] Understandably, during the training of a network model, a global L2 loss function, loss2, can be established to learn the values ​​of γ and k.

[0054]

[0055] Learning the values ​​of γ and k can be considered a network model optimization problem; therefore, the total loss function of the convolutional neural network is:

[0056] loss total =loss1+loss2 (6)

[0057] Therefore, by learning, the optimal solutions for γ and k can be obtained, the kernel selection threshold I can be calculated, the kernels of each convolutional layer can be removed, and the training can be re-corrected to adapt to the updated network model, thereby reducing the kernel parameters while ensuring accuracy.

[0058] The above scheme utilizes the group sparsity matrix to perform kernel pruning and kernel decomposition on the convolution kernel. Two operations can be completed with a single matrix multiplication. At the same time, during training, the matrix 2 norm of the convolution kernel of each convolutional layer is calculated to obtain the kernel selection threshold, and convolution kernels are removed from the convolutional layers. The whole method can effectively reduce model parameters, reduce the memory usage of the model on the hardware device, and speed up the forward inference calculation.

[0059] Please see Figure 6 , Figure 6 This is a flowchart illustrating an embodiment of the convolutional neural network porting method of this application. The convolutional neural network porting method in this embodiment includes the following steps:

[0060] S61: Obtain the initial convolutional neural network in the first device.

[0061] S62: The initial convolutional neural network is compressed using a convolutional neural network compression method to obtain a compressed convolutional neural network. The compression method for the convolutional neural network includes any of the above-mentioned convolutional neural network compression methods.

[0062] S63: The compressed convolutional neural network is ported to the second device.

[0063] Understandably, the first device has ample storage space and fast processing speed, and the initial convolutional neural network (CNN) can be an image recognition network. Therefore, users can utilize the abundant data resources of the first device and implement image recognition through a complex initial CNN. However, the initial CNN places high demands on the hardware's storage and processing capabilities. Users cannot directly port the initial CNN from the first device to a lightweight second device for image recognition. The second device could be a mobile phone or embedded device. Therefore, any of the aforementioned CNN compression methods can be used to compress the initial CNN in the first device. While maintaining network accuracy, redundant parameters are released, and unnecessary computations are eliminated, resulting in a compressed CNN. This compressed CNN can then be ported to the second device for application, saving power, space, and time resources. This promotes the application of CNNs on lightweight second devices, making various computer vision tasks based on CNNs more relevant to daily life, rather than being limited to high-performance first devices.

[0064] Please see Figure 7 , Figure 7This is a schematic diagram of an embodiment of the convolutional neural network compression device of this application. The convolutional neural network compression device 60 in this embodiment includes an acquisition module 600, a processing module 602, and a training module 604 connected to each other. The acquisition module 600 is used to acquire an initial convolutional neural network to be compressed, and add a sparse convolutional layer after each convolutional layer to be compressed in the initial convolutional neural network. The processing module 602 is used to acquire the weight parameters of each sparse convolutional layer, and perform pruning and decomposition processing on each convolutional layer to be compressed based on the weight parameters to obtain a sparsed convolutional neural network. The training module 604 is used to train the sparsed convolutional neural network, and use the trained convolutional neural network as the compressed convolutional neural network.

[0065] In one embodiment, the sparse convolutional layer is a 1*1 convolutional layer; the processing module 602 performs the following steps: obtaining the weight parameters of each sparse convolutional layer, and pruning and decomposing each convolutional layer to be compressed based on the weight parameters to obtain a sparsified convolutional neural network. This includes: obtaining the group sparse matrix corresponding to each sparse convolutional layer based on the weight parameters of each sparse convolutional layer; for a certain convolutional kernel, multiplying the convolutional kernel by the row simplification matrix of the group sparse matrix to obtain a new convolutional kernel; and converting a single heavyweight convolutional kernel W into a lightweight convolutional kernel and a 1*1 convolutional kernel using the new convolutional kernel and the column simplification matrix of the group sparse matrix to obtain the sparsified convolutional neural network.

[0066] In one embodiment, the training module 604 performs the step of training the sparsed convolutional neural network and using the trained convolutional neural network as a compressed convolutional neural network, including: adding relevant regularization terms to the loss function of the convolutional neural network and updating the parameters of the convolutional neural network according to the loss function of the convolutional neural network.

[0067] In one embodiment, the relevant regularization term includes a weight decay regularization term.

[0068] In one embodiment, the relevant regularization term includes a sparse regularization term; the sparse regularization term is used to constrain the parameters of the group sparse matrix, constraining some parameters to 0.

[0069] In one embodiment, the relevant regularization term further includes a kernel sorting regularization term; the training module 604 performs the step of adding relevant regularization terms to the loss function of the convolutional neural network and updating the parameters of the convolutional neural network according to the loss function of the convolutional neural network, specifically including: calculating the matrix 2 norm of each convolutional kernel in each convolutional layer, and sorting the convolutional kernels according to the value of the matrix 2 norm; obtaining a kernel filtering threshold according to the kernel sorting relationship of all convolutional layers; and removing convolutional kernels in each convolutional layer according to the kernel filtering threshold to obtain the updated convolutional neural network.

[0070] In one embodiment, the relevant regularization term further includes a kernel sorting regularization term; the training module 604 performs the step of adding relevant regularization terms to the loss function of the convolutional neural network and updating the parameters of the convolutional neural network according to the loss function of the convolutional neural network, specifically including: adding weight decay regularization term, sparsity regularization term and kernel sorting regularization term to the loss function of the convolutional neural network to obtain a total loss function; training according to the total loss function to determine the optimal solution of the weight parameters of each sparse convolutional layer, so as to obtain the sparsified convolutional neural network.

[0071] For details regarding the compression method of the convolutional neural network implemented by the compression device 60 of this application, please refer to the above-described embodiment of the compression method of the convolutional neural network, which will not be repeated here.

[0072] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of an embodiment of the electronic device of this application. The electronic device 70 in this embodiment includes a processor 702 and a memory 701 interconnected; the memory 701 is used to store program instructions, and the processor 702 is used to execute the program instructions stored in the memory 701 to implement the steps of any of the above-described convolutional neural network compression methods or convolutional neural network porting methods embodiments. In a specific implementation scenario, the electronic device 70 may include, but is not limited to, a microcomputer or a server.

[0073] Specifically, processor 702 controls itself and memory 701 to implement the steps of any of the above-described convolutional neural network compression methods or convolutional neural network porting methods. Processor 702 can also be referred to as a CPU (Central Processing Unit). Processor 702 may be an integrated circuit chip with signal processing capabilities. Processor 702 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 702 can be implemented using integrated circuit chips.

[0074] In the above scheme, processor 702 adds a sparse convolutional layer after each convolutional layer to be compressed in the convolutional neural network. Based on the weight parameters of each sparse convolutional layer, it performs pruning and decomposition on each convolutional layer to be compressed, thus obtaining a sparsified convolutional neural network. By fusing convolutional kernel pruning and decomposition through sparse convolutional layers, pruning and decomposition can be performed simultaneously, which can compress the model, reduce model parameters, and accelerate model computation.

[0075] Please see Figure 9 , Figure 9 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 80 of the present application stores program instructions 800 thereon. When the program instructions 800 are executed by a processor, they implement the steps in any of the above-described embodiments of the compression method or porting method of the convolutional neural network.

[0076] The computer-readable storage medium 80 can be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or it can be a server that stores the program instructions 800. The server can send the stored program instructions 800 to other devices for execution, or it can execute the stored program instructions 800 itself.

[0077] In the several embodiments provided in this application, it should be understood that the disclosed methods, devices, and apparatuses can be implemented in other ways. For example, the device and apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0078] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0079] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0080] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A compression method for convolutional neural networks, characterized in that, The compression method for the convolutional neural network includes: Obtain the initial convolutional neural network to be compressed, and add a sparse convolutional layer after each convolutional layer to be compressed in the initial convolutional neural network; The weight parameters of each sparse convolutional layer are obtained, and the convolutional layers to be compressed are pruned and decomposed based on the weight parameters to obtain a sparse convolutional neural network. The sparsed convolutional neural network is trained, and the trained convolutional neural network is used as the compressed convolutional neural network. The convolutional neural network is used to process computer vision tasks; The process of obtaining the weight parameters of each sparse convolutional layer, and then pruning and decomposing each convolutional layer to be compressed based on the weight parameters to obtain a sparsed convolutional neural network includes: Based on the weight parameters of each sparse convolutional layer, the group sparse matrix corresponding to each sparse convolutional layer is obtained; For a given convolutional kernel, the kernel is multiplied by the row simplification matrix of the group sparse matrix to obtain a new convolutional kernel. Then, using the new convolutional kernel and the column simplification matrix of the group sparse matrix, the single heavyweight convolutional kernel is transformed into a lightweight convolutional kernel and a 1*1 convolutional kernel to obtain the sparsified convolutional neural network.

2. The compression method for convolutional neural networks according to claim 1, characterized in that, The step of training the sparsed convolutional neural network and using the trained convolutional neural network as a compressed convolutional neural network includes: A relevant regularization term is added to the loss function of the convolutional neural network, and the parameters of the convolutional neural network are updated according to the loss function.

3. The compression method for convolutional neural networks according to claim 2, characterized in that, The relevant regularization terms include weight decay regularization terms.

4. The compression method for convolutional neural networks according to claim 2, characterized in that, The relevant regularization terms include sparse regularization terms; The sparse regularization term is used to constrain the parameters of the group sparse matrix, constraining some parameters to 0.

5. The compression method for convolutional neural networks according to claim 2, characterized in that, The relevant regularization terms also include convolution kernel sorting regularization terms; The step of adding a relevant regularization term to the loss function of the convolutional neural network and updating the parameters of the convolutional neural network according to the loss function includes: Calculate the matrix 2 norm of each convolutional kernel in each convolutional layer, and sort the convolutional kernels according to the value of the matrix 2 norm; Based on the kernel sorting relationship of all convolutional layers, obtain the kernel filtering threshold; The convolutional kernels in each convolutional layer are removed according to the convolutional kernel selection threshold to obtain the updated convolutional neural network.

6. The compression method for convolutional neural networks according to claim 2, characterized in that, The step of adding a relevant regularization term to the loss function of the convolutional neural network and updating the parameters of the convolutional neural network according to the loss function includes: The weight decay regularization term, the sparsity regularization term, and the kernel sorting regularization term are added to the loss function of the convolutional neural network to obtain the total loss function; Training is performed based on the total loss function to determine the optimal solution for the weight parameters of each sparse convolutional layer, thereby obtaining the sparsified convolutional neural network.

7. A compression device for a convolutional neural network, characterized in that, The convolutional neural network is used to process computer vision tasks, and the compression device for the convolutional neural network includes: The acquisition module is used to acquire the initial convolutional neural network to be compressed, and add a sparse convolutional layer after each convolutional layer to be compressed in the initial convolutional neural network. The processing module is used to obtain the weight parameters of each sparse convolutional layer, and perform pruning and decomposition processing on each convolutional layer to be compressed based on the weight parameters to obtain a sparse convolutional neural network. The training module is used to train the sparsed convolutional neural network and use the trained convolutional neural network as a compressed convolutional neural network. The processing module is further configured to obtain the group sparse matrix corresponding to each sparse convolutional layer according to the weight parameters of each sparse convolutional layer; for a certain convolutional kernel, multiply the convolutional kernel by the row simplification matrix of the group sparse matrix to obtain a new convolutional kernel; and convert a single heavyweight convolutional kernel into a lightweight convolutional kernel and a 1*1 convolutional kernel through the new convolutional kernel and the column simplification matrix of the group sparse matrix to obtain the sparsified convolutional neural network.

8. An electronic device, characterized in that, The electronic device includes a processor and a memory interconnected with each other; The memory is used to store program instructions, and the processor is used to execute the program instructions to implement the compression method of the convolutional neural network as described in any one of claims 1-6.

9. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the compression method of the convolutional neural network according to any one of claims 1 to 6.

Images