Convolutional neural network pruning method and device based on feature similarity and feature compensation

Abstract

This invention discloses a convolutional neural network pruning method and apparatus based on feature similarity and feature compensation. The method includes: adjusting the size of the input images in the dataset to a fixed size, standardizing the pixel values ​​of the images, and increasing the training data using image enhancement techniques; initializing the network structure and setting model parameters, and training the model using the training data; for the trained model, obtaining the similarity between convolutional kernels, performing cluster analysis based on the similarity, grouping each layer of convolutional kernels into similar groups, selecting the convolutional kernels to be retained in each similar group, and generating a new network structure; copying the parameters of the retained convolutional kernels into the new network structure, and compensating the weight parameters of the pruned convolutional kernels in each similar group into the retained convolutional kernels through parameter superposition; performing model accuracy recovery training using the original dataset, and saving the model parameters and network structure. This pruning method can maintain model accuracy.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and model compression, specifically to a method and apparatus for pruning convolutional neural networks based on feature similarity and feature compensation. Background Technology

[0002] With the continuous improvement of machine computing power, Convolutional Neural Networks (CNNs) have made significant progress in the field of computer vision, such as image recognition, object detection, and image segmentation. However, the performance of CNN models is closely related to their complexity. To achieve optimal results in various computer vision tasks, deeper and wider networks are often required, resulting in huge floating-point computations and a large number of parameters. For computational tasks requiring real-time performance and privacy, it is necessary to perform model inference directly on edge devices, but resource-constrained edge devices often struggle to deploy large-scale network models. Therefore, reducing the floating-point computation and parameter count of models is of great significance for edge device deployment.

[0003] To overcome the challenge of deployment on edge devices, many application scenarios employ network pruning methods. Addressing the massive floating-point computations and parameter counts, weights or network structures with minimal impact on model accuracy can be pruned. Network pruning schemes are categorized into unstructured pruning and structured pruning. Unstructured pruning, due to the sparse weight matrix after pruning, struggles to achieve compression and computational acceleration without dedicated computing hardware. Structured pruning compresses the network size by trimming network structures such as convolutional kernels and layers, while retaining the original convolutional structure. Most structured pruning schemes assign importance metrics to each convolutional kernel or channel, using a globally set threshold to prune kernels with importance metrics below the threshold. The pruned model exhibits significant size reduction compared to the original model, thus partially addressing the issue of directly deploying network models on edge devices.

[0004] Most structured pruning schemes focus primarily on the design of convolutional kernels or channel importance metrics, neglecting the similarity of convolutional kernel or channel features. During the inference process of a convolutional neural network, the output feature maps of different convolutional kernels at the same level exhibit similarity. Pruning convolutional kernels with similar behaviors can reduce the loss of model accuracy while compressing the model size. Furthermore, under normal model training conditions, the pruned network parameters contain a significant amount of feature information, which can positively correlate with model accuracy. During network pruning, this feature information needs to be passed to the preserved network structure through feature compensation. Therefore, considering feature similarity and feature compensation is essential for structured pruning. Summary of the Invention

[0005] Purpose of the invention: To address the shortcomings of existing technologies, this invention provides a convolutional neural network pruning method and apparatus based on feature similarity and feature compensation. This method can reduce the loss of model accuracy while decreasing the amount of floating-point calculations and parameters, thereby enabling the model to be deployed directly on edge devices, improving the real-time performance of inference and protecting privacy and security.

[0006] Technical Solution: To achieve the above-mentioned objectives, the technical solution of this invention is as follows:

[0007] A convolutional neural network pruning method based on feature similarity and feature compensation includes the following steps:

[0008] The input images in the dataset are resized to a fixed size, the pixel values ​​of the images are standardized, and the training data of the original dataset is increased by image enhancement techniques.

[0009] Initialize the network structure and set the model parameters, and train the model using the training data;

[0010] For the trained model, the similarity between convolutional kernels is obtained, and cluster analysis is performed based on the similarity. Each layer of convolutional kernels is grouped into similar groups, and the convolutional kernels that are retained in each similar group are selected to generate a new network structure.

[0011] The parameters of the retained convolutional kernels are copied into the new network structure, and the weight parameters of the pruned convolutional kernels in each similar group are compensated to the retained convolutional kernels by parameter superposition.

[0012] The original dataset is used for model accuracy recovery training, and the model parameters and network structure are saved.

[0013] According to an embodiment of the present invention, obtaining the similarity between convolutional kernels includes: stretching the three-dimensional tensor of each convolutional kernel into a one-dimensional tensor, calculating the cosine similarity between the one-dimensional tensors, and thereby measuring the similarity between convolutional kernels.

[0014] According to an embodiment of the present invention, the tensor stretching process is represented as follows:

[0015]

[0016] in, It is the j-th convolutional kernel in the i-th layer, which is composed of n i h i ×w i Composed of two-dimensional tensors, which are stretched to form one-dimensional tensors. n i n represents the number of input feature maps in the i-th layer. i+1h represents the number of output feature maps of the i-th layer. i with w i These represent the height and width of the two-dimensional tensor in each three-dimensional tensor, respectively.

[0017] According to an embodiment of the present invention, similarity grouping of each convolutional kernel layer includes: for a neural network with L convolutional kernel layers, using the k-means clustering analysis algorithm to perform similarity grouping of each convolutional kernel layer, wherein the number of groups is determined by the pruning rate p set for each layer. i The number of groups in the i-th layer convolutional kernel is determined to be k. i =p i ×n i+1 where 1≤i≤L, n i+1 This represents the number of output feature maps of the i-th layer.

[0018] According to an embodiment of the present invention, selecting the convolutional kernel to be retained in each similar group includes: using the one-dimensional tensor stretching result, calculating the sum of the absolute values ​​of the weight parameters of each convolutional kernel, and selecting the convolutional kernel with the largest sum of the absolute values ​​of the weight parameters in each similar group as the convolutional kernel to be retained.

[0019] According to an embodiment of the present invention, generating a new network structure includes: setting an index for each convolutional kernel in each layer, saving the indices of the convolutional kernels to be retained in a mask array, and generating a new network structure based on the mask array, wherein the mask array is represented as:

[0020]

[0021] mask i Let represent the mask array for layer i, where 1 indicates that the j-th convolutional kernel in layer i is retained, 0 indicates that the j-th convolutional kernel in layer i is pruned, I represents the set of retained convolutional kernels, and U represents the set of pruned convolutional kernels.

[0022] According to an embodiment of the present invention, the parameter superposition method is as follows: the weight parameters of each group of pruned convolution kernels are shared with the retained convolution kernels. The sharing refers to superimposing the tensors at the corresponding positions of the convolution kernels in the group as the convolution kernels after sharing the parameters.

[0023] A convolutional neural network pruning device based on feature similarity and feature compensation, comprising:

[0024] The data preprocessing module is configured to resize the input images in the dataset to a fixed size, standardize the pixel values ​​of the images, and increase the training data of the original dataset through image enhancement techniques.

[0025] The model training module is configured to initialize the network structure and set the model parameters, and train the model using training data;

[0026] The network pruning module is configured to obtain the similarity between convolutional kernels for the trained model, perform cluster analysis based on the similarity, group the convolutional kernels of each layer into similar groups, select the convolutional kernels to be retained in each similar group, and generate a new network structure.

[0027] The parameter compensation module is configured to copy the parameters of the retained convolutional kernels into the new network structure and compensate the weight parameters of the pruned convolutional kernels in each similar group to the retained convolutional kernels by parameter superposition.

[0028] The accuracy recovery module is configured to perform model accuracy recovery training using the original dataset, saving model parameters and network structure.

[0029] The present invention also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the convolutional neural network pruning method based on feature similarity and feature compensation as described above.

[0030] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the convolutional neural network pruning method based on feature similarity and feature compensation as described above.

[0031] Compared with existing technologies, this invention has the following advantages and beneficial effects: Firstly, addressing the importance of convolutional kernel similarity and redundant parameters, this invention groups convolutional kernels with similar outputs at each layer using a clustering method, retaining one kernel in each group, thereby achieving network compression. Secondly, addressing the issue that most network pruning methods do not fully consider the impact of pruned structure parameters on network performance, this invention, during network pruning, compensates for the parameters of pruned convolutional kernels in each group by superimposing them into the retained convolutional kernels, allowing the network to obtain more information and improving network performance. Thirdly, using this invention, it is possible to reduce the model's floating-point computation and parameter count while minimizing the loss of model accuracy, thus enabling the model to be deployed directly on edge devices, improving inference real-time performance and protecting privacy and security. Attached Figure Description

[0032] Figure 1 This is a flowchart of the convolutional neural network pruning method of the present invention;

[0033] Figure 2 This is a schematic diagram of the network structure generated before model training;

[0034] Figure 3 This is a schematic diagram of the network pruning process;

[0035] Figure 4 This is a schematic diagram of the model accuracy recovery training after pruning. Detailed Implementation

[0036] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0037] The network pruning method of this invention compresses the trained model. Before pruning, the network structure and dataset need to be determined, and the model needs to be trained. After pruning, the original dataset is used to train the pruned model to restore accuracy. (Refer to...) Figure 1 The present invention provides a convolutional neural network pruning method based on feature similarity and feature compensation, comprising the following steps:

[0038] Step (1) adjusts the size of the input images in the dataset to a fixed size, standardizes the pixel values ​​of the images, and increases the training data of the original dataset through image enhancement technology.

[0039] The input image size is adjusted to a fixed size to facilitate network input. In this embodiment of the invention, for the VGG network structure, the input image size is fixed at 3×32×32, and for the ResNet network structure, the input image size is fixed at 3×224×224.

[0040] Image enhancement techniques, such as rotation, flipping, and cropping, can increase the diversity of the dataset, thereby increasing the training data for the network and improving its generalization ability.

[0041] Step (2): Initialize the network structure and set the model parameters, and train the model using the training data.

[0042] For model training, different network structures are initialized according to different needs. Figure 2 This is an example of the generated network structure. In this embodiment of the invention, the optimization algorithm is set to SGD (Stochastic Gradient Descent), the initial learning rate is 0.1, the learning rate decay is 0.1, the momentum is 0.9, the weight decay is 5e-4, the batch size is 128, and the learning rate decays every 30 rounds. The model is trained using the enhanced training data.

[0043] Step (3): For the trained model, obtain the similarity between convolutional kernels, perform cluster analysis based on the similarity, group each layer of convolutional kernels into similar groups, select the convolutional kernels to be retained in each similar group, and generate a new network structure.

[0044] Unlike setting the pruning rate globally, this invention adopts a layer-by-layer pruning rate setting scheme. In this way, the compression degree of the model can be freely set according to the redundancy situation, so that the importance of the pre-convolutional layer that extracts more feature information does not need to be compared with the post-convolutional layer that has more redundant information, thus limiting the importance comparison of each layer to the current layer.

[0045] For a trained model, convolutional kernels are obtained by traversing the module of each layer. To facilitate the calculation of cosine similarity between different convolutional kernels, the three-dimensional convolutional kernel tensor is stretched into a one-dimensional convolutional kernel tensor. The tensor stretching process can be represented by formula (1):

[0046]

[0047] in, It is the j-th convolutional kernel in the i-th layer, which is composed of n i h i ×w i Composed of two-dimensional tensors, which are stretched to form one-dimensional tensors. n i n represents the number of input feature maps in the i-th layer. i+1 h represents the number of output feature maps of the i-th layer. i with w i These represent the height and width of the two-dimensional tensor in each three-dimensional tensor, respectively.

[0048] After tensor stretching is completed, the calculation of the cosine similarity between one-dimensional tensors can be expressed by formula (2):

[0049]

[0050] in, and These represent the m-th and n-th convolutional kernels in the i-th layer, respectively.

[0051] The k-means clustering algorithm is used to group convolutional kernels based on similarity. For a network with L convolutional layers, the number of groups in each kernel clustering is determined by the pruning rate p set for each layer. i (1≤i≤L) determines the number of groups in the i-th convolutional kernel, which can be represented by formula (3):

[0052] k i =p i ×n i+1 (3)

[0053] After performing the above operations, the similarity grouping of convolutional kernels in each convolutional layer can be obtained. Since the convolutional kernels are similar, their corresponding output feature maps are also similar. When the next convolutional layer performs convolution operations on these similar feature maps, it will produce similar results. Therefore, only one convolutional kernel is retained for each similar group to fully consider the influence of feature similarity. In the model inference process, larger weight parameters have a greater impact on the model inference results. For each similar group, the importance of each convolutional kernel is measured by the sum of the absolute values ​​of the weight parameters in the group. Only the convolutional kernel with the largest sum of the absolute values ​​of the weight parameters is retained in each group. The sum of the absolute values ​​of the weight parameters of the convolutional kernel can be expressed by formula (4):

[0054]

[0055] in, This represents the sum of the absolute values ​​of the weight parameters of the j-th convolutional kernel in the i-th layer.

[0056] After determining the convolutional kernels retained in each group, the pruned network structure is also determined. To facilitate copying the parameters of the retained convolutional kernels from the original model to the new network structure, and for feature compensation to utilize redundant parameters, a mask array needs to be set in each layer to mark the indices of the retained convolutional kernels from the original model. This mask array can be represented by formula (5):

[0057]

[0058] mask i Let represent the mask array of the i-th layer, where 1 indicates that the j-th convolutional kernel in the i-th layer is retained, 0 indicates that the j-th convolutional kernel in the i-th layer is pruned, I represents the set of retained convolutional kernels, and U represents the set of pruned convolutional kernels.

[0059] Step (4) involves copying the retained convolutional kernel parameters into the new network structure and compensating the weight parameters of the pruned convolutional kernels in each similar group into the retained convolutional kernels by parameter superposition.

[0060] After obtaining the mask array, a new network structure is generated based on the mask array, and the parameters of the retained convolutional kernels are copied into the new network structure. After copying, feature compensation is performed based on the grouping and the retained convolutional kernels. The method of this invention considers the weight parameters of the pruned convolutional kernels. For conventionally trained models, directly judging the importance of convolutional kernels by some metric has poor interpretability and poor accuracy before the model is retrained. Since convolutional kernels in the same group have similar behaviors, their output feature maps are also quite similar. Therefore, the method of this invention shares the weight parameters of each group of pruned convolutional kernels with the retained convolutional kernels through superposition, thereby achieving feature compensation. The parameter superposition method is as follows: the weight parameters of each group of pruned convolutional kernels are shared with the retained convolutional kernels. The sharing refers to superimposing the tensors at the corresponding positions of the convolutional kernels in the group as the convolutional kernels after sharing the parameters.

[0061] For example, in the i-th layer, the convolution kernel resemblance, The maximum sum of the absolute values ​​of the weight parameters will... and The weight parameters are shared by superposition. The convolution kernel stacking can be represented by formula (6):

[0062]

[0063] Where q1, q2, q3 ∈ [1, n i+1 ], This represents the convolution kernel after sharing parameters. and Represent and The nth i A two-dimensional tensor Indicate and The two-dimensional tensors at corresponding positions are superimposed. This represents superimposing the one-dimensional tensors at corresponding positions within a two-dimensional tensor. and The relevant parameters will be pruned in the newly generated network.

[0064] The above description, combined with steps (3) and (4), discloses the core technical points of the network pruning and feature compensation of the present invention, and its flowchart is shown below. Figure 3As shown, after clustering, it is necessary to calculate the maximum value of the sum of the absolute values ​​of the weight parameters of each convolutional kernel within the group. This maximum value is used to determine which convolutional kernels should be retained in each similar group. The sum of the absolute values ​​of the weight parameters can be calculated using a stretched one-dimensional tensor. While determining the convolutional kernels to be retained, an index can be assigned to each kernel, and the index of the retained kernels is recorded in a mask array. A new network structure is generated based on the mask array, and the weight parameters of the retained convolutional kernels are copied into the new network structure. In the new network structure, based on the results of each group and the index of the retained convolutional kernels, the weight parameters of the pruned convolutional kernels in each group are superimposed onto the retained convolutional kernels, achieving feature compensation. Figure 3 In the i-th convolutional layer, different backgrounds are used to represent the convolutional kernels in different groups. The convolutional kernels that are pruned in a certain group are marked in gray. Since the convolutional kernel is pruned, it does not affect the output feature map of the (i+1)-th layer and is represented by a dashed line.

[0065] Step (5): Use the original dataset to perform model accuracy recovery training and save the model parameters and network structure.

[0066] Figure 4 This diagram illustrates the precision recovery training of the pruned model. After network pruning, the pruned network structure is obtained. The pruned parts are represented by dashed lines. For different models, fine-grained learning rates are specified, and the original dataset is used for precision recovery training. Finally, the model parameters and network structure are saved for deployment and use.

[0067] To address the challenge of balancing pruning intensity and model accuracy in convolutional neural networks, this invention considers the existence of similar output feature maps in the convolutional results of each layer. These similar feature maps, when used as input to the next convolutional layer, produce similar results. This invention prunes redundant parameters by removing convolutional kernels corresponding to similar output feature maps. During network pruning, the weights of pruned convolutional kernels are directly discarded; however, these pruned weights often positively impact model accuracy. This invention compensates for these parameters by superimposing them back into the network. Furthermore, the deeper the model, the more redundant parameters there are. For shallower layers, the pruning ratio can be reduced; for deeper layers, the pruning ratio can be increased to preserve effective information.

[0068] Based on the same inventive concept as the method, the present invention also provides a convolutional neural network pruning device based on feature similarity and feature compensation, comprising:

[0069] The data preprocessing module is configured to resize the input images in the dataset to a fixed size, standardize the pixel values ​​of the images, and increase the training data of the original dataset through image enhancement techniques.

[0070] The model training module is configured to initialize the network structure and set the model parameters, and train the model using training data;

[0071] The network pruning module is configured to obtain the similarity between convolutional kernels for the trained model, perform cluster analysis based on the similarity, group the convolutional kernels of each layer into similar groups, select the convolutional kernels to be retained in each similar group, and generate a new network structure.

[0072] The parameter compensation module is configured to copy the parameters of the retained convolutional kernels into the new network structure and compensate the weight parameters of the pruned convolutional kernels in each similar group to the retained convolutional kernels by parameter superposition.

[0073] The accuracy recovery module is configured to perform model accuracy recovery training using the original dataset, saving model parameters and network structure.

[0074] It should be understood that the convolutional neural network pruning device in the embodiments of the present invention can implement all the technical solutions in the above method embodiments. The functions of each functional module can be specifically implemented according to the methods in the above method embodiments. The specific implementation process can be referred to the relevant descriptions in the above embodiments, which will not be repeated here.

[0075] The present invention also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the convolutional neural network pruning method based on feature similarity and feature compensation as described above.

[0076] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the convolutional neural network pruning method based on feature similarity and feature compensation as described above.

[0077] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, computer devices, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0078] This invention is described with reference to a flowchart of a method according to embodiments of the invention. It should be understood that each step in the flowchart and combinations thereof can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the process. Figure 1 A device for a function specified in one or more processes.

[0079] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 The function specified in one or more processes.

[0080] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 Steps of a specified function in one or more processes.

[0081] The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A convolutional neural network pruning method based on feature similarity and feature compensation, characterized in that, Includes the following steps: The input images in the dataset are resized to a fixed size, the pixel values ​​of the images are standardized, and the training data of the original dataset is increased by image enhancement techniques. Initialize the network structure and set the model parameters, and train the model using the training data; For the trained model, the similarity between convolutional kernels is obtained, and cluster analysis is performed based on the similarity. Each layer of convolutional kernels is grouped into similar groups, and the convolutional kernels that are retained in each similar group are selected to generate a new network structure. The process of obtaining the similarity between convolutional kernels includes: stretching the three-dimensional tensor of each convolutional kernel into a one-dimensional tensor and calculating the cosine similarity between the one-dimensional tensors; selecting the convolutional kernel to be retained in each similarity group includes: using the one-dimensional tensor stretching result, calculating the sum of the absolute values ​​of the weight parameters of each convolutional kernel, and selecting the convolutional kernel with the largest sum of the absolute values ​​of the weight parameters in each similarity group as the convolutional kernel to be retained. The retained convolutional kernel parameters are copied into the new network structure. The weight parameters of the pruned convolutional kernels in each similar group are compensated to the retained convolutional kernels by parameter superposition. The parameter superposition method is as follows: the weight parameters of each group of pruned convolutional kernels are shared with the retained convolutional kernels. The sharing means superimposing the tensors at the corresponding positions of the convolutional kernels in the group as the convolutional kernels after sharing the parameters. The original dataset is used for model accuracy recovery training, and the model parameters and network structure are saved.

2. The method according to claim 1, characterized in that, The tensor stretching process is represented as: ； in, It is the j-th convolutional kernel in the i-th layer, which is composed of indivual Composed of two-dimensional tensors, which are stretched to form one-dimensional tensors. , This represents the number of input feature maps in the i-th layer. This represents the number of output feature maps of the i-th layer. and These represent the height and width of the two-dimensional tensor in each three-dimensional tensor, respectively.

3. The method according to claim 1, characterized in that, Grouping convolutional kernels based on similarity involves: for a neural network with L convolutional kernels, using the k-means clustering algorithm to group each kernel based on similarity, with the number of groups determined by the pruning rate set for each layer. The number of groups in the i-th layer convolutional kernel is determined as follows: ,in , This represents the number of output feature maps of the i-th layer.

4. The method according to claim 1, characterized in that, Generating a new network structure involves: setting an index for each convolutional kernel in each layer, saving the indices of the convolutional kernels to be retained into a mask array, and generating a new network structure based on the mask array, where the mask array is represented as follows: ； in This represents the mask array for the i-th layer, where 1 indicates that the j-th convolutional kernel in the i-th layer is preserved, and 0 indicates that the j-th convolutional kernel in the i-th layer is pruned. This represents the set of convolution kernels that are preserved. This represents the set of pruned convolution kernels.

5. A convolutional neural network pruning device based on feature similarity and feature compensation, characterized in that, include: The data preprocessing module is configured to resize the input images in the dataset to a fixed size, standardize the pixel values ​​of the images, and increase the training data of the original dataset through image enhancement techniques. The model training module is configured to initialize the network structure and set the model parameters, and train the model using training data; The network pruning module is configured to obtain the similarity between convolutional kernels for the trained model, perform cluster analysis based on the similarity, group the convolutional kernels of each layer into similar groups, select the convolutional kernels to be retained in each similar group, and generate a new network structure. The process of obtaining the similarity between convolutional kernels includes: stretching the three-dimensional tensor of each convolutional kernel into a one-dimensional tensor and calculating the cosine similarity between the one-dimensional tensors; selecting the convolutional kernel to be retained in each similarity group includes: using the one-dimensional tensor stretching result, calculating the sum of the absolute values ​​of the weight parameters of each convolutional kernel, and selecting the convolutional kernel with the largest sum of the absolute values ​​of the weight parameters in each similarity group as the convolutional kernel to be retained. The parameter compensation module is configured to copy the parameters of the retained convolutional kernels into the new network structure, and to compensate the weight parameters of the pruned convolutional kernels in each similar group to the retained convolutional kernels by parameter superposition. The parameter superposition method is as follows: the weight parameters of each group of pruned convolutional kernels are shared with the retained convolutional kernels. The sharing refers to superimposing the tensors at the corresponding positions of the convolutional kernels in the group as the convolutional kernels after sharing the parameters. The accuracy recovery module is configured to perform model accuracy recovery training using the original dataset, saving model parameters and network structure.

6. A computer device, characterized in that, include: One or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the convolutional neural network pruning method based on feature similarity and feature compensation as described in any one of claims 1-4.

7. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the convolutional neural network pruning method based on feature similarity and feature compensation as described in any one of claims 1-4.

Images