A compression method of a deep neural network model and a related system

Abstract

This invention belongs to the field of data network models and discloses a compression method and related system for deep neural network models. The pruning method of this invention reduces the number of parameters in the model by removing unimportant weights. This not only simplifies the model structure but also reduces computational complexity and memory requirements. The pruned model requires fewer computational resources, especially during the inference stage, which is crucial for running on resource-constrained devices such as mobile devices and embedded systems. This invention reduces redundant computations, and the pruned model can accelerate the training and inference processes, improving overall running speed. The pruning method of this invention can also have a regularization effect, reducing the risk of overfitting and allowing the model to perform better when processing unseen data. The pruning method of this invention can significantly reduce the model's storage requirements and energy consumption, saving substantial resources for deployment in large-scale service or cloud computing environments.

Description

Technical Field

[0001] This invention belongs to the field of data network models, specifically relating to a compression method and related system for deep neural network models. Background Technology

[0002] In recent years, deep neural networks have become the mainstream computational model in artificial intelligence. They have demonstrated superior performance in many fields such as image recognition, speech recognition, and natural language processing. Currently popular and efficient network architectures include Res-NET, GoogLeNet, and VGG-Net. However, with the improvement of network performance, the number of network layers has increased from a few layers to a dozen layers, and now to more than one hundred layers. Deep neural networks contain many convolutional layers, and the computation of convolutional layers accounts for the vast majority of the network's computation. The more convolutional layers there are, the greater the computational cost. For example, in the VGG-Net-16 network architecture, 13 convolutional layers account for approximately 90-95% of the computation; fully connected (FC) layers account for approximately 5-10% of the computation time and 95% of the parameters. This leads to an increasing demand for computational and storage resources, requiring the use of high-performance GPUs or GPU clusters to implement neural network computations. Embedded devices have limited computing power and storage capacity, which restricts the application of deep networks in embedded devices. Therefore, many researchers have devoted themselves to the compression and acceleration of deep neural networks, with the aim of reducing the computational load and storage of the network while maintaining network performance, so that it can be deployed in embedded devices.

[0003] There are many methods for compressing and accelerating deep neural networks, such as matrix factorization, low-precision weights, knowledge distillation, and pruning. Most of these methods operate on the weight matrix itself, meaning smaller weights contribute less, and redundant weights are eliminated by removing as many small weights as possible. However, using the magnitude of the weights themselves as a basis for judging their importance lacks theoretical basis; some small weights may be very important. Summary of the Invention

[0004] The purpose of this invention is to overcome the problem of data bias caused by manually assigning weights, and to provide a method and system for compressing deep neural network models.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides a method for compressing a deep neural network model, comprising the following steps:

[0007] Obtain the dataset of the fully connected layer in the deep neural network model, and get the weight matrix of the fully connected layer dataset;

[0008] A gradient-based pruning method is used to process the weight matrix of the fully connected layer dataset to obtain the pruned weight matrix of the fully connected layer dataset.

[0009] The weight matrix of the pruned fully connected layer dataset is restored to the pruned fully connected layer dataset;

[0010] The pruned fully connected layer dataset is fed into the deep neural network model for retraining to obtain the final deep neural network model.

[0011] A further improvement of this invention lies in the following specific method for processing the weight matrix of the fully connected layer dataset using a gradient-based pruning method:

[0012] Based on the weight matrix of the fully connected layer dataset, obtain the gradient matrix of the weight matrix;

[0013] Obtain the maximum absolute value of the gradient in the gradient matrix, and based on the maximum absolute value of the gradient in the gradient matrix, obtain the step size and candidate threshold.

[0014] Count the number of gradients whose gradient values ​​are less than the candidate threshold in the gradient matrix of the weight matrix;

[0015] The gradient matrix of the weight matrix is ​​pruned based on the number of gradients less than the candidate threshold, thus completing the processing.

[0016] A further improvement of this invention is that, when pruning the gradient matrix of the weight matrix based on the number of gradients less than the candidate threshold, it is determined whether the number of remaining parameters after pruning is the same between adjacent candidate thresholds. If they are the same, the current candidate threshold is saved as the final threshold; if they are different, pruning continues.

[0017] A further improvement of this invention lies in the following specific method of pruning:

[0018] The pruning process is complete by replacing the weight matrix corresponding to gradient values ​​that are less than the final threshold with 0.

[0019] A further improvement of this invention lies in the following method for calculating the gradient matrix of the weight matrix:

[0020]

[0021] in, Let be the gradient matrix of the weight matrix, and Loss be the missing function. This is the weight matrix of the dataset for the fully connected layer.

[0022] A further improvement of this invention is that the step size is calculated as follows:

[0023]

[0024] Where step is the step size, |g| max is the maximum absolute value of the gradient in the gradient matrix, and m is any threshold among the candidate thresholds;

[0025] The candidate threshold is calculated as follows:

[0026] T m =step×m

[0027] Among them, T m is the candidate threshold.

[0028] Secondly, the present invention provides a compression system for deep neural network models, comprising:

[0029] The weight matrix acquisition module is used to obtain the dataset of the fully connected layer in the deep neural network model and obtain the weight matrix of the fully connected layer dataset.

[0030] The pruning module is used to process the weight matrix of the fully connected layer dataset using a gradient-based pruning method to obtain the pruned weight matrix of the fully connected layer dataset.

[0031] The restore module is used to restore the weight matrix of the pruned fully connected layer dataset to the pruned fully connected layer dataset.

[0032] The retraining module is used to feed the pruned fully connected layer dataset into the deep neural network model for retraining, resulting in the final deep neural network model.

[0033] A further improvement of the present invention is that the pruning module includes:

[0034] The gradient matrix acquisition module is used to obtain the gradient matrix of the weight matrix based on the weight matrix of the fully connected layer dataset.

[0035] The step size and candidate threshold acquisition module is used to obtain the maximum absolute value of the gradient in the gradient matrix, and obtain the step size and candidate threshold based on the maximum absolute value of the gradient in the gradient matrix.

[0036] The gradient count acquisition module is used to count the number of gradients whose gradient values ​​are less than the candidate threshold in the gradient matrix of the weight matrix;

[0037] The gradient matrix processing module is used to prune the gradient matrix of the weight matrix based on the number of gradients less than the candidate threshold, thus completing the processing.

[0038] Thirdly, the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of a method for compressing a deep neural network model.

[0039] Fourthly, the present invention provides a storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of a compression method for a deep neural network model.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] This invention reduces the number of model parameters by removing unimportant weights. This not only simplifies the model structure but also reduces computational complexity and memory requirements. The pruned model requires fewer computational resources, especially during inference, which is crucial for running on resource-constrained devices such as mobile devices and embedded systems. This invention reduces redundant computations, allowing the pruned model to accelerate training and inference processes and improve overall performance. The pruning mechanism of this invention acts as a regularization mechanism, reducing the risk of overfitting and enabling the model to perform better on unseen data. The pruning significantly reduces the model's storage requirements and energy consumption, saving substantial resources for deployments in large-scale service or cloud computing environments. The pruned model is more streamlined, potentially improving interpretability, as the remaining weights and nodes often represent the features most important to the task. This gradient-based pruning method identifies and retains important weights, thereby significantly reducing the number of parameters while maintaining model performance as much as possible. In summary, this invention can be used in conjunction with various deep neural network architectures and different tasks, offering high flexibility and adaptability.

[0042] This invention solves the problem of threshold selection, enabling each network layer to find a suitable value. While compressing parameters to accelerate computation, this invention also reduces overfitting, resulting in a certain degree of performance improvement after pruning. However, for network structures containing Batch Normalization (BN) layers, although this invention does not improve network performance, it still maintains the accuracy of the network model while pruning parameters. Generally, deeper networks perform better. This invention enables a low-layer AlexNet to achieve similar performance to a high-layer VGG-Net, but with significantly less computational cost. Attached Figure Description

[0043] Figure 1 This is a flowchart of the present invention;

[0044] Figure 2 This is a system diagram of the present invention;

[0045] Figure 3 This is a system diagram of the pruning module in this invention;

[0046] Figure 4 This is a system diagram for an embodiment. Detailed Implementation

[0047] To further understand the content of this invention, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments are merely illustrative and not limiting of the invention.

[0048] See Figure 1 A method for compressing a deep neural network model includes the following steps:

[0049] S1, obtain the dataset of the fully connected layer in the deep neural network model, and obtain the weight matrix of the fully connected layer dataset.

[0050] S2 uses a gradient-based pruning method to process the weight matrix of the fully connected layer dataset, resulting in the pruned weight matrix of the fully connected layer dataset.

[0051] S3 restores the weight matrix of the pruned fully connected layer dataset to the pruned fully connected layer dataset.

[0052] S4. The pruned fully connected layer dataset is fed into the deep neural network model for retraining to obtain the final deep neural network model.

[0053] See Figure 2 A compression system for deep neural network models, comprising:

[0054] The weight matrix acquisition module is used to obtain the dataset of the fully connected layer in the deep neural network model and obtain the weight matrix of the fully connected layer dataset.

[0055] The pruning module is used to process the weight matrix of the fully connected layer dataset using a gradient-based pruning method to obtain the pruned weight matrix of the fully connected layer dataset.

[0056] The restore module is used to restore the weight matrix of the pruned fully connected layer dataset to the pruned fully connected layer dataset.

[0057] The retraining module is used to feed the pruned fully connected layer dataset into the deep neural network model for retraining, resulting in the final deep neural network model.

[0058] See Figure 3 The pruning module includes:

[0059] The gradient matrix acquisition module is used to obtain the gradient matrix of the weight matrix based on the weight matrix of the fully connected layer dataset.

[0060] The step size and candidate threshold acquisition module is used to obtain the maximum absolute value of the gradient in the gradient matrix, and obtain the step size and candidate threshold based on the maximum absolute value of the gradient in the gradient matrix.

[0061] The gradient count acquisition module is used to count the number of gradients whose gradient values ​​are less than the candidate threshold in the gradient matrix of the weight matrix;

[0062] The gradient matrix processing module is used to prune the gradient matrix of the weight matrix based on the number of gradients less than the candidate threshold, thus completing the processing.

[0063] Example 1:

[0064] Let be the weight matrix of the fully connected layer dataset, and k×k×d×n be the shape of the weight matrix. The specific method for processing the weight matrix of the fully connected layer dataset using gradient-based pruning is as follows:

[0065] Step 1: Calculate the gradient matrix of the weight matrix using the following formula. The shape remains unchanged.

[0066]

[0067] Loss represents the missing function.

[0068] Step two, in the gradient matrix Find the maximum absolute value of the gradient |g| max Set m=100, and calculate the step size step and candidate threshold T using the following formula. m T m This represents the m-th threshold.

[0069]

[0070] T m =step×m

[0071] m = 1, 2, 3, ..., 100

[0072] Step 3, based on the candidate threshold T m In the gradient matrix The statistical gradient value is less than the candidate threshold T m The number of gradients, denoted as N. m .

[0073] Step 4: Calculate the candidate threshold T. mWhen pruning, the number of remaining parameters in the model is RN[m].

[0074] RN[m]=k×k×d×nN m

[0075] Step 5: Repeat steps 2 through 4 m times.

[0076] Step 6, calculate the candidate threshold T. m+1 The number of remaining parameters RN[m+1] of the model after pruning and the number of candidate thresholds T. m The difference between the number of remaining parameters RN[m] in the model after pruning.

[0077] ΔRN[m]=RN[m+1]-RN[m]

[0078] Step 7: Determine if the above formula is true. If it is true, save the current value of m and calculate the candidate threshold T. m Obtain the final threshold T = T m If this is not true, repeat step six.

[0079] ΔRN[m]>Value

[0080] Step 8: Set the weights corresponding to gradient values ​​that are less than the final threshold T to 0 to complete the pruning step.

[0081] Generally, the principle of pruning is to retain important weights and remove unimportant weights. m This represents a rectangular frame containing the parts to be filtered out. Based on the filtering criteria in step seven, we can deduce that this frame is symmetrical about 0. And T... m The value changes from small to large, corresponding to the bounding box changing from a black rectangle to a red rectangle. During the traversal of m, the value of each candidate threshold T is recorded. mNext, we need to determine the number of parameters to retain, RN[m]. Since a larger m value results in a larger bounding box area, fewer parameters should be retained. We observe that when m is small, the change in RN[m] is significant, but after m reaches a certain size, RN[m] remains almost unchanged. This is because most gradients are distributed near 0; in other words, the number of parameters is very large near 0, and increasing the bounding box by a single step can cause a significant change in RN[m]. Conversely, in areas with fewer parameters, increasing the bounding box by a single step results in a less noticeable change in RN[m]. Next, we use the formula from step six to subtract the m-th RN[m] value from the (m+1)-th RN[m] value to obtain a reduction ΔRN[m]. We then find the value where the change in ΔRN[m] is relatively small, which is the Value. Note that this Value needs to be manually set according to different network structures. When ΔRN[m] reaches -1000, the RN[m] curve begins to flatten out; therefore, -1000 is a suitable Value for AlexNet. The Value values ​​in VGG-Net and VGG-BN-Net mentioned below are obtained using the same method. If ΔRN[m] > Value, then the value of m at this time is returned. The purpose is to calculate the threshold T at this time using the formula in step two. m That is, we get the final threshold T = T m After obtaining the final threshold T, pruning is performed according to step eight, and finally, retraining is performed to restore the model accuracy.

[0082] Example 2:

[0083] The present invention was used to prune AlexNet, and then the pruned network was retrained 5000 times. The results are shown in Table 1.

[0084]

[0085] This invention demonstrates excellent performance in fully connected layers, achieving a compression ratio of nearly 14 times while only decreasing accuracy by 0.63%. Fully connected layers Fc6 and Fc7 are compressed by 99.9% and 98.0%, respectively. The original network requires 7.1 x 10^6 steps for a single forward inference operation. 7 Each multiplication, requiring only one forward inference after compressing the fully connected layer, results in 5.8 x 10 multiplications. 7 The computational cost of each multiplication only decreased by 20%. As the number of pruned layers increases, the network performance degrades more significantly. When compressing two fully connected layers Fc6 and Fc7, and the convolutional layer conv5, the accuracy decreased by 2.53%, the model size was compressed by a factor of 31, and a single forward inference required 2.9 x 10^6 computations. 7The computational cost of each multiplication was reduced by 60%. This shows that although convolutional layers have a small number of parameters, they have a particularly large computational cost. Compressing convolutional layers can achieve a greater speedup effect, and compressing fully connected layers can further reduce the storage space occupied by the computational model.

[0086] Example 3:

[0087] VGG-Net pruning on CIFAR-10.

[0088] The standard VGG-Net model is used on the ImageNet dataset, with input image sizes of 128*128, while CIFAR-10 images are 32*32. To avoid overfitting due to excessive parameters, the VGG-Net structure is set as described above. After 50,000 training iterations, the test set accuracy is 84%, the model size is 63MB, and the value is set to -1000. The results of pruning the conv4-2, conv4-3, conv5-2, conv5-3, Fc14, and Fc1 layers of the VGG-Net model using this invention show that both convolutional and fully connected layers can achieve over 98% parameter reduction using the AFTP algorithm, while still maintaining a significant improvement in network performance. After approximately 20,000 retraining iterations, the highest accuracy reaches 90.6%, the model size is 21MB, and the computational cost is reduced by 55.3%. Before pruning, the test set accuracy was already relatively stable at around 83%, making further improvement difficult, but after pruning, the accuracy improved significantly. Because the retained weights are all important weights—those with large gradients that significantly impact 1OS—the network performance changes slightly more with each training iteration after weight updates, but it's still generally higher than the performance of the network before pruning. This is because removing several redundant weights with gradients close to zero effectively prevents overfitting. Those weights with gradients close to zero are like a polynomial function f(x) = C1x. 1 +C2x 2 +..+C3x 3 The more terms with large exponents at the very end of the gradient, the better the function fits the training data, but the lower its generalization ability and the lower accuracy on the test set. This invention can eliminate these redundant gradients, improving network performance while compressing parameters.

[0089] Example 4:

[0090] Pruning VGG-BN-Net on CIFAR-10.

[0091] Compared to VGG-Net, the VGG-BN-Net network architecture adds Batch-Normalization and Dropout layers to accelerate training and improve the network's initial performance. After training, VGG-BN-Net achieves a test set accuracy of 91.17%. Following the method described above for obtaining the value, a value of -10000 is selected for fully connected layers, and a value of -20000 is selected for pruning. After retraining, the accuracy decreases to 91.03% (a decrease of 0.14%), the computational cost decreases by 35.3%, and the number of parameters is only 29.3% of the previous amount. This demonstrates that the present invention has a better pruning effect.

[0092] Example 5:

[0093] Please see Figure 4 As shown, the present invention also provides an electronic device 100 for a method of compressing a deep neural network model; the electronic device 100 includes a memory 101, at least one processor 102, a computer program 103 stored in the memory 101 and executable on the at least one processor 102, and at least one communication bus 104.

[0094] The memory 101 can be used to store the computer program 103. The processor 102 implements the steps of the compression method for a deep neural network model described in Embodiment 1 by running or executing the computer program stored in the memory 101 and calling the data stored in the memory 101. The memory 101 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device 100 (such as audio data), etc. In addition, the memory 101 may include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0095] The at least one processor 102 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 102 may be a microprocessor or any conventional processor. The processor 102 is the control center of the electronic device 100, connecting various parts of the electronic device 100 via various interfaces and lines.

[0096] The memory 101 in the electronic device 100 stores multiple instructions to implement a compression method for a deep neural network model, and the processor 102 can execute the multiple instructions to achieve the following:

[0097] Obtain the dataset of the fully connected layer in the deep neural network model, and get the weight matrix of the fully connected layer dataset;

[0098] A gradient-based pruning method is used to process the weight matrix of the fully connected layer dataset to obtain the pruned weight matrix of the fully connected layer dataset.

[0099] The weight matrix of the pruned fully connected layer dataset is restored to the pruned fully connected layer dataset;

[0100] The pruned fully connected layer dataset is fed into the deep neural network model for retraining to obtain the final deep neural network model.

[0101] Example 6:

[0102] If the modules / units integrated in the electronic device 100 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0103] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0104] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0105] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0106] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for compressing a deep neural network model, characterized in that, Includes the following steps: Obtain the dataset of the fully connected layer in the deep neural network model, and get the weight matrix of the fully connected layer dataset; A gradient-based pruning method is used to process the weight matrix of the fully connected layer dataset to obtain the pruned weight matrix of the fully connected layer dataset. The specific method is as follows: Based on the weight matrix of the fully connected layer dataset, obtain the gradient matrix of the weight matrix; Obtain the maximum absolute value of the gradient in the gradient matrix. Based on the maximum absolute value of the gradient in the gradient matrix, obtain the step size and candidate threshold. The step size is calculated as follows: in, Step size, This represents the maximum absolute value of the gradient in the gradient matrix. For any of the candidate thresholds; The candidate threshold is calculated as follows: in, The candidate threshold; Count the number of gradients whose gradient values ​​are less than the candidate threshold in the gradient matrix of the weight matrix; The gradient matrix of the weight matrix is ​​pruned based on the number of gradients less than the candidate threshold, thus completing the processing. When pruning the gradient matrix of the weight matrix based on the number of gradients less than the candidate threshold, it is determined whether the number of remaining parameters after pruning is the same between adjacent candidate thresholds. If they are the same, the current candidate threshold is saved as the final threshold; if they are different, pruning continues. The weight matrix of the pruned fully connected layer dataset is restored to the pruned fully connected layer dataset; The pruned fully connected layer dataset is fed into the deep neural network model for retraining to obtain the final deep neural network model.

2. The method for compressing a deep neural network model according to claim 1, characterized in that, The specific methods for pruning are as follows: The pruning process is complete by replacing the weight matrix corresponding to gradient values ​​that are less than the final threshold with 0.

3. The method for compressing a deep neural network model according to claim 1, characterized in that, The gradient matrix of the weight matrix is ​​calculated as follows: in, The gradient matrix of the weight matrix. For missing functions, This is the weight matrix of the dataset for the fully connected layer.

4. A compression system for a deep neural network model, based on the compression method for a deep neural network model as described in claim 1, characterized in that, include: The weight matrix acquisition module is used to obtain the dataset of the fully connected layer in the deep neural network model and obtain the weight matrix of the fully connected layer dataset. The pruning module is used to process the weight matrix of the fully connected layer dataset using a gradient-based pruning method to obtain the pruned weight matrix of the fully connected layer dataset. The restore module is used to restore the weight matrix of the pruned fully connected layer dataset to the pruned fully connected layer dataset. The retraining module is used to feed the pruned fully connected layer dataset into the deep neural network model for retraining, resulting in the final deep neural network model.

5. The compression system for a deep neural network model according to claim 4, characterized in that, The pruning module includes: The gradient matrix acquisition module is used to obtain the gradient matrix of the weight matrix based on the weight matrix of the fully connected layer dataset. The step size and candidate threshold acquisition module is used to obtain the maximum absolute value of the gradient in the gradient matrix, and obtain the step size and candidate threshold based on the maximum absolute value of the gradient in the gradient matrix. The gradient count acquisition module is used to count the number of gradients whose gradient values ​​are less than the candidate threshold in the gradient matrix of the weight matrix; The gradient matrix processing module is used to prune the gradient matrix of the weight matrix based on the number of gradients less than the candidate threshold, thus completing the processing.

6. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the compression method for a deep neural network model according to any one of claims 1 to 3.

7. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the compression method for a deep neural network model according to any one of claims 1 to 3.

Images