Neural network model pruning method and system
By introducing LRP modules and sparse filters into the neural network model, the propagation rules are simplified, solving the computational complexity problem of existing neural network models when deployed on resource-constrained devices, and improving the stability and interpretability of the model.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NAT UNIV OF DEFENSE TECH
- Filing Date
- 2026-04-10
- Publication Date
- 2026-05-08
AI Technical Summary
Existing neural network model pruning methods suffer from high inference latency, increased power consumption and memory bandwidth usage when deployed on resource-constrained devices. Furthermore, existing metric methods are computationally complex and difficult to migrate and scale across different network architectures.
By establishing gating and filtering mechanisms, simplifying propagation rules, replacing the standard layer with LRP linear and nonlinear modules, and adding correlation gating and sparse filters to the residual module, correlation propagation is controlled, reducing computational load and implementation complexity.
It effectively reduces the complexity and computational cost of model pruning, improves the consistency and interpretability of metrics in residual networks, reduces resource requirements, and simplifies the design dependencies of nonlinear layers.
Smart Images
Figure CN121998016A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of model compression, and more particularly to a method and system for pruning neural network models. Background Technology
[0002] With the development of computer science and the continuous improvement of computing power, the scale and structural complexity of deep learning models are constantly increasing. When such models are deployed to resource-constrained edge devices (such as embedded systems, mobile terminals, etc.), problems such as increased inference latency, increased power consumption and memory bandwidth usage, and even insufficient computing power or storage often occur, thus limiting their application in high real-time and high energy efficiency scenarios.
[0003] Existing importance metrics can be broadly categorized into three types: (1) Amplitude-based metrics. These metrics are based on amplitude indicators such as the absolute value of weights and are widely used due to their simplicity and low cost. However, this type of method struggles to characterize the nonlinear relationship between parameters and performance, as well as cross-layer coupling, and is prone to creating a suboptimal trade-off between model compactness and accuracy. (2) First-order gradient-based metrics. These metrics are based on the first-order term of the Taylor expansion of the loss function and utilize gradient information to measure the sensitivity of parameters to the loss, providing a more refined correlation with the objective function. However, they are essentially still a first-order approximation and fail to adequately capture the interactions between different parameters. (3) Second-order Hessian-based metrics. These metrics evaluate the joint effects between parameters using second-order information, where the off-diagonal elements of the Hessian matrix can reflect cross-parameter / cross-layer interactions, typically resulting in better pruning performance. However, this type of method has high computational and storage overhead and does not fully utilize the joint effects, often requiring customized design for specific network architectures, leading to complex implementation and limited portability and scalability.
[0004] In recent years, interpretability-based pruning has gradually gained attention. This approach utilizes interpretability attribution methods to quantify the "relevance / contribution" of network units and performs pruning accordingly. Existing techniques have demonstrated feasibility on small- to medium-sized CNNs; further, the impact of propagation rule selection and hyperparameter settings at different layers on pruning effects has been explored. Nevertheless, this direction remains largely exploratory: compared to mature first- / second-order methods, there is still room for improvement in terms of uniformity and generalizability, and a unified framework that can simultaneously cover CNNs and Transformers is still lacking.
[0005] To address the common problems of existing pruning metrics, how to avoid relying on complex propagation schemes for nonlinear layer design when pruning the model, thereby reducing implementation complexity and computational load, is a technical problem that urgently needs to be solved in this field. Summary of the Invention
[0006] To address the aforementioned technical problems, the purpose of this application is to provide a neural network model pruning method and system. By establishing a gating and filtering mechanism, the propagation rules are significantly simplified, avoiding reliance on complex propagation schemes for nonlinear layer design, and reducing complexity and computational load.
[0007] To achieve the above objectives, this application provides a method and system for pruning neural network models.
[0008] The above-mentioned objective of this application is achieved through the following technical solution: A neural network model pruning method, comprising: Obtain the first neural network model; The standard layers in the first neural network model are replaced to obtain the second neural network model; Add a correlation gate to each residual module in the second neural network model to control the propagation of correlation. Add a sparse filter to the output of the linear layer in the second neural network model to remove noise during the propagation of correlation, and finally obtain the third neural network model.
[0009] Preferably, it further includes: Acquire image data; The image data is input into the third neural network model, and pruning is performed based on the calculated component correlation scores to obtain the fourth neural network model.
[0010] Preferably, replacing the standard layer in the first neural network model includes: The first neural network model is modified by rewriting only the computational logic rules of forward propagation and backward propagation, replacing all linear layers with LRP linear modules, and nonlinear layers and normalization layers with LRP nonlinear modules.
[0011] Preferably, all linear layers are replaced with LRP linear modules, specifically: During forward propagation, the LRP linear module performs the same operations as the original linear layer, outputs feature values that are completely consistent with the original model, and caches the input tensor and output tensor. During backpropagation, the LRP linear module calculates a stabilization factor based on the output tensor, performs an element-wise product operation with the input tensor and parameters to generate an output, and simultaneously calculates the correlation score of the parameters.
[0012] Preferably, the nonlinear layer and the normalization layer are replaced with LRP nonlinear modules, specifically: During forward propagation, the LRP nonlinear module does not change the original propagation calculation logic of the first neural network model; During backpropagation, the LRP nonlinear module uses the identity mapping rule to directly pass and output the correlation tensor returned by the successor layer.
[0013] Preferably, a correlation gating mechanism is added to each residual module in the second neural network model to control the propagation of correlation, including: During forward propagation, the identity mapping rule is used for propagation through the aforementioned correlation gating; During backpropagation, the correlation-gated output is a zero tensor with the same shape as the input.
[0014] Preferably, a sparse filter is added to the output of the linear layer in the second neural network model to remove noise during correlation propagation, including: During forward propagation, the sparse filter is used to propagate using the identity mapping rule; During backpropagation, the sparse filter calculates the correlation score of the input correlation tensor according to the feature dimension, retains the preset correlation tensor, and forces the remaining part to be sparsified. The retained correlation tensor is then compensated by an energy compensation coefficient.
[0015] Preferably, the formula for calculating the component correlation score is as follows: ; in, For the j-th sample; N is the total number of samples; Indicates the first The sample at the th The correlation scores generated on each parameter.
[0016] A neural network model pruning system, comprising: The acquisition module is used to acquire the first neural network model; The replacement module is used to replace the standard layers in the first neural network model to obtain a second neural network model; The model pruning module adds correlation gating to each residual module in the second neural network model to control correlation propagation. It also adds sparse filters to the output of the linear layers in the second neural network model to remove noise during correlation propagation, ultimately obtaining the third neural network model.
[0017] Preferably, the acquisition module is further configured to acquire image data; The model pruning module is further configured to: input the image data into the third neural network model, prune the data according to the calculated component correlation scores, and obtain a fourth neural network model.
[0018] This application implements a pre-defined path control strategy during the correlation backpropagation process of the residual structure through a designed correlation gating mechanism: only backbone correlations are allowed to propagate across blocks along the residual path, and truncated functional branch correlations are settled within the block for local contribution evaluation. This strategy effectively reduces the risk of correlation distortion caused by cross-branch mixing, simplifies the propagation link, and enhances the consistency and comparability of metrics in the residual network. Furthermore, by using a sparse filter, sparsity constraints are introduced during layer-by-layer propagation, thereby suppressing noise accumulation and numerical amplification, mitigating the instability caused by deep continuous propagation, ensuring the robustness and interpretability of the attribution distribution, and ultimately reducing implementation complexity and improving stability. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 This is a schematic diagram of a neural network model pruning method in an embodiment of this application; Figure 2 This is a structural diagram of a neural network model pruning system in an embodiment of this application. Detailed Implementation
[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0022] Furthermore, the technical features in the various embodiments or individual embodiments provided in this application can be arbitrarily combined with each other to form a feasible technical solution. Such combination is not constrained by the order of steps and / or the structural composition mode, but must be based on the ability of a person skilled in the art to implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed in this application.
[0023] In the embodiments provided in this application, it should be understood that the disclosed methods and systems can be implemented in other ways. The system embodiments described below are merely illustrative. For example, the division of units and modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or modules can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling communication connection between the various components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, and can be electrical, mechanical or other forms.
[0024] In addition, each functional unit in the various embodiments of this application can be integrated into a single processor, or each unit can be a separate device, or two or more units can be integrated into a single device; each functional unit in the various embodiments of this application can be implemented in hardware or in the form of hardware plus software functional units.
[0025] Those skilled in the art will understand that all or part of the steps of the following method embodiments can be implemented by program instructions and related hardware. The aforementioned program instructions can be stored in a computer-readable storage medium. When the program instructions are executed, they perform the steps of the following method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.
[0026] It should be understood that the use of terms such as "system," "device," "unit," and / or "module" in this application is merely one method of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.
[0027] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "a plurality of" or "several" means two or more, unless otherwise explicitly specified.
[0028] It should be noted that the structures, proportions, sizes, etc., shown in the accompanying drawings of this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed in the specification, and are not intended to limit the conditions under which this application can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size should still fall within the scope of the technical content disclosed in this application, provided that they do not affect the effects and purposes that this application can produce.
[0029] If a flowchart is used in this application, it is used to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.
[0030] It should also be noted that, in this document, terms such as “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase “comprising one…” does not exclude the presence of other identical elements in the article or device that includes the aforementioned element.
[0031] LRP (Layer-wise Relevance Propagation): Layer-wise relevance propagation.
[0032] The implementation method of this application is written in a progressive manner.
[0033] like Figure 1 As shown in the figure, this application provides a neural network model pruning method, including: S1. Obtain the first neural network model; Specifically, taking the ViT-Base model as an example, it includes a patch embedding layer, 12 Transformer encoder modules, and a classification head. Each encoder module contains a multi-head self-attention (MSA) sublayer and a multilayer perceptron (MLP) sublayer, and all contain residual connections.
[0034] S2. Replace the standard layers in the first neural network model to obtain the second neural network model; S3. Add correlation gating to each residual module in the second neural network model to control the propagation of correlation. Add a sparse filter to the output of the linear layer in the second neural network model to remove noise during the propagation of correlation, and finally obtain the third neural network model.
[0035] Specifically, in the ViT-Base model, the residual network structure ( To correct attribution bias in deep networks, a custom relevance gate is inserted at the beginning of the functional path of each residual block—after the data split point in the forward propagation and before the input of the functional module sequence (MSA or MLP). This forces the relevance flow to propagate only along the residual path to shallower layers. This mechanism "truncates" the relevance of the functional path at the module level, evaluating it only based on the module's internal importance, preventing it from being mixed with shallower relevance flows during cross-block propagation. This alleviates the depth bias caused by the superposition of residual accumulation effects and proportional allocation mechanisms, significantly improving the comparability and distinguishability of scores from modules of different depths.
[0036] By designing a correlation gating system to control the data propagation process according to a predetermined strategy, bias and distortion caused by cross-branch accumulation are reduced, correlation propagation is simplified, and the consistency and comparability of metrics in the residual structure are improved. Furthermore, by using a sparse filter to introduce sparsity constraints during the layer-by-layer propagation process, noise accumulation and numerical amplification are suppressed, the instability caused by deep continuous propagation is alleviated, the robustness and interpretability of the attribution distribution are guaranteed, and the implementation complexity is reduced while stability is improved.
[0037] In some embodiments, it also includes: Acquire image data; Specifically, the image data is a subset of the ImageNet training set (50 samples are randomly selected for each category).
[0038] The image data is input into the third neural network model, and pruning is performed based on the calculated component correlation scores to obtain the fourth neural network model.
[0039] Specifically, the prepared image data is input into the reconstructed ViT-Base model. Through forward propagation, the predicted output (Logits) is obtained. The numerical values of the corresponding target category (Target Logits) in the predicted results are extracted and used as the initial correlation signal. During backpropagation, the correlation tensor signal flows backward in the computation graph in the form of gradients. The correlation score of each parameter is automatically calculated by PyTorch's automatic integration component. When pruning, since structured pruning is performed on a component-by-component basis (such as convolution kernels, neurons, and attention heads), the obtained parameter-level scores need to be aggregated into component-level scores, namely component correlation scores. Within each component group, the components are globally sorted according to the calculated component correlation scores, and the components to be pruned are selected for pruning.
[0040] After obtaining the component relevance score, define Given the set of all pruningable component groups in the network, a pruning strategy (i.e., an individual in an EA) is represented as a pruning rate vector. ,in The pruning ratio represents the ViT-Base module type. For a given strategy... Within each component group, based on component relevance score Sort by size from largest to smallest. Based on ratio. Keep the component with the highest score, and mark the component with the lowest score as the set to be pruned. ; To minimize accuracy loss while satisfying compression ratios (e.g., computational cost, number of parameters), the fitness function is defined as the sum of the correlations of the pruned components, calculated as follows: ; The smaller the calculated fitness value, the fewer the components removed by the pruning strategy contain in the model decision-related parts, i.e., the most redundant parts are removed, thus theoretically minimizing the interference with model accuracy. After N (e.g., 20) iterations of the EA algorithm, the resulting pruning strategy p is the globally optimal pruning ratio configuration. The model is then pruned according to this pruning strategy. By uniformly calculating the pruningable components, performance degradation can be minimized while satisfying resource constraints.
[0041] In other embodiments, replacing the standard layers in the first neural network model includes: The first neural network model is modified by rewriting only the computational logic rules of forward propagation and backward propagation, replacing all linear layers with LRP linear modules, and nonlinear layers and normalization layers with LRP nonlinear modules.
[0042] Specifically, the standard layers in the ViT-Base model are replaced. The replaced ViT-Base model does not change the parameter weights, but only modifies the connection method of the computation graph and the behavior of the operators.
[0043] All linear operators in the ViT-Base model (including nn.Linear, nn.Conv2D, matrix multiplication in attention computation, etc.) can be replaced using an automatic differentiation engine based on a deep learning framework (such as PyTorch's autograd), thereby rewriting the computational logic of forward and backward propagation; and nonlinear activation layers (such as GELU, ReLU) and normalization layers (such as LayerNorm) in the ViT-Base model can be replaced.
[0044] In other embodiments, all linear layers are replaced with LRP linear modules, specifically: During forward propagation, the LRP linear module performs the same operations as the original linear layer, outputs feature values that are completely consistent with the original model, and caches the input and output tensors. Specifically, the ViT-Base model performs the same operations as the original standard linear layers (such as nn.Linear, nn.Conv2D, etc.), as follows: ; Where x is the input data; w is the linear layer parameter; b is the bias term.
[0045] This ensures that the output feature values are completely consistent with the original model. During this process, both the input and output tensors are cached.
[0046] During backpropagation, the LRP linear module calculates the stabilization factor based on the output tensor, performs element-wise product operations with the input tensor and parameters to generate the output, and simultaneously calculates the correlation score of the parameters.
[0047] Specifically, during backpropagation, when the correlation tensor returned from the successor layer is received, the conventional derivative chain rule is not followed; instead, a new propagation rule is applied: when the correlation tensor returned from the successor layer is received... At that time, the output tensor of the forward propagation using the buffer is used. The stabilization factor is calculated using the following formula: ; in, To prevent small quantities with a denominator of zero.
[0048] Based on the formula: ; Where: x is the input tensor during forward propagation.
[0049] This allows us to obtain the correlation tensor that needs to be passed to the next layer during backpropagation.
[0050] Simultaneously, calculate the relevance score of the weight parameters: ; And write it into the gradient attribute (.grad) of parameter w.
[0051] This allows the framework to automatically accumulate the relevance score of each parameter to the model decision during backpropagation, instead of the traditional loss gradient.
[0052] In some embodiments, all linear layers are replaced with LRP nonlinear modules, specifically: During forward propagation, the LRP nonlinear module does not change the original propagation calculation logic of the first neural network model; Specifically, the LRP nonlinear module after replacing the ViT-Base model has the same computational logic as the standard module (nn.GELU, nn.LayerNorm, etc.) during the forward propagation stage, thus ensuring that the feature values do not change during forward inference.
[0053] During backpropagation, the LRP nonlinear module uses the identity mapping rule to directly pass and output the correlation tensor returned by the successor layer.
[0054] Specifically, during backpropagation, the identity mapping rule is used, that is, without performing any operations, the correlation tensor returned by the successor layer is directly and losslessly passed to the previous layer as input. ).
[0055] In some embodiments, a correlation gating is added to each residual module in the second neural network model to control the propagation of correlation, including: During forward propagation, the identity mapping rule is used for propagation through correlation gating; During backpropagation, the output is a zero tensor with the same shape as the input, which is obtained by correlation gating.
[0056] Specifically, to correct attribution bias in deep networks, a relevance gate is inserted at the beginning of the functional path of the residual network structure in the ViT-Base model—that is, after the data split point in forward propagation and before the input of the functional module sequence (MSA or MLP). During forward propagation, the gate performs an identity mapping, ensuring that the inference result remains unchanged. During backpropagation, when the relevance tensor passes through this gate along the functional path from the later layers, its value is forced to zero. The backpropagation output is a zero tensor with the same shape as the input, thus forcing the relevance flow to continue propagating only along the residual path to shallower layers. This mechanism effectively eliminates over-attribution caused by functional path feedback loops in deep networks, while preserving the local parameter relevances calculated within the functional modules based on the current layer input.
[0057] In other embodiments, a sparse filter is added to the output of the linear layer in the second neural network model to remove noise during correlation propagation, including: During forward propagation, propagation is performed using an identity mapping rule through a sparse filter; During backpropagation, a sparse filter is used to calculate the correlation score of the input correlation tensor according to the feature dimension. The preset correlation tensor is retained, while the rest is forced to be sparsified. The retained correlation tensor is compensated by an energy compensation coefficient.
[0058] Specifically, a sparse filter is added after the output of the ViT-Base model to perform an identity mapping during forward propagation, that is, directly outputting the received input correlation tensor. This process does not change the value of the feature map, thus ensuring that the forward inference result is completely consistent with the original model. During the backpropagation stage, a "sort-truncation-compensation" operation is performed on the received correlation tensor: The input correlation tensor is sorted by absolute value according to the feature dimension, i.e. the correlation score is calculated. The values with the largest absolute value (e.g., 50%) are selected and retained. The retained values are regarded as effective signals with high signal-to-noise ratio. The remaining values are forced to zero (i.e. sparsification). These removed values are regarded as noise or weak contribution in the propagation process. At the same time, to avoid the loss of the total amount of correlation tensors due to the retained data, it is necessary to compensate for the remaining correlation tensors. The formula for calculating the energy compensation coefficient is: ; in, This is the sum of the correlation scores of all correlation tensors; The sum of the correlation scores of the retained correlation tensors.
[0059] This sparse filter ensures that the linear layer receives and outputs correlation signals with high signal-to-noise ratio and stable values, effectively mitigating attribution ambiguity caused by the accumulation of small noise in deep networks and preventing numerical anomalies such as "zero as denominator".
[0060] In some embodiments, the formula for calculating the component relevance score is as follows: ; in, For the j-th sample; N is the total number of samples; Indicates the first The sample at the th The correlation scores generated on each parameter.
[0061] like Figure 2 As shown, a neural network model pruning system includes: Module 101 is used to acquire the first neural network model; Replacement module 102 is used to replace the standard layers in the first neural network model to obtain the second neural network model; The model pruning module 103 is used to add correlation gating to each residual module in the second neural network model to control the propagation of correlation, and to add sparse filters to the output of the linear layers in the second neural network model to remove noise during the propagation of correlation, and finally obtain the third neural network model.
[0062] In some embodiments, the acquisition module 101 is further configured to acquire image data; The model pruning module 103 is also used to: input image data into the third neural network model, prune according to the calculated component correlation scores, and obtain the fourth neural network model.
[0063] By controlling the data propagation process according to a predetermined strategy through correlation gating, unnecessary accumulation and bias propagation of cross-branch correlations are suppressed from the source, improving the consistency and comparability of measurements in the residual structure. Furthermore, by introducing sparsity constraints during the layer-by-layer propagation process through sparse filters, noise accumulation and numerical amplification are suppressed, alleviating the instability caused by deep continuous propagation and ensuring the robustness and interpretability of the attribution distribution. After pruning the neural model by combining the above mechanisms, the reliance on complex propagation schemes for nonlinear layer design is avoided, reducing the complexity and computational cost of pruning.
[0064] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0065] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0066] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for pruning a neural network model, characterized in that, include: Obtain the first neural network model; The standard layers in the first neural network model are replaced to obtain the second neural network model; Add a correlation gate to each residual module in the second neural network model to control the propagation of correlation. Add a sparse filter to the output of the linear layer in the second neural network model to remove noise during the propagation of correlation, and finally obtain the third neural network model.
2. The neural network model pruning method according to claim 1, characterized in that, Also includes: Acquire image data; The image data is input into the third neural network model, and pruning is performed based on the calculated component correlation scores to obtain the fourth neural network model.
3. The neural network model pruning method according to claim 1, characterized in that, The standard layers in the first neural network model are replaced, including: The first neural network model is modified by rewriting only the computational logic rules of forward propagation and backward propagation, replacing all linear layers with LRP linear modules, and nonlinear layers and normalization layers with LRP nonlinear modules.
4. The neural network model pruning method according to claim 3, characterized in that, Replace all linear layers with LRP linear modules, specifically: During forward propagation, the LRP linear module performs the same operations as the original linear layer, outputs feature values that are completely consistent with the original model, and caches the input tensor and output tensor. During backpropagation, the LRP linear module calculates a stabilization factor based on the output tensor, performs an element-wise product operation with the input tensor and parameters to generate an output, and simultaneously calculates the correlation score of the parameters.
5. The neural network model pruning method according to claim 3, characterized in that, The nonlinear layers and normalization layers are replaced with LRP nonlinear modules, specifically: During forward propagation, the LRP nonlinear module does not change the original propagation calculation logic of the first neural network model; During backpropagation, the LRP nonlinear module uses the identity mapping rule to directly pass and output the correlation tensor returned by the successor layer.
6. The neural network model pruning method according to claim 1, characterized in that, In each residual module of the second neural network model, a correlation gating is added to control the propagation of correlation, including: During forward propagation, the identity mapping rule is used for propagation through the aforementioned correlation gating; During backpropagation, the correlation-gated output is a zero tensor with the same shape as the input.
7. The neural network model pruning method according to claim 1, characterized in that, A sparse filter is added to the output of the linear layer in the second neural network model to remove noise during correlation propagation, including: During forward propagation, the sparse filter is used to propagate using the identity mapping rule; During backpropagation, the sparse filter calculates the correlation score of the input correlation tensor according to the feature dimension, retains the preset correlation tensor, and forces the remaining part to be sparsified. The retained correlation tensor is then compensated by an energy compensation coefficient.
8. The neural network model pruning method according to claim 2, characterized in that, The formula for calculating the component relevance score is as follows: ； in, For the j-th sample; N is the total number of samples; Indicates the first The sample at the th The correlation scores generated on each parameter.
9. A neural network model pruning system, characterized in that, include: The acquisition module is used to acquire the first neural network model; The replacement module is used to replace the standard layers in the first neural network model to obtain a second neural network model; The model pruning module adds correlation gating to each residual module in the second neural network model to control correlation propagation. It also adds sparse filters to the output of the linear layers in the second neural network model to remove noise during correlation propagation, ultimately obtaining the third neural network model.
10. The neural network model pruning system according to claim 9, characterized in that, The acquisition module is further configured to: acquire image data; the model pruning module is further configured to input the image data into the third neural network model, and prune according to the calculated component correlation score to obtain a fourth neural network model.