Neural network global one-time structured pruning method, system, device, and medium

Abstract

The present application relates to a kind of neural network global one-time structured pruning method, system, equipment and medium, wherein, method includes: by single forward propagation, in the neural network to be pruned using calibration dataset and parallelly capturing the input activation tensor of all target layers;According to input activation tensor, the intermediate neuron weight of each target layer is synchronously calculated differential entropy index and amplitude response intensity, and after normalization and fusion, static global importance atlas is formed;According to the importance threshold value determined according to preset pruning rate, based on global importance atlas, one-time generates the index set of all global pruning to be pruned whose mixed importance score is below importance threshold value;Based on index set, the weight matrix of each target layer is executed one-time physical structured pruning.By doing so, the present application generates static global importance atlas by single forward propagation and parallelly capturing activation tensor, and completes no-mask one-time pruning by physical structured pruning.

Description

Technical Field

[0001] This invention relates to the field of data model processing technology, and in particular to a method, system, device and medium for global one-time structured pruning of neural networks. Background Technology

[0002] In the field of neural network model compression, existing methods mostly focus on unstructured pruning. Its randomly distributed sparse pattern leads to low compression efficiency of model resources (it cannot effectively reduce the need for structured storage), and actual inference acceleration heavily relies on the support of sparse computing units provided by dedicated hardware, which greatly limits the universality of engineering deployment.

[0003] For example, existing unstructured neural network pruning methods (such as Wanda) employ a layer-dependent processing flow: first, the importance score of the neurons in the first layer is calculated based on the original input data; after pruning the first layer, forward propagation is immediately performed, using the perturbated intermediate outputs as the input benchmark for subsequent layers. However, this hierarchical processing mechanism leads to a series of technical drawbacks.

[0004] Since pruning is inherently an irreversible information loss process, the approximation error generated by pruning in the first layer is directly injected into subsequent layers through the forward propagation path. When calculating the importance of secondary neurons, the system has to evaluate them based on distorted input activation values, which introduces a systematic bias into the reliability of importance scoring. As the number of processing layers increases, the errors between layers accumulate through this forward dependency, and the input activation data of deeper layers will accumulate the pruning errors of all previous layers, resulting in a progressively deteriorating accuracy in importance assessment.

[0005] The implementation mechanism of this method also brings significant computational resource consumption problems. Since the forward propagation of the entire set of samples needs to be re-executed after each single-layer pruning to update the input of subsequent layers, the required computational cost increases linearly with network depth when dealing with modern neural networks with a large number of layers and complex structures.

[0006] Furthermore, existing methods rely on calibration datasets that are often not aligned with the distribution of the real input data of the network to be pruned when pruning general pre-trained models. For models that have undergone professional fine-tuning, the cost of constructing calibration data is high and its effectiveness is difficult to guarantee. In addition, relying on static activation data obtained in the preprocessing stage as the basis for importance assessment cannot effectively represent the real activation patterns of the model in the dynamic inference process. This results in a fundamental deviation between pruning decisions derived from static data analysis and actual operating scenarios.

[0007] Therefore, existing technologies face multiple technical bottlenecks when dealing with multi-layer neural network pruning tasks, especially when dealing with professionally fine-tuned models (non-pre-trained models), such as error propagation amplification, low computational efficiency, and distorted importance assessment. New solutions are urgently needed to overcome the core constraints of the current methodological system. Summary of the Invention

[0008] (a) Technical problems to be solved

[0009] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a method, system, device and medium for global one-time structured pruning of neural networks. It solves the technical problems of existing pruning techniques, which rely on the combined effect of unstructured sparse patterns and serialized layer-by-layer processing mechanisms, resulting in low model storage compression efficiency, limited hardware acceleration benefits, and inevitably causing error propagation, low computational efficiency and distortion of importance assessment.

[0010] (II) Technical Solution

[0011] To achieve the above objectives, the main technical solutions adopted by the present invention include:

[0012] In a first aspect, embodiments of the present invention provide a method for global one-time structured pruning of neural networks based on a hybrid amplitude entropy importance map, comprising:

[0013] By using a single forward propagation, the input activation tensors of all target layers are captured in parallel in the neural network to be pruned using a calibration dataset.

[0014] Based on the input activation tensor, the differential entropy index and amplitude response intensity are calculated synchronously for the association weight reorganization of intermediate neurons in each target layer. After normalization and fusion, a static global importance map is formed.

[0015] The importance threshold is determined according to the preset pruning rate, and an index set of all global interneurons with scores below the importance threshold is generated at once based on the global importance map.

[0016] Based on the index set, a one-time physical structured pruning is performed on the weight matrix of each target layer.

[0017] Optionally, the input activation tensors of all target layers in the neural network to be pruned are captured in parallel using a calibration dataset through a single forward propagation, including:

[0018] Load the selected neural network to be pruned, and obtain the calibration dataset by random sampling from the original training data;

[0019] The calibration dataset is input into the neural network to be pruned, and a single forward propagation computation is performed.

[0020] During the forward propagation process, each target feedforward network layer and input activation tensor are intercepted in parallel and at one time through pre-implanted hooks, and the input activation tensor is stored iteratively to ensure that only a single storage result after the fusion of the current input activation tensor and the historical input activation tensor is always retained.

[0021] In this context, each word hidden state vector in the input activation tensor is generated based on the corresponding context during the decoding stage and is used to predict the final output word. The word hidden state vector is the smallest operable data unit in the input activation tensor that carries independent semantic context information.

[0022] Optionally, based on the input activation tensor, the differential entropy index and amplitude response intensity are simultaneously calculated for the association weight reorganization of intermediate neurons in each target layer. After normalization and fusion, a static global importance map is formed, including:

[0023] Configure the association weight reassembly of each intermediate neuron in the target feedforward network layer. The association weight reassembly includes a specified row of the gated projection layer weight matrix, a specified row of the up projection layer weight matrix, and a specified column of the down projection layer weight matrix.

[0024] Based on the input activation tensor, calculate the product matrix of each weight in the association weight reorganization and the activation response intensity index of the corresponding feature channel, and perform aggregation operation according to the input or output dimension to generate amplitude response intensity score.

[0025] In the downcast layer, the intermediate activation tensor generated by the modulation of gated projection and upcast output is reconstructed in two dimensions and the feature channels are separated. The differential entropy index of each intermediate neuron is calculated by kernel density estimation and numerical integration.

[0026] The amplitude response intensity fraction and the differential entropy index are normalized respectively;

[0027] The normalized amplitude result is weighted and fused with the differential entropy result to generate a static global importance map representing the undisturbed intrinsic information content.

[0028] Optionally, based on the input activation tensor, the product matrix of each weight in the association weight reorganization and the activation response intensity index of the corresponding feature channel is calculated, and aggregation is performed according to the input or output dimension to generate the amplitude response intensity score, including:

[0029] In the gated projection layer, locate the weight matrix row corresponding to the target neuron, traverse all the weight elements corresponding to the input channels in the weight matrix row, multiply the absolute value of each weight element by the L2 norm of the feature channel corresponding to the input activation tensor, and sum them along the direction of the input feature channel to obtain the importance score of the gated row.

[0030] In the up-projection layer, locate the row of the weight matrix corresponding to the target neuron, traverse all weight elements corresponding to all input channels in the row of the weight matrix, multiply the absolute value of each weight element by the L2 norm of the feature channel corresponding to the input activation tensor, and sum them along the direction of the input feature channel to obtain the importance score of the up-row;

[0031] In the downcast projection layer, the weight matrix column corresponding to the target neuron is located. The weight elements corresponding to all output channels in the weight matrix column are traversed. The absolute value of each weight element is multiplied by the L2 norm of the feature channel corresponding to the intermediate activation tensor. The summation is carried out along the output feature channel to obtain the importance score of the downcast column. The intermediate activation tensor is obtained by performing a Hadamard product operation between the output of the gated projection layer and the output of the upcast projection layer after the SiLU function transformation.

[0032] The gating row importance score, the up row importance score, and the down column importance score are added together to generate the amplitude response strength score.

[0033] Optionally, in the downcast projection layer, the intermediate activation tensor generated based on the modulation of gated projection and upcast projection outputs is reconstructed in two dimensions and its feature channels are separated. The differential entropy index of each intermediate neuron is calculated by kernel density estimation and numerical integration, including:

[0034] In the downlink projection layer, the intermediate activation tensor, which is generated in three-dimensional form based on the modulation of gated projection and uplink projection output, is reconstructed into a two-dimensional matrix with independent observation samples as rows and feature dimensions as columns along the batch dimension and sequence dimension.

[0035] Extract the set of activation values ​​corresponding to each intermediate neuron in the two-dimensional matrix column by column;

[0036] A kernel function is applied to the set of activation values ​​for each intermediate neuron to smooth the data and construct a continuous probability density function that characterizes the distribution of activation values ​​of the neuron.

[0037] Based on the continuous probability density function, the log-probability weighted integral of the activation value distribution is calculated within a preset numerical integration interval, and the differential entropy index is approximated by numerical integration.

[0038] Optionally, an importance threshold is determined according to a preset pruning rate, and an index set of all global interneurons to be pruned with scores below the importance threshold is generated at once based on the global importance map, including:

[0039] Based on the preset network pruning ratio, percentile statistics are performed on the global graph containing the normalized importance scores of all neurons to determine the corresponding global importance threshold.

[0040] The global importance map is scanned element by element, and the numerical relationship between the importance score of each neuron and the global importance threshold is compared in real time.

[0041] The location information of neurons whose importance scores are consistently below the global importance threshold is synchronously recorded across all target feedforward network layers into a unified pruning index container, forming a global index set of intermediate neurons to be pruned.

[0042] Optionally, based on the index set, performing a one-time physical structured pruning on the weight matrix of each target layer includes:

[0043] Search for the target feedforward network layer containing the interneurons to be pruned along the network depth direction layer by layer to generate a queue of layers to be processed.

[0044] Create a transactional temporary mirror copy of the weight matrix for each target feedforward network layer in the queue of layers to be processed, and establish a backup of the original weight state;

[0045] For the queue of layers to be processed, based on the coordinate information of the intermediate neurons to be pruned recorded in the index set, the input dimension of the weight matrices of the gated projection layer and the up-projection layer is reduced, and the row vectors of the weight matrix corresponding to the intermediate neurons to be pruned are deleted.

[0046] Simultaneously, output dimension reduction is performed on the weight matrix of the downprojection layer, and column vectors of the weight matrix corresponding to the intermediate neurons to be pruned are deleted.

[0047] Based on the backup of the original weight state, a preset inter-layer connection dimension matching verification is performed on all target feedforward network layers that have completed the reduction operation.

[0048] If the dimension matching verification passes, the reduced weight matrix will be atomically replaced in batches to the corresponding network layers;

[0049] If the dimension matching verification fails, the transaction rollback mechanism is automatically triggered to restore the weight matrix of all target feedforward network layers to the original backup state.

[0050] After performing a one-time structured row and / or column dimension deletion on all target feedforward network layers, a compressed model without masking operations is generated.

[0051] Secondly, embodiments of the present invention provide a global one-time structured pruning system for neural networks based on a hybrid amplitude entropy importance map, comprising:

[0052] The tensor capture module is used to capture the input activation tensors of all target layers in parallel using a calibration dataset in a single forward propagation in the neural network to be pruned.

[0053] The map output module is used to simultaneously calculate the differential entropy index and amplitude response intensity of the association weight reorganization of the intermediate neurons of each target layer based on the input activation tensor, and form a static global importance map after normalization and fusion.

[0054] The neuron selection module is used to determine the importance threshold according to the preset pruning rate and generate an index set of all global intermediate neurons with scores lower than the importance threshold in one go based on the global importance map;

[0055] The structured pruning module is used to perform one-time physical structured pruning on the weight matrix of each target layer based on the index set.

[0056] Thirdly, embodiments of the present invention provide a global one-time structured pruning device for neural networks based on a hybrid amplitude entropy importance map, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute the global one-time structured pruning method for neural networks based on a hybrid amplitude entropy importance map as described above.

[0057] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the global one-time structured pruning method for neural networks based on a hybrid amplitude entropy importance graph as described above.

[0058] (III) Beneficial Effects

[0059] The beneficial effects of this invention are as follows: First, by capturing the static input activation tensors of all target layers in parallel within a single forward propagation in the neural network to be pruned, it overcomes the limitations of traditional layer-by-layer dynamic serial capture, eliminates the propagation and accumulation of errors in multi-layer networks, and simultaneously reduces the magnitude of computational resource consumption and improves the utilization rate of hardware execution units. Based on this, a multi-dimensional fusion evaluation strategy using differential entropy and amplitude response intensity is innovatively adopted to achieve cross-layer joint analysis of the weight groups associated with intermediate neurons. Structured correlation modeling eliminates memory access fragmentation caused by unstructured sparse patterns, resulting in a static global importance map with spatial consistency. This ensures that importance scoring is no longer limited by activation fluctuations at local levels, effectively avoiding the problem of misjudgment accumulation caused by inconsistent importance evaluation standards between layers in traditional methods.

[0060] Furthermore, by directly mapping the normalized score distribution of the static global importance graph to a preset pruning rate, the screening thresholds for all intermediate neurons to be pruned are determined at once, avoiding the risk of evaluation fluctuations during the iterative optimization process, and achieving global optimality of pruning decisions while ensuring hardware friendliness.

[0061] Ultimately, based on the execution mechanism of one-time physical structured pruning, the row and column dimensions of redundant parameters are directly removed instead of relying on mask preservation. This not only eliminates the additional overhead introduced by mask calculation in traditional pruning methods, but also achieves a dual improvement in storage space and computational efficiency through the physical reconstruction of the weight matrix. This allows the compressed model to be deployed directly without additional dimension calibration or mask compensation mechanisms, thereby avoiding the problem of instantaneous model failure caused by step-by-step pruning in traditional methods.

[0062] Therefore, this invention eliminates the need for costly iterations to observe minute perturbations after pruning. It abandons the traditional linear evaluation paradigm based on activation amplitude and uses entropy-driven high-dimensional spatial structured representation. By taking a global "snapshot" on the neural network to be pruned, it can identify and prune the neurons that should be removed and are least functionally important. Attached Figure Description

[0063] Figure 1 A flowchart illustrating the method provided in an embodiment of the present invention;

[0064] Figure 2 This is a schematic diagram illustrating the specific process of step S1 of the method provided in this embodiment of the invention;

[0065] Figure 3 This is a detailed flowchart illustrating step S2 of the method provided in this embodiment of the invention;

[0066] Figure 4 This is a detailed flowchart illustrating step S22 of the method provided in this embodiment of the invention;

[0067] Figure 5 This is a detailed flowchart illustrating step S23 of the method provided in this embodiment of the invention;

[0068] Figure 6 This is a detailed flowchart illustrating step S3 of the method provided in this embodiment of the invention;

[0069] Figure 7 This is a detailed flowchart illustrating step S4 of the method provided in this embodiment of the invention;

[0070] Figure 8 A schematic diagram showing the comparison of the variance of activation values ​​of neurons to be pruned under traditional pruning schemes and the pruning scheme of the present invention, provided in an embodiment of the present invention.

[0071] Figure 9 This is a schematic diagram showing the comparison of the variance of activation values ​​of preserved neurons under the pruning scheme of the present invention, provided in an embodiment of the invention.

[0072] Figure 10 A comparison of the real-time performance accuracy of pruning provided in the embodiments of the present invention;

[0073] Figure 11 A comparison of the accuracy of pruning recovery performance provided in the embodiments of the present invention;

[0074] Figure 12 This is a schematic diagram of the joint distribution of pruning channel amplitude and entropy provided in an embodiment of the present invention. Detailed Implementation

[0075] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0076] Before proceeding, to facilitate understanding of the technical solution provided by this invention, some concepts will be introduced below:

[0077] FFN: Feed-Forward Network;

[0078] Hooks: Hooks are a mechanism that intercepts and records intermediate results during the forward / backward propagation of a neural network.

[0079] gate_proj: gated projection layer;

[0080] up_proj: Upward projection layer;

[0081] down_proj: Downward projection layer;

[0082] A down_in Input activation tensor;

[0083] PyTorch Hook: A callback mechanism that allows capturing or modifying intermediate layer data during model forward / backward propagation, often used to obtain parameters or gradients in real time;

[0084] mlp.downproj: A linear projection layer in the Transformer feedforward network responsible for compressing the high-dimensional gated activations (gate_proj ⊥ up_proj results) back to the original model dimensions;

[0085] KDE: Kernel Density Estimation;

[0086] Language_model: The language model module for large language models (especially multimodal models);

[0087] batch_size(B): The number of samples in one training step;

[0088] seq_len(L): The length (number) of the sequence used to generate tokens;

[0089] Prefill: The first step in language model inference, which only processes the input content;

[0090] Generate: The generation stage, the autoregressive decoding stage of the language model, processes and generates tokens;

[0091] Token: Lexicon, the smallest linguistic structural unit that a language model can process;

[0092] InternVL3: A multimodal large-scale model of the scholar's myriad aspects, developed by the Shanghai Artificial Intelligence Laboratory;

[0093] Qwen3: Tongyi Qianwen Big Language Model, developed by Alibaba.

[0094] like Figure 1 As shown in the embodiment of the present invention, a one-time global structured pruning method for neural networks based on a hybrid amplitude entropy importance map is proposed, comprising: capturing the input activation tensors of all target layers in parallel using a calibration dataset in the neural network to be pruned through a single forward propagation; calculating the differential entropy index and amplitude response intensity simultaneously by recombining the association weights of the intermediate neurons of each target layer according to the input activation tensors, and forming a static global importance map after normalization and fusion; determining the importance threshold according to a preset pruning rate, and generating an index set of all global intermediate neurons to be pruned with scores lower than the importance threshold based on the global importance map; and performing one-time physical structured pruning on the weight matrix of each target layer based on the index set.

[0095] First, this invention achieves parallel capture of the static input activation tensors of all target layers in a pruned neural network through a single forward propagation, overcoming the limitations of traditional layer-by-layer dynamic serial capture. This eliminates the accumulation of errors propagating across multiple layers, simultaneously reducing computational resource consumption and improving hardware execution unit utilization. Building upon this, a multi-dimensional fusion evaluation strategy using differential entropy and amplitude response intensity is innovatively employed to achieve cross-layer joint analysis of the weight groups associated with intermediate neurons. Structured correlation modeling eliminates memory access fragmentation caused by unstructured sparse patterns, resulting in a static global importance map with spatial consistency. This ensures that importance scoring is no longer limited by activation fluctuations at local levels, effectively avoiding the accumulation of misjudgments caused by inconsistent importance evaluation standards between layers in traditional methods.

[0096] Furthermore, by directly mapping the normalized score distribution of the static global importance graph to a preset pruning rate, the screening thresholds for all intermediate neurons to be pruned are determined at once, avoiding the risk of evaluation fluctuations during the iterative optimization process, and achieving global optimality of pruning decisions while ensuring hardware friendliness.

[0097] Ultimately, based on the execution mechanism of one-time physical structured pruning, the row and column dimensions of redundant parameters are directly removed instead of relying on mask preservation. This not only eliminates the additional overhead introduced by mask calculation in traditional pruning methods, but also achieves a dual improvement in storage space and computational efficiency through the physical reconstruction of the weight matrix. This allows the compressed model to be deployed directly without additional dimension calibration or mask compensation mechanisms, thereby avoiding the problem of instantaneous model failure caused by step-by-step pruning in traditional methods.

[0098] Therefore, this invention eliminates the need for costly iterations to observe minute perturbations after pruning. It abandons the traditional linear evaluation paradigm based on activation amplitude and uses entropy-driven high-dimensional spatial structured representation. By taking a global "snapshot" on the neural network to be pruned, it can identify and prune the neurons that should be removed and are least functionally important.

[0099] Therefore, this invention eliminates the need for costly iterations to observe minute perturbations after pruning, abandoning the traditional linear evaluation paradigm based on activation amplitude. By introducing a high-dimensional perspective analysis mechanism based on entropy information, it can identify and prune the least important neurons that should be removed by taking a single global "snapshot" on the neural network to be pruned.

[0100] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and that the scope of the present invention can be fully conveyed to those skilled in the art.

[0101] Specifically, embodiments of the present invention provide a global one-time structured pruning method for neural networks based on a hybrid amplitude entropy importance map, comprising:

[0102] S1. Through a single forward propagation, the input activation tensors of all target layers are captured in parallel using the calibration dataset in the neural network to be pruned.

[0103] Furthermore, such as Figure 2 As shown, step S1 includes:

[0104] S11. Load the selected neural network to be pruned and obtain the calibration dataset by randomly sampling from the original training data. The neural network to be pruned is a pre-trained model, such as the InternVL series, Qwen series, or Llama series network models. The calibration dataset uses a data size of 30-128 samples.

[0105] S12. Input the calibration dataset into the neural network to be pruned and perform a single complete forward propagation calculation.

[0106] S13. During the forward propagation, pre-implanted hooks are used to intercept each target feedforward network layer (as the target layer) and its input activation tensor in parallel and all at once. The input activation tensor is then iteratively stored to ensure that only a single storage result, combining the current input activation tensor with historical input activation tensors, is always retained. During the forward propagation, hooks are set up to capture all target FFN layers (e.g., down_proj) and input activation tensors (e.g., A) in parallel and all at once. down_in The captured activation tensors are stored iteratively, and each time a data is computed, it is combined with the result of the previous one, so that only one tensor is always retained.

[0107] It is important to understand that each word hidden state vector in the input activation tensor is generated based on the corresponding context during the decoding stage and is used to predict the final output word. The word hidden state vector is the smallest operable data unit in the input activation tensor that carries independent semantic context information.

[0108] S2. Based on the input activation tensor, the differential entropy index and amplitude response intensity are calculated synchronously for the association weight reorganization of the intermediate neurons of each target layer. After normalization and fusion, a static global importance map is formed.

[0109] In this step, for each captured activation tensor, the entropy score and magnitude score of each intermediate feature channel are calculated independently and in parallel. Entropy modeling involves data reshaping, KDE probability density estimation, and numerical integration, ultimately generating an entropy importance score for all neurons in the model to be pruned. Then, all calculated entropy and magnitude scores are aggregated to form a static, global importance map. This map records the intrinsic information content of each target neuron in the model in its original, undisturbed state.

[0110] Furthermore, such as Figure 3 As shown, step S2 includes:

[0111] S21. Configure the association weight reassembly for each intermediate neuron in the target feedforward network layer. This reassembly includes a specified row of the gated projection layer weight matrix, a specified row of the up-projection layer weight matrix, and a specified column of the down-projection layer weight matrix. This configuration is based on the following considerations: the feature representation of the i-th intermediate neuron can be divided into two stages. In the expansion stage, it is jointly represented by the i-th row of the up-projection matrix and the gated projection matrix; while in the contraction stage, it is represented by the i-th column of the down-projection matrix. Therefore, the pruning operation for a single neuron requires simultaneous execution of three operations—removing the i-th row of the up-projection matrix and the gated projection matrix, and removing the i-th column of the down-projection matrix—to ensure dimensional consistency of the data flow.

[0112] S22. Based on the input activation tensor, calculate the product matrix of each weight in the association weight reorganization and the activation response intensity index of the corresponding feature channel, and perform aggregation operation according to the input or output dimension to generate amplitude response intensity scores. The activation response intensity index is selected from, but is not limited to, statistical measures such as L2 norm, absolute mean, or standard deviation.

[0113] Furthermore, such as Figure 4 As shown, step S22 includes:

[0114] S221. Locate the weight matrix row corresponding to the target neuron in the gated projection layer, traverse all weight elements corresponding to all input channels in the weight matrix row, multiply the absolute value of each weight element by the L2 norm of the feature channel corresponding to the input activation tensor, and sum them along the direction of the input feature channel to obtain the importance score of the gated row.

[0115] S222. Locate the weight matrix row corresponding to the target neuron in the up-projection layer, traverse all weight elements corresponding to all input channels in the weight matrix row, multiply the absolute value of each weight element by the L2 norm of the feature channel corresponding to the input activation tensor, and sum them along the direction of the input feature channel to obtain the importance score of the up-row.

[0116] S223. Locate the weight matrix column corresponding to the target neuron in the downcast projection layer, traverse all weight elements corresponding to all output channels in the weight matrix column, multiply the absolute value of each weight element by the L2 norm of the feature channel corresponding to the intermediate activation tensor, and sum them along the direction of the output feature channel to obtain the importance score of the downcast column; wherein, the intermediate activation tensor is obtained by performing a Hadamard product operation between the output of the gated projection layer and the output of the upcast projection layer after the SiLU activation function transformation.

[0117] S224. Add the gating row importance score, the up row importance score, and the down column importance score to generate the amplitude response strength score.

[0118] In one embodiment, the goal is to compute an importance score I for each intermediate neuron in the FFN. j This is to enable structured pruning. Pruning the j-th intermediate neuron means removing a set of weights associated with it, thus maintaining the integrity of matrix operations.

[0119] The goal of this invention is to remove entire neurons that process intermediate features, which corresponds to entire rows or columns in a matrix. Therefore, individual weights cannot be evaluated independently. A single importance score must be calculated for a weighted group. This "group" contains all weights associated with the same intermediate neuron. Failure to do so, for example, by independently pruning the i-th row of the up-projection layer `up_proj` and the k-th column of the down-projection layer `down_proj`, disrupts the computational flow of the FFN, leading to structural misalignment.

[0120] For the j-th intermediate neuron (j is the index of the intermediate layer neuron, j∈{1,...,d}) ffn}), and its corresponding weight reassembly includes: the gate_proj weight matrix of the gated projection layer. The j-th row, d model Let d be the input and output feature dimensions. ffn The intermediate feature dimension; the weight matrix of the up-projection layer up_proj. The j-th row; and the down_proj weight matrix of the down projection layer. The j-th column.

[0121] And, the definitions of the underlying components upon which subsequent calculations depend:

[0122] Input to activate: B is the number of samples, and L is the number of tokens generated;

[0123] gate_proj weight: ;

[0124] up_proj weight: ;

[0125] down_proj weight: ;

[0126] To obtain the amplitude response intensity score, the importance of up_proj and gate_proj is first evaluated. For W... u and W g To evaluate the importance of a row, all weights in the j-th row collectively determine the output of the j-th intermediate neuron. Specifically, evaluating a row involves the following steps:

[0127] (1) Calculate the input activation norm: These weights act on the input X. Calculate each input feature (d...model The L2 norm of the (batch and sequence dimensions) is used to aggregate information from both the batch and sequence dimensions.

[0128]

[0129] In the formula, the norm vector N in The overall activation intensity of each output channel is measured. X:,:,k represents the sum of the activation values ​​of channel k under all sample token generation (overall activation intensity).

[0130] (2) Calculate the importance matrix of the weights: First, calculate a temporary importance matrix, where each element is the product of the weight size and the corresponding input activation norm.

[0131] ;

[0132] These two temporary matrices M u M g All shapes are .

[0133] (3) Aggregation to obtain row importance: In order to obtain a single score for the j-th row (i.e., the j-th neuron), it is necessary to cross columns (input dimension d). model This is aggregated. This represents the average response magnitude of all outputs from neurons in the gate_proj and up_proj layers.

[0134] ;

[0135] In the formula, I u [j] represents the total importance score of neurons in the ascending projection layer, I g [j] represents the total importance score of neurons in the gated projection layer. It is worth emphasizing that, given a fixed input dimension, the perturbation of neuron importance ranking by summation and averaging is merely linear scaling, without changing the relative order. This property allows for flexible selection of computational methods in pruning algorithm design without worrying about affecting the final pruning result.

[0136] Secondly, the importance of down_proj was evaluated, and for W... d To evaluate the importance of a column, since the j-th column receives input from the j-th interneuron, the evaluation process involves the following steps:

[0137] (1) Calculate the input activation and norm of down_proj: the input A of down_proj down_in It is the Hadamard product of the output activations of the gated projection layer and the output activations of the uplink projection layer. (Alternatively, the input matrix of the mlp.downproj layer can be obtained directly via a PyTorch Hook).

[0138] ;

[0139] In the formula, SiLU is the activation function and T is the transpose.

[0140] (2) Calculate each intermediate feature (d) ffn The L2 norm of the vector N (dimension N). down_in The overall output activation intensity of each interneuron was measured.

[0141] ;

[0142] In the formula, A down_in:,:,j This represents the overall activation strength of channel j in the down_proj layer across all samples that generate tokens.

[0143] (3) Calculate the importance matrix of the down-projection layer:

[0144] ;

[0145] This matrix M d The shape is .

[0146] (4) Aggregation to obtain column importance: In order to obtain a single score for the j-th column (corresponding to the j-th neuron), it is necessary to span across rows (output dimension d). model Aggregate.

[0147] ;

[0148] In this way, I u I g and I d The dimensions and physical meanings are perfectly aligned: their j-th elements all represent the importance of the j-th interneuron. Ultimately, the total importance score of the j-th interneuron is the sum of the contributions from the three parts:

[0149] ;

[0150] In vector form:

[0151] ;

[0152] Furthermore, since entropy needs to be incorporated for overall importance assessment later, normalization must be performed separately to ensure that the assessment of a neuron is holistic and functionally balanced. Neurons that consistently play a vital role throughout the entire lifecycle of the information flow are rewarded, while neurons that may generate significant activation only at a particular stage but ultimately prove ineffective are pruned.

[0153] ;

[0154] In the formula, Norm represents min-max normalization.

[0155] This final vector I final It can be used for all d ffn The intermediate neurons are sorted, and the neuron with the lowest score is pruned. Assuming the language_model module of this network has a total of k layers, a magnitude importance matrix can be obtained:

[0156] .

[0157] In the formula, I mag [i, j] represents the amplitude response intensity score of the j-th neuron in the i-th layer of the FFN layer of the language_model.

[0158] S23. In the downcast projection layer, the intermediate activation tensor generated based on the modulation of gated projection and upcast projection output is reconstructed in two dimensions and the feature channels are separated. The differential entropy index of each intermediate neuron is calculated by kernel density estimation and numerical integration.

[0159] Furthermore, such as Figure 5 As shown, step S23 includes:

[0160] S231. In the downlink projection layer, the intermediate activation tensor in three-dimensional form generated based on the modulation of gated projection and uplink projection output is reconstructed along the batch dimension and sequence dimension into a two-dimensional matrix with independent observation samples as rows and feature dimensions as columns.

[0161] S232. Extract the set of activation values ​​corresponding to each intermediate neuron in the two-dimensional matrix column by column.

[0162] S233. Perform kernel function smoothing on the set of activation values ​​of each intermediate neuron to construct a continuous probability density function that characterizes the distribution of activation values ​​of the neuron.

[0163] S234. Based on the continuous probability density function, calculate the log-probability weighted integral of the activation value distribution within the preset numerical integration interval, and approximate the difference entropy index through numerical integration.

[0164] S24. Normalize the amplitude response intensity fraction and the differential entropy index respectively.

[0165] S25. The normalized amplitude result and the differential entropy result are weighted and fused to generate a static global importance map representing the undisturbed intrinsic information content and stored persistently.

[0166] It is important to clarify that this "intrinsic information content" is not a single indicator, but a comprehensive score obtained by weighted fusion of features from two orthogonal dimensions. Signal strength is calculated by normalizing the norm of the neuron's output weights to characterize the neuron's influence in the forward propagation of network information. Activation diversity is calculated by distributing the neuron's activation values ​​on a set of calibration datasets to characterize the richness of the neuron's information encoding across different contexts.

[0167] In another embodiment, the importance of the information entropy metric is determined by assigning an importance score to each intermediate neuron based on the amount of information it transmits. The core idea is that a neuron with a very stable and predictable activation value distribution (low entropy) carries less information and may be redundant; while a neuron with a wide activation value distribution and high uncertainty (high entropy) carries more information and is therefore more important.

[0168] Unlike magnitude methods that evaluate the product of weights and activation values, the entropy method directly models the distribution of activation values. Its evaluation focuses solely on the input activation of the down_proj layer; this choice is crucial because the activation tensor A... down_in It is the final result produced by the combined action of `gate_proj` and `up_proj`, representing the complete information flow through the FFN layer. Modeling this information bottleneck allows for the most direct evaluation of the combined information contribution of each intermediate channel.

[0169] The input to down_proj is the final convergence point of the information flows from up_proj and gate_proj, and the uncertainty of its distribution best represents the information characteristics after processing by the entire FFN layer. This avoids unreasonable mathematical combinations of indicators with different distributions.

[0170] Specifically, determining the importance of information entropy measurement includes:

[0171] (1) Data Reshaping: In order to model the activation value distribution of each intermediate feature channel, the three-dimensional activation tensor A is reshaped. down_in Reshape into a two-dimensional matrix:

[0172] ;

[0173] In this matrix, rows ( ): Represents the sample dimension. Each row is an independent observation sample. B is the number of samples, L is the number of generated tokens, and only the activation values ​​during the decoding stage are stored. Column (d ffn ): Represents the feature dimension. Each column contains the activation values ​​of a single intermediate neuron across all samples.

[0174] (2) Separate feature channels: process matrix A′ column by column. down_in .make For A′ down_in The j-th column. This vector C. j That is, the j-th interneuron is in all ( Generate a set of activation values ​​on the token, with the goal of calculating the entropy of the distribution represented by this vector.

[0175] (3) Probability density estimation: For each activation vector C j First, we need to estimate the continuous probability density function (PDF) behind these discrete sample points, denoted as . Assuming the activation vector C follows a true distribution p, we can model the activation vector C using KDE probability density estimation to obtain an estimated distribution, which is used to approximate the true distribution. x That is, a random variable in the activation vector C.

[0176] Each sample is decoded k times to obtain a total of B×k activation samples. Experiments show that selecting only the first 32-64 activation samples of the decoded sequence is sufficient to achieve optimal modeling results. Traditional histogram methods are not robust enough and are limited by bin selection. Therefore, kernel density estimation (KDE) is adopted, which is a more advanced nonparametric method that can generate smooth density curves, especially suitable for cases with small sample sizes. The mathematical form of KDE is:

[0177] ;

[0178] In the formula, K is the kernel function (a Gaussian kernel is used in this embodiment of the invention), h is the bandwidth hyperparameter, and C ji This represents the i-th sample value of the j-th activation vector.

[0179] (4) Calculate the difference entropy: With a continuous probability density function Then, its difference entropy H(C) can be calculated. j It quantifies the uncertainty of the distribution:

[0180] ;

[0181] And we approximate this infinite integral with the sum of a finite interval.

[0182] ;

[0183] In the formula, KDE creates a continuous function. For numerical computation, it needs to be discretized. We can first collect the maximum and minimum values ​​of the original activation vector to obtain a sampling interval S, and then set the required number of sampling points N. Then Δx is the sampling interval S / N. A smaller sampling interval results in a more accurate estimate, but the computational cost increases accordingly. The number of sampling points N is a hyperparameter; an empirical value of 100 is chosen.

[0184] The entropy value calculated for each interneuron is used as its final importance score.

[0185] ;

[0186] Assuming the language_model module of this network has a total of k layers, we can obtain an entropy-aware importance matrix:

[0187] ;

[0188] Based on the above calculation formulas and parameters, the calculated I ent and execution and I mag Vectors are normalized in the same way (e.g., converted to the 0-1 range or rank) to eliminate scale differences.

[0189] By combining the two weighted averages, the final importance score vector is obtained:

[0190] ;

[0191] In the formula, α is a pre-defined weight parameter, α∈[0,1].

[0192] Ultimately, the cross-layer I model They are aggregated into a global static graph, providing a unified assessment of neuron importance across the model.

[0193] It is important to emphasize that the composition of the sample is crucial to ensuring the accuracy of the entropy model.

[0194] The first, correct approach (modeling the activation value distribution during the generation phase): Samples should consist of activation values ​​from different independent sequences. For example, during the decoding phase, multiple different prompts are collected to generate the token's activation values ​​during the generation phase. That is, vector C j Each element in the dataset comes from a distinct input context p(token|distinct_context). This ensures that the samples are approximately independent and identically distributed, thus allowing the estimation to... It can accurately reflect the general activation behavior pattern of the j-th neuron under various conditions.

[0195] The second incorrect approach (modeling the activation value distribution during the pre-filling stage): If activation values ​​are collected from different token positions within the same long sequence to form samples, then the measured value will not be the universal activation distribution of neurons. In this case, the changes in activation values ​​will primarily reflect the internal language structure of that specific sequence, rather than the inherent randomness of neurons during the decoding stage.

[0196] Therefore, unlike the general pruning algorithm for obtaining activation values ​​in the prefill stage, it is necessary to take the activation values ​​of tokens generated in the generation stage from different samples. Experiments have shown that modeling the initial k (16-32) tokens generated is better than random sampling and modeling using the activation values ​​of all tokens.

[0197] S3. Determine the importance threshold according to the preset pruning rate, and generate an index set of all global interneurons with scores lower than the importance threshold in one go based on the global importance map.

[0198] Furthermore, such as Figure 6 As shown, step S3 includes:

[0199] S31. Based on the preset network pruning ratio, perform percentile statistics on the global graph containing the normalized importance scores of all neurons to determine the corresponding global importance threshold. Based on the preset global pruning rate (e.g., removing 25% of neurons), determine a uniform entropy importance score threshold.

[0200] S32. Scan the global importance map element by element and compare the numerical relationship between the importance score of each neuron and the global importance threshold in real time.

[0201] S33. The location information of neurons whose importance scores are consistently below the global importance threshold is synchronously recorded across all target feedforward network layers into a unified pruning index container, forming a global index set of intermediate neurons to be pruned.

[0202] S4. Based on the index set, perform one-time physical structured pruning on the weight matrix of each target layer.

[0203] Furthermore, such as Figure 7 As shown, step S4 includes:

[0204] S41. Search for the target feedforward network layer containing the interneurons to be pruned along the network depth direction layer by layer to generate a queue of layers to be processed.

[0205] S42. Create a transactional temporary mirror copy of the weight matrix for each target feedforward network layer in the queue of layers to be processed, and establish a backup of the original weight state.

[0206] S43. For the queue of layers to be processed, based on the coordinate information of the intermediate neurons to be pruned recorded in the index set, perform input dimension reduction on the weight matrices of the gated projection layer and the uplink projection layer, and delete the row vectors of the weight matrix corresponding to the intermediate neurons to be pruned.

[0207] In this step, the weight matrix of each FFN layer is physically and permanently modified based on the global set of indices to be pruned:

[0208] For the gate_proj and up_proj layers, delete the corresponding rows in their weight matrices.

[0209] For the down_proj layer, delete the corresponding column in its weight matrix.

[0210] S44. Simultaneously, perform output dimension reduction on the weight matrix of the downprojection layer and delete the column vectors of the weight matrix corresponding to the intermediate neurons to be pruned.

[0211] S45. Based on the backup of the original weight state, perform inter-layer connection dimension matching verification on all target feedforward network layers that have completed the reduction operation. The inter-layer connection dimension matching verification involves performing a structured verification of the dimensions of the input / output matrices between adjacent layers of the neural network to ensure that the weight matrix after pruning maintains the mathematical computability of the feedforward computation chain during cross-layer transmission.

[0212] S46. If the dimension matching verification passes, the reduced weight matrix is ​​atomically replaced in batches to the corresponding network layer; if the dimension matching verification fails, the transaction rollback mechanism is automatically triggered to restore the weight matrix of all target feedforward network layers to the original backup state.

[0213] S47. After performing a one-time structured row and / or column dimension deletion on all target feedforward network layers, a compressed model without masking is generated. The resulting model is smaller and can be directly used for subsequent fine-tuning or inference without any masking.

[0214] In one specific embodiment, dynamic autoregressive pruning is implemented based on multiple models from the InternVL3-2B and Qwen3 series. The specific steps are as follows:

[0215] First, prepare the system and environment by instantiating the model and loading the pre-trained InternVL3-2B model along with its visual encoder and word segmenter. If the model is distributed across multiple GPUs, the device mapping relationships for each layer need to be recorded to ensure hardware consistency for subsequent computations and data collection.

[0216] Next, prepare a calibration dataset D containing M samples, D = {S1, ..., S2}. M Each sample S m =(Im P m ), where I m For image data, P m This involves providing text prompts; subsequently, identifying all FFN layers in the model that require pruning. For example, for the language model part of InternVL3-2B, this corresponds to the mlp module in each layer, which contains three linear sub-layers: gate_proj, up_proj, and down_proj; for the language model part of InternVL3-2B, this contains three linear sub-layers: w1, w2, and w3. The Qwen3 series uses a similar approach, which will not be elaborated further.

[0217] Secondly, traditional pruning methods typically only extract activation values ​​from the pre-filling stage, but the dynamic behavior of the model during the autoregressive generation process is more informative. This invention uses hook technology to capture the internal state of the decoding stage in real time. Taking InternVL3-2B as an example, the specific process is detailed below:

[0218] (1) Layer encapsulation:

[0219] Define and instantiate a Wrapper class. This class contains member variables for online calculation of statistics, which may include:

[0220] Norm_l2 / Norm_l1: Used to store and update L2 / L1 norms.

[0221] min_vals, max_vals: Used to determine the maximum and minimum values ​​of the activation values.

[0222] activation_input_matrix: Used to store the activation value for generating the token.

[0223] Create a Wrapper instance for all layer goals gate_proj, up_proj, and down_proj sub-layers in the model and store it in a dictionary.

[0224] (2) Hook registration and execution:

[0225] Iterate through each sample S in dataset D m =(I m P m For each sample, execute the following loop:

[0226] a. Register hooks: Before processing all samples, register a forward hook for all target sub-layers.

[0227] b. Execute generation: Call the model's generate function, with the sample S as input. mThe model will first perform a prefill, and then enter the autoregressive decoding loop to generate new tokens one by one. The key point is that the Wrapper distinguishes between the Prefill and Generate phases by checking the sequence length L of the input activation tensor, and it will only record the activation value when the sequence length L is 1 (i.e. in the decoding loop), ensuring that the collected data strictly comes from the dynamic generation process.

[0228] c. Data Accumulation: Throughout the generation process, hooks are triggered at each decoding step. The Wrapper instance continuously accumulates statistical data (records channel activation values, updates the L2 norm, and updates various statistical values ​​such as maximum, minimum, variance, and mean as needed).

[0229] d. Remove hooks: After the generation process of all samples is completed, remove all hooks uniformly.

[0230] (3) The above data collection process only needs to be executed once: traverse the complete dataset D once. During this stage, the Wrapper determines the activation value of each channel and completes the calculation of statistical values ​​such as the L2 norm.

[0231] Next, after completing the collection of statistical data, the magnitude importance score and entropy importance score are calculated for each FFN layer according to the specific importance calculation content in step S2 above, and a unified importance score vector is generated by weighted fusion.

[0232] Next, structured pruning is performed. All intermediate feature channels are sorted according to the final importance score vector to determine the indices of the channels to be pruned. The following simultaneous pruning is implemented:

[0233] down_proj: Removes the column from its weight matrix that corresponds to the index of the channel to be pruned.

[0234] gate_proj: Removes the row in its weight matrix that corresponds to the index of the channel to be pruned.

[0235] up_proj: Removes the row in its weight matrix that corresponds to the index of the channel to be pruned.

[0236] Accordingly, the bias vectors of gate_proj and up_proj are clipped.

[0237] pass Figure 8 and Figure 9 Comparative experiments allow for in-depth analysis of the effectiveness of different pruning strategies. The experiments pruned the model using both the state-of-the-art (SOTA) method based solely on amplitude and the method of this invention, recording the variance of the activation response of the channels to be pruned. The results show:

[0238] The variance of channels pruned using the amplitude-only method is significantly higher than that of the method in this invention. This indicates that channels deemed "unimportant" by the traditional amplitude-only method still exhibit differential responses to different input samples, implying potential information value, and direct pruning may lead to a loss in model performance.

[0239] The variance of the channels pruned by the method of this invention approaches zero, verifying the rigor of its selection criteria—channels marked as redundant are indeed low-response, low-information regions. Furthermore, the variance distribution trends of the high-importance channels selected by the two methods are highly consistent, indicating that the core functional channels of the model possess stable identifiability, whether based on amplitude or comprehensive evaluation.

[0240] refer to Figure 12 It shows the amplitude and entropy distribution of retained and deleted neurons in a certain layer when the pruning ratio is 50%, further revealing the selection logic of different strategies:

[0241] Pure amplitude pruning ( Figure 12 (Illustration of a medium rectangle): While retaining high-amplitude channels, a large number of high-entropy channels (rich in information but with low amplitude) are mistakenly deleted. These channels may play a key role in complex semantic reasoning, and their loss will affect the model's generalization ability.

[0242] Pure entropy pruning ( Figure 12 (Diagram of upper-middle triangle): When high-entropy channels are retained, high-amplitude channels (high activation intensity but redundant information) are incorrectly removed, resulting in a decrease in the model's ability to express basic features.

[0243] The present invention solution ( Figure 12 (Diagram of the lower-middle triangle): By comprehensively evaluating amplitude and entropy, only low-amplitude and low-entropy redundant channels are removed, while high-amplitude (functionally critical) and high-entropy (information-rich) channels are retained, achieving an optimal balance between performance and efficiency.

[0244] Next, to verify the actual effectiveness of the method, referring to Experiment 1, this embodiment of the invention was tested on a professionally fine-tuned model. Its core task was to identify chart information and output structured JSON data as required. The experiment used 400 charts as test samples and set the following comparison schemes: amplitude only, entropy only, and the scheme of this invention. To provide a more intuitive comparison, an unstructured state-of-the-art (SOTA) scheme (e.g., Wanda) was also set.

[0245] The evaluation criteria are strictly divided into:

[0246] 1. Format accuracy: If the format is inaccurate, i.e. cannot be correctly parsed into JSON format, the accuracy of the table's content is 0.

[0247] 2. Content accuracy: Compare the accuracy of key-value pairs in the chart, taking into account the accuracy of both key values ​​and key names.

[0248] 3. Real-time performance comparison: Compare results with pruning rates of 10% to 70%, without recovery training.

[0249] 4. Comparison of recovery performance: Results with pruning rates of 10% and 50% were used in the recovery training.

[0250] The experimental results can be referenced. Figure 10 and Figure 11 Whether it is the immediate performance of pruning or the performance of recovery training, the method provided by this invention is superior to other existing solutions used as benchmarks. In high pruning rate scenarios (above 60%), it can even approach or surpass unstructured solutions.

[0251] To further verify the generalization ability of the pruning model, this invention tests its performance on general test metrics. The experiment uses the original unpruned model (Base) as a benchmark, comparing the performance degradation under different pruning rates. Referring to Tables 1 to 4 below, it can be seen that the structured pruning scheme of this invention performs no worse than existing state-of-the-art (SOTA) solutions on all metrics. The structured pruning scheme of this invention is not only suitable for professional graph parsing tasks, but also demonstrates significant advantages in general language tasks.

[0252] Table 1

[0253]

[0254] Table 1: Zero-shot performance of the model on language inference benchmarks. The model was calibrated on the Alpaca (Qwen3-1.7B) and VQA (InternVL3-2B) datasets. Magnitude represents the application of pure magnitude pruning criteria in structured pruning, derived from Wanda. Esparse is another method combining magnitude and entropy for pruning, primarily using entropy as a modifier for magnitude. Entropy is a pure entropy pruning scheme, and Ours is the scheme of this invention. ARC-c, ARC-e, BoolQ, HellaSwag, OBQA, RTE, and WinoGrande are all benchmarks related to natural language processing, used to measure a model's language generation capabilities from multiple task perspectives. The results demonstrate that the pruning performance of this invention outperforms state-of-the-art pruning methods on both large language models and multimodal models.

[0255] Table 2

[0256]

[0257] Table 2: This table reports the perplexity and average inference accuracy on WikiText2 for Qwen (LLM) and InternVL (VLM,*) models with different parameter values. This table clearly demonstrates that the method of this invention is also a superior choice for applying to larger-sized models.

[0258] Table 3

[0259]

[0260] Table 3: AI2D, ChartQA, GQA, MME-cog, MME-per, and OKVQA are all benchmark tests related to multimodal tasks, used to measure the ability of a multimodal model to complete image-related tasks from multiple task perspectives. These results further validate the versatility and advantages of the method presented in this invention.

[0261] Table 4

[0262]

[0263] Table 4: This table records the average accuracy of the pruned model on language benchmark tasks after capturing activation values ​​at different inference stages of the model to perform structured and unstructured pruning. The results in the table indicate that capturing activation values ​​in the pre-filling stage is only suitable for unstructured pruning methods; to achieve better results, structured pruning requires capturing activation values ​​in the generation stage.

[0264] Additionally, this invention provides a global one-time structured pruning system for neural networks based on a hybrid amplitude entropy importance map, comprising: a tensor capture module, used to capture the input activation tensors of all target layers in parallel using a calibration dataset in a single forward propagation; a map output module, used to simultaneously calculate the differential entropy index and amplitude response intensity based on the input activation tensors and the association weights of intermediate neurons in each target layer, and form a static global importance map after normalization and fusion; a neuron selection module, used to determine an importance threshold according to a preset pruning rate, and generate an index set of all global intermediate neurons with scores below the importance threshold based on the global importance map; and a structured pruning module, used to perform one-time physical structured pruning on the weight matrices of each target layer based on the index set.

[0265] Then, an embodiment of the present invention provides a neural network global one-time structured pruning device based on a hybrid amplitude entropy importance map, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute the neural network global one-time structured pruning method based on a hybrid amplitude entropy importance map as described above.

[0266] Furthermore, embodiments of the present invention provide a computer-readable storage medium storing computer-executable instructions. When the executable instructions are executed by a processor, they implement the global one-time structured pruning method for neural networks based on a hybrid amplitude entropy importance graph as described above.

[0267] In summary, this invention provides a method, system, device, and medium for global one-time structured pruning of neural networks, achieving efficient model compression through the following logical links: First, a single forward propagation is performed in the neural network to be pruned using a calibration dataset, capturing the original input activation tensors of all target layers in parallel, ensuring that all calculations are based on the unmodified initial model state. Next, for the reassembly of neuron weights in each target layer, the differential entropy index and amplitude response intensity are calculated simultaneously, and after normalization and dynamic fusion, a global static importance map is generated. Subsequently, an importance threshold is determined based on a preset pruning rate, and a set of global intermediate neuron indices below the threshold is directly selected from the map. Finally, a one-time physical pruning operation is performed on the weight matrices of all target layers.

[0268] The core advantages of this method are reflected in three aspects:

[0269] First, the method of this invention performs only one forward propagation on the original, unmodified, and error-free model. The importance calculations for all layers are based on their most ideal and accurate input activations. The computation is dependency-free; the importance calculation for layer L+1 is completely independent of the pruning decisions for layer L. They are computationally parallel and independent. Layer L+1 sees the original input, and layer L also sees the original input. There is no "error propagation" path between them. The result is a complete "global importance map" representing the original state of the model. Pruning decisions are made based on this single, uncontaminated true map. All neurons to be deleted are identified and executed all at once. There is no opportunity for pruning errors in one layer to affect the decision-making process of the next layer.

[0270] Second, structured pruning directly reduces the parameter scale, eliminating the need for complex mask management or special hardware adaptation, thus significantly reducing deployment complexity.

[0271] Third, based on the solid theoretical foundation of "relative information entropy ranking stability," the effectiveness of pruning is ensured, which can significantly reduce model complexity while preserving model performance to the greatest extent. At the same time, it takes into account the token information of the autoregressive generation process, avoiding the model performance loss caused by only considering pre-filled text prompts. It is especially suitable for pruning and compression of professional models (i.e., models fine-tuned for professional fields or vertical categories) (when pruning is tested on general models, the performance is still better than the existing state-of-the-art structured pruning methods).

[0272] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.

[0273] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the technical solution should be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0274] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the invention's technical solutions and their equivalents, then the invention should also include these modifications and variations.

Claims

1. A global one-time structured pruning method for neural networks based on a hybrid amplitude entropy importance graph, characterized in that, include: The selected neural networks to be pruned, distributed across multiple GPUs, are instantiated and loaded. The device mapping relationship of each layer of the neural network to be pruned is recorded to ensure hardware consistency for computation and data collection. Through a single forward propagation, the input activation tensors of all target layers in the neural network to be pruned are captured in parallel using a calibration dataset. During the forward propagation, each target feedforward network layer and its input activation tensor are intercepted in parallel and at once using pre-implanted hooks. The input activation tensors are stored iteratively to ensure that only a single storage result of the fusion of the current input activation tensor and the historical input activation tensors is always retained. Each word hidden state vector in the input activation tensor is generated based on the corresponding context during the decoding stage and is used to predict the final output word. The word hidden state vector is the smallest operable data unit in the input activation tensor that carries independent semantic context information. Based on the input activation tensor, the differential entropy index and amplitude response intensity are calculated synchronously for the association weight reorganization of intermediate neurons in each target layer. After normalization and fusion, a static global importance map is formed. The association weight reorganization includes specified rows of the gated projection layer weight matrix, specified rows of the up projection layer weight matrix, and specified columns of the down projection layer weight matrix. The importance threshold is determined according to the preset pruning rate, and an index set of all global interneurons with scores below the importance threshold is generated at once based on the global importance map. Based on the index set, a one-time physical structured pruning is performed on the weight matrix of each target layer, deleting the row / column vectors of the weight matrix and performing inter-layer connection dimension matching verification. If the dimension matching verification passes, the reduced weight matrix is ​​batch-replaced into the corresponding network layer; if the dimension matching verification fails, the transaction rollback mechanism is automatically triggered, restoring the weight matrix of all target feedforward network layers to the original backup state. After completing the one-time structured row and / or column dimension deletion of all target feedforward network layers, a compressed network without masking operations is generated. The pruned compressed network is directly deployed for performing graph parsing tasks and general language tasks.

2. The global one-time structured pruning method for neural networks based on hybrid amplitude entropy importance maps as described in claim 1, characterized in that, Based on the input activation tensor, the differential entropy index and amplitude response intensity are simultaneously calculated by recombining the association weights of intermediate neurons in each target layer. After normalization and fusion, a static global importance map is formed, including: Reconfigure the association weights of each intermediate neuron in the target feedforward network layer; Based on the input activation tensor, calculate the product matrix of each weight in the association weight reorganization and the activation response intensity index of the corresponding feature channel, and perform aggregation operation according to the input or output dimension to generate amplitude response intensity score. In the downcast layer, the intermediate activation tensor generated by the modulation of gated projection and upcast output is reconstructed in two dimensions and the feature channels are separated. The differential entropy index of each intermediate neuron is calculated by kernel density estimation and numerical integration. The amplitude response intensity fraction and the differential entropy index are normalized respectively; The normalized amplitude result is weighted and fused with the differential entropy result to generate a static global importance map representing the undisturbed intrinsic information content.

3. The global one-time structured pruning method for neural networks based on hybrid amplitude entropy importance maps as described in claim 2, characterized in that, Based on the input activation tensor, the product matrix of each weight in the association weight reorganization and the activation response intensity index of the corresponding feature channel is calculated. Aggregation operations are performed according to the input or output dimension to generate amplitude response intensity scores, including: In the gated projection layer, locate the weight matrix row corresponding to the target neuron, traverse all the weight elements corresponding to the input channels in the weight matrix row, multiply the absolute value of each weight element by the L2 norm of the feature channel corresponding to the input activation tensor, and sum them along the direction of the input feature channel to obtain the importance score of the gated row. In the up-projection layer, locate the row of the weight matrix corresponding to the target neuron, traverse all weight elements corresponding to all input channels in the row of the weight matrix, multiply the absolute value of each weight element by the L2 norm of the feature channel corresponding to the input activation tensor, and sum them along the direction of the input feature channel to obtain the importance score of the up-row; In the downcast projection layer, the weight matrix column corresponding to the target neuron is located. The weight elements corresponding to all output channels in the weight matrix column are traversed. The absolute value of each weight element is multiplied by the L2 norm of the feature channel corresponding to the intermediate activation tensor. The summation is carried out along the output feature channel to obtain the importance score of the downcast column. The intermediate activation tensor is obtained by performing a Hadamard product operation between the output of the gated projection layer and the output of the upcast projection layer after the SiLU function transformation. The gating row importance score, the up row importance score, and the down column importance score are added together to generate the amplitude response strength score.

4. The global one-time structured pruning method for neural networks based on hybrid amplitude entropy importance maps as described in claim 2, characterized in that, In the downcast projection layer, the intermediate activation tensor generated based on the modulation of gated projection and upcast output is reconstructed in two dimensions and its feature channels are separated. The differential entropy index of each intermediate neuron is calculated by kernel density estimation and numerical integration, including: In the downlink projection layer, the intermediate activation tensor, which is generated in three-dimensional form based on the modulation of gated projection and uplink projection output, is reconstructed into a two-dimensional matrix with independent observation samples as rows and feature dimensions as columns along the batch dimension and sequence dimension. Extract the set of activation values ​​corresponding to each intermediate neuron in the two-dimensional matrix column by column; A kernel function is applied to the set of activation values ​​for each intermediate neuron to smooth the data and construct a continuous probability density function that characterizes the distribution of activation values ​​of the neuron. Based on the continuous probability density function, the log-probability weighted integral of the activation value distribution is calculated within a preset numerical integration interval, and the differential entropy index is approximated by numerical integration.

5. The global one-time structured pruning method for neural networks based on hybrid amplitude entropy importance maps as described in any one of claims 1-4, characterized in that, An importance threshold is determined according to a preset pruning rate. Based on the global importance map, an index set of all global interneurons with scores below the importance threshold is generated in one go, including: Based on the preset network pruning ratio, percentile statistics are performed on the global graph containing the normalized importance scores of all neurons to determine the corresponding global importance threshold. The global importance map is scanned element by element, and the numerical relationship between the importance score of each neuron and the global importance threshold is compared in real time. The location information of neurons whose importance scores are consistently below the global importance threshold is synchronously recorded across all target feedforward network layers into a unified pruning index container, forming a global index set of intermediate neurons to be pruned.

6. The global one-time structured pruning method for neural networks based on hybrid amplitude entropy importance maps as described in any one of claims 1-4, characterized in that, Based on the index set, a one-time physical structured pruning is performed on the weight matrix of each target layer, deleting the row / column vectors of the weight matrix and performing inter-layer connection dimension matching verification. If the dimension matching verification passes, the reduced weight matrix is ​​batch-replaced into the corresponding network layer. If the dimension matching verification fails, the transaction rollback mechanism is automatically triggered, restoring the weight matrices of all target feedforward network layers to their original backup state. After completing the one-time structured row and / or column dimension deletion of all target feedforward network layers, a compressed network without masking operations is generated. The pruned compressed network is then directly deployed for performing graph parsing tasks and general language tasks, including: Search for the target feedforward network layer containing the interneurons to be pruned along the network depth direction layer by layer to generate a queue of layers to be processed. Create a transactional temporary mirror copy of the weight matrix for each target feedforward network layer in the queue of layers to be processed, and establish a backup of the original weight state; For the queue of layers to be processed, based on the coordinate information of the intermediate neurons to be pruned recorded in the index set, the input dimension of the weight matrices of the gated projection layer and the up-projection layer is reduced, and the row vectors of the weight matrix corresponding to the intermediate neurons to be pruned are deleted. Simultaneously, output dimension reduction is performed on the weight matrix of the downprojection layer, and column vectors of the weight matrix corresponding to the intermediate neurons to be pruned are deleted. Based on the backup of the original weight state, a preset inter-layer connection dimension matching verification is performed on all target feedforward network layers that have completed the reduction operation. If the dimension matching verification passes, the reduced weight matrix will be replaced in batches to the corresponding network layers; If the dimension matching verification fails, the transaction rollback mechanism is automatically triggered to restore the weight matrix of all target feedforward network layers to the original backup state. After performing a one-time structured row and / or column dimension deletion of all target feedforward network layers, a compressed network without masking operations is generated.

7. A global one-time structured pruning system for neural networks based on a hybrid amplitude entropy importance map, characterized in that, include: The tensor capture module instantiates and loads selected neural networks to be pruned, distributed across multiple GPUs, and records the device mapping relationships of each layer of the neural network to be pruned to ensure hardware consistency in computation and data collection. Through a single forward propagation, it captures the input activation tensors of all target layers in the neural network to be pruned in parallel using a calibration dataset. During the forward propagation, each target feedforward network layer and its input activation tensor are captured in parallel and at once using pre-implanted hooks, and the input activation tensors are iteratively stored to ensure that only a single storage result of the fusion of the current input activation tensor and the historical input activation tensors is always retained. Each word hidden state vector in the input activation tensor is generated based on the corresponding context during the decoding stage and is used to predict the final output word. The word hidden state vector is the smallest operable data unit in the input activation tensor that carries independent semantic context information. The graph output module is used to simultaneously calculate the differential entropy index and amplitude response intensity of the association weight reorganization of the intermediate neurons of each target layer based on the input activation tensor. After normalization and fusion, a static global importance graph is formed. The association weight reorganization includes specified rows of the gated projection layer weight matrix, specified rows of the up projection layer weight matrix, and specified columns of the down projection layer weight matrix. The neuron selection module is used to determine the importance threshold according to the preset pruning rate and generate an index set of all global intermediate neurons with scores lower than the importance threshold in one go based on the global importance map; The structured pruning module performs a one-time physical structured pruning of the weight matrices of each target layer based on the index set. It deletes row / column vectors of the weight matrix and performs inter-layer connection dimension matching verification. If the dimension matching verification passes, the shrunk weight matrices are batch-replaced into the corresponding network layers. If the dimension matching verification fails, a transaction rollback mechanism is automatically triggered to restore the weight matrices of all target feedforward network layers to their original backup state. After completing the one-time structured row and / or column dimension deletion of all target feedforward network layers, a compressed network without masking operations is generated. The pruned compressed network is then directly deployed for graph parsing tasks and general language tasks.

8. A neural network global one-time structured pruning device based on a hybrid amplitude entropy importance map, characterized in that, include: At least one processor; and memory that is communicatively connected to at least one processor; The memory stores instructions that can be executed by at least one processor, which are executed by at least one processor to enable the at least one processor to perform the global one-time structured pruning method for neural networks based on the hybrid amplitude entropy importance map as described in any one of claims 1-6.

9. A computer-readable storage medium storing computer-executable instructions thereon, characterized in that, When the executable instructions are executed by the processor, they implement the global one-time structured pruning method for neural networks based on the hybrid amplitude entropy importance map as described in any one of claims 1-6.

Images