Model filter compression method based on gradient guidance and terminal equipment

Abstract

The invention relates to the technical field of deep learning and neural network model compression, and discloses a model filter compression method based on gradient guidance and terminal equipment in order to solve the technical problems of high reasoning cost and difficult deployment of an FSMN model caused by a high-order FIR filter. A trained FSMN model is obtained, the model comprises at least one high-order finite impulse response filter layer, the weight of the high-order finite impulse response filter layer is defined as the step b, and the gradient of a final loss function of the FSMN model relative to the gradient is determined and calculated; and step c, searching and determining an infinite impulse response filter of which the order is lower than that of the FIR filter based on the guidance of the gradient, and defining the weight of the infinite impulse response filter as step d, and replacing the generated compressed FSMN model with the infinite impulse response filter. The problem that the FSMN model is high in reasoning cost is solved, and the parameter quantity and the calculation complexity of the model are remarkably reduced.

Description

Technical Field

[0001] This invention relates to the field of deep learning and neural network model compression technology, and in particular to a gradient-guided model filter compression method for efficiently compressing high-order finite impulse response filters in feedforward sequence memory networks, thereby reducing model inference complexity while ensuring or even improving the final task performance of the model. Background Technology

[0002] Feedforward Sequential Memory Networks (FSMNs) are a class of neural network architectures proposed to overcome the vanishing or exploding gradient problems encountered by traditional recurrent neural networks (RNNs) during training. The core idea is to use a feedforward finite impulse response (FIR) filter layer (i.e., a tapped delay network) to replace the recursive states of the RNN, thereby encoding historical information within a fixed window size (look-ahead and look-back). An FSMN module and its FIR filtering form are as follows: Figure 1 As shown.

[0003] Its advantages: FSMN is a completely feedforward structure, which avoids the gradient problem of RNN and can be trained in parallel efficiently.

[0004] Its disadvantages: In order to simulate long-term dependencies, the FIR filter of FSMN needs to be of a very high order (i.e., a large number of taps, such as tens or even hundreds), which makes its parameter count and inference computation far exceed those of RNN with equivalent performance.

[0005] In traditional model compression, common techniques include low-rank decomposition (such as SVD). The goal of traditional low-rank decomposition is to minimize the weight approximation error, i.e. ,in It is the original weight. These are the compressed weights. This optimization objective is related to the model's final performance (such as the loss function). () are separate.

[0006] Disadvantages of existing technology:

[0007] FSMN has high inference costs: While FSMN solves the training efficiency problem, its high-order FIR structure leads to huge memory consumption and inference computation. This makes it difficult to deploy trained FSMN models on devices with limited computing and memory resources (such as mobile phones and edge computing devices).

[0008] Performance loss of traditional compression methods: If attempting to compress FSMN, for example using traditional signal processing methods (such as least squares, Prony's method, etc.) to approximate its higher-order FIR filter with a low-order infinite impulse response (IIR) filter, this method is a local optimization (only concerned with the response error of the filter itself). This local approximation error accumulates layer by layer in deep networks, leading to a significant deterioration in the final model performance (such as the word error rate WER in speech recognition or the model's loss).

[0009] The application gap of advanced compression methods: Although joint loss optimization compression is conceptually advanced, its current applications are limited to low-rank decompositions such as SVD, and it has not yet been used to solve the structural compression problem of FIR filters to IIR filters in FSMN. Therefore, there is an urgent need for a filter compression method that can significantly reduce the inference cost of FSMN models while ensuring or even improving the final performance of the model. Summary of the Invention

[0010] The purpose of this invention is to provide a gradient-guided model filter compression method and terminal device to solve the technical problems of high inference cost and difficult deployment caused by high-order FIR filters in FSMN models, and to avoid the performance loss caused by traditional compression methods.

[0011] To achieve the above objectives, this invention provides a gradient-guided model filter compression method suitable for compressing high-order finite impulse response (FIR) filters in feedforward sequence memory networks (FSMNs), comprising the following steps:

[0012] Step a: Obtain a trained FSMN model, which contains at least one high-order finite impulse response filter layer, and define its weights as follows. ;

[0013] Step b, determine one or more of the above. And calculate the final loss function of the FSMN model with respect to the... The gradient;

[0014] Step c: Based on the gradient guidance, search for and determine an infinite impulse response filter with an order lower than the FIR filter, and define its weights as follows. ;

[0015] Step d, using the Replace the This generates a compressed FSMN model.

[0016] Further, in step c, the guidance based on the gradient includes:

[0017] Step c1. In a common representation space, compute the... Compared to The approximate error vector;

[0018] Step c2. Select the one that satisfies conditional .

[0019] Furthermore, the common representation space is a time-domain impulse response space, wherein the gradient is in The gradient is calculated in the coefficient space; the approximate error vector is the... impulse response (truncation to) order ) and the The difference between the coefficients (i.e., the impulse response).

[0020] Furthermore, the common representation space is a frequency response space, wherein the gradient is the frequency domain gradient obtained through a transformation (such as a DFT). The approximation error vector is the... frequency response With the frequency response The difference between them.

[0021] Furthermore, the selection in step c also includes a constraint: the approximation error vector The norm must be less than a preset perturbation threshold. (For example ).

[0022] Furthermore, in step c, the candidate candidate that satisfies the constraints is selected. In the middle, make the inner product To reach the minimum (i.e., maximize performance improvement) .

[0023] Furthermore, in step c, the candidate candidate that satisfies the constraints is selected. Among them, the one with the lowest filter order (i.e., the fewest parameters) .

[0024] Furthermore, the method is executed after the FSMN model has been trained, without the need to retrain or fine-tune the generated compressed model.

[0025] The gradient-guided model filter compression method described above is applied to speech recognition, natural language processing, or other sequence data processing tasks, wherein the final loss function... This corresponds to word error rate (WER), cross-entropy loss, or perplexity.

[0026] The present invention also provides a terminal device, the terminal device including a memory and a processor, the memory storing a computer program executable on the processor, the processor being configured to perform the method as described above.

[0027] The gradient-guided model filter compression method and terminal device provided by this invention have the following advantages:

[0028] This invention proposes a post-training compression method. During the process of replacing FIR filters with low-order IIR filters, the global gradient of the FIR weights in the model's final loss function guides the selection of candidate IIR filters, ensuring that the perturbations introduced by compression help reduce or at least not degrade model performance. This addresses the high inference cost problem of the FSMN model by significantly reducing the number of model parameters and computational complexity through replacing its high-order FIR filters with low-order IIR filters.

[0029] More importantly, this invention utilizes the idea of ​​joint loss optimization to ensure that this FIR-to-IIR structural replacement does not lead to a loss of final model performance (Loss or WER), resulting in performance improvement. At the same time, the entire compression process is efficient and does not require retraining or fine-tuning. Attached Figure Description

[0030] Figure 1 A schematic diagram of the FSMN module and its FIR filtering method;

[0031] Figure 2 A schematic diagram illustrating the specific implementation steps of this invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0033] This invention proposes a gradient-guided FSMN-Filter Approximation (GFA) method and terminal device, which belongs to a post-trained model compression technology and is suitable for efficient and low-loss compression of high-order FIR filters in trained feedforward sequence memory networks (FSMN).

[0034] This invention utilizes high-order FIR filters in FSMN. Replace with a low-order IIR filter The process is considered to introduce a compression error. We utilize the model's final loss. right gradient To guide The choice, finding one that can make of This allows for parameter compression while maintaining or even reducing the model's total loss. .

[0035] The overall architecture of this invention includes the following four functional modules:

[0036] Module A: FSMN Training Module

[0037] The input is the complete training dataset, and the output is a fully trained FSMN model. It contains one or more FSMN layers, each containing a higher-order FIR filter, whose weights are denoted as... .

[0038] Module B: Global Gradient Calculation Module

[0039] The input is the "model" of the trained model. "and calibration dataset", the output is "final loss" right gradient ".

[0040] Module C: Filter Approximation and Constraint Checking Module (GFA Core)

[0041] This module receives and As input.

[0042] Internally, it contains an "IIR candidate search loop" (e.g., by IIR order). search).

[0043] For each candidate ,calculate .

[0044] A "constraint check unit" is used to perform two checks:

[0045] Error amplitude constraint: ?

[0046] Gradient direction constraint: ?

[0047] The output is "optimal low-order IIR filter". ".

[0048] Module D: Model Reconstruction Module

[0049] Input the original "model" "and the selected optimal " The output is "compressed FSMN model". ",in Already replace.

[0050] like Figure 2 As shown, the specific implementation steps of the present invention are as follows:

[0051] Step 1: Obtain the trained FSMN model

[0052] Obtain a fully trained FSMN model The model It contains one or more FSMN layers, each containing a higher-order FIR filter, whose weights are denoted as... .

[0053] Step 2: Prepare calibration data and computing environment

[0054] Prepare a small, representative calibration dataset, for example, a few hundred samples drawn from the validation set. This dataset is used to compute gradients and does not need to be used for training. Set an error magnitude constraint threshold. ( Can be set to (On a scale of magnitude), used to ensure the effectiveness of gradient approximation.

[0055] Step 3: Perform filter replacement layer by layer

[0056] Traverse the model in order Each FSMN layer in the process (let the current layer be the th layer) layer).

[0057] Step 4: Calculate the global loss gradient

[0058] Perform one forward and back propagation on the calibration dataset to compute the model's final, global loss function. (For example, WER or cross-entropy loss in speech recognition) Regarding the current... Layer FIR filter weights gradient .

[0059] Note: Even if the model has converged, this gradient is almost non-zero in practical engineering. Revealed The direction of the impact of minute changes on final performance.

[0060] Step 5: Gradient-guided IIR filter search

[0061] The goal of this step is to find a low-order IIR filter. , used to replace .

[0062] 1. Set the search range: Set the target order range for the IIR filter, for example, from... Starting from the first order, the search reaches the preset maximum order. ( much smaller (order).

[0063] 2. Initialize the candidate list: Create an empty candidate list. .

[0064] 3. Iterative search: For :

[0065] a. Generate candidate IIRs: Find one using any standard filter fitting algorithm (such as Prony's method, Steiglitz-McBride method, or simple gradient descent). Rank to make it initially approach .

[0066] b. Calculate the error vector: .

[0067] because ( (FIR) and The (P, Q order IIR) structures are different and cannot be directly subtracted. We must... Effect mapping to Where Space, to compute with gradient Corresponding error .

[0068] In b, the following two embodiments are included:

[0069] Example 1

[0070] Error calculation based on impulse response (time domain) is the preferred embodiment of this invention because it most directly corresponds to... The parameter space. i. The coefficient itself is its finite impulse response (FIR). ii. Calculate candidate The infinite impulse response (IIR) of the finite impulse response, and its preceding... Item, obtained .

[0071] iii. Define time-domain error : (that is, in) In 3D space, the difference in impulse response between two filters.

[0072] iv. This Vectors and gradients They have exactly the same dimensions and physical meaning (i.e., for each time step) (response weights).

[0073] Example 2

[0074] Error calculation based on frequency response (frequency domain) is suitable for scenarios with higher requirements for frequency domain matching.

[0075] i. Transform the gradient: Convert the time-domain (coefficient) gradient The frequency gradient is obtained by mapping it to the frequency domain through the Discrete Fourier Transform (DFT) or other transformations. The gradient represents the gradient at each frequency point. The response change on the final The impact.

[0076] ii. Calculate the frequency domain response: Calculate separately and frequency response and .

[0077] iii. Define frequency domain error :

[0078] iv. Calculate the inner product in the frequency domain: The inner product calculation in step (d) is transformed into being performed in the frequency domain: (Note: _t represents taking the real part, and _t represents the conjugate transpose, used in complex inner products.

[0079] c. Check constraints (1) - amplitude constraints:

[0080] Verify whether the error is within the effective perturbation range, i.e. . ( (For the Frobenius norm). Or (Using frequency domain error), if this constraint is not met, it indicates that the order is incorrect. The IIR filter introduces too much noise and is nearly ineffective; this should be skipped. .

[0081] d. Check constraints (2) - gradient constraints (core): If the magnitude constraint is satisfied, calculate the inner product of the gradient and the error. (Implementation Example 1) or (Implementation Example 2)

[0082] e. Store in the candidate list: If (That is, this error helps reduce the global loss), then... Save to candidate list .

[0083] Step Six: Optimal IIR Filter Selection

[0084] After the search loop (step five) ends, the candidate list is adjusted according to different compression strategies. Choose the optimal one This includes the following strategies:

[0085] Strategy 1 (Lossless Optimization):

[0086] Goal: To achieve optimal performance.

[0087] Operation: From list Select to make inner product Minimum (i.e.) The one that is the most negative This corresponds to minimizing the maximization of the model's loss.

[0088] Strategy 2 (Compact Optimization):

[0089] Objective: To achieve the maximum compression ratio.

[0090] Operation: From list Choose the one with the smallest order that one This corresponds to using the fewest parameters without increasing the loss.

[0091] Step 7: Model Reconstruction

[0092] Current layer of Replace with the optimal one selected in step six .

[0093] Step 8: Loop and Complete

[0094] Repeat steps three through seven until the model is complete. All (or all specified) FSMN layers are replaced. The final model is obtained. This is the compressed high-performance model.

[0095] The gradient-guided model filter compression method and terminal device provided by this invention have the following advantages:

[0096] By using high-order FIR filters in FSMN Replace with a low-order IIR filter The process is considered to introduce a compression error. We utilize the model's final loss. right gradient To guide The choice, finding one that can make of This allows for parameter compression while maintaining or even reducing the model's total loss. This significantly reduces the number of model parameters and computational complexity.

[0097] More importantly, this invention utilizes the idea of ​​joint loss optimization to ensure that this FIR-to-IIR structural replacement does not lead to a loss of the final model performance (Loss or WER), and may even bring performance improvement. At the same time, the entire compression process is efficient and does not require retraining or fine-tuning.

[0098] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A gradient-guided model filter compression method, applicable to compressing high-order finite impulse response (FIR) filters in feedforward sequence memory networks (FSMNs), characterized in that... Includes the following steps: Step a: Obtain a trained FSMN model, which contains at least one high-order finite impulse response (FIR) filter layer, and define its weights as follows. ; Step b, determine one or more of the above. And calculate the final loss function of the FSMN model ( Regarding the above gradient ( ); Step c, based on the gradient ( Guided by the principle of [missing information], a search is conducted to determine an infinite impulse response (IIR) filter of order lower than the FIR filter, and its weights are defined as [missing information]. ; Step d, using the Replace the This generates a compressed FSMN model.

2. The gradient-guided model filter compression method according to claim 1, characterized in that, In step c, based on the gradient ( The guidance includes: Step c1. In a common representation space, compute the... Compared to The approximate error vector ( ); Step c2. Select the one that satisfies conditional .

3. The gradient-guided model filter compression method according to claim 2, characterized in that, The common representation space is the time-domain impulse response space, wherein the gradient ( ) is in The gradient is calculated in the coefficient space; the approximate error vector ( ) is the stated The impulse response and the The difference between the coefficients.

4. The gradient-guided model filter compression method according to claim 2, characterized in that, The common representation space is the frequency response space, wherein the gradient ( The frequency domain gradient is obtained through transformation. The approximate error vector ( ) is the stated frequency response With the frequency response The difference between them.

5. The gradient-guided model filter compression method according to any one of claims 2 to 4, characterized in that, The selection in step c also includes a constraint: the approximation error vector The norm must be less than a preset perturbation threshold. .

6. The gradient-guided model filter compression method according to any one of claims 2 to 4, characterized in that, Step c selects all candidates that satisfy the constraints. In the middle, make the inner product Reaching the minimum value .

7. The gradient-guided model filter compression method according to any one of claims 2 to 4, characterized in that, Step c selects all candidates that satisfy the constraints. Among them, the one with the lowest filter order .

8. The gradient-guided model filter compression method according to claim 5, characterized in that, Applied to speech recognition, natural language processing, or other sequence data processing tasks, wherein the final loss function This corresponds to word error rate (WER), cross-entropy loss, or perplexity.

9. A terminal device, characterized in that, The terminal device includes a memory and a processor, the memory storing a computer program executable on the processor, and the processor being configured to perform the method as described in any one of claims 1 to 8.

Images