Irregular Pruning for ASR Neural Network Weight Matrices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conformer models for automatic speech recognition (ASR) require significant computational resources and power to process audio corpora, posing a challenge for efficient acceleration on hardware devices like GPUs.
Innovation Solution
Implementing an irregular pruning method that divides the weight matrix of neural network layers into blocks, selecting a predetermined percentage of weights for block-wise pruning, and setting them to zero, thereby reducing computational and memory requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a conformer model is used to achieve state-of-the-art ASR performance, then recognition accuracy is improved, but computational resource requirements and power consumption increase significantly
Solution Approach 1:
The weight matrix is divided into multiple blocks, allowing selective pruning of specific blocks while preserving others. This segmentation enables partial reduction of computational resources while maintaining critical functionality for accurate speech recognition
Solution Approach 2:
Different pruning ratios are applied to different blocks of the weight matrix based on their importance. Critical blocks retain more weights to maintain recognition accuracy, while less critical blocks are pruned more aggressively to reduce power consumption
2Adaptability or versatility
If a conformer model is used to capture both local and global context, then modeling capability is improved, but device complexity and computational overhead increase
Solution Approach 1:
The weight matrix is divided into multiple blocks, allowing selective pruning of specific blocks while preserving others. This segmentation enables partial reduction of computational resources while maintaining critical functionality for accurate speech recognition
Solution Approach 2:
Different pruning ratios are applied to different blocks of the weight matrix based on their importance. Critical blocks retain more weights to maintain recognition accuracy, while less critical blocks are pruned more aggressively to reduce power consumption
3Measurement precision
If full weight matrices are maintained for all neural network layers, then model accuracy is preserved, but memory requirements and inference time increase
Solution Approach 1:
The weight matrix is divided into multiple blocks, allowing selective pruning of specific blocks while preserving others. This segmentation enables partial reduction of computational resources while maintaining critical functionality for accurate speech recognition
Solution Approach 2:
Different pruning ratios are applied to different blocks of the weight matrix based on their importance. Critical blocks retain more weights to maintain recognition accuracy, while less critical blocks are pruned more aggressively to reduce power consumption
Data Source
AI summary
A method and an apparatus for automatic speech recognition are provided. The method includes: generating a weight matrix for a layer of a plurality of layers in a neural network; dividing the weight matrix into a plurality of blocks, each block including a plurality of weights; selecting a pre-determined percentage of weights from at least one block for block-wise pruning; and generating a block-wise pruned weight matrix by setting the pre-determined percentage of weights selected from the at least one block to zero. The weight matrix includes a set of weights associated with the layer, the plurality of layers includes a first layer receiving a first input associated with one or more audio feature sequences, and the plurality of layers are executed on one or more processors. The method efficiently accelerates model inference using irregular pruning.