Audio Feature Hyper-Block Reorientation for Low-Complexity DNNs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning-based audio processing methods are computationally and memory-intensive, making them unsuitable for deployment on devices with limited resources such as DSPs and embedded systems, and fail to utilize depth-wise correlations and TF feature correlations effectively.
Innovation Solution
A method involving channelwise feature reorientation and merging with neighbouring features to reshape input features into a hyper-block format, followed by neural network processing, which reduces complexity and leverages depth-wise correlations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deep learning-based audio processing methods are used, then processing performance is improved, but computational complexity and memory requirements increase
Solution Approach 1:
The input feature map is divided into multiple non-overlapping local regions along the frequency dimension. Each region is processed independently by separate neural network branches, allowing the model to handle complex audio processing tasks while reducing the computational burden on each individual branch compared to processing the entire feature map as a whole.
2Reliability
If deep learning-based audio processing methods are used, then processing performance is improved, but memory requirements increase
Solution Approach 1:
By segmenting the feature map into local frequency regions and processing them independently, the memory required for storing intermediate activations and gradients is significantly reduced compared to processing the entire feature map in a single unified network, enabling deployment on devices with limited memory resources.
3Device complexity
If channelwise subband splitting is used, then computational complexity is reduced, but depth-wise correlations and TF feature correlations are not utilized
Solution Approach 1:
The patent introduces a new dimension (depth dimension) within each local frequency region to preserve and exploit depth-wise correlations. By stacking feature maps along this depth dimension and applying convolutional operations, the model can capture correlations across different feature channels while maintaining the computational efficiency of local region processing.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
There is disclosed an apparatus (10) for processing an input information signal (20), comprising: a feature extractor (100) for extracting a set of features (102) from the input information signal (20), the set of features (102) having a first shape, a feature block generator (200, CR1) configured to reshape the set of features (102) from a first shape to a second shape by: segmenting (210) the set of features (102) into at least 3 feature subsets (212, 212a-212h), each feature subset (212, 212a-212h) having a specific shape, performing a subsampling (220, 220a, 220b) of the feature subsets (212) to generate a plurality of feature blocks (222, 222a-222d) by stacking a selected set of the subsampled feature subsets feature subsets (212, 212a-212h) along a first dimension, creating (230) a feature hyper-block (332) of the second shape by stacking the plurality of feature blocks (222, 222a-222d) along a second dimension, a neural network processor (300, D1) configured to process the feature hyper block (332) to obtain a first result (380); and an output post-processor (P2, 700) configured to post-process the result (380) to construct an output information signal (40) and/or to output a classification result.