Sparse Matrix Accelerator for Signal Transform Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Communicating large amounts of data between a primary processor and a neural network co-processor for neural network computing is inefficient and slow, particularly due to the need for transforming data between time-domain and frequency-domain representations using conventional signal processing methods.
Innovation Solution
Implementing a sparse matrix accelerator that performs mathematical transforms natively, utilizing matrix permutation operations to re-order input vectors and decompose transforms into smaller sub-transforms, thereby reducing computational complexity and enabling efficient processing of sparse data structures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional signal processing methods are used to transform data between time-domain and frequency-domain, then the transforms can be performed, but the data transfer requirements are large and the processing is slow
Solution Approach 1:
The patent segments the transform processing into two distinct parts: (1) sparse matrix multiplication operations performed on the neural network co-processor, and (2) final inverse transform operations performed on the primary processor. This segmentation allows the majority of computational work to be done locally on the co-processor with minimal data transfer, while the primary processor only handles the final reconstruction step, thereby reducing overall data transfer volume and improving processing speed
Solution Approach 2:
The patent introduces sparse matrix representation as an intermediary data structure that enables efficient computation. By representing transform matrices in sparse format and using specialized sparse matrix multiplication algorithms, the system can perform transform operations with reduced computational complexity (O(N log2 N) vs O(N^2)) and minimal data movement between processors
2Productivity
If data is communicated between primary processor and neural network co-processor for neural network computing, then the computing can be performed, but the communication is inefficient and slow
Solution Approach 1:
The patent extracts the transform computation functionality from the primary processor and relocates it to the neural network co-processor. By implementing sparse matrix multiplication units directly on the co-processor, the system eliminates the need to transfer large amounts of data back and forth between processors, keeping the computational workload localized and minimizing communication overhead
Solution Approach 2:
The patent changes the computational dimension by introducing a new processing paradigm: sparse matrix multiplication. This approach transforms the traditional dense matrix operations into sparse operations that can be performed efficiently on the co-processor with specialized hardware, adding a new dimension of computational efficiency to the system architecture
Data Source
AI summary
Methods and apparatus for accelerating transforms via sparse matrix operations. Conventional processing architectures use bit-reversed addressing and a “butterfly” operation to perform digital signal processing techniques (such as the FFT, DFT, DCT, etc.). However, bit-reversed addressing may also be performed as a single sparse matrix permutation; similarly, butterfly operations may also be represented as a number of multi-matrix multiplications. Exemplary sparse matrix processors can perform these operations locally with great efficiency. Importantly, instead of sending data from a machine learning (ML) co-processor to a DSP to perform signal processing functions (and then back to the ML co-processor); the entire sequence may be performed on a sparse ML processor. This may greatly improve system power consumption and may entirely obviate the need for a separate DSP in certain (e.g., embedded) systems.