AI Accelerator Vector Processing for Sparse 1D Convolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for speech signal processing using convolutional neural networks face inefficiencies due to the need for high-precision floating-point computations and the presence of zeros in speech and weight data, leading to unnecessary computations when applying stride and/or dilation rates.
Innovation Solution
A one-dimensional neural network accelerator employs a vector processing circuit to partition data into sub-blocks, generating addresses to access these sub-blocks and efficiently remove unnecessary computations by rearranging input and weight data, using a floating-point format to perform vector processing for convolution computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If stride and/or dilation rate are applied in convolution computation, then the receptive field is expanded and spatial downsampling is achieved, but unnecessary computations are generated due to zeros in the expanded data structure
Solution Approach 1:
The patent extracts and removes zero elements from the input data before convolution computation. The data aligner identifies and eliminates zeros that are generated by stride and dilation rate operations, so that only non-zero elements are processed by the convolution computation units, thereby eliminating unnecessary computations.
Solution Approach 2:
The patent performs preliminary data alignment and zero removal before the actual convolution computation. The data aligner pre-processes the input data by removing zeros and aligning non-zero elements to appropriate positions, so that the subsequent convolution computation operates only on necessary data elements.
2Measurement precision
If floating-point data format is used for speech data and weight data, then high precision is maintained, but computation complexity increases compared to integer formats
Solution Approach 1:
The patent employs dedicated floating-point computation units within the AI accelerator that are self-optimized for floating-point operations. These specialized units handle floating-point arithmetic efficiently without requiring complex software management, allowing high-precision speech data processing while maintaining hardware efficiency.
3Loss of energy
If data is rearranged into sub-blocks for processing, then unnecessary computations are removed, but additional processing steps are required
Solution Approach 1:
The patent segments the input data into multiple sub-blocks that are processed independently by parallel computation units. The data aligner divides the input data stream into sub-blocks, removes zeros from each sub-block, and feeds them to parallel convolution computation units, enabling efficient power utilization while reducing overall processing complexity through parallelization.
Data Source
AI summary
Provided are an artificial intelligence (AI) accelerator, a system-on-chip (SoC) and electronic device including the AI accelerator, and an operating method of the AI accelerator. The AI accelerator includes a vector processing circuit (VPC) configured to, according to commands, perform a rearrangement on input data into sub-blocks, configured to generate addresses for the rearranged sub-blocks, and configured to perform vector processing for a convolution computation.