Sparse Accelerator With Internal Sparsity Masks for Non-2-D Convolutions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine-learning accelerators struggle to efficiently execute non-2-D convolution operations due to increased processing latency, size, and power consumption when offloading layers to different hardware or including dedicated hardware, which degrades performance metrics like TOPs/mm2 and TOPs/W.
Innovation Solution
A sparse accelerator that generates sparsity information internally and manipulates sparsity bit masks to execute non-2-D convolution operations, such as depthwise and grouped convolutions, reducing the need for external data retrieval and dedicated hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If non-2-D convolution operations are offloaded to different hardware or dedicated hardware is included, then operational capability is improved, but processing latency increases and performance metrics (TOPs/mm2, TOPs/W) deteriorate
Solution Approach 1:
The sparse accelerator is designed to execute multiple types of convolution operations (2-D, depthwise, grouped, dilated) using a unified hardware architecture. The same convolution execution units can be configured through programmable parameters to perform different operation types, eliminating the need for dedicated hardware for each operation type and avoiding the performance degradation associated with hardware offloading.
2Adaptability or versatility
If non-2-D convolution operations are offloaded to different hardware or dedicated hardware is included, then operational capability is improved, but device size and power consumption increase
Solution Approach 1:
The accelerator uses a universal hardware core that can be reconfigured via software to perform different convolution operations. This eliminates the need for multiple dedicated hardware blocks, thereby reducing device size and power consumption while maintaining the ability to execute diverse operations including depthwise, grouped, and dilated convolutions.
Solution Approach 2:
The accelerator employs dynamically reconfigurable parameters and control logic that allow the same hardware resources to adapt to different operation types. This dynamic reconfiguration enables operational versatility without requiring static dedicated hardware for each operation, thus reducing overall power consumption and device footprint.
3Ease of operation
If sparsity information is retrieved from external sources, then sparsity processing is enabled, but bandwidth requirements and processing overhead increase
Solution Approach 1:
The sparse accelerator generates sparsity information internally within its own architecture through dedicated sparsity generation circuitry. This self-service approach eliminates the need to retrieve sparsity information from external memory or sources, thereby reducing bandwidth requirements and processing overhead while enabling efficient sparsity-aware computation.
Data Source
AI summary
Methods, apparatus, systems and articles of manufacture are disclosed to perform machine-learning model operations on sparse accelerators. An example apparatus includes first circuitry, second circuitry to generate sparsity data based on an acceleration operation, and third circuitry to instruct one or more data buffers to provide at least one of activation data or weight data based on the sparsity data to the first circuitry, the first circuitry to execute the acceleration operation based on the at least one of the activation data or the weight data.