Sparse Tensor Accelerator Architecture for In-Line Format Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning (ML) processing engines face inefficiencies in handling a mix of sparse and dense tensors, leading to increased processing time, power consumption, and storage requirements, with challenges in converting between tensor forms.
Innovation Solution
A processor architecture with adapter input and output circuits that efficiently process sparse and dense tensors by in-line decompression and compression, aligning tensor samples with locations, and utilizing arithmetic logic units (ALUs) for calculations, allowing parallel processing and flexible tensor format conversion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If sparse tensor compression is used to reduce storage size and data footprint, then storage requirements and access bandwidth are reduced, but processing time increases due to the need for decompression and alignment operations
Solution Approach 1:
The patent pre-calculates and stores location mapping information that enables direct computation of output locations from input locations. This preliminary action avoids the need for complex runtime decompression and alignment operations, thereby reducing processing time while maintaining compressed sparse tensor storage benefits
Solution Approach 2:
The patent introduces an intermediary location mapping mechanism that bridges sparse tensor locations and output positions. This intermediary structure enables efficient indirect addressing and alignment without requiring full decompression, thus resolving the contradiction between compressed storage and processing speed
2Ease of operation
If dense tensor form is used to simplify processing operations, then processing operations become cleaner and more straightforward, but storage size and access bandwidth requirements increase
Solution Approach 1:
The patent segments the tensor processing into location handling and value computation components. By separating location management from actual arithmetic operations, the system can work with compressed sparse tensors using simplified operations, achieving both storage efficiency and operational simplicity
Solution Approach 2:
The patent changes the representation parameters of tensors from dense to sparse format, using location-value pairs instead of full arrays. This parameter change reduces storage size while the system maintains ease of operation through specialized processing circuits designed for sparse tensor operations
3Adaptability or versatility
If in-line decompression and compression operations are performed to enable flexible tensor format conversion, then adaptability between sparse and dense forms is improved, but device complexity and processing overhead increase
Solution Approach 1:
The patent designs processing circuits that can handle both sparse and dense tensor formats using the same fundamental operation pipeline. The adapter circuits provide multi-functionality by supporting format conversion, location processing, and arithmetic operations in a unified architecture, reducing overall device complexity despite the added adaptability
Data Source
AI summary
In an example, a processor for machine learning calculations is described. An adapter input circuit is operable to receive an input tensor. The adapter input circuit includes channels. A first channel of the channels is operable to process samples of the input tensor to generate pre-processed samples and to obtain locations of the samples. A location processor, coupled to the first channel, is operable to determine output locations in response to the locations. An arithmetic logic unit (ALU), coupled to the channels, is operable to calculate output samples from the pre-processed samples. An adapter output circuit, coupled to the location processor and the ALU, operable to process the output locations and the output samples to generate an output tensor.