Tensor Tile Walking for Convolution Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Performing convolutions on large input tensors purely in software or firmware is resource prohibitive due to the number of execution cycles and time required to process the entire tensor.
Innovation Solution
A hybrid firmware-hardware technique that partitions the input tensor into multiple tiles and uses a combination of firmware and hardware acceleration to perform per-tile convolution operations through tile walking, with the complex part of the tile walk implemented in firmware and the detailed cell-value-based tile walking performed by a hardware accelerator.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If software or firmware is used to perform convolution operations on large input tensors, then implementation flexibility is maintained, but processing time and resource requirements become prohibitive
Solution Approach 1:
The patent divides the input tensor into multiple smaller tiles and processes them individually through tile walking. This segmentation reduces the computational burden on any single processing unit, enabling parallelization and reducing overall processing time while managing resource requirements through systematic division of work.
Solution Approach 2:
The patent introduces a tile walking mechanism as an intermediary layer between the firmware controller and hardware accelerator. This mediator coordinates the interaction between software control logic and hardware computation units, enabling efficient data transfer and processing through structured tile-based iteration patterns.
2Loss of time
If hardware acceleration is used for tile walking, then processing time is reduced, but implementation complexity increases
Solution Approach 1:
By segmenting the tensor into tiles and using hardware acceleration for per-tile operations, the patent reduces processing time through parallel computation while managing complexity through modular architecture where each tile can be processed independently using standardized hardware routines.
Solution Approach 2:
The patent performs preliminary actions by pre-dividing the tensor into tiles and pre-configuring the tile walking pattern before executing the convolution operation. This preparation enables the hardware accelerator to execute processing operations more efficiently without requiring complex real-time decision-making during computation.
Data Source
AI summary
An example apparatus to perform a convolution on an input tensor includes a parameters generator to: generate a horizontal hardware execution parameter for a horizontal dimension of the input tensor based on a kernel parameter and a layer parameter; and generate a vertical hardware execution parameter for a vertical dimension of the input tensor based on the kernel parameter and the layer parameter; an accelerator interface to configure a hardware accelerator circuitry based on the horizontal and vertical hardware execution parameters; a horizontal Iterator controller to determine when the hardware accelerator circuitry completes the first horizontal iteration of the convolution; and a vertical Iterator controller to determine when the hardware accelerator circuitry completes the first vertical iteration of the convolution.