Neural Processor Zero Padding via Partitioned Tensor Registers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processing methods consume significant CPU bandwidth and increase power consumption due to reliance on central processing units for extensive computing operations, particularly in configurations involving convolutional neural networks.
Innovation Solution
Implementing zero-padding in convolution operations within a neural processor circuit by dividing input tensors into partitions and using registers to manage zero-padding patterns, where active and inactive values are processed differently, and updating these patterns across compute cycles to optimize convolution operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If convolution operations are performed using CPU and main memory, then ease of configuration is improved, but CPU bandwidth consumption and power consumption increase
Solution Approach 1:
The patent divides the input tensor into multiple partitions and processes them in parallel using multiple compute circuits. This segmentation allows the system to distribute computational load across dedicated hardware units rather than relying on a single CPU, reducing overall power consumption while maintaining configurability through software updates to the neural network model.
Solution Approach 2:
The patent introduces a specialized neural processor as an intermediary between the CPU and the computation task. This dedicated hardware unit handles convolution operations, freeing the CPU from intensive computational work and reducing its bandwidth consumption and power usage, while the neural processor itself is optimized for energy-efficient matrix operations.
2Reliability
If zero-padding is implemented for all compute circuits in each kernel row, then computational completeness is improved, but processing efficiency deteriorates due to unnecessary computations
Solution Approach 1:
The patent applies zero-padding selectively to specific compute circuits based on their position in the kernel row and the current processing cycle. Instead of uniformly padding all circuits, the system determines which circuits require padding by checking if their corresponding input indices fall within valid tensor boundaries. This local differentiation ensures computational correctness where needed while avoiding unnecessary operations elsewhere, thereby improving processing efficiency.
Solution Approach 2:
The patent implements partial zero-padding by applying the padding operation only to the extent necessary for each compute circuit. The system calculates the exact range of valid input indices and applies zero-padding only to compute circuits that would otherwise access out-of-bounds memory locations. This partial action approach maintains computational completeness for valid operations while eliminating excessive padding operations that would waste processing cycles.
Data Source
AI summary
Embodiments relate to a method of efficient zero-padding in convolution. The method includes accessing a partition among a plurality of partitions of an input tensor. The input tensor is divided into the plurality of partitions in a raster-scan direction. For each row of a kernel for performing convolution on the input tensor, a register is populated with a set of values indicating a zero-padding pattern. For a compute cycle in the row of the kernel, computations associated with the convolution are performed based in part on the zero-padding pattern. After that, an updated zero-padding pattern representing a zero-padding pattern for a next cycle in the row of the kernel is generated. The set of values in the register is updated to the updated zero-padding pattern.