Partition Channel Convolution Engine for AI Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware solutions struggle to efficiently perform convolution operations for complex artificial intelligence problems, particularly depthwise convolutions, due to complexity and inefficiencies in memory organization and data processing.
Innovation Solution
A processor system utilizing a partition channel convolution processor that processes multiple portions of a data convolution matrix concurrently by applying different weight matrices to each channel, with vector units performing multiply and add operations to enhance throughput and power efficiency, particularly for activation data with small dimensions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional optimization approaches for depthwise convolution are used, then convolution performance is improved, but hardware complexity and memory organization complexity increase significantly
Solution Approach 1:
The patent implements a universal convolution engine that can perform multiple types of convolution operations (standard convolution, depthwise convolution, separable convolution) using the same hardware infrastructure. The engine uses a unified data layout (NHWC format) and a single instruction set to handle different convolution types, eliminating the need for specialized hardware for each operation type while maintaining high performance.
Solution Approach 2:
The patent segments the convolution computation into distinct phases (data loading, computation, result storage) and uses a partitioned data layout where channels are divided into groups. This segmentation allows efficient memory access patterns and enables the same hardware to handle both standard and depthwise convolutions by simply changing which channel groups are processed together.
2Speed
If specialized hardware is used to optimize depthwise convolution, then computation speed increases, but adaptability to different matrix operations decreases
Solution Approach 1:
The convolution engine is designed as a universal processor that can execute multiple matrix operations including standard convolution, depthwise convolution, separable convolution, and general matrix multiplication. The same hardware unit responds to different operation types through a unified instruction set, achieving both high speed for specialized operations and broad adaptability for various matrix computations.
Solution Approach 2:
The patent implements dynamic configuration of the convolution engine through software control. The same hardware can be reconfigured on-the-fly to perform different operation types by loading appropriate kernels and setting control parameters. This dynamic adaptability allows the engine to optimize for speed when performing depthwise convolution while maintaining the capability to handle other matrix operations when needed.
3Productivity
If complex memory organization schemes are implemented, then convolution efficiency is improved, but ease of operation and implementation difficulty increase
Solution Approach 1:
The patent adopts a local optimization strategy where the data layout (NHWC format with channel grouping) is optimized specifically for convolution operations while maintaining compatibility with standard memory access patterns. This localized approach improves convolution efficiency without requiring global restructuring of the memory system, making it easier to implement and operate while still achieving high performance.
Data Source
AI summary
A processor system comprises two groups of registers and a hardware channel convolution processor unit. The first group of registers is configured to store data elements of channels of a portion of a convolution data matrix. Each register stores at least one data element from each channel. The second group of registers is configured to store data elements of convolution weight matrices including a separate matrix for each channel. Each register stores at least one data element from each matrix. The hardware channel convolution processor unit is configured to multiply each data element in a first and second portion of the first group of registers with a corresponding data element in the second group of registers to determine corresponding multiplication results and sum together the multiplication results for each specific channel to determine two corresponding channel convolution result data elements in a corresponding channel convolution result matrix.