Tensor Shape Conversion for Accelerator Format Alignment

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional format conversion techniques for hardware accelerators often result in increased data size and require redundant buffers due to alignment issues, making them inefficient for processing tensor data.

Innovation Solution

A method that calculates a modified shape and format for input tensors by finding divisors to align dimensions, allowing for efficient conversion to a format compatible with hardware accelerators, which can then process data without redundant buffers and increased size, enabling quicker processing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If conventional format conversion techniques are used to meet accelerator data format requirements, then data format compatibility is improved, but data size increases and redundant buffers are required

Engineering Contradiction:

Improvedata format compatibilityVSAvoiddata size

Core Design Contradiction:

Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

The patent changes the data representation parameters by converting tensors from NHWC format with original shape to NCHW format with modified shape, where the shape transformation uses mathematical relationships (S1, S2, S3 divisors) to remap dimensions. This parameter change achieves format compatibility without increasing data quantity or requiring redundant buffers

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If conventional format conversion techniques are used to meet accelerator data format requirements, then data format compatibility is improved, but memory buffer requirements increase

Engineering Contradiction:

Improvedata format compatibilityVSAvoidbuffer memory size

Core Design Contradiction:

Adaptability or versatilityVSArea of stationary object

Solution Approach 1:

The patent transforms the shape parameters of input tensors using divisor relationships (S1, S2, S3) to convert from NHWC to NCHW format. This parameter transformation allows the data to be reorganized in-place within the same buffer memory, eliminating the need for additional redundant buffers while achieving accelerator compatibility

Inventive Principle:

Principle #35Parameter changes

3Productivity

If n-dimension tensor shape is converted to modified shape format, then processing efficiency is improved, but computational complexity increases

Engineering Contradiction:

Improveprocessing efficiencyVSAvoidshape conversion complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent segments the n-dimension shape conversion into discrete divisor calculations (S1, S2, S3) where each dimension transformation is handled independently through systematic division operations. This segmentation makes the complex shape conversion manageable and systematic, enabling efficient implementation despite the increased computational steps

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS20230153318A1Shape and data format conversion for accelerators

Publication Date: 2023.05.18 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

A method for converting a shape and a format of tensor data to meet a specific data format of a hardware accelerator is provided. The method receives input tensors L1 and L2, each being constants having a data format of < X x Y x Z >, and each further having an n-dimension input tensor shape as <Xn x Xn-1 x Xn-2 x ... x X1>. The method stores input tensor shape. The method calculates an n-dimension modified shape of the input tensors by (a) setting a largest divisor of (Xn x Xn-1 x...x X1 ) ≤ L1 to S1, (b) setting a largest divisor of ((Xn x Xn-1 x...x X1 ) / S1) ≤ L2 to S2, (c) setting (((Xn x Xn-1 x... x X1 ) / (S1 x S2)) to S3, and (d) returning the n-dimension modified shape as < S3 x S2 x S1>.