Tensor Layout Transposition for Low-Latency Neural Data Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware configurations for convolutional neural networks and matrix multiplication operations require different data arrangement formats, leading to inefficiencies and increased latency in data conversion tasks between different neural network frameworks.
Innovation Solution
A data conversion method and apparatus that transposes dimensions of a data combination to convert between different data arrangement formats, such as NCHW and NHWC, using a read module, transpose module, and write module to offload data conversion tasks from processing devices like accelerators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data conversion is performed using existing hardware configurations for different neural network frameworks, then compatibility between frameworks is improved, but data conversion latency increases and processing device burden increases
Solution Approach 1:
The data conversion process is segmented into distinct stages: reading data in the first format, transposing dimensions through dedicated transpose modules, and writing in the second format. This segmentation allows each module to be optimized independently and enables parallel processing of different data batches, reducing overall conversion latency while maintaining framework compatibility.
Solution Approach 2:
The patent introduces intermediate buffer memory and transpose modules as mediators between data storage and processing units. These intermediaries handle the dimension transposition operation, isolating the conversion overhead from the main processing path and preventing it from blocking subsequent processing operations, thereby reducing latency.
2Adaptability or versatility
If data conversion tasks are performed on processing devices like accelerators, then conversion functionality is provided, but device burden and processing overhead increase
Solution Approach 1:
The data conversion functionality is extracted from the main processing device and implemented as separate, dedicated conversion modules. These modules handle dimension transposition independently, removing the conversion burden from the accelerator and allowing it to focus on its primary computation tasks, thereby reducing device complexity and overhead.
Solution Approach 2:
The transpose modules are designed as universal components that can handle dimension transposition for different data formats and frameworks. By creating multi-functional conversion modules that can adapt to various neural network framework requirements, the system provides broad compatibility without requiring separate specialized hardware for each framework, reducing overall device complexity.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
A data conversion method and apparatus (200/300), an electronic device (400) and a storage medium (600) for converting dimensions of a first data combination, the first data combination includes at least one batch, dimensions of data to which each of the at least one batch corresponds includes a first dimension, a second dimension, and a third dimension. The data conversion method includes: reading n elements in the first data combination according to a first-dimension direction to obtain a first processing group, a first element to an n-th element in the first processing group are arranged according to the first-dimension direction, and n is a positive integer; performing a transpose on the first dimension and the third dimension of the first processing group to obtain a second processing group, a first element to an n-th element in the second processing group are arranged in a third-dimension direction; and writing the first element to the n-th element in the second processing group to a first storage.