Tensor Dimension Ordering for Efficient Contractions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current techniques for performing tensor operations are inefficient due to poor organization of tensor operands in memory, leading to suboptimal use of computing resources.
Innovation Solution
The system sorts and rearranges tensor dimensions based on dependent operations, optimizing storage and contraction processes to improve data access and reduce overhead, using a processor to fuse modes and arrange dimensions for efficient tensor contractions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If tensor operands are stored in conventional memory organization, then storage simplicity is maintained, but computing resource efficiency deteriorates
Solution Approach 1:
The patent applies preliminary action by sorting and reordering tensor dimensions before storage in memory. The system determines the contraction order of tensor operations in advance and arranges tensor dimensions accordingly before the actual computation begins. This pre-organization of data ensures that during contraction operations, the required elements are already in optimal positions, eliminating the need for complex runtime reorganization and improving computing resource efficiency without adding operational complexity.
Solution Approach 2:
The patent utilizes dimensionality change by transforming the conventional storage layout of tensors into a reordered layout that reflects the contraction operation structure. By changing the dimensional arrangement in memory (而非 computation), the system enables more efficient access patterns during tensor contractions. This dimensional reorganization allows processing elements to access consecutive memory locations during computation, improving cache utilization and reducing memory bandwidth requirements.
2Productivity
If tensor dimensions are not sorted, then data access simplicity is maintained, but operation overhead increases
Solution Approach 1:
The system performs dimension sorting as a preliminary action before tensor contraction operations. By analyzing the contraction indices and determining the optimal dimension order in advance, the patent reorganizes tensor elements so that contracted dimensions are properly aligned. This eliminates the need for complex runtime dimension management and reduces operation overhead, as the processing elements can directly access sorted dimensions without additional indexing or reorganization during computation.
3Ease of operation
If conventional storage methods are used, then implementation simplicity is maintained, but data access patterns become suboptimal
Solution Approach 1:
The patent implements preliminary sorting of tensor dimensions based on the contraction operation requirements. Before storing tensor operands in memory, the system determines the optimal dimension ordering that will enable efficient access during contraction. This pre-arranged storage method ensures that when processing elements need to access tensor elements during contraction, the data is already in the optimal location, improving data access efficiency without requiring complex access patterns during computation.
Solution Approach 2:
The patent changes the storage dimensionality arrangement from conventional layouts to contraction-optimized layouts. By reordering dimensions in the stored tensor representation, the system creates a memory layout where contracted dimensions are contiguous or easily accessible. This dimensional transformation in storage enables processing elements to efficiently access required elements during contraction operations, improving data access patterns without complicating the access logic.
Data Source
AI summary
Apparatuses, systems, and techniques to store tensor operands. In at least one embodiment, modes of one or more tensor operands are sorted based, at least in part, on one or more performance metrics of one or more tensor operations to be performed using said one or more tensor operands.