Tensor Operation-Space Mapping for Lower Memory Traffic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing techniques, such as neural network and graphics processing, face challenges in efficiently handling large amounts of data due to the complexity of managing diverse operations and the need for external memory transactions.
Innovation Solution
A processor architecture is introduced with a handling unit, storage elements, and execution units that transform operation spaces into local spaces for efficient data dispatch, allowing for simplified execution units and easy addition of new operations without hardware modification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data processing techniques process considerable amounts of data using operations, then processing capability is improved, but external memory transactions increase
Solution Approach 1:
The operation space is divided into multiple portions, and each portion is transformed into operation-specific local spaces. This segmentation allows execution units to process data in smaller, more efficient chunks with localized data access patterns, reducing the need for frequent external memory transactions while maintaining high processing capability.
Solution Approach 2:
The patent transforms the global operation space into multiple operation-specific local spaces, effectively adding a dimensional transformation layer. This allows data to be accessed from local storage elements rather than external memory, reducing transaction overhead while preserving the ability to process considerable amounts of data.
2Adaptability or versatility
If diverse operations are supported in the processor, then versatility is improved, but device complexity increases
Solution Approach 1:
The handling unit implements a universal operation space transformation mechanism that can handle diverse operations through a unified approach. By transforming the global operation space into operation-specific local spaces, the system supports multiple operation types (neural network processing, graphics processing, etc.) without requiring separate specialized hardware for each operation type, thus maintaining versatility while controlling complexity.
Solution Approach 2:
The patent uses parameter transformations to adapt the processor to diverse operations. The operation space transformation involves changing coordinate systems and data layouts based on the specific operation type, allowing the same hardware architecture to efficiently support different operations by adjusting transformation parameters rather than requiring structural modifications.
3Productivity
If operation space transformation is performed for each portion, then data dispatch efficiency is improved, but processing overhead increases
Solution Approach 1:
The operation space transformation is performed in advance before data processing begins. The handling unit transforms the global operation space into operation-specific local spaces and prepares the transformed data in local storage elements beforehand. This preliminary action ensures that when execution units need data, it is already in the optimal local format, minimizing runtime overhead and maximizing data dispatch efficiency.
Data Source
AI summary
A processor and method for handling data, by obtaining operations from storage, analyzing each of the operations to determine an associated operation space, and generating at least one operation set, wherein the operations of the operation set have substantially similar operation spaces. Receiving input data in the form of a tensor; and allocate the input data, as the input to a given operation of the operation set. The input data having the predetermined input characteristics associated with the given operation. Executing the given operations using the input to produces an output with the known output characteristics. Storing in a segment being associated with an operation of the operation set, the input data; and the output associated with the operation of the operation set.