Execution Unit Dataflow Without Intermediate Memory Writes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in efficiently handling and processing large amounts of data for neural network and graphics processing tasks, particularly in managing data operations and optimizing hardware for diverse neural network structures without requiring complex transforms and external memory transactions.
Innovation Solution
A processing unit with a handling unit and execution unit that parses task data into invocation data, maps operations to non-local storage, and processes data without storing intermediate results, allowing for simplified execution and easy addition of new operations by transforming operation spaces to section-specific local spaces.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is processed using traditional methods with external memory transactions, then data processing capability is maintained, but device complexity and processing time increase
Solution Approach 1:
The patent extracts the data processing operations from external memory transactions and relocates them to the execution unit's internal resources. The execution unit performs operations on data directly from non-local storage without requiring complex external memory transactions, thereby reducing device complexity while maintaining processing capability.
Solution Approach 2:
The patent segments the data processing task into operations that can be performed within the execution unit's internal architecture. By dividing the processing into manageable segments that utilize the execution unit's internal resources rather than external memory, the system reduces overall complexity while maintaining productivity.
2Productivity
If complex transforms are used to handle data operations, then data processing capability is maintained, but processing time increases
Solution Approach 1:
The patent extracts complex transform operations from the data processing workflow and eliminates them by performing operations directly on data in non-local storage. This extraction of unnecessary transforms reduces processing time while preserving the essential data processing capability.
Solution Approach 2:
The patent performs preliminary actions by pre-loading data into non-local storage and preparing operation invocations in advance. This allows the execution unit to process data more efficiently without requiring time-consuming transforms during the actual processing operation.
3Productivity
If hardware is optimized for specific neural network structures, then processing performance improves, but adaptability to diverse structures decreases
Solution Approach 1:
The patent implements universality by designing the execution unit to handle diverse neural network structures through a unified architecture. The execution unit can process various operation types (convolution, pooling, normalization, etc.) on different data structures without requiring hardware modifications, thereby maintaining high processing performance across diverse applications.
Solution Approach 2:
The patent utilizes parameter changes to adapt the execution unit to different neural network structures. By modifying operational parameters such as operation type, data format, and processing mode, the same hardware can efficiently handle diverse neural network architectures without physical reconfiguration.
4Reliability
If intermediate results are stored in non-local storage, then data availability is improved, but storage requirements and processing complexity increase
Solution Approach 1:
The patent extracts the storage function from complex storage management systems and simplifies it to basic non-local storage. By taking out the complexity of storage management and using simple, direct storage mechanisms, the system maintains data availability while reducing overall processing complexity.
Solution Approach 2:
The patent implements self-service by allowing the execution unit to directly access and manage data in non-local storage without requiring complex storage management overhead. The execution unit autonomously handles data retrieval and processing, eliminating the need for complex centralized storage management systems.
Data Source
AI summary
A data processing unit is provided comprising a handling unit configured to send invocation data including the first and second operation to an execution unit to cause the execution unit to process the invocation data. The execution unit processes the data by: obtaining data from a non-local storage based on a logical source pipe of a first operation, performing the first and a second operation for portions of the data received from the logical source pipe. In response to the output of the first operation and input of the second operation referring to a logical forwarding pipe, the execution unit performs processing for a portion of the data for the first and second operation without storing the output data of the first operation in the non-local storage.