Intermediate Buffer Allocation for Directed-Graph Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing processors face challenges in efficiently handling and managing data storage for complex operations represented as directed graphs, particularly in tasks such as neural network processing, leading to inefficiencies in execution and resource utilization.
Innovation Solution
A processor is designed with storage, execution circuitry, and a handling unit that allocates physical storage locations for intermediate data blocks and generates execution instructions to optimize data handling, allowing flexible and efficient execution of operations in directed graphs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If intermediate data blocks are stored in external memory, then data can be preserved for later operations, but data transfer time and processing efficiency deteriorate
Solution Approach 1:
The patent introduces an intermediate buffer memory within the processing unit that acts as a mediator between external memory and processing elements. This buffer temporarily stores intermediate data blocks, allowing fast access during subsequent operations without requiring repeated external memory transfers, thus resolving the contradiction between data preservation and transfer time
Solution Approach 2:
The handling unit pre-allocates and pre-fills buffer memory locations with intermediate data blocks before they are needed by subsequent operations. This preliminary action ensures data is readily available in fast memory when required, eliminating wait time and improving processing efficiency while maintaining data integrity
2Speed
If dedicated storage locations are allocated for each intermediate data block, then data access efficiency is improved, but storage resource utilization deteriorates
Solution Approach 1:
The buffer memory is designed as a universal storage resource that can serve multiple operations and multiple intermediate data blocks. The handling unit dynamically manages this shared buffer, allocating and deallocating space based on operational needs, thus achieving fast access without requiring dedicated storage for each data block
Solution Approach 2:
The patent implements dynamic buffer management where the handling unit continuously tracks the contents and locations of intermediate data blocks in the buffer. This dynamic approach allows flexible reallocation of buffer space, optimizing both access speed and resource utilization by adapting to changing operational requirements
3Productivity
If intermediate data blocks are reused across operations, then processing efficiency is improved, but data management complexity deteriorates
Solution Approach 1:
The handling unit implements a feedback mechanism that tracks which intermediate data blocks are currently stored in the buffer and which operations require them. This feedback information allows the handling unit to intelligently manage buffer contents, reuse data blocks when appropriate, and automatically evict or overwrite data when necessary, simplifying the complexity of data reuse management
Data Source
AI summary
A processor comprising storage, execution circuitry and a handling unit configured to obtain task data that describes a task to be executed, the task comprising a plurality of operations representable as a directed graph of operations, a consumption operation comprising reading of an intermediate block of intermediate data values generated by a production operation in determining a final block of final data values based on the intermediate block. The handling unit allocates a physical storage location of the storage for storing the intermediate block, generates location data indicative of the physical storage location and generates and sends execution instructions to instruct the execution circuitry to at least partly execute the production operation to generate the intermediate block and to store the intermediate block in the physical storage location, the execution instructions comprising the location data.