Dynamically-sized data structures on reconfigurable processors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional compilers and Von Neumann architectures are inadequate for efficiently executing machine learning and artificial intelligence applications, which require massively parallel computations and dynamic data structures, as they lack support for asynchronous data exchange between parallel threads and cannot handle dynamically-sized data outputs effectively.
Innovation Solution
A data processing system utilizing a reconfigurable processor with coarse-grained reconfigurable units that can implement dynamically-sized data structures by generating control data indicative of output size, allowing operations to process inputs based on this data, and using control circuitry to manage memory allocation and data flow efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional compilers and Von Neumann architectures are used, then general-purpose computing is maintained, but efficiently executing machine learning and AI applications with dynamically-sized data structures is not achieved
Solution Approach 1:
The patent implements dynamically-sized data structures on the data flow architecture by allowing buffer allocations to be determined at runtime rather than compile-time. The system uses control data to indicate actual output sizes of operations, enabling buffers to be dynamically allocated to match the true size of data structures during execution, thus achieving both efficiency and adaptability for ML/AI workloads
Solution Approach 2:
The invention changes the parameter of data structure size from static (compile-time determined) to dynamic (runtime determined). By introducing control data that carries size information and using this to allocate buffers dynamically, the system can adapt to varying data sizes in ML/AI applications while maintaining efficient parallel execution
2Device complexity
If fixed-size buffers are allocated for all operations, then memory allocation is simplified, but garbage data issues occur and computational efficiency is reduced
Solution Approach 1:
The patent introduces a feedback mechanism where the actual output size of each operation is determined and communicated back to the memory allocation system. Control data carries this size information, which is then used to allocate appropriately-sized buffers, preventing garbage data issues while maintaining efficient computation by avoiding over-allocation
Solution Approach 2:
The system performs preliminary determination of output sizes before buffer allocation by using control data to indicate the actual size of operation outputs. This preliminary action allows buffers to be allocated with the exact size needed, eliminating garbage data problems and optimizing memory usage before the actual computation proceeds
3Productivity
If dynamically-sized data structures are supported, then ML/AI application efficiency is improved, but control circuitry and data flow management complexity increases
Solution Approach 1:
The patent implements a universal control data structure that can carry size information for any operation output, making the control mechanism applicable across all operations in the data flow graph. This universal approach manages complexity by using a single, standardized control mechanism rather than operation-specific control logic for each data structure
Data Source
AI summary
A data processing system for implementing operations that generate a dynamically-sized output is presented. The data processing system includes a reconfigurable processor that is configured to implement a first operation, a second operation, a recording unit, and a control unit. The first operation generates an output, wherein a size of the output is unknown during a configuration phase. The second operation receives the output of the first operation as an input. The recording unit generates control data that is indicative of the size of the output. The control unit that provides the control data to the second operation, wherein the second operation processes the input based on the control data.