Circular Line Buffer for Neural Network Vision Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The performance of artificial neural networks in processing large image files is hindered by limitations in bus transfer rates, leading to frequent data transfers and significant overhead during vision processing tasks.
Innovation Solution
A system comprising a central processing unit (CPU), an accelerator, and a communication bus, with a local memory buffer configured as a circular line buffer and a data transfer scheduler that utilizes a ping-pong buffer to manage data transfer between system memory and processing engines, minimizing off-chip data transfers and optimizing data locality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is transferred frequently between system memory and local memory buffer, then data access is maintained, but data transfer overhead increases significantly
Solution Approach 1:
The system pre-loads data into the local memory buffer before it is needed by the processing engines. The data transfer scheduler anticipates data requirements and performs transfers in advance, reducing the frequency of transfers during actual processing and thereby reducing overall transfer overhead.
Solution Approach 2:
The circular line buffer enables continuous data flow to processing engines without interruption. By maintaining a continuous supply of data in the local buffer, the system avoids frequent stops and starts in data transfer operations, reducing the overhead associated with initiating and managing discrete transfer operations.
2Productivity
If large quantities of data are transferred between storage devices and on-chip buffers, then processing capability is utilized, but bus transfer rate limitations cause performance degradation
Solution Approach 1:
The local memory buffer acts as an intermediary between the system memory and the processing engines. It decouples the processing capability from the bus transfer rate by providing a local storage area that can be rapidly accessed by processing engines without requiring continuous high-speed bus transfers.
Solution Approach 2:
The buffer is divided into multiple segments including a circular line buffer portion and a ping-pong buffer portion. This segmentation allows different portions of the buffer to serve different functions simultaneously, enabling efficient data flow management and maximizing the utilization of processing capability while working around bus transfer limitations.
3Productivity
If the local memory buffer is configured as a circular line buffer, then data locality is optimized, but buffer management complexity increases
Solution Approach 1:
The circular line buffer uses periodic indexing to manage data storage and retrieval. By using modulo arithmetic with the buffer size, the system achieves efficient circular buffering with simple wraparound logic, optimizing data locality while keeping the management mechanism relatively simple through periodic rather than complex address calculation.
4Productivity
If a ping-pong buffer is used for transferring output data, then concurrent processing and transfer are enabled, but buffer structure complexity increases
Solution Approach 1:
The ping-pong buffer extracts the data transfer function from the processing function by providing a dedicated buffer region specifically for output data. This separation allows processing engines to write output data to one buffer region while another region is being transferred to system memory, enabling concurrency without requiring complex integrated buffer management.
Data Source
AI summary
A method and system including a central processing unit (CPU), an accelerator, a communication bus and a system memory device for dynamically processing an image file are described. The accelerator includes a local memory buffer, a data transfer scheduler, and a plurality of processing engines. The data transfer scheduler is arranged to manage data transfer between the system memory device and the local memory buffer, wherein the data transfer includes data associated with the image file. The local memory buffer is configured as a circular line buffer, and the data transfer scheduler includes a ping-pong buffer for transferring output data from the one of the processing engines to the system memory device. The local memory buffer is configured to execute cross-layer usage of data associated with the image file.