Convolutional Neural Network Accelerator Input Data Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing convolutional neural network calculations in heterogeneous computing systems face inefficiencies due to high host load and bandwidth requirements, primarily because the host needs to actively move and process data, perform register configurations, and wait for accelerator completion, which increases calculation time and reduces throughput.
Innovation Solution
An input data conversion device within the accelerator system that includes a scheduling unit, data moving unit, row data caching unit, and row data expansion unit, which preprocesses input data by expanding it into filter windows, reducing the need for host involvement and optimizing data movement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the host actively moves and processes data for convolutional neural network calculations, then the calculation can be performed, but the host computing power is excessively consumed and calculation time increases
Solution Approach 1:
The accelerator is designed to autonomously perform input data conversion and filtering operations without host intervention. The accelerator includes dedicated hardware units for data conversion, filter window generation, and convolution calculation, enabling it to independently complete the entire CNN calculation process including preprocessing tasks that traditionally required host involvement.
Solution Approach 2:
The patent extracts data preprocessing functions (input data conversion, filter window generation) from the host system and integrates them into the accelerator. This separation allows the host to focus on high-level task management while the accelerator handles computationally intensive operations including its own data preparation, reducing host workload and improving overall system efficiency.
2Reliability
If the host performs register configurations and data movement preparation, then the accelerator can be properly initialized, but system time loss increases
Solution Approach 1:
The accelerator performs input data conversion and filter preparation in advance during its autonomous operation cycle. The device converts input data into the required format and generates filter windows before the actual convolution calculation begins, ensuring that all preprocessing is completed preliminarily without requiring repeated host intervention during the calculation process.
Solution Approach 2:
The accelerator autonomously manages its own data preparation and register configuration through integrated control units. The control unit generates control signals that automatically coordinate data movement and preprocessing operations within the accelerator, eliminating the need for manual host-based register configuration and reducing system time loss.
3Productivity
If data is moved and processed through the host, then calculation can proceed, but bandwidth requirements increase
Solution Approach 1:
The patent extracts data preprocessing and filtering operations from the host-memory interface path and relocates them to the accelerator's internal processing units. This extraction eliminates unnecessary data movement between host and accelerator, as the accelerator performs conversions and filter generations using its own internal memory and processing units, thereby reducing bandwidth requirements.
Solution Approach 2:
The accelerator acts as an intermediary between the input data and the convolution calculation process. It receives raw input data, performs all necessary conversions and filter preparations internally, and then feeds the processed data to the convolution units, eliminating the need for the host to act as an intermediary and reducing overall data movement volume.
Data Source
AI summary
Embodiments of the present invention provide an input data conversion device, comprising: a scheduling unit, to generate a control command according to the size/dimension of the input data and the size/stride of a filter, to control actions of a data moving unit and a row data expansion unit; the data moving unit, to actively read an input data from a system storage space according to the control command of the scheduling unit; a row data caching unit, to store the input data read in; and the row data expansion unit, to read one row of the input data from the row data caching unit each time, and then to expand the row of data into one row data in different filter windows according to the sizes of the filter windows.