Neural Network Accelerator Data Classification for Lower Input Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In neural networks, the varying types and sizes of data lead to increased latency in computations due to the need for different data processing requirements, which existing accelerators fail to efficiently manage.
Innovation Solution
A data processing system that includes a host and an accelerator, where input data is classified based on a data classification criterion, with the host transmitting data identification information to the accelerator, which then classifies and distributes the data to corresponding buffers for pre-processing and computation, allowing for precision transformation and feedback of result data for further processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If different types and sizes of data are applied as various types of neural networks are used, then the neural network can handle diverse computational tasks, but the time taken to perform actual computations increases
Solution Approach 1:
The accelerator divides the data processing pipeline into separate functional units: a data classification unit that categorizes input data into feature data, parameters, and biases; a pre-processing unit that performs precision transformation; and a computation unit that executes calculations. This segmentation allows each unit to specialize in specific data types and operations, reducing overall computation time while maintaining versatility across different neural network configurations
Solution Approach 2:
The system performs data classification and pre-processing operations before the main computation phase. The data classification unit categorizes incoming data in advance, and the pre-processing unit transforms precision levels prior to computation. These preliminary actions prepare data in advance, eliminating the need for time-consuming processing during the critical computation phase, thus reducing total computation time while handling diverse data types
2Loss of time
If a simple buffer is used to reduce latency for data transmission between host and accelerator, then data transmission latency is reduced, but the system cannot efficiently manage varying data types and sizes
Solution Approach 1:
The buffer is designed as a multi-functional component that can store and manage multiple types of data (feature data, parameters, biases) with different sizes and precision requirements. The buffer works in conjunction with the data classification unit that directs different data types to appropriate buffer regions, enabling the buffer to handle diverse data management needs while maintaining fast access speeds for reducing transmission latency
Data Source
AI summary
A data processing system includes a host and an accelerator. The host transmits, to the accelerator, input data together with data identification information based on a data classification criterion. The accelerator classifies the input data as any one of feature data, a parameter, and a bias based on the data identification information when the input data is received from the host, distributes the input data, performs pre-processing on the feature data, and outputs computed result data to the host or feeds the result data back so that computation processing is performed on the result data again.