Multi-Precision Data Pipeline for Neural Network Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current artificial neural networks face inefficiencies in hardware utilization and computation due to reliance on CPU/GPU architectures for non-linear functions, leading to low hardware efficiency and the need for a universal architecture that can accommodate various neural network demands.
Innovation Solution
An extensible multi-precision data pipeline system and method that includes a local buffer, input tensor shaper, cascaded pipeline, and output tensor shaper to convert and route data through computational stages, supporting non-linear functions and arithmetic operations with mixed precision, allowing for efficient computation and reduced memory access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If CPU/GPU architecture is used to perform non-linear functions in artificial neural networks, then computational flexibility is maintained, but hardware utilization efficiency deteriorates
Solution Approach 1:
The system segments the data processing pipeline into distinct functional stages: input tensor shaper, cascaded pipeline with multiple computation stages, and output tensor shaper. Each stage is optimized for specific operations, allowing parallel processing while maintaining flexibility for different neural network functions.
Solution Approach 2:
The cascaded pipeline is designed with dynamic configuration capabilities, where computation stages can be selectively activated or deactivated based on the specific neural network requirements. This allows the hardware to adapt to different computational demands while maintaining high utilization efficiency.
2Speed
If ASIC hardware is used to accelerate computations, then processing speed improves, but data pipeline efficiency deteriorates due to frequent pausing for CPU/GPU interaction
Solution Approach 1:
The system merges the advantages of ASIC hardware acceleration with efficient data pipeline design by integrating multiple computation stages within a unified hardware architecture. This eliminates the need for frequent data transfer between hardware and CPU/GPU, keeping the data pipeline continuous and efficient.
Solution Approach 2:
The cascaded pipeline architecture enables continuous data flow through multiple computation stages without interruption. Intermediate results are processed immediately in subsequent stages rather than being returned to CPU/GPU, maintaining continuous useful action throughout the pipeline.
3Device complexity
If fixed-precision implementation is used, then hardware complexity is reduced, but computational precision deteriorates
Solution Approach 1:
The system dynamically changes precision parameters based on the specific computational requirements of different neural network layers and operations. This allows the hardware to maintain lower complexity while achieving high precision where needed by adjusting data representation parameters rather than using fixed high-precision formats throughout.
Data Source
AI summary
An extensible multi-precision data pipeline system, comprising, a local buffer that stores an input local data set in a local storage format, an input tensor shaper coupled to the local buffer that reads the input local data set and converts the input local data set into an input tensor data set having a tensor format of vector width N by tensor length L, a cascaded pipeline coupled to the input tensor shaper that routes the input tensor data set through at least one function stage resulting in an output tensor data set, an output tensor shaper coupled to the cascaded pipeline that converts the output tensor data set into an output local data set having the local storage format and wherein the output tensor shaper writes the output local data set to the local buffer.