Neural Network Processor Fetch Unit Data Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network processors face challenges in achieving high processing rates at a low cost, particularly in deep learning applications where the number of nodes increases, leading to inefficiencies in memory access and data transmission.
Innovation Solution
A calculation processing device with a fetch unit that includes routers with data processing mapping tables and a fetch network controller, which reconfigures the data processing mapping tables to create a software topology for efficient data processing, allowing for high processing rates while minimizing costs by optimizing data transmission and reducing the need for large buffers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of nodes in the neural network increases to improve processing capability, then the calculation power increases, but memory access inefficiency and data transmission bottlenecks worsen
Solution Approach 1:
The system segments memory into multiple banks and divides the neural network into multiple channels, where each channel can access different memory banks simultaneously. This segmentation allows parallel data retrieval operations, reducing memory access time even as the total number of nodes increases.
Solution Approach 2:
The patent introduces a channel dimension to the traditional memory access architecture. Instead of single-threaded sequential access, multiple channels can access memory banks in parallel, effectively adding a dimensional aspect to data retrieval operations and reducing bottlenecks.
2Productivity
If more memory banks are added to increase data throughput, then the data processing capacity improves, but the device complexity and cost increase
Solution Approach 1:
Each memory bank is designed to be universally accessible by multiple channels, and each channel can operate with any memory bank. This multi-functional design allows the system to achieve high throughput without requiring dedicated memory banks for each channel, reducing overall complexity.
Solution Approach 2:
The system dynamically assigns memory banks to channels based on current computational needs rather than having fixed assignments. This dynamic allocation allows flexible resource utilization and reduces the need for additional memory banks to handle peak loads.
3Device complexity
If traditional memory access methods are used to simplify the architecture, then the device complexity is reduced, but the processing rate decreases
Solution Approach 1:
Data is pre-loaded and cached in memory banks before being needed for computation. The system performs preliminary data retrieval and staging operations, so that when computation is required, data is already available in ready-state memory locations, reducing access latency without complex architecture.
4Productivity
If larger buffers are allocated to handle increased data flow, then data transmission capacity improves, but the cost and device complexity increase
Solution Approach 1:
The system maintains continuous data flow through pipelined operations where data is constantly being retrieved, processed, and transmitted without idle periods. This continuous operation maximizes the utilization of existing buffer resources, eliminating the need for larger buffers to handle interruptions or bottlenecks.
Data Source
AI summary
The present specification discloses a calculation processing device which has a high processing rate with low cost. The calculation processing device according to the present specification is a calculation processing device including a fetch unit which reads, from a memory, data required for a calculation to perform processing of a neural network, and provides the data to a calculation unit. The fetch unit may comprise: multiple routers each having a data processing mapping table in which a scheme of processing input data is recorded according to a node identifier (ID) of the input data; and a fetch network controller which controls respective data processing mapping tables of the multiple routers.