NMP Unit Data Loading Method for Deep Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional Near Memory Processing (NMP) solutions face challenges in efficiently transferring data from the main memory to the vector processing unit and incurring high overhead for reading and writing operations, particularly in deep learning recommendation models like DLRM, due to limitations in data movement through the Network-on-Chip (NOC).
Innovation Solution
A computing device and method that utilize an NMP unit to perform operations on data items in memory, with instructions including addresses and operation types, allowing for efficient data processing and result storage, and reducing data movement overhead by using a flush instruction to make results visible to the processing unit and read/write instructions to manage data transfer effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transferred from main memory to vector processing unit through NOC, then data can be accessed by processing unit, but data transfer efficiency is low and overhead is high
Solution Approach 1:
The patent introduces an NMP unit as an intermediary component between the main memory and the vector processing unit. This NMP unit can directly access memory data and perform processing operations, acting as a mediator that reduces the need for frequent data transfers through the NOC bottleneck, thereby improving data transfer efficiency and reducing transfer time.
Solution Approach 2:
The patent segments the data processing function by separating the NMP unit from the main vector processing unit. The NMP unit handles specific near-memory processing tasks independently, allowing parallel operation and reducing the burden on the main processing unit, which improves overall system productivity and reduces data transfer overhead.
2Adaptability or versatility
If NMP unit performs operations on memory data, then processing flexibility is improved, but data movement overhead increases
Solution Approach 1:
The NMP unit is designed to perform processing operations directly on data residing in memory without requiring extensive data movement to and from the processing unit. This self-service capability allows the NMP unit to autonomously execute operations on memory data, improving processing flexibility while minimizing data movement overhead and energy consumption.
3Ease of operation
If read operations are performed frequently to retrieve data, then data accessibility is improved, but overhead for reading entries increases
Solution Approach 1:
The NMP unit performs preliminary processing operations directly on data in memory before the main processing unit needs to access it. This preliminary action includes preparing and organizing data in advance, which improves data accessibility when needed while reducing the overhead of reading and processing entries later.
Data Source
AI summary
The embodiments of the disclosure relate to a computing device and a method for loading data. According to the method, the first processing unit sends a first instruction to the NMP unit. The first instruction includes a first address, a plurality of second addresses, and an operation type. In response to the first instruction, the NMP unit performs operations associated with the operation type on multiple data items on the multiple second addresses of the first memory, so as to generate the operation result. The NMP unit stores the operation result to the first address of the first memory. The first processing unit issues a flush instruction to make the operation result on the first address visible to the first processing unit. The first processing unit issues a read instruction to read the operation result on the first address to the first processing unit.