Memory Format Conversion for Direct GPU and NPU Data Exchange
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Auxiliary processors like GPUs and NPUs face difficulties in directly performing quantization and data format conversion, leading to data bottlenecks and a significant burden on the CPU when reading and writing data to DRAMs.
Innovation Solution
A memory device and system with a format converter that can convert data formats according to commands from a host device, using clipping, scaling, and bit discarding operations to handle different data types without CPU mediation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If auxiliary processors read and write data to DRAMs through the CPU for quantization and data format conversion, then data can be transferred between memory and processor, but data bottlenecks occur and CPU burden increases significantly
Solution Approach 1:
The patent introduces a format converter as an intermediary component located in the memory device that performs data format conversion between the memory and auxiliary processors. This mediator handles the quantization and format conversion tasks that previously required CPU intervention, thereby reducing CPU burden while enabling efficient data transfer between memory and processor without bottlenecking.
Solution Approach 2:
The memory device is equipped with an integrated format converter that enables self-service data format conversion. The memory device can autonomously convert data formats and perform quantization operations without requiring CPU mediation, thus improving data transfer efficiency and eliminating the need for CPU-based processing of these tasks.
2Productivity
If auxiliary processors use specialized arithmetic operations, then computational efficiency is improved, but they cannot directly perform quantization and data format conversion
Solution Approach 1:
The patent merges the format conversion functionality with the memory device architecture by integrating a format converter into the memory subsystem. This combination allows auxiliary processors to directly access converted data from memory without needing separate CPU-based conversion steps, thereby maintaining computational efficiency while gaining direct data format conversion capability.
Solution Approach 2:
The memory device is enhanced with multi-functionality by incorporating the format converter, enabling it to perform both data storage and data format conversion operations. This universal capability allows the memory device to serve multiple functions: storing data, converting data formats, and facilitating direct access to auxiliary processors, thus improving adaptability without sacrificing computational efficiency.
Data Source
AI summary
A memory device comprises a control logic circuit configured to receive a format conversion command instructing conversion of first data having a first data type into second data having a second data type, and generate format information based on the format conversion command, and a format converter configured to receive the first data and generate the second data by converting the first data based on the format information, wherein the format converter is configured to: in response to the first and second data types being different, clip the first data according to a clipping range determined based on the format information, multiply the clipped first data by a scaling factor based on the format information, discard a fractional part of a result of the multiplication, and in response to the first and second data types being the same, discard some bits of the first data according to the format information.