Fused Load Store Instructions for Direct Memory Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for data transfer between processing cores in massively parallel systems face bottlenecks due to high memory access latency and register pressure, particularly during large data transfers like matrix multiplications, which can slow down processing systems in data-intensive computations such as AI and deep learning.
Innovation Solution
The introduction of specialized instructions, such as fused load and store instructions (LDGSTS), which allow data to be directly transferred from global memory to shared memory without passing through processor registers or cache, reducing the need for intermediate storage and minimizing data movement through Memory Input Output Interface (MIO).
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is transferred through processor registers and cache memory, then data can be accessed by processing cores, but memory access latency and register pressure increase significantly
Solution Approach 1:
The patent extracts the data transfer path from the traditional processor register-cache-memory hierarchy by introducing dedicated memory-to-memory transfer instructions. These instructions create a direct data path between global memory and shared memory, bypassing the intermediate processor registers and cache layers that normally mediate data access. This extraction eliminates the latency penalties and register pressure associated with the conventional access path.
Solution Approach 2:
The patent introduces specialized memory transfer instructions as intermediaries that facilitate direct data movement between global memory and shared memory. These instructions act as mediators that operate independently of the processor core's register file and cache hierarchy, enabling efficient data transfer without requiring intermediate storage in processor registers or cache memory.
2Ease of operation
If data is stored in processor registers during memory transfer, then data can be manipulated by processing cores, but register pressure increases and slows down processing
Solution Approach 1:
The patent removes data from the processor register file during memory transfer operations by providing direct memory-to-memory transfer capabilities. Instead of loading data into processor registers and then storing it to shared memory, the system enables direct transfer from global memory to shared memory, extracting the data from the register-based processing path entirely.
Solution Approach 2:
The patent implements copy instructions that enable direct duplication of data from global memory to shared memory without requiring intermediate copies in processor registers. This copying mechanism operates at the memory subsystem level, bypassing the processor's register file and avoiding the register pressure that would result from traditional load-store sequences.
3Quantity of substance
If traditional load and store instructions are used for data transfer, then data can be moved between memory locations, but memory access overheads and energy consumption increase
Solution Approach 1:
The patent merges the load and store operations into a single unified memory transfer instruction. Instead of executing separate load (from global memory to registers) and store (from registers to shared memory) instructions, the system provides combined instructions that perform both operations atomically in a single step, eliminating redundant memory access overheads and reducing energy consumption.
Solution Approach 2:
The patent enables continuous data transfer between global memory and shared memory by providing dedicated memory-to-memory transfer instructions that operate independently of the processor's instruction execution cycle. This continuous transfer capability eliminates the start-stop nature of traditional load-store sequences, maintaining steady data flow and reducing overall energy consumption for bulk data transfers.
Data Source
AI summary
A technique for block data transfer is disclosed that reduces data transfer and memory access overheads and significantly reduces multiprocessor activity and energy consumption. Threads executing on a multiprocessor needing data stored in global memory can request and store the needed data in on-chip shared memory, which can be accessed by the threads multiple times. The data can be loaded from global memory and stored in shared memory using an instruction which directs the data into the shared memory without storing the data in registers and/or cache memory of the multiprocessor during the data transfer.