Private-Memory Sequential Access for SIMT Vector Data Loading
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing GPU architectures face inefficiencies in loading and storing data for multiple concurrent threads, requiring vector operations to prepare data addresses and using a vector address bus, which complicates data management.
Innovation Solution
The proposed solution involves using a scalar memory address to efficiently load and store vector data for multiple threads in a SIMT computing system. This is achieved by generating a scalar memory address based on thread block dimension information and addressing parameters, allowing K words of data to be loaded or stored from K consecutive memory addresses starting at the scalar memory address.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If vector operations are used to prepare data addresses for multiple concurrent threads, then data can be loaded and stored for multiple threads, but the system requires a vector address bus and increased device complexity
Solution Approach 1:
The patent merges the address generation function into the memory port itself, combining address calculation and data access operations into a single integrated component. This eliminates the need for separate vector address bus operations while maintaining the capability to service multiple threads, thereby reducing device complexity without sacrificing productivity
Solution Approach 2:
The memory port is designed with multi-functionality, serving both as an address generation unit and a data access interface. By making the memory port universal—capable of both generating addresses for multiple threads and performing actual data transactions—the patent eliminates the need for dedicated vector address bus hardware, thus reducing complexity while preserving efficient multi-threaded data access
2Productivity
If vector operations are used to prepare data addresses, then multiple thread data can be managed, but the data management process becomes more complex
Solution Approach 1:
The patent combines address generation and data management functions into the memory port, eliminating the need for separate vector operation units. This integration simplifies the data management process by unified the control logic, allowing the system to handle multiple thread data with reduced operational complexity while maintaining high productivity
3Productivity
If a vector address bus is used for data access, then multiple threads can access memory simultaneously, but the system efficiency is reduced due to preparatory vector operations
Solution Approach 1:
The patent performs address generation as a preliminary action within the memory port itself, using scalar registers to store thread block dimension information and addressing parameters that are pre-configured. This eliminates the need for time-consuming vector operations during runtime, as addresses are ready in advance through scalar registration, thus reducing time loss while maintaining parallel access capability
Solution Approach 2:
The patent uses scalar registers to store and replicate address information for multiple threads. By copying thread block dimension information and addressing parameters into scalar registers, the system can generate addresses for multiple threads simultaneously without requiring vector operations, thereby reducing the time required for address preparation while preserving simultaneous memory access capability
Data Source
AI summary
Processors, systems and methods are provided for thread level parallel processing where threads in a warp are concurrently executed. A method may include sending thread block dimension information and addressing parameters stored in scalar registers of a sequencer by the sequencer to a memory port, generating a scalar memory address by the memory port and based on the thread block dimension information and the addressing parameters, and loading K words of data for a warp to a vector register from K consecutive memory addresses starting at the scalar memory address by a memory interface and a memory unit. The scalar memory address may point to a storage location of a piece of data for a first thread of the warp. And K may be a warp size representing a number of threads to be concurrently processed by executing a vector instruction.