Shared memory access method in GPU and system thereof

Abstract

The application provides a shared memory access method in a GPU and a system thereof. The method comprises: receiving shared memory read instructions issued by multiple threads; in response to the shared memory read instructions, converting original read addresses corresponding to each thread into access addresses according to a predetermined address mapping rule, so that access addresses of multiple data elements corresponding to the same thread are discontinuous; reading the data elements with the same element serial number from multiple storage banks of the shared memory in parallel according to the access addresses; and writing the read data elements into the target register file address space. The shared memory access method in the GPU and the system thereof effectively realize data layout unification of the shared memory and the register file, thereby eliminating or significantly reducing storage bank conflicts from the source, improving data transfer efficiency, and maintaining high compatibility of existing software.

Description

Technical Field

[0001] This invention relates to the field of GPU and parallel computing technology, and more specifically, to a method and system for accessing shared memory in a GPU. Background Technology With the advent of artificial intelligence (AI), deep learning, and the era of large models, graphics processing units (GPUs) have become the core accelerators for high-performance computing. The scale of modern AI model parameters has jumped from billions to trillions, and the demand for computational throughput and storage bandwidth during training and inference phases has grown exponentially. During model inference and training, frequent reads and writes of intermediate data by numerous operators have gradually made on-chip storage bandwidth one of the performance bottlenecks. In the GPU on-chip storage system, register files, shared memory (SHM), and global memory constitute key storage layers. Among them, shared memory, as on-chip static random access memory (SRAM) for high-speed data exchange between threads within a thread block, has extremely low access latency and high access bandwidth. Specifically, under ideal conditions without memory conflicts, a single shared memory access can typically be completed within a single memory access cycle, which physically corresponds to one or a few clock cycles. Therefore, shared memory has become a key hardware resource for optimizing memory-intensive computational kernels (including but not limited to matrix multiplication, convolution operations, and the self-attention mechanism in Transformer models). The memory access cycle refers to the logical access time window occupied by a thread bundle when executing a shared memory access instruction, which triggers and completes a shared memory read / write operation. Within this time window, multiple threads within the thread bundle initiate shared memory access requests in parallel. The clock cycle is the smallest unit of time defined by the global clock signal of the processor or storage subsystem, used to measure the length of time required for a synchronous operation or state update in the hardware logic circuit.

[0002] In actual hardware implementations, to improve chip yield and reduce manufacturing costs, large-capacity SRAM is usually not implemented as a single unit, but rather composed of multiple smaller SRAM cells. However, the actual performance of shared memory is often limited by SRAM bank conflicts. Shared memory is typically composed of 32 or more independent small-capacity SRAM modules (called banks), each bank usually has only a limited number of read / write ports, typically a single-port or equivalent single-port structure. This means that each bank can only support one 32-bit or 64-bit data access per clock cycle. When multiple threads within the same warp simultaneously access different addresses in the same bank, because the bank cannot serve multiple different address requests simultaneously in one clock cycle, the hardware must serialize these accesses. This causes the actual number of memory access cycles for a single shared memory instruction to increase exponentially; that is, a single SIMT (Single Instruction, Multiple Thread) instruction requires multiple SRAM memory access cycles to complete the data reading for all threads. This conflict is particularly prominent in large AI model computations, because typical computational patterns such as matrix multiplication (GEMM) and self-attention mechanisms require repeated loading and reuse of data blocks (tiles) in shared memory. Any extra memory access cycles will be repeatedly amplified, becoming the main performance bottleneck.

[0003] The root cause of this problem lies in the long-standing data layout mismatch between shared memory and register files. For example... Figure 1 As shown, in traditional designs, shared memory inherits the "horizontally compact" layout of the early burst transport protocol. Figure 1 This diagram illustrates the data element layout in a scenario where each thread within a thread bundle needs to move four data elements between the register file and shared memory. In this mode, each thread loads four consecutive 32-bit data elements (e1 to e4) at a time. Because the shared memory uses a traditional horizontally contiguous and tightly packed layout, e1, e2, e3, and e4 of the same thread are stored sequentially and consecutively in the address space. For example, thread 0's e1 to e4 occupy consecutive addresses, followed immediately by thread 1. Different colors in the diagram represent data elements from different threads (thd0, thd1, etc.), and Addr0-3 represent the addresses in the shared memory. Each column corresponds to an SRAM bank. When multiple threads within a thread bundle simultaneously read data elements with the same index, these data elements are actually distributed in the same column in the shared memory. The pattern of low-order address changes determines that they are concentrated in a few banks, leading to bank conflicts.

[0004] To support SIMT parallel computing, register files employ a completely different "vertical arrangement" layout, such as... Figure 2As shown, data elements with the same element number from all threads within a warp are stored in the same register row or the same set of register addresses. This allows the Arithmetic Logic Unit (ALU) to simultaneously read the corresponding elements of the entire warp within one clock cycle and complete vectorized calculations.

[0005] It is this fundamental difference in layout that causes the hardware to have to read shared memory multiple times and then reassemble it in an internal buffer when loading data from shared memory to registers, before finally writing the data correctly to the corresponding line in the register file. For example... Figure 3 As shown, in a traditional shared memory layout, four data elements—thd0 e1, thd2 e1, thd4 e1, and thd6 e1—are located at different addresses within the same memory bank, yet they need to be written to the same line in the register file simultaneously. Since each bank only supports one access per clock cycle, the hardware must read these four elements sequentially from the bank over four clock cycles (similar for other threads with the same element number). Even without any other concurrent memory access conflicts, this fixed multi-clock-cycle overhead caused by layout mismatch is unavoidable. Conversely, writing data from registers to shared memory incurs the same "transposition" cost. This additional memory access cycle overhead introduced by shared memory bank conflicts is amplified significantly in large AI model computations because shared memory often exhibits the strongest data locality and highest reuse rate; any increase in memory access latency directly constrains overall computational efficiency.

[0006] In existing technologies, the industry has attempted various methods to mitigate shared memory bank conflicts at both the software and hardware levels. These include inserting padding into shared memory to alter address mapping patterns, manually transposing or rearranging data at the software level, and optimizing matrix layout and broadcast for Tensor Cores. For example, WO2022253075A1 (A compilation method and related apparatus) uses a compiler to statically analyze the warp and shared memory access relationship at the intermediate representation level and adjust the mapping order to reduce bank conflicts; US20210124582A1 (Techniques for efficiently transferring data to a processor) optimizes shared memory access by rearranging data to different banks and combining fused loading instructions; and US8108625B1 (Shared memory with parallel access and access conflict resolution mechanism) dynamically resolves conflicts through hardware conflict detection and serialization mechanisms without modifying the data layout. These methods can improve performance in specific scenarios, but their general applicability is usually limited, and some methods require programmer intervention, resulting in high code complexity. More importantly, these solutions fail to address the fundamental inconsistency between shared memory and register layout at the architectural level. While unifying the layout of all storage layers at the hardware level could theoretically eliminate conflicts, it would severely damage the compatibility of the existing Unified Computing Device Architecture (CUDA) ecosystem. Hundreds of millions of existing kernel codes worldwide assume that shared memory adopts a traditional horizontally compact arrangement. Once the underlying layout changes, existing code may need to be recompiled, debugged, or even rewritten, which is almost infeasible.

[0007] Therefore, there is an urgent need for a method that can optimize shared memory access from the source while maintaining existing software compatibility, in order to reduce or eliminate bank conflicts, shorten the number of memory access cycles required for a single instruction to complete shared memory access, and improve the efficiency of on-chip data movement in the GPU. Summary of the Invention

[0008] This invention provides a shared memory access method and system in a GPU. By introducing address mapping rules at the hardware level, the data elements read from shared memory are logically consistent with the arrangement order in the register file without modifying the existing software code. This eliminates the natural multi-clock-cycle serialization overhead introduced by memory conflicts and layout mismatches during the transfer of shared memory to the register file, achieving near-conflict-free memory access while maximizing software compatibility.

[0009] Specifically, this invention introduces a dedicated shared memory address translation unit in the hardware memory access path. The software still issues shared memory access instructions in the traditional way (read / write instructions, i.e., ld / st instructions remain unchanged). Before or after the instruction reaches shared memory, the hardware rapidly translates the original read address, mapping the access addresses of data elements with the same element number within the warp to different memory banks within the shared memory. After address translation, data elements with the same element number from all threads within the warp logically form a layout consistent with the register file, thus achieving a one-time complete transfer from shared memory to the register file, eliminating the need for batch reading and internal reassembly of data across multiple clock cycles.

[0010] This invention does not limit specific address mapping rules, as long as an equivalent layout uniformity effect can be achieved. For example, lightweight operations such as addition, shifting and constant multiplication can be used. These operations can be completed in the existing address generation pipeline with extremely low latency and almost no additional critical path latency is introduced.

[0011] In actual operation, the benefits of the new layout are maximized when all 32 threads within a thread bundle are active. If some threads are inactive (e.g., due to branching), the hardware can dynamically revert to the traditional layout, reading or writing data elements according to the original read addresses, thus avoiding space waste. To further improve performance, this invention also proposes a memory access instruction merging optimization strategy: 1-2 additional access instructions to be executed are cached in the shared memory access unit for each thread bundle. When adjacent instructions are detected to be able to be concatenated into a complete 32-thread access, address translation and layout unification mechanisms are uniformly applied, thereby maximizing the bandwidth advantage brought by the new mechanism.

[0012] In a first aspect, the present invention provides a method for accessing shared memory in a GPU, characterized in that the method includes: Receive shared memory read instructions from multiple threads, wherein the original read address in the read instruction is read according to a first layout, wherein the first layout is that multiple data elements corresponding to the same thread are arranged consecutively at the original read address; In response to the shared memory read instruction, the original read address corresponding to each thread is converted into an access address according to a predetermined address mapping rule, so that the access addresses of multiple data elements corresponding to the same thread are not contiguous. The address mapping rule is configured to map the access addresses of data elements with the same element number in each thread to different storage banks in the shared memory. The data in the shared memory is arranged such that data elements with the same element number in different threads are stored in different storage banks. Based on the access address, data elements with the same element number are read in parallel from multiple storage banks in the shared memory; The read data elements are written into the target register file address space, wherein data elements corresponding to multiple threads with the same element number are written to the same address in the target register file address space.

[0013] In a second aspect, the present invention also provides a shared memory access system in a GPU, characterized in that the system includes shared memory, a target register file, a receiving device, an address translation unit, a reading device, and a writing device; The receiving device is used to receive shared memory read instructions issued by multiple threads, wherein the original read address in the read instruction is read according to a first layout, wherein the first layout is that multiple data elements corresponding to the same thread are arranged consecutively at the original read address; The address translation unit is used to respond to the shared memory read instruction and convert the original read address corresponding to each thread into an access address according to a predetermined address mapping rule, so that the access addresses of multiple data elements corresponding to the same thread are not contiguous. The address mapping rule is configured to map the access addresses corresponding to data elements with the same element number in each thread to different storage banks in the shared memory. The data in the shared memory is arranged such that data elements with the same element number in different threads are stored in different storage banks. The reading device is used to read data elements with the same element number in parallel from multiple storage banks of shared memory according to the access address; The writing device is used to write the read data elements into the target register file address space, wherein data elements corresponding to multiple threads with the same element number are written to the same address in the target register file address space.

[0014] Compared with existing technologies, it has the following beneficial effects: First, by converting the original read address through a predetermined address mapping rule, the access addresses of multiple threads with the same element index within a thread bundle are mapped to different storage banks in shared memory. This enables parallel reading within the same memory access cycle, avoiding the serialization latency caused by multiple threads accessing the same storage bank in traditional layouts. The memory access cycle refers to the logical shared memory access time window triggered by a shared memory access instruction when the thread bundle executes it. Even without other concurrent memory access conflicts, the fixed multi-clock-cycle overhead caused by layout mismatch is eliminated, significantly improving the effective bandwidth of shared memory. In scenarios where each thread within a thread bundle needs to move four 32-bit data elements between the register file and shared memory, since the data path between the register file and shared memory only supports parallel movement of threads with the same element index at a time (i.e., only e1, e2, e3, or e4 are moved each time), a complete data movement requires multiple shared memory access processes. In traditional layout schemes, multi-threaded access to the same element index easily leads to memory conflicts, and each shared memory access typically requires approximately 4 consecutive clock cycles to complete. Therefore, completing the transfer of 4 data elements requires approximately: 4 (element index count) × 4 (conflict unwinding cycles per access) = 16 clock cycles. However, under the address mapping and layout unification mechanism of this invention, data elements with the same element index are mapped to different memory banks, achieving conflict-free parallel access, making a single shared memory access physically close to a single clock cycle. Therefore, completing the data transfer requires only: 4 (element index count) × 1 (conflict-free access cycle) = 4 clock cycles. Thus, under the same data transfer task, the required number of clock cycles is reduced from approximately 16 to approximately 4, a reduction of approximately 12 clock cycles, corresponding to a memory access time reduction of approximately 75%, significantly improving the efficiency of data transfer from shared memory to register files.

[0015] Secondly, the converted data elements are logically aligned with the register file's order. Multiple thread elements with the same element number can be directly written to the same line in the register file, eliminating the need for multiple reads of shared memory and internal buffer reassembly in traditional schemes. This reduces the number of consecutive clock cycles required for a single data transfer from shared memory to the register file from multiple clock cycles in traditional schemes to nearly a single clock cycle, significantly reducing memory access latency and improving overall computational efficiency. Symmetrical performance gains are also achieved in the write direction, thus simultaneously accelerating both read and write operations of shared memory.

[0016] Third, the address mapping rules are implemented in the hardware address translation unit, and the ld / st instructions issued by the software do not need to be modified. Existing CUDA kernel code can be run directly without recompiling or rewriting. This method optimizes shared memory access from the source without disrupting the existing ecosystem, making it suitable for large-scale AI training and inference tasks. In real AI computing kernels, such as FP16 GEMM and Attention operations, shared memory access-related instructions typically account for 30% to 60% of the total kernel execution clock cycles. Therefore, in the above scenarios, the overall computing performance is expected to improve by approximately 15% to 40%, with the specific improvement depending on the kernel's memory access pattern and the intensity of shared memory usage.

[0017] Fourth, when the number of effective threads within a thread bundle is less than the maximum number of threads, adjacent memory access instructions can be cached, and it can be determined whether they can be concatenated into a complete active set of thread bundles. This dynamically determines whether to apply address mapping rules, avoiding the waste of shared memory space caused by the strong binding of threads and storage locations in traditional solutions, and improving resource utilization efficiency in real-world scenarios. This mechanism enables the invention to maintain high acceleration not only in regularized full-thread bundle computation modes but also in irregular memory access scenarios such as transpose, normalization, and Softmax that frequently occur in models like Transformer.

[0018] Fifth, the address translation unit is implemented based on simple operations such as addition and bit shifting, resulting in extremely low latency and almost no additional critical path overhead. It can be seamlessly integrated into existing GPU memory access paths, demonstrating good feasibility and engineering value. Its hardware overhead is limited to a small address translation module, which significantly reduces chip area and power consumption compared to increasing the number of banks, adding ports, or expanding SRAM capacity.

[0019] In summary, this invention solves the fundamental problem of mismatch between shared memory and register file layout at the architecture level while maintaining software compatibility. It significantly improves the on-chip data transfer efficiency and overall computing performance of the GPU, and can be widely used in application scenarios that are highly sensitive to on-chip bandwidth, such as cloud training, edge inference, and scientific computing. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1This is a schematic diagram of the shared memory data element layout in a data transfer scenario where each thread has 4 data elements, as is currently the case in the technology. Figure 2 This is a schematic diagram of the data element layout in a register file in the prior art; Figure 3 This is a schematic diagram illustrating the mapping relationship between shared memory data elements and register file data elements in existing technologies; Figure 4 This is a flowchart of a shared memory access method in a GPU provided in an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the mapping relationship between shared memory data elements and register file data elements provided in an embodiment of the present invention; Figure 6 This is a schematic diagram illustrating the layout changes of shared memory data elements provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the addressing method for shared memory in existing technology; Figure 8 This is a block diagram of a shared memory access system in a GPU provided in an embodiment of the present invention. Detailed Implementation

[0022] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Invention Overview As mentioned above, this invention provides a shared memory access method and system in a GPU. By introducing address mapping rules at the hardware level to convert shared memory access addresses, the data layout of shared memory and register files is unified, thereby eliminating or significantly reducing memory conflicts from the source, improving data transfer efficiency, and maintaining high compatibility with existing software.

[0024] Exemplary methods Figure 4 This is a flowchart of a shared memory access method in a GPU provided by an embodiment of the present invention. This embodiment includes the following steps: S101: Receive shared memory read instructions from multiple threads, wherein the original read address in the read instruction is read according to a first layout, wherein the first layout is that multiple data elements corresponding to the same thread are arranged consecutively at the original read address.

[0025] The multiple threads belong to a warp. A warp is the basic execution unit in the GPU Single Instruction Multithreaded (SIMT) architecture, typically containing 32 threads. Multiple threads within a warp execute the same read instruction within the same memory access cycle, but each thread can generate different shared memory access addresses based on its own thread number and computational needs to read the required data elements from shared memory. These data elements can be 32-bit or 64-bit wide data units.

[0026] The original read addresses in the read instruction are read according to the first layout. For any thread within a thread bundle, the original read addresses of the multiple data elements it needs to access occupy a continuous address range. For example, when a thread needs to load four 32-bit data elements, the original read addresses of the multiple data elements corresponding to that thread are sequentially consecutive. The original read addresses of data elements corresponding to different threads are interleaved. For ease of explanation, Figure 6 Each data element is labeled with its corresponding position number (e.g., 1, 2, 3, etc.), such as Figure 6 In the first layout shown in the upper part, the original read addresses of the data elements Thd0 e1, Thd0 e2, Thd0 e3, and Thd0 e4 corresponding to thread 0 (Thd0) occupy the first continuous address range, corresponding to position numbers 1, 2, 3, and 4 respectively. The original read addresses of the data elements corresponding to thread 1 immediately follow and occupy the next continuous address range, corresponding to position numbers 5, 6, 7, and 8 respectively. The remaining threads are arranged in sequence.

[0027] In the first layout, when multiple threads within a thread bundle simultaneously access data elements with the same element number within the same memory access cycle, the original read addresses corresponding to different threads are read according to the first layout, which easily maps them to the same memory bank or a few memory banks, such as... Figure 6 The first layout shown in the upper part has thd0 e1 / thd2 e1 / thd4 e1 / thd6e1 located on the same bank, which causes a storage conflict and results in the serialization of access requests.

[0028] S102: In response to the shared memory read instruction, the original read address corresponding to each thread is converted into an access address according to a predetermined address mapping rule, so that the access addresses of multiple data elements corresponding to the same thread are not contiguous. The address mapping rule is configured to map the access addresses corresponding to data elements with the same element number in each thread to different storage banks in the shared memory. The data in the shared memory is arranged such that data elements with the same element number in different threads are stored in different storage banks.

[0029] The core idea of ​​this invention is to unify the data arrangement on shared memory and global memory to the same layout as the register file. The register file uses a vertical arrangement, meaning that multiple threads with the same element number within a warp are stored in the same register line or the same address group. This allows the arithmetic logic unit to simultaneously read the corresponding elements of the entire warp within a single cycle, achieving efficient vectorized computation. If shared memory and global memory also use the same layout, there will be no SRAM memory conflicts between registers and shared memory, greatly improving data transfer efficiency.

[0030] This invention chooses to achieve compatibility optimization without modifying any software code, relying solely on hardware changes. Specifically, a dedicated shared memory address translation unit is introduced into the shared memory access path. The software-issued ld / st instructions remain unchanged; the hardware, before or after the instructions reach shared memory, translates the original read addresses for each thread into access addresses, ensuring that the access addresses of multiple data elements corresponding to the same thread are not contiguous. Figure 6 The second layout shown in the lower half, after conversion, the access addresses of Thd0 e1, Thd0 e2, Thd0 e3, and Thd0 e4 correspond to position numbers 1, 9, 17, and 25 respectively, ultimately achieving a one-time complete transfer from shared memory to registers, without the need for multiple reassemblies.

[0031] like Figure 5 As shown, the upper part illustrates the shared memory data arrangement method using address translation in this invention. Multiple thread data elements with the same element number are arranged consecutively horizontally and distributed across different memory banks; different element numbers of the same thread are aligned vertically and distributed across different rows within the same memory bank. This arrangement is logically identical to the register file layout shown in the lower part: the register file uses a vertical arrangement, with one row corresponding to multiple thread data elements with the same element number, and multiple rows corresponding to different element numbers.

[0032] Data elements read from shared memory can be directly mapped to the same line position in the register file, enabling complete data transfer within a single memory access cycle and eliminating the overhead of multiple reads and reassemblies caused by layout mismatch in traditional layouts.

[0033] Figure 6 This demonstrates the changing positions of data elements in shared memory. The upper half shows the conventional layout of existing technology, namely the first layout, where data elements of the same thread are horizontally continuous, and multi-threaded data with the same element number are vertically aligned. The lower half shows the layout of this invention, where multi-threaded data elements with the same element number are arranged horizontally in a row, and data elements with different element numbers of the same thread are vertically spaced, consistent with the vertical layout of register files, that is, data elements with the same element number in different threads are stored in different memory banks.

[0034] Prior to step S102, the method further includes executing a first write instruction to the shared memory.

[0035] Before executing the shared memory read instruction, the data elements in the shared memory have been written according to the address mapping rules of this invention, that is, the first write instruction for the shared memory is executed, and the data is organized according to the second layout (multiple thread data with the same data element sequence number occupy the same address, that is, the same row position). When the shared memory read instruction is received in step S102, the data elements in the shared memory are physically stored according to the converted layout (logically corresponding to the read view of the second layout), but from the software perspective, the original read address is still calculated based on the first layout.

[0036] The underlying addressing method of shared memory is as follows: Figure 7 The address shown is fixed, and it increments in row-major order. That is, it first increments horizontally within the same row, and then jumps vertically to the next row to continue addressing. This addressing rule is determined by the shared memory hardware architecture, including the mapping relationship between address and bank and row position, and usually does not change with software access mode.

[0037] While maintaining the shared memory addressing method, this invention sets up an address translation unit to perform real-time remapping of the original read address at the hardware level. This changes the actual storage location of the data element under the fixed addressing path, thereby completing the logical transposition of the data location without changing the software memory access instructions and the underlying addressing rules. This achieves consistency between the shared memory data layout and the register file layout, allowing the data read from the shared memory to directly correspond to the arrangement order in the register file. This enables thread-beam level data to be moved without conflict within a single memory access cycle.

[0038] The address mapping rules are implemented by a hardware address translation unit located in the shared memory access path. This hardware address translation unit is situated between the shared memory access instruction issued by the thread bundle and the shared memory storage addressing logic. It is used to translate the original read address generated by the thread to obtain the corresponding access address without altering the software-visible address space.

[0039] The hardware address translation unit performs address translation operations on the original read addresses of each thread within the thread bundle, according to a predetermined address mapping rule. This address translation operation does not involve multiplication, table lookup, or complex control logic. Instead, it rearranges or combines the high and / or low bits of the original read address based on addition and bit shift operations to generate the access address.

[0040] Since the hardware address translation unit consists of only adders, shifters and a small amount of combinational logic, its operation latency is extremely low. It can complete the address translation within one clock cycle, without introducing additional memory access pipelines or critical path overhead, and can be seamlessly integrated into the existing GPU shared memory access path.

[0041] Furthermore, since the address translation process is completed entirely at the hardware level, the shared memory load and store instructions issued by the software do not need to be modified. Kernel code under existing CUDA or other GPU programming models can run directly on hardware that supports the address translation unit, thereby improving the efficiency of shared memory access while maintaining software compatibility.

[0042] Specifically, for the j-th data element of any thread i within a thread bundle, a thread-level base offset and an element-level segment offset are introduced based on the original read address generated by that thread to form the access address.

[0043] The thread-level base offset is determined based on the thread number i, and is used to distribute the access addresses of different threads with the same element number in different memory banks of the shared memory; the element-level segment offset is determined based on the element number j, and is used to distribute the access addresses of different element numbers of the same thread in different rows of the same memory bank of the shared memory.

[0044] The core of the address mapping rule is to perform a linear transformation on the original read address of each data element of each thread, thereby realizing the logical rearrangement of data elements in shared memory, which reduces memory conflicts and achieves consistency with the register file layout.

[0045] For any thread i (i=0,1,...,31) and its j-th data element (j=0,1,...,ele_num-1) within a thread bundle, the hardware address translation unit first obtains the original read address calculated by the thread based on the original layout. Here, ele_num represents the number of 32-bit data elements that each thread needs to load (or store) in a single shared memory access instruction, typically taking values ​​of 1, 2, 4, 8, 16, or 32, depending on the instruction type. Then, based on this original read address, two types of offsets are introduced for adjustment to form the final translated access address.

[0046] The thread-level base offset is directly related to the thread number *i*, and is typically an address increment proportional to *i* (e.g., *i* × the size of the data element in bytes). Its purpose is to horizontally distribute access addresses from different threads with the same element index *j* across shared memory banks. Through this offset, accesses to elements with the same index that might otherwise be concentrated in the same or a few memory banks are evenly or conflict-free distributed across different memory banks, thus avoiding conflicts caused by multiple threads accessing the same memory bank simultaneously within a warp.

[0047] The size of the element-level segment offset is directly related to the element index j, and is usually a large address increment proportional to j (e.g., j × large offset). Its purpose is to ensure that the access addresses of different element indices (i.e., different j) of the same thread are distributed with significant spacing along the row direction within the same memory bank of shared memory. Typically, this offset causes elements e1, e2, e3, e4, etc., of the same thread to fall in different row positions within the same memory bank, thus logically forming a layout view that matches the vertical arrangement of the register file.

[0048] In the register file, multiple data elements (such as e1, e2, e3, etc.) of each thread within a thread bundle are typically allocated in the same contiguous register address space, i.e., in the same column. Because the GPU's Single Instruction Multithreaded (SIMT) architecture requires multiple data elements of the same thread to have a defined layout in the registers, this contiguous allocation method facilitates the Arithmetic Logic Unit (ALU) accessing these data in a fixed order to perform vectorized computations. If the data elements of the same thread are arbitrarily scattered in the register file, the order of data access within the thread will be uncertain, increasing control complexity and potentially disrupting the correspondence between registers and the ALU, preventing the computational units from correctly aligning their operations.

[0049] This invention introduces thread-level base offsets and element-level segmented offsets to the original read address of shared memory, performing linear remapping for each data element of each thread. The combination of these two types of offsets not only ensures that data with the same element index j from different threads are mapped to different storage banks in shared memory, eliminating bank conflicts and enabling parallel access within the same memory access cycle, but also ensures that different element indices j from the same thread fall in the same column in the register file, maintaining the sequential consistency of data within the thread.

[0050] More specifically, the address mapping rule is as follows: For the j-th data element of thread i, the access address satisfies the following relationship: The access address = the original read address (j × data element size in bytes) + (i × data element size in bytes) + (j × large span offset); The above The (j × data element size in bytes) term is the original element offset offset term, used to eliminate offsets caused by contiguous storage of data elements within the same thread at the original read address. By subtracting this offset, the thread's access address is pulled back to the base address, providing the correct starting point for subsequent offset stacking. This is a preprocessing step for implementing layout reset, ensuring that address translation is calculated based on the thread base address.

[0051] The + (i × data element size in bytes) term is the thread-level base offset; The + (j × large span offset) term is the element-level segmented offset; The large span offset is set to an integer multiple of at least one full memory bank width, where the memory bank width is the number of bytes that a single memory bank in the shared memory can be addressed contiguously.

[0052] The data element size can be 4 bytes (corresponding to a 32-bit data element), and the large span offset can be 128 bytes.

[0053] All multiplications in the above formulas are constant multiplications (×4, ×128), which can be efficiently implemented in hardware using shifting and addition. The entire address translation process requires only a small number of adders and shifters and can be completed within one clock cycle without introducing additional critical path latency. This transformation is applicable to both read operations (ld.shared) and write operations (st.shared) of shared memory, ensuring the consistency of the shared memory read and write layout.

[0054] To more clearly describe this transformation process, a pseudocode implementation example is given below: Thread_num_per_share_mem_entry = 32 / ele_num; For(i=0;i <thread_num;i++):loop_thread_idx For(j=0;j <ele_num;j++):loop_element_idx Share_mem_addr_old[i][j]= share_mem_base[i][j]+ j*4; Share_mem_addr_new[i][j]= Share_mem_addr_old[i][j]–j*4 + i*4 + j*128; End End Where Share_mem_addr_old[i][j] represents the access address of the j-th data element in shared memory for thread i under the conventional scheme. This invention does not limit the specific transformation algorithm, as long as it can achieve an equivalent layout rearrangement effect. This method supports a maximum of 32 threads, with each thread handling a maximum of 32 data elements.

[0055] In scenarios where each thread within a thread bundle needs to move four data elements between the register file and shared memory (ele_num=4), the size of the address region occupied before and after the translation remains the same, without increasing the space requirement of shared memory. The address translation scheme adopted in this invention has the same storage space occupation as traditional schemes, and the layout diagram after the translation also shows this characteristic.

[0056] Through the specific address mapping rules described above, this invention can completely change the conflict mode of traditional shared memory at the hardware address level without modifying the software code, achieving low-conflict or conflict-free parallel access at the thread bundle level. At the same time, it ensures that the arrangement order of data elements in shared memory is completely consistent with the layout logic of the register file. After the transformation, data elements with the same element number in the warp logically correspond to the same line position in the register file. The hardware can read these data elements from multiple memory banks in parallel within the same memory access cycle and directly write them to the corresponding line position in the register file, eliminating the multiple memory accesses and data reassembly processes required in traditional solutions.

[0057] However, this transformation method binds the thread number to a specific storage location. When not all threads within a thread bundle are active (e.g., some threads become inactive due to branching), forcing the application address transformation may lead to a decrease in the utilization of shared memory space.

[0058] Therefore, this invention supports using this address translation scheme when all threads are active to achieve optimal performance; when the number of active threads is less than the full number of threads in the thread bundle, it can fall back to the traditional layout method for access, thereby avoiding space waste. Before executing the first write instruction of the shared memory, the method further includes: determining whether the number of active threads has reached a preset threshold, the preset threshold being the full number of threads in the thread bundle (e.g., 32); if it has, then executing the address mapping rule; if it has not, then bypassing the address mapping rule and reading data elements according to the original read address.

[0059] To further improve the performance and compatibility of this invention in actual operation, this invention proposes a memory access instruction caching and merging mechanism, which is applicable to selectively enabling the address mapping rule when the number of effective threads in a thread bundle is less than the full number of threads.

[0060] Specifically, the step of determining whether the number of valid threads has reached a preset threshold further includes: when the number of valid threads within a thread bundle has not reached the preset threshold, firstly, caching the current shared memory read instruction and adjacent subsequent memory access instructions within a predetermined time window or instruction count window; then, determining whether the number of valid threads in the cached memory access instructions has reached the preset threshold. If the target is reached, the address mapping rule is executed to convert the access address of the cached instruction and use the converted address to complete the data reading. If the address mapping rule is not met, the data element is read according to the original read address.

[0061] In typical GPU architectures, memory access units are usually located outside the computation units, and the scheduling and execution of shared memory access instructions have a certain buffer space. Based on this structural characteristic, this invention caches the current shared memory read instruction and adjacent subsequent memory access instructions within a predetermined time window or instruction count window for each thread bundle.

[0062] When adjacent memory access instructions (including the current instruction and subsequent instructions) are detected to form a complete set of 32 active threads, the address mapping rules are uniformly applied for address translation. This utilizes the optimized memory access method proposed in this invention to achieve higher storage space utilization and memory access efficiency. The specific concatenation method can be flexibly adjusted according to hardware implementation and scheduling strategy, for example, based on thread masks, active thread identifiers, or the number of active threads within the instruction window.

[0063] This mechanism avoids wasting storage space due to forced application address translation when there are insufficient active threads within a thread bundle. When active threads can be supplemented by adjacent instructions, the address mapping optimization of this invention is given priority, thereby achieving a balance between performance and space utilization under different load scenarios.

[0064] S103: Based on the access address, read the data elements with the same element number in parallel from multiple storage banks of shared memory.

[0065] The access address ensures that multiple threads within a thread bundle with the same element index j have their access requests mapped to different memory banks. The shared memory hardware supports simultaneous independent access to multiple banks, with each bank responding to one request per cycle. Therefore, when all relevant memory banks receive only one thread's read request, the hardware can complete the reading of all data elements in parallel within the same memory access cycle.

[0066] Compared to traditional methods, this step avoids the serialization latency caused by multiple threads accessing the same memory bank with the same sequence number. Even in the absence of other concurrency conflicts, it eliminates the fixed multi-clock-cycle overhead caused by layout mismatch, enabling data transfer from shared memory to the register file to be completed in a near-ideal single clock cycle.

[0067] S104: Write the read data elements into the target register file address space, wherein data elements corresponding to multiple threads with the same element number are written into the same address in the target register file address space, i.e., the same line position.

[0068] Since the layout of the converted data elements is logically consistent with the register file, multiple thread data with the same element number correspond exactly to the same line position in the register file. Therefore, no additional reorganization or multiple write operations are required, and the corresponding line address of the register file can be written at once in the same cycle.

[0069] This invention applies not only to shared memory read operations but also to shared memory write operations, ensuring consistency between shared memory read and write operations. The method further includes: Receive a shared memory write instruction, wherein the data to be written is organized in the target register file address space according to a second layout. The second layout is that data elements corresponding to multiple threads with the same element number within a thread bundle occupy the same address in the register file address space, i.e., the same row position, while data elements with different element numbers of the same thread are arranged continuously along the column direction in the register file.

[0070] In response to the shared memory write instruction, the original write address corresponding to each thread is converted into an access address according to the predetermined address mapping rule. Since the address mapping rule is a reversible linear address transformation, during the shared memory write process, the write address can be converted using the same address mapping rule as the read phase, so that the second layout data in the register file is remapped to the first layout location in the shared memory, thereby achieving consistency in the data layout between the shared memory and the register file in both the read and write directions.

[0071] Based on the access address, data elements organized in the second layout in the register file are written in parallel to multiple storage banks in shared memory.

[0072] Exemplary System Accordingly, embodiments of the present invention also provide a shared memory access system in a GPU. Figure 8 This is a block diagram of a shared memory access system in a GPU provided in an embodiment of the present invention, such as... Figure 8As shown, the system 100 provided in this embodiment includes: shared memory 101, target register file 102, receiving device 103, address translation unit 104, reading device 105, and writing device 106; The receiving device 103 is used to receive shared memory read instructions issued by multiple threads, wherein the original read address in the read instruction is read according to a first layout, and the first layout is that multiple data elements corresponding to the same thread are arranged consecutively at the original read address; The address translation unit 104 is used to respond to the shared memory read instruction and convert the original read address corresponding to each thread into an access address according to a predetermined address mapping rule, so that the access addresses of multiple data elements corresponding to the same thread are not contiguous. The address mapping rule is configured to map the access addresses corresponding to data elements with the same element number in each thread to different storage banks in the shared memory. The data in the shared memory 101 is arranged such that data elements with the same element number in different threads are stored in different storage banks. The reading device 105 is used to read the data elements with the same element number in parallel from multiple storage banks of the shared memory 101 according to the access address; The writing device 106 is used to write the read data elements into the target register file address space, wherein data elements corresponding to multiple threads with the same element number are written to the same address in the target register file address space.

[0073] The address translation unit 104 is also used for: For the j-th data element of thread i, a thread-level base offset and an element-level segment offset are introduced on the basis of the original read address; The thread-level base offset is determined based on the thread number i, and is used to distribute the access addresses of different threads with the same element number in different memory banks of shared memory 101. The element-level segment offset is determined based on the element index j, and is used to distribute the access addresses of different element indices of the same thread in different rows of the same storage bank of shared memory 101.

[0074] The address mapping rule is as follows: For the j-th data element of thread i, the access address satisfies the following relationship: The access address = the original read address (j × data element size in bytes) + (i × data element size in bytes) + (j × large span offset); The above The (j×data element byte size) item is the original element offset offset item, which is used to eliminate the offset caused by the continuous storage of data elements within the same thread in the original read address; The + (i × data element size in bytes) term is the thread-level base offset; The + (j × large span offset) term is the element-level segmented offset; The large span offset is set to an integer multiple of at least one full memory bank width, wherein the memory bank width is the number of bytes that a single memory bank in shared memory 101 can be addressed consecutively.

[0075] The data element has a size of 4 bytes; The large span offset is 128 bytes.

[0076] The writing device 106 is also used to execute a first write instruction of the shared memory 101; The system 100 also includes: The judging device 107 is used to judge whether the number of valid threads has reached a preset threshold. If the condition is met, the address translation unit 104 is further configured to execute the address mapping rule; If the target is not met, the reading device 105 is also used to bypass the address mapping rule and read data elements according to the original reading address.

[0077] If the above conditions are not met, then: The judgment device 107 further includes a cache device 108, which is used to cache the current shared memory read instruction and the subsequent memory access instructions adjacent within a predetermined time window or instruction number window if the number of effective threads does not reach a preset threshold. The judgment device 107 is also used to determine whether the number of valid threads in the cached memory access instructions has reached a preset threshold.

[0078] The multiple threads belong to a thread bundle, and the preset threshold is the full number of threads in the thread bundle.

[0079] The receiving device 103 is also used to receive a shared memory write instruction, wherein the data to be written is organized in the target register file address space according to a second layout, the second layout being data elements corresponding to multiple threads with the same element number, occupying the same row position in the target register file address space; The address translation unit 104 is also configured to, in response to the shared memory write instruction, translate the original write address corresponding to each thread according to the predetermined address mapping rule; The writing device 106 is also used to write data elements organized in the second layout in the register file in parallel to multiple storage banks of the shared memory 101 according to the access address.

[0080] The address mapping rules are implemented in the hardware address translation unit.

[0081] The hardware address translation unit generates the access address based on addition and bitwise operations.

[0082] It should be noted that although the operations of the shared memory access method in the GPU of the present invention are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0083] Furthermore, although several devices, units, or modules of the shared memory access system in the GPU have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the invention, the features and functions of two or more modules described above can be embodied in a single module. Conversely, the features and functions of a single module described above can be further divided and embodied by multiple modules.

[0084] While the spirit and principles of the invention have been described with reference to several specific embodiments, it should be understood that the invention is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for ease of description. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

[0085] This invention provides: 1. A method for accessing shared memory in a GPU, characterized in that the method includes: Receive shared memory read instructions from multiple threads, wherein the original read address in the read instruction is read according to a first layout, wherein the first layout is that multiple data elements corresponding to the same thread are arranged consecutively at the original read address; In response to the shared memory read instruction, the original read address corresponding to each thread is converted into an access address according to a predetermined address mapping rule, so that the access addresses of multiple data elements corresponding to the same thread are not contiguous. The address mapping rule is configured to map the access addresses of data elements with the same element number in each thread to different storage banks in the shared memory. The data in the shared memory is arranged such that data elements with the same element number in different threads are stored in different storage banks. Based on the access address, data elements with the same element number are read in parallel from multiple storage banks in the shared memory; The read data elements are written into the target register file address space, wherein data elements corresponding to multiple threads with the same element number are written to the same address in the target register file address space.

[0086] 2. The shared memory access method in the GPU according to item 1, characterized in that the step of converting the original read address corresponding to each thread into an access address according to a predetermined address mapping rule in response to the shared memory read instruction, so that the access addresses of multiple data elements corresponding to the same thread are not contiguous, specifically comprises: For the j-th data element of thread i, a thread-level base offset and an element-level segment offset are introduced on the basis of the original read address; The thread-level base offset is determined based on the thread number i, and is used to distribute the access addresses of different threads with the same element number in different memory banks of the shared memory. The element-level segment offset is determined based on the element index j, and is used to distribute the access addresses of different element indices of the same thread in different rows of the same storage bank of shared memory.

[0087] 3. The shared memory access method in the GPU described in item 2, characterized in that the address mapping rule is specifically as follows: For the j-th data element of thread i, the access address satisfies the following relationship: The access address = the original read address (j × data element size in bytes) + (i × data element size in bytes) + (j × large span offset); The above The (j×data element byte size) item is the original element offset offset item, which is used to eliminate the offset caused by the continuous storage of data elements within the same thread in the original read address; The + (i × data element size in bytes) term is the thread-level base offset; The + (j × large span offset) term is the element-level segmented offset; The large span offset is set to an integer multiple of at least one full memory bank width, where the memory bank width is the number of bytes that a single memory bank in the shared memory can be addressed contiguously.

[0088] 4. The shared memory access method in the GPU according to item 3, wherein the data element byte size is 4 bytes; The large span offset is 128 bytes.

[0089] 5. The shared memory access method in a GPU according to any one of items 1-3, characterized in that, before the step of converting the original read address corresponding to each thread into an access address according to a predetermined address mapping rule in response to the shared memory read instruction, so that the access addresses of multiple data elements corresponding to the same thread are not contiguous, the method further includes: Execute the first write instruction of the shared memory; Before the step of executing the first write instruction of the shared memory, the method further includes: Determine whether the number of valid threads has reached a preset threshold; If the condition is met, then the address mapping rule is executed; If the address mapping rule is not met, the data element is read according to the original read address.

[0090] 6. The shared memory access method in the GPU according to item 5, characterized in that the step of determining whether the number of effective threads has reached a preset threshold further includes: If the number of valid threads does not reach the preset threshold, cache the current shared memory read instruction and the subsequent memory access instructions that are adjacent within the predetermined time window or instruction number window. Determine whether the number of valid threads in the cached memory access instructions has reached a preset threshold.

[0091] 7. The shared memory access method in a GPU according to item 5 or 6, wherein the plurality of threads belong to a thread bundle, and the preset threshold is the full number of threads in the thread bundle.

[0092] 8. The shared memory access method in a GPU according to any one of items 1-3, characterized in that the method comprises: Receive a shared memory write instruction, wherein the data to be written is organized in the target register file address space according to a second layout, the second layout being data elements corresponding to multiple threads with the same element number, occupying the same address in the target register file address space; In response to the shared memory write instruction, the original write address corresponding to each thread is converted into an access address according to the predetermined address mapping rule; Based on the access address, data elements organized in the second layout in the register file are written in parallel to multiple storage banks in shared memory.

[0093] 9. The shared memory access method in a GPU according to any one of items 1-3, characterized in that the address mapping rule is implemented in a hardware address translation unit.

[0094] 10. The shared memory access method in a GPU according to item 9, characterized in that the hardware address translation unit generates the access address based on addition and bitwise operations.

[0095] 11. A shared memory access system in a GPU, characterized in that the system includes shared memory, a target register file, a receiving device, an address translation unit, a reading device, and a writing device; The receiving device is used to receive shared memory read instructions issued by multiple threads, wherein the original read address in the read instruction is read according to a first layout, wherein the first layout is that multiple data elements corresponding to the same thread are arranged consecutively at the original read address; The address translation unit is used to respond to the shared memory read instruction and convert the original read address corresponding to each thread into an access address according to a predetermined address mapping rule, so that the access addresses of multiple data elements corresponding to the same thread are not contiguous. The address mapping rule is configured to map the access addresses corresponding to data elements with the same element number in each thread to different storage banks in the shared memory. The data in the shared memory is arranged such that data elements with the same element number in different threads are stored in different storage banks. The reading device is used to read data elements with the same element number in parallel from multiple storage banks of shared memory according to the access address; The writing device is used to write the read data elements into the target register file address space, wherein data elements corresponding to multiple threads with the same element number are written to the same address in the target register file address space.

[0096] 12. The shared memory access system in the GPU according to claim 11, wherein the address translation unit is further configured to: For the j-th data element of thread i, a thread-level base offset and an element-level segment offset are introduced on the basis of the original read address; The thread-level base offset is determined based on the thread number i, and is used to distribute the access addresses of different threads with the same element number in different memory banks of the shared memory. The element-level segment offset is determined based on the element index j, and is used to distribute the access addresses of different element indices of the same thread in different rows of the same storage bank of shared memory.

[0097] 13. The shared memory access system in the GPU according to item 12, characterized in that the address mapping rule is specifically as follows: For the j-th data element of thread i, the access address satisfies the following relationship: The access address = the original read address (j × data element size in bytes) + (i × data element size in bytes) + (j × large span offset); The above The (j×data element byte size) item is the original element offset offset item, which is used to eliminate the offset caused by the continuous storage of data elements within the same thread in the original read address; The + (i × data element size in bytes) term is the thread-level base offset; The + (j × large span offset) term is the element-level segmented offset; The large span offset is set to an integer multiple of at least one full memory bank width, where the memory bank width is the number of bytes that a single memory bank in the shared memory can be addressed contiguously.

[0098] 14. The shared memory access system in a GPU according to claim 13, wherein the data element byte size is 4 bytes; The large span offset is 128 bytes.

[0099] 15. A shared memory access system in a GPU according to any one of claims 11-13, wherein the writing device is further configured to execute a first write instruction of the shared memory; The system also includes: A judgment device is used to determine whether the number of valid threads has reached a preset threshold. If the condition is met, the address translation unit is further configured to execute the address mapping rule. If the target is not met, the reading device is also used to bypass the address mapping rule and read data elements according to the original reading address.

[0100] 16. The shared memory access system in the GPU according to item 15, wherein the determining device further comprises: a caching device, which is used to cache the current shared memory read instruction and the subsequent memory access instructions adjacent within a predetermined time window or instruction number window if the number of effective threads does not reach a preset threshold. The judgment device is also used to determine whether the number of valid threads in the cached memory access instructions has reached a preset threshold.

[0101] 17. The shared memory access system in a GPU according to claim 15 or 16, wherein the plurality of threads belong to a thread bundle, and the preset threshold is the full number of threads in the thread bundle.

[0102] 18. A shared memory access system in a GPU according to any one of claims 11-13, wherein the receiving device is further configured to receive a shared memory write instruction, wherein the data to be written is organized in a target register file address space according to a second layout, the second layout being data elements corresponding to multiple threads having the same element number occupying the same row position in the target register file address space. The address translation unit is also used to respond to the shared memory write instruction and, according to the predetermined address mapping rule, convert the original write address corresponding to each thread into an access address. The writing device is further configured to write data elements organized in a second layout in the register file in parallel to multiple storage banks of shared memory according to the access address.

[0103] 19. A shared memory access system in a GPU according to any one of claims 11-13, characterized in that the address mapping rules are implemented in a hardware address translation unit.

[0104] 20. The shared memory access system in a GPU according to claim 19, wherein the hardware address translation unit generates the access address based on addition and bitwise operations.

Claims

1. A shared memory access method in a GPU, characterized in that, The method includes: Receive shared memory read instructions from multiple threads, wherein the original read address in the read instruction is read according to a first layout, wherein the first layout is that multiple data elements corresponding to the same thread are arranged consecutively at the original read address; In response to the shared memory read instruction, the original read address corresponding to each thread is converted into an access address according to a predetermined address mapping rule, so that the access addresses of multiple data elements corresponding to the same thread are not contiguous. The address mapping rule is configured to map the access addresses corresponding to data elements with the same element number in each thread to different storage banks in the shared memory. The data in the shared memory is arranged such that data elements with the same element number in different threads are stored in different storage banks. Based on the access address, data elements with the same element number are read in parallel from multiple storage banks in the shared memory; The read data elements are written into the target register file address space, wherein data elements corresponding to multiple threads with the same element number are written to the same address in the target register file address space.

2. The shared memory access method in a GPU according to claim 1, characterized in that, The step of responding to the shared memory read instruction and converting the original read address corresponding to each thread into an access address according to a predetermined address mapping rule, so that the access addresses of multiple data elements corresponding to the same thread are not contiguous, specifically includes: For the j-th data element of thread i, a thread-level base offset and an element-level segment offset are introduced on the basis of the original read address; The thread-level base offset is determined based on the thread number i, and is used to distribute the access addresses of different threads with the same element number in different memory banks of the shared memory. The element-level segment offset is determined based on the element index j, and is used to distribute the access addresses of different element indices of the same thread in different rows of the same storage bank of shared memory.

3. The shared memory access method in a GPU according to claim 2, characterized in that, The address mapping rule is as follows: For the j-th data element of thread i, the access address satisfies the following relationship: The access address = the original read address (j × data element size in bytes) + (i × data element size in bytes) + (j × large span offset); The above The (j×data element byte size) item is the original element offset offset item, which is used to eliminate the offset caused by the continuous storage of data elements within the same thread in the original read address; The + (i × data element size in bytes) term is the thread-level base offset; The + (j × large span offset) term is the element-level segmented offset; The large span offset is set to an integer multiple of at least one full memory bank width, where the memory bank width is the number of bytes that a single memory bank in the shared memory can be addressed contiguously.

4. The shared memory access method in a GPU according to claim 3, characterized in that, The data element has a size of 4 bytes; The large span offset is 128 bytes.

5. The shared memory access method in a GPU according to any one of claims 1-3, characterized in that, Before the step of converting the original read address corresponding to each thread into an access address according to a predetermined address mapping rule in response to the shared memory read instruction, so that the access addresses of multiple data elements corresponding to the same thread are not contiguous, the method further includes: Execute the first write instruction of the shared memory; Before the step of executing the first write instruction of the shared memory, the method further includes: Determine whether the number of valid threads has reached a preset threshold; If the condition is met, then the address mapping rule is executed; If the address mapping rule is not met, the data element is read according to the original read address.

6. The shared memory access method in a GPU according to claim 5, characterized in that, The step of determining whether the number of valid threads has reached a preset threshold also includes: If the number of valid threads does not reach the preset threshold, cache the current shared memory read instruction and the subsequent memory access instructions that are adjacent within the predetermined time window or instruction number window. Determine whether the number of valid threads in the cached memory access instructions has reached a preset threshold.

7. The shared memory access method in a GPU according to claim 5 or 6, characterized in that, The multiple threads belong to a thread bundle, and the preset threshold is the full number of threads in the thread bundle.

8. The shared memory access method in a GPU according to any one of claims 1-3, characterized in that, The method includes: Receive a shared memory write instruction, wherein the data to be written is organized in the target register file address space according to a second layout, the second layout being data elements corresponding to multiple threads with the same element number, occupying the same address in the target register file address space; In response to the shared memory write instruction, the original write address corresponding to each thread is converted into an access address according to the predetermined address mapping rule; Based on the access address, data elements organized in the second layout in the register file are written in parallel to multiple storage banks in shared memory.

9. The shared memory access method in a GPU according to any one of claims 1-3, characterized in that, The address mapping rules are implemented in the hardware address translation unit.

10. A shared memory access system in a GPU, characterized in that, The system includes shared memory, a target register file, a receiving device, an address translation unit, a reading device, and a writing device; The receiving device is used to receive shared memory read instructions issued by multiple threads, wherein the original read address in the read instruction is read according to a first layout, wherein the first layout is that multiple data elements corresponding to the same thread are arranged consecutively at the original read address; The address translation unit is used to respond to the shared memory read instruction and convert the original read address corresponding to each thread into an access address according to a predetermined address mapping rule, so that the access addresses of multiple data elements corresponding to the same thread are not contiguous. The address mapping rule is configured to map the access addresses corresponding to data elements with the same element number in each thread to different storage banks in the shared memory. The data in the shared memory is arranged such that data elements with the same element number in different threads are stored in different storage banks. The reading device is used to read data elements with the same element number in parallel from multiple storage banks of shared memory according to the access address; The writing device is used to write the read data elements into the target register file address space, wherein data elements corresponding to multiple threads with the same element number are written to the same address in the target register file address space.

Images