A hardware-software cooperative address mapping scheme for efficient processing-in-memory

Abstract

To achieve higher performance by reducing row contention, an approach is provided for implementing a hardware-software cooperative address mapping scheme that enables mapping of data elements that are accessed together in the same row of one bank or across the same row of different banks. Using an intra-bank frame striping policy (IBFS), corresponding subsets of data elements are interleaved in a single row of a bank. Using an intra-channel frame striping policy (ICFS), corresponding subsets of data elements are interleaved in a single channel row of a channel. A memory controller utilizes ICFS and / or IBFS to efficiently store and access data elements in a memory, such as a processing-in-memory (PIM)-enabled memory.

Description

[Background technology]

[0001] The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Thus, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Furthermore, it should not be assumed that any of the approaches described in this section are well-understood, conventional, or conventional merely by virtue of their inclusion in this section.

[0002] Data access latency at the dynamic random access memory (DRAM) bank level is critical due to the physical nature of the DRAM cells and supporting peripheral circuitry. Each DRAM bank contains a row buffer that can read a single row at a time from the 2D memory array stored in the respective bank. To access a data element stored in a bank, the row of the 2D memory array containing the data element must be activated and then read into the row buffer. The data element can then be accessed from the row buffer. To access any next data element that falls in another row, a precharge operation must be performed before the next row can be activated. This situation is commonly known as a row-conflict. Such row-activate-precharge-activate cycles impose strict timing requirements of tens of nanoseconds that must be followed to ensure the integrity of the stored data and power management.

[0003] One way to avoid row conflicts is to access as many data elements as possible from the same row. However, there is no guarantee that adjacent data elements that fall in the same operating system page or contiguous physical address space, i.e., frame, will always be accessed together. The arrangement of data elements within a physical memory module, i.e., DRAM channels, banks, rows, and columns, depends on the physical address-to-physical memory mapping scheme used by the memory controller that controls access to the DRAM. Typically, to achieve memory-level parallelism (MLP), data elements from the same page are interleaved across the available number of channels. However, simple interleaving can result in excessive row conflicts when performing operations involving operands from different pages. Therefore, a better approach to addressing row conflicts in computer memory is needed.

[0004] Embodiments are illustrated by way of example, and not by way of limitation, in the accompanying drawings, in which like reference numerals refer to similar elements and in which: [Brief explanation of the drawings]

[0005] [Figure 1] FIG. 1 is a block diagram illustrating a memory controller coupled to a processing-in-memory (PIM) capable memory for implementing a cooperative address mapping scheme as described herein. [Figure 2] FIG. 1 illustrates a memory system having two channels. [Figure 3] 3 illustrates how physical address bits are mapped for indexing in the memory system shown in FIG. 2. [Figure 4A] FIG. 1 illustrates channel 0, bank 0 of a PIM-enabled memory with address interleaved memory mapping. [Figure 4B] FIG. 1 illustrates channel 0, bank 0 of a PIM-enabled memory with IBFS (intra-bank frame striping) address mapping. [Figure 5A]FIG. 1 illustrates channel 0 of a PIM-enabled memory with address-interleaved memory mapping. [Figure 5B] FIG. 1 illustrates channel 0 of a PIM-enabled memory with ICFS (Intra-Channel Frame Striping) address mapping. [Figure 6] FIG. 1 is a flow diagram illustrating an approach for accessing memory using IBFS and ICFS address mapping. DETAILED DESCRIPTION OF THE INVENTION

[0006] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments. However, it will be apparent to those skilled in the art that the embodiments may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the embodiments.

[0007] I. Overview II. Architecture III. Address Mapping Method A. Address interleaved memory mapping method B. Intrabank Frame Striping Policy C. Intra-channel frame striping policy D. Implementation Details E. Exemplary Procedures F. Applications

[0008] I. Overview To achieve higher processing performance by reducing row contention, an approach is provided for implementing a hardware-software cooperative address mapping scheme that enables mapping data elements used by memory instructions accessed together in the same row of one bank or across the same row of different banks. In one embodiment, an intra-bank frame striping policy (IBFS) is used to interleave corresponding subsets of data elements in a single row of a bank. In another embodiment, an intra-channel frame striping policy (ICFS) is used to interleave corresponding subsets of data elements in a single channel row of a channel. A memory controller utilizes IBFS or ICFS to efficiently store and access data elements in a memory, such as a processing-in-memory (PIM) capable memory, depending on the placement of PIM execution units in the memory bank or channel, respectively.

[0009] Although embodiments are described herein in the context of PIM-enabled memory systems, such embodiments are not limited to PIM-enabled memory systems per se and may be applied to other memory systems.

[0010] The approach described herein achieves a reduction in the number of row activations using IBFS and ICFS policies, respectively, compared to existing techniques while concurrently accessing data elements corresponding to large data structures. Such reduction results in superior PIM performance and energy efficiency.

[0011] II. Architecture FIG. 1 is a block diagram illustrating a memory controller 102 coupled to a processing-in-memory (PIM) capable memory 104 for implementing a cooperative address mapping scheme as described herein.

[0012] 1 includes a memory controller 102 and a PIM-enabled memory 104. The PIM-enabled memory 104 includes a PIM execution unit 106 and a memory structure 108. In one embodiment, the memory structure 108 comprises a bank. In another embodiment, the memory structure 108 comprises a channel including multiple banks. In such an embodiment, the PIM execution unit is coupled to the memory structure 108, which comprises a channel having multiple banks.

[0013] The memory controller 102 includes mapping logic 110 configured to manage the storage and access of data elements within the memory structure 108. Using the mapping logic 110, the memory controller 102 identifies specific row and column index values ​​for storing and accessing data elements within the memory structure 108. For example, the mapping logic 110 may implement an intra-bank frame striping (IBFS) policy that provides logic for storing data elements in a bank in a particular configuration. Such logic may also be used to access such data elements once they are stored from the bank. In another example, the mapping logic 110 may implement an intra-channel frame striping policy (ICFS) that provides logic for storing data elements in a channel in a particular configuration. Such logic may also be used to access such data elements once they are stored from the channel. The mapping logic 110 is implemented by computer hardware, computer software, or any combination of computer hardware and computer software. In some embodiments, the mapping logic is stored external to the memory controller 102 and is used by the memory controller 102 to perform the operations described herein.

[0014] The PIM-enabled memory 104 includes a memory structure 108 having a row decoder 112 and a column decoder 114. The memory structure 108 is coupled to a PIM execution unit 106. The row decoder 112 is configured to receive a row index value from the memory controller 102 and decode the received row index value to identify a row of the memory structure 108 that corresponds to the received row index value. The column decoder 114 is configured to receive a column index value from the memory controller 102 and decode the received column index value to identify a column of the memory structure 108 that corresponds to the received column index value. The combination of the decoded row index value and the decoded column index value is used to retrieve a value from a corresponding cell of the memory structure 108. The value retrieved from the cell is sent to the PIM execution unit 106 or returned to the memory controller 102, depending on the operation. The PIM execution unit includes logic to generate a result based on the value and send the result to the memory structure 108 and / or the memory controller 102.

[0015] In some embodiments, the memory structure 108 comprises a channel including multiple banks. In such embodiments, a separate row decoder 112 and column decoder 114 are associated with each of the multiple banks of the channel. The memory controller 102 includes logic for sending a bank index value to select a bank from the multiple banks, and then sending row and column index values ​​to open the row and access the data element. The retrieved data element is sent to the PIM execution unit 106 or returned to the memory controller 102, depending on the operation. The PIM execution unit includes logic for generating a result based on the data element and sending the result to the memory structure 108 and / or the memory controller 102.

[0016] III. Address Mapping Method A. Address interleaved memory mapping method FIG. 2 illustrates a memory system with two channels (channel 0 and channel 1). Each channel has four banks (banks 0, 1, 2, and 3). Assume there is one rank per channel, i.e., all four banks in each channel constitute one rank. Each bank comprises a 4×8 memory array. In some embodiments, each bank is coupled to a separate PIM execution unit. In some embodiments, each bank in a channel is coupled to the same PIM execution unit. Assume each cell in each memory structure holds one data element. Therefore, a PIM-enabled memory with two channels can store 256 (2×4×4×8) data elements, which can be represented by eight physical address bits. Assume the physical address space has four frames (frames 0-3), with four pages A, B, C, and D located in frames 0-3, respectively. Each page has 64 data elements. In an address interleaved memory mapping scheme, data elements from pages A, B, C, D are interleaved across channels, then across banks within each channel, then across columns within each bank, then across rows, i.e., the interleaving order is channel, bank, column, and row.

[0017] Figure 2 shows how 64 data elements from each page are distributed across a memory system using an address-interleaved memory mapping scheme. For example, for page A, which contains 64 data elements A0-A63, Figure 2 shows data element A0 indexed at row 0, column 0 of bank 0 of channel 0. Data element A1 is located at row 0, column 0 of bank 0 of channel 1. Data element A2 is located at row 0, column 0 of bank 1 of channel 0. Data element A3 is located at row 0, column 0 of bank 1 of channel 1, and so on.

[0018] Figure 3 shows how physical address bits are mapped for indexing in a PIM-enabled memory using address interleaved memory mapping. For example, bit 0 represents the channel number. Bits 1 and 2 represent the bank number. Bits 3-5 represent the column number, and bits 6-7 represent the row number.

[0019] As an example of performing a PIM operation using the memory system shown in Figure 2, assume a PIM calculation is being performed that adds two matrices element by element and stores the result in a third matrix. The matrices are defined as a, b, and c, stored in separate arrays and assigned to pages A, B, and C, respectively. To perform the calculation A[i] = B[i] + C[i] for all data elements i in pages A, B, and C, row R1 is opened to read the i-th data element of B. This element is stored in a temporary buffer. row R2 is opened to read the i-th data element of C. The addition of the data elements is then performed. The result of the addition is stored in the same temporary buffer. row R0 is opened to store the result in the i-th data element of A. The same process is repeated in each bank until all data elements have been processed.

[0020] Therefore, to perform a single addition operation and produce one result element, three row conflicts are encountered. Row conflicts hinder PIM efficiency from both a timing and energy perspective. In this example, a PIM arithmetic / logic operation may require a single memory clock cycle, but each row conflict may take 10 memory clock cycles, making data element placement a bottleneck in achieving high throughput.

[0021] B. Intrabank Frame Striping Policy The Intrabank Frame Striping (IBFS) policy stripes data elements from consecutive frames onto the same row within each bank. Co-located The data elements are mapped in different ways depending on the implementation. together (co-located)For example, corresponding subsets of data elements from consecutive frames may be interleaved in a single row or multiple rows of a bank. IBFS will be described with reference to the exemplary memory system shown in FIG. 2. In this example, the physical address space has four frames, namely, frames 0-3, and four pages, A, B, C, and D, are assigned to frames 0-3, respectively. For ease of explanation, an example of IBFS will be shown using a single bank.

[0022] Assume that bank 0 of channel 0 from FIG. 2 is used to illustrate IBFS. FIG. 4A illustrates bank 0 of channel 0 with an address interleaved memory mapping scheme as described with reference to FIG. 2. FIG. 4B illustrates bank 0 of channel 0 from FIG. 4A with IBFS address mapping. As shown in FIG. 4B, in IBFS mapping, for a data element i in pages A, B, and C, to perform the calculation A[i]=B[i]+C[i], one specific row is opened to read the i-th data element of B. The data element is then stored in a temporary buffer. The i-th data element of C is also read from the same row. An addition of the data elements is then performed. The result of the addition is stored in the same temporary buffer. To store the result in the i-th data element of A, the result is stored in the same row. The same process is repeated in each bank until all data elements have been processed.

[0023] Therefore, using IBFS mapping, only one row needs to be opened to perform the calculation A[i]=B[i]+C[i]. For example, to perform the first operation of the calculation A[i]=B[i]+C[i], i.e., the calculation for i=0, only row R0 and data elements B0, C0, and A0 from row R0 are required. Furthermore, using IBFS mapping, for example, the second operation of the calculation A[i]=B[i]+C[i], for i=8, can be performed using data elements A8, B8, and C8 from the same row R0. For the calculation A[i]=B[i]+C[i], the bank-by-bank PIM configuration achieves up to a six-fold reduction in the number of row conflicts using IBFS compared to previous techniques using address interleaved mapping for the memory system of FIG. 2. Therefore, PIM throughput is significantly improved using IBFS mapping.

[0024] To implement the IBFS policy, each row of the bank Co-locatedA number of stripes (NoS) is defined, indicating the number of frames to be placed in each row. Each such contiguous group of frames is called a superset of frames. As shown in FIG. 4B, row R0 contains an equal number of data elements from four contiguous frames, i.e., two data elements from each of pages A, B, C, and D. The data elements are arranged inside each row in a "striped" fashion, with each stripe containing elements from a single page. As shown in FIG. 4B, the NoS is 4 and each stripe size is 2. All of the rows (e.g., R0-R3) used to hold data elements from each of pages A, B, C, and D comply with the NoS and stripe size parameters, and such a set of rows is called a superset of rows. Thus, eight data elements from page A (e.g., A0, A8, A16, A24, A32, A40, A48, and A56) that would have been placed in row R0 using an address-interleaved memory mapping scheme are distributed across a superset of rows R0-R3 using IBFS mapping (e.g., A0, A8 to R0, A16, A24 to R1, A32, A40 to R2, and A48 and A56 to R3). Similarly, data elements from pages B, C, and D are distributed across the rows of the superset. Within each row, the stripe starting locations of data elements coming from different frames depend on the frame's relative position with respect to the first frame in the superset.

[0025] The following formulas are used to provide row and column index values ​​in physical memory according to the IBFS mapping. The same bits of the physical address as shown in FIG. 3 are used for row and column addressing, but different formulas are utilized by the memory controller to decode the bits. The following formulas are a function of external parameters such as the operating system's page size and NoS. The formulas are flexible and can be used in memory systems with different parameters. They are shown here for a system where all parameters are powers of two. Extensions to support parameters that are not powers of two will be apparent to those skilled in the art. Additionally, exemplary memory system parameters and operating system parameters are provided below along with sample calculations. Memory system specific parameters: Bank size = 32 Total banks = Banks per channel x Total channels = 4 x 2 = 8 Rows per bank = 4 Columns per row = 8 OS / user specific parameters: Frame size (i.e. page size) = 64 Number of stripes = 4 Required calculations: Stripe size = columns per row / number of stripes = 8 / 4 = 2 Frames per bank = bank size / (frame size / total banks) = 32 / (64 / 8) = 4 Rows per frame per bank = Rows per bank / Frames per bank = 4 / 4 = 1 The formula required for IBFS mapping is: IBFS column address = IBFS column base + IBFS column offset IBFS column base = relative frame position x stripe size = (number of frames % number of stripes) x stripe size IBFS column offset = column address % stripe size IBFS Row Address = IBFS Row Base + IBFS Row Offset 1 + IBFS Row Offset 2 IBFS row base = # of supersets x rows per frame per bank x # of stripes = (# of frames / # of stripes) x rows per frame per bank x # of stripes IBFS Row Offset 1 = (Row Address % Rows per Frame per Bank) x Number of Stripes IBFS row offset 2 = column address / stripe size

[0026] In some embodiments, for the above formula, the frame size is a multiple of total banks x columns per row. The number of stripes (NoS) is a power of 2, and the maximum allowed value is the value of columns per row. Column address and row address refer to basic binary numbers represented by the column address bits and row address bits, respectively. In one embodiment, all operations are integer operations.

[0027] C. Intra-channel frame striping policy The intra-channel frame striping policy (ICFS) stripes data elements from consecutive frames into corresponding rows across the banks of each channel. together (co-located) The data elements are mapped in different ways depending on the implementation. together (co-located) For example, corresponding subsets of data elements from consecutive frames may be interleaved across a single channel row or across multiple channel rows of a channel. ICFS will be described with reference to the exemplary memory system shown in FIG. 2. In this example, the physical address space has four frames, namely, frames 0-3, and four pages, A, B, C, and D, are assigned to frames 0-3, respectively. For ease of explanation, an example of ICFS will be shown using a single channel.

[0028] Assume that channel 0 from FIG. 2 is used to illustrate ICFS. FIG. 5A illustrates channel 0 with an address interleaved memory mapping scheme as described with respect to FIG. 2. FIG. 5B illustrates channel 0 from FIG. 2 with ICFS address mapping. As shown in FIG. 5B, to perform the calculation A[i]=B[i]+C[i] for an element i in pages A, B, and C using ICFS mapping, one particular row of Bank 1 is opened to read the i-th data element of B. The data element is then stored in a temporary buffer. One particular row of Bank 2 is opened to read the i-th data element of C. An addition of the data elements is then performed. The result of the addition is stored in the same temporary buffer. To store the result in the i-th data element of A, one particular row of Bank 0 is opened, and the result is stored back in that row of Bank 0.

[0029] Therefore, using ICFS mapping, only three rows across three different banks need to be opened to perform the calculation A[i]=B[i]+C[i]. For example, to perform the first operation of the calculation A[i]=B[i]+C[i], i.e., the calculation for i=0, only row R0 from bank 0, row R0 from bank 1, and row R0 from bank 2 are required for data elements A0, B0, and C0 from the corresponding rows. Furthermore, using ICFS mapping, for example, when i=8, 16, 24, 32, 40, 48, and 56, seven additional operations of the calculation A[i]=B[i]+C[i] can be performed using the same set of rows across three banks. For the calculation A[i]=B[i]+C[i], the per-channel PIM system achieves up to a 24-fold reduction in the number of row conflicts using ICFS compared to previous techniques using address interleaving mapping for the memory of FIG. 2. Therefore, PIM throughput is significantly improved using ICFS mapping.

[0030] To implement ICFS policies, a channel-level view of rows, called a "channel row," is utilized. A channel row comprises the same rows from all banks of a respective channel. For example, from FIG. 5B, a channel row comprises R0 from bank 0, R0 from bank 1, R0 from bank 2, and R0 from bank 3. The channel-level view of rows is contrasted with the bank-level view of rows, called a "row" or "bank row," which is utilized for IBFS mapping.

[0031] Each channel row Colocated A number of stripes (NoS) is defined that indicates the number of frames to be placed in a stripe. Each such contiguous group of frames is called a superset of frames. As mentioned above, all of the bank rows with the same number are considered together as a channel row, e.g., row R0 of banks 0, 1, 2, and 3 are considered together as channel row R0. In ICFS mapping, a bank row is considered one single unit, and stripe size is measured using this unit.

[0032] As shown in FIG. 5B, channel row R0 contains an equal number of bank-row-sized data elements from four consecutive frames, i.e., one such element from each of pages A, B, C, and D, arranged in a "striped" format across the banks, with each stripe containing elements from a single page. As shown in FIG. 5B, the NoS is 4 and each stripe size is 1. All channel rows (R0 through R3) used to hold data elements from each of pages A, B, C, and D are aligned according to the NoS and stripe size parameters, and such a set of channel rows is called a superset of channel rows. Thus, the four bank-row-sized data elements from page A would have been placed in channel row R0 using an address interleaved memory mapping scheme and are distributed across the superset of channel rows R0 through R3 using ICFS mapping (e.g., R0 of bank 0 is in channel row R0, R0 of bank 1 is in channel row R1, R0 of bank 2 is in channel row R2, and R0 of bank 3 is in channel row R3). Similarly, data elements from pages B, C, and D are distributed across the channel rows of the superset. Within each channel row, the stripe starting positions of data elements coming from different frames depend on the frame's position relative to the first frame in the superset.

[0033] The following formulas are used to provide row index and bank index values ​​in physical memory according to the ICFS mapping. The same bits of the physical address as shown in FIG. 3 are used for row and bank addressing, but different formulas are utilized by the memory controller to decode the bits. The formulas are a function of external parameters such as OS page size and NoS. The formulas are flexible and can be used in memory systems with different parameters. They are shown here for a system where all parameters are powers of two. Extensions to support parameters that are not powers of two will be apparent to those skilled in the art. Additionally, exemplary memory system parameters and operating system parameters are provided below along with sample calculations. Memory system specific parameters: Channel Size = 16 Total Channels = 2 Channel Rows per Channel = 4 Bank per channel row = 4 Parameters selected by the user: Frame size (i.e. page size) = 8 Number of stripes = 4 Required calculations: Stripe size = Banks per channel row / Number of stripes = 4 / 4 = 1 Frames per channel = channel size / (frame size / total channels) = 16 / (8 / 2) = 4 Channel rows per frame per channel = Channel rows per channel / Frames per channel = 4 / 4 = 1 The formula required for ICFS mapping is: ICFS Bank Address = ICFS Bank Base + ICFS Bank Offset ICFS bank base = relative frame position x stripe size = (number of frames % number of stripes) x stripe size ICFS bank offset = bank address % stripe size ICFS Row Address = ICFS Row Base + ICFS Row Offset 1 + ICFS Row Offset 2 ICFS row base = number of supersets x channel rows per frame per channel x number of stripes = (number of frames / number of stripes) x channel rows per frame per channel x number of stripes ICFS Row Offset 1 = (Row Address % Channel Rows per Frame per Channel) x Number of Stripes ICFS row offset 2 = bank address / stripe size

[0034] In some embodiments, for the above formula, the frame size is a multiple of total channels x banks per channel. The number of stripes (NoS) is a power of 2, with the maximum allowed value being the number of banks per channel. Bank address and row address refer to the base binary number represented by the bank address bits and row address bits, respectively. In one embodiment, all operations are integer operations.

[0035] D. Implementation Details IBFS and ICFS are built into the memory controller alongside existing policies. The formulas provided include division / multiplication / modulo operations by power-of-two numbers, which can be efficiently achieved by shift operations. The memory controller is configured to use generic IBFS and ICFS depending on whether the system is non-PIM / PIM-per-bank capable / PIM-per-channel capable. For example, for a system configuration in which PIM execution units are coupled to memory structures at the bank level, IBFS may be selected for use by the memory controller. For a system configuration in which PIM execution units are coupled to memory structures at the channel level, ICFS may be selected for use by the memory controller.

[0036] Parameters such as frame size and NoS are provided to PIM-enabled memory systems using IBFS and ICFS. Frame size selection is provided according to operating system specifications. NoS can be statically provided according to user preferences at system startup (e.g., via the Basic Input / Output System) or dynamically provided to achieve flexibility.

[0037] Modifications to the operating system and compiler are made to allow a PIM-aware memory system to take advantage of IBFS and ICFS. The operating system can recognize memory allocations (mallocs) that are associated with each other by receiving hints from the compiler. The programmer can provide hints to the compiler indicating which data structures are involved in the PIM calculation. The compiler then determines the related data structures that participate together in the PIM calculation, called PIM data structures. The compiler selects a number of such PIM data structures up to the NoS and assigns them a single color. Each set of PIM data structures is assigned a distinct color. Whenever a malloc call is made for any such PIM data structure, the PIM data structure is allocated the required number of pages in the virtual address space. The operating system is notified of the distinct colors associated with the virtual pages, which are the colors associated with each PIM data structure. At this point, no physical memory (frames) are allocated. The same process occurs when any other PIM data structure from the same set is allocated. Then, when a program accesses such a page, the operating system allocates physical memory for the requested page with the same color within a chunk of the superset of frames. If each such data structure has n pages, one superset of frames is allocated on demand to the i-th page of the PIM data structure, another superset of frames is allocated on demand to the (i+1)-th page of the PIM data structure, and so on up to the n-th page.

[0038] For example, consider page allocations for a[], b[], and c[] for the PIM calculation a[] = b[] + c[]. Assume each allocation requires four pages. All 12 (3 x 4) pages are allocated red. The operating system ensures that page[i] of all allocations are similar, i.e., a request for allocation of page[0] of a[] triggers allocation of page[0] for b[] and c[]. All of the allocation requests are filled with consecutive frames contained in the same superset of frames. Instead of triggering allocation of page[0] of b[] and c[], allocation is triggered when a page is requested. However, a frame is reserved for an unallocated page within the same superset of frames to which page[0] of [] is allocated. To avoid fragmentation, any unused frames in the superset of frames can be returned to the operating system to be allocated in future requests.

[0039] When a program issues any PIM command that specifies a virtual address, the virtual address is translated into a physical address according to the normal address translation process. The physical address is mapped in a PIM-friendly way because the memory controller is configured to use IBFS and ICFS.

[0040] As mentioned above, IBFS and ICFS can be implemented as static or dynamic schemes. In a static scheme, the NoS is bound to an arbitrary constant (e.g., 2 / 4 / 8) at system startup according to user preference. The memory structure is mapped with a specified NoS, and the mapping cannot be changed without restarting the system with a different NoS. The static scheme does not require any extra steps beyond those proposed above.

[0041] The dynamic scheme provides flexibility and a finer degree of control over memory mapping by enabling PIM-friendly address mapping for pages corresponding to PIM data structures and simultaneously generic address mapping for other memory pages according to user preferences. As proposed above, the compiler identifies different sets of related PIM data structures with required NoS and marks each such set with a distinct color. During physical memory allocation for pages of any such set, the operating system allocation process described above ensures a superset of the required frames, such that the superset contains the NoS number of frames. Any data structure and / or page can be associated with only one NoS value throughout its lifetime. Whenever any memory access, e.g., a read or write-back from a storage file to these frames, or any access from a processor core, is made to these pages, the memory controller is always informed of the corresponding NoS information along with the physical memory address received at the memory controller after virtual-to-physical memory address translation for any given virtual address of the page. The memory controller recognizes the NoS information contained in each such memory request and decodes the physical memory address per IBFS or ICFS. Therefore, the NoS information for each data structure / page must be tracked and communicated to the memory controller along with the physical memory address of any memory access request.

[0042] To achieve the communication of NoS information to the memory controller, four possible approaches are utilized: (i) an instruction-based approach, (ii) a page table entry (PTE)-based approach, (iii) a configuration register approach, and (iv) a mode register-based approach. In the instruction-based approach, the instruction set architecture (ISA) is extended to include special load / store instructions for accessing any memory structure annotated by the user to be used in PIM calculations. In a compiler pass in which the compiler first selects a set of associated PIM data structures to be accessed along with the necessary NoS information, the compiler tracks the NoS information corresponding to each such memory structure, for example, in its symbol table. Later, for these memory structures, as needed, the compiler issues special load / store instructions (e.g., LOAD_PIM, STORE_PIM) with the NoS information embedded in the instruction. In one embodiment, for a memory system that supports only one IBFS or ICFS at a time, no dynamic specifier is required to communicate which of the IBFS or ICFS needs to be used by the memory controller. In another embodiment, for a memory system that dynamically enables both IBFS and ICFS, a single bit policy indicator information is embedded in special load / store instructions for PIM data structures along with NoS information used by the memory controller to select between IBFS or ICFS. The policy indicator, like the NoS information described herein, is determined and managed by the compiler.

[0043] The memory controller is capable of recognizing such special read / store commands, finding the necessary NoS information from the received command, and correctly decoding the physical memory address mapping according to IBFS, ICFS, or generic address mapping. Depending on the implementation choice, different numbers of bits can be used to convey the NoS information. For example, using only two bits, four different NoS values ​​of 1, 2, 4, and 8 are encoded, where an NoS value of 1 indicates a default mapping policy, such as an address-interleaved memory mapping scheme. For normal read / store commands where no NoS information is provided, the memory controller uses an NoS value of 1 and defaults to a default mapping policy, such as an address-interleaved memory mapping scheme.

[0044] In the PTE-based approach, for each allocated page, a set of bits in the PTE is used to hold the corresponding NoS information for the page. Only two bits are required to encode four different NoS values: 1, 2, 4, and 8, where a NoS value of 1 indicates a default mapping policy, such as an address-interleaved memory mapping scheme. A default NoS value of 1 is also stored for any normal page that does not belong to any PIM calculation. These bits in the PTE indicating the NoS value are cached in a translation lookaside buffer (TLB) entry along with the physical memory address. For any memory access request, the corresponding NoS information is included along with the physical memory address. When any such memory request reaches the memory controller, the memory controller recognizes the NoS information included with such memory request and decodes the physical memory address mapping according to the IBFS, ICFS, or generic address mapping. In one embodiment, for a memory system that supports only one IBFS or ICFS at a time, no dynamic specifier is required to convey whether the IBFS or ICFS needs to be used by the memory controller. In another embodiment, for a memory system that dynamically enables both IBFS and ICFS, a single bit policy indicator information is included in any memory access request along with NoS information that is used by the memory controller to select between IBFS or ICFS. The policy indicator is selected and managed similarly to the NoS information in the PTE-based approach.

[0045] In the configuration register-based approach, for ranges of physical memory addresses that store PIM data structures, one or more configuration registers associated with the memory controller are used to hold corresponding NoS information for each physical address range. The aggregation of PIM data structures into ranges of physical addresses and the programming of one or more configuration registers with NoS information are managed by software (e.g., an operating system) or hardware. Only two bits are required to encode four different NoS values: 2, 4, 8, and 16. Because the configuration registers only hold the address ranges that hold PIM data structures, this approach does not require explicitly storing indicators for default mapping policies, such as an address-interleaved memory mapping scheme. For any memory access request received by the memory controller, the associated physical memory address is used to query the configuration registers. If the address falls within a range programmed in one or more configuration registers, the corresponding NoS information from the configuration registers is associated with the memory access. For normal accesses that do not access PIM data structures, the configuration registers do not hold an entry, and a default mapping, such as an NoS value of 1, is used. The memory controller recognizes the NoS information and decodes the physical memory address mapping according to IBFS or ICFS or generic address mapping. In an embodiment in which the memory system dynamically supports both IBFS and ICFS, a single bit policy indicator information is also maintained in the configuration register along with the NoS information to indicate whether IBFS or ICFS should be used by the memory controller for the corresponding physical address range. The policy indicator is selected and managed similarly to the NoS information in the configuration register-based approach.

[0046] In a mode register-based approach, software running on the processor or hardware writes NoS information for all subsequent memory accesses to a single mode register associated with the memory controller. Only two bits are available to encode four different NoS values: 1, 2, 4, and 8, with a NoS value of 1 indicating a default mapping policy, such as an address-interleaved memory mapping scheme. In one embodiment, all memory accesses following programming the mode register use the NoS information from the mode register. The mode register may be updated by software as frequently as necessary to change the NoS information associated with subsequent memory accesses. In another embodiment, instead of a single mode register, the memory controller has a mode register for each memory request generator (e.g., each core or each thread) in the system. In such an embodiment, the settings from each mode register apply only to memory requests originating from the associated requester. In an embodiment in which the memory system dynamically supports both IBFS and ICFS, a single-bit policy indicator information is also written into the mode register along with the NoS information to indicate whether IBFS or ICFS should be used by the memory controller for subsequent memory accesses. The policy indicators are selected and managed similarly to the NoS information in the mode register-based approach.

[0047] The proposed hardware / software cooperative address mapping scheme provides a reduction in the number of row activations while accessing data elements corresponding to large data structures accessed with IBFS and ICFS policies, respectively, compared to previous schemes for the memory system of Figure 2. Such a reduction results in superior PIM performance and energy efficiency.

[0048] E. Exemplary Procedures 6 is a flow diagram 600 illustrating an approach for accessing memory using IBFS and ICFS address mapping. For purposes of explanation, FIG. 6 will be described with respect to the architecture shown in FIG.

[0049] In step 602, a memory access request for one data element of a plurality of data elements stored in memory is received by memory controller 102. In some embodiments, the memory comprises a process-in-memory (PIM) enabled memory. The memory access request comprises a physical memory address and a number of stripes (NoS) value. For example, a host processor may submit a memory access request and send the request to memory controller 102.

[0050] In one embodiment, the NoS value is derived from a specific instruction set architecture (ISA) instruction. In another embodiment, the NoS value is derived from one or more bits of a page table entry (PTE). In another embodiment, the NoS value is derived from one or more configuration registers associated with a range of physical memory addresses that includes the physical memory address of the request. In another embodiment, the NoS value is derived from a mode register associated with a memory controller.

[0051] In step 604, using the mapping, row and column index values ​​are calculated for the data elements by memory controller 102 based on the received physical memory addresses and NoS values. For example, memory controller 102 uses mapping logic 110 to calculate row and column index values ​​for the data elements.

[0052] In one embodiment, the mapping comprises an IBFS mapping. In such an embodiment, the memory comprises a plurality of banks, each bank of the plurality of banks including a plurality of rows, a plurality of columns, and a corresponding execution unit. In some embodiments, the memory comprises a PIM-enabled memory, and the corresponding execution unit comprises a PIM execution unit. The plurality of data elements are stored in the memory by grouping, for a particular bank of the plurality of banks, data elements of the plurality of data elements corresponding to distinct operands of a computation into a single row of the plurality of rows of the particular bank.

[0053] In another embodiment, the mapping comprises an ICFS mapping. In such an embodiment, the memory comprises a plurality of channels, each channel of the plurality of channels comprising a plurality of banks, a plurality of channel rows, and a corresponding execution unit. In some embodiments, the memory comprises a PIM-enabled memory, and the corresponding execution unit comprises a PIM execution unit. Each channel row of the plurality of channel rows comprises the same row from all of the plurality of banks for a respective channel of the plurality of channels. The plurality of data elements are stored in the memory by grouping, for a particular channel of the plurality of channels, data elements of the plurality of data elements corresponding to separate operands of a computation into a single channel row of the plurality of channel rows for the particular channel.

[0054] In step 606, data elements are retrieved from memory by memory controller 102 based on the row index value and column index value. In one embodiment, for IBFS, memory controller 102 sends a row index value to row decoder 112 and a column index value to column decoder 114 to open a row and access a data element. In another embodiment, memory structure 108 is viewed as a channel having multiple banks, and for ICFS, memory controller 102 sends a bank index value to select a bank from the multiple banks, and then sends row index and column index values ​​to open a row and access a data element. PIM execution unit 106 aggregates such data elements, performs the necessary operations, and sends the results to memory structure 108 and / or memory controller 102.

[0055] In one embodiment, for a memory system that supports only one of IBFS or ICFS at a time, no dynamic specifier is required to communicate whether IBFS or ICFS should be used by the memory controller. In another embodiment, for a memory system that dynamically allows both IBFS and ICFS, policy indicator information is communicated along with NoS information that is used by the memory controller to select between IBFS or ICFS. The policy indicator information is determined and managed similarly to the NoS information described herein.

[0056] F. Applications In some embodiments, techniques such as IBFS and ICFS can be useful when applied to various machine learning (ML) applications. For example, ML applications implementing deep neural networks (DNNs) typically use operations such as adding two large matrices element-wise and storing the result in another matrix (e.g., A[i] = B[i] + C[i]). In a DNN using a residual unit with multiple layers as components, one layer of each residual unit may perform the element-wise matrix addition operation. Using IBFS and ICFS techniques, if corresponding subsets of data elements of such matrices (e.g., A[], B[], C[]) are interleaved in the same DRAM bank-level row or channel-level row, fewer row conflicts are encountered while accessing these data elements to perform the element-wise matrix addition operation. Therefore, IBFS and ICFS enable DNNs to perform the same operations with fewer row conflicts, effectively improving the speed of various ML applications.

Claims

1. A memory controller, It has mapping logic, The mapping logic: determining where a plurality of data elements are stored in a memory for a computation involving the plurality of data elements, the memory having a plurality of banks, each bank of the plurality of banks including a plurality of rows, a plurality of columns, and a corresponding execution unit; for a particular bank of the plurality of banks, interleaving a corresponding subset of data elements of the plurality of data elements into a single row of the plurality of rows of the particular bank, each corresponding subset of data elements corresponding to a different page; configured to: Memory controller.

2. a particular execution unit corresponding to the particular bank performs a first operation of a computation based on the corresponding subset of data elements interleaved in the single row of the plurality of rows of the particular bank; the particular execution unit performs a further operation of the computation based on the corresponding subset of data elements interleaved in the single row of the plurality of rows of the particular bank. The memory controller of claim 1 .

3. The mapping logic: and further configured, in response to a memory access request for a particular data element of the plurality of data elements, the memory access request including a physical memory address and an associated Number of Stripes (NoS) value, to decode the physical memory address based on the NoS value to generate a bank index, a row index value, and a column index value for the particular data element. The memory controller of claim 1 .

4. the corresponding subset of data elements of the plurality of data elements are stored in the particular bank of the plurality of banks based on a Number of Stripes (NoS) value that defines the number of frames to be placed together in a single row of the plurality of rows of the particular bank; The memory controller of claim 1 .

5. A memory controller, It has mapping logic, The mapping logic: determining, for a computation involving a plurality of data elements, where the plurality of data elements are stored in a memory, the memory having a plurality of channels, each channel of the plurality of channels including a plurality of banks, a plurality of channel rows, and a corresponding execution unit; each channel row in the plurality of channel rows is included in a corresponding channel and includes the same row from all banks in the plurality of banks in the corresponding channel; for a particular channel of the plurality of channels, interleaving a corresponding subset of data elements of the plurality of data elements onto a single channel row of the plurality of channel rows for the particular channel; configured to: Memory controller.

6. a particular execution unit corresponding to the particular bank performs a first operation of a calculation based on the corresponding subset of data elements interleaved in the single channel row of the plurality of channel rows of the particular channel; the particular execution unit performs an additional operation of the computation based on the corresponding subset of data elements interleaved in the single channel row of the plurality of channel rows of the particular channel. The memory controller of claim 5.

7. The mapping logic: and further configured, in response to a memory access request for a particular data element of the plurality of data elements, the memory access request including a physical memory address and a number of stripes (NoS) value, to decode the physical memory address based on the NoS value to generate a bank index value and a row index value for the particular data element. The memory controller of claim 5.

8. the corresponding subset of data elements of the plurality of data elements are stored in the particular channel based on a Number of Stripes (NoS) value defining a number of frames to be placed together in a single channel row of the plurality of channel rows for each channel of the plurality of channels; The memory controller of claim 5.

9. Based on the NoS value, a plurality of frames are arranged at the same location in each of the plurality of channel rows of the specific channel among the plurality of channels. The memory controller of claim 8.

10. 1. A method comprising: receiving a memory access request for a data element from a plurality of data elements stored in a memory, the memory access request including a physical memory address and a number of stripes (NoS) value; the mapping logic calculating an index value for the data element based on the physical memory address and the NoS value; the memory controller retrieving the data element from a memory structure based on the index value. method.

11. the index values ​​include a row index value and a column or bank index value for the data element; the data elements are retrieved from the memory based on the row index value and the column or bank index value. The method of claim 10.

12. the memory comprises a plurality of banks, each bank of the plurality of banks including a plurality of rows, a plurality of columns, and a corresponding execution unit; the plurality of data elements are stored in the memory by interleaving, for a particular bank of the plurality of banks, a corresponding subset of data elements of the plurality of data elements into a single row of the plurality of rows of the particular bank; The method of claim 10.

13. the memory comprises a plurality of channels, each channel of the plurality of channels including a plurality of banks, a plurality of channel rows, and a corresponding execution unit; each channel row in the plurality of channel rows is included in a corresponding channel and includes the same row from all banks in the plurality of banks in the corresponding channel; the plurality of data elements are stored in the memory by interleaving, for a particular channel of the plurality of channels, a corresponding subset of data elements of the plurality of data elements onto a single channel row of the plurality of channel rows for the particular channel; The method of claim 10.

Images