Approach for managing near and non-near memory processing instructions in a memory controller - Patents.com

The memory controller efficiently manages PIM and non-PIM instructions using a single page table to track memory element status, improving computational efficiency and reducing power consumption by integrating both types of instructions.

JP2024545215APending Publication Date: 2024-12-05ADVANCED MICRO DEVICES INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2024535347
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-12-23
Filing Date
2022-12-01
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Existing memory controllers struggle to efficiently manage both Processing In Memory (PIM) and non-PIM instructions due to their different processing requirements, leading to performance penalties or increased costs when handling these instructions separately.

Method used

A memory controller approach that uses a single page table to track the status of memory elements, allowing simultaneous management of PIM and non-PIM instructions by configuring memory elements based on the most recent instruction, reducing the need for separate handling and avoiding performance penalties.

Benefits of technology

This approach enhances computational efficiency and reduces power consumption by enabling seamless integration of PIM and non-PIM instructions without the need for dedicated controllers, thus optimizing memory operations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024545215000001_ABST
    Figure 2024545215000001_ABST
Patent Text Reader

Abstract

An approach is provided for managing PIM and non-PIM instructions in a memory controller. The memory controller queues PIM and non-PIM instructions and selects the next instruction to be processed based on various selection criteria. The memory controller maintains and uses a page table to properly configure memory elements, such as banks, in a memory module for the next memory instruction, whether a PIM or non-PIM instruction. The page table tracks the status of the memory elements at the time of the most recent memory instruction issued. The page table includes an "All Bank" entry that indicates the status of the banks after processing the most recent PIM instruction. For example, the All Bank entry indicates whether all banks have open rows, and if so, specifies the open rows in all banks.
Need to check novelty before this filing date? Find Prior Art

Description

[Background technology]

[0001] The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Thus, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Moreover, it should not be assumed that any of the approaches described in this section are well understood, conventional, or conventional by virtue of their inclusion in this section.

[0002] Since computational throughput scales faster than memory bandwidth, various techniques have been developed to keep data supplied to the growing computational capacity. Processing In Memory (PIM) embeds processing power within the memory module so that tasks can be processed directly within the memory module. In the context of Dynamic Random-Access Memory (DRAM), an exemplary PIM configuration includes vector computation elements and local registers. The vector computation elements and local registers allow the memory module to perform some computations, such as arithmetic calculations, locally. This allows the memory controller to launch local computations in multiple memory modules in parallel without requiring data movement across the memory module interface, which can significantly improve performance, especially for data-intensive workloads.

[0003] One of the challenges of PIM is how to process both PIM and non-PIM instructions in a memory controller given their different processing requirements. For example, non-PIM instructions are directed to a single DRAM bank, while PIM instructions are broadcast instructions directed to all banks in a DRAM module. Thus, traditional page tables that track the state of individual banks are insufficient for PIM instructions. As another example, non-PIM instructions typically require host-memory module data communication to write data to memory and read data from memory. In contrast, PIM instructions use a "fire and forget" semantic that does not trigger host-memory module data communication other than providing metadata over the data bus to process the PIM instruction. Exemplary metadata includes information such as instruction index value, process or thread ID, constant values ​​representing source operands, PIM register index, etc. Thus, the memory controller must process PIM instructions differently than non-PIM instructions.

[0004] One possible solution to address these issues is to separate the PIM and non-PIM instruction traffic as they arrive at the memory controller by blocking either of the two instruction streams and processing them separately. However, this approach can cause a large performance penalty due to delaying the processing of either of the two instruction streams. Another solution is to use a dedicated memory controller for PIM instructions, which comes with increased area and power costs. In view of the above, therefore, there is a need for an approach for processing both PIM and non-PIM instructions by a memory controller that avoids the limitations and costs of current solutions.

[0005] Embodiments are illustrated by way of example, and not by way of limitation, in the accompanying figures, in which like references refer to similar elements and in which: FIG. [Brief description of the drawings]

[0006] [Figure 1A] 1 is a flow diagram illustrating an approach for managing broadcast near memory operation instructions and non-near memory operation instructions in a memory controller. [Figure 1B] FIG. 2 is a block diagram showing a memory configuration. [Figure 1C] FIG. 2 illustrates an example embodiment of a memory controller including an instruction queue, processing logic, instruction metadata, and a page table. [Figure 1D] FIG. 1 is a block diagram logically illustrating the two-stage process used by the scheduler, including a first stage picker and a second stage picker. [Figure 2A] FIG. 1D illustrates a page table that is an example embodiment of the page table of FIG. 1C. [Figure 2B] FIG. 11 is a flow diagram illustrating the use of a page table to perform non-PIM to non-PIM operations. [Figure 2C] FIG. 13 illustrates the state of the page table after a non-PIM instruction directed to row 4 of bank 3 has been processed. [Figure 2D] FIG. 1 is a flow diagram illustrating a transition from non-PIM to PIM. [Figure 2E] FIG. 11 illustrates the page table after all bank entries have been updated to specify that row 5 is open in all banks. [Figure 2F] FIG. 13 illustrates the page table after all bank entries have been updated to change open rows from 5 to 3. [Figure 2G] FIG. 1 is a flow diagram illustrating an approach for performing PIM to PIM processing. [Figure 2H] FIG. 13 illustrates the state of the page table after a PIM instruction directed to row 3 of all banks has been processed. [Figure 2I] FIG. 1 is a flow diagram illustrating a transition from PIM to non-PIM. [Figure 2J] FIG. 13 illustrates the page table after all bank entries have been invalidated and the "PGT 0" entry has been updated to indicate that row 8 is open in bank 0. [Diagram 3] FIG. 1 is a flow diagram illustrating an approach for processing PIM and non-PIM instructions in a memory controller. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments. However, it will be apparent to those skilled in the art that the embodiments may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the embodiments. I. Overview II. Structure III. SELECTION OF INSTRUCTIONS FOR PROCESSING IV. Exemplary Page Table with "All Banks" Status V. Non-PIM to Non-PIM Processing VI. Transition from Non-PIM to PIM VII. PIM to PIM Processing VIII. Transition from PIM to non-PIM

[0008] (I. Overview) An approach is provided for managing PIM and non-PIM instructions in a memory controller that queues PIM and non-PIM instructions and selects the next instruction to process based on various selection criteria, including but not limited to instruction allocation bandwidth, number of pending instructions, amount of time or number of cycles since the most recent instruction was issued, presence of a marker, or global age.

[0009] The memory controller maintains and uses page tables to properly configure memory elements, such as banks, in a memory module for the next memory instruction, whether a PIM instruction or a non-PIM instruction. The page table tracks the status of the memory elements at the time of the most recent memory instruction issued. For example, the page table indicates whether a bank has open rows, and if so, specifies the open rows in the bank. According to one embodiment, the page table includes an "All banks" entry that indicates the status of the banks after processing the most recent PIM instruction. For example, the All banks entry indicates whether all banks have open rows, and if so, specifies the open rows in all banks.

[0010] The approach of using a single full bank entry in a page table to track the status of a memory element after processing the most recent PIM instruction allows the memory controller to efficiently manage PIM and non-PIM instructions without having to separate PIM and non-PIM instructions, which can cause a large performance penalty in terms of both computational cost and power consumption. The approach also avoids having a dedicated memory controller for PIM instructions, which increases the area and power cost of scheduling instructions to memory. The embodiments include the memory controller issuing PIM instructions in program order to maintain correctness. Although the embodiments are described herein in the context of PIM and non-PIM instructions for illustration, the embodiments are applicable to any type of near memory processing instructions and non-near memory processing instructions.

[0011] 1A is a flow diagram 100 illustrating an approach for managing PIM and non-near memory processing instructions in a memory controller. In step 102, the memory controller receives and, for example, stores memory instructions in one or more queues. These may include any number and types of PIM and non-PIM instructions.

[0012] The memory controller selects a memory instruction to be processed in step 104. For example, the memory controller may select the memory instruction from one or more queues based on various selection criteria.

[0013] In step 106, the memory controller configures the target memory elements for the selected memory instruction. According to one embodiment, the memory controller uses the page table, and in particular all bank entries in the page table, to determine the current status of the memory elements and configures the target memory elements accordingly, as described in more detail below.

[0014] In step 108, the memory controller issues the selected memory instruction, and in step 110, the memory controller updates the status data of the memory device, which includes updating the page tables, as described in more detail below.

[0015] (II. Structure) 1B is a block diagram illustrating a computing configuration 120. The computing configuration 120 includes a memory controller 130 and a memory module 140. The computing configuration 120 may be implemented in any type of computing device, including, for example, a processor and a system on a chip (SoC). The memory controller 130 and the memory module 140 are communicatively coupled via an instruction bus 150 and a data bus 160. As used herein, the term "bus" refers to any type of wired or wireless electrical coupling, such as wires, electrical lines, and / or wireless communication lines. Although embodiments are illustrated and described herein in the context of a single memory module 140 for illustrative purposes, the embodiments are applicable to memory configurations having any number of memory modules. The computing configuration 120 may include other elements not illustrated and described herein for illustrative purposes.

[0016] Memory module 140 may be, for example, a DRAM memory module and includes a memory array having N+1 banks (Bank 0 through Bank N) with corresponding PIM execution units (PIM Execution Unit 0 through PIM Execution Unit N). Each bank includes cells defined by a two-dimensional array, although embodiments are not limited to DRAM memory modules or banks per se, or to banks having memory cells defined by a two-dimensional array, but are applicable to other memory cell organizations. Memory arrays and banks may include other elements, such as buffers and decoders, which for purposes of illustration are not shown in the figures and will not be described herein.

[0017] A PIM execution unit includes processing logic and storage, such as local registers, for processing PIM instructions, e.g., to perform logical operations, arithmetic operations, etc. While the example of Figure 1 shows a separate PIM execution unit for each bank, in an embodiment, a fewer number of PIM execution units, and corresponding processing logic and storage, service multiple (or all) banks, including any type of near memory operations.

[0018] Memory controller 130 manages the flow of data to and from memory module 140 and may be implemented as a stand-alone element, for example, on a separate chip from the microprocessor, on the same chip as but separate from the microprocessor, or integrated within the microprocessor as an integrated memory controller. Embodiments are applicable to any type of memory controller 130, and according to one embodiment, memory controller 130 includes processing logic configured to issue instructions to memory module 140.

[0019] 1C illustrates an example embodiment of memory controller 130, including instruction queue 170, scheduler 172, processing logic 174, instruction metadata 176, and page table 178. Memory controller 130 may vary depending on the particular embodiment and may include fewer or additional elements that, for purposes of illustration, are not shown in the figures or described herein. Additionally, the functionality provided by the various elements of memory controller 130 may be combined in any manner depending on the particular embodiment.

[0020] The instruction queue 170 stores memory instructions received by the memory controller 130 from one or more threads executing, for example, on a Central Processing Unit (CPU), Graphics Processing Unit (GPU), or other processor. According to one embodiment, the memory instructions include PIM instructions and non-PIM instructions. A PIM instruction is a broadcast memory instruction directed to multiple memory elements in a memory module, such as multiple banks in a DRAM memory module. The target memory elements are specified by one or more bit values, such as a bit mask in the PIM instruction, specifying any number, including all, of the available target memory elements. The PIM instruction causes some operation, such as a logical operation and / or a calculation, to be performed by multiple, including all, of the specified memory elements in the memory module 130. As one non-limiting example, the PIM instruction specifies that in each target bank, a value is read from memory at a specified row and column into a local register, an arithmetic operation is performed on the value, and the result is stored back in memory. Examples of non-near memory processing instructions include, but are not limited to, load (read) instructions, store (write) instructions, etc. Unlike PIM instructions, which are broadcast memory processing instructions and are generally directed to multiple target banks, non-near memory processing instructions are directed to a single bank, i.e., they are bank-specific.

[0021] Instruction queue 170 stores instructions issued by memory controller 130. According to one embodiment, memory controller 130 stores PIM and non-PIM instructions in instruction queue 170 as they are received. Instructions are stored in open slots in instruction queue 170, and instructions are grouped in the order they are received, by type or regardless of type, depending on the particular embodiment. Instruction queue 170 is implemented by any type of storage capable of storing memory instructions. Although embodiments are illustrated and described herein in the context of instruction queue 170 being implemented as a single element, embodiments are not limited to this example, and according to embodiments, instruction queue 170 is implemented by a separate instruction queue for each of multiple elements, e.g., banks in memory module 140. Scheduler 172 schedules memory instructions for processing. According to one embodiment, scheduler 172 selects instructions to be processed based on various selection criteria, as described in more detail below.

[0022] Processing logic 174 stores memory instructions in instruction queue 170 and may be implemented by computer hardware, computer software, or any combination of computer hardware and computer software. Instruction metadata 176 specifies one or more attributes of the memory instructions stored in instruction queue 170, such as the type of instruction (e.g., PIM or non-PIM), the date and / or time the memory instruction was stored in instruction queue 170, the relative ordering of the memory instructions within instruction queue 170, the type of instruction, the instruction status, etc.

[0023] The page table 178 includes data specifying the current state of one or more memory elements in the memory module and is used by the processing logic 174 to configure the memory elements to process memory instructions. For example, the page table 178 includes an entry for each bank of the memory module 140, with each entry specifying whether the corresponding bank currently has an open (activated) row from a most recent non-PIM instruction directed to that bank, and which rows, if any, are currently open, e.g., via a row ID. When a row is opened or closed (deactivated), the processing logic 174 updates the corresponding entry in the page table 178. According to one embodiment, the page table 178 also includes a full bank entry that specifies whether the bank has an open row from a most recent PIM instruction, and if so, which row. The use of the full bank entries by the memory controller 130 to manage PIM and non-PIM instructions is described in more detail below.

[0024] III. SELECTION OF INSTRUCTIONS FOR PROCESSING According to one embodiment, scheduler 172 is configured to use a two-stage process to select a memory instruction to be processed. In a first stage, scheduler 172 selects between a PIM instruction and a group of candidate non-PIM instructions based on various criteria. In a second stage, scheduler 172 selects a particular PIM instruction from instruction queue 170 or a particular non-PIM instruction from the group of candidate non-PIM instructions from instruction queue 170 depending on whether a PIM instruction or a group of candidate non-PIM instructions was selected in the first stage.

[0025] 1D is a block diagram 180 that logically illustrates the two-stage process used by scheduler 172, including first-stage picker 180 and second-stage picker 182. First-stage picker 180 and second-stage picker 182 represent functionality of scheduler 172 implemented in computer hardware, computer software, or any combination of computer hardware and software.

[0026] The first stage picker 180 is configured to select between PIM instructions or a group of candidate non-PIM instructions from the instruction queue 170 based on selection criteria that vary depending on the particular implementation. For the group of candidate non-PIM instructions, the ready non-PIM instructions in the instruction queue 170 are classified as page hit, page miss, or page conflict based on the bank and row specified by each instruction and the status of the bank and row in the page table 178. An instruction ready is defined by the next instruction to be issued, the earliest time this instruction can be issued without violating memory timing parameters, the memory element (bank) status of the instruction, and the last instruction issued to the channel. According to an embodiment, the non-PIM instructions are further classified by other criteria including, but not limited to, real-time traffic, e.g., display refresh, I / O device (Input-Output devices) traffic, etc. One ready candidate non-PIM page hit instruction, one ready candidate non-PIM page miss instruction, and one ready candidate non-PIM page conflict instruction are selected for the group of candidate non-PIM instructions.

[0027] According to one embodiment, the selection criteria includes an instruction bandwidth allocation. The instruction bandwidth is allocated between PIM and non-PIM instructions, and if either of the allocations is consumed, first stage picker 180 switches to other types of instructions. For example, if the instruction bandwidth allocated to PIM instructions is consumed, then first stage picker 180 switches to processing non-PIM instructions. Similarly, if the instruction bandwidth allocated to non-PIM instructions is consumed, then first stage picker 180 switches to processing PIM instructions. The instruction bandwidth allocation is specified, for example, by system configuration information.

[0028] According to another embodiment, the selection criteria includes a threshold number of pending instructions. In this embodiment, the first stage picker 180 switches between processing PIM instructions and non-PIM instructions when the number of pending PIM instructions or non-PIM instructions, respectively, reaches a threshold. Depending on the particular embodiment, the same pending instruction threshold is used for both PIM and non-PIM instructions, or separate thresholds are used, which may be specified via configuration information of the program. For example, a programmer may specify a lower pending PIM instruction threshold for workloads that include large PIM code regions. According to another embodiment, the selection criteria includes a number of cycles or time since a particular type of instruction was issued. For example, the first stage picker 180 switches from processing PIM instructions to processing non-PIM instructions, or vice versa, when a specified number of cycles or amount of time has passed since the last non-PIM instruction was processed. This is useful in situations where quality of service requirements are met.

[0029] According to another embodiment, the selection criteria includes the presence of a particular marker. For example, a switch is made between processing PIM instructions and non-PIM instructions based on the presence of an instruction having a specified marker in instruction queue 170. The markers indicate, for example, the endpoints of a group of PIM or non-PIM instructions and may be specified by software, for example, added by a software developer, or may be inserted by memory controller 130 when a memory address dependency, for example, a data hazard such as Write-After-Read (WAR), Write-After-Write (WAW), Read-After-Write (RAW), etc., is detected between PIM and non-PIM instructions, even from different processor threads.

[0030] According to yet another embodiment, the selection criteria includes a global age, in which first stage picker 180 selects between processing PIM instructions and non-PIM instructions based on the oldest instruction type in instruction queue 170 based on instruction metadata 176. For example, if the oldest instruction in instruction queue 170 is a PIM instruction, then a PIM instruction is selected for processing, and vice versa.

[0031] According to one embodiment, the burst is used as a factor in selecting the next instruction to be processed. For example, if the current instruction is a PIM instruction, then the next PIM instruction in program order that is of the same type as the current PIM instruction, i.e., read or write, is selected in preference to a non-PIM instruction, even if a non-PIM instruction would normally be selected based on other selection criteria. This increases processing efficiency by generating larger read or write PIM instruction bursts and reducing the cost associated with switching between reads and writes, e.g., switching between read turnarounds and write turnarounds.

[0032] All of the above selection criteria may be used individually or in any combination to achieve a desired level of balance between the processing of PIM instructions and the processing of non-PIM instructions, depending on the particular embodiment.

[0033] If first stage picker 180 selects a PIM instruction for processing, then second stage picker 182 selects the oldest PIM instruction from instruction queue 170 based on instruction metadata 176 indicating when the instruction was enqueued in instruction queue 170. This maintains the program order of the PIM instructions and avoids architectural hazards in the PIM pipeline. If the selected PIM instruction is not ready, then scheduler 172 waits until the selected PIM instruction is ready.

[0034] If first stage picker 180 selects a group of candidate non-PIM instructions, then second stage picker 182 selects one of the candidate non-PIM instructions, i.e., a ready candidate non-PIM page hit instruction, a page miss instruction, or a page conflict instruction. According to one embodiment, second stage picker 182 selects the oldest of these three ready candidate non-PIM instructions based on instruction metadata 176, although this is not required to maintain accuracy of the non-PIM instructions.

[0035] According to one embodiment, memory controller 130 is configured to manage PIM request tokens to ensure proper processing of PIM instructions. This includes converting received PIM instructions that are normally processed as memory write instructions to memory read instructions so that write buffering techniques, e.g., write bursts, are not used by scheduler 172 when selecting PIM instructions from instruction queue 170. This allows scheduler 172 to immediately select PIM instructions, similar to non-PIM instructions. Before sending a PIM instruction to memory controller 130, the data fabric reserves a data buffer token to ensure that the corresponding metadata is copied to a dedicated PIM data buffer in memory controller 130. Since no data is returned from memory when issuing a PIM instruction, the data buffer token is only held until the PIM instruction is issued, after which the data buffer holding the metadata of the PIM instruction may be reused. The metadata of the PIM instruction is used by memory controller 130 or transferred to the PIM device while the PIM instruction is being issued. Alternatively, if sufficient notification is available, then the PIM instruction metadata may be provided to memory controller 130 via notification.

[0036] IV. An Exemplary Page Table with "All Banks" Status 2A illustrates a page table 200, an exemplary embodiment of the page table 178 of FIG. 1C. The memory controller 130 uses the page table 200 to track the status of the memory module banks and configure the banks appropriately for the next memory instruction. According to one embodiment, the page table 200 includes an entry for each bank that tracks the status of the corresponding bank after the most recent non-near memory processing instruction, i.e., a non-PIM instruction. The page table 200 also includes a single entry that tracks the status of all banks after the most recent PIM instruction, i.e., a PIM instruction.

[0037] In the example of FIG. 2A, page table 200 includes entries “PGT 0” through “PGT N” that indicate the status of each bank after the most recent non-PIM instruction. Each entry specifies a status, a bank ID, whether a row is open for the bank, and a row ID for the open row, if applicable. A status value of “Valid” indicates that the bank identified by the bank ID has a row currently open, and a status value of “Invalid” indicates that the bank identified by the bank ID does not have a row currently open. The bank ID is data that identifies the bank corresponding to the entry, and the open matrix indicates whether the bank has a row currently open (Y) or does not have a row currently open (N). The row ID is data that identifies the row currently open for the bank, and is “n / a” if the bank does not have a row currently open. As an example, the status data in a second entry (PGT 1) in page table 200 specifies that bank 1 has row 5 currently open. One or more of the entries "PGT 0" to "PGT N" are checked and / or updated every time a non-PIM command is issued.

[0038] The first entry in page table 200 is a new all-bank entry that specifies whether a bank currently has an open row as a result of processing the most recent PIM instruction, and if so, which row. For example, assume memory controller 130 issues a broadcast PIM instruction that specifies row 6, i.e., all banks perform a particular memory operation, which may include local computation, on row 6 and a particular column. Processing logic 174 updates all bank entries with a status value of “Valid”, an open row value of “Y”, and a row ID of 6. Processing logic 174 also invalidates other bank-specific entries PGT 0-PGT N in page table 178 for non-PIM instructions as necessary, i.e., if the preceding instruction issued was a non-PIM instruction, as will be described in more detail below. When memory controller 130 issues successive PIM instructions, processing logic 174 updates the row ID value with the row specified by the most recent PIM instruction. This continues until memory controller 130 issues a non-PIM instruction, after which processing logic 174 invalidates all bank entries (i.e., by changing the status to “Invalid”, changing the open row to “N”, and changing the row ID to “n / a”) and updates the entries for the particular target bank specified by the non-PIM instruction, as described in more detail below. Thus, a status of “Valid” in all bank entries indicates that the last instruction issued by memory controller 130 was a broadcast PIM instruction issued to all banks or a specified subset of banks, while a status of “Invalid” in all bank entries indicates that the last instruction issued by memory controller 130 was not a broadcast PIM instruction, but instead was a bank-specific non-near memory processing instruction.

[0039] (V. Non-PIM to non-PIM processing) 2B is a flow diagram illustrating using page table 200 to perform "Non-PIM to Non-PIM processing," i.e., processing a non-PIM instruction when the most recently issued instruction was also a non-PIM instruction. In step 202, a non-PIM instruction is selected for processing. For example, memory controller 130 selects a non-PIM instruction from instruction queue 170 based on selection criteria. In this example, assume that the non-PIM instruction is directed to bank 3, row 4.

[0040] At step 204, all bank statuses are checked. According to one embodiment, processing logic 174 checks the status of all bank entries in page table 200. In this example, because the previous instruction processed was a non-PIM instruction, all bank entries in page table 200 have a status of "Invalid" and processing logic 174 proceeds to check the page table entries for the selected instruction to determine the status. In this example, processing logic 174 examines the fourth entry for bank 3, "PGT 3," which indicates that bank 3 does not currently have a row open. Thus, there is a page miss in page table 200 and at step 206, processing logic opens a row in the bank required by the instruction. In this example, processing logic 174 issues a start execution instruction to open row 4 in bank 3. Control then passes to step 212 where the processing logic 174 updates the page table to record the change in status of bank 3, i.e., changing the status field from "Invalid" to "Valid", the open row field from "N" to "Y", and the row ID field from "n / a" to "4". The processing logic 212 then issues an instruction.

[0041] If, in step 204, row 4 of bank 3 is currently open, then this represents a page hit and control passes to step 212. However, if, in step 204, a different row of bank 3 is currently open, then this is a page conflict and in step 208, processing logic 174 closes the currently open row, for example, by issuing a precharge instruction to bank 3. In step 210, processing logic 174 then opens the row needed to process the selected instruction (row 3 in this example) by issuing a begin execute instruction to open row 4 in bank 3. Control then passes to step 212, as described above.

[0042] Steps 202-212 are repeated for any number of non-PIM instructions, with processing logic 174 opening and closing rows in banks as necessary and then updating page table 200. Figure 2C shows the page table after it has been updated by processing logic 174 after row 4 of bank 3 has been opened.

[0043] (VI. Transition from Non-PIM to PIM) Processing logic 174 eventually selects the PIM instruction to be processed and switches from non-PIM mode to PIM mode, also referred to herein as "Non-PIM to PIM Transition." According to one embodiment, once the decision to transition from non-PIM to PIM is made, scheduler 172 blocks all pending non-PIM instructions that have not yet been issued to memory module 140 and waits for the issued non-PIM instructions to be removed from instruction queue 170. FIG. 2C illustrates the state of page table 200 at this point after a non-PIM instruction directed to row 4 of bank 3 has been processed, with all bank entries being invalid. FIG. 2D is a flow diagram illustrating the transition from non-PIM to PIM, i.e., switching from processing non-PIM instructions to PIM instructions.

[0044] In step 220, a PIM instruction is selected for processing, for example, using the selection criteria described herein above. In this example, assume that processing logic 174 selects a PIM instruction that executes a memory instruction in row 5 of all banks.

[0045] In step 222, the all bank status is checked. According to one embodiment, the processing logic 174 checks the status of all bank entries in the page table 200. In this example, because the previous instruction processed was a non-PIM instruction, all bank entries in the page table 200 have a status of "Invalid," indicating a page conflict, and control proceeds to step 224, where all of the open rows in the bank are closed. For example, the processing logic 174 accomplishes this by issuing an all bank precharge command or a separate precharge command for each bank that has an open row. The decision of whether to use an all bank precharge command or a separate precharge command for each bank that has an open row may be made based on a comparison of the cost of issuing a separate precharge command to the banks that have the open rows compared to the cost of issuing an all bank precharge command. Exemplary costs include computational cost, power consumption, etc. For example, in a situation where only a small number of banks have open rows and it is cheaper to issue a separate precharge command to those banks, then a separate precharge command is used instead of an all bank precharge command. However, if multiple banks have open rows and it is computationally less expensive to issue a single all-bank precharge command, then an all-bank precharge command is issued instead of individual precharge commands for each bank.

[0046] At step 226, the row specified by the PIM instruction is opened in all banks. For example, processing logic 174 issues a per bank start execution instruction to open the row, or an all bank start execution instruction. In this example, processing logic 174 opens row 5 in all banks. At step 228, the page table is updated and the PIM instruction is processed. For example, processing logic 174 updates all bank entries in page table 200. FIG. 2E illustrates page table 200 after all bank entries have been updated to specify that row 5 is open in all banks. Using all bank entries in this manner provides the technical advantage of not having to update all of the entries in page table 200 to specify that row 5 is open. Instead, only all bank entries need to be updated, which reduces computational resources and power consumption.

[0047] (VII. PIM to PIM Processing) As PIM instructions continue to be selected for processing, the processing logic 174 performs what is called “PIM to PIM Processing.” Figure 2G is a flow diagram illustrating an approach for performing PIM to PIM processing.

[0048] In step 230, another PIM instruction is selected for processing, for example, using the selection criteria described herein above. In this example, assume that processing logic 174 selects a PIM instruction that executes a memory instruction in row 3 of all banks.

[0049] In step 232, the full bank status is checked. According to one embodiment, processing logic 174 checks the status of all bank entries in page table 200. In this example, since the previous instruction processed was a PIM instruction, all bank entries in page table 200 currently have a status of "Valid." A status of Valid in all bank entries means that a row is currently open in all banks, and represents either a page hit or a page conflict, depending on whether the row specified by the PIM instruction is the same or different from the row currently open in the bank, as indicated by all bank entries in page table 200. In this example, this represents a page conflict, since the PIM instruction specified a memory operation to be performed on row 3 of the bank, and row 5 is currently open. If the current PIM instruction specified row 5, this represents a page hit. A status of Valid represents a page hit if the row specified by the PIM instruction is the same as the row currently open in all banks, as indicated by all bank entries in page table 200.

[0050] In this example, the check of all bank entries results in a page conflict, so in step 234, the open row in all banks is closed. For example, processing logic 174 accomplishes this by issuing an all bank precharge command. In step 236, the row specified by the PIM command is opened in all banks. For example, processing logic 174 issues an all bank begin execute command or a begin execute command for each bank to open the row. In this example, processing logic 174 opens row 3 in all banks. If the check of all bank status in step 232 results in a page hit, i.e., the row specified by the PIM command is the same as the currently open row, steps 234 and 236 are not performed.

[0051] In step 238, the page table is updated and the PIM instruction is processed. For example, processing logic 174 updates all bank entries in page table 200 to change the open rows from 5 to 3. FIG. 2F shows page table 200 after all bank entries have been updated to change the open rows from 5 to 3. As discussed above, using all bank entries in this manner provides a technical advantage in that it is not necessary to update all of the entries in page table 200 to specify that row 3 is open. Instead, only all bank entries need to be updated, which reduces computational resources and power consumption.

[0052] (VIII. Transition from PIM to non-PIM) After processing the PIM instructions, which may be any number and type of PIM instructions, processing logic 174 selects a non-PIM instruction to be processed and switches from PIM mode back to non-PIM mode, also referred to herein as a "PIM to non-PIM Transition." Once this decision is made, scheduler 172 completes processing of the current PIM instruction and stops selecting further PIM instructions. Scheduler 172 also unblocks all non-PIM traffic. Figure 2H shows the state of page table 200 after PIM instructions directed to row 3 in all banks have been processed. Thus, row 3 is open in all banks.

[0053] 2I is a flow diagram illustrating a PIM to non-PIM transition, i.e., switching from processing a PIM instruction to a non-PIM instruction. In step 240, a non-PIM instruction is selected for processing, for example, using the selection criteria described herein above. In this example, assume that processing logic 174 selects a non-PIM instruction that performs a memory operation in bank 0, row 8.

[0054] In step 242, the all bank status is checked. According to one embodiment, the processing logic 174 checks the status of all bank entries in the page table 200. In this example, since the previous instruction processed was a PIM instruction, all bank entries in the page table 200 have a status of "Valid," meaning that all of the banks have open rows, so in step 244, the open rows in all of the banks are closed. For example, the processing logic 174 accomplishes this by issuing an all bank precharge command.

[0055] In step 246, the row specified by the non-PIM instruction is opened in the specified bank. For example, processing logic 174 issues a begin execution instruction to bank 0 to open row 8. In step 248, the page table is updated and the non-PIM instruction is processed. In this example, processing logic 174 invalidates all bank entries and updates entry "PGT 0" in page table 200 to specify that row 8 is open to bank 0. Figure 2J shows page table 200 after all bank entries have been invalidated and the "PGT 0" entry has been updated to indicate that row 8 is open in bank 0.

[0056] After the process of Figure 2I, any number of non-PIM instructions are processed using the non-PIM to non-PIM processing described herein above. The above approach for the memory controller to process both non-PIM and PIM instructions using a single entry in the page table for PIM instructions is efficient and avoids having to maintain a separate entry for each bank in the page table for PIM instructions.

[0057] 3 is a flow diagram 300 illustrating an approach for processing PIM and non-PIM instructions in a memory controller. The process begins with memory controller 130 selecting the next instruction to be processed, as described herein above. In step 302, it is determined whether the selected instruction is a PIM instruction. If not, then the selected instruction is a non-PIM instruction and in step 304, the status of all bank entries in the page table is checked. For example, processing logic 174 checks all bank entries in page table 200. In step 306, it is determined whether all bank entries in the page table are valid. For example, processing logic 174 determines whether the status of all bank entries in page table 200 is "Valid" or "Invalid."

[0058] If, in step 306, it is determined that the status of all bank entries is invalid, then the most recent instruction processed is a non-PIM instruction and, in step 308, the page table entry for the target bank of the instruction is checked and the page state is returned. For example, assume that a non-PIM instruction is directed to bank 3, row 4. The processing logic 174 examines the fourth entry for bank 3 in page table 200, "PGT 3". If the status is "Invalid", bank 3 does not currently have an open row and the processing logic returns a page state of "Miss". If the status is "Valid" and bank 3 currently has row 4 open, the processing logic 174 returns a page state of "Hit". If the status is "Valid" and bank 3 has other open rows, the processing logic 174 returns a page state of "Conflict". Control then proceeds to step 310 and the process is complete.

[0059] Returning to step 306, if it is determined that the status of all bank entries is valid, then the most recent instruction processed is a PIM instruction and all of the banks have the same open row. In step 312, all bank entries in the page table are invalidated and the entry for the target bank of the instruction is checked and a page status of "Conflict" is returned. In step 314, all rows in all banks are closed. For example, the processing logic 174 issues an all bank precharge command to close all rows in all banks and control passes to step 310 and the process is complete. In step 316, the rows required to process the instruction are opened in the target bank. Because the current instruction is a non-PIM instruction, the row specified by the instruction is opened in the target bank. For example, the processing logic 174 issues a start execute command for the bank and row specified by the non-PIM instruction selected for processing.

[0060] Returning to step 302, if it is determined that the selected instruction is a PIM instruction, then in step 318, the status of all bank entries in the page table is checked. For example, processing logic 174 checks all bank entries in page table 200. In step 320, it is determined whether all bank entries in the page table are valid. For example, processing logic 174 determines whether the status of all bank entries in page table 200 is "Valid" or "Invalid". If the status of all bank entries is "Invalid", then the most recently processed instruction is a non-PIM instruction, and in step 322, all bank entries in the page table are verified. For example, processing logic 174 changes the status of all bank entries in the page table to "Valid". Control then proceeds again to step 314, where all rows in all banks are closed to close rows opened by the previous non-PIM instruction. In step 316, rows required to process the instruction are opened in the target bank. Because the current instruction is a non-PIM instruction, the row specified in the PIM instruction is opened in all banks, such as by processing logic 174 issuing a start execution instruction in all banks for the row specified in the PIM instruction.

[0061] Returning to step 320, if it is determined that all bank entries in the page table are valid, then all banks have an open row because the most recently issued instruction was also a PIM instruction. In step 324, the page status of all bank entries is determined and returned. For example, the processing logic 174 checks the currently open row specified by all bank entries in the page table 200, and if the open row is the same as the row specified by the current (PIM) instruction, then the page status is a page "Hit" and the process is completed in step 310 because the row required to process the current (PIM) instruction is already open in all banks. If the currently open row is different from the row specified by the current (PIM) instruction, then the page status is a page "Conflict" and in step 326, all rows are closed and the row specified by the current (PIM) instruction is opened in all banks. The process then is completed in step 310.

Claims

1. 1. A memory controller, comprising: processing logic configured to select for processing a broadcast near memory processing instruction or a non-near memory processing instruction based on a selection criterion; Memory controller.

2. The selection criteria include one or more of instruction allocation bandwidth, number of pending instructions, time or number of cycles since the last instruction was issued, presence of a marker, or global age. The memory controller of claim 1 .

3. The processing logic further comprises: configured to maintain in a page table a status of two or more memory elements after a most recent broadcast near-memory operation instruction; The memory controller of claim 1 .

4. the status of two or more memory elements following the most recent broadcast near memory operation is maintained in a single entry of the page table. The memory controller of claim 3.

5. the status indicating currently open rows in the two or more memory devices; The memory controller of claim 3.

6. The processing logic, in response to the processed preceding memory instruction being a non-near memory processing instruction and selecting for processing the broadcast near memory processing instruction, closing all open rows in two or more memory devices in the memory module; opening rows specified by the broadcast near-memory operation instruction in two or more memory devices in the memory module; updating a single status for the two or more memory elements in a page table to indicate that a row specified by the broadcast near memory operation instruction is open in the two or more memory elements; [0023] The memory controller of claim 1 .

7. The processing logic, in response to the processed preceding memory instruction being a broadcast near memory processing instruction and selecting for processing the non-near memory processing instruction, closing all open rows in two or more memory devices in the memory module; opening a particular row in a particular memory device of two or more memory devices in the memory module specified by the non-near memory operation instruction; updating a status for the particular memory device in a page table to indicate that the particular row is open in the particular memory device; [0023] The memory controller of claim 1 .

8. the broadcast near memory processing instruction selected for processing is the oldest broadcast near memory processing instruction among a plurality of broadcast near memory processing instructions; The memory controller of claim 1 .

9. 1. A method comprising: the memory controller selecting for processing the broadcast near memory operation instruction or the non-near memory operation instruction based on a selection criterion; method.

10. The selection criteria include one or more of instruction allocation bandwidth, number of pending instructions, time or number of cycles since the last instruction was issued, presence of a marker, or global age.

10. The method of claim 9.

11. maintaining in a page table a status of two or more memory elements after a most recent broadcast near memory operation instruction; 10. The method of claim 9.

12. the status of two or more memory elements following the most recent broadcast near memory operation is maintained in a single entry of the page table.

12. The method of claim 11.

13. the status indicating currently open rows in the two or more memory devices; 12. The method of claim 11.

14. responsive to the processed preceding memory instruction being a non-near memory processing instruction and selecting for processing the broadcast near memory processing instruction, closing all open rows in two or more memory devices in the memory module; opening rows specified by the broadcast near-memory operation instruction in two or more memory devices in the memory module; updating a single status for the two or more memory elements in a page table to indicate that the row specified by the broadcast near memory operation instruction is open in the two or more memory elements.

10. The method of claim 9.

15. responsive to the processed preceding memory instruction being a broadcast near memory processing instruction and selecting for processing the non-near memory processing instruction, closing all open rows in two or more memory devices in the memory module; opening a particular row in a particular memory device of two or more memory devices in the memory module specified by the non-near memory operation instruction; updating a status for the particular memory device in a page table to indicate that the particular row is open in the particular memory device.

10. The method of claim 9.

16. the broadcast near memory processing instruction selected for processing is the oldest broadcast near memory processing instruction among a plurality of broadcast near memory processing instructions; 10. The method of claim 9.

17. 1. A processor comprising: An instruction queue; A memory controller, The memory controller includes: storing broadcast near memory operation instructions and non-near memory operation instructions in the instruction queue; selecting for processing the broadcast near memory processing instruction or the non-near memory processing instruction based on a selection criterion; [0023] Processor.

18. The memory controller is configured to maintain a status of two or more memory elements following a most recent broadcast near-memory operation in a single entry of a page table; the status indicating currently open rows in the two or more memory devices; 20. The processor of claim 17.

19. the memory controller, in response to the processed preceding memory instruction being a non-near memory processing instruction and selecting for processing the broadcast near memory processing instruction, closing all open rows in two or more memory devices in the memory module; opening rows specified by the broadcast near-memory operation instruction in two or more memory devices in the memory module; updating a single status for the two or more memory elements in a page table to indicate that a row specified by the broadcast near memory operation instruction is open in the two or more memory elements; [0023] 20. The processor of claim 17.

20. the memory controller, in response to the processed preceding memory instruction being a broadcast near memory processing instruction and selecting for processing the non-near memory processing instruction, closing all open rows in two or more memory devices in the memory module; opening a particular row in a particular memory device of two or more memory devices in the memory module specified by the non-near memory operation instruction; updating a status for the particular memory device in a page table to indicate that the particular row is open in the particular memory device; [0023] 20. The processor of claim 17.