Reducing latency of memory operations in a memory controller

Abstract

This application relates to reducing latency of memory operations in a memory controller. Methods, systems, memory controllers, apparatuses, and machine-readable media are disclosed in some instances that minimize this stall time by returning a memory write acknowledgement after a memory controller input multiplexer selects a write command, rather than when the memory write command has been executed. Because the memory controller performs ordering of the memory immediately after a packet is selected at the input multiplexer, the ordering of previous and subsequent requests to the same address location is preserved, and the response is provided early enough to enable the processor to continue its operations sooner without any adverse impact.

Description

[0001] Statement regarding government support

[0002] This invention was made with the support of the U.S. government, under DARPA License No. HR00111890003. The U.S. government holds certain rights to this invention. Technical Field

[0003] In some instances, this disclosure relates to chiplet technology. In some instances, this disclosure relates to memory controller chiplets. In some instances, this disclosure relates to reducing the latency of memory operations of chiplet memory controllers. Background Technology

[0004] Chiplets are an emerging technology that integrates various processing functions. Typically, a chiplet system consists of discrete modules (each module being a "chiplet") integrated on an interposer and interconnected on demand via one or more established networks in many instances to provide a system with the desired functionality. The interposer and the included chiplets can be packaged together for interconnection with other components of a larger system. Each chiplet may contain one or more individual integrated circuits or "chips" (ICs), and possibly discrete circuit components, and is typically coupled to a corresponding substrate for attachment to the interposer. Most or all of the chiplets in the system will be individually configured to communicate via the one or more established networks.

[0005] Chiplets, as individual modules of a system, differ from systems implemented on a single chip containing different device blocks (e.g., intellectual property (IP) blocks) on a substrate (e.g., a single die), such as a system-on-a-chip (SoC) or multiple discrete packaged devices integrated on a printed circuit board (PCB). Generally, chiplets offer better performance (e.g., lower power consumption, reduced latency, etc.) than discrete packaged devices, and chiplets offer greater manufacturing benefits than a single die chip. These manufacturing benefits may include higher yields or reduced development costs and time.

[0006] A chiplet system may comprise, for example, one or more application (or processor) chiplets and one or more support chiplets. Here, the distinction between application and support chiplets is merely a reference to possible design schemes for a chiplet system. Thus, for example, a synthetic vision chiplet system may comprise (by way of example only) application chiplets for generating synthetic vision output and support chiplets, such as memory controller chiplets, sensor interface chiplets, or communication chiplets. In typical use cases, synthetic vision designers may design the application chiplets and obtain the support chiplets from other sources. Therefore, design expenditures (e.g., in terms of time or complexity) are reduced by eliminating the need to design and manufacture the functionality embodied in the support chiplets. Chiplets also support tight integration of IP blocks, which may be difficult in other cases, such as when IP blocks are manufactured using different processing technologies or with different feature sizes (or different contact technologies or pitches). Therefore, multiple ICs or IC assemblies with different physical, electrical, or communication characteristics can be assembled in a modular manner to provide an assembly with the required functionality. Chiplet systems also facilitate adaptation to the needs of different larger systems into which the chiplet system will be incorporated. In practice, ICs or other assemblies can be optimized for power, speed, or heat generation for specific functions—as might happen with sensors—and can be integrated with other devices more easily than if they were to be integrated onto a single die. Furthermore, by reducing the overall size of the die, small chiplets often have higher yields than more complex single-die devices. Summary of the Invention

[0007] In one aspect, this application provides an apparatus comprising: a memory array; an input queue coupled to the memory array and configured to receive a memory request from a request processor; and logic coupled to the input queue and the memory array and configured to: select a memory request from the input queue, the memory request being received from the request processor; determine that the memory request is a write request for writing a requested value to a requested location in the memory of the memory array; in response to determining that the memory request is a write request: send a response to the request processor, the response being sent before executing the write request; and after sending the response to the request processor, execute the write request for writing the requested value to the requested location in the memory.

[0008] In another aspect, this application provides a method comprising: selecting a memory request from an input queue, the memory request being received from a request processor; determining that the memory request is a write request to write a requested value to a requested location in memory; in response to determining that the memory request is a write request: sending a response to the request processor, the response being sent before executing the write request; and after sending the response to the request processor, executing the write request to write the requested value to the requested location in memory.

[0009] In another aspect, this application provides a non-transitory machine-readable medium for storing instructions that, when executed by a machine, cause the machine to perform operations, including: selecting a memory request from an input queue, the memory request being received from a request processor; determining that the memory request is a write request to write a requested value to a requested location in memory; in response to determining that the memory request is a write request: sending a response to the request processor, the response being sent before executing the write request; and after sending the response to the request processor, executing the write request to write the requested value to the requested location in memory. Attached Figure Description

[0010] This disclosure will be more fully understood from the detailed description given below and the accompanying drawings of various embodiments thereof. However, the drawings should not be construed as limiting this disclosure to the specific embodiments, but are merely for illustration and understanding.

[0011] Figure 1A and 1B Examples of chiplet systems according to some of the embodiments of this disclosure are shown.

[0012] Figure 2 The components of an example memory controller chiplet according to some examples of this disclosure are shown.

[0013] Figure 3-5 A schematic diagram of a memory controller according to some examples of this disclosure is shown.

[0014] Figure 6 A flowchart illustrating a method for processing memory write requests according to some examples of this disclosure is shown.

[0015] Figure 7 This is a block diagram illustrating examples of machines in which or which can be used to operate embodiments of the present disclosure, according to some examples of the present disclosure. Detailed Implementation

[0016] Figure 1, described below, provides an example of a chiplet system and the components operating therein. As described below, such a chiplet system may include a memory controller chiplet that controls one or more off-die memory devices, which may reside on other chiplets in some instances. In a chiplet system, minimizing processor downtime is crucial for maximizing performance, for example, when the processor is waiting for a response from the memory controller. For instance, a processor requesting a memory write may not perform any other operations of the specific procedure requesting the memory write until the memory write is acknowledged. Once acknowledgement is received from the memory controller, the procedure can continue with subsequent instructions.

[0017] Methods, systems, memory controllers, apparatuses, and machine-readable media are disclosed in some instances that minimize this downtime by returning a memory write acknowledgment after a write command has been selected for processing by the memory controller input multiplexer, rather than after the memory write command has already been executed. Because the memory controller performs memory sorting immediately after a packet is selected at the input multiplexer, the ordering of previous and subsequent requests is preserved, and providing an early response allows the processor to continue its operation earlier without any adverse effects.

[0018] Figure 1A and 1B An example of a chiplet system 110 according to an embodiment is shown. Figure 1A This is an illustration of a chiplet system 110 mounted on a peripheral board 105, which can be connected to a wider range of computer systems via, for example, peripheral component interconnect (PCIe). The chiplet system 110 includes a package substrate 115, an insert 120, and four chips: an application chiplet 125, a host interface chiplet 135, a memory controller chiplet 140, and a memory device chiplet 150. Other systems may include numerous additional chipsets to provide additional functionality, as will become clear from the following discussion. The package of the chiplet system 110 is shown with a cover or shroud 165, but other packaging technologies and structures can be used for the chiplet system. Figure 1B This is a block diagram for the purpose of clearly labeling the components in a chiplet system.

[0019] Application chip 125 is shown as including a network on chip (NOC) 130 to support a chiplet network 155 for inter-chiplet communication. In an example embodiment, NOC 130 may be included on application chip 125. In an example, NOC 130 may be defined in response to selected supporting chips (e.g., chips 135, 140, and 150), thereby allowing the designer to select an appropriate number of chiplet network connections or switches for NOC 130. In an example, NOC 130 may reside on a separate chiplet or even within insert 120. In the example discussed herein, NOC 130 implements a chiplet protocol interface (CPI) network.

[0020] CPI is a packet-based network that supports virtual channels to enable flexible, high-speed interaction between chiplets. CPI supports bridging within-chiplet networks to chiplet networks 155. For example, the Advanced Extensible Interface (AXI) is a widely used specification for designing intra-chip communication. However, the AXI specification covers a large number of physical design options, such as the number of physical channels, signal timing, and power. In a single chip, these options are typically selected to meet design goals such as power consumption and speed. However, to achieve flexibility in chiplet systems, adapters such as CPI are used to connect various AXI design options that can be implemented in various chiplets. By enabling physical-to-virtual channel mapping and encapsulating time-based signaling using packet-based protocols, CPI bridges within-chiplet networks 155.

[0021] CPI can use various physical layers to transmit packets. A physical layer may contain simple conductive connections or drivers for increasing voltage or otherwise facilitating signal transmission over longer distances. An example of such a physical layer may include an Advanced Interface Bus (AIB), which can be implemented in insert 120 in various instances. The AIB uses source-synchronous data transfer with a forwarding clock to transmit and receive data. Packets are transmitted across the AIB at Single Data Rate (SDR) or Double Data Rate (DDR) relative to the transmission clock. The AIB supports various channel widths. In SDR mode, the AIB channel width is a multiple of 20 bits (20, 40, 60, ...), and in DDR mode, it is a multiple of 40 bits (40, 80, 120, ...). The AIB channel width includes both transmitted and received signals. Channels can be configured with a symmetrical number of transmit (TX) and receive (RX) inputs / outputs (I / O), or an asymmetrical number of transmitters and receivers (e.g., all transmitters or all receivers). The channel can act as either the AIB master clock or a secondary clock, depending on the chiplet providing the master clock. The AIB I / O unit supports three timing modes: asynchronous (i.e., non-timing), SDR, and DDR. In various instances, the non-timing mode is used for the clock and some control signals. SDR mode can use a dedicated SDR-only I / O unit or a dual SDR / DDR I / O unit.

[0022] In this example, the CPI packet protocol (e.g., point-to-point or routable) can use symmetrical receive and transmit I / O units within an AIB channel. The CPI streaming protocol allows for more flexible use of AIB I / O units. In this example, the AIB channel for streaming mode can be configured with I / O units as all TX, all RX, or half RX and half RX. The CPI packet protocol can use the AIB channel in SDR or DDR operating modes. In this example, the AIB channel is configured in increments of 80 I / O units (i.e., 40 TX and 40 RX) for SDR mode and in increments of 40 I / O units for DDR mode. The CPI streaming protocol can use the AIB channel in SDR or DDR operating modes. Here, in this example, the AIB channel increments by 40 I / O units for both SDR and DDR modes. In this example, each AIB channel is assigned a unique interface identifier. This identifier is used during CPI reset and initialization to determine paired AIB channels between neighboring chiplets. In this example, the interface identifier is a 20-bit value, comprising a 7-bit chiplet identifier, a 7-bit column identifier, and a 6-bit link identifier. The AIB physical layer uses an AIB out-of-band shift register to transmit the interface identifier. Bits 32-51 of the shift register are used to transmit the 20-bit interface identifier in both directions across the AIB interface.

[0023] AIB defines a stacked group of AIB channels as an AIB channel column. An AIB channel column has a certain number of AIB channels, plus an auxiliary channel. The auxiliary channel contains signals used for AIB initialization. All AIB channels in a column (except the auxiliary channel) have the same configuration (e.g., all TX, all RX, or half TX and half RX, and the same number of data I / O signals). In this example, AIB channels are numbered in consecutive increments, starting with the AIB channel adjacent to the AUX channel. The AIB channel adjacent to the AUX is defined as AIB channel zero.

[0024] Typically, the CPI interface on each chiplet may include serialization-deserialization (SERDES) hardware. SERDES interconnects are suitable for high-speed signaling applications requiring low signal counts. However, for multiplexing and demultiplexing, error detection or correction (e.g., using block-level cyclic redundancy check (CRC)), link-level retries, or forward error correction, SERDE can lead to additional power consumption and longer latency. However, when low latency or power consumption is a primary concern for ultra-short distances, chiplet interconnects with parallel interfaces having clock rates that allow data transmission with minimal latency can be utilized. CPIs contain elements that minimize latency and power consumption in these ultra-short distance chiplet interconnects.

[0025] For flow control, CPI employs a credit-based technique. A receiver, such as application chip 125, provides credits representing available buffers to a sender, such as memory controller chip 140. In this example, the CPI receiver contains buffers for each virtual channel for a given transmission time unit. Therefore, if the CPI receiver supports five messages and a single virtual channel in time, the receiver has five buffers arranged in five rows (e.g., one row per unit time). If four virtual channels are supported, the receiver has twenty buffers arranged in five rows. Each buffer holds the payload of one CPI packet.

[0026] When a sender transmits data to a receiver, it decrements its available credits based on the transmission. Once the receiver has exhausted all its credits, the sender stops sending packets to the receiver. This ensures that the receiver always has an available buffer to store transmissions.

[0027] When the receiver processes the received packet and releases the buffer, it sends the available buffer space back to the sender. This credit is then made available to the sender, allowing the transmission of additional information.

[0028] Also shown is a chiplet mesh network 160 using direct, chiplet-to-chiplet technology without requiring NOC 130. The chiplet mesh network 160 can be implemented in CPI or another chiplet-to-chiplet protocol. The chiplet mesh network 160 typically enables chiplet pipelines, where one chiplet acts as a pipeline interface, while other chipslets in the pipeline only interface with themselves.

[0029] In addition, dedicated device interfaces, such as one or more industry-standard memory interfaces 145 (e.g., synchronous memory interfaces, such as DDR5, DDR6), can also be used to interconnect chiplets. This connects chiplet systems or individual chiplets to external devices (e.g., to larger systems via a desired interface, such as a PCIe interface). For example, in one instance, such an external interface can be implemented via a host interface chiplet 135, which, in the depicted example, provides a PCIe interface external to the chiplet system 110. Such dedicated interfaces 145 are typically used when industry practices or standards have converged on them. The illustrated example of connecting a memory controller chiplet 140 to a Double Data Rate (DDR) interface 145 of a dynamic random access memory (DRAM) memory device 150 is such an industry practice.

[0030] Among the various possible supporting chiplets, the memory controller chiplet 140 may be present in the chiplet system 110 due to the ubiquitous use of storage devices for computer processing and the cutting-edge technologies used for memory devices. Therefore, using the memory device chiplet 150 and the memory controller chiplet 140, both manufactured by others, allows chiplet system designers to obtain robust products from established manufacturers. Typically, the memory controller chiplet 140 provides a memory device-specific interface for reading, writing, or erasing data. Typically, the memory controller chiplet 140 can provide additional features such as error detection, error correction, maintenance operations, or atomic operation execution. For certain types of memory, maintenance operations tend to be specific to the memory device 150, such as garbage collection in NAND flash or storage class memory, or temperature regulation (e.g., cross-temperature management) in NAND flash memory. In instances, maintenance operations may involve logic-to-physical (L2P) mapping or management to provide an indirection level between the physical and logical representations of data. In other types of memory (such as DRAM), some memory operations, such as refresh, may be controlled by the host processor or memory controller at some times, and by the DRAM memory device at other times, or by logic associated with one or more DRAM devices (e.g., interface chips (in this example, buffers)).

[0031] An atomic transaction is one or more data manipulation operations, for example, that can be performed by the memory controller chiplet 140. In other chiplet systems, atomic transactions can be performed by other chipsets. For example, an application chiplet 125 can specify an "increment" atomic transaction in a command, which includes a memory address and a possible increment value. Upon receiving the command, the memory controller chiplet 140 retrieves a number from the specified memory address, increments the number by the amount specified in the command, and stores the result. Upon successful completion, the memory controller chiplet 140 provides the application chiplet 125 with an indication that the command was successful. Atomic transactions avoid transferring data across the chiplet mesh network 160, thereby reducing the execution latency of such commands.

[0032] Atomic transactions can be categorized into built-in atoms or programmable (e.g., custom) atomic transactions. Built-in atomic transactions are a finite set of operations that are immutably implemented in hardware. Programmable atomic transactions are small programs with one or more instructions (e.g., instruction sets) that can be executed on programmable atomic units (PAUs) (e.g., custom atomic units (CAUs)) of the memory controller chiplet 140. Figure 1 illustrates an example of a memory controller chiplet discussing PAUs.

[0033] The memory device chiplet 150 can be a volatile memory device or a non-volatile memory, or any combination thereof. Examples of volatile memory devices include, but are not limited to, random access memory (RAM)—such as DRAM, synchronous DRAM (SDRAM), graphics double data rate type 6 SDRAM (GDDR6 SDRAM), etc. Examples of non-volatile memory devices include, but are not limited to, NAND flash memory, memory-class memory (e.g., phase-change memory or memristor-based technology), ferroelectric RAM (FeRAM), etc. The illustrated example includes memory device 150 as a chiplet; however, memory device 150 can reside elsewhere, such as in different packages on peripheral board 105. For many applications, multiple memory device chipslets can be provided. In the example, these memory device chipslets can each implement one or more memory technologies. In the example, the memory chiplet can include multiple stacked memory dies of different technologies, such as stacked or otherwise communicating with one or more DRAM devices. The memory controller 140 can also be used to coordinate the operation between multiple memory chips in the chiplet system 110; for example, using one or more memory chips in one or more levels of cache storage devices, and using one or more additional memory chips as main memory. The chiplet system 110 may also include multiple memory controllers 140, which can be used to provide memory control functions for individual processors, sensors, networks, etc. The chiplet architecture of systems such as 110 has the advantage of being able to adapt to different memory storage technologies and different memory interfaces through updated chiplet configurations without requiring redesign of the rest of the system architecture.

[0034] Figure 2Components of an example memory controller chiplet 205 according to an embodiment are shown. The memory controller chiplet 205 includes a cache 210, a cache controller 215, an off-die memory controller 220 (e.g., communicating with off-die memory 275), a network communication interface 225 (e.g., interfacing with chiplet network 285 and communicating with other chiplets), and a set of atom and merging units 250. Members of this set may include, for example, a write merging unit 255, a memory dangerous unit 260, a built-in atom unit 265 (for performing built-in atom transactions), or a programmable atom unit (PAU) 270 (for performing programmable atom transactions). The various components are shown logically, and they are not necessarily implemented. For example, the built-in atom unit 265 may include different means along the path to off-die memory. For example, the built-in atom unit 265 may be in an interface means / buffer on the memory chiplet, as discussed above. In contrast, the programmable atom unit 270 can be implemented in a separate processor on the memory controller chiplet 205 (but in various instances, it can be implemented in other locations, such as on the memory chiplet).

[0035] The off-die memory controller 220 is directly coupled to the off-die memory 275 (e.g., via a bus or other communication connection) to provide write operations to, and read operations from, the one or more off-die memories, such as off-die memory 275 and off-die memory 280. In the depicted example, the off-die memory controller 220 is also coupled to outputs to the atom and merge unit 250 and inputs to the cache controller 215 (e.g., a memory-side cache controller).

[0036] In the instance configuration, the cache controller 215 is directly coupled to the cache 210 and can be coupled to the network communication interface 225 for input (e.g., incoming read or write requests) and coupled to the off-die memory controller 220.

[0037] Network communication interface 225 includes packet decoder 230, network input queue 235, packet encoder 240, and network output queue 245 to support packet-based chiplet network 285, such as CPI. Chiplet network 285 can provide packet routing between processors, memory controllers, mixed-thread processors, configurable processing circuitry, or communication interfaces. In such packet-based communication systems, each packet typically contains destination and source addressing, as well as any data payload or instructions. In examples, chiplet network 285 can be implemented, depending on the configuration, as a series of crossbars with a folded Clos configuration, or as a mesh network with additional connectivity.

[0038] In various instances, the chiplet network 285 may be part of an asynchronous switching architecture. Here, data packets can be routed along any of the various paths, such that the arrival of any selected data packet at its addressed destination can occur at any of a plurality of different times depending on the route. Alternatively, the chiplet network 285 may be implemented at least partially as a synchronous communication network, such as a synchronous mesh communication network. Both of these configurations of the communication network are intended for use in embodiments according to this disclosure.

[0039] The memory controller chip 205 can receive packets having, for example, a source address, a read request, and a physical address. In response, the off-die memory controller 220 or the cache controller 215 reads data from the specified physical address (which may be in off-die memory 275 or cache 210) and assembles a response packet to the source address containing the requested data. Similarly, the memory controller chip 205 can receive packets having a source address, a write request, and a physical address. In response, the memory controller chip 205 writes data to the specified physical address (which may be in cache 210 or off-die memory 275 or 280) and assembles a response packet to the source address containing confirmation that the data has been stored in memory.

[0040] Therefore, the memory controller chiplet 205 can receive read and write requests via the chiplet network 285 and (if possible) process the requests using the cache controller 215, which interfaces with the cache 210. If the cache controller 215 cannot process the request, the off-die memory controller 220 processes the request by communicating with the off-die memory 275 or 280, the atom and merge unit 250, or both. As mentioned above, one or more levels of caches can also be implemented in the off-die memory 275 or 280; and in some such instances, these caches can be directly accessed by the cache controller 215. Data read by the off-die memory controller 220 can be cached in the cache 210 by the cache controller 215 for later use.

[0041] Atom and merge unit 250 is coupled to receive (as input) the output of off-die memory controller 220 and provides the output to cache 210, network communication interface 225, or directly to chiplet network 285. Memory hazard unit 260, write merge unit 255, and built-in (e.g., predetermined) atom unit 265 may each be implemented as a state machine with other combinational logic circuitry (e.g., adders, shifters, comparators, AND gates, OR gates, XOR gates, or any suitable combination thereof) or other logic circuitry. These components may also include one or more registers or buffers for storing operands or other data. PAU 270 may be implemented as one or more processor cores or control circuitry and various state machines with other combinational logic circuitry or other logic circuitry, and may also include one or more registers, buffers, or memories for storing addresses, executable instructions, operands, and other data, or may be implemented as a processor.

[0042] Write merging unit 255 receives read data and request data, and merges the request data and read data to create a single unit with read data and source address for use in a response or return packet. Write merging unit 255 provides the merged data to the write port of cache 210 (or equivalently, to cache controller 215 for writing to cache 210). Optionally, write merging unit 255 provides the merged data to network communication interface 225 to encode the response or return packet and prepare it for transmission on chiplet network 285.

[0043] When the requested data is for a built-in atomic operation, the built-in atomic unit 265 receives the request and reads the data from the write merging unit 255 or directly from the off-chip memory controller 220. An atomic transaction is performed, and the resulting data is written to cache 210 or provided to network communication interface 225 using write merging unit 255 to encode the response or return data packet and prepare it for transmission on chiplet network 285.

[0044] Built-in atomic units 265 handle predefined atomic transactions, such as fetch and increment or compare and swap. In instances, these transactions perform simple read-modify-write operations on a single memory location of 32 bytes or less. Atomic memory transactions are initiated from request packets transmitted via chiplet network 285. The request packet has a physical address, atomic operator type, operand size, and optionally up to 32 bytes of data. Atomic transactions perform read-modify-write operations on cache lines of cache 210, thereby filling the cache memory as needed. Atomic transaction responses can be simple complete responses or responses with up to 32 bytes of data. Instance atomic memory transactions include fetch and AND, fetch and OR, fetch and XOR, fetch and add, fetch and subtract, fetch and increment, fetch and decrement, fetch and minimum, fetch and maximum, fetch and swap, and compare and swap. In various instance embodiments, 32-bit and 64-bit operations, as well as operations on 16 or 32 bytes of data, are supported. The methods disclosed herein are also compatible with hardware that supports larger or smaller operations and more or less data.

[0045] Built-in atomic transactions can also involve requests for "standard" atomic criteria regarding the requested data, such as relatively simple single-cycle integer atoms—e.g., fetch and increment or compare and swap—with throughput similar to regular memory read or write operations that do not involve atomic operations. For these operations, cache controller 215 can typically preserve cache lines in cache 210 by setting a danger bit (in hardware) so that the cache line cannot be read by another process during translation. Data is obtained from off-chip memory 275 or cache 210 and provided to built-in atomic unit 265 to perform the requested atomic transaction. After the atomic transaction, in addition to providing the obtained data to packet encoder 240 for encoding outgoing packets for transmission over chiplet network 285, built-in atomic unit 265 also provides the obtained data to write merging unit 255, which writes the obtained data back to cache 210. After the obtained data is written to cache 210, memory danger unit 260 clears any corresponding danger bits that were set.

[0046] The PAU 270 achieves high performance (high throughput and low latency) for programmable atomic transactions (also known as "custom atomic transactions" or "custom atomic operations"), comparable to built-in atomic transactions. In response to an atomic transaction request specifying a programmable atomic transaction and a memory address, instead of performing multiple memory accesses, the circuitry in the memory controller chiplet 205 transmits the atomic transaction request to the PAU 270 and sets a dangerous bit stored in a memory danger register corresponding to the memory address of the memory row used in the atomic operation, ensuring that no other operations (read, write, or atomic transaction) are performed on that memory row. The dangerous bit is then cleared upon completion of the atomic transaction. The additional direct data path provided by the PAU 270 for executing programmable atomic transactions allows for additional write operations without any limitations imposed by the communication network bandwidth or any increase in communication network congestion.

[0047] The PAU 270 includes a multi-threaded processor, such as a RISC-VIS-based multi-threaded processor with one or more processor cores, and further with an extended instruction set for performing programmable atomic transactions. When the extended instruction set for performing programmable atomic transactions is provided, the PAU 270 processor can be embodied as one or more hybrid-threaded processors. In some example embodiments, the PAU 270 processor provides bucket-loop instantaneous thread switching to maintain a high instruction / clock rate.

[0048] The PAU 270 may include local memory, such as static random access memory (SRAM), NAND, phase-change memory, etc. Local memory may include registers, instruction memory, and cache. Local memory can be accessed by the processor through the memory controller.

[0049] Programmable atomic transactions, which can be executed by PAU 270, involve requests for programmable atomic transactions of requested data. Users can prepare programming code in the form of one or more instructions to provide such programmable atomic transactions. For example, a programmable atomic transaction can be a fairly simple multi-loop operation, such as floating-point addition, or a relatively complex multi-instruction operation, such as Bloom filter insertion. Programmable atomic transactions can be the same as or different from predetermined atomic transactions, as long as they are defined by the user and not the system vendor. For these operations, cache controller 215 can retain cache lines in cache 210 by setting a danger bit (in hardware) so that the cache lines cannot be read by another process during translation. Data is obtained from cache 210 or off-chip memory 275 or 280 and provided to PAU 270 to execute the requested programmable atomic transaction. After the atomic operation, PAU 270 provides the obtained data to network communication interface 225 to directly encode outgoing data packets containing the obtained data for transmission over chiplet network 285. Furthermore, PAU 270 provides the obtained data to cache controller 215, which then writes the obtained data to cache 210. After writing the obtained data to cache 210, cache controller 215 clears any corresponding dangerous bits that were set.

[0050] In selected instances, the method for programmable atomic transactions involves providing multiple generic programmable atomic transaction request types that can be sent from origin such as a processor or other system component to memory controller chiplet 205 via chiplet network 285. Cache controller 215 or off-die memory controller 220 recognizes the request as a programmable atomic transaction and forwards it to PAU 270. In a representative embodiment, PAU 270: (1) is a programmable processing element capable of efficiently executing user-defined atomic transactions; (2) can perform load and store to memory, arithmetic and logical operations, and control flow decisions; and (3) utilizes a RISC-V ISA and a new set of dedicated instructions to facilitate interaction with such controllers 215, 220, thereby atomically executing user-defined transactions. In an ideal instance, the RISC-V ISA contains a complete set of instructions supporting high-level language operators and data types. When included in memory controller chiplet 205, PAU 270 can utilize the RISC-V ISA, but typically supports a more limited instruction set and a limited register file size to reduce the die size of the unit.

[0051] As mentioned above, before writing the read data to cache 210, the memory danger unit 260 clears the danger bit set for a reserved cache line. Therefore, when the write merging unit 255 receives a request and read data, the memory danger unit 260 can transmit a reset or clear signal to cache 210 to reset the memory danger bit set for a reserved cache line. Furthermore, resetting this danger bit also releases pending read or write requests involving a specified (or reserved) cache line, thereby providing the pending read or write request to the incoming request multiplexer for selection and processing.

[0052] Figure 3 A functional schematic diagram of a memory controller 300 associated with a memory (345) according to some examples of this disclosure is shown. Figure 3 This is another example of the memory controller chiplet 205, and many related features are shown. Figure 2 The same components are shown. For example, caches 320 and 385 are instances of cache 210; DRAM 345 is an instance of off-die memory 275-280; atomic / write merge 380 and programmable atomic unit 395 may be instances of atomic and merge unit 250; Figure 3 Other components can be Figure 2Examples of other components include the off-chip memory controller 220 and the cache controller 215. The NOC request queue 305 receives requests from the on-chip network and provides small queues. The atomic request queue 310 receives requests from programmable atomic cells and provides small queues. The inflow request multiplexer (IRM) 315 selects among the sources of inflow memory requests. These three sources, in order of priority, are: memory dangerous requests, atomic requests, and inflow NOC requests. Cache (read) 302 and cache (write) 385 are SRAM data caches. This diagram shows the cache as two separate blocks (302 and 385), one providing read access and the other providing write access. Delay block 325 provides one or more pipeline stages to simulate the latency of SRAM cache read operations. A cache miss requires memory access to place the required data into the cache. In some instances, as shown, the memory is DRAM, but in other instances, other types of memory may be used. During this access time, the memory row is unavailable for other requests. The memory danger blocks (set block 330 and clear block 390) maintain a danger bit table indicating which memory rows are inaccessible. Incoming requests attempting to access rows with danger are held by the memory danger blocks until the danger is cleared. Once the danger is cleared, the request is resent via the incoming request multiplexer. The memory row tag address is hashed as a danger bit index. The number of danger bits can be selected to set the danger collision probability to a sufficiently low level. The incoming DRAM control multiplexer (IDCM) 335 selects from incoming NOC requests and cache eviction requests. Group request queue 340—each individually managed DRAM group has a dedicated group request queue to hold requests until a request can be scheduled on the associated DRAM group.

[0053] Scheduler 342 selects across group queue 340 to select a request for an available DRAM group. DRAM 345 represents an external DRAM device. Request hit data queue 350 holds request data from cache hits until selected. Request miss data queue 355 holds data read from DRAM until selected. Miss request queue 360 ​​holds cache miss request packet information until a request is selected. Hit request queue 365 holds cache hit request packet information until selected. Data selection multiplexer (DSM) 370 selects between DRAM read data and cache hit read data. The selected data is written to the SRAM cache. Request selection multiplexer (RSM) 375 selects between hit request queue 360 ​​and miss request queue 365.

[0054] Atom / Write Merge 380 merges request data and DRAM read data, or, if the request is a built-in atom, memory data and request data are used as inputs to the atom operation. Cache (Write) Block 385 represents the write port of the SRAM cache. Data from the NOC write request and data from the DRAM read operation are written to the SRAM cache. Memory Danger (Clear) Block 390 represents a danger clearing operation for dangerous memory structures. Clearing dangers may release pending NOC requests and send them to the inflow request multiplexer. Programmable Atom Cell 395 handles programmable atom transactions. NOC Outflow Response Multiplexer (ORM) 397 selects between memory controller responses and custom atom cell responses and sends the selection to the NOC.

[0055] Figure 4 This shows how to process write commands. Figure 3 The memory controller. The process is the same regardless of whether the requested SRAM cache hit or miss, and is indicated by dashed lines. For example, write commands received from the NOC can be queued in the NOC request queue 305. The input request multiplexer 315 can select a write request from the NOC request queue 305. A memory hazard can be checked to determine if the memory address is busy. If busy, the write request will be delayed, waiting for the memory hazard to be cleared. Once the memory hazard is cleared, the write request is processed by the hit request queue 365, and once selected by the request selection multiplexer 375, it is written to the cache (write) 385. The cache (read) function can determine if the requested address is already in the cache (in which case the value will be updated). Figure 5 This illustrates the processing of write commands with cache evicting. Figure 3 The memory controller. Figure 5 In the write operation, for a write miss, the operation looks up the line in the cache to be used for the current write operation. If the line's contents are invalid (e.g., there are unwritten changes), the line is evicted (written to memory). The line is now translated to the address of the new write. At this point, data is written to the cache as if it were a write hit (because the line's label now matches the address of the write request). The dashed lines show the flow as evicted data items are sent to IDCM 335, Group 340, Scheduler 342, and DRAM 345.

[0056] As previously described, minimizing processor latency is crucial for maximizing performance. For example, when a processor issues a write command to the memory controller, it waits for the controller to respond. By sending a response after the memory controller input multiplexer (e.g., IRM 315) selects the write, the processor can continue processing other instructions, thus minimizing the amount of time the processor must wait for the memory write request to complete. From the time the request is selected, the memory controller sorts the requests line by line until the request is executed in the memory-side cache or DRAM. Once the memory controller selects the request, it returns a write complete response, ensuring that the order of previous and subsequent requests for the same address location is preserved. This applies even if memory hazards are not cleared, as these requests are queued in order.

[0057] Figure 6 A flowchart illustrating a method 600 for processing memory write requests according to some embodiments of the present disclosure is shown. At operation 605, the memory controller may receive a memory request. The request may be a read request, a write request, a built-in atomic transaction request, a custom atomic transaction request, etc. The request may be received from a request processor on the same chip as the memory controller, another chip in a chiplet system (e.g., via an inter-chip network, such as CPI), etc. At operation 607, the controller may place the request in a queue. For example, an on-chip network request queue 305, an atomic request queue 310 (if the request is for a built-in atomic transaction), etc.

[0058] At operation 610, a request can be pulled from one of the queues. For example, via input request multiplexer 315. If the request is not a write request, then... Figure 4 The flow proceeds to operation 620, which is then interpreted. If the request is a write request, then at operation 615, a response is sent to the processor that sent the request. For example, the response can be sent via an inter-chip network, an on-chip network, etc. The response is sent before the request is written to memory. For example, the response can be sent before or simultaneously with sending a memory request to DRAM345 or a memory write request to the cache. Once the response is sent, the memory request can be serviced at operation 620, such as setting a memory danger bit, writing to the cache, evicting a value from the cache, etc.

[0059] As mentioned earlier, once a write request is selected, the ordering of memory requests is guaranteed. That is, previously selected requests are completed first, and subsequently selected requests are completed after the currently selected request.

[0060] Figure 7A block diagram of an example machine 700 is shown, in which, or by means of any one or more techniques (e.g., methods) discussed herein can be implemented. As described herein, an example may contain logic or several components or mechanisms in, or be operable by, machine 700. A circuit system (e.g., a processing circuit system) is a series of circuits implemented in the tangible entity of machine 700, containing hardware (e.g., simple circuits, gates, logic, etc.). The components of the circuit system can be flexible over time. The circuit system contains components that can perform a specified operation individually or in combination during operation. In an example, the hardware of the circuit system may be immutably designed to perform a specific operation (e.g., hardwiring). In an example, the hardware of the circuit system may contain variablely connected physical components (e.g., execution units, transistors, simple circuits, etc.) and a machine-readable medium that encodes instructions for a specific operation with physical modifications (e.g., magnetically, electrically, movable placement of constant aggregate particles, etc.). When the physical components are connected, the basic electrical properties of the hardware composition change, for example, from an insulator to a conductor or vice versa. Instructions enable embedded hardware (e.g., an execution unit or loading mechanism) to generate components of a circuit system via variable connections, thereby performing specific operations when in operation. Thus, in one example, when the device is in operation, the machine-readable medium element is either part of the circuit system or another component communicatively coupled to the circuit system. In another example, any physical component may be used for more than one component of more than one circuit system. For example, during operation, an execution unit may be used at one point in time for a first circuit of a first circuit system and at another point in time for a second circuit of the first circuit system or a third circuit of the second circuit system. Additional examples of these components of machine 700 are as follows.

[0061] In alternative embodiments, machine 700 may be used as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, machine 700 may operate as a server machine, a client machine, or both in a server-client network environment. In an example, machine 700 may act as a peer-to-peer (P2P) (or other distributed) network environment. Machine 700 may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, network device, network router, switch, or bridge, or any machine capable of executing instructions (sequentially or otherwise) specifying the actions to be taken by said machine. Furthermore, although only a single machine is shown, the term "machine" should also be considered as encompassing any collection of machines that individually or jointly execute a set (or more) of instructions to perform any of the methods discussed herein, such as cloud computing, Software as a Service (SaaS), and other computer cluster configurations.

[0062] Machine (e.g., computer system) 700 may include a hardware processor 702 (e.g., a central processing unit (CPU), graphics processing unit (GPU), hardware processor core, or any combination thereof), main memory 704, static memory (e.g., memory or storage device for firmware, microcode, basic input / output (BIOS), unified extensible firmware interface (UEFI), etc.) 706, and mass storage device 708 (e.g., hard disk drive, tape drive, flash memory, or other block device), some or all of which may communicate with each other via interconnect (e.g., bus) 730. Machine 700 may further include a display unit 710, an alphanumeric input device 712 (e.g., keyboard), and a user interface (UI) navigation device 714 (e.g., mouse). In an example, the display unit 710, input device 712, and UI navigation device 714 may be a touchscreen display. Machine 700 may also include a mass storage device (e.g., a drive unit) 708, a signal generation device 718 (e.g., a speaker), a network interface device 720, and one or more sensors 716, such as a Global Positioning System (GPS) sensor, a compass, an accelerometer, or other sensors. Machine 700 may include an output controller 728, for example, a serial (e.g., Universal Serial Bus (USB), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection, to communicate with or control one or more peripheral devices (e.g., a printer, a card reader, etc.).

[0063] The registers of processor 702, main memory 704, static memory 706, or mass storage device 708 may be or contain machine-readable medium 722, on which one or more sets of data structures or instructions 724 (e.g., software) embodying or for use with any one or more of the technologies or functions described herein. Instructions 724 may also reside wholly or at least partially in any of the registers of processor 702, main memory 704, static memory 706, or mass storage device 708 during execution by machine 700. In an example, one or any combination of hardware processor 702, main memory 704, static memory 706, or mass storage device 708 may constitute machine-readable medium 722. Although machine-readable medium 722 is shown as a single medium, the term "machine-readable medium" may include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) configured to store the one or more instructions 724.

[0064] The term "machine-readable medium" can include any medium capable of storing, encoding, or transmitting instructions executable by machine 700 and causing machine 700 to perform any one or more of the technologies disclosed herein, or any medium capable of storing, encoding, or transmitting data structures used by or associated with such instructions. Examples of non-limiting machine-readable media include solid-state memory, optical media, magnetic media, and signals (e.g., radio frequency signals, other photon-based signals, sound signals, etc.). In examples, non-transitory machine-readable media include machine-readable media having a plurality of particles with constant mass (e.g., stationary), and are therefore compositions of matter. Thus, a non-transitory machine-readable medium is a machine-readable medium that does not contain transiently propagating signals. Specific examples of non-transitory machine-readable media include: non-volatile memory, such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable hard disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0065] In an example, information stored on or otherwise provided on a machine-readable medium 722 may represent instructions 724, such as instructions 724 themselves or a format from which instructions 724 can be derived. The format from which instructions 724 can be derived may include source code, encoded instructions (e.g., in compressed or encrypted form), packaging instructions (e.g., splitting into multiple packages), etc. The information representing instructions 724 in the machine-readable medium 722 can be processed by a processing circuitry system into instructions to perform any of the operations discussed herein. For example, deriving instructions 724 from information (e.g., processing performed by a processing circuitry system) may include: compiling (e.g., from source code, object code, etc.), interpreting, loading, organizing (e.g., dynamic or static linking), encoding, decoding, encrypting, decrypting, packaging, unpacking, or otherwise manipulating the information into instructions 724.

[0066] In an example, the derivation of instruction 724 may involve (e.g., by a processing circuitry system) the assembly, compilation, or interpretation of information to create instruction 724 from some intermediate or preprocessed format provided by machine-readable medium 722. When information is provided in multiple parts, it can be combined, unpacked, and modified to create instruction 724. For example, information may be contained in multiple compressed source code packages (or object code, or binary executable code, etc.) on one or more remote servers. The source code packages may be encrypted during transmission over a network, decrypted, decompressed, assembled (e.g., linked), and compiled or interpreted on a local machine (e.g., into a library, as a standalone executable, etc.) and executed by the local machine.

[0067] Instruction 724 may further use a transmission medium via network interface device 720 to transmit or receive data through communication network 726 using any of a plurality of transmission protocols (e.g., Frame Relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.). Example communication networks may include Local Area Networks (LANs), Wide Area Networks (WANs), packet data networks (e.g., the Internet), mobile phone networks (e.g., cellular networks), ordinary legacy telephone (POTS) networks, and wireless data networks (e.g., the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard series (referred to as...). ), IEEE 802.16 series of standards (referred to as This includes standards such as IEEE 802.15.4, peer-to-peer (P2P) networks, etc. In an example, network interface device 720 may include one or more physical jacks (e.g., Ethernet, coaxial, or telephone jacks) or one or more antennas to connect to communication network 726. In an example, network interface device 720 may include multiple antennas to perform wireless communication using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) technologies. The term "transmission medium" should be considered as any intangible medium capable of storing, encoding, or transmitting instructions executable by machine 700, and includes digital or analog communication signals or other intangible media to facilitate communication of such software. The transmission medium is a machine-readable medium. To better illustrate the methods and apparatus described herein, a set of non-limiting example embodiments are described below as examples identified by numbers.

[0068] Other notes and examples

[0069] Example 1 is a device comprising: a memory array; an input queue coupled to the memory array and configured to receive a memory request from a request processor; and logic coupled to the input queue and the memory array and configured to: select a memory request from the input queue, the memory request being received from the request processor; determine that the memory request is a write request to write a requested value to a requested location in the memory of the memory array; in response to determining that the memory request is a write request: send a response to the request processor, the response being sent before executing the write request; and after sending the response to the request processor, execute the write request to write the requested value to the requested location in the memory.

[0070] In Example 2, the subject of Example 1 includes the logic wherein the logic is configured to execute the write request to write the requested value to the requested location in memory by configuring the request value to be written to the cache.

[0071] In Example 3, the subject of Examples 1-2 includes the memory request being received from the external interface of the device at the input queue.

[0072] In Example 4, the subject of Example 3 includes the memory request being received via an on-chip network interface.

[0073] In Example 5, the subject matter of Examples 1-4 includes the logic on the first chiplet and the request processor on the second chiplet.

[0074] In Example 6, the subject of Examples 1-5 includes the logic wherein the memory request is selected from the input queue by configuring the memory request to be selected from multiple input queues using an input multiplexer.

[0075] In Example 7, the subject of Examples 1-6 includes the memory array comprising volatile memory that communicates with the logic via a memory interface on one or more second chips.

[0076] In Example 8, the subject of Examples 1-7 includes the logic configured to perform the ordering of the write requests such that a previous write request to the requested location is completed first, and a subsequent write command to the requested location is completed after the write request.

[0077] In Example 9, the subject of Examples 1-8 includes wherein the logic is configured to execute the write request to write the requested value to the requested location in the memory by configuring the request value to be written to the memory cell.

[0078] In Example 10, the subject of Examples 1-9 includes the logic wherein the logic is configured to execute the write request to write the requested value to the requested location in memory by configuring the requested value to be written to the write cache corresponding to the memory.

[0079] Example 11 is a method comprising: selecting a memory request from an input queue, the memory request being received from a request processor; determining that the memory request is a write request to write a requested value to a requested location in memory; in response to determining that the memory request is a write request: sending a response to the request processor, the response being sent before executing the write request; and after sending the response to the request processor, executing the write request to write the requested value to the requested location in memory.

[0080] In Example 12, the subject of Example 11 includes, wherein performing the write request to write the requested value to the requested location in memory includes writing the requested value to the cache.

[0081] In Example 13, the subject of Examples 11-12 includes the memory request received across the external interface of the chiplet system.

[0082] In Example 14, the subject of Example 13 includes the memory request being received via an on-chip network interface.

[0083] In Example 15, the subject of Examples 11-14 includes the following: selecting the memory request from the input queues includes selecting the memory request from multiple input queues using an input multiplexer.

[0084] In Example 16, the subject of Examples 11-15 includes performing the ordering of the write requests such that a previous write request to the requested location is completed first, and a subsequent write command to the requested location is completed after the write request.

[0085] In Example 17, the subject of Examples 11-16 includes, wherein performing the write request to write the requested value to the requested location in the memory includes writing the requested value to a memory cell in the memory.

[0086] In Example 18, the subject of Examples 11-17 includes, wherein performing the write request to write the requested value to the requested location in memory includes writing the requested value to a write cache corresponding to the memory.

[0087] Example 19 is a non-transitory machine-readable medium storing instructions that, when executed by a machine, cause the machine to perform operations including: selecting a memory request from an input queue, the memory request being received from a request processor; determining that the memory request is a write request to write a requested value to a requested location in memory; in response to determining that the memory request is a write request: sending a response to the request processor, the response being sent before executing the write request; and after sending the response to the request processor, executing the write request to write the requested value to the requested location in memory.

[0088] In Example 20, the subject of Example 19 includes an operation in which the write request to write a requested value to a requested location in memory includes writing the requested value to a cache.

[0089] In Example 21, the subject of Examples 19-20 includes the memory request received across the chiplet system's external interface.

[0090] In Example 22, the subject of Example 21 includes the memory request being received via an on-chip network interface.

[0091] In Example 23, the subject of Examples 19-22 includes an operation in which the memory request is selected from the input queues, which includes using an input multiplexer to select the memory request from multiple input queues.

[0092] In Example 24, the subject of Examples 19-23 includes the operation further comprising performing the write requests in a sequence such that a previous write request to the requested location is completed first, and a subsequent write command to the requested location is completed after the write request.

[0093] In Example 25, the subject of Examples 19-24 includes an operation in which the write request to write the requested value to the requested location in the memory includes writing the requested value to a memory cell of the memory.

[0094] In Example 26, the subject of Examples 19-25 includes an operation in which the write request to write the requested value to the requested location in memory includes writing the requested value to a write cache corresponding to the memory.

[0095] Example 27 is an apparatus comprising: means for selecting a memory request from an input queue, the memory request being received from a request processor; means for determining that the memory request is a write request for writing a requested value to a requested location in memory; means for sending a response to the request processor in response to determining that the memory request is a write request, the response being sent before executing the write request; and means for executing the write request for writing the requested value to the requested location in memory after sending the response to the request processor.

[0096] In Example 28, the subject of Example 27 includes a component for performing the write request to write the requested value to a requested location in memory, which includes a component for writing the requested value to a cache.

[0097] In Example 29, the subject of Examples 27-28 includes the memory request received across the chiplet system's external interface.

[0098] In Example 30, the subject of Example 29 includes the memory request being received via an on-chip network interface.

[0099] In Example 31, the subject matter of Examples 27-30 includes a component for selecting the memory request from the input queue, which includes a component for selecting the memory request from multiple input queues using an input multiplexer.

[0100] In Example 32, the subject of Examples 27-31 includes components for performing the ordering of the write requests such that a previous write request to the requested location is completed first, and a subsequent write command to the requested location is completed after the write request.

[0101] In Example 33, the subject matter of Examples 27-32 includes a component for performing a write request to write the requested value to the requested location in the memory, which includes a component for writing the requested value to the memory.

[0102] In Example 34, the subject matter of Examples 27-33 includes a component for performing a write request to write the requested value to the requested location in memory, which includes a component for writing the requested value to a write cache corresponding to the memory.

[0103] Example 35 is at least one machine-readable medium containing instructions that, when executed by a processing circuitry system, cause the processing circuitry system to perform operations to implement any one of Examples 1-34.

[0104] Example 36 is a device that includes components for implementing any one of Examples 1-34.

[0105] Example 37 is a system for implementing any one of Examples 1-34.

[0106] Example 38 is a method for implementing any of Examples 1-34.

[0107] The above detailed description includes reference to the accompanying drawings, which form part of the detailed description. The drawings illustrate specific embodiments in which the invention may be practiced by means of illustration. These embodiments are also referred to herein as “examples.” Such examples may include elements other than those shown or described. However, the inventors also contemplate that only examples of those elements shown or described are provided herein. Furthermore, the inventors also contemplate examples (or aspects thereof) of any combination or arrangement of those elements shown or described with respect to a particular example (or one or more aspects thereof) or with respect to other examples (or one or more aspects thereof) shown or described herein.

[0108] In this document, the term "a" is used as is common in patent literature to include one or more, independent of any other example or use of "at least one" or "one or more". In this document, the term "or" is used to refer to a non-exclusive "or", such that unless otherwise indicated, "A or B" may include "A but not B", "B but not A", and "A and B". In the appended claims, the terms "comprising" and "in which" are used as concise equivalents to the corresponding terms "including" and "wherein". Furthermore, in the appended claims, the terms "comprising" and "including" are open-ended, meaning that a system, apparatus, article, or process that includes elements other than those listed after such terms is still considered to be within the scope of the claims. Additionally, in the appended claims, the terms "first", "second", and "third", etc., are used only as designations and are not intended to impose numerical requirements on their objects.

[0109] The above description is intended to be illustrative and not restrictive. For example, the above examples (or one or more aspects thereof) can be used in combination with each other. Other embodiments may be used by those skilled in the art upon review of the above description. It is submitted on the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the above detailed description, various features may be grouped together to simplify this disclosure. This should not be construed as expecting that any unclaimed disclosed features are necessary for any claim. In fact, the subject matter of the invention may lie in less than all the features of a particular disclosed embodiment. Therefore, the appended claims are hereby incorporated into the detailed description, wherein each claim exists independently as a separate embodiment, and these embodiments are contemplated to be combined or arranged in various ways. The scope of the invention should be determined by reference to the appended claims and the full scope of the equivalents to which these claims are given.

Claims

1. An apparatus comprising: Memory array; A first queue, coupled to the memory array and configured to receive memory requests from a request processor; Logic, coupled to the first queue, the second queue, and the memory array, and configured as follows: A memory request is selected from the first queue using a multiplexer, the multiplexer multiplexing between the first queue and the second queue, and the memory request is received from the request processor. The memory request is determined to be a write request to write the requested value to the requested location in the memory of the memory array; In response to determining that the memory request is a write request: A response is sent to the request processor before the write request is executed; After sending the response to the request processor, the write request is executed to write the requested value to the requested location in the memory; as well as The write requests are ordered such that previous write requests to the requested location are completed first, and subsequent write commands to the requested location are completed after the write requests.

2. The device of claim 1, wherein the logic is configured to execute the write request to write the requested value to the requested location in memory by configuring the request value to be written to a cache.

3. The device of claim 1, wherein the memory request is received from an external interface of the device at the first queue.

4. The device of claim 3, wherein the memory request is received via an on-chip network interface.

5. The device of claim 1, wherein the logic is on a first chiplet and the request processor is on a second chiplet.

6. The device of claim 1, wherein the memory array includes volatile memory communicating with the logic via a memory interface on one or more second chips.

7. The device of claim 1, wherein the logic is configured to execute the write request to write the request value to the requested location in the memory by means of a memory cell configured to write the request value to the memory.

8. The device of claim 1, wherein the logic is configured to execute the write request to write the requested value to the requested location in the memory by configuring the request value to be written to a write cache corresponding to the memory.

9. A method comprising: A memory request is selected from a first queue using a multiplexer, which multiplexes the memory request between the first queue and a second queue, and the memory request is received from a request processor. The memory request is determined to be a write request to write the requested value to the requested location in the memory; In response to determining that the memory request is a write request: A response is sent to the request processor before the write request is executed; After sending the response to the request processor, the write request is executed to write the requested value to the requested location in the memory; as well as The write requests are ordered such that previous write requests to the requested location are completed first, and subsequent write commands to the requested location are completed after the write requests.

10. The method of claim 9, wherein executing the write request to write the requested value to the requested location in memory includes writing the requested value to a cache.

11. The method of claim 9, wherein the memory request is received across an external interface of the chiplet system.

12. The method of claim 11, wherein the memory request is received via an on-chip network interface.

13. The method of claim 9, wherein executing the write request to write the requested value to the requested location in the memory comprises writing the requested value to a memory cell of the memory.

14. The method of claim 9, wherein executing the write request to write the requested value to the requested location in memory comprises writing the requested value to a write cache corresponding to the memory.

15. A non-transitory machine-readable medium storing instructions that, when executed by a machine, cause the machine to perform an operation, comprising: A memory request is selected from a first queue using a multiplexer, which multiplexes the memory request between the first queue and a second queue, and the memory request is received from a request processor. The memory request is determined to be a write request to write the requested value to the requested location in the memory; In response to determining that the memory request is a write request: A response is sent to the request processor before the write request is executed; After sending the response to the request processor, the write request is executed to write the requested value to the requested location in the memory; as well as The write requests are ordered such that previous write requests to the requested location are completed first, and subsequent write commands to the requested location are completed after the write requests.

16. The non-transitory machine-readable medium of claim 15, wherein the operation of performing the write request to write the requested value to the requested location in memory includes writing the requested value to a cache.

Images