Network credit return mechanism

Abstract

Embodiments of the present disclosure relate to systems and methods for reducing design complexity and critical path timing challenges of credit return logic. A wide bus supports simultaneous transmission of multiple flits, with each lane of the wide bus transmitting one flit. A source device transmitting flits on the wide bus selects from multiple credit return options to ensure that only one of the multiple flits transmitted simultaneously contains a credit return value. In some example embodiments, a sink device only checks the flit of one lane (e.g., lane 0) of the wide bus for credit return data. In other example embodiments, the sink device uses a bitwise OR to combine the credit return data of all received flits in a single cycle.

Description

[0001] Priority application

[0002] This application claims priority to U.S. Application Serial No. 17 / 007,814, filed August 31, 2020, which is incorporated herein by reference in its entirety.

[0003] Statement regarding government support

[0004] This invention was developed with the support of the U.S. government under DARPA Contract No. HR00111830003. The U.S. government owns certain rights to this invention. Technical Field

[0005] Embodiments of this disclosure generally relate to network protocols, and more specifically, to networking using simplified credit return logic. In particular instances, protocols may be implemented to manage network communications in chiplet-based systems. Background Technology

[0006] A sending device using a credit-based flow control system reduces its available credit before sending data to a receiving device. The receiving device buffers the data. After the data is removed from the buffer and processed, the receiving device sends a response message to the sending device. In response to receiving the response message, the sending device increases its available credit.

[0007] Chiplets are an emerging technology for integrating various processing functionalities. Typically, a chiplet system consists of discrete chips (e.g., integrated circuits (ICs) on different substrates or dies) integrated on an interposer and packaged together. This arrangement differs from a single chip (e.g., an IC) containing different device blocks (e.g., intellectual property (IP) blocks) on a single substrate (e.g., a single die), such as a system-on-a-chip (SoC) or a discrete packaged device integrated on a board. Generally, chiplets offer better performance (e.g., lower power consumption, reduced latency, etc.) than discrete packaged devices, and offer greater manufacturing benefits than single-die chips. These manufacturing benefits can include higher yields or reduced development costs and time.

[0008] Chiplet systems generally consist of one or more application chiplets and support chiplets. Here, the distinction between application and support chiplets is merely a reference to possible design scenarios for chiplet systems. Thus, for example, a synthetic vision chiplet system may include application chiplets for generating synthetic vision output and support chiplets, such as memory controller chiplets, sensor interface chiplets, or communication chiplets. In typical use cases, synthetic vision designers may design the application chiplets and obtain support chiplets from other sources. Therefore, design costs (e.g., in terms of time or complexity) are reduced by avoiding the design and manufacturing of the functionality embodied in the support chiplets. Chiplets also support tight integration of intellectual property blocks that might otherwise be difficult to achieve, such as those using different feature sizes. Therefore, for example, devices designed during previous-generation manufacturing with larger feature sizes, or those in which feature sizes are optimized for power, speed, or heat (which can occur in sensors), are easier to integrate with devices of different feature sizes than to attempt to do so on a single die. Furthermore, by reducing the overall die size, chiplet yields tend to be higher than those of more complex single-die devices. Attached Figure Description

[0009] This disclosure will be more fully understood from the detailed description given below and the accompanying drawings of various embodiments thereof. However, the drawings should not be construed as limiting this disclosure to the specific embodiments, but are for explanation and understanding only.

[0010] Figure 1A and 1B An example of a chiplet system according to an embodiment is described.

[0011] Figure 2 This describes the components of an example of a memory controller chiplet according to an embodiment.

[0012] Figure 3 This describes an example of routing between chiplets using a chiplet protocol interface (CPI) network according to an embodiment.

[0013] Figure 4 This is a block diagram of a data packet comprising multiple microchips according to some embodiments of the present disclosure.

[0014] Figure 5 This is a block diagram of a credit return data packet according to some embodiments of the present disclosure.

[0015] Figure 6 This is a block diagram illustrating a received chip stream and a virtual channel queue of received chips according to some embodiments of the present disclosure.

[0016] Figure 7This is a flowchart illustrating the operation of a method performed by a circuit when updating credit data according to some embodiments of the present disclosure.

[0017] Figure 8 This is a flowchart illustrating the operation of a method performed by a circuit when updating credit data according to some embodiments of the present disclosure.

[0018] Figure 9 This is a block diagram of an example computer system in which embodiments of the present disclosure may be operated. Detailed Implementation

[0019] This disclosure relates to systems and methods for reducing the design complexity and critical path timing challenges of credit return logic. A wide bus supports the simultaneous transmission of multiple microchips, with each channel of the wide bus transmitting one microchip. A source device transmitting a microchip on the wide bus selects from multiple credit return options to ensure that only one of the concurrently transmitted microchips contains a credit return value. In some exemplary embodiments, a receiving device examines only the microchips of a predetermined channel (e.g., channel 0) of the wide bus to obtain credit return data, and a transmitting device uses only said predetermined channel to transmit the credit return data. In other exemplary embodiments, the receiving device uses bitwise OR to combine the credit return data of all received microchips in a single cycle, and the transmitting device uses any delay to transmit the credit return data. When the default credit return value is all zero, the bitwise OR produces only the credit return data of the microchip containing said credit return data, regardless of which channel the credit return data is carried on.

[0020] The benefits of the embodiments disclosed herein include the elimination of additional circuitry (e.g., adders for summing credits returned from each channel), thereby reducing design complexity and avoiding added latency in the receiver's critical path. The processing loops consumed in transmitting, receiving, and processing credit return data are reduced. Additionally, power consumption during processing is reduced. The performance of systems including communication devices is also improved due to reduced networking overhead. Other benefits will be apparent to those skilled in the art who will recognize the benefits of this disclosure.

[0021] Figure 1A and 1B An example of a chiplet system 110 according to an embodiment is described. Figure 1AThis is an illustration of a chiplet system 110 mounted on a peripheral board 105, which can be connected to a wider range of computer systems, for example, via a peripheral component interconnect (PCIe) high-speed connection. The chiplet system 110 includes a package substrate 115, an interposer 120, and four chips: an application chiplet 125, a host interface chiplet 135, a memory controller chiplet 140, and a memory device chiplet 150. Other systems may include numerous additional chipsets to provide additional functionality, as will be apparent from the following discussion. The package of the chiplet system 110 is illustrated with a cap or cover plate 165, but other packaging technologies and structures used for chiplet systems may be used. Figure 1B This is a block diagram for the purpose of clearly labeling the components in a chiplet system.

[0022] Application chip 125 is described as including a network on-chip (NOC) 130 to support a chiplet network 155 for inter-chiplet communication. In an exemplary embodiment, NOC 130 may be included on application chip 125. In an example, NOC 130 may be defined in response to selected supporting chipsets (e.g., chipsets 135, 140, and 150), thus enabling designers to select an appropriate number of chiplet network connections or switches for NOC 130. In an example, NOC 130 may reside on a single chiplet or even within interposer 120. In the example discussed herein, NOC 130 implements a CPI network.

[0023] CPI is a packet-based network that supports virtual channels to enable flexible and high-speed interaction between chiplets. CPI bridges the chiplet-internal network to chiplet network 155. For example, the Advanced Extensible Interface (AXI) is a widely used specification for designing intra-chip communication. However, the AXI specification covers a large number of physical design options, such as the number of physical channels, signal timing, and power. Within a single chip, these options are generally selected to meet design goals, such as power consumption and speed. However, to achieve flexibility in chiplet systems, adapters such as CPI are used to intersect between various AXI design options that can be implemented in various chiplets. By implementing a mapping from physical channels to virtual channels and encapsulating time-based signaling using packetization protocols, CPI bridges the chiplet-internal network across chiplet network 155.

[0024] CPI can use a variety of different physical layers to transmit packets. A physical layer may contain simple conductive connections or drivers to increase voltage, or otherwise facilitate signal transmission over longer distances. An example of such a physical layer may include an Advanced Interface Bus (AIB), which in various instances may be implemented in intermediate layer 120. The AIB uses source-synchronous data transfer with a forwarding clock to transmit and receive data. Packets are transmitted across the AIB at Single Data Rate (SDR) or Double Data Rate (DDR) relative to the transmitted clock. The AIB supports various channel widths. When operating in SDR mode, the AIB channel width is a multiple of 20 bits (20, 40, 60, ...), and for DDR mode, the AIB channel width is a multiple of 40 bits (40, 80, 120, ...). The AIB channel width includes both transmit (TX) and receive (RX) signals. Channels can be configured to have a symmetrical number of TX and RX inputs / outputs (I / O), or an asymmetrical number of transmitters and receivers (e.g., all transmitters or all receivers). The AIB channel can act as either a master or slave device, depending on which chip provides the master clock. The AIB I / O unit supports three clock modes: asynchronous (i.e., non-timed), SDR, and DDR. In various instances, the non-timed mode is used for clocking and some control signals. SDR mode can use dedicated SDR-only I / O units or dual-purpose SDR / DDR I / O units.

[0025] In this example, the CPI packet protocol (e.g., point-to-point or routable) can use symmetrical receive and transmit I / O units within an AIB channel. The CPI streaming protocol allows for more flexible utilization of AIB I / O units. In this example, a streaming AIB channel can configure I / O units as all TX, all RX, or half TX and half RX. The CPI packet protocol can use AIB channels in SDR or DDR operating modes. In this example, AIB channels are configured in increments of 80 I / O units (i.e., 40 TX and 40 RX) for SDR mode and in increments of 40 I / O units for DDR mode. The CPI streaming protocol can use AIB channels in either SDR or DDR operating modes. Here, in this example, the AIB channels are configured in increments of 40 I / O units for both SDR and DDR modes. In this example, a unique interface identifier is assigned to each AIB channel. This identifier is used during CPI reset and initialization to determine paired AIB channels across adjacent chiplets. In this example, the interface identifier is a 20-bit value comprising a seven-bit chiplet identifier, a seven-bit column identifier, and a six-bit link identifier. The AIB physical layer uses an AIB out-of-band shift register to transmit the interface identifier. Bits 32 to 51 of the shift register are used to transmit the 20-bit interface identifier in both directions across the AIB interface.

[0026] AIB defines a stacked group of AIB channels as an AIB channel column. An AIB channel column has a certain number of AIB channels, plus an auxiliary (AUX) channel. The AUX channel contains signals used for AIB initialization. All AIB channels within a column (except for the AUX channel) have the same configuration (e.g., all TX, all RX, or half TX and half RX, and the same number of data I / O signals). In this example, AIB channels are numbered sequentially in ascending order, starting with the AUX channel. The AIB channel adjacent to the AUX channel is defined as AIB channel zero.

[0027] Typically, the CPI interface on an individual chiplet may include serialization-deserialization (SERDES) hardware. SERDES interconnects are well-suited for scenarios requiring high-speed signaling and low signal counts. However, SERDES can introduce additional power consumption and longer latency for multiplexing and demultiplexing, error detection or correction (e.g., using block-level cyclic redundancy check (CRC)), link-level retries, or forward error correction. However, when low latency or power consumption is a primary concern for ultra-short-distance chiplet-to-chiplet interconnects, parallel interfaces that allow data transfer with minimal latency can be utilized. CPIs contain components designed to minimize both latency and power consumption in these ultra-short-distance chiplet interconnects.

[0028] For flow control, CPI employs a credit-based technique. For example, the receiving side of application chip 125, or the sender side of memory controller chip 140, provides credits indicating available buffers. In this example, the CPI receiver contains buffers for each virtual channel for a given transmission time unit. Therefore, if the CPI receiver supports five messages and a single virtual channel in time, the receiver has five buffers arranged in five rows (e.g., one row per unit time). If four virtual channels are supported, the receiver has twenty buffers arranged in five rows. Each buffer holds the payload of one CPI packet.

[0029] When a sender transmits to a receiver, the sender decrements its available credits based on the transmission. Once the receiver has exhausted all its credits, the sender stops sending packets to the receiver. This ensures that the receiver always has an available buffer to store transmissions.

[0030] When the receiver processes the received packet and releases the buffer, it relays the available buffer space back to the sender. The sender can then use this credit to allow the transmission of additional information.

[0031] It also describes a chiplet mesh network 160 that uses direct chiplet-to-chiplet technology without requiring NOC 130. The chiplet mesh network 160 can be implemented in CPI or another chiplet-to-chiplet protocol. The chiplet mesh network 160 typically implements a chiplet pipeline, where one chiplet acts as an interface to the pipeline, while other chipslets in the pipeline interface only interface with themselves.

[0032] Additionally, dedicated device interfaces, such as one or more industry-standard memory interfaces 145 (e.g., synchronous memory interfaces, such as DDR5, DDR6), can also be used to interconnect chiplets. Connections from a chiplet system or individual chiplets to external devices (e.g., larger systems) can be made via the desired interface (e.g., a PCIe interface). In an example, this external interface can be implemented via a host interface chiplet 135, which, in the depicted example, provides a PCIe interface external to the chiplet system 110. Such interfaces are typically used when industry practice or standards have converged on them. The illustrated example of connecting a memory controller chiplet 140 to the DDR interface 145 of a dynamic random access memory (DRAM) memory device chiplet 150 exemplifies this industry practice.

[0033] Among the various possible supporting chiplets, the memory controller chiplet 140 is likely to be present in the chiplet system 110 due to the near-ubiquitous use of storage for computer processing and the advanced technology of memory devices. Therefore, using the memory device chiplet 150 and memory controller chiplet 140, both designed by other designers, allows chiplet system designers to obtain robust products from established manufacturers. Typically, the memory controller chiplet 140 provides a memory device-specific interface for reading, writing, or erasing data. The memory controller chiplet 140 often provides additional features such as error detection, error correction, maintenance operations, or atomic operation execution. For some types of memory, maintenance operations are often specific to the memory device chiplet 150, such as garbage collection in NAND flash or memory-class memory and temperature regulation (e.g., cross-temperature management) in NAND flash memory. In instances, maintenance operations may involve logic-to-physical (L2P) mapping or management to provide an indirection hierarchy between the physical and logical representations of data. In other types of memory, such as DRAM, some memory operations, such as refresh, may be controlled at certain times by the host processor or memory controller, and at other times by the DRAM memory device or logic associated with one or more DRAM devices, such as an interface chip (in this example, a buffer).

[0034] Atomic operations are data manipulations that can be performed, for example, by the memory controller chiplet 140. In other chiplet systems, atomic operations can be performed by other chipsets. For example, an atomic operation can be specified as an "increment" in a command by the application chiplet 125, the command containing a memory address and possibly an increment value. Upon receiving the command, the memory controller chiplet 140 retrieves a number from the specified memory address, increments the number by the amount specified in the command, and stores the result. Upon successful completion, the memory controller chiplet 140 provides the application chiplet 125 with an indication that the command was successful. Atomic operations avoid transferring data across the chiplet mesh network 160, thereby reducing latency in executing such commands.

[0035] Atomic operations can be categorized into built-in atoms or programmable (e.g., custom) atoms. Built-in atoms are a finite set of operations implemented immutably in the hardware. Programmable atoms are applets that can run on programmable atom units (PAUs) (e.g., custom atom units (CAUs)) of the memory controller chiplet 140. Figure 1 illustrates an example of a memory controller chiplet including PAUs.

[0036] Memory device chip 150 may be or include any combination of volatile memory devices or non-volatile memory. Examples of volatile memory devices include, but are not limited to, random access memory (RAM, such as DRAM), synchronous DRAM (SDRAM), and graphics dual data rate type 6 SDRAM (GDDR6 SDRAM), etc. Examples of non-volatile memory devices include, but are not limited to, NAND flash memory, memory-class memory (e.g., phase-change memory or memristor-based technology), and ferroelectric RAM (FeRAM), etc. The illustrated examples include memory devices as memory device chip 150; however, the memory devices may reside elsewhere, such as in different packages on board 105. For many applications, multiple memory device chips may be provided. In examples, these memory device chips may each implement one or more memory technologies. In examples, the memory chip may include multiple stacked memory dies of different technologies (e.g., one or more SRAM devices stacked or otherwise communicating with one or more DRAM devices). The memory controller chiplet 140 can also be used to coordinate the operation between multiple memory chipsets in the chiplet system 110 (e.g., utilizing one or more memory chipsets in one or more tiers of cache memory and using one or more additional memory chipsets as main memory). The chiplet system 110 may also include multiple memory controller chipsets 140, which can be used to provide memory control functionality for individual processors, sensors, networks, etc. For example, the chiplet architecture of the chiplet system 110 provides the benefit of allowing updated chiplet configurations to be adapted to different memory storage technologies and different memory interfaces without requiring redesign of the rest of the system architecture.

[0037] Figure 2 The components of an example of a memory controller chiplet 205 according to an embodiment are described below. The memory controller chiplet 205 includes a cache 210, a cache controller 215, an off-die memory controller 220 (e.g., for communicating with off-die memory 275), a network communication interface 225 (e.g., for interfacing with a chiplet network 285 and communicating with other chiplets), and a set of atom and merge operation units 250. This set of components may include, for example, a write merge unit 255, a danger clearing unit 260, a built-in atom unit 265, or a PAU 270. The various components are described logically, and they may not necessarily be implemented. For example, a built-in atom unit 265 may include different means along the path to off-die memory. For example, a built-in atom unit may be in an interface means / buffer on the memory chiplet, as discussed above. In contrast, a PAU 270 may be implemented in a separate processor on the memory controller chiplet 205 (but in various instances, it may be implemented in other locations, such as on the memory chiplet itself).

[0038] The off-die memory controller 220 is directly coupled to the off-die memory 275 (e.g., via a bus or other communication connection) to provide write and read operations to and from the one or more off-die memories (e.g., off-die memory 275 and off-die memory 280). In the depicted example, the off-die memory controller 220 is also coupled for output to the atom and merge operation unit 250 and for input to the cache controller 215 (e.g., a memory-side cache controller).

[0039] In the instance configuration, the cache controller 215 is directly coupled to the cache 210 and can be coupled to the network communication interface 225 for input (e.g., incoming read or write requests) and coupled to the off-die memory controller 220.

[0040] Network communication interface 225 includes packet decoder 230, network input queue 235, packet encoder 240, network output queue 245, and credit checker and updater 290 to support packet-based chiplet network 285, such as CPI. Chiplet network 285 can provide packet routing between processors, memory controllers, mixed-thread processors, configurable processing circuitry, or communication interfaces. In this packet-based communication system, each packet typically contains destination and source addressing, as well as any data payload or instructions. In some instances, depending on the configuration, chiplet network 285 can be implemented as a collection of crossbars with a folded Clos configuration, or as a mesh network providing additional connectivity.

[0041] Packet decoder 230 decodes incoming packets, thereby converting them into internal memory instructions provided to cache controller 215. Additionally, credit return data may be included in the incoming packets and provided to credit checker and updater 290 to update available credits (e.g., credits of a virtual channel available for the source device of the incoming packets).

[0042] In various instances, the chiplet network 285 may be part of an asynchronous switching architecture. Here, data packets may be routed along any of various paths, such that any selected data packet may arrive at the addressed destination at any time among multiple different times, depending on the route. Additionally, the chiplet network 285 may be implemented at least partially as a synchronous communication network, such as a synchronous mesh communication network. Two configurations of the communication network are contemplated for use according to this disclosure, for example.

[0043] The memory controller chip 205 can receive packets having, for example, a source address, a read request, and a physical address. In response, the off-die memory controller 220 or the cache controller 215 reads data from the specified physical address (which may be in off-die memory 275 or cache 210) and assembles a response packet into a source address containing the requested data. Similarly, the memory controller chip 205 can receive packets having a source address, a write request, and a physical address. In response, the memory controller chip 205 writes data to the specified physical address (which may be in cache 210 or off-die memory 275 or 280) and assembles a response packet into a source address containing confirmation that data has been stored in memory.

[0044] Therefore, where possible, the memory controller chiplet 205 can receive read and write requests via chiplet network 285 and process the requests using cache controller 215, which interfaces with cache 210. If cache controller 215 cannot process the request, off-chip memory controller 220 processes the request by communicating with off-chip memory 275 or 280, atom and merge operation unit 250, or both. As described above, one or more cache levels can also be implemented in off-chip memory 275 or 280, and in some such instances, can be directly accessed by cache controller 215. Data read by off-chip memory controller 220 can be cached in cache 210 by cache controller 215 for later use.

[0045] The atomic and merging operation unit 250 is coupled to receive (as input) the output of the off-die memory controller 220 and provides the output to the cache 210, the network communication interface 225, or directly to the chiplet network 285. The memory danger clearing (reset) unit 260, the write merging unit 255, and the built-in (e.g., predetermined) atomic operation unit 265 can each be implemented as a state machine with other combinational logic circuitry (e.g., adders, shifters, comparators, AND gates, OR gates, XOR gates, or any suitable combination thereof) or other logic circuitry. These components may also include one or more registers or buffers to store operands or other data. The PAU 270 can be implemented as one or more processor cores or control circuitry systems, and various state machines with other combinational logic circuitry or other logic circuitry systems, and may also include one or more registers, buffers, or memories to store addresses, executable instructions, operands, and other data, or may be implemented as a processor.

[0046] The write merging unit 255 receives read data and request data, and merges the request data and read data to produce a single unit containing the read data and the source address to be used in the response or return data packet. The write merging unit 255 provides the merged data to the write port of cache 210 (or equivalently, to cache controller 215 for writing to cache 210). Optionally, the write merging unit 255 provides the merged data to network communication interface 225 for encoding and preparing response or return data packets for transmission on chiplet network 285.

[0047] When requested data is used for a built-in atomic operation, the built-in atomic operation unit 265 receives the request and reads the data from the write merging unit 255 or directly from the off-chip memory controller 220. The atomic operation is performed, and using the write merging unit 255, the resulting data is written to the cache 210 or provided to the network communication interface 225 to encode and prepare a response or return data packet for transmission on the chiplet network 285.

[0048] Built-in atomic operation unit 265 handles predefined atomic operations, such as fetch and increment or compare and swap. In examples, these operations perform simple read-modify-write operations on a single memory location of 32 bytes or less. An atomic memory operation is initiated from a request packet transmitted via chiplet network 285. The request packet has a physical address, atomic operator type, operand size, and optionally up to 32 bytes of data. The atomic operation performs a read-modify-write operation on a cache line of cache 210, thereby filling the cache memory as necessary. The atomic operator response can be a simple complete response or a response with up to 32 bytes of data. Example atomic memory operators include fetch and AND, fetch and OR, fetch and XOR, fetch and add, fetch and subtract, fetch and increment, fetch and decrement, fetch and minimum, fetch and maximum, fetch and swap, and compare and swap. In various exemplary embodiments, 32-bit and 64-bit operations and operations on 16 or 32 bytes of data are supported. The methods disclosed herein are also compatible with hardware that supports larger or smaller operations and more or less data.

[0049] Built-in atomic operations may also involve requests for "standard" atomic operations on requested data, such as relatively simple single-loop integer atoms, such as fetch and increment or compare and swap, whose throughput will be the same as regular memory read or write operations that do not involve atomic operations. For these operations, cache controller 215 can typically preserve cache lines in cache 210 by setting a danger bit (in hardware) so that the cache line cannot be read by another process during translation. Data is obtained from off-chip memory 275 or cache 210 and provided to built-in atomic operation unit 265 to perform the requested atomic operation. After the atomic operation, in addition to providing the obtained data to packet encoder 240 to encode outgoing packets for transmission on chiplet network 285, built-in atomic operation unit 265 also provides the obtained data to write merging unit 255, which writes the obtained data back to cache 210. After the obtained data is written to cache 210, memory danger clearing unit 260 clears any corresponding danger bits that were set.

[0050] Before transmitting a packet to the chiplet network 285, the packet encoder 240 or the network output queue 245 can use a credit checker and updater 290 to verify that sufficient credit is available for transmission. For example, a packet costing six credits for the destination device's virtual channel 0 can only be sent if at least six credits are available (e.g., because it contains six chips or will be transmitted in six transmission loops). If a packet is sent, the available credits will decrease by six. Available credits can be increased after receiving a credit return packet (or another packet containing credit return data) from the destination device. Therefore, by using credits, the sending and receiving devices can communicate without overflowing the receive buffer at the receiving device.

[0051] The PAU 270 implements high-performance (high throughput and low latency) programmable atomic operations (also known as "custom atomic operations"), comparable to the performance of built-in atomic operations. Instead of performing multiple memory accesses, in response to an atomic operation request specifying a programmable atomic operation and a memory address, the circuitry in the memory controller chiplet 205 passes the atomic operation request to the PAU 270 and sets a danger bit in a memory danger register corresponding to the memory address used in the atomic operation to ensure that no other operation (read, write, or atomic) is performed on that memory line. The danger bit is then cleared after the atomic operation is completed. The additional direct data path provided to the PAU 270 for performing programmable atomic operations allows for additional write operations without being limited by the bandwidth of the communication network and without increasing any congestion on the communication network.

[0052] The PAU 270 includes a multi-threaded processor, such as a RISC-VIS-based multi-threaded processor, having one or more processor cores and further having an extended instruction set for performing programmable atomic operations. When equipped with the extended instruction set for performing programmable atomic operations, the PAU 270 can be embodied as one or more hybrid-threaded processors. In some exemplary embodiments, the PAU 270 provides bucket-style polling instantaneous thread switching to maintain a high instruction-per-clock rate.

[0053] Programmable atomic operations can be executed by PAU 270, which relate to requests for programmable atomic operations on requested data. Users can prepare programming code to provide such programmable atomic operations. For example, programmable atomic operations can be relatively simple multi-loop operations, such as floating-point addition, or relatively complex multi-instruction operations, such as Bloom filter insert. Programmable atomic operations can be the same as or different from predetermined atomic operations, as long as they are defined by the user and not the system vendor. For these operations, cache controller 215 can preserve cache lines in cache 210 by setting a danger bit (in hardware), preventing the cache lines from being read by another process during translation. Data is obtained from cache 210 or off-chip memory 275 or 280 and provided to PAU 270 to execute the requested programmable atomic operation. After the atomic operation, PAU 270 provides the resulting data to network communication interface 225 to directly encode outgoing data packets containing the resulting data for transmission on chiplet network 285. Additionally, PAU270 provides the obtained data to cache controller 215, which then writes the obtained data to cache 210. After writing the obtained data to cache 210, cache controller 215 clears any corresponding dangerous bits that were set.

[0054] In the selected example, the approach taken for programmable atomic operations is to provide multiple general-purpose custom atomic request types, which can be sent from a source of origin, such as a processor or other system component, to the memory controller chiplet 205 via chiplet network 285. Cache controller 215 or off-die memory controller 220 recognizes the request as a custom atom and forwards the request to PAU 270. In a representative embodiment, PAU 270: (1) is a programmable processing element capable of efficiently performing user-defined atomic operations; (2) can perform load and store operations on memory, arithmetic and logic operations, and control flow decisions; and (3) utilizes a RISC-V ISA with a new set of dedicated instructions to facilitate interaction with such controllers 215, 220, thereby performing user-defined operations atomically. In a desirable example, the RISC-V ISA contains a complete instruction set supporting high-level language operators and data types. PAU 270 may utilize the RISC-V ISA, but will typically support a more limited instruction set and a limited register file size to reduce the die size of the unit when included within the memory controller chiplet 205.

[0055] As mentioned above, before writing the read data to cache 210, the memory danger clearing unit 260 clears the set danger bits of the reserved cache line. Therefore, when the write merging unit 255 receives a request and read data, the memory danger clearing unit 260 can transmit a reset or clear signal to cache 210 to reset the set memory danger bits of the reserved cache line. Furthermore, resetting this danger bit will also release pending read or write requests involving the specified (or reserved) cache line, thereby providing the pending read or write requests to the inbound request multiplexer for selection and processing.

[0056] Figure 3 This describes an example of routing between chiplets in chiplet layout 300 using a CPI network according to an embodiment. Chiplet layout 300 includes chiplets 310A, 310B, 310C, 310D, 310E, 310F, 310G, and 310H. Chipslets 310A to 310H are interconnected by a network including nodes 330A, 330B, 330C, 330D, 330E, 330F, 330G, and 330H. Each of chiplets 310A to 310H includes a hardware transceiver labeled 320A to 320H.

[0057] AIBs can be used to transfer CPI packets between chiplets 310. AIBs provide physical layer functionality. The physical layer uses source-synchronous data transfer with a forwarding clock to transmit and receive data. Packets are transferred across the AIB relative to the transmitted clock in SDR or DDR. AIBs support various channel widths. When operating in SDR mode, AIB channel widths are multiples of 20 bits (20, 40, 60, ...), and for DDR mode, AIB channel widths are multiples of 40 bits (40, 80, 120, ...). The AIB channel width includes both TX and RX signals. Channels can be configured to have a symmetrical number of TX and RX I / Os, or an asymmetrical number of transmitters and receivers (e.g., all transmitters or all receivers). A channel can act as an AIB master or slave depending on which chiplet provides the master clock.

[0058] The AIB adapter provides interfaces to the AIB link layer and to the AIB physical layer (PHY). The AIB adapter provides a data hierarchy register, a power-on reset sequencer, and a control signal shift register.

[0059] The AIB physical layer consists of AIB I / O units. AIB I / O units (implemented by hardware transceiver 320 in some exemplary embodiments) can be input-only, output-only, or bidirectional. An AIB channel consists of a set of AIB I / O units, and the number of units depends on the configuration of the AIB channel. A received signal on a chiplet is connected to a transmission signal on a pair of chipslets. In some exemplary embodiments, each column includes an AUX channel and data channels numbered 0 to N.

[0060] AIB channels are typically configured as half TX data and half RX data; all TX data; or all RX data plus associated clock and promiscuous control. In some exemplary embodiments, the number of TX data signals relative to the number of RX data signals is determined at design time and cannot be configured as part of system initialization.

[0061] The CPI packet protocol (point-to-point and routable) uses symmetrical receive and transmit I / O units within the AIB channel. The CPI streaming protocol allows for more flexible use of AIB I / O units. In some exemplary embodiments, the I / O units for the streaming mode of the AIB channel can be configured as all TX, all RX, or half TX and half RX.

[0062] Data packets are routed between chiplets 310 via network nodes 330. Node 330 can determine the next node 330 to forward a received data packet to based on one or more data fields of the data packet. For example, source or destination address, source or destination port, virtual channel, or any suitable combination thereof can be hashed to select consecutive network nodes or available network paths. This method of path selection can be used to balance network traffic.

[0063] Therefore, in Figure 3 The diagram illustrates the data path from chiplet 310A to chiplet 310D. Data packets are sent from hardware transceiver 320A to network node 330A; forwarded by network node 330A to network node 330C; forwarded by network node 330C to network node 330D; and delivered by network node 330D to hardware transceiver 320D of chiplet 310D.

[0064] Figure 3 The diagram also illustrates a second data path from chiplet 310A to chiplet 310G. Data packets are sent from hardware transceiver 320A to network node 330A; forwarded by network node 330A to network node 330B; forwarded by network node 330B to network node 330D; forwarded by network node 330D to network node 330C; forwarded by network node 330C to network node 330E; forwarded by network node 330E to network node 330F; forwarded by network node 330F to network node 330H; forwarded by network node 330H to network node 330G; and delivered by network node 330G to hardware transceiver 320G of chiplet 310G. (The last sentence appears to be a fragment and doesn't translate directly.) Figure 3 Visually, it is obvious that multiple paths through the network can be used for data transfer between any pair of chiplets.

[0065] The AIB I / O unit supports three clock modes: asynchronous (i.e., non-timed), SDR, and DDR. Non-timed mode is used for clocking and some control signals. SDR mode can use a dedicated SDR-only I / O unit or a dual-purpose SDR / DDR I / O unit.

[0066] The CPI packet protocol (point-to-point and routable) can use the AIB channel in either SDR or DDR operating mode. In some exemplary embodiments, the AIB channel increments by 80 I / O units (i.e., 40 TX and 40 RX) for SDR mode and by 40 I / O units for DDR mode.

[0067] The CPI streaming protocol can use AIB channels in either SDR or DDR operating mode. In some exemplary embodiments, the AIB channels are incremented by 40 I / O units for both modes (SDR and DDR).

[0068] A unique interface identifier is assigned to each AIB channel. This identifier is used during CPI reset and initialization to determine paired AIB channels across adjacent chiplets. In some exemplary embodiments, the interface identifier is a 20-bit value comprising a seven-bit chiplet identifier, a seven-bit column identifier, and a six-bit link identifier. The AIB physical layer uses an AIB out-of-band shift register to transmit the interface identifier. Bits 32 to 51 of the shift register are used to transmit the 20-bit interface identifier in both directions across the AIB interface.

[0069] In some exemplary embodiments, AIB channels are numbered in ascending order, starting with the AIB channel adjacent to the AUX channel. The AIB channel adjacent to the AUX channel is defined as AIB channel zero.

[0070] Figure 3 An example is provided to illustrate eight chiplets 310 connected by a network comprising eight nodes 330. More or fewer chiplets 310 and more or fewer nodes 330 can be included in the chiplet network, thereby allowing the creation of networks of chiplets of any size.

[0071] Figure 4 This is a block diagram of a data packet 400 comprising multiple micro-slices according to some embodiments of the present disclosure. The data packet 400 is divided into flow control units (micro-slices), each of which consists of 36 bits. The first micro-slice of the data packet 400 includes a control path field 405, a path field 410, a destination identifier (DID) field 415, a sequence continuation (SC) field 420, a length field 425, and a command field 430. The second micro-slice includes address fields 435 and 445, a transaction ID (TID) field 440, and a reservation (RSV) field 450. Each remaining micro-slice includes a credit return (CR) / RSV field (e.g., CR / RSV fields 455 and 465) and a data field (e.g., data fields 460 and 470).

[0072] The control path field 405 is a two-bit field that indicates whether the CR / RSV fields of later segments in the packet contain CR data, RSV data, or should be ignored, and whether the path field 410 should be used to control packet ordering. In some exemplary embodiments, a value of 0 or 1 in the control path field 405 indicates that CR / RSV fields 455 and 465 contain credit return data; a value of 2 or 3 in the control path field 405 indicates that CR / RSV fields 455 and 465 contain RSV data; a value of 0 indicates that path field 410 should be ignored; a value of 1 or 3 indicates that path field 410 is used to determine the path of packet 400; and a value of 2 indicates that a single path ordering will be used. In some exemplary embodiments, a one-bit field is used. Alternatively, the high-order bit of the control path field 405 may be considered as a one-bit field controlling whether CR / RSV fields 455 and 465 contain credit return data or RSV data.

[0073] The path field 410 is an eight-bit field. When the control path field 405 indicates that the path field 410 is used to determine the path for packet 400, all packets with the same value for the path field 410 are guaranteed to take the same path across the network. Therefore, the order of the packets will remain unchanged between the sender and receiver. If the control path field 405 indicates that a single path sorting will be used, the path for each packet is determined as if the path field 410 were set to zero. Therefore, all packets take the same path and the order will remain unchanged, regardless of the actual value of the path field 410 for each packet. If the control path field 405 indicates that the path field 410 will be ignored, the packets are routed without considering the value of the path field 410, and the packets can be received by the receiver in a different order than the order in which they were sent by the sender. However, this avoids network congestion and allows for higher throughput in the device.

[0074] The DID field 415 stores a twelve-bit DID. The DID uniquely identifies the destination (e.g., a destination chip) within the network. It ensures that all packets in a sequence with the SC field 420 are delivered in order. The length field 425 is a five-bit field indicating the number of chips comprising packet 400. The interpretation of the length field 425 can be non-linear. For example, a value of 0-22 can be interpreted as 0-22 chips in packet 400, and a value of 23-27 can be interpreted as 33-37 chips in packet 400 (i.e., 10 more than the indicated value). Other values ​​for the length field 425 can be vendor-defined, not protocol-defined.

[0075] Commands in data packet 400 are stored in command field 430, a seven-bit field. Commands can be write commands, read commands, predefined atomic operation commands, custom atomic operation commands, read responses, acknowledgment responses, or vendor-specific commands. Additionally, commands can indicate the virtual channel of data packet 400. For example, different commands can be used for different virtual channels, or bits 1, 2, 3, or 4 of the seven-bit command field 430 can be used to indicate the virtual channel, and the remaining bits can be used to indicate the command. The following table illustrates protocol- and command-based virtual channels according to some exemplary embodiments.

[0076] Virtual Channel CPI Agreement AXI Protocol 0 Read / Write Requests Write request 1 Read / Write Response Write response 2 Unused Read request 3 Unused Read response 4 Priority read / write requests Priority write requests 5 Priority read / write response Priority write response 6 Unused Priority read request 7 Unused Priority read response

[0077] The address of the command can be indicated in the path field 410, address fields 435 and 445, or any suitable combination thereof. For example, the high 31 bits of an 8-byte aligned 34-bit address can be indicated by concatenating address field 435, path field 410, and address field 445 sequentially (most significant bits first). The TID field 440 is used to match the response to the request. For example, if the first packet 400 is a read request identifying the memory location to be read, then the response packet 400 containing the read data will contain the same value in the TID field 440.

[0078] Memory access commands can identify the number of bytes to be written or accessed, the memory space to be accessed (e.g., off-die memory 275 or instruction memory for custom atomic operations), or any suitable combination thereof. In some exemplary embodiments, the command may instruct additional bits of a later microchip to identify the command. For example, a multi-byte command can be sent using a vendor-specific command in the seven-bit command field 430 and the multi-byte command can be stored using a portion or all of the data field 460. Thus, for certain values ​​of the command field 430, package 400 contains only one header microchip (e.g., ...). Figure 4 The first header micropie shown contains fields 405 to 430. For other values ​​of command field 430, package 400 contains a predetermined additional number of header micropieces or a predetermined total number of header micropieces.

[0079] If CR is enabled, two bits of CR / RSV field 455 and 465 identify whether the credit return is for virtual channel 0, 1, 2 or 3, and the other two bits of CR / RSV field 455 and 465 indicate whether the number of credits to be returned is 0, 1, 2 or 3.

[0080] Figure 5This is a block diagram of a credit return data packet 500 according to some embodiments of the present disclosure. The data packet 500 includes a single 36-bit micropie. The micropie includes four credit return fields 505, 510, 515, and 520, a length field 530, and reserved fields 525 and 535. The length field 530 is set to zero to indicate that no additional micropie constitutes the data packet 500. Reserved fields 525 and 535 are unused and should be set to zero.

[0081] Each of the credit return fields 505 to 520 is a five-bit field. If credit is being returned to a low virtual channel, the first bit of the credit return fields 505 to 520 is set to zero. If credit is being returned to a high virtual channel, the first bit of the credit return fields 505 to 520 is set to 1. The remaining four bits of each credit return field 505 to 520 indicate the number of credits being returned (0 to 15). Therefore, the CR0 credit return field 520 returns 0 to 15 credits to virtual channel 0 or 4; the CR1 credit return field 515 returns 0 to 15 credits to virtual channel 1 or 5; the CR2 credit return field 510 returns 0 to 15 credits to virtual channel 2 or 6; and the CR3 credit return field 505 returns 0 to 15 credits to virtual channel 3 or 7.

[0082] Therefore, early credit return for credit-based flow control can be implemented by using the CR / RSV fields 455 and 465 of packet 400 when sending other packets (such as acknowledgment packets) or by using packet 500.

[0083] Based on the packet size and the number of credits available at the destination, the source determines whether there are enough credits to deliver the entire packet. In a chip-based flow control credit scheme, the number of credits used to deliver the packet is the number of chips in the packet. In a transfer cycle-based flow control scheme, the number of credits used to deliver the packet is the number of transfer cycles used to deliver the packet. This can be determined using the following equation:

[0084] )

[0085] In this equation, the cycle is the number of transfer cycles required to transmit a packet, the upper bound is a function that rounds the fractional value to the next integer, f0 is a 0-based offset of the position of the first chip in the wide bus, and the width is the bus width measured in chips. Therefore, when width = 1, f0 is always 0, and the number of cycles is simply the size of the packet in the chip. When width = 2 and f0 is 0, the number of cycles is half the size of the packet, rounded. When width = 2 and f0 is 1, packets with an even number of chips require additional transfer cycles to transmit (since only one chip is transmitted on the first transfer cycle and one chip is transmitted on the last transfer cycle).

[0086] Figure 6 This is a block diagram 600 illustrating a received chip stream and a virtual channel queue of received chips according to some embodiments of the present disclosure. The interface for receiving the chip stream has three channel widths. Therefore, one chip is received simultaneously (or at least simultaneously) on each of channels 610A, 610B, and 610C. The received chips are sequentially shown in rows 620A, 620B, 620C, 620D, and 620E. After reception, the chips are stored in virtual channel 0 channel queues 630A, 630B, and 630C or virtual channel 1 channel queues 650A, 650B, and 650C. Virtual channel 0 channel queues 630A to 630C store chips in rows 640A, 640B, 640C, and 640D. Virtual channel 1 channel queues 650A to 650C store chips in rows 660A, 660B, and 660C. Each channel 610A to 610C physically receives data on a portion of the wide bus. Virtual channels are not physical and do not directly correspond to channels 610A to 610C. For example, a packet destined for virtual channel 0 may be a write packet entering the write queue for processing. Write packets can span multiple physical channels and begin and end on any physical channel.

[0087] The received fragment stream consists of three packets: P1, P2, and P3. P1 is destined for virtual channel 0 and contains four fragments. P2 is destined for virtual channel 1 and contains five fragments. P3 is destined for virtual channel 0 and contains three fragments. The virtual channel for each packet can be determined based on the destination identifier 415 in the header.

[0088] At the end of each clock cycle, based on the destination of the microchips in a row, an entire row of received data (e.g., one of rows 620A to 620E) is copied, either entirely or partially, to zero or more virtual channel queues. For example, row 620A is partially copied to row 640A because two microchips in row 620A go to virtual channel 0. Data in idle microchips of row 620A may be copied or the data may not be written to row 640A. As another example, row 620B is partially copied to both row 640B and row 660A because row 620B contains at least one microchip for both virtual channel 0 and virtual channel 1.

[0089] A credit represents a single entry in each of the three channel queues for a specific virtual channel. This allows for the use of a single read and write queue pointer across the three channel queues for each virtual channel. A side effect of this queue organization is that if any chip is written to the set of channel queues associated with the virtual channel, the entry in each of the three channel queues is considered to be written for said virtual channel (e.g., using a single write pointer across the three channel queues used for the virtual channel). Packets written to the same virtual channel in the associated set of virtual channel queues may have one or more free chips between packets within the queue. The number of free entries depends on the end and start channel positions of the two packets.

[0090] Instead of copying each chip individually to a single-channel queue, the timing of the circuit design is easier to meet. For example, relative to row 620A, copying the first chip of P1 to the first entry of the single-channel queue increments the destination pointer, and then copying the second chip of P1 to the second entry of the single-channel queue. Each of these operations needs to be completed before the next clock cycle of the bus on which the chip stream is received, using a higher internal clock rate.

[0091] Figure 7 This is a flowchart illustrating the operation of a method 700 performed by circuitry when updating credit data according to some embodiments of the present disclosure. Method 700 includes operations 710, 720, and 730. By way of example and not limitation, method 700 is described as using... Figure 4 The data packets are executed by the devices shown in Figures 1 to 3.

[0092] In operation 710, the transmission device (e.g., Figure 3 The small chip 310A is implemented. Figure 2 The memory controller chip 205 (or communicating with the memory controller chip) identifies the credits to be returned for multiple virtual channels. For example, for virtual channels 0, 1, and 2, the credits can be prepared to be returned to... Figure 3 The small chip 310D.

[0093] In operation 720, the transmission device selects one of a plurality of virtual channels to which the credits will be returned. For example, the virtual channel with the most credits to be returned can be selected, the next virtual channel in the circular queue can be selected (e.g., such that virtual channel 0 is selected first, then virtual channel 1, then virtual channel 2, then virtual channel 3, and so on, until all virtual channels have been selected and virtual channel 0 is selected again), or another selection algorithm can be used.

[0094] In operation 730, on a transmission cycle, based on the selection of a virtual channel, the transmission device sets the credit return field of the first microchip using the identified credit of the selected virtual channel and clears the credit return field of the second microchip. For example, using a wide bus that transmits two microchips per transmission cycle, the CR / RSV field 455 of the first microchip is set to indicate the return credit on the selected virtual channel, and the CR / RSV 465 of the second microchip transmitted on the same transmission cycle is cleared to at least have no return credit.

[0095] The operation of method 700 can be performed by the credit checker and updater 290 of the transmission device. By using method 700 in a transmission device coupled to a network, the network increases packet throughput by using a wide bus that transmits multiple chips in each transmission cycle without modifying the chip format, but the transmission device ensures that only one chip returns credit in each transmission cycle. Therefore, throughput is enhanced compared to systems using a single-chip wide bus. Circuit complexity is reduced compared to systems using a multi-chip wide bus that supports returning credit on multiple chips in a transmission cycle, thereby reducing cost and increasing manufacturing yield.

[0096] Figure 8 This is a flowchart illustrating the operation of method 800 performed by circuitry when updating credit data according to some embodiments of the present disclosure. Method 800 includes operations 810, 820, and 830. By way of example and not limitation, method 800 is described as using... Figure 4 The data packets are executed by the devices shown in Figures 1 to 3.

[0097] In operation 810, the receiving device (e.g., Figure 3 The small chip 310D is implemented. Figure 2 The receiving device (or communicating with the memory controller chip 205) receives multiple microchips, including a first microchip and a second microchip, on a transfer cycle. For example, the receiving device can receive multiple microchips transmitted on a CPI network via a transmitting device performing method 700. The transfer cycle is a common duration defined by one or more clock cycles during which all the multiple microchips are received. Microchips received during the common duration are received simultaneously, even if they are not actually received simultaneously.

[0098] In operation 820, the receiving device updates the credit data using the first credit return field of the first microchip. For example, the CR / RSV field 455 may contain credit return data for the virtual channel, and the receiving device uses the credit return data to update the number of available credits for the virtual channel.

[0099] In operation 830, the receiving device avoids updating credit data using the second credit return value of the second microchip. For example, the receiving device may ignore CR / RSV field 465 transmitted on the same transmission cycle as CR / RSV field 455. The use of the first credit return field and the ignoring of the second credit return field may be based on the position of the first microchip on the wide bus or the values ​​of the first and second return values. For example, the credit return data of the microchip in the first channel of the wide bus may be checked, while microchips in other channels may be ignored. As another example, the credit return fields of all microchips received on the transmission cycle may be logically ORed together, such that if the transmitting device sets the credit return fields of all other microchips to zero, the credit return field of any single microchip will remain unchanged. In the latter example, the transmitting device may provide credit return data on any channel, as long as the credit return data is provided on only one channel.

[0100] In some exemplary embodiments, before or after updating the credit data in operation 820, the credit return field of micro-pieces other than the first micro-piece is verified to be zero (or otherwise does not contain credit return data). In these exemplary embodiments, the presence of credit return data in multiple micro-pieces received on a single transmission cycle can trigger an error handling circuitry system to improve the error condition of the receiving device, send an error response to the transmitting device, or both.

[0101] Various data formats can be used for the credit return data of the first micro-piece. For example, Figure 4 The CR / RSV field 455 contains credit return data for a single virtual channel within the microchip, but Figure 5 The CR3, CR2, CR1, and CR0 fields of the credit return data packet 500 contain credit return data for up to four virtual channels in the microchip. Processing the first credit return field in operation 820 can be a subset of processing all credit return fields for the first microchip. Therefore, support for the credit return data packet 500 is maintained as long as only one microchip receives credit return data per transmission cycle.

[0102] The operation of method 800 can be performed by the credit checker and updater 290 of the transmitting device. By using method 800 in a receiving device coupled to a network, the network increases packet throughput by using a wide bus that transmits multiple chips in each transmission cycle without modifying the chip format, but the receiving device processes the credit return credit of a single chip in each transmission cycle. Therefore, throughput is enhanced compared to systems using a single-chip wide bus. Circuit complexity is reduced compared to systems using a multi-chip wide bus that supports credit return on multiple chips in a transmission cycle, thereby reducing cost and increasing manufacturing yield.

[0103] Figure 9The block diagram illustrating example machine 900 illustrates any or more of the techniques (e.g., methods) discussed herein that can be implemented using, in, or through said machine. As described herein, an example may include, or be operated by, logic or components or mechanisms in machine 900. A circuit system (e.g., a processing circuit system) is a collection of circuits implemented in the tangible entity of machine 900, which includes hardware (e.g., simple circuits, gates, logic, etc.). The membership of a circuit system can be flexible over time. A circuit system includes members that can perform specific tasks individually or in combination during operation. In an example, the hardware of a circuit system can be designed in an immutable manner to perform a specific operation (e.g., hardwiring). In an example, the hardware of a circuit system may include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.) that include machine-readable media that are physically modified (e.g., invariant magnetic properties of concentrated particles, electrically movable placement, etc.) to encode instructions for a specific operation. When connecting physical components, the fundamental electrical properties of the hardware components are altered, for example, from an insulator to a conductor, or vice versa. Instructions enable embedded hardware (e.g., an execution unit or loading mechanism) to hardware-create components of a circuit system via variable connections to perform specific operations when in operation. Thus, in an example, a machine-readable medium element is part of the circuit system or communicatively coupled to other components of the circuit system during device operation. In an example, any one of the physical components can be used in more than one part of more than one circuit system. For example, under operation, an execution unit may be used at one point in time for a first circuit in a first circuit system and reused by a second circuit in the first circuit system, or reused at a different time by a third circuit in a second circuit system. Further examples of these components of machine 900 are given below.

[0104] In alternative embodiments, machine 900 may operate as a standalone device or be connected (e.g., networked) to other machines. In a networked deployment, machine 900 may operate as a server machine, a client machine, or both in a server-client network environment. In an example, machine 900 may act as a peer-to-peer (P2P) (or other distributed) network environment. Machine 900 may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, network appliance, network router, switch, or bridge, or any machine capable of executing (sequentially or otherwise) instructions specifying actions to be taken by said machine. Furthermore, while only a single machine is described, the term "machine" should also be considered as encompassing any collection of machines that individually or jointly execute one or more sets of instructions to perform any one or more of the methodologies discussed herein, such as cloud computing, Software as a Service (SaaS), and other computer cluster configurations.

[0105] Machine (e.g., computer system) 900 may include a hardware processor 902 (e.g., a central processing unit (CPU), graphics processing unit (GPU), hardware processor core, or any combination thereof), main memory 904, static memory (e.g., memory or storage device for firmware, microcode, basic input / output (BIOS), unified extensible firmware interface (UEFI), etc.) 906, and mass storage device 908 (e.g., hard disk drive, tape drive, flash memory, or other block device), some or all of which may communicate with each other via interconnect (e.g., bus) 930. Machine 900 may further include a display unit 910, an alphanumeric input device 912 (e.g., keyboard), and a user interface (UI) navigation device 914 (e.g., mouse). In an example, the display unit 910, input device 912, and UI navigation device 914 may be a touchscreen display. Machine 900 may additionally include a signal generating device 918 (e.g., a speaker), a network interface device 920, and one or more sensors 916, such as a Global Positioning System (GPS) sensor, a compass, an accelerometer, or other sensors. Machine 900 may include an output controller 928, for example, serial (e.g., Universal Serial Bus (USB), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection, to communicate with or control one or more peripheral devices (e.g., a printer, a card reader, etc.).

[0106] The registers of processor 902, main memory 904, static memory 906, or mass storage device 908 may be or contain a machine-readable medium 922 on which one or more sets of data structures or instructions 924 (e.g., software) are stored, said sets of data structures or instructions embodying, or being utilized by, any one or more of the techniques or functions described herein. Instructions 924 may also reside wholly or at least partially in any of the registers of processor 902, main memory 904, static memory 906, or mass storage device 908 during execution by machine 900. In an example, one or any combination of hardware processor 902, main memory 904, static memory 906, or mass storage device 908 may constitute machine-readable medium 922. Although machine-readable medium 922 is described as a single medium, the term "machine-readable medium" may include a single medium or multiple media (e.g., a centralized or distributed database, or associated cache and server) configured to store one or more instructions 924.

[0107] The term "machine-readable medium" can include any medium capable of storing, encoding, or transmitting instructions for execution by machine 900 and causing machine 900 to perform any or more of the technologies disclosed herein, or any medium capable of storing, encoding, or transmitting data structures used by or associated with such instructions. Examples of non-limiting machine-readable media may include solid-state memory, optical media, magnetic media, and signals (e.g., radio frequency signals, other photon-based signals, sound signals, etc.). In examples, non-transitory machine-readable media include machine-readable media having a plurality of particles having constant (e.g., rest) mass and therefore being composed of matter. Therefore, a non-transitory machine-readable medium is a machine-readable medium that does not contain transiently propagating signals. Specific examples of non-transitory machine-readable media may include: non-volatile memory, such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable hard disks; magneto-optical disks; and compact optical disc read-only memory (CD-ROM) and digital multifunction optical disc read-only memory (DVD-ROM).

[0108] In an example, information stored or otherwise provided on a machine-readable medium 922 may represent instructions 924, such as instructions 924 themselves or a format from which instructions 924 can be derived. This format from which instructions 924 can be derived may include source code, encoded instructions (e.g., in compressed or encrypted form), encapsulated instructions (e.g., split into multiple encapsulations), etc. The information representing instructions 924 in the machine-readable medium 922 may be processed by a processing circuitry system into the instructions to perform any of the operations discussed herein. For example, deriving instructions 924 from information (e.g., through processing circuitry system processing) may include: compiling (e.g., from source code, object code, etc.), interpreting, loading, organizing (e.g., dynamically or statically linking), encoding, decoding, encrypting, decrypting, encapsulating, decapsulating, or otherwise manipulating the information into instructions 924.

[0109] In an example, the derivation of instruction 924 may involve (e.g., by processing a circuit system) assembling, compiling, or decompiling information to create instruction 924 from some intermediate or preprocessed format provided by machine-readable medium 922. When information is provided in multiple parts, the information may be combined, decapsulated, and modified to create instruction 924. For example, information may be contained in multiple compressed source packages (or object code, or binary executable code, etc.) on one or more remote servers. The source packages may be encrypted when transmitted over a network and, if necessary, decrypted, decompressed, assembled (e.g., linked), and compiled or decompiled at the local machine (e.g., into a stand-alone executable library, etc.) and executed by the local machine.

[0110] Instruction 924 may further utilize any of a plurality of transmission protocols (e.g., Frame Relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.) to transmit or receive over communication network 926 via network interface device 920 using a transmission medium. Example communication networks may include local area networks (LANs), wide area networks (WANs), packet data networks (e.g., the Internet), mobile phone networks (e.g., cellular networks), simple old-style telephone (POTS) networks, and wireless data networks (e.g., IEEE 802.11 series standards known as Wi-Fi®, IEEE 802.16 series standards known as WiMax®), IEEE 802.15.4 series standards, peer-to-peer (P2P) networks, etc. In an example, network interface device 920 may include one or more physical jacks (e.g., Ethernet, coaxial, or telephone jacks) or one or more antennas for connection to communication network 926. In an example, network interface device 920 may include multiple antennas to wirelessly communicate using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) technologies. The term "transmission medium" should be considered as any intangible medium capable of storing, encoding, or carrying instructions for execution by machine 900, and includes digital or analog communication signals or other intangible media used to facilitate communication of this software. The transmission medium is a machine-readable medium.

[0111] In the foregoing description, some exemplary embodiments of this disclosure have been described. It will be apparent that various modifications may be made to this disclosure without departing from the broader spirit and scope of the examples set forth in the appended claims. Therefore, the description and drawings should be viewed in an illustrative rather than restrictive sense. The following is a non-exhaustive list of examples of embodiments of this disclosure.

[0112] Example 1 is a system comprising: a memory device; a memory controller coupled to the memory device; and logic configured to perform operations including: simultaneously receiving a plurality of flow control units (microchips) comprising a first microchip and a second microchip, the first microchip having a first credit return field and the second microchip having a second credit return field; updating credit data using credit indicated in only one of the first credit return field of the first microchip and the second credit return field of the second microchip.

[0113] In Example 2, the subject matter according to Example 1 includes, wherein the operation further includes: determining, based on a first channel of the first microchip and a second channel of the second microchip, to update the credit data using the first credit return field and not to update the credit data using the second credit return field.

[0114] In Example 3, the subject matter according to Examples 1 to 2 includes, wherein the operation further includes: verifying that the second credit return field has a zero value before updating the credit data using the first credit return field.

[0115] In Example 4, the subject matter according to Examples 1 to 3 includes wherein: the plurality of micro-pieces includes a third micro-piece having a third credit return field; the operation further includes determining which credit return field of the plurality of micro-pieces contains the indicated credit; and the credit update of the credit data using only one of the credit indicated in the first credit return field of the first micro-piece and the second credit return field of the second micro-piece in response to determining that only the second credit return field of the second micro-piece indicates the credit, and includes updating the credit data using only the credit indicated by the second credit return field of the second micro-piece.

[0116] In Example 5, the object described in Examples 1 to 4 includes, wherein the operation further includes: combining the first credit return field of the first micro-piece with the second credit return field of the second micro-piece using a bitwise OR operation.

[0117] In Example 6, the object according to Examples 1 to 5 includes the operation further comprising: identifying a virtual channel from the first microchip; and the update of the credit data includes updating the credit data of the identified virtual channel.

[0118] In Example 7, the object described in Examples 1 to 6 includes, wherein the operation further includes: identifying a first virtual channel, a first number of credits, a second virtual channel, and a second number of credits from the first microchip; and wherein the updating of the credit data includes updating the credit data of the first virtual channel using the first number of credits and updating the credit data of the second virtual channel using the second number of credits.

[0119] In Example 8, the subject matter according to Example 7 includes: the identification of the first virtual channel includes determining, from a 1-bit first field of the first microchip, whether the first number of credits is used for the first virtual channel or the third virtual channel; and the identification of the second virtual channel includes determining, from a 1-bit second field of the first microchip, whether the second number of credits is used for the second virtual channel or the fourth virtual channel.

[0120] In Example 9, the subject matter according to Examples 1 to 8 includes, wherein the receipt of the plurality of microchips includes receiving the plurality of microchips via a chiplet protocol interface (CPI).

[0121] Example 10 is a method comprising: simultaneously receiving a plurality of flow control units (micro-chips) including a first micro-chip and a second micro-chip, during a common duration defined by one or more clock cycles, the first micro-chip having a first credit return field and the second micro-chip having a second credit return field; and using credit update credit data indicated in only one of the first credit return field of the first micro-chip and the second credit return field of the second micro-chip.

[0122] In Example 11, the subject matter described in Example 10 includes a first channel of the first microchip and a second channel of the second microchip, and it is determined that the credit data is updated using the first credit return field and not updated using the second credit return field.

[0123] In Example 12, the subject matter as described in Examples 10 to 11 includes verifying that the second credit return field has a zero value before updating the credit data using the first credit return field.

[0124] In Example 13, the subject matter according to Examples 10 to 12 includes wherein: the plurality of micro-pieces includes a third micro-piece having a third credit return field; the method further includes determining which credit return field of the plurality of micro-pieces contains the indicated credit; and updating the credit data using the credit indicated in only one of the first credit return field of the first micro-piece and the second credit return field of the second micro-piece in response to determining that only the second credit return field of the second micro-piece indicates the credit, and includes updating the credit data using only the credit indicated by the second credit return field of the second micro-piece.

[0125] In Example 14, the subject matter according to Examples 10 to 13 includes combining the first credit return field of the first micro-piece with the second credit return field of the second micro-piece using a bitwise OR operation.

[0126] In Example 15, the subject matter according to Examples 10 to 14 includes a virtual channel identified from the first microchip; and the update of the credit data includes updating the credit data of the identified virtual channel.

[0127] In Example 16, the subject matter according to Examples 10 to 15 includes identifying a first virtual channel, a first number of credits, a second virtual channel, and a second number of credits from the first microchip; and wherein the updating of the credit data includes updating the credit data of the first virtual channel using the first number of credits and updating the credit data of the second virtual channel using the second number of credits.

[0128] In Example 17, the subject matter according to Example 16 includes: the identification of the first virtual channel includes determining, from a 1-bit first field of the first microchip, whether the first number of credits is used for the first virtual channel or the third virtual channel; and the identification of the second virtual channel includes determining, from a 1-bit second field of the first microchip, whether the second number of credits is used for the second virtual channel or the fourth virtual channel.

[0129] In Example 18, the subject matter according to Examples 10 to 17 includes, wherein the receipt of the plurality of microchips includes receiving the plurality of microchips via a chiplet protocol interface (CPI).

[0130] Example 19 is a non-transitory machine-readable medium storing instructions that, when executed by a system, cause the system to perform operations including: simultaneously receiving a plurality of flow control units (microchips) comprising a first microchip and a second microchip, the first microchip having a first credit return field and the second microchip having a second credit return field, during a common duration defined by one or more clock cycles; and using credit update credit data indicated in only one of the first credit return field of the first microchip and the second credit return field of the second microchip.

[0131] In Example 20, the subject matter according to Example 19 includes the operation further comprising: determining, based on a first channel of the first microchip and a second channel of the second microchip, to update the credit data using the first credit return field and not to update the credit data using the second credit return field.

[0132] Example 21 is at least one machine-readable medium containing instructions that, when executed by a processing circuit system, cause the processing circuit system to perform operations to implement any one of Examples 1 to 20.

[0133] Example 22 is an apparatus that includes means for implementing any one of Examples 1 to 20.

[0134] Example 23 is a system for implementing any one of Examples 1 through 20.

[0135] Example 24 is a method for implementing any one of Examples 1 through 20.

Claims

1. A memory system comprising: Memory devices; A memory controller coupled to the memory device; and Logic circuits configured to perform operations including the following: Simultaneously receive multiple micro-pieces including a first micro-piece and a second micro-piece, wherein the first micro-piece has a first credit return field and the second micro-piece has a second credit return field; Based on the first channel of the first microchip and the second channel of the second microchip, it is determined that the credit data is updated using the credit indicated in the first credit return field instead of updating the credit data using the second credit return field; and Based on the determination, the credit data is updated using the credit indicated in the first credit return field and not using the second credit return field.

2. The memory system of claim 1, wherein updating the credit data using the credit indicated in the first credit return field is further based on verification that the second credit return field has a zero value.

3. The memory system according to claim 1, wherein: The plurality of micro-pieces includes a third micro-piece with a third credit return field; The operation further includes determining which credit return field of the plurality of micro-pieces contains the indicated credit; and In response to determining that only the second credit return field of the second micro-pie indicates credit, the credit data is updated using only the credit indicated by the second credit return field of the second micro-pie.

4. The memory system of claim 1, wherein the operation further comprises: The first credit return field of the first microchip is combined with the second credit return field of the second microchip using a bitwise OR operation.

5. The memory system of claim 1, wherein the operation further comprises: Identify the virtual channel from the first microchip; and The update of the credit data includes updating the credit data of the identified virtual channel.

6. The memory system of claim 1, wherein the operation further comprises: Identify a first virtual channel, a first number of credits, a second virtual channel, and a second number of credits from the first microchip; and The update of the credit data includes updating the credit data of the first virtual channel using the first number of credits and updating the credit data of the second virtual channel using the second number of credits.

7. The memory system according to claim 6, wherein: The identification of the first virtual channel includes determining, from a 1-bit first field of the first microchip, whether the first number of credits is used for the first virtual channel or the third virtual channel; and The identification of the second virtual channel includes determining, from the 1-bit second field of the first microchip, whether the second number of credits is used for the second virtual channel or the fourth virtual channel.

8. The memory system of claim 1, wherein receiving the plurality of microchips includes receiving the plurality of microchips via a chiplet protocol interface (CPI).

9. A method for a memory system, the method comprising: During a common duration defined by one or more clock cycles, multiple micro-chips, including a first micro-chip and a second micro-chip, are simultaneously received, the first micro-chip having a first credit return field and the second micro-chip having a second credit return field; Based on the first channel of the first microchip and the second channel of the second microchip, it is determined that the credit data is updated using the credit indicated in the first credit return field instead of updating the credit data using the second credit return field; and Based on the determination, the credit data is updated using the credit indicated in the first credit return field and not using the second credit return field.

10. The method of claim 9, wherein updating the credit data using the credit indicated in the first credit return field is further based on verification that the second credit return field has a zero value.

11. The method according to claim 9, wherein: The plurality of micro-pieces includes a third micro-piece with a third credit return field; The method further includes determining which credit return field of the plurality of micro-pieces contains the indicated credit; and In response to determining that only the second credit return field of the second micro-pie indicates credit, the credit data is updated using only the credit indicated by the second credit return field of the second micro-pie.

12. The method of claim 9, further comprising: The first credit return field of the first microchip is combined with the second credit return field of the second microchip using a bitwise OR operation.

13. The method of claim 9, further comprising: Identify the virtual channel from the first microchip; and The update of the credit data includes updating the credit data of the identified virtual channel.

14. The method of claim 9, further comprising: Identify a first virtual channel, a first number of credits, a second virtual channel, and a second number of credits from the first microchip; and The update of the credit data includes updating the credit data of the first virtual channel using the first number of credits and updating the credit data of the second virtual channel using the second number of credits.

15. The method of claim 14, wherein: The identification of the first virtual channel includes determining, from a 1-bit first field of the first microchip, whether the first number of credits is used for the first virtual channel or the third virtual channel; and The identification of the second virtual channel includes determining, from the 1-bit second field of the first microchip, whether the second number of credits is used for the second virtual channel or the fourth virtual channel.

16. The method of claim 9, wherein receiving the plurality of microchips includes receiving the plurality of microchips via a chiplet protocol interface (CPI).

17. A non-transitory machine-readable medium storing instructions, which, when executed by a memory system, cause the memory system to perform operations including: During a common duration defined by one or more clock cycles, multiple micro-chips, including a first micro-chip and a second micro-chip, are simultaneously received, the first micro-chip having a first credit return field and the second micro-chip having a second credit return field; Based on the first channel of the first microchip and the second channel of the second microchip, it is determined that the credit data is updated using the credit indicated in the first credit return field instead of updating the credit data using the second credit return field; and Based on the determination, the credit data is updated using the credit indicated in the first credit return field and not using the second credit return field.

18. The non-transitory machine-readable medium of claim 17, wherein updating the credit data using the credit indicated in the first credit return field is further based on verification that the second credit return field has a zero value.

19. The non-transitory machine-readable medium of claim 17, wherein the operation further comprises: The first credit return field of the first microchip is combined with the second credit return field of the second microchip using a bitwise OR operation.

20. The non-transitory machine-readable medium according to claim 17, wherein: The operation further includes determining which credit return field of the plurality of micro-pieces contains the indicated credit; and In response to determining that only the second credit return field of the second micro-pie indicates the credit, the credit data is updated using only the credit indicated by the second credit return field of the second micro-pie.

21. The non-transitory machine-readable medium of claim 17, wherein the operation further comprises: Identify the virtual channel from the first microchip; and The update of the credit data includes updating the credit data of the identified virtual channel.

22. The non-transitory machine-readable medium of claim 17, wherein the operation further comprises: Identify a first virtual channel, a first number of credits, a second virtual channel, and a second number of credits from the first microchip; and The update of the credit data includes updating the credit data of the first virtual channel using the first number of credits and updating the credit data of the second virtual channel using the second number of credits.

Images