Systems and methods for flow control using multiple flow control unit interfaces

Abstract

Embodiments of the present disclosure relate to systems and methods for flow control using a multi-flit interface. A credit return field is used in a credit-based flow control system to indicate that one or more credits are being returned from a receiving device to a sending device. Based on the number of credits available, the sending device determines whether the sending device or to wait until more credits are returned. The amount of buffer space used by a receiver to store a packet is determined by the number of transfer cycles used to receive the packet, rather than the number of flits comprising the packet. This is accomplished by making the buffer as wide as the bus. The receiver returns credits to the transmitter based on the number of buffer rows used to store the received packet, rather than the number of flits comprising the packet.

Description

[0001] Priority application

[0002] This application claims priority to U.S. Application No. 17 / 007,468, filed on August 31, 2020, the entire contents of which are incorporated herein by reference.

[0003] Statement regarding government support

[0004] This invention was carried out with the support of the United States government under DARPA Agreement HR00111830003. The United States government has certain rights to this invention. Technical Field

[0005] Embodiments of this disclosure generally relate to network protocols and, more specifically, to virtual channels networking between a simplified flow control credit agency using packets and a multi-flow control unit (“flit”) interface. Background Technology

[0006] Packets are divided into fragments for transmission by the transmitting device via the bus. Before transmitting a packet to the receiving device, the transmitting device verifies that there are enough credits available for the receiving device to accept all fragments of the packet. In many conventional systems, one credit is used for each fragment. The first fragment of the packet is a header fragment containing data for routing through the network. The header fragment is followed by zero or more body fragments.

[0007] Chiplets are an emerging technology for integrating various processing functionalities. Generally, a chiplet system consists of discrete chips (e.g., integrated circuits (ICs) on different substrates or dies) integrated and packaged together on an interposer. This arrangement differs from a single chip (e.g., an IC) containing different device blocks (e.g., intellectual property (IP) blocks) on a single substrate (e.g., a single die) or carefully packaged devices integrated on a board. Generally, chiplets offer better performance (e.g., lower power consumption, reduced latency) than carefully packaged devices, and offer greater manufacturing benefits than single-die chips. These manufacturing benefits can include higher yields or reduced development costs and time.

[0008] Chiplet systems typically consist of one or more application chiplets and support chiplets. Here, the distinction between application and support chiplets is used only to refer to possible design schemes of chiplet systems. Thus, for example, a synthetic vision chiplet system may include application chiplets for generating synthetic vision output and support chiplets, such as memory controller chiplets, sensor interface chiplets, or communication chiplets. In typical use cases, synthetic vision designers may design the application chiplet and obtain the support chiplets from other sources. Therefore, design costs (e.g., in terms of time or complexity) are reduced because the functionality embodied in the design and manufacturing of the support chiplet is avoided. Chiplets also support the tight integration of IP blocks that might otherwise be difficult, such as the tight integration of IP blocks with different feature sizes. Therefore, devices designed with larger feature sizes during previous generation manufacturing, or those where the feature sizes are optimized for power, speed, or heat (as might be the case with sensors), can be integrated with devices of different feature sizes, which is easier than attempting to do so on a single die. Furthermore, by reducing the overall die size, chiplet yields tend to be higher than those of more complex single-die devices.

[0009] In chiplet systems, communication between chiplets can include packet-based communication across networks or direct connections. In conventional systems, the efficiency of transmission and storage in this packet-based communication can affect the overall performance of the chiplet system. Attached Figure Description

[0010] This disclosure will be more fully understood from the detailed description given below and from the accompanying drawings of various embodiments thereof. However, the drawings should not be construed as limiting this disclosure to the particular embodiments, but are for explanation and understanding only.

[0011] Figure 1A and 1B An example of a chiplet system according to an embodiment is described.

[0012] Figure 2 This describes the components of an example of a memory controller chiplet according to an embodiment.

[0013] Figure 3 This describes an example of routing between chiplets in a chiplet layout using a chiplet protocol interface (CPI) network, according to an embodiment.

[0014] Figure 4 This is a block diagram of data groups including multiple flits according to some embodiments of the present disclosure.

[0015] Figure 5 This is a block diagram illustrating a received flit stream and a virtual channel queue of received flits according to some embodiments of the present disclosure.

[0016] Figure 6 This is a flowchart illustrating the operation of a method performed by a circuit when sending multiple flits in a transfer cycle according to some embodiments of the present disclosure.

[0017] Figure 7 This is a flowchart illustrating the operation of a method performed by a circuit when receiving multiple flits in a transfer cycle according to some embodiments of the present disclosure.

[0018] Figure 8 This is a composite flowchart illustrating the operation of a composite method performed by a first circuit and a second circuit according to some embodiments of the present disclosure, wherein the first method is performed by the first circuit when sending one or more packets and the second method is performed by the second circuit when receiving one or more packets.

[0019] Figure 9 This is a block diagram of an example computer system in which embodiments of the present disclosure may be operated. Detailed Implementation

[0020] This disclosure relates to systems and methods for flow control using multi-flit interfaces. A credit return field in a credit-based flow control system is used to indicate that one or more credits are returning from a receiving device to a transmitting device. Based on the number of available credits, the transmitting device determines whether to send a packet to the receiving device or wait for more credits to return. The credit return field may be a field in a data packet carrying other data (e.g., when using two-way communication, such as in Voice over Internet Protocol (VoIP) applications) or may be in a separate data packet sent specifically for the purpose of returning credits. In systems using virtual channels, credit-based flow control can track the individual credit balance for each virtual channel at the destination.

[0021] In the example system described herein, the amount of buffer space used by the receiver to store packets is determined by the number of transmission cycles used to receive packets, rather than the number of flits containing the packets. This is achieved by making the buffer as wide as the bus (e.g., if the bus receives three flits per transmission cycle, then the buffer is three flits wide). The receiver returns credit to the transmitter based on the number of buffer rows used to store the received packets, rather than the number of flits containing the packets. This reduces computation on the receiver because lower row numbers are counted instead of higher flit numbers.

[0022] To support this credit-based traffic system on the transmitter side, the amount of credit consumed by sending packets can be determined by the number of transmission cycles used to send packets, rather than the number of flits that include packets.

[0023] One benefit of the embodiments of this disclosure is that it uses fewer data packets to return a lower number of credits, representing the number of wide buffer entries consumed rather than the number of received flits. Furthermore, compared to a receiver device using a single wide buffer for a multi-flit wide bus, data processing during the transfer of flits from the bus to the buffer is reduced. Therefore, the processing loops consumed in processing network traffic and returning credits are reduced. Additionally, power consumption during processing network traffic and returning credits is reduced. The performance of systems including communication devices is also improved due to the reduced network overhead. Other benefits will be apparent to those skilled in the art who benefit from this disclosure.

[0024] The embodiments of the present invention are advantageous in a particular context of chiplet systems, in which one or more memory controller chiplets provide an interface between the system's memory and other chiplets (e.g., application chiplets, external interface chiplets, and the like). Processing-intensive operations require relatively more memory accesses and / or more data transfers through the memory controller. Therefore, the described efficiency improvements in reducing processing loops and power consumption in network performance are related to communication with the memory controller and other chiplets, thereby improving overall system performance.

[0025] Figure 1A and 1B An example of a chiplet system 110 according to an embodiment is described. Figure 1A This is an illustration of a chiplet system 110 mounted on a peripheral board 105, which can be connected to a wider range of computer systems, for example, via peripheral component interconnect (PCIe). The chiplet system 110 includes a package substrate 115, an interposer 120, and four chips: an application chiplet 125, a host interface chiplet 135, a memory controller chiplet 140, and a memory device chiplet 150. Other systems may include numerous additional chipsets to provide additional functionality, as will become clear from the following discussion. The package of the chiplet system 110 is described as having a cap or cover 165, but other packaging technologies and structures used for chiplet systems may be used. Figure 1B It is a block diagram used to clearly label the components in a chiplet system.

[0026] Application chip 125 is described as including a network on chip (NOC) 130 to support a chiplet network 155 for inter-chiplet communication. In an example embodiment, NOC 130 may be included on application chip 125. In an example, NOC 130 may be defined in response to the selected supporting chips (e.g., host interface chip 135, memory controller chip 140, and memory device chip 150), thus enabling the designer to select an appropriate number of chiplet network connections or switches for NOC 130. In an example, NOC 130 may reside on a single chiplet or even within intermediary layer 120. In the example discussed herein, NOC 130 implements a chiplet protocol interface (CPI) network.

[0027] CPI is a packet-based network that supports flexible and high-speed interaction between chiplets via virtual channels. CPI bridges the chiplet-internal network to the chiplet network 155. For example, the Advanced Extensible Interface (AXI) is a widely used specification for designing intra-chip communication. However, the AXI specification covers various physical design options, such as the number of physical channels, signal timing, and power. Within a single chip, these options are typically selected to meet design goals, such as power consumption and speed. However, to achieve flexibility in chiplet systems, adapters such as CPI are used to intersect between various AXI design options that can be implemented in various chiplets. By enabling physical-to-virtual channel mapping and encapsulating time-based signaling with a packetization protocol, CPI bridges the chiplet-internal network 155.

[0028] CPI can use various physical layers to transmit packets. A physical layer may contain simple conductive connections or may contain drivers to increase voltage or otherwise facilitate signal transmission over longer distances. An instance of this physical layer may contain an Advanced Interface Bus (AIB), which in various instances may be implemented in intermediate layer 120. The AIB uses source-synchronous data transfer with a forwarding clock to transmit and receive data. Packets are transmitted across the AIB at a single data rate (SDR) or double data rate (DDR) relative to the transmitted clock. Various channel widths are supported by the AIB. AIB channel widths are multiples of 20 bits (20, 40, 60, ...) when operating in SDR mode and multiples of 40 bits (40, 80, 120, ...) in DDR mode. The AIB channel width includes both transmitted and received signals. Channels can be configured to have a symmetrical number of transmit (TX) and receive (RX) inputs / outputs (I / O) or an asymmetrical number of transmitters and receivers (e.g., all transmitters or all receivers). The channel can act as an AIB master or slave, depending on which chiplet provides the master clock. AIB I / O cells support three timing modes: asynchronous (i.e., non-timing), SDR, and DDR. In various instances, non-timing mode is used for clock and some control signals. SDR mode can use dedicated SDR-only I / O cells or dual-purpose SDR / DDR I / O cells.

[0029] In this example, CPI packet protocols (e.g., point-to-point or routable) can use symmetric receive and transmit I / O cells within an AIB channel. CPI streaming protocols allow for more flexible use of AIB I / O cells. In this example, AIB channels for streaming modes can configure I / O cells as all TX, all RX, or half TX and half RX. CPI packet protocols can use AIB channels in SDR or DDR operating modes. In this example, AIB channels are configured in increments of 80 I / O cells (i.e., 40 TX and 40 RX) for SDR mode and 40 I / O cells for DDR mode. CPI streaming protocols can use AIB channels in SDR or DDR operating modes. Here, in this example, AIB channels are configured in increments of 40 I / O cells for both SDR and DDR modes. In this example, a unique interface identifier is assigned to each AIB channel. This identifier is used during CPI reset and initialization to determine paired AIB channels across neighboring chiplets. In this example, the interface identifier is a 20-bit value comprising a 7-bit chiplet identifier, a 7-bit column identifier, and a 6-bit link identifier. The AIB physical layer uses an AIB out-of-band shift register to transmit the interface identifier. The 20-bit interface identifier is transmitted in both directions across the AIB interface using bits 32 to 51 of the shift register.

[0030] AIB defines a set of stacked AIB channels as an AIB channel column. An AIB channel column has a certain number of AIB channels plus auxiliary channels. The auxiliary channels contain signals used for AIB initialization. All AIB channels within a column (except for the auxiliary channels) have the same configuration (e.g., full TX, full RX, or half TX and half RX, and the same number of data I / O signals). In this example, AIB channels are numbered in consecutive ascending order, starting with the AIB channel adjacent to the AUX channel. The AIB channel adjacent to the AUX channel is defined as AIB channel zero.

[0031] Generally, the CPI interface on an individual chiplet may include serialization-deserialization (SERDES) hardware. SERDES interconnects are well-suited for schemes where high-speed signaling with low signal counts is desired. However, SERDES can result in additional power consumption and longer latency for multiplexing and demultiplexing, error detection or correction (e.g., using block-level cyclic redundancy check (CRC)), link-level retries, or forward error correction. However, when low latency or power consumption is a primary concern for very short-range chiplet-to-chiplet interconnects, parallel interfaces with clock rates that allow data transfer with minimal latency can be utilized. CPIs incorporate elements that reduce both latency and power consumption in these very short-range chiplet interconnects.

[0032] CPI employs a credit-based technique for flow control. The receiver (e.g., application chip 125) provides a credit representing available buffers to a transmitter, such as memory controller chip 140. In this example, the CPI receiver contains buffers for each virtual channel for a given transmission time unit. Therefore, if the CPI receiver supports five time messages and a single virtual channel, the receiver has five buffers arranged in five rows (e.g., one row per unit time). If four virtual channels are supported, the receiver has twenty buffers arranged in five rows. Each buffer holds the payload of one CPI packet.

[0033] As the sender transmits to the receiver, it decrements the available credits based on the transmission. Once all of the receiver's credits are exhausted, the sender stops sending packets to the receiver. This ensures that the receiver has an available buffer to store transmissions.

[0034] When the receiver processes the received packet and releases the buffer, the receiver will send the available buffer space back to the sender. This credit return can then be used by the sender to allow the transmission of additional information.

[0035] It also describes a chiplet mesh network 160 using direct chiplet-to-chiplet technology, eliminating the need for NOC 130. The chiplet mesh network 160 can be implemented in CPI or another chiplet-to-chiplet protocol. The chiplet mesh network 160 typically implements a chiplet pipeline, where one chiplet acts as the interface to the pipeline, while other chipslets in the pipeline only interface with themselves.

[0036] Additionally, dedicated device interfaces (such as one or more industry-standard memory interfaces 145, for example, synchronous memory interfaces, such as DDR5, DDR6) can also be used to interconnect chiplets. Chiplet systems or individual chiplets can be connected to external devices (such as larger systems) via desired interfaces (such as PCIe interfaces). In an example, such an external interface can be implemented via a host interface chiplet 135, which, in the depicted example, provides a PCIe interface external to the chiplet system 110. This dedicated interface 145 is typically used when industry practice or standards converge on this interface. The illustrated example of connecting the memory controller chiplet 140 to the dual data rate (DDR) interface 145 of the dynamic random access memory (DRAM) memory device chiplet 150 is such an industry practice.

[0037] Among the various possible chiplets that support them, the memory controller chiplet 140 is most likely to be present in the chiplet system 110, as storage is ubiquitous in computer processing and is a cutting-edge technology for memory devices. Therefore, using memory device chiplets 150 and memory controller chiplets 140 manufactured by other companies empowers chiplet system designers to use robust products from sophisticated manufacturers. Generally, the memory controller chiplet 140 provides a memory device-specific interface for reading, writing, or erasing data. Typically, the memory controller chiplet 140 can provide additional features such as error detection, error correction, maintenance operations, or atomic operation execution. For some types of memory, maintenance operations are often specific to the memory device chiplet 150, such as the collection of discarded items in NAND flash or storage-class memory, or temperature regulation of NAND flash memory (e.g., cross-temperature management). In instances, maintenance operations may include logic-to-physical (L2P) mapping or management to provide an indirection hierarchy between the physical and logical representations of data. In other types of memory (such as DRAM), some memory operations (such as refresh) may be controlled by the host processor or memory controller at some times, and by the DRAM memory device or logic associated with one or more DRAM devices (such as interface chips (in this example, buffers)) at other times.

[0038] Atomic operations are data manipulations that can be performed, for example, by memory controller chiplet 140. In other chiplet systems, atomic operations can be performed by other chipsets. For example, an "incremental" atomic operation can be specified by application chiplet 125 in a command that includes a memory address and a possible increment value. Upon receiving the command, memory controller chiplet 140 retrieves a number from the specified memory address, increments it by the amount specified in the command, and stores the result. Upon successful completion, memory controller chiplet 140 provides application chiplet 125 with an indication that the command was successful. Atomic operations avoid data transfer across chiplet network 160, resulting in reduced latency execution of such commands.

[0039] Atomic operations can be classified as built-in atoms or programmable (e.g., custom) atoms. Built-in atoms are a finite set of operations implemented invariantly in the hardware. Programmable atoms are applets that can run on programmable atomic units (PAUs) (e.g., custom atomic units (CAUs)) of the memory controller chiplet 140. Figure 1 illustrates an example of a memory controller chiplet with a PAU.

[0040] Memory device chip 150 may be or include any combination of volatile memory devices or non-volatile memory. Examples of volatile memory devices include (but are not limited to) random access memory (RAM), such as DRAM, synchronous DRAM (SDRAM), graphics dual data rate type 6 SDRAM (GDDR6 SDRAM), etc. Examples of non-volatile memory devices include (but are not limited to) NAND flash memory, memory-type memory (e.g., phase-change memory or memristor-based technology), ferroelectric RAM (FeRAM), etc. The illustrated example includes memory device chip 150; however, memory device chip 150 may reside elsewhere, such as in different packages on board 105. For many applications, multiple memory device chips may be provided. In the example, these memory device chips may each implement one or more memory technologies. In the example, the memory chip may include multiple stacked memory dies of different technologies, such as one or more SRAM devices that are stacked or otherwise communicate with one or more DRAM devices. The memory controller chiplet 140 can also be used to coordinate the operation between multiple memory chips in the chiplet system 110, for example, by utilizing one or more memory chips in one or more levels of cache storage and using one or more additional memory chips as main memory. The chiplet system 110 may also include multiple memory controller chips 140, which can also be used to provide memory control functionality for individual processors, sensors, networks, etc. The chiplet architecture (such as the chiplet system 110) provides the advantage of allowing adaptation to different memory storage technologies and different memory interfaces by updating the chiplet configuration without redesigning the rest of the system architecture.

[0041] Figure 2 The components of an example of a memory controller chiplet 205 according to an embodiment are described below. The memory controller chiplet 205 includes a cache 210, a cache controller 215, an off-die memory controller 220 (e.g., for communicating with off-die memory 175), a network communication interface 225 (e.g., for interfacing with chiplet network 180 and communicating with other chiplets), and a set of atomic and merge operations 250. Members of this set may include, for example, a write merge unit 255, a memory danger clearance unit 260, a built-in atomic operation unit 265, or a PAU 270. The various components are logically described and not necessarily implemented as such. For example, the built-in atomic operation unit 265 may include different means along the path to off-die memory. For example, the built-in atomic unit may be in an interface means / buffer on the memory chiplet, as discussed above. In contrast, the PAU 270 may be implemented in a separate processor on the memory controller chiplet 205 (but in various instances, it may be implemented elsewhere, such as on the memory chiplet).

[0042] The off-die memory controller 220 is directly coupled to the off-die memory 275 (e.g., via a bus or other communication connection) to provide write and read operations to and from one or more off-die memories, such as the off-die memory 275 and the chiplet network 280. In the depicted example, the off-die memory controller 220 is also coupled for output to the atom and merge operation 250 and for input to the cache controller 215 (e.g., a memory-side cache controller).

[0043] In the instance configuration, the cache controller 215 is directly coupled to the cache 210 and can be coupled to the network communication interface 225 for input (e.g., incoming read or write requests) and coupled to the off-chip memory controller 220 for output.

[0044] Network communication interface 225 includes packet decoder 230, network input queue 235, packet encoder 240, and network output queue 245 to support packet-based chiplet network 280, such as CPI. Chiplet network 280 can provide packet routing between and within processors, memory controllers, mixed-thread processors, configurable processing circuitry, or communication interfaces. In this packet-based communication system, each packet typically contains destination and source addressing, as well as any data payload or instructions. In an example, chiplet network 280 may be implemented as a collection of crossbars with a folded Clos configuration or providing additional connectivity in a mesh network, depending on the configuration.

[0045] In various instances, the chiplet network 280 may be part of an asynchronous switching architecture. Here, data packets can be routed along any of various paths, such that any selected data packet can arrive at its addressed destination at any of multiple different times, depending on the routing. Alternatively, the chiplet network 280 may be implemented at least partially as a synchronous communication network, such as a synchronous mesh communication network. According to this disclosure, two configurations of the communication network can be considered for use in examples.

[0046] The memory controller chip 205 can receive packets having, for example, a source address, a read request, and a physical address. In response, the off-die memory controller 220 or the cache controller 215 reads data from the specified physical address (which may be in off-die memory 275 or cache 210) and assembles a response packet to the source address containing the requested data. Similarly, the memory controller chip 205 can receive packets having a source address, a write request, and a physical address. In response, the memory controller chip 205 writes data to the specified physical address (which may be in cache 210 or off-die memory 275 or 280) and assembles a response packet to the source address containing confirmation that the data has been stored in memory.

[0047] Therefore, the memory controller chiplet 205 can receive read and write requests via the chiplet network 280 and process the requests using the cache controller 215, which interfaces with the cache 210, if possible. If the request cannot be handled by the cache controller 215, then the off-die memory controller 220 handles the request by communicating with the off-die memory 275 or 280, the atom and merge operation 250, or both. As described above, one or more levels of cache can also be implemented in the off-die memory 275 or 280, and in some such instances, they can be directly accessed via the cache controller 215. Data read by the off-die memory controller 220 can be cached in the cache 210 by the cache controller 215 for later use.

[0048] Atom and merge operation 250 is coupled to receive (as input) the output of off-die memory controller 220 and provides output to cache 210, network communication interface 225, or directly to chiplet network 280. Memory danger clearing unit 260, write merging unit 255, and built-in atom operation unit 265 can each be implemented as a state machine with other combinational logic circuitry (e.g., adders, shifters, comparators, AND gates, OR gates, XOR gates, or any suitable combination thereof) or other logic circuitry. These components may also include one or more registers or buffers to store operands or other data. PAU 270 can be implemented as one or more processor cores or control circuitry and various state machines with other combinational logic circuitry or other logic circuitry, and may also include one or more registers, buffers, or memories to store addresses, executable instructions, operands, and other data, or may be implemented as a processor.

[0049] Write merging unit 255 receives read data and request data and merges the request data and read data to create a single unit with the read data and source address used in response or return data packets. Write merging unit 255 provides the merged data to the write port of cache 210 (or equivalently, to cache controller 215 for writing to cache 210). Optionally, write merging unit 255 provides the merged data to network communication interface 225 to encode and prepare response or return data packets for transmission on chiplet network 280.

[0050] When requested data is used for a built-in atomic operation, the built-in atomic operation unit 265 receives the request and reads the data from the write merging unit 255 or directly from the off-chip memory controller 220. The atomic operation is performed, and using the write merging unit 255, the resulting data is written to the cache 210 or provided to the network communication interface 225 to encode and prepare response or return data packets for transmission over the chiplet network 280.

[0051] Built-in atomic operation unit 265 handles predefined atomic operations, such as fetch and increment or compare and swap. In examples, these operations perform simple read-modify-write operations on a single memory location of 32 bytes or less. Atomic memory operations begin with a request packet transmitted via chiplet network 280. The request packet has a physical address, atomic operator type, operand size, and optionally up to 32 bytes of data. The atomic operation performs a read-modify-write operation on cache memory lines of cache 210 to fill the cache memory as needed. The atomic operator response can be a simple complete response or a response with up to 32 bytes of data. Example atomic memory operators include fetch and AND, fetch and OR, fetch and XOR, fetch and addition, fetch and subtraction, fetch and increment, fetch and decrement, fetch and minimum, fetch and maximum, fetch and swap, and compare and swap. In various example embodiments, 32-bit and 64-bit operations and operations on 16 or 32 bytes of data are supported. The methods disclosed herein are also compatible with hardware that supports more or fewer operations and more or less data.

[0052] Built-in atomic operations may also involve requests for "standard" atomic operations on the requested data, such as relatively simple, single-cycle integer atoms (e.g., fetch and increment or compare and swap), which will occur with the same amount of processing as regular memory read or write operations that do not involve atomic operations. For these operations, cache controller 215 typically reserves cache lines in cache 210 by setting a danger bit (in hardware) so that the cache line cannot be read by another process while it is in transition. Data is obtained from off-die memory 275 or cache 210 and provided to built-in atomic operation unit 265 to perform the requested atomic operation. After the atomic operation, in addition to providing the obtained data to data packet encoder 240 to encode outgoing data packets for transmission on chiplet network 280, built-in atomic operation unit 265 provides the obtained data to write merging unit 255, which also writes the obtained data to cache 210. After the obtained data is written to cache 210, any corresponding danger bits set are cleared by memory danger clearing unit 260.

[0053] The PAU 270 implements high-performance (high throughput and low latency) programmable atomic operations (also known as "custom atomic operations") comparable to built-in atomic operations. Instead of performing multiple memory accesses, in response to an atomic operation request specifying a programmable atomic operation and a memory address, the circuitry in the memory controller chiplet 205 transmits the atomic operation request to the PAU 270 and sets a danger bit in a memory danger register corresponding to the memory address used in the atomic operation to ensure that no other operation (read, write, or atomic) is performed on that memory line. The danger bit is then cleared after the atomic operation is completed. The additional direct data path provided for the PAU 270 to perform programmable atomic operations allows for additional write operations without being limited by the bandwidth of the communication network and without increasing network congestion.

[0054] The PAU 270 includes a multi-threaded processor, such as a RISC-VIS-based multi-threaded processor, having one or more processor cores and further having an extended instruction set for performing programmable atomic operations. When equipped with an extended instruction set for performing programmable atomic operations, the PAU 270 can be embodied as one or more hybrid-threaded processors. In some example embodiments, the PAU 270 provides bucket-style polling instantaneous thread switching to maintain a high instruction-per-clock rate.

[0055] Programmable atomic operations can be executed by PAU 270, which involve requesting programmable atomic operations on requested data. The user can prepare programming code to provide this programmable atomic operation. For example, the programmable atomic operation can be a relatively simple multi-loop operation (e.g., floating-point addition) or a relatively complex multi-instruction operation (e.g., a Bloom filter plug-in). The programmable atomic operation can be the same as or different from a predetermined atomic operation, as long as it is defined by the user and not the system vendor. For these operations, cache controller 215 can reserve cache lines in cache 210 by setting a danger bit (in hardware) so that the cache lines cannot be read by another process while they are in transition. Data is obtained from cache 210 or off-chip memory 275 or 280 and provided to PAU 270 to execute the requested programmable atomic operation. After the atomic operation, PAU 270 provides the obtained data to network communication interface 225 to directly encode outgoing data packets containing the obtained data for transmission on chiplet network 280. Additionally, PAU 270 provides the obtained data to cache controller 215, which in turn writes the obtained data to cache 210. After the obtained data is written to cache 210, cache controller 215 clears any corresponding dangerous bits that have been set.

[0056] In the selected example, the approach taken for programmable atomic operations is to provide multiple generic custom atomic request types, which can be sent from the original source, such as a processor or other system component, to the memory controller chiplet 205 via chiplet network 280. Cache controller 215 or off-die memory controller 220 recognizes the request as a custom atom and forwards the request to PAU 270. In a representative embodiment, PAU 270: (1) is a programmable processing element capable of efficiently performing user-defined atomic operations; (2) can perform load and store operations on memory, arithmetic and logical operations, and control flow decisions; and (3) utilizes a RISC-V ISA and a new set of specialized instructions to facilitate interaction with this cache controller 215 and off-die memory controller 220 to atomically perform user-defined operations. In a desirable example, the RISC-V ISA contains a full set of instructions supporting one of the high-level language operators and data types. PAU 270 may utilize the RISC-V ISA, but will typically support a more limited instruction set and a limited register file size when included within memory controller chiplet 205 to reduce the die size of the unit.

[0057] As mentioned above, before writing read data to cache 210, the memory danger clearing unit 260 clears the danger bits set for reserved cache lines. Therefore, when a request and read data are received by write merging unit 255, a reset or clear signal can be transmitted from memory danger clearing unit 260 to cache 210 to reset the memory danger bits set for reserved cache lines. Furthermore, resetting these danger bits also releases pending read or write requests involving specified (or reserved) cache lines, allowing pending read / write requests to be provided to the incoming request multiplexer for selection and processing.

[0058] Figure 3 This describes an example of routing between chiplets in chiplet layout 300 using a Chiplet Protocol Interface (CPI) network according to an embodiment. Chiplet layout 300 includes chiplets 310A, 310B, 310C, 310D, 310E, 310F, 310G, and 310H. Chipslets 310A to 310H are interconnected via a network including nodes 330A, 330B, 330C, 330D, 330E, 330F, 330G, and 330H. Each of chiplets 310A to 310H includes hardware transceivers labeled 320A to 320H.

[0059] CPI packets can be transmitted between chiplets 310 using the Advanced Interface Bus (AIB). The AIB provides physical layer functionality. The physical layer uses source-synchronous data transfer with a forwarding clock to transmit and receive data. Packets are transmitted across the AIB at Single Data Rate (SDR) or Double Data Rate (DDR) relative to the transmitted clock. Various channel widths are supported by the AIB. AIB channel widths are multiples of 20 bits (20, 40, 60, ...) when operating in SDR mode and multiples of 40 bits (40, 80, 120, ...) in DDR mode. The AIB channel width includes both transmitted and received signals. Channels can be configured to have a symmetrical number of transmit (TX) and receive (RX) inputs / outputs (I / O) or an asymmetrical number of transmitters and receivers (e.g., all transmitters or all receivers). A channel can act as an AIB master or slave, depending on which chiplet provides the master clock.

[0060] The AIB adapter provides interfaces to the AIB link layer and to the AIB physical layer (PHY). The AIB adapter provides a data hierarchy register, a power-on reset sequencer, and a control signal shift register.

[0061] The AIB physical layer consists of AIB I / O cells. AIB I / O cells (implemented by hardware transceiver 320 in some embodiments) can be input-only, output-only, or bidirectional. An AIB channel consists of a set of AIB I / O cells, the number of which depends on the AIB channel configuration. A received signal on a chiplet is connected to a pair of transmitted signals on the chipsets. In some embodiments, each column includes auxiliary (AUX) channels numbered 0 to N and a data channel.

[0062] AIB channels are typically configured as half TX data plus half RX data, all TX data, or all RX data plus associated clock and various control signals. In some example implementations, the number of TX and RX data signals is determined at design time and cannot be configured as part of the system initialization.

[0063] The CPI packet protocol (point-to-point and routable) uses symmetric receive and transmit I / O cells within the AIB channel. The CPI streaming protocol allows for more flexible use of AIB I / O cells. In some example implementations, the AIB channel for streaming mode can configure I / O cells as all TX, all RX, or half TX and half RX.

[0064] Data packets are routed between chiplets 310 via network nodes 330. Node 330 can determine the next node 330 to which a received data packet should be forwarded based on one or more data fields of the data packet. For example, source or destination address, source or destination port, virtual channel, or any suitable combination thereof can be hashed to select consecutive network nodes or available network paths. Path selection in this manner can be used to balance network traffic.

[0065] therefore, Figure 3 The diagram illustrates the data path from chiplet 310A to chiplet 310D. Data packets are sent from hardware transceiver 320A to network node 330A, forwarded by network node 330A to network node 330C, forwarded by network node 330C to network node 330D, and finally transmitted by network node 330D to hardware transceiver 320D of chiplet 310D.

[0066] Figure 3 The diagram also illustrates a second data path from chiplet 310A to chiplet 310G. Data packets are sent from hardware transceiver 320A to network node 330A, forwarded by network node 330A to network node 330B, forwarded by network node 330B to network node 330D, forwarded by network node 330D to network node 330C, forwarded by network node 330C to network node 330E, forwarded by network node 330E to network node 330F, forwarded by network node 330F to network node 330H, forwarded by network node 330H to network node 330G, and finally transmitted by network node 330G to hardware transceiver 320G of chiplet 310G. Figure 3 As can be clearly seen, multiple paths through the network can be used to transfer data between any pair of small chips.

[0067] AIB I / O cells support three timing modes: asynchronous (i.e., non-timing), SDR, and DDR. Non-timing mode is used for clock and some control signals. SDR mode can use dedicated SDR-only I / O cells or dual-purpose SDR / DDR I / O cells.

[0068] The CPI packet protocol (point-to-point and routable) can use AIB channels in either SDR or DDR operating modes. In some example implementations, the AIB channels should be incremented by 80 I / O cells (i.e., 40 TX and 40 RX) for SDR mode and by 40 I / O cells for DDR mode.

[0069] The CPI streaming protocol can use AIB channels in either SDR or DDR operating mode. In some example implementations, the AIB channels are incremented by 40 I / O cells for both modes (SDR and DDR).

[0070] Each AIB channel is assigned a unique interface identifier. This identifier is used during CPI reset and initialization to determine the paired AIB channels across neighboring chiplets. In some implementations, the interface identifier is a 20-bit value comprising a 7-bit chiplet identifier, a 7-bit column identifier, and a 6-bit link identifier. The AIB physical layer uses an AIB out-of-band shift register to transmit the interface identifier. The 20-bit interface identifier is transmitted in both directions across the AIB interface using bits 32 to 51 of the shift register.

[0071] AIB defines a set of stacked AIB channels as an AIB channel column. For example... Figure 3 As shown, the AIB channel column has a certain number of AIB channels plus auxiliary channels. The auxiliary channels contain signals for AIB initialization. All AIB channels within the column (except for the auxiliary channels) have the same configuration (e.g., full TX, full RX, or half TX and half RX, and the same number of data I / O signals). The AIB channel column is arranged in a specific orientation around the periphery of the chiplet to allow interoperability between chiplets. Although each of the chiplets 310... Figure 3 The chip is shown as having one hardware transceiver 320, but it should be noted that each side of the chip may have zero or more hardware transceivers.

[0072] In some example implementations, AIB channels are numbered in consecutive ascending order, starting with the AIB channel adjacent to the AUX channel. The AIB channel adjacent to the AUX is defined as AIB channel zero.

[0073] For example, Figure 3 The diagram shows eight chiplets 310 connected by a network comprising eight nodes 330. More or fewer chiplets 310 and more or fewer nodes 330 can be included in the chiplet network to allow the creation of chiplet networks of any size.

[0074] Figure 4 This is a block diagram of a data packet 400 comprising multiple flits according to some embodiments of the present disclosure. The data packet 400 is adapted to be processed by... Figure 3 Network 300 communicates with other small chips. Figure 2 The memory controller chip 205 is used. Data packets 400 are divided into flow control units (flits), each of which consists of 36 bits in this example. Data packets 400 include a header 410 and zero or more body flits (e.g., body flit 420 and body flit 430). The header 410 may contain a destination identifier field, a length field, a command field, or any suitable combination thereof. Body flits 420 and 430 may contain data, additional control fields, or both.

[0075] The destination identifier uniquely identifies the destination in the network (e.g., a virtual channel on a physical destination chip). In direct wired connections that do not use networks with multiple destinations, the destination identifier can be omitted. The length field indicates the number of flits comprising data packet 400. The interpretation of the length field can be non-linear. For example, values ​​0 to 22 can be interpreted as 0 to 22 flits in data packet 400, and values ​​23 to 27 can be interpreted as 33 to 37 flits in data packet 400 (i.e., 10 more than the indicated value). In this example, the packet size is measured in flits, independent of the bus width or the number of bits per flit. In other example embodiments, the packet size is measured in bits or bytes. In other example embodiments, the packet size is fixed and the length field can be omitted. Other values ​​for the body flit 430 can be vendor-defined rather than protocol-defined.

[0076] Commands used for data packet 400 can be, for example, write commands, read commands, predefined atomic operation commands, custom atomic operation commands, read responses, acknowledgment responses, or vendor-specific commands. Additionally, commands can indicate virtual channels for data packet 400. For example, different commands can be used for different virtual channels.

[0077] The memory access command can further identify the number of bytes to be written or accessed, the memory space to be accessed (e.g., off-die memory 375 or instruction memory for custom atomic operations), or any suitable combination thereof. In some example embodiments, the command may indicate additional bit identification commands for a subsequent flit. For example, a multi-byte command can be sent by using a vendor-specific command in the command field of header 410 and using part or all of the body flit 420 containing the larger command.

[0078] Figure 5 This is a block diagram 500 illustrating a received flit stream and a virtual channel queue of received flits according to some embodiments of the present disclosure. The interface on which the flit stream is received is three channel wide. Therefore, one flit is received simultaneously on each of channels 510A, 510B, and 510C. Received flits are sequentially shown in rows 520A, 520B, 520C, 520D, and 520E. After reception, the flits are stored in virtual channel 0 channel queues 530A, 530B, and 530C or virtual channel 1 channel queues 550A, 550B, and 550C. Virtual channel 0 channel queues 530A to 530C store the flits in rows 540A, 540B, 540C, 540D, and 540E. Virtual channel 1 channel queues 550A to 550C will store flit in rows 560A, 560B, 560C, 560D, and 560E.

[0079] The received flit stream displays three packets: P1, P2, and P3. P1 is designated for virtual channel 0 and includes four flits. P2 is designated for virtual channel 1 and includes five flits. P3 is designated for virtual channel 0 and includes three flits. The virtual channel for each packet can be determined based on the destination identifier in header 410.

[0080] On each clock cycle, based on the destination of the flit in a row, all or part of the received data in that row (e.g., one of rows 520A to 520E) is copied to zero or more of the virtual channel queues. For example, row 520A is partially copied to row 540A because two flits in row 520A are designated for virtual channel 0. Data in the free flits of row 520A may be copied or left unwritten in row 540A. As another example, row 520B is partially copied to both row 540B and row 560A because row 520B contains at least one flit from both virtual channel 0 and virtual channel 1.

[0081] Rows 540E, 560D, and 560E can be used to store additional data. In some example embodiments, the buffered data is cleared (e.g., replaced with 0 or another data pattern) after it has been read.

[0082] By copying the entire row into a queue with multiple channels instead of copying each flit individually into a single-channel queue, the timing of the circuit design is easier to satisfy. For example, regarding row 520A, the first flit of P1 will be copied to the first entry of the single-channel queue, the destination pointer will be incremented, and then the second flit of P1 will be copied to the second entry of the single-channel queue. Each of these operations needs to be completed using a higher internal clock rate before the next clock cycle of the bus on which the stream of flits is received.

[0083] In a credit-based flow control system, the transmission device maintains a data structure indicating the number of credits available for data transfer to the destination. The initial number of credits can be provided from the destination to the source (e.g., upon power-on). Figure 5 As seen in the example, the receiving device has five entries of buffer space available for each virtual channel. Therefore, if the device communicates only with a single transmitter, it is safe to allocate five credits to each virtual channel. After transmission, the transmitting device reduces the number of available credits for the virtual channel used. After processing the data, the receiving device sends one or more credit return packets. These credit return packets indicate the amount of buffer space that has been freed up by processing the data and is now available to store additional data from the source. In this way, the source can keep track of the amount of available buffer space at the destination and avoid overflow.

[0084] Figure 6This illustrates some embodiments of the present disclosure consisting of circuits (e.g.) Figure 2 The flowchart illustrates the operation of method 600, executed when the small chip 205 sends multiple flits on a transfer cycle. Method 600 includes operations 610 and 620. By way of example and not limitation, method 600 is described as being used by the apparatus of Figures 1 to 3. Figure 4 Data is processed in packets. In method 600, packet transmission is associated with credits, such that any number of flits transmitted in a single transmission cycle are associated with a single credit.

[0085] In operation 610, logic (e.g., one or more components of chip 310A, such as network interface 225) transmits multiple flits to a destination (e.g., chip 310D) on a transmission cycle, with two or more of the flits directed to a single virtual channel. For example, any flit in rows 520A to 520D can be sent. Rows 520A, 520B, and 520D contain two or more flits directed to virtual channel 0; row 520C contains two or more flits directed to virtual channel 1. Each flit may contain an indicator of the virtual channel it is designated for. Alternatively, header 410 may contain an indicator of the virtual channel for the entire packet and the number of flits in the packet (or another measure of the packet size). Based on the packet size and the virtual channel identified in the header, the circuitry determines that each flit in the packet is also used for the same virtual channel.

[0086] In operation 620, logic (e.g., one or more components of chip 310A) modifies the data representing the number of credits by reducing the number of credits for the destination virtual channel by 1, based on two or more fragments transmitted on a transmission cycle and directed to a single virtual channel. Therefore, for row 520A, even if more than one fragment is transmitted for virtual channel 0, the number of credits for virtual channel 0 is reduced by 1. In some example embodiments, the number of credits for each virtual channel having one or more fragments transmitted in a transmission cycle is reduced by 1. In these example embodiments, chip 310A reduces the number of credits for both virtual channel 0 and virtual channel 1 after transmitting the fragment of row 520A.

[0087] Compared to a solution that uses a credit to represent a flit, fewer credits are needed to send data packets over a multi-flit interface using credits based on the transmission cycle rather than credits based on the flit.

[0088] Figure 7This is a flowchart illustrating the operation of method 700 performed by a circuit receiving multiple flits in a transfer cycle according to some embodiments of the present disclosure. Method 700 includes operations 710, 720, and 730. By way of example and not limitation, method 700 is described as being used by the apparatus of Figures 1 to 3. Figure 4 Data is grouped and executed.

[0089] In operation 710, the circuit receives multiple flits from a source on a transmission cycle, with two or more of the flits directed to a first virtual channel and one of the flits directed to a second virtual channel. For example, the network interface 225 of chip 310D may receive flits of line 520B from chip 310A on a transmission cycle, including two flits directed to virtual channel 0 and one flit directed to virtual channel 1. In some example embodiments, each flit contains an indicator of the virtual channel to which it is directed. In other example embodiments, the header flit of a packet indicates the virtual channel and the size of the packet; subsequently, the flit determines the portion of the packet to which it is directed based on the packet size and thus determines which virtual channel is indicated in the header.

[0090] In operation 720, the circuit copies two or more flits to the first buffer associated with the first virtual channel. For example, the flit directed to virtual channel 0 in line 520B is copied to both in the channel queue of virtual channel 0 in line 540B.

[0091] In operation 730, the circuit copies the flit to the second buffer associated with the second virtual channel. Continuing this example, the flit of virtual channel 1 in line 520B is copied to channel queue 550C of virtual channel 1 in line 560A. Operations 720 and 730 can be performed on a single clock cycle, simultaneously, or both.

[0092] Compared to solutions that use a single-channel queue for each virtual channel, the circuitry using method 700 can copy all data from the virtual channel without causing delays in copying, pointer incrementing, or the copy loop. Furthermore, energy consumption is reduced by masking the data in the received flit stream so that virtual channel data is not written to the channel queues of other virtual channels (as shown by the "Do Not Write" values ​​in rows 540A, 540D, 560A, and 560C).

[0093] Figure 8This is a composite flowchart illustrating the operation of a composite method 800 performed by a first circuit and a second circuit according to some embodiments of the present disclosure, including a first method 810 performed by the first circuit when transmitting one or more packets and a second method 820 performed by the second circuit when receiving one or more packets. Method 810 includes operations 830, 840, and 880. Method 820 includes operations 850, 860, and 870. Solid arrows indicate the sequence of operations within each method 810 and 820. Dashed arrows show the relationship between the two methods 810 and 820. Overall, the composite method 800 includes operations 830 to 880. By way of example and not limitation, method 800 is described as being used by the apparatus of Figures 1 to 3. Figure 4 Data is grouped and executed.

[0094] In operation 830, the source (e.g., chip 310A) selects one or more packets based on the number of available credits in the credit pool of the destination for each selected packet. For example, several packets may be queued for transmission from the source. Each packet identifies its destination (e.g., physical device and virtual channel). Each packet also has a size (e.g., the number of flits to be used to transmit the packet). Based on the packet size and the number of available credits at the destination, the source determines whether there are enough credits available to transmit the entire packet. In a flit-based flow control credit scheme, the number of credits used to transmit the packet is the number of flits in the packet. In a transmission cycle-based flow control scheme, the number of credits used to transmit the packet is the number of transmission cycles used to transmit the packet. This can be determined using the following equation:

[0095] )

[0096] In this equation, `cycle` is the number of transmission cycles that will be used to transmit a packet, `ceiling` is a function that rounds the fractional value to the next integer, `f0` is a 0-based offset from the position of the first flit in the wide bus, and `width` is the width of the bus, measured in flits. Therefore, when width = 1, `f0` is always 0, and the number of cycles is only the size of the packet (in flits). When width = 2 and `f0` is 0, the number of cycles is half the size of the packet, rounded. When width = 2 and `f0` is 1, packets with an even number of flits are transmitted using an additional transmission cycle (since only one flit is transmitted on the first transmission cycle and one flit is transmitted on the last transmission cycle).

[0097] In operation 840, the source transmits one or more packet fragments via the bus to update the packet credit pool based on the number of transmission cycles used to transmit each packet, rather than the number of fragments per packet. (Reference) Figure 5The stream of flits is sent in three packets: P1, P2, and P3. P1, despite being four flits long, consumes two credits because the entire packet is transmitted in two transmission cycles corresponding to lines 520A and 520B. P2, despite being five flits long, consumes three credits. P3, despite being three flits long, consumes two credits. Therefore, in a transmission cycle in which flits for multiple destinations are sent, the number of available credits for each destination is reduced by 1, regardless of the distribution of flits across destinations. In some implementations, the credit pool is updated before sending any flit of a packet to reduce the number of transmission cycles that will be used to send the packet. In other implementations, the credit pool is updated on each transmission cycle to reduce the number of available credits by 1 each time one or more flits are transmitted.

[0098] In operation 850, the destination receives a flit via the bus to queue the received flit for processing. Continue Figure 5 In one example, the receiving chip 310D stores the flit in the virtual channel queue based on the virtual channel identified in the header of each packet.

[0099] In operation 860, the destination circuit converts the received flit into one or more packets. Therefore, packets P1, P2, and P3 are reconstructed from the received flit.

[0100] In operation 870, for each received packet, the destination circuit replenishes credits based on the number of transmission cycles used to transmit the packet, rather than the number of packet flits. For example, a credit return packet may be sent from chip 310D to chip 310A to indicate that two credits are returned after processing P1, three credits are returned after processing P2, or two credits are returned after processing P3. After processing the packet, storing the rows that constitute the packet's flits can be used to dispose of new data received via the bus.

[0101] In operation 880, the source circuit receives credit return packets and updates the credit pool based on the data in the credit return packets. For example, the credit return packets may indicate the destination circuit (using the identifier of chip 310D), the virtual channel, and the number of credits returned. By updating the credit pool to reflect the release line at the destination, the source circuit prepares for another iteration of method 800 to send more data packets to the destination without overflowing the receive buffer.

[0102] The system execution, including both source and destination chips, includes method 800, which coordinates methods 810 and 820. By using method 800, the system can use credit-based flow control to prevent destination buffer overflow, utilize the ability of a wide bus to transmit multiple flits per transfer cycle, and avoid the complexities caused by using a single-channel buffer and a wide bus.

[0103] Figure 9 This describes a block diagram of an example machine 900 in which any or more of the techniques (e.g., methods) discussed herein can be implemented, in, or through it. As described herein, an example may contain logic or several components or mechanisms in or operable by the machine 900. A circuit system (e.g., a processing circuit system) is a collection of circuits implemented in a tangible entity of the machine 900 containing hardware (e.g., simple circuits, gates, logic, etc.). The members of the circuit system can change flexibly over time. The circuit system contains members that can perform specific operations individually or in combination during operation. In an example, the hardware of the circuit system may be designed invariably to implement specific operations (e.g., hardwired). In an example, the hardware of the circuit system may contain variable-connection physical components (e.g., execution units, transistors, simple circuits, etc.) that contain machine-readable media that are physically modified (e.g., magnetically, electrically, or with movable placement of invariant mass particles, etc.) to encode instructions for specific operations. For example, when connecting physical components, the basic electrical properties of the hardware composition change from insulators to conductors, and vice versa. Instructions enable embedded hardware (such as an execution unit or loading mechanism) to create members of a circuit system within the hardware via variable connections to perform specific operations during operation. Thus, in an example, a machine-readable media element is part of the circuit system or communicatively coupled to other components of the circuit system during device operation. In an example, any of the physical components can be used in more than one member of more than one circuit system. For instance, under operation, an execution unit may be used at one point in time in a first circuit of a first circuit system and may be reused at different times by a second circuit in the first circuit system or a third circuit in the second circuit system. Additional examples of these components with respect to machine 900 are as follows.

[0104] In alternative embodiments, machine 900 may operate as a standalone device or be connected (e.g., networked) to other machines. In a networked deployment, machine 900 may operate as a server machine, a client machine, or both in a server-client network environment. In an example, machine 900 may act as a peer-to-peer (P2P) (or other distributed) network environment. Machine 900 may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, network device, network router, switch, or bridge, or any machine capable of executing instructions (sequentially or otherwise) specifying actions to be taken by the machine. Furthermore, while only a single machine is described, the term "machine" should also be considered to include any collection of machines that individually or collectively execute a set (or more) of instructions to perform any or more of the methods discussed herein, such as cloud computing, Software as a Service (SaaS), or other computer cluster configurations.

[0105] Machine (e.g., computer system) 900 may include a hardware processor 902 (e.g., a central processing unit (CPU), graphics processing unit (GPU), hardware processor core, or any combination thereof), main memory 904, static memory (e.g., memory or storage device for firmware, microcode, basic input-output (BIOS), unified extensible firmware interface (UEFI), etc.) 906, and mass storage device 908 (e.g., hard disk drive, tape drive, flash memory, or other block device), some or all of which may communicate with each other via interconnect (e.g., bus) 930. Machine 900 may further include a display unit 910, an alphanumeric input device 912 (e.g., keyboard), and a user interface (UI) navigation device 914 (e.g., mouse). In an example, the display unit 910, input device 912, and UI navigation device 914 may be a touchscreen display. Machine 900 may additionally include a storage device (e.g., a drive unit) 908, a signal generating device 918 (e.g., a speaker), a network interface device 920, and one or more sensors 916, such as a Global Positioning System (GPS) sensor, a compass, an accelerometer, or other sensors. Machine 900 may include an output controller 928, for example, a serial (e.g., Universal Serial Bus (USB)), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection to transmit or control one or more peripheral devices (e.g., a printer, a card reader, etc.).

[0106] The registers of processor 902, main memory 904, static memory 906, or mass storage device 908 may be or contain machine-readable media 922, on which one or more sets of data structures or instructions 924 (e.g., software) embodying or utilized by any one or more of the techniques or functions described herein. Instructions 924 may also reside wholly or at least partially in any of the registers of processor 902, main memory 904, static memory 906, or mass storage device 908 during execution by machine 900. In an example, one or any combination of hardware processor 902, main memory 904, static memory 906, or mass storage device 908 may constitute machine-readable media 922. Although machine-readable storage media 922 is described as a single medium, the term "machine-readable media" may include a single medium or multiple media (e.g., a centralized or distributed database or associated cache and server) configured to store one or more instructions 924.

[0107] The term "machine-readable media" may include any medium capable of storing, encoding, or carrying instructions executable by machine 900 and causing machine 900 to perform any or more of the technologies disclosed herein, or any medium capable of storing, encoding, or carrying data structures used by or associated with such instructions. Examples of non-limiting machine-readable media may include solid-state memory, optical media, magnetic media, and signals (e.g., radio frequency signals, other photon-based signals, sound signals, etc.). In examples, non-transitory machine-readable media includes machine-readable media having a plurality of particles containing invariant (e.g., rest) mass and thus being composed of matter. Therefore, non-transitory machine-readable media is machine-readable media that does not contain transiently propagating signals. Specific examples of non-transitory machine-readable media may include: non-volatile memory, such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0108] In an example, information stored or otherwise provided on machine-readable medium 922 may represent instructions 924, such as instructions 924 themselves or a format from which instructions 924 may be derived. This format from which instructions 924 may be derived may contain source code, encoded instructions (e.g., in compressed or encrypted form), encapsulation instructions (e.g., divided into multiple encapsulations), or the like. The information representing instructions 924 in machine-readable medium 922 may be processed by a processing circuitry system into instructions for performing any of the operations discussed herein. For example, deriving instructions 924 from information (e.g., processed by a processing circuitry system) may include: compiling (e.g., from source code, object code, etc.), interpreting, loading, organizing (e.g., dynamic or static linking), encoding, decoding, encrypting, decrypting, encapsulating, unencapsulating, or otherwise manipulating the information as instructions 924.

[0109] In an example, the derivation of instruction 924 may contain assembly, compilation, or interpretation information (e.g., by processing a circuit system) to create instruction 924 in some intermediate or preprocessed format provided by machine-readable media 922. Information provided in multiple parts may be combined, unpacked, and modified to create instruction 924. For example, the information may be in multiple compressed source code packages (or object code or binary executable code, etc.) on one or more remote servers. The source code packages may be encrypted during network transfer and decrypted, decompressed, assembled (e.g., linked) (as needed), and compiled or interpreted (e.g., into libraries, standalone executables, etc.) on the local machine and executed by the local machine.

[0110] Instruction 924 may further utilize any of several transmission protocols (such as Frame Relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.) to transmit or receive data via a transmission medium through network interface device 920 over communication network 926. Example communication networks may include local area networks (LANs), wide area networks (WANs), packet data networks (such as the Internet), mobile phone networks (such as cellular networks), conventional telephone (POTS) networks, and wireless data networks (such as the IEEE 802.11 series of standards (known as Wi-Fi®), the IEEE 802.16 series of standards (known as WiMax®)), the IEEE 802.15.4 series of standards, peer-to-peer (P2P) networks, etc. In examples, network interface device 920 may include one or more physical jacks (such as Ethernet, coaxial, or telephone jacks) or one or more antennas for connection to communication network 926. In an example, network interface device 920 may include multiple antennas to communicate wirelessly using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) technologies. The term "transmission medium" should be considered as any intangible medium capable of storing, encoding, or carrying instructions executable by machine 900, and containing digital or analog communication signals or other intangible media to facilitate communication of this software. The transmission medium is a machine-readable medium.

[0111] In the foregoing description, some exemplary embodiments of this disclosure have been described. It will be apparent that various modifications may be made thereto without departing from the broader scope of this disclosure as set forth in the appended claims. Therefore, the specification and drawings should be considered illustrative rather than limiting. The following is a non-exhaustive list of examples of embodiments of this disclosure.

[0112] Example 1 is a system comprising: logic configured to perform operations including: transmitting a plurality of flow control units (flits) to a destination on a transmission cycle, two or more of the plurality of flits being portions of packets directed to the destination virtual channel; and modifying data representing the number of credits by decreasing the number of credits of the virtual channel at the destination by 1, based on the fact that two or more of the plurality of flits are transmitted on the transmission cycle and are portions of the packets directed to the virtual channel.

[0113] In Example 2, the object of Example 1 includes: one of the plurality of flits being directed to a second virtual channel of the destination; and the operation further includes: based on the flit being directed to the second virtual channel, reducing the second number of credits of the second virtual channel of the destination by 1.

[0114] In Example 3, the subject matter of Examples 1 to 2 includes the operation further comprising: determining to send the packet based on the number of credits of the virtual channel at the destination, the count of the plurality of fragments, and the count of fragments in the packet before transmitting the plurality of fragments.

[0115] In Example 4, the subject of Example 3 includes: the plurality of flits comprising one or more flits directed to the second virtual channel; and the determination to send the packet is further based on a count of the one or more flits of the second virtual channel.

[0116] In Example 5, the object of Examples 1 to 4 includes at least one of the flits of the virtual channel leading to the destination, which includes an identifier of the virtual channel.

[0117] Example 6 is a non-transitory machine-readable medium storing instructions that, when executed by a system, cause the system to perform operations including: transmitting multiple flow control units (flits) to a destination on a transmission cycle, two or more of the multiple flits being portions of packets directed to the destination virtual channel; and modifying data representing the number of credits by decreasing the number of credits of the virtual channel at the destination by 1, based on the two or more of the multiple flits transmitted on the transmission cycle and being portions of the packets directed to the virtual channel.

[0118] In Example 7, the subject of Example 6 includes: one of the plurality of flits being directed to a second virtual channel; and the operation further includes: based on the flit being directed to the second virtual channel, reducing the second number of credits of the second virtual channel at the destination by 1.

[0119] In Example 8, the subject matter of Examples 6 and 7 includes the operation further comprising: determining to send the packet based on the number of credits of the virtual channel at the destination, the count of the plurality of fragments, and the count of fragments in the packet before transmitting the plurality of fragments.

[0120] In Example 9, the subject of Example 8 includes: the plurality of flits includes one or more flits directed to the second virtual channel; and the determination to send the packet is further based on a count of the one or more flits directed to the second virtual channel.

[0121] In Example 10, the object of Examples 6 to 9 includes at least one of the flits of the virtual channel leading to the destination, which includes an identifier of the virtual channel.

[0122] Example 11 is a method comprising: transmitting a plurality of flow control units (flits) to a destination on a transmission cycle, two or more of the plurality of flits being portions of packets directed to the destination virtual channel; and modifying data representing the number of credits by decreasing the number of credits of the virtual channel at the destination by 1, based on the fact that two or more of the plurality of flits are transmitted on the transmission cycle and are portions of packets directed to the virtual channel.

[0123] In Example 12, the object of Example 11 includes: one of the plurality of flits being directed to a second virtual channel; and the method further includes: based on the flit being directed to the second virtual channel, reducing the second number of credits of the second virtual channel at the destination by 1.

[0124] In Example 13, the subject matter of Examples 11 to 12 includes the method further comprising: determining to send the packet based on the number of credits of the virtual channel at the destination, the count of the plurality of fragments, and the count of fragments in the packet before transmitting the plurality of fragments.

[0125] In Example 14, the subject of Example 13 includes wherein: the plurality of flits includes one or more flits directed to the second virtual channel; and the determination to send the packet is further based on a count of the one or more flits of the second virtual channel.

[0126] In Example 15, the object of Examples 11 to 14 includes at least one of the flits of the virtual channel leading to the destination, which includes an identifier of the virtual channel.

[0127] Example 16 is a chiplet system comprising: a first chiplet including a memory controller and further including logic configured to perform operations including: receiving a plurality of flow control units (flits) from a source on a transmission cycle, two or more of the plurality of flits being directed to a first virtual channel, one of the plurality of flits being directed to a second virtual channel; copying the two or more flits to a first buffer associated with the first virtual channel; and copying the flits to a second buffer associated with the second virtual channel.

[0128] In Example 17, the subject of Example 16 includes: copying the two or more flits to the first buffer is part of copying the plurality of flits to the first buffer; and the operation further includes: setting a bitmask associated with the first buffer, the bitmask indicating which of the plurality of flits is directed to the first virtual channel.

[0129] In Example 18, the subject matter of Examples 16 to 17 includes: a processing device configured to perform a second operation comprising: assembling data packets from data in the first buffer, the data packets comprising a set of flits containing the two or more flits and additional flits received on one or more additional transmission cycles, each of the set of flits being directed to the first virtual channel; and causing the hardware transceiver to send a credit return packet to the source, the credit return packet causing a credit return for each transmission cycle on which at least one of the set of flits is received, the credit indicating available buffer space for the source.

[0130] In Example 19, the subject of Example 18 includes the credit return packet indicating the first virtual channel.

[0131] Example 20 is a non-transitory machine-readable medium storing instructions that, when executed by a system, cause the system to perform operations including: receiving a plurality of flow control units (flits) from a source on a transmission cycle, two or more of the plurality of flits being directed to a first virtual channel, one of the plurality of flits being used for a second virtual channel; copying the two or more flits to a first buffer associated with the first virtual channel; and copying the flits to a second buffer associated with the second virtual channel.

[0132] In Example 21, the subject of Example 20 includes: copying the two or more flits to the first buffer is part of copying the plurality of flits to the first buffer; and the operation further includes: setting a bitmask associated with the first buffer, the bitmask indicating which of the plurality of flits is used for the first virtual channel.

[0133] In Example 22, the subject matter of Examples 20 to 21 includes the operation further comprising: assembling data packets from the first buffer, the data packets comprising a set of flits containing the two or more flits and additional flits received on one or more additional transmission cycles, each of the set of flits being directed to the first virtual channel; and sending a credit return packet to the source, the credit return packet causing a credit return for each transmission cycle on which at least one of the set of flits is received, the credit indicating available buffer space for the source.

[0134] In Example 23, the subject of Example 22 includes the credit return packet indicating the first virtual channel.

[0135] Example 24 is a method comprising: receiving, on a transmission cycle, a plurality of flow control units (flits), two or more of the plurality of flits being directed to a first virtual channel, one of the plurality of flits being directed to a second virtual channel; copying the two or more flits to a first buffer associated with the first virtual channel; and copying the flits to a second buffer associated with the second virtual channel.

[0136] In Example 25, the subject of Example 24 includes: copying the two or more flits to the first buffer is part of copying the plurality of flits to the first buffer; and the method further includes: setting a bitmask associated with the first buffer, the bitmask indicating which of the plurality of flits is directed to the first virtual channel.

[0137] In Example 26, the subject matter of Examples 24 to 25 includes: a data packet assembled by a processing device from the first buffer, the data packet comprising a set of flits containing the two or more flits and additional flits received on one or more additional transmission cycles, each of the set of flits being directed to the first virtual channel; and a credit return packet caused by the processing device to be sent to the source by a hardware transceiver, the credit return packet causing a credit return for each transmission cycle on which at least one of the set of flits is received, the credit indicating available buffer space of the source.

[0138] In Example 27, the subject of Example 26 includes the credit return packet indicating the first virtual channel.

[0139] Example 28 is at least one machine-readable medium containing instructions that, when executed by a processing circuitry system, cause the processing circuitry system to perform operations to implement any of Examples 1 to 27.

[0140] Example 29 is an apparatus that includes components for implementing any of Examples 1 through 27.

[0141] Example 30 is a system for implementing any of Examples 1 through 27.

[0142] Example 31 is a method for implementing any of Examples 1 through 27.

Claims

1. A system for flow control, comprising: Logic components configured to perform operations including: Multiple flow control unit flits are transmitted to the destination on the transmission cycle, and two or more of the multiple flits are part of a packet directed to the first virtual channel to the destination; and Based on two or more of the plurality of flits sent on the transmission cycle and being part of the packet directed to the first virtual channel, the data representing the number of credits is modified by reducing the number of credits in the first virtual channel to the destination by 1.

2. The system according to claim 1, wherein: One of the plurality of flits is directed to the second virtual channel of the destination; and The operation further includes: Based on the flit being directed to the second virtual channel, the second number of credits of the second virtual channel at the destination is reduced by 1.

3. The system according to claim 1, wherein the operation further comprises: Before transmitting the plurality of flits, the packet is determined to be sent based on the number of credits of the first virtual channel at the destination, the count of the plurality of flits, and the count of flits in the packet.

4. The system according to claim 3, wherein: The plurality of flits includes one or more flits directed to the second virtual channel; and The determination to send the packet is further based on the count of the one or more flits of the second virtual channel.

5. The system according to claim 1, wherein: At least one of the flits of the first virtual channel leading to the destination includes an identifier of the first virtual channel.

6. A non-transitory machine-readable medium storing instructions, which, when executed by a system, cause the system to perform operations including: Multiple flow control unit fragments are transmitted to the destination on a transmission cycle, two or more of which are portions of packets directed to the first virtual channel to the destination; and Based on two or more of the plurality of flits sent on the transmission cycle and being part of the packet directed to the first virtual channel, the data representing the number of credits is modified by reducing the number of credits in the first virtual channel to the destination by 1.

7. The non-transitory machine-readable medium according to claim 6, wherein: One of the plurality of flits is directed to the second virtual channel; and The operation further includes: Based on the flit being directed to the second virtual channel, the second number of credits of the second virtual channel at the destination is reduced by 1.

8. The non-transitory machine-readable medium of claim 6, wherein the operation further comprises: Before transmitting the plurality of flits, the packet is determined to be sent based on the number of credits of the first virtual channel at the destination, the count of the plurality of flits, and the count of flits in the packet.

9. The non-transitory machine-readable medium according to claim 8, wherein: The plurality of flits includes one or more flits directed to the second virtual channel; and The determination to send the packet is further based on the count of the one or more flits directed to the second virtual channel.

10. The non-transitory machine-readable medium according to claim 6, wherein: At least one of the flits of the first virtual channel leading to the destination includes an identifier of the first virtual channel.

11. A method for flow control, comprising: Multiple flow control unit flits are transmitted to the destination on the transmission cycle, and two or more of the multiple flits are part of a packet directed to the first virtual channel to the destination; and Based on two or more of the plurality of flits sent on the transmission cycle and being part of the packet directed to the first virtual channel, the data representing the number of credits is modified by reducing the number of credits in the first virtual channel to the destination by 1.

12. The method according to claim 11, wherein: One of the plurality of flits is directed to the second virtual channel; and The method further includes: Based on the flit being directed to the second virtual channel, the second number of credits of the second virtual channel at the destination is reduced by 1.

13. The method of claim 11, wherein the method further comprises: Before transmitting the plurality of flits, the packet is determined to be sent based on the number of credits of the first virtual channel at the destination, the count of the plurality of flits, and the count of flits in the packet.

14. The method of claim 13, wherein: The plurality of flits includes one or more flits directed to the second virtual channel; and The determination to send the packet is further based on the count of the one or more flits of the second virtual channel.

15. The method according to claim 11, wherein: At least one of the flits of the first virtual channel leading to the destination includes an identifier of the first virtual channel.

Images