Packet arbitration for buffered packets in network devices

Abstract

Apparatuses and techniques for packet arbitration of buffered packets in a network device are described herein. A packet can be received at an input of the network device. The packet can be placed in a buffer for the input, and a characteristic of the packet can be obtained. A record for the packet is written into a data structure that is independent of the buffer, the record including the characteristic. Based on the characteristic of the packet in the record, arbitration can then be performed among multiple packets to select a next packet from the buffer to pass to an output.

Description

[0001] Priority of application

[0002] This application claims priority to U.S. Application Serial No. 17 / 074,988, filed October 20, 2020, which is incorporated herein by reference in its entirety. Background Technology

[0003] Chiplets are an emerging technology for integrating various processing functionalities. Generally, a chiplet system consists of discrete modules (each referred to as a "chiplet") integrated on an interposer layer and, in many instances, interconnected via one or more established networks to provide the desired functionality to the system. The interposer layer and its contained chiplets can be packaged together for easy interconnection with other components of a larger system. Each chiplet may contain one or more individual integrated circuits (ICs) or "chips," which may be combined with discrete circuit components and coupled to a corresponding substrate for attachment to the interposer layer. Most or all of the chiplets in the system will be individually configured to communicate via one or more established networks.

[0004] Chipsets, configured as individual modules within a system, differ from a system implemented on a single chip containing different device blocks (e.g., intellectual property (IP) blocks) on a substrate (e.g., a single die), such as a system-on-a-chip (SoC) or multiple discrete packaged devices integrated on a printed circuit board (PCB). Generally, chiplets offer better performance (e.g., lower power consumption, reduced latency, etc.) than discrete packaged devices, and offer greater manufacturing benefits than a single die chip. These manufacturing benefits may include higher yields or reduced development costs and time.

[0005] Chiplet systems can include, for example, one or more application (or processor) chiplets and one or more support chiplets. Here, the distinction between application chiplets and support chiplets is merely a reference to possible design scenarios for chiplet systems. Thus, for example, a synthetic vision chiplet system may include (by example only) application chiplets for generating synthetic vision output, and support chiplets such as memory controller chiplets, sensor interface chiplets, or communication chiplets. In typical use cases, synthetic vision designers can design the application chiplets and obtain support chiplets from other sources. Therefore, design costs (e.g., in terms of time or complexity) are reduced by avoiding the design and manufacture of the functionality embodied in the support chiplets. Chiplets also support the tight integration of IP blocks that might otherwise be difficult, such as IP blocks manufactured using different processing technologies or with different feature sizes (or utilizing different contact technologies or pitches). Therefore, multiple ICs or IC assemblies with different physical, electrical, or communication characteristics can be assembled in a modular manner to provide a combination that implements the desired functionality. Chiplet systems also facilitate adaptation to the needs of different larger systems into which the chiplet system will be incorporated. In examples, ICs or other assemblies can be optimized for power, speed, or heat generation for specific functions, as may be the case with sensors. Compared to attempting integration with other devices on a single die, these ICs or other assemblies can be integrated much more easily. Furthermore, by reducing the overall size of the die, the yield of small chips often exceeds that of more complex single-die devices. Attached Figure Description

[0006] This disclosure will be more fully understood from the detailed description given below and the accompanying drawings of various embodiments thereof. However, the drawings should not be construed as limiting this disclosure to the specific embodiments, but are for explanation and understanding only.

[0007] Figure 1A and 1B An example of a chiplet system according to an embodiment is described.

[0008] Figure 2 This describes the components of an example of a memory controller chiplet according to an embodiment.

[0009] Figure 3 This describes the processing flow through the packet buffer according to an embodiment.

[0010] Figure 4 This is a flowchart illustrating an example of a method for packet arbitration of buffered packets in a network device according to an embodiment.

[0011] Figure 5 This is a block diagram of an instance of a machine, which embodiments of this disclosure may utilize, operate in, or be performed via. Detailed Implementation

[0012] Figure 1, described below, provides an example of a chiplet system and components operating therein. The chiplet system can use packet-based networking to communicate between chiplets (e.g., the chiplet protocol interface described below). This packet-based approach allows for flexible and efficient tolerances to varying response time delays between chiplets with different implementations or to their relative positions within the chiplet system. Generally, such packet-based network interfaces implement buffers on the input interfaces to store received packets before they are serviced by the output ports of, for example, a network device. When multiple inputs or multiple outputs are present, the network device may include an arbitrator to pass packets from inputs to outputs. A simple arbitrator implementation may serve the input buffers in a polling manner, sequentially retrieving packets (or decoding packet information) from the header of each input buffer and passing them to the appropriate output.

[0013] More sophisticated arbiter implementations may consider packet characteristics, such as size, destination output port, etc., and select packets from the header of the input buffer based on these characteristics to prevent wasted loops, for example, when contention exists at the output port. However, arbiter implementations that store packets in buffers and arbitrate reads from these buffers typically only arbitrate packets visible at the header of said buffer (e.g., the output). Since packets are deeper in a given input buffer, this strategy can still lead to contention at the output.

[0014] To improve arbitrator performance, the arbitrator can be configured to consider more packets than are visible at the output of the input buffer. This is achieved by maintaining a packet feature data structure that stores the packets within the input buffer. The arbitrator then selects from possible packets to remove from the input buffer and passes the packet features to the output. Recording packet information related to reading the arbitration in a separate structure allows the arbitrator to consider more packets. The arbitrator can use information such as packet size and position within the buffer to limit packets and avoid collisions when reading packets from the buffer. An arbitrator that knows more packets than those only at the head of the input buffer can improve efficiency and reduce latency. Additional details and examples are provided below.

[0015] Figure 1A and 1B An example of a chiplet system 110 according to an embodiment is described. Figure 1AThis is an illustration of a chiplet system 110 mounted on a peripheral board 105, which can be connected to a wider range of computer systems, for example, via peripheral component interconnect (PCIe). The chiplet system 110 includes a package substrate 115, an interposer 120, and four chips: an application chiplet 125, a host interface chiplet 135, a memory controller chiplet 140, and a memory device chiplet 150. Other systems may include numerous additional chipsets to provide additional functionality, as will be apparent from the following discussion. The package of the chiplet system 110 is illustrated with a cap or cover plate 165, but other packaging technologies and structures used for chiplet systems may be used. Figure 1B This is a block diagram for the purpose of clearly labeling the components in a chiplet system.

[0016] Application chip 125 is described as including an on-chip network (NOC) 130 to support a chiplet network 155 for inter-chiplet communication. In an example embodiment, NOC 130 may be included on application chip 125. In an example, NOC 130 may be defined in response to selected supporting chips (e.g., chips 135, 140, and 150), thus enabling designers to select an appropriate number of chiplet network connections or switches for NOC 130. In an example, NOC 130 may reside on a single chiplet or even within interposer 120. In the example discussed herein, NOC 130 implements a chiplet protocol interface (CPI) network.

[0017] CPI is a packet-based network that supports virtual channels to enable flexible and high-speed interaction between chiplets. CPI bridges the chiplet-internal network to chiplet network 155. For example, the Advanced Extensible Interface (AXI) is a widely used specification for designing intra-chip communication. However, the AXI specification covers a large number of physical design options, such as the number of physical channels, signal timing, and power. Within a single chip, these options are typically selected to meet design goals such as power consumption and speed. However, to achieve flexibility in chiplet systems, adapters such as CPI are used to intersect between various AXI design options that can be implemented in various chiplets. By implementing a mapping from physical channels to virtual channels and encapsulating time-based signaling using packetization protocols, CPI bridges the intra-chiplet network 155.

[0018] CPI can utilize various physical layers to transmit packets. A physical layer may contain simple conductive connections, or it may contain drivers to increase voltage, or otherwise facilitate signal transmission over longer distances. An example of such a physical layer may include an Advanced Interface Bus (AIB), which in various instances may be implemented in intermediate layer 120. The AIB uses source-synchronous data transfer with a forwarding clock to transmit and receive data. Packets are transmitted across the AIB at Single Data Rate (SDR) or Double Data Rate (DDR) relative to the transmitted clock. The AIB supports various channel widths. When operating in SDR mode, the AIB channel width is a multiple of 20 bits (20, 40, 60, ...), and for DDR mode, the AIB channel width is a multiple of 40 bits (40, 80, 120, ...). The AIB channel width includes both transmitted and received signals. Channels can be configured with a symmetrical number of transmit (TX) and receive (RX) inputs / outputs (I / O), or with an asymmetrical number of transmitters and receivers (e.g., all transmitters or all receivers). The channel can act as either AIB master or slave depending on which chip provides the primary clock. The AIB I / O unit supports three clock modes: asynchronous (i.e., non-timed), SDR, and DDR. In various instances, the non-timed mode is used for clocking and some control signals. SDR mode can use a dedicated SDR-only I / O unit or a dual-purpose SDR / DDR I / O unit.

[0019] In this example, the CPI packet protocol (e.g., point-to-point or routable) can use symmetrical receive and transmit I / O units within an AIB channel. The CPI streaming protocol allows for more flexible utilization of AIB I / O units. In this example, the streaming mode AIB channel can be configured with all TX, all RX, or half TX and half RX. The CPI packet protocol can use AIB channels in SDR or DDR operating modes. In this example, the AIB channel is configured in increments of 80 I / O units (i.e., 40 TX and 40 RX) for SDR mode and in increments of 40 I / O units for DDR mode. The CPI streaming protocol can use AIB channels in SDR or DDR operating modes. Here, in this example, the AIB channel is configured in increments of 40 I / O units for both SDR and DDR modes. In this example, a unique interface identifier is assigned to each AIB channel. This identifier is used during CPI reset and initialization to determine paired AIB channels across neighboring chiplets. In this example, the interface identifier is a 20-bit value comprising a seven-bit chiplet identifier, a seven-bit column identifier, and a six-bit link identifier. The AIB physical layer uses an AIB out-of-band shift register to transmit the interface identifier. Bits 32 to 51 of the shift register are used to transmit the 20-bit interface identifier in both directions across the AIB interface.

[0020] AIB defines a stacked group of AIB channels as an AIB channel column. An AIB channel column has a certain number of AIB channels, plus an auxiliary (AUX) channel. The auxiliary channel contains signals used for AIB initialization. All AIB channels within a column (except the auxiliary channel) have the same configuration (e.g., all TX, all RX, or half TX and half RX, and the same number of data I / O signals). In this example, AIB channels are numbered sequentially in ascending order, starting with the AIB channel adjacent to the AUX channel. The AIB channel adjacent to the AUX channel is defined as AIB channel zero.

[0021] Generally, the CPI interface on an individual chiplet may include serialization-deserialization (SERDES) hardware. SERDES interconnects are well-suited for scenarios requiring high-speed signaling and low signal counts. However, SERDES can introduce additional power consumption and longer latency for multiplexing and demultiplexing, error detection or correction (e.g., using block-level cyclic redundancy check (CRC)), link-level retries, or forward error correction. However, when low latency or power consumption is a primary concern for ultra-short-distance chiplet-to-chiplet interconnects, parallel interfaces that allow data transmission with minimal latency can be utilized. CPIs contain elements designed to minimize both latency and power consumption in these ultra-short-distance chiplet interconnects.

[0022] For flow control, CPI employs a credit-based technique. For example, the sender side of the application chip 125, such as the memory controller chip 140, provides credits indicating available buffers. In this example, the CPI receiver contains buffers for each virtual channel for a given transmission time unit. Therefore, if the CPI receiver supports five messages and a single virtual channel in time, the receiver has five buffers arranged in five rows (e.g., one row per unit time). If four virtual channels are supported, the receiver has twenty buffers arranged in five rows. Each buffer holds the payload of one CPI packet.

[0023] When a sender transmits data to a receiver, the sender decrements its available credits based on the transmission. Once all of the receiver's credits are exhausted, the sender stops sending packets to the receiver. This ensures that the receiver always has an available buffer to store transmissions.

[0024] When the receiver processes the received packet and releases the buffer, it relays the available buffer space back to the sender. The sender can then use this credit to allow the transmission of additional information.

[0025] It also describes a chiplet mesh network 160 that uses direct chiplet-to-chiplet technology without requiring NOC 130. The chiplet mesh network 160 can be implemented in CPI or another chiplet-to-chiplet protocol. The chiplet mesh network 160 typically implements a chiplet pipeline, where one chiplet acts as an interface to the pipeline, while other chipslets in the pipeline interface only interface with themselves.

[0026] Additionally, dedicated device interfaces, such as one or more industry-standard memory interfaces 145 (e.g., synchronous memory interfaces, such as DDR5, DDR6), can also be used to interconnect chiplets. Connections from chiplet systems or individual chiplets to external devices (e.g., larger systems can be connected via the desired interface (e.g., a PCIe interface)). In an example, this external interface can be implemented via a host interface chiplet 135, which, in the depicted example, provides a PCIe interface external to the chiplet system 110. Such interfaces are typically used when industry practice or standards have converged on them. The illustrated example of connecting the memory controller chiplet 140 to the dual data rate (DDR) interface 145 of the dynamic random access memory (DRAM) memory device 150 is merely one such industry practice.

[0027] Among the various possible supporting chiplets, the memory controller chiplet 140 is likely to be present in the chiplet system 110 due to the ubiquitous use of memory for computer processing and the application of advanced technologies to memory devices. Therefore, using the memory device chiplet 150 and the memory controller chiplet 140, both designed by other designers, allows chiplet system designers to obtain robust products manufactured by established companies. Typically, the memory controller chiplet 140 provides a memory device-specific interface for reading, writing, or erasing data. Typically, the memory controller chiplet 140 can provide additional features such as error detection, error correction, maintenance operations, or atomic operator execution. For some types of memory, maintenance operations are often specific to the memory device 150, such as garbage collection in NAND flash or storage-class memory, or temperature regulation (e.g., cross-temperature management) in NAND flash memory. In instances, maintenance operations may involve logic-to-physical (L2P) mapping or management to provide an indirection hierarchy between the physical and logical representations of data. In other types of memory, such as DRAM, some memory operations, such as refresh, may be controlled at some times by the host processor or memory controller, and at other times by the DRAM memory device or logic associated with one or more DRAM devices, such as interface chips (in this example, buffers).

[0028] Atomic operators are data manipulations that can be performed, for example, by memory controller chiplet 140. In other chiplet systems, atomic operators can be performed by other chipsets. For example, an application chiplet 125 can specify an "increment" atomic operator in a command that includes a memory address and may include an increment value. Upon receiving the command, memory controller chiplet 140 retrieves a number from the specified memory address, increments the number by the amount specified in the command, and stores the result. Upon successful completion, memory controller chiplet 140 provides an indication of command success to application chiplet 125. Atomic operators avoid data transfer across chiplet network 160, thereby reducing latency in executing such commands.

[0029] Atomic operators can be classified as built-in atoms or programmable (e.g., custom) atoms. Built-in atoms are a finite set of operations implemented invariantly in the hardware. Programmable atoms are applets that can be executed on programmable atom units (PAUs) (e.g., custom atom units (CAUs)) of the memory controller chiplet 140. Figure 1 illustrates an example of a memory controller chiplet with a PAU.

[0030] The memory device chiplet 150 may be or contain any combination of volatile memory devices or non-volatile memory. Examples of volatile memory devices include, but are not limited to, random access memory (RAM), such as DRAM, synchronous DRAM (SDRAM), graphics dual data rate type 6 SDRAM (GDDR6 SDRAM), etc. Examples of non-volatile memory devices include, but are not limited to, NAND flash memory, memory-class memory (e.g., phase-change memory or memristor-based technology), ferroelectric RAM (FeRAM), etc. The illustrated example includes the memory device 150 as a chiplet; however, the memory device 150 may reside elsewhere, such as in different packages on peripheral board 105. For many applications, multiple memory device chipsets may be provided. In the example, these memory device chipsets may each implement one or more memory technologies. In the example, the memory chiplet may contain multiple stacked memory dies of different technologies, such as one or more static random access memory (SRAM) devices stacked with or otherwise communicating with one or more dynamic random access memory (DRAM) devices. The memory controller 140 can also be used to coordinate the operation between multiple memory chips in the chiplet system 110; for example, utilizing one or more memory chips in one or more tiers of cache memory and using one or more additional memory chips as main memory. The chiplet system 110 may also include multiple memory controllers 140, which can be used to provide memory control functionality for individual processors, sensors, networks, etc. For example, the chiplet architecture of the chiplet system 110 offers the advantage of allowing for adaptation to different memory storage technologies and different memory interfaces through updated chiplet configurations, without requiring redesign of the rest of the system architecture.

[0031] Figure 2The components of an example of a memory controller chiplet 205 according to an embodiment are described below. The memory controller chiplet 205 includes a cache 210, a cache controller 215, an off-die memory controller 220 (e.g., for communicating with off-die memory 275), a network communication interface 225 (e.g., for interfacing with a chiplet network 285 and communicating with other chiplets), and a set of atomic and merging units 250. Members of this set may include, for example, a write merging unit 255, a memory dangerous unit 260, a built-in atomic unit 265, or a PAU 270. The various components are described logically, and they may not necessarily be implemented. For example, a built-in atomic unit 265 may include different means along the path to off-die memory. For example, a built-in atomic unit 265 may be in an interface means / buffer on the memory chiplet, as discussed above. In contrast, a programmable atomic unit 270 may be implemented in a separate processor on the memory controller chiplet 205 (but in various instances, it may be implemented in other locations, such as on the memory chiplet itself).

[0032] The off-die memory controller 220 is directly coupled to the off-die memory 275 (e.g., via a bus or other communication connection) to provide write operations to and read operations from the one or more off-die memories, such as off-die memory 275 and off-die memory 280. In the depicted example, the off-die memory controller 220 is also coupled to outputs to the atom and merge unit 250 and inputs to the cache controller 215 (e.g., a memory-side cache controller).

[0033] In the instance configuration, the cache controller 215 is directly coupled to the cache 210 and can be coupled to the network communication interface 225 for input (e.g., incoming read or write requests) and coupled to the off-die memory controller 220.

[0034] Network communication interface 225 includes packet decoder 230, network input queue 235, packet encoder 240, and network output queue 245 to support packet-based chiplet network 285, such as CPI. Chiplet network 285 can provide packet routing between and within processors, memory controllers, mixed-thread processors, configurable processing circuitry, or communication interfaces. In such packet-based communication systems, each packet typically contains destination and source addressing, as well as any data payload or instructions. In some instances, depending on the configuration, chiplet network 285 may be implemented as a collection of crossbar switches with a folded Clos configuration, or as a mesh network providing additional connectivity.

[0035] Network communication interface 225 can be configured to implement packet arbitration of buffered packets. In this context, memory controller 205 can be considered a network device. The same mechanism can also be applied to any chiplet in a chiplet system, such as application chiplet 125 in chiplet system 110, or to a dedicated network device, such as NOC 130. In an example of an NOC, the network device can be a switch. However, for simplicity, network communication interface 225 is used as an example network device for the described packet arbitration technique.

[0036] Network communication interface 225 is configured to receive packets at the input. As illustrated, the input is connected to chiplet network 285 and passes packets to input queue 235. As described above in Figure 1, the CPI to AXI bridge may contain multiple inputs from a single hardware connection via virtual channels. Therefore, packets specify virtual channels, which are treated as distinct hardware inputs once received. In this type of arrangement, network input queue 235 may contain multiple buffers, with at least one buffer for each virtual input.

[0037] Network communication interface 225 is configured to place received packets in a buffer for input. Therefore, when a packet for input A is received, buffer A is used to store the packet. In an example, the buffer is implemented as a first-in, first-out (FIFO) queue, with packets inserted (e.g., pushed) at the tail of the buffer and removed (popped) from the head. Actual hardware implementations can vary using, for example, head and tail pointers of contiguous regions of random access memory. Multiple buffers can be implemented in memory, kept separate by constraints on the addresses used to access the memory. This arrangement can also be effective because packets do not need to move within memory toward the head of the queue. Instead, as the queue pointer moves, the relative position of any given packet moves without copying the packet to a new memory address. However, other buffer implementations can also be used.

[0038] Network communication interface 225 is configured to obtain packet characteristics, for example, through a packet analyzer circuitry. To obtain packet characteristics, network communication interface 225 may read information from a packet (e.g., its destination), measure a packet (e.g., its size), or record the packet's context (e.g., the position the packet occupies in the input buffer, or the time the packet spends in the buffer). The specific packet characteristics obtained by network communication interface 225 can vary widely, but given other packets being arbitrated, information related to addressing and arbitrating the packet to the output is typically considered. Therefore, the size and position of the packet help determine how long a given output will be unavailable (e.g., how many clock cycles) when the packet is passed to the output. In this example, for different values ​​of, for example, packet context (e.g., its position in the buffer), network communication interface 225 is configured to continuously monitor (e.g., by polling or in an event-driven manner) the context and obtain updated packet characteristics as they become available.

[0039] Network communication interface 225 is configured to write packet records into a data structure, for example, via a packet analyzer circuitry. The written record contains packet characteristics obtained by network communication interface 225. This data structure may be called ArbInfo because it contains information related to arbitration. In this instance, the data structure is independent of the buffer. Maintaining the data structure separately from the buffer allows for a simpler or more flexible design for both the buffer and the data structure. In this instance, when packet characteristics are dynamic, such as the packet position in the buffer, network communication interface 225 is configured to update the packet record in the data structure as updated information about the packet becomes available. In this instance, the record contains multiple characteristics of the packet. Therefore, a single record is used to store multiple packet characteristics for a single packet.

[0040] Network communication interface 225 is configured to perform arbitration among multiple packets to select the next packet from the buffer for delivery to the output. Arbitration can be performed by a dedicated arbitrator circuitry system (e.g., Figure 3Arbitrator 330, as described herein, is used. Arbitration is based on (e.g., consider) one or more characteristics of packets written in a packet record stored in a data structure. In an example, when the record contains multiple packet characteristics, arbitration among multiple packets to select the next packet from the buffer is based on multiple characteristics of the packets stored in the record in the data structure. Therefore, in this example, more than one characteristic, or all characteristics, are used in the arbitration. Primarily, arbitration attempts to reduce contention on the output to prevent read stalling of the input buffers. Given three input buffers, an example of packet characteristic-based arbitration could include a packet at the head of buffer A going to output Z and being larger, a packet at the head of buffer B going to output Y and a second packet in buffer B going to output Z, and a packet at the head of buffer C going to output Y. Here, the arbitrator could select the head of buffer A and skip buffer B to select the head of buffer C on the next clock cycle, because buffer B would stall on the second packet waiting for buffer A to finish reading. Alternatively, if the first two packets of buffer B are smaller, you can choose to pass them to outputs Z and Y from buffer B, and then pass the larger packets from buffers A and C to these outputs.

[0041] In this example, a data structure is one of a set of data structures corresponding to a buffer and used to store characteristics of the buffered packets considered in arbitration. In this example, the cardinality (e.g., number) of the data structure set is equal to the number of multiple packets considered. Therefore, if a packet from multiple buffers is considered in each round of arbitration, there is one data structure for each buffer. In this example, when multiple packets come from different buffers, the next packet is the queue head packet for the buffer. Here, the only packet the arbitrator can read from a given buffer is at the head of the buffer. However, deeper buffer contexts (e.g., packets not at the head of the buffer) are considered when deciding which buffer head to read and pass to the output. While it is possible to remove out-of-order packets from the buffer, this typically complicates arbitration and may require additional circuitry to prevent packets from remaining in the buffer for too long. In this example, the cardinality of the data structure set is equal to the number of virtual queues in the buffer. This example considers a single buffer with virtual sub-buffers, for example, based on virtual channels existing in many CPI packets. In this example, the number of virtual queues is equal to the number of buffer outputs in the network device. This example illustrates a structure that facilitates CPI to AXI bridging, where the virtual channel is related to the hardwired channel in chiplet communication technology.

[0042] In various instances, the chiplet network 285 may be part of an asynchronous switching architecture. Here, data packets may be routed along any of various paths, such that any selected data packet may arrive at the addressed destination at any time among multiple different times, depending on the route. Furthermore, the chiplet network 285 may be implemented at least partially as a synchronous communication network, such as a synchronous mesh communication network. Two configurations of the communication network are considered, for example, according to this disclosure.

[0043] The memory controller chip 205 can receive packets having, for example, a source address, a read request, and a physical address. In response, the off-die memory controller 220 or the cache controller 215 reads data from the specified physical address (which may be in off-die memory 275 or cache 210) and assembles a response packet into a source address containing the requested data. Similarly, the memory controller chip 205 can receive packets having a source address, a write request, and a physical address. In response, the memory controller chip 205 writes data to the specified physical address (which may be in cache 210 or off-die memory 275 or 280) and assembles a response packet into a source address containing confirmation that data has been stored in memory.

[0044] Therefore, where possible, the memory controller chiplet 205 can receive read and write requests via chiplet network 285 and process the requests using cache controller 215, which interfaces with cache 210. If cache controller 215 cannot process the request, off-die memory controller 220 processes the request by communicating with off-die memory 275 or 280, atom and merging unit 250, or both. As described above, one or more levels of cache can also be implemented in off-die memory 275 or 280; and in some such instances, they can be directly accessed by cache controller 215. Data read by off-die memory controller 220 can be cached in cache 210 by cache controller 215 for later use.

[0045] Atom and merging unit 250 is coupled to receive (as input) the output of off-die memory controller 220 and provides the output to cache 210, network communication interface 225, or directly to chiplet network 285. Memory hazard unit 260, write merging unit 255, and built-in (e.g., predetermined) atom unit 265 can each be implemented as a state machine with other combinational logic circuitry (e.g., adders, shifters, comparators, AND gates, OR gates, XOR gates, or any suitable combination thereof) or other logic circuitry. These components may also include one or more registers or buffers to store operands or other data. PAU 270 can be implemented as one or more processor cores or control circuitry, and various state machines with other combinational logic circuitry or other logic circuitry, and may also include one or more registers, buffers, or memories to store addresses, executable instructions, operands, and other data, or may be implemented as a processor.

[0046] The write merging unit 255 receives read data and request data, and merges the request data and read data to produce a single unit containing the read data and the source address to be used in the response or return data packet. The write merging unit 255 provides the merged data to the write port of cache 210 (or equivalently, to cache controller 215 for writing to cache 210). Optionally, the write merging unit 255 provides the merged data to network communication interface 225 to encode and prepare response or return data packets for transmission on chiplet network 285.

[0047] When requested data is used for a built-in atomic operator, the built-in atomic unit 265 receives the request and reads the data from the write-merge unit 255 or directly from the off-chip memory controller 220. The atomic operator is executed, and the resulting data is written to cache 210 using write-merge unit 255, or provided to network communication interface 225 to encode and prepare response or return packets for transmission on chiplet network 285.

[0048] Built-in atomic unit 265 processes predefined atomic operators, such as fetch and increment or compare and swap. In examples, these operations perform simple read-modify-write operations on a single memory location of 32 bytes or less. Atomic memory operations are initiated from request packets transmitted via chiplet network 285. The request packet has a physical address, atomic operator type, operand size, and optionally up to 32 bytes of data. The atomic operators perform read-modify-write operations on cache lines of cache 210, thereby filling the cache memory as needed. The atomic operator response can be a simple complete response or a response with up to 32 bytes of data. Example atomic memory operators include fetch and AND, fetch and OR, fetch and XOR, fetch and add, fetch and subtract, fetch and increment, fetch and decrement, fetch and minimum, fetch and maximum, fetch and swap, and compare and swap. In various example embodiments, 32-bit and 64-bit operations and operations on 16 or 32 bytes of data are supported. The methods disclosed herein are also compatible with hardware that supports larger or smaller operations and more or less data.

[0049] Built-in atomic operators may also involve requests for "standard" atomic operators on the requested data, such as relatively simple single-loop integer atoms, such as fetch and increment or compare and swap, whose throughput will be the same as a regular memory read or write operation that does not involve atomic operators. For these operations, cache controller 215 can typically preserve cache lines in cache 210 by setting a danger bit (in hardware) so that the cache line cannot be read by another process during conversion. Data is obtained from off-chip memory 275 or cache 210 and provided to built-in atomic unit 265 to execute the requested atomic operator. After the atomic operator, in addition to providing the obtained data to packet encoder 240 to encode outgoing packets for transmission on chiplet network 285, built-in atomic unit 265 also provides the obtained data to write merging unit 255, which writes the obtained data back to cache 210. After the obtained data is written to cache 210, memory danger unit 260 clears any corresponding danger bits that were set.

[0050] The PAU 270 implements high-performance (high throughput and low latency) programmable atomic operators (also known as "custom atomic transactions" or "custom atomic operators"), comparable to the performance of built-in atomic operators. Instead of performing multiple memory accesses, in response to an atomic operator request specifying a programmable atomic operator and a memory address, the circuitry in the memory controller chiplet 205 transmits the atomic operator request to the PAU 270 and sets a danger bit in a memory danger register corresponding to the memory address used in the atomic operator, ensuring that no other operations (read, write, or atomic) are performed on that memory line. The danger bit is then cleared after the atomic operator completes. The additional direct data path provided to the PAU 270 for executing programmable atomic operators allows for additional write operations without being limited by the bandwidth of the communication network and without increasing any congestion on the communication network.

[0051] The PAU 270 includes, for example, a multi-threaded processor based on a RISC-VIS ISA multi-threaded processor, having one or more processor cores and further having an extended instruction set for executing programmable atomic operators. When equipped with the extended instruction set for executing programmable atomic operators, the PAU 270 can be embodied as one or more hybrid-threaded processors. In some example embodiments, the PAU 270 provides bucket-style polling instantaneous thread switching to maintain a high instruction-per-clock rate.

[0052] Programmable atomic operators can be performed by PAU 270, which relate to requests for programmable atomic operators with respect to requested data. Users can prepare programming code to provide these programmable atomic operators. For example, programmable atomic operators can be relatively simple multi-loop operations, such as floating-point addition, or relatively complex multi-instruction operations, such as Bloom filter insertion. Programmable atomic operators can be the same as or different from predetermined atomic operators because they are defined by the user, not the system vendor. For these operations, cache controller 215 can preserve cache lines in cache 210 by setting a danger bit (in hardware) so that the cache lines cannot be read by another process during transitions. Data is obtained from cache 210 or off-chip memory 275 or 280 and provided to PAU 270 to execute the requested programmable atomic operators. After the atomic operators, PAU 270 provides the resulting data to network communication interface 225 to directly encode outgoing data packets containing the resulting data for transmission on chiplet network 285. Additionally, PAU 270 provides the obtained data to cache controller 215, which then writes the obtained data to cache 210. After writing the obtained data to cache 210, cache control circuit 215 clears any corresponding dangerous bits that were set.

[0053] In the selected example, the approach taken for programmable atomic operators is to provide multiple generic custom atomic request types that can be sent from an originating source, such as a processor or other system component, to the memory controller chiplet 205 via chiplet network 285. Cache controller 215 or off-die memory controller 220 recognizes the request as a custom atom and forwards the request to PAU 270. In a representative embodiment, PAU 270: (1) is a programmable processing element capable of efficiently executing user-defined atomic operators; (2) performs load and store operations on memory, arithmetic and logic operations, and control flow decisions; and (3) utilizes a RISC-V ISA with a new set of dedicated instructions to facilitate interaction with such controllers 215, 220, thereby atomically executing user-defined operations. In desirable examples, the RISC-V ISA contains a complete instruction set supporting high-level language operators and data types. PAU 270 may utilize RISC-V ISA, but typically supports a more limited instruction set and a limited register file size to reduce the die size of the unit when included within the memory controller chiplet 205.

[0054] As mentioned above, before writing the read data to cache 210, the memory danger clearing unit 260 clears the danger bits set in the reserved cache line. Therefore, when the write merging unit 255 receives a request and read data, the memory danger clearing unit 260 can transmit a reset or clear signal to cache 210 to reset the memory danger bits set in the reserved cache line. Furthermore, resetting this danger bit will also release pending read or write requests involving the specified (or reserved) cache line, thereby providing the pending read or write requests to the inbound request multiplexer for selection and processing.

[0055] Figure 3 The processing flow through the packet buffer according to the embodiment is described. As illustrated, packet 310 is received at the input interface 305 of the network device. The packet is buffered (e.g., placed in buffer 315). As illustrated, buffer 315 is divided into three sub-buffers, indicated by different shades. For each sub-buffer, data structures are maintained, as described in data structures 355, 350, and 320. All data structures maintain a record of the packet in buffer 315, such as record 325 for packet 310.

[0056] Arbitrator 330 is implemented as a circuit block in a network device and is communicatively coupled to buffer 315 and data structures 355, 350, and 320 during operation. During a round of arbitration (e.g., in each clock cycle), the arbitrator consults data structures 355, 350, or 320 to determine from which sub-buffer to read a group of packets and deliver it to the appropriate output, such as output interface 335, output interface 340, or output interface 345. The actual technique used by arbitrator 330 for comparison records can vary. For example, arbitrator 330 may sum the packet sizes of the first three packets in each sub-buffer and select the smallest packet. In an example, arbitrator 330 may use a threshold regarding packet size at the beginning of the sub-buffer and only look deeper into the second sub-buffer to determine whether to deliver the packet immediately or wait if the size is higher than the threshold.

[0057] Recording packet information related to reading arbitration in a separate structure allows the arbitrator to consider more packets. For example, information such as packet size and position within the buffer allows the arbitrator to limit packets and avoid collisions when reading packets from the input buffer. This implementation allows the arbitrator reading from the buffer to consider more packets for arbitration, which improves efficiency and reduces latency.

[0058] Figure 4This is a flowchart illustrating an example of a method 400 for packet arbitration of buffered packets in a network device according to an embodiment. The operation of method 400 is controlled by computer hardware, such as the NOC interface in NOC 130 or other chiplets, such as memory controller chiplet 140, as shown in FIG1. Figure 2 (For example, network interface 225 or network input queue 235) or Figure 5 (For example, processing circuitry systems) describes the execution of computer hardware.

[0059] At operation 405, packets are received at the input of the network device.

[0060] At operation 410, the packet is placed in the buffer for input.

[0061] At operation 415, the characteristics of the packet are obtained. In this example, the packet characteristic is the size of the packet, and obtaining the packet characteristic includes measuring the size of the packet. In this example, the packet characteristic is the position of the packet in the buffer.

[0062] At operation 420, a record of the packet is written to a buffer-independent data structure. The written record contains the packet characteristics obtained at operation 415. In this example, when the packet characteristics represent the packet's position within the buffer, the record can be updated to indicate the packet's new position in the buffer in response to movement of the packet within the buffer. Here, movement within the buffer may involve copying the packet from one location in the buffer to another. Alternatively, the packet may remain in the same physical memory device (e.g., a memory block, register, etc.) but move relative to the head or tail pointer of the buffer. In this example, the packet movement is relative to the head of the buffer, but avoids the need for additional memory copying to physically move the packet. In this example, the record contains multiple characteristics of the packet.

[0063] At operation 425, arbitration is performed among multiple packets to select the next packet from the buffer to be passed to the output. Here, arbitration is based on packet characteristics obtained from records in the data structure. In this example, when a record contains multiple packet characteristics, arbitration among multiple packets to select the next packet from the buffer is based on the multiple characteristics of the packets stored in the records in the data structure.

[0064] In this example, a data structure is one of a set of data structures corresponding to a buffer and used to store characteristics of the buffered packets considered in arbitration. In this example, the cardinality of the data structure set is equal to the number of packets. In this example, multiple packets come from different buffers, and the next packet is a queue header packet for the buffer. The cardinality of the data structure set is equal to the number of virtual queues in the buffer. In this example, the number of virtual queues is equal to the number of buffer outputs in the network device.

[0065] Figure 5This illustrates a block diagram of an example machine 500, which may utilize, be in, or be operated by any one or more of the techniques (e.g., methods) discussed herein. As described herein, an example may contain, or be operated by, logic or components or mechanisms within machine 500. A circuit system (e.g., a processing circuit system) is a collection of circuits implemented in the tangible entity of machine 500, comprising hardware (e.g., simple circuits, gates, logic, etc.). The membership of a circuit system may be flexible over time. A circuit system includes members that can perform specific operations individually or in combination during operation. In an example, the hardware of a circuit system may be designed in an immutable manner to perform a specific operation (e.g., hardwiring). In an example, the hardware of a circuit system may include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.) that include machine-readable media that are physically modified (e.g., invariant magnetic properties of concentrated particles, electrically movable placement, etc.) to encode instructions for a specific operation. When connecting physical components, the fundamental electrical properties of the hardware components are altered, for example, from an insulator to a conductor, or vice versa. Instructions enable embedded hardware (e.g., an execution unit or loading mechanism) to hardware-create components of a circuit system via variable connections to perform specific operations when in operation. Thus, in an example, a machine-readable media element is part of the circuit system or communicatively coupled to other components of the circuit system during device operation. In an example, any one of the physical components can be used in more than one component of more than one circuit system. For example, in operation, an execution unit can be used at one point in time for a first circuit in a first circuit system and reused at different times by a second circuit in the first circuit system, or by a third circuit in the second circuit system. Further examples of these components of machine 500 are given below.

[0066] In alternative embodiments, machine 500 may act as a standalone device or be connected (e.g., networked) to other machines. In a networked deployment, machine 500 may operate as a server machine, a client machine, or both in a server-client network environment. In an example, machine 500 may act as a peer-to-peer (P2P) (or other distributed) network environment. Machine 500 may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, network appliance, network router, switch, or bridge, or any machine capable of executing (sequentially or otherwise) instructions specifying actions to be taken by said machine. Furthermore, while only a single machine is described, the term "machine" should also be considered as encompassing any collection of machines that individually or jointly execute one or more sets of instructions to perform any one or more of the methodologies discussed herein, such as cloud computing, Software as a Service (SaaS), and other computer cluster configurations.

[0067] Machine (e.g., computer system) 500 may include a hardware processor 502 (e.g., a central processing unit (CPU), graphics processing unit (GPU), hardware processor core, or any combination thereof), main memory 504, static memory (e.g., memory or storage device for firmware, microcode, basic input / output (BIOS), unified extensible firmware interface (UEFI), etc.) 506, and mass storage device 508 (e.g., hard disk drive, tape drive, flash memory, or other block device), some or all of which may communicate with each other via interconnect (e.g., bus) 530. Machine 500 may further include a display unit 510, an alphanumeric input device 512 (e.g., keyboard), and a user interface (UI) navigation device 514 (e.g., mouse). In an example, the display unit 510, input device 512, and UI navigation device 514 may be a touchscreen display. Machine 500 may additionally include a storage device (e.g., a drive unit) 508, a signal generation device 518 (e.g., a speaker), a network interface device 520, and one or more sensors 516, such as a Global Positioning System (GPS) sensor, a compass, an accelerometer, or other sensors. Machine 500 may include an output controller 528, which is connected, for example, serially (e.g., Universal Serial Bus (USB)), in parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) to communicate with or control one or more peripheral devices (e.g., a printer, a card reader, etc.).

[0068] The registers of processor 502, main memory 504, static memory 506, or mass storage device 508 may be or contain a machine-readable medium 522 on which one or more sets of data structures or instructions 524 (e.g., software) are stored, said sets of data structures or instructions embodying, or being utilized by, any one or more of the techniques or functions described herein. Instructions 524 may also reside wholly or at least partially in any of the registers of processor 502, main memory 504, static memory 506, or mass storage device 508 during execution by machine 500. In an example, one or any combination of hardware processor 502, main memory 504, static memory 506, or mass storage device 508 may constitute machine-readable medium 522. While machine-readable medium 522 is described as a single medium, the term "machine-readable medium" may include a single medium or multiple media (e.g., a centralized or distributed database, or associated cache and server) configured to store one or more instructions 524.

[0069] The term "machine-readable media" can include any medium capable of storing, encoding, or transmitting instructions for execution by machine 500 and causing machine 500 to perform any or more of the technologies disclosed herein, or any medium capable of storing, encoding, or transmitting data structures used by or associated with such instructions. Examples of non-limiting machine-readable media can include solid-state memory, optical media, magnetic media, and signals (e.g., radio frequency signals, other photon-based signals, sound signals, etc.). In examples, non-transitory machine-readable media includes machine-readable media having a plurality of particles having constant (e.g., rest) mass and therefore being composed of matter. Therefore, non-transitory machine-readable media is machine-readable media that does not contain transiently propagating signals. Specific examples of non-transitory machine-readable media can include: non-volatile memory, such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0070] In an example, information stored or otherwise provided on machine-readable medium 522 may represent instructions 524, such as instructions 524 themselves or a format from which instructions 524 can be derived. This format from which instructions 524 can be derived may include source code, encoded instructions (e.g., in compressed or encrypted form), encapsulation instructions (e.g., splitting into multiple encapsulations), etc. The information representing instructions 524 on machine-readable medium 522 may be processed by a processing circuitry system into the instructions to perform any of the operations discussed herein. For example, deriving instructions 524 from information (e.g., through processing circuitry system processing) may include: compiling (e.g., from source code, object code, etc.), interpreting, loading, organizing (e.g., dynamically or statically linking), encoding, decoding, encrypting, decrypting, encapsulating, decapsulating, or otherwise manipulating the information into instructions 524.

[0071] In an example, the derivation of instruction 524 may involve (e.g., by processing a circuit system) assembling, compiling, or decompiling information to create instruction 524 from some intermediate or preprocessed format provided by machine-readable media 522. When information is provided in multiple parts, the information may be combined, decapsulated, and modified to create instruction 524. For example, information may be contained in multiple compressed source code packages (or object code, or binary executable code, etc.) on one or more remote servers. The source code packages may be encrypted when transmitted over a network and, if necessary, decrypted, decompressed, assembled (e.g., linked), and compiled or decompiled at the local machine (e.g., into a stand-alone executable library, etc.) and executed by the local machine.

[0072] Instruction 524 may further utilize any of a plurality of transport protocols (e.g., Frame Relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.) to transmit or receive over communication network 526 via network interface device 520 using a transport medium. Example communication networks may include local area networks (LANs), wide area networks (WANs), packet data networks (e.g., the Internet), mobile phone networks (e.g., cellular networks), simple old-fashioned telephone (POTS) networks, and wireless data networks (e.g., referred to as…). The Institute of Electrical and Electronics Engineers (IEEE) 802.11 series of standards, known as The IEEE 802.16 series of standards, the IEEE 802.15.4 series of standards, peer-to-peer (P2P) networks, etc. In an example, network interface device 520 may include one or more physical jacks (e.g., Ethernet, coaxial, or telephone jacks) or one or more antennas to connect to communication network 526. In an example, network interface device 520 may include multiple antennas to wirelessly communicate using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) technologies. The term "transmission medium" should be considered as including any intangible medium capable of storing, encoding, or carrying instructions for execution by machine 500, and including digital or analog communication signals or other intangible media used to facilitate communication of this software. The transmission medium is a machine-readable medium. For better illustration of the methods and apparatus described herein, a set of non-limiting exemplary embodiments are set forth below as numbered examples.

[0073] Example 1 is a network device comprising: an input configured to receive packets; a buffer for the input to store the packets; an output; a packet analyzer circuitry configured to: obtain characteristics of the packets; and write a record of the packets into a data structure independent of the buffer, the record containing the characteristics; and an arbitrator configured to perform arbitration among a plurality of packets based on the characteristics of the packets obtained from the record in the data structure to select a next packet from the buffer to be passed to the output.

[0074] In Example 2, according to the subject matter described in Example 1, the characteristic of the package is the size of the package, and the package analyzer circuitry is configured to measure the size of the package in order to obtain the characteristic of the package.

[0075] In Example 3, the subject matter according to any one of Examples 1 to 2, wherein the characteristic of the package is the position of the package in the buffer.

[0076] In Example 4, according to the subject matter described in Example 3, the packet analyzer circuitry is configured to update the record to include the new position in response to the packet moving to a new position in the buffer.

[0077] In Example 5, the subject matter according to any one of Examples 1 to 4, wherein the record contains multiple features of the package.

[0078] In Example 6, according to the subject matter described in Example 5, arbitration among the plurality of packets to select the next packet from the buffer is based on the plurality of characteristics of the packet stored in the record in the data structure.

[0079] In Example 7, the subject matter according to any one of Examples 1 to 6, wherein the data structure is one of a set of data structures corresponding to the buffer and used to store the characteristics of the buffer packet considered in the arbitration.

[0080] In Example 8, according to the subject matter described in Example 7, the cardinality of the data structure set is equal to the number of the plurality of packages.

[0081] In Example 9, according to the subject matter described in Example 8, the cardinality of the data structure set is equal to the number of virtual queues in the buffer.

[0082] In Example 10, according to the object described in Example 9, the number of virtual queues is equal to the number of outputs of the buffer in the network device.

[0083] In Example 11, the subject matter according to any one of Examples 1 to 10, wherein the network device is a switch in a chip system.

[0084] In Example 12, according to the subject matter described in Example 11, the switch is on a chiplet containing a memory controller for the chiplet system.

[0085] In Example 13, the subject matter according to any one of Examples 1 to 12, wherein the plurality of packets come from different buffers, and wherein the next packet is a queue head packet for the buffer.

[0086] Example 14 is a method comprising: receiving a packet at an input of a network device; placing the packet in a buffer for the input; obtaining a characteristic of the packet; writing a record of the packet into a data structure independent of the buffer, the record containing the characteristic; and performing arbitration among a plurality of packets based on the characteristic of the packet obtained from the record in the data structure to select the next packet from the buffer for delivery to an output.

[0087] In Example 15, the subject matter according to Example 14, wherein the characteristic of the package is the size of the package, and wherein obtaining the characteristic of the package includes measuring the size of the package.

[0088] In Example 16, the subject matter according to any one of Examples 14 to 15, wherein the characteristic of the package is the position of the package in the buffer.

[0089] In Example 17, the subject matter according to Example 16 includes updating the record to include the new position in response to the package moving to a new position in the buffer.

[0090] In Example 18, the subject matter according to any one of Examples 14 to 17, wherein the record contains a plurality of features of the package.

[0091] In Example 19, according to the subject matter described in Example 18, arbitration among the plurality of packets to select the next packet from the buffer is based on the plurality of characteristics of the packet stored in the record in the data structure.

[0092] In Example 20, the subject matter according to any one of Examples 14 to 19, wherein the data structure is one of a set of data structures corresponding to the buffer and used to store the characteristics of the buffer packet considered in the arbitration.

[0093] In Example 21, according to the subject matter described in Example 20, the cardinality of the data structure set is equal to the number of the plurality of packages.

[0094] In Example 22, according to the object described in Example 21, the cardinality of the data structure set is equal to the number of virtual queues in the buffer.

[0095] In Example 23, according to the object described in Example 22, the number of virtual queues is equal to the number of outputs of the buffer in the network device.

[0096] In Example 24, the subject matter according to any one of Examples 14 to 23, wherein the network device is a switch in a chip system.

[0097] In Example 25, the subject matter according to Example 24 is wherein the switch is on a chiplet containing a memory controller for the chiplet system.

[0098] In Example 26, the subject matter according to any one of Examples 14 to 25, wherein the plurality of packets come from different buffers, and wherein the next packet is a queue head packet for the buffer.

[0099] Example 27 is a machine-readable medium containing instructions that, when executed by the circuitry of a network device, cause the network device to perform operations including: receiving a packet at an input of the network device; placing the packet in a buffer for the input; obtaining characteristics of the packet; writing a record of the packet into a data structure independent of the buffer, the record containing the characteristics; and performing arbitration among a plurality of packets based on the characteristics of the packet obtained from the record in the data structure to select the next packet from the buffer for delivery to an output.

[0100] In Example 28, the subject matter according to Example 27, wherein the characteristic of the package is the size of the package, and wherein obtaining the characteristic of the package includes measuring the size of the package.

[0101] In Example 29, the subject matter according to any one of Examples 27 to 28, wherein the characteristic of the package is the position of the package in the buffer.

[0102] In Example 30, the subject matter according to Example 29, wherein the operation includes updating the record to include the new position in response to the package moving to a new position in the buffer.

[0103] In Example 31, the subject matter according to any one of Examples 27 to 30, wherein the record contains a plurality of features of the package.

[0104] In Example 32, according to the subject matter described in Example 31, arbitration among the plurality of packets to select the next packet from the buffer is based on the plurality of characteristics of the packet stored in the record in the data structure.

[0105] In Example 33, the subject matter according to any one of Examples 27 to 32, wherein the data structure is one of a set of data structures corresponding to the buffer and used to store the characteristics of the buffer packet considered in the arbitration.

[0106] In Example 34, according to the subject matter described in Example 33, the cardinality of the data structure set is equal to the number of the plurality of packages.

[0107] In Example 35, according to the subject matter described in Example 34, the cardinality of the data structure set is equal to the number of virtual queues in the buffer.

[0108] In Example 36, according to the object described in Example 35, the number of virtual queues is equal to the number of outputs of the buffer in the network device.

[0109] In Example 37, the subject matter according to any one of Examples 27 to 36, wherein the network device is a switch in a chip system.

[0110] In Example 38, the subject matter according to Example 37 is wherein the switch is on a chiplet containing a memory controller for the chiplet system.

[0111] In Example 39, the subject matter according to any one of Examples 27 to 38, wherein the plurality of packets come from different buffers, and wherein the next packet is a queue head packet for the buffer.

[0112] Example 40 is a system comprising: means for receiving a packet at an input of a network device; means for placing the packet in a buffer for the input; means for obtaining characteristics of the packet; means for writing a record of the packet into a data structure independent of the buffer, the record containing the characteristics; and means for performing arbitration among a plurality of packets based on the characteristics of the packet obtained from the record in the data structure to select the next packet from the buffer for delivery to an output.

[0113] In Example 41, the subject matter according to Example 40, wherein the characteristic of the package is the size of the package, and wherein the means for obtaining the characteristic of the package includes means for measuring the size of the package.

[0114] In Example 42, the subject matter according to any one of Examples 40 to 41, wherein the characteristic of the package is the position of the package in the buffer.

[0115] In Example 43, the object according to Example 42 includes means for updating the record to include the new position in response to the package moving to a new position in the buffer.

[0116] In Example 44, the subject matter according to any one of Examples 40 to 43, wherein the record contains a plurality of features of the package.

[0117] In Example 45, according to the subject matter described in Example 44, arbitration among the plurality of packets to select the next packet from the buffer is based on the plurality of characteristics of the packet stored in the record in the data structure.

[0118] In Example 46, the subject matter according to any one of Examples 40 to 45, wherein the data structure is one of a set of data structures corresponding to the buffer and used to store the characteristics of the buffer packet considered in the arbitration.

[0119] In Example 47, according to the subject matter described in Example 46, the cardinality of the data structure set is equal to the number of the plurality of packages.

[0120] In Example 48, according to the subject matter described in Example 47, the cardinality of the data structure set is equal to the number of virtual queues in the buffer.

[0121] In Example 49, according to the object described in Example 48, the number of virtual queues is equal to the number of outputs of the buffer in the network device.

[0122] In Example 50, the subject matter according to any one of Examples 40 to 49, wherein the network device is a switch in a chip system.

[0123] In Example 51, according to the subject matter described in Example 50, the switch is on a chiplet containing a memory controller for the chiplet system.

[0124] In Example 52, the subject matter according to any one of Examples 40 to 51, wherein the plurality of packets come from different buffers, and wherein the next packet is a queue head packet for the buffer.

[0125] Example 53 is at least one machine-readable medium containing instructions that, when executed by a processing circuitry system, cause the processing circuitry system to perform operations to implement any one of Examples 1 to 52.

[0126] Example 54 is an apparatus that includes means for implementing any one of Examples 1 to 52.

[0127] Example 55 is a system for implementing any one of Examples 1 through 52.

[0128] Example 56 is a method for implementing any one of Examples 1 through 52.

[0129] The above detailed description includes reference to the accompanying drawings, which form a part of the detailed description. The drawings illustrate, by means of illustration, specific embodiments in which the invention may be practiced. These embodiments are also referred to herein as “examples.” Such examples may include elements other than those shown or described. However, the inventors also contemplate that only examples of those elements shown or described are provided herein. Furthermore, the inventors also contemplate examples (or aspects thereof) of any combination or arrangement of those elements shown or described with respect to a particular example (or one or more aspects thereof) or with respect to other examples (or one or more aspects thereof) shown or described herein.

[0130] In this document, as is common in patent documents, the terms “a” or “an” are used to include one or more, regardless of any other examples or uses of “at least one” or “one or more”. In this document, unless otherwise indicated, the term “or” is used to refer to a non-exclusive “or”, such that “A or B” may include “A but not B”, “B but not A”, and “A and B”. In the appended claims, the terms “including” and “in which” are used as concise English equivalents to the corresponding terms “comprising” and “wherein”. Furthermore, in the following claims, the terms “comprising” and “including” are open-ended, meaning that a system, apparatus, article, or process that includes elements other than those listed after this term in the claims is still considered to be within the scope of the claims. Additionally, in the following claims, the terms “first,” “second,” and “third,” etc., are used merely as labels and are not intended to impose numerical requirements on their objects.

[0131] The above description is illustrative and not restrictive. For example, the examples described above (or one or more aspects thereof) may be used in combination with each other. Other embodiments may be used by those skilled in the art upon review of the above description. The abstract is submitted under the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Similarly, in the above detailed description, various features may be grouped together to simplify this disclosure. This should not be construed as expecting that unclaimed features are necessary for any claim. In fact, the subject matter of the invention may lie in fewer features than all of the particular disclosed embodiments. Therefore, the appended claims are hereby incorporated into the detailed description, wherein each claim exists independently as a separate embodiment, and these embodiments can be combined or arranged in various ways. The scope of the invention should be determined by reference to the appended claims and the full scope of the equivalents granted by the claims.

Claims

1. An apparatus for a chiplet system, comprising: Input, which can be coupled to the network and configured to receive packets; A memory configured to implement a buffer for the input to hold the packet; The output can be coupled to the network; The package analyzer circuit system is configured to: Obtain the characteristics of the package, wherein the characteristics are the position of the package in the buffer; and Write the record of the packet into a data structure independent of the buffer, the record containing the feature; and An arbitrator configured to perform arbitration among multiple packets based on the characteristics of the packet obtained from the record in the data structure to select the next packet from the buffer to be passed to the output, wherein the next packet is not the packet, and wherein, in response to the packet shifting to a new position in the buffer due to the next packet being passed to the output, the arbitrator is configured to update the position of the packet in the buffer in the record for the packet in the data structure.

2. The device of claim 1, wherein the record comprises a plurality of features of the package, the plurality of features comprising a second feature of the package, which is the size of the package.

3. The device of claim 2, wherein arbitration of the plurality of packets to select the next packet from the buffer is based on the plurality of characteristics of the packet stored in the record in the data structure.

4. The device of claim 1, wherein the data structure is one of a set of data structures corresponding to the buffer and used to store the characteristics of the buffer packet considered in the arbitration.

5. The device of claim 4, wherein the cardinality of the data structure set is equal to the number of the plurality of packets.

6. The device of claim 5, wherein the cardinality of the data structure set is equal to the number of virtual queues in the buffer, and each of the plurality of packets is at the head of one of the virtual queues.

7. The device according to claim 1, wherein the device is a switch in the chip system.

8. The device of claim 7, wherein the switch is on a chiplet containing a memory controller for the chiplet system.

9. The device of claim 1, wherein the plurality of packets come from different buffers, and wherein the next packet is a queue head packet for the buffers.

10. A method for a chiplet system, comprising: Receive packets at the input of the network device; Place the packet in the buffer used for the input; Obtain the characteristics of the package, wherein the characteristic is the position of the package in the buffer; Write the record of the packet into a data structure independent of the buffer, the record containing the feature; and Arbitration is performed among multiple packets based on the characteristics of the packet obtained from the record in the data structure to select the next packet from the buffer to be passed to the output, wherein the next packet is not the packet, and wherein performing the arbitration includes updating the position of the packet in the buffer in the record for the packet in the data structure.

11. The method of claim 10, wherein the record comprises a plurality of features of the package, the plurality of features comprising a second feature of the package, which is the size of the package.

12. The method of claim 11, wherein arbitration among the plurality of packets to select the next packet from the buffer is based on the plurality of characteristics of the packet stored in the record of the data structure.

13. The method of claim 10, wherein the data structure is one of a set of data structures corresponding to the buffer and used to store features of the buffer packets considered in the arbitration.

14. The method of claim 13, wherein the cardinality of the data structure set is equal to the number of the plurality of packages.

15. The method of claim 14, wherein the cardinality of the data structure set is equal to the number of virtual queues in the buffer, and each of the plurality of packets is at the head of one of the virtual queues.

16. The method of claim 10, wherein the network device is a switch in the chiplet system.

17. The method of claim 16, wherein the switch is on a chiplet containing a memory controller for the chiplet system.

18. The method of claim 10, wherein the plurality of packets come from different buffers, and wherein the next packet is a queue head packet for the buffers.

19. A non-transitory machine-readable medium comprising instructions that, when executed by the circuitry of a network device, cause the network device to perform operations, the operations including: Receive packets at the input of the network device; Place the packet in the buffer used for the input; Obtain the characteristics of the package, wherein the characteristic is the position of the package in the buffer; Write the record of the packet into a data structure independent of the buffer, the record containing the feature; and Arbitration is performed among multiple packets based on the characteristics of the packet obtained from the record in the data structure to select the next packet from the buffer to be passed to the output, wherein the next packet is not the packet, and wherein performing the arbitration includes updating the position of the packet in the buffer in the record for the packet in the data structure.

20. The machine-readable medium of claim 19, wherein the record comprises a plurality of features of the package, the plurality of features comprising a second feature of the package, which is the size of the package.

21. The machine-readable medium of claim 20, wherein arbitration among the plurality of packets to select the next packet from the buffer is based on the plurality of characteristics of the packet stored in the record of the data structure.

22. The machine-readable medium of claim 19, wherein the data structure is one of a set of data structures corresponding to the buffer and used to store features of the buffer packet considered in the arbitration.

23. The machine-readable medium of claim 22, wherein the cardinality of the data structure set is equal to the number of the plurality of packets.

24. The machine-readable medium of claim 23, wherein the cardinality of the set of data structures is equal to the number of virtual queues in the buffer, and each of the plurality of packets is at the head of one of the virtual queues.

25. The machine-readable medium of claim 19, wherein the network device is a switch in a chip system.

26. The machine-readable medium of claim 25, wherein the switch is on a chiplet containing a memory controller for the chiplet system.

27. The machine-readable medium of claim 19, wherein the plurality of packets originate from different buffers, and wherein the next packet is a queue head packet for the buffers.

Images