Link layer data packaging and packet flow control scheme

Abstract

Systems, devices and methods for performing efficient data transfer in a computing system are disclosed. The computing system includes multiple fabric interfaces in a client and a fabric. A packet transmitter in the fabric interface includes multiple queues, each queue is used to store packets of a corresponding type. The packet transmitter includes multiple queue arbiters, each queue arbiter is used to select candidate packets from a corresponding queue in the multiple queues. The packet transmitter includes a buffer for storing link packets, and the buffer includes data storage space for storing multiple candidate packets. The packet transmitter selects qualified candidate packets from the multiple queues and inserts the candidate packets into the link packets. The packet arbiter avoids data conflicts at the receiver by considering the mismatch between the rate at which candidate packets are inserted into the link packets and the rate at which available data storage space is generated in the receive queue in the receiver.

Description

Background of the Invention

[0002] Description of Related Technology

[0003] In computing systems, some types of applications perform functions that can take advantage of parallel processing and shared memory better than other applications. Examples of such applications include machine learning applications, entertainment and real-time applications, and some commercial, scientific, medical, and other applications. Although some processor architectures include more than one processing unit (e.g., CPU, GPU, etc.) or processing core, in some cases, one or two additional processing units or cores coupled to memory do not necessarily provide sufficient parallelism to provide the desired performance level.

[0004] In addition to read and write access commands and corresponding data, consistency probes, interrupts, and other communication messages are also transmitted in the system through the communication fabric (or fabric). Examples of interconnects in the fabric are bus architectures, crossbar-based architectures, network-on-chip (NoC) communication subsystems, communication channels between dies, silicon interposers for stacking chips side by side, through silicon vias (TSVs) for vertically stacking specialized dies on top of processor dies, etc.

[0005] In many cases, a fabric has multiple physical channels, each supporting relatively wide packets. When data is transmitted within a single fabric, the fabric reduces latency because a relatively large number of physical lines can be used. However, when separate die are connected together by the fabric and when separate processing nodes are connected together (each processing node has a corresponding fabric), data is transmitted through a significantly reduced number of physical lines, which limits the available bandwidth. In some cases, the link physical lines transmit data at a data rate that is a multiple of the data rate of the physical lines on the die. However, when communicating between die and between nodes, the bandwidth is still significantly reduced.

[0006] In addition to the aforementioned inefficiencies in transferring data, packets of varying types from different sources are combined together many times along with control information. One or more of the sources of varying packet types have different bus widths and carry varying amounts of information between packets within a single source. These packets of varying sizes from different sources are inefficiently combined together many times, further reducing link bandwidth efficiency.

[0007] Still further, when the rate at which packets are sent from the transmitter does not match the rate at which packets are received at the receiver, data corruption occurs at the receiver through data collisions. In one example, the transmitter is a graphics processing unit (GPU) in a processing unit, and the receiver is a fabric. A conservative approach of reducing the send data rate of packets from the transmitter prevents data corruption, but also further reduces link bandwidth efficiency.

[0008] In view of the above, efficient methods and systems for performing efficient data transfers in computing systems are desired. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Advantages of the methods and mechanisms described herein may be better understood by referring to the following description in conjunction with the accompanying drawings, in which:

[0010] Figure 1 is a block diagram of one embodiment of a computing system.

[0011] Figure 2 is a block diagram of one implementation of a packet transmitter.

[0012] Figure 3 is a flow chart of one embodiment of a method for performing efficient data transfer in a computing system.

[0013] Figure 4 is a flow chart of another embodiment of a method for performing efficient data transfer in a computing system.

[0014] Figure 5 is a flow chart of another embodiment of a method for performing efficient data transfer in a computing system.

[0015] Figure 6 is a block diagram of one embodiment of a packet receiver.

[0016] Figure 7 is a flow chart of another embodiment of a method for performing efficient data transfer in a computing system.

[0017] Although the present invention is susceptible to various modifications and alternative forms, specific embodiments are shown in the drawings by way of example and are described in detail herein. However, it should be understood that the drawings and their detailed description are not intended to limit the present invention to the specific forms disclosed, but on the contrary, the present invention will cover all modifications, equivalents and alternatives falling within the scope of the present invention as defined by the appended claims. DETAILED DESCRIPTION

[0018] In the following description, numerous specific details are set forth to provide a thorough understanding of the methods and mechanisms presented herein. However, it will be appreciated by those of ordinary skill in the art that various embodiments may be practiced without these specific details. In some cases, well-known structures, components, signals, computer program instructions, and techniques are not shown in detail to avoid confusion with the methods described herein. It should be understood that in order to make the description simple and clear, the elements shown in the figures may not be drawn to scale. For example, some sizes of the elements may be enlarged relative to other elements.

[0019] Various systems, devices, methods, and computer-readable media for performing efficient data transfer in a computing system are disclosed. In various embodiments, the computing system includes one or more clients for processing applications. Examples of clients include general-purpose central processing units (CPUs), graphics processing units (GPUs), accelerated processing units (APUs), input / output (I / O) devices, etc. The computing system also includes at least a power controller, and a plurality of link interfaces for transmitting messages between clients.

[0020] In various embodiments, one or more clients and a fabric include a fabric interface, each fabric interface having a packet transmitter and a packet receiver. The packet transmitter includes a plurality of queues, each queue being used to store packets of a corresponding type. Examples of packet types include request types, response types, probe types, and token or credit types. Other examples of types include data types in packets, such as commands or messages, which are control types. For write requests, in some embodiments, write requests are divided into write commands inserted into commands or control packets. Write requests are also divided into separate write data packets corresponding to write commands. Read requests are sent as read commands inserted into commands or control packets. The response to a read command is sent as a read message inserted into a command or control packet. Read response data is inserted into separate data packets.

[0021] In some embodiments, two or more of the plurality of queues store packets of different sizes. In one embodiment, the packet transmitter further comprises a plurality of queue arbiters, each of which is used to select candidate packets from a corresponding queue in the plurality of queues. The packet transmitter further comprises a buffer for storing link packets. In various embodiments, the link packet comprises a data storage space for storing a plurality of candidate packets. The link packet is sent to the receiver via a link that is no longer busy transmitting other data. In some embodiments, the buffer for storing the link packet comprises a plurality of registers, a flip-flop circuit or other sequential elements.

[0022] The packet transmitter also includes a packing arbiter that selects qualified candidate packets from a plurality of queues and inserts the candidate packets into the link packets. In some embodiments, the packing arbiter selects candidate packets based on one or more attributes and inserts them into the link buffer, wherein the one or more attributes include age, priority level of packet type, quality of service parameters, source identifier, application identifier or type (such as real-time application), indication of business type (such as real-time business, bandwidth requirements or delay tolerance requirements, etc.). In various embodiments, the packing arbiter also determines whether the fill link packet has remaining available data storage space for a given candidate packet. In some embodiments, the packing arbiter also determines whether the fill link packet has remaining available data storage space aligned on a given boundary of a given candidate packet.

[0023] The packetization arbiter includes control logic for taking into account the mismatch between the rate at which candidate packets are packetized (inserted) into link packets and the rate at which available data storage space is created in the receive queue in the receiver. The mismatch is also caused by the network in the fabric adding and removing clock cycles when the link is available for transmitting link packets.

[0024] In one example, when the fabric interface detects a data transfer error and resends the data, idle clock cycles are inserted on the link from the perspective of the packet transmitter. In some cases, due to bus utilization and arbitration throughout the network in the fabric, these idle cycles are not maintained between two link packets sent from the packet transmitter, but are deleted. Therefore, in order to avoid data collisions at the receiver's receive queue, the packet transmitter maintains a first count of enabled network cycles (each count allows the next link packet to use the link connected to the packet transmitter), and a second count of the number of candidate packets of a given type inserted into the link packet. The packet transmitter stores a third count of the minimum number of cycles between link packets received by the receiver to avoid data collisions in the receive queue storing packets of the given type. The packet transmitter uses these counts to determine when to insert a given candidate packet into a link packet.

[0025] See also Figure 1 , a generalized block diagram of one embodiment of a computing system 100 is shown. As shown, computing system 100 includes a communication fabric 120 between each of client 110, memory controller 130, power controller 170, and link interface 180. In some embodiments, the components of processing node 100 are individual dies on an integrated circuit (IC) such as a system on a chip (SOC). In other embodiments, the components are individual dies in a system in package (SiP) or a multi-chip module (MCM).

[0026] In the illustrated embodiment, client 110 includes central processing unit (CPU) 112, graphics processing unit (GPU) 114 and hub 116. Hub 116 is used to communicate with multimedia engine 118. CPU 112, GPU 114 and multimedia engine 118 are examples of computing resources capable of processing application programs. Although not shown, in other embodiments, other types of computing resources are included in client 110. Each of one or more processor cores in CPU 112 includes a circuit for executing instructions according to a given selected instruction set architecture (ISA). In various embodiments, each of the processor cores in CPU 112 includes a superscalar, multithreaded microarchitecture for processing instructions of a given ISA. In one embodiment, GPU 114 includes a high parallel data microarchitecture with a large number of parallel execution channels. In one embodiment, microarchitecture uses single instruction multiple data (SIMD) pipelines for parallel execution channels. Multimedia engine 118 includes a processor for processing audio data and visual data of multimedia applications.

[0027] In one embodiment, power controller 170 collects data, such as a predetermined sampling signal, from client 110 during a given sampling interval. In some embodiments, on-die current sensors and temperature sensors in node 100 also send information to power controller 170. In some embodiments, power controller 170 selects a corresponding power-performance state (P-state) for each of the computing resources in client 110. The P-state includes at least an operating voltage and an operating clock frequency.

[0028] Although a single memory controller 130 is shown, in other embodiments, another number of memory controllers are used in processing node 100. In various embodiments, memory controller 130 receives memory requests from client 110 through fabric 120, schedules the memory requests, and sends the scheduled memory requests to one or more of system memory and main memory. Memory controller 130 also receives responses from system memory and main memory and sends the responses to the corresponding sources of the requests in client 110. In various embodiments, system memory is filled with data from main memory through I / O controller and bus 160 and memory bus 150. The corresponding cache fill line with the requested block is conveyed from main memory to the corresponding cache memory subsystem in the cache memory subsystem in client 110 to complete the original memory request. The cache fill line is placed in one or more levels of cache.

[0029] In some embodiments, the address space of processing node 100 is divided between at least CPU 112, GPU 114 and hub 116 and one or more other components (such as input / output (I / O) peripheral devices (not shown)) and other types of computing resources. Maintaining memory mapping determines which addresses are mapped to which component, and therefore determines which of CPU 112, GPU 114 and hub 116 a memory request for a particular address should be routed to. In one embodiment, system memory is one of a variety of dynamic random access memories (DRAMs), and memory controller 130 supports a corresponding protocol. The protocol determines the values ​​used for information transfer, such as the number of data transfers per clock cycle, signal voltage levels, signal timing, signal and clock phases, and clock frequencies. In some embodiments, main memory is one of a variety of types of non-volatile random access secondary data storage devices. Examples of main memory are hard disk drives (HDDs) and solid state drives (SSDs).

[0030] In various embodiments, communication fabric 120 (or fabric 120) transmits traffic back and forth between computing resources 110 and memory controller 130, and includes interfaces for supporting corresponding communication protocols. "Traffic" includes data such as commands, messages, probes, interrupts, and data corresponding to commands and messages. In some embodiments, fabric 120 includes at least queues for storing requests and responses, selection logic for arbitrating between received requests before sending them across the internal network, logic for building packets and decoding them, and logic for selecting routes for packets.

[0031] In some embodiments, fabric transmission interface (FTI) 180 supports the communication between computing system 100 and other computing systems or processing nodes by transmitting business on link. In various embodiments, the business sent across the link includes the indication of the operating state of one or more processing nodes, power-down request, response to request, interruption and other information. As shown in the figure, client 110 includes FTI 113 for CPU 112, FTI 115 for GPU 114 and FTI 117 for hub 116. In addition, memory controller 130 includes FTI 132, and power controller 170 includes FTI 172. Texture 120 includes multiple texture transmission interfaces (not shown for ease of explanation). In some embodiments, each link connected to the texture transmission interface is a point-to-point communication channel. In other embodiments, one or more links connected to the texture transmission interface are end-to-end communication channels.

[0032] At the physical level, a link includes one or more channels. In some embodiments, the fabric transport interface includes control logic and buffers or queues for communication. In some embodiments, the fabric transport interface and the corresponding link include communication protocol connections such as PCIe (Peripheral Component Interconnect Express), InfiniBand, RapidIO, HyperTransport, Advanced eXtensible Interface (AXI), etc.

[0033] In some embodiments, each of the fabric transmission interfaces includes multiple queues, each queue is used to store packets of corresponding types. In addition, there are multiple queue arbiters, each arbiter is used to select candidate packets from corresponding queues in the multiple queues. One or more of the fabric transmission interfaces in the computing system 100 include a buffer for storing link packets, and the buffer includes data storage space for storing multiple candidate packets. In some embodiments, two or more of the multiple queues store candidate packets of different sizes. The packing arbiter in the fabric transmission interface selects qualified candidate packets from the multiple queues and inserts these candidate packets into the link packet. In some embodiments, the packing arbiter also determines whether the filling link packet has the remaining available data storage space aligned on the given boundary of the given candidate packet. The packet receiver in the fabric transmission interface includes a receiving queue for receiving the link packet sent by the packet transmitter. The packing arbiter in the packet transmitter avoids data conflicts at the receiver by considering the mismatch between the rate at which the candidate packets are inserted into the link packet and the rate at which the available data storage space is generated in the receiving queue in the receiver.

[0034] refer to Figure 2 , a generalized block diagram of one embodiment of a packet transmitter 200 is shown. In the illustrated embodiment, the packet transmitter 200 includes queues 210-214, each queue being used to store packets of a corresponding type. In some embodiments, the packets are flow control units ("flits"). A flit is a subset of a larger packet. A flit typically carries data and control information, such as header information and trailer information of a larger packet. Although the data for transmission is described as packets routed in a network, in other embodiments, the data for transmission is a bit stream or byte stream in a point-to-point interconnect. In various embodiments, queues 210-214 store control packets to be sent on a fabric link. Corresponding data packets (such as larger packets corresponding to a flit) are sent from another source.

[0035] In some embodiments, one or more other sources (such as a source for larger data packets) share the fabric link with the packet transmitter 200. Therefore, the fabric link is not always available to send control packets. The packet transmitter 200 also includes a packet buffer arbiter 250, which is used to insert one or more of the candidate packets 230-234 into the link packet 260. The link packet 260 has sufficient data storage space to store two or more of the fabric transmission interface (FTI) candidate packets 230-234. In one embodiment, the packet transmitter 200 sends the link packet 260 on the fabric link when two requirements are met. The first requirement is that the fabric transmission interface sends a signal to the packet transmitter 200 indicating that the fabric link is available. The second requirement is that the packet transmitter 200 determines that the link packet 260 is not empty.

[0036] The examples of control packet types stored in queues 210-214 include request types, response types, detection types, and token or credit types. Other examples of packet types are also included in other embodiments. As shown, queue 210 stores packets of "type 1", which are control request types in one embodiment. Queue 212 stores packets of "type 2", which are control response types in one embodiment. Queue 214 stores packets of "type N", which are control tokens or credit types in one embodiment.

[0037] The queue arbiter 220 selects a fabric transport interface (FTI) candidate packet 230 from the queue 210. In some embodiments, the queue arbiter 220 selects the FTI candidate packet 230 based on one or more attributes, including age, priority level of packet type, priority level of packet, quality of service parameter, source identifier, application identifier or type (such as real-time application), indication of traffic type (such as real-time traffic), bandwidth requirement or delay tolerance requirement, etc. In a similar manner, the queue arbiters 222-224 select FTI candidate packets 232-234 from the queues 212-214.

[0038] In some embodiments, the queue arbiters 220-224 select FTI candidate packets 230-234 from the queues 210-214 at each clock cycle. In other embodiments, the queue arbiters 220-224 select FTI candidate packets 230-234 from the queues 210-214 after a previously selected FTI candidate packet 230-234 has been inserted into the link packet 260. In still other embodiments, the queue arbiters 220-224 select FTI candidate packets 230-234 from the queues 210-214 after a previously selected FTI candidate packet 230-234 has been selected by the packet buffer arbiter 250 but has not been inserted into the link packet 260 due to unavailable data storage space in the link packet 260, unavailable data storage space aligned with a given boundary in the link packet 260, or other unsatisfied conditions.

[0039] As shown, each of the FTI candidate packets 230-234 has a different data size 240-244. In some embodiments, two or more of the FTI candidate packets 230-234 have the same data size. In one embodiment, one or more of the queues 210-214 store FTI packets of different sizes. Therefore, when a given queue arbiter in the queue arbiters 222-224 selects an FTI candidate packet, the selected FTI candidate packet has a size different from the previously selected FTI candidate packet selected from the same queue. In an example, when the queue 210 stores a packet of a request type, the queue 210 stores an uncompressed request packet of size 128 bits, and stores a compressed request packet of size 64 bits. Therefore, some entries of the queue 210 store FTI candidate packets whose data size is different from the data size 240.

[0040] As shown, link packet 260 is divided into sectors 262-266. In one embodiment, each sector is 32 bits. Other data sizes of sectors 262-266 are possible and can be considered. In one embodiment, link packet 260 includes eight sectors. In other embodiments, link packet 260 includes another number of sectors. In various embodiments, link packet 260 is stored in a register. In other embodiments, other sequential storage elements are used. In addition to storing data such as controlling FTI candidate packets, other metadata (not shown) are also stored, such as which sectors in sectors 262-266 are available and which sectors are assigned. Packet buffer arbiter 250 receives FTI candidate packets 230-234, and selects one or more of the received FTI candidate packets 230-234 to be inserted into the available data storage space in link packet 260. In some embodiments, the packet buffer arbiter 250 can select one of the FTI candidate packets 230-234 per clock cycle and insert it into the link packet 260. In another embodiment, the packet buffer arbiter 250 can select two or more of the FTI candidate packets 230-234 per clock cycle and insert them into the link packet 260.

[0041] In various embodiments, the packet buffer arbiter 250 includes a combination of combinational logic and sequential elements. The packet buffer arbiter 250 is implemented in hardware, software, or a combination of hardware and software. In some embodiments, the packet buffer arbiter 250 selects one or more of the FTI candidate packets 230-234 based on one or more attributes and inserts them into the link packet 260, wherein the one or more attributes include packet age, priority level of packet type, quality of service parameters, source identifiers, application identifiers or types (such as real-time applications), indications of business types (such as real-time business), bandwidth requirements or delay tolerance requirements, etc. In some embodiments, the packet buffer arbiter 250 selects one of the FTI candidate packets 230-234 in each clock cycle for insertion (packing). In other embodiments, the packet buffer arbiter 250 selects two or more of the FTI candidate packets 230-234 in each clock cycle for insertion (packing).

[0042] In various embodiments, the packet buffer arbiter 250 also determines whether the link packet 260 has remaining available data storage space for a given FTI candidate packet in the FTI candidate packets 232-234. In some embodiments, the packet buffer arbiter 250 also determines whether the link packet 260 has remaining available data storage space aligned on a given boundary of a given FTI candidate packet in the FTI candidate packets 232-234. In some embodiments, the packet arbiter 250 includes control logic that is used to take into account the mismatch between the rate at which the FTI candidate packets 232-234 are packetized (inserted) into the link packet 260 and the rate at which available data storage space is generated in the receive queue in the receiver at the other end of the fabric link. This mismatch is also caused by the network in the fabric adding and deleting idle clock cycles ("bubbles") when the link is available for transmitting link packets.

[0043] In one example, idle clock cycles ("bubbles") are inserted onto the fabric link when the fabric interface detects a data transfer error and resends the data. In some cases, due to bus utilization and arbitration throughout the network in the fabric, these idle cycles are not maintained between two link packets sent from the packet transmitter 200 to the receiver, but are deleted. Therefore, to avoid data collisions at the receiver's receive queue, in one embodiment, the packet buffer arbiter 250 stores a count of the minimum number of cycles between link packets received by the packet receiver to avoid data collisions in the receive queue storing packets of a given type.

[0044] In one example, the packet receiver on the other end of the fabric link has a receiving queue of a given type (e.g., a control response packet), and this receiving queue has two available entries to allocate received packets. In order to simplify the access logic and meet the timing requirements, in some embodiments, two identical entries in the receiving queue are available for allocating received packets, rather than allocating received packets of a given type in any entry. In the next clock cycle, the information corresponding to the two allocated packets of a given type will be relocated to other entries. In one embodiment, the information is moved down (or up) two entries. In another embodiment, the information is moved down (or up) a single entry. In yet another embodiment, the information is classified based on attributes with the information stored in other entries. Examples of attributes are provided above. In various embodiments, the packing buffer arbiter 250 in the packet transmitter 200 is aware of the implementation in the packet receiver, and uses this cognition when the FTI candidate packets 230-234 are inserted into the link packet 260.

[0045] Using the above example, the packet buffer arbiter 250 realizes that the minimum number of clock cycles (or cycles) between link packets received by the packet receiver for avoiding data collisions is one cycle. For example, in clock cycle 1, the packet receiver may receive two control FTI packets of a given type (e.g., control response packets) and allocate them in two entries (e.g., entries 0 and 1) of the receive queue of the given type. In clock cycle 2, the receive queue transfers the information corresponding to the most recently received control FTI packets from the two entries (e.g., entries 0 and 1) to other entries in the receive queue. For example, in one embodiment, the information stored in the entries of the receive queue is sorted for arbitration purposes.

[0046] In clock cycle 2, two entries (e.g., entries 0 and 1) cannot receive new information. If new information of one or more new control FTI packets of a given type is received, a data conflict occurs in two entries (e.g., entries 0 and 1). In clock cycle 3, two entries (e.g., entries 0 and 1) are available for receiving new information again for allocation. Therefore, in this example, for a given type of control FTI packet, the packet receiver needs to be used to avoid a minimum of one clock cycle number between the link packets received by the packet receiver for data conflicts of the given type of control FTI packets. In various embodiments, the minimum clock cycle number between the received link packets for avoiding data conflicts at the packet receiver of the first type of control FTI packets is different from the minimum clock cycle number of the second type of control FTI packets. In various embodiments, the packet buffer arbiter 250 stores a separate value for the minimum clock cycle number of each of the control types such as "type 1" to "type N".

[0047] In some embodiments, the packet buffer arbiter 250 additionally maintains a first count of enabled network cycles, each of which allows the next link packet 260 from the packet transmitter 200 to use the fabric link. In some embodiments, the packet buffer arbiter 250 also maintains a second count of the number of FTI candidate packets of a given type inserted into the link packet 260. Thus, the packet buffer arbiter 250 maintains a separate second count for each of the control types such as "Type 1" to "Type N". In various embodiments, the packet buffer arbiter 250 determines the sum of the FTI packets of a given type by adding three operands. The first operand is the second count of the given type. The second operand is the minimum number of clock cycles for storage of the given type. And the third operand is one corresponding to the FTI candidate packet, which will fit into one or more consecutive sectors of sectors 262-266, wherein the consecutive sectors are aligned with a given boundary. In one embodiment, if the packet buffer arbiter 250 determines that the sum is less than the first count, the packet buffer arbiter 250 inserts the FTI candidate packet into the link packet 260 and increments the second count.

[0048] Reference now Figure 3 , showing an embodiment of a method 300 for performing efficient data transfer in a computing system. For discussion purposes, the embodiments (and Figures 4 to 5 and Figure 7 However, it should be noted that in various embodiments of the described method, one or more of the described elements are performed simultaneously, in a different order than shown, or omitted entirely. Other additional elements may also be performed as needed. Any of the various systems or devices described herein is configured to implement method 300.

[0049] A packet is received for transmission on a link (block 302). In one example, a processing unit generates a control request packet to transmit to a destination. In another example, a memory controller generates a control response packet to transmit to a requesting processing unit. The fabric transport interface includes a packet transmitter that receives the control packet. Based on the packet type, the received packets are stored in a plurality of queues (block 304). As previously described, examples of control packet types include a request type, a response type, a probe type, and a token or credit type. Other examples of packet types are also included in other embodiments.

[0050] A candidate packet is selected from each queue (block 306). In some embodiments, the queue arbiter selects a candidate packet from a queue based on one or more attributes, including age, priority level of the packet type, priority level of the packet, quality of service parameters, source identifier, application identifier or type (such as a real-time application), indication of a traffic type (such as real-time traffic), bandwidth requirements or delay tolerance requirements, etc.

[0051] If the link to the receiver is available and the link packet is non-empty (the "yes" branch of conditional block 308), the packetized link packet is sent over the available link to the receiver (block 310). In some embodiments, the fabric transport interface sends a signal or other indication to the packet transmitter specifying that the fabric link is available for sending the packetized link packet. If the link to the receiver is not available or the link packet is empty (the "no" branch of conditional block 308), one or more candidate packets are conditionally packaged into the link packet (block 312).

[0052] The condition includes whether the link packet has enough data storage space for receiving a given candidate packet. In some embodiments, it is also determined whether the link packet has enough available data storage space aligned on a given boundary of a given candidate packet. If the link packet is full (the "yes" branch of conditional box 314), then start waiting for the link to the receiver to become available (box 316). Then, the control flow of method 300 moves to box 310. If the link packet is not full (the "no" branch of conditional box 314), the control flow of method 300 returns to conditional box 308, in which it is determined whether the link to the receiver is available and whether the link packet is empty.

[0053] Now turn to Figure 4 , showing an embodiment of a method 400 for performing efficient data transfer in a computing system. Candidate packets are received from a plurality of queues, each queue storing packets of a corresponding type (box 402). In some embodiments, a queue arbiter selects a candidate packet from a queue at each clock cycle. In other embodiments, the queue arbiter selects a candidate packet from a queue after a previously selected candidate packet has been inserted into a link packet. In still other embodiments, the queue arbiter selects a candidate packet from a queue after a previously selected candidate packet has been selected by a packet buffer arbiter but has not been inserted into a link packet due to unavailable data storage space in the link packet, unavailable data storage space aligned with a given boundary in the link packet, or other unsatisfied conditions.

[0054] A set of available queues is identified (box 404). In some embodiments, when a given queue stores one or more candidate packets, the given queue is in the set of available queues. In one embodiment, one or more of the plurality of queues have corresponding limits on the number of packets inserted from the queue into the link packet before the link packet is sent on the available link. In some embodiments, the limits are stored in configuration and status registers. In various embodiments, the registers are programmable. A queue with the highest priority is selected from the set of available queues (box 410). In some embodiments, the priority is based on one or more attributes, and the one or more attributes include packet age, priority level of packet type, quality of service parameters, source identifier, application identifier or type (such as real-time application), indication of business type (such as real-time business), bandwidth requirements or delay tolerance requirements, etc.

[0055] If the candidate packet of the selected queue satisfies the insertion condition (the "yes" branch of conditional box 412), the candidate packet is inserted into the link packet (box 418). As previously described, in some embodiments, a determination is made as to whether the link packet has remaining available data storage space aligned on a given boundary of the candidate packet. A determination is also made as to whether inserting the candidate packet into the link packet will avoid data collisions at the receiver by considering the mismatch between the rate at which the candidate packets are inserted into the link packet and the rate at which available data storage space is created in the receive queue at the receiver.

[0056] If the candidate packet of the selected queue does not meet the insertion condition (the "No" branch of conditional box 412), the selected queue is notified to select another candidate packet (box 414). In one embodiment, the next candidate packet is selected by the selected queue in the next clock cycle. In some embodiments, the priority of the selected queue is reduced until the candidate packet is inserted into the link packet or the link packet is sent (box 416). In other embodiments, the priority of the selected queue is maintained. In other embodiments, the priority of the selected queue is maintained, but the selected queue is deleted from the set of available queues. Then, the control flow of method 400 returns to box 404, in which the set of available queues is identified. In some embodiments, the loop from box 404 to box 416 or box 418 is repeated multiple times within one clock cycle.

[0057] Reference now Figure 5, one embodiment of a method 500 for performing efficient data transfer in a computing system is shown. A candidate packet is received from a selected queue (block 502). In one embodiment, a packetization buffer arbiter for packetizing link packets receives the candidate packet. If there is no available space at the receiver for the candidate packet of the selected queue (the "no" branch of conditional block 504), the selected queue is deleted from the set of available queues (block 506).

[0058] If there is available space at the receiver for a candidate packet for the selected queue (the "yes" branch of conditional block 504), but there is no available space in the link packet for a candidate packet for the selected queue (the "no" branch of conditional block 508), then the selected queue is deleted from the set of available queues (block 506). If there is available space in the link packet for a candidate packet for the selected queue (the "yes" branch of conditional block 508), but the available space in the link packet does not align with a given boundary (the "no" branch of conditional block 510), then the selected queue is deleted from the set of available queues (block 506). If the available space in the link packet aligns with the given boundary (the "yes" branch of conditional block 510), then the candidate packet is inserted into the link packet (block 512).

[0059] refer to Figure 6 , a generalized block diagram of one embodiment of a packet receiver 600 is shown. In the illustrated embodiment, the packet receiver 600 includes queues 610-614, each queue being used to store packets of a corresponding type. As shown, the packet receiver 600 receives a link packet 660 from a fabric link. The link packet 660 has sufficient data storage space to store two or more fabric transport interface (FTI) packets. The distribution demultiplexer logic 650 (or logic 650) analyzes the sectors 662-666 in the link packet 600, determines how many FTI control packets there are in the link packet 660 and their locations in the link packet 660, and sends the FTI control packets to the queues 610-614 for storage.

[0060] The example of the control packet type stored in the queues 610-614 includes a request type, a response type, a detection type, and a token or credit type. Other examples of packet types are also included in other embodiments. As shown, queue 610 stores a packet of "type 1", which is a control request type in one embodiment. Queue 612 stores a packet of "type 2", which is a control response type in one embodiment. Queue 614 stores a packet of "type N", which is a control token or credit type in one embodiment.

[0061] As shown, link packet 660 is divided into sectors 662-666. In one embodiment, each sector is 32 bits. Other data sizes of sectors 662-666 are possible and can be considered. In one embodiment, link packet 660 includes eight sectors. In other embodiments, link packet 660 includes another number of sectors. In addition to storing data such as control FTI packets, other metadata (not shown) are also stored, such as which of sectors 662-666 is allocated and where the control FTI packet begins, because the control FTI packet has a variable data size in some embodiments.

[0062] In various embodiments, logic 650 includes a combination of combinatorial logic and sequential elements. Logic 650 is implemented with hardware, software or a combination thereof. As shown, logic 650 deletes one or more control FTI packets from link packet 660, and sends them to queues 610-614. In various embodiments, each of FTI candidate packets 630-634 has different data sizes 640-644. In some embodiments, two or more of FTI candidate packets 630-634 have the same data size. In one embodiment, one or more of queues 610-614 store FTI packets of different sizes. Therefore, when logic 650 sends FTI packets such as FTI packets 630 to queue 610, the FTI packets have a size different from the previous FTI packets sent to the same queue. In an example, when queue 610 stores a packet of a request type, queue 610 stores an uncompressed request packet of size 128 bits, and stores a compressed request packet of size 64 bits. Therefore, some entries of queue 610 store FTI candidate packets having a data size different from data size 640 .

[0063] In one embodiment, each of the queues 610-614 has two available entries to distribute the packet received. In order to simplify the access logic and meet the timing requirements, in some embodiments, two identical entries in the queues 610-614 are available for distributing the packet received, rather than distributing FTI packets 630-634 in any entries. As shown in the figure, two entries at the top are used to receive one or more of the FTI packets 630-634. In other embodiments, the entries and other positions of another quantity in the queues 610-614 are used. In some embodiments, valid fields 611,613 and 615 are used to indicate available entries. In one embodiment, the negative value indication available entries stored in valid fields 611,613 and 615, and the assertion value indication distributed entries stored in valid fields 611,613 and 615.

[0064] In the next clock cycle, the newly allocated information in queues 610-614 is relocated to other entries. In one embodiment, the information is moved down (or up) two entries. In another embodiment, the information is moved down (or up) a single entry. In yet another embodiment, the information is classified with the information stored in other entries based on attributes. Examples of attributes are provided above. In various embodiments, ( Figure 2 The) packet transmitter 200 is aware of the implementation in the packet receiver 600 and uses this knowledge when inserting the FTI candidate packet into the link packet.

[0065] In some embodiments, logic 650 sends tokens or credits 652 for each of the packet types to the packet transmitter over the sideband connection. In some embodiments, when a stored FTI packet is sent for processing, the number of available entries in one or more of queues 610-614 increases beyond a default value, such as two entries, based on deallocation of entries in queues 610-614. In such cases, the packet transmitter updates its stored value to the minimum number of cycles between link packets received by packet receiver 600 to avoid data collisions in queues 610-614.

[0066] Reference now Figure 7 , showing one embodiment of a method 700 for performing efficient data transfer in a computing system. A first count is stored, wherein the first count is a minimum number of cycles between link packets received by a receiver to avoid data collisions in a receive queue storing packets of a given type (box 702). A link packet including one or more command packets is sent to the receiver over a link (box 704). A second count of enabled network cycles is reset, wherein each enabled network cycle allows a next link packet to use the link (box 706). A third count is reset, wherein the third count is a number of candidate packets of a given type that are inserted into the link packet (box 708).

[0067] If the link is enabled by the network in the fabric (the "yes" branch of conditional box 710), the second count is incremented (box 712). If there is a candidate packet of the given type that meets the insertion condition (the "yes" branch of conditional box 714), a sum is determined by adding the first count, the third count, and one corresponding to the candidate packet (box 716). If the sum is less than the second count (the "yes" branch of conditional box 720), the candidate packet of the given type is inserted into the link packet (box 722). In addition, the third count is incremented (box 724). Then, the control flow of method 700 moves to box 726. Similarly, if there is no candidate packet of the given type that meets the insertion condition (the "no" branch of conditional box 714), or if the sum is not less than the second count (the "no" branch of conditional box 720), the control flow of method 700 moves to conditional box 726.

[0068] If the link packet is ready to be sent (the "yes" branch of conditional box 726), the control flow of method 700 returns to conditional box 704, in which a link packet including one or more command packets is sent to the receiver via the link. In some embodiments, the link packet is considered ready when the fabric transmission interface sends a signal or other indication that the link is available and the link packet is non-empty. If the link packet is not yet ready to be sent (the "no" branch of conditional box 726), method 700 waits until the next clock cycle (box 728). For example, if the fabric transmission interface does not send a signal or other indication that the link is available or the link packet is empty, the link packet is not yet ready to be sent. Then, the control flow of method 700 returns to conditional box 710 through box "A", where it is determined whether the link is enabled by the network in the fabric.

[0069] In various embodiments, the previously described methods and / or mechanisms are implemented using program instructions of a software application. The program instructions describe the behavior of the hardware in a high-level programming language (such as C). Alternatively, a hardware design language (HDL) such as Verilog is used. The program instructions are stored on a non-transitory computer-readable storage medium. Numerous types of storage media are available. During use, a computing system can access the storage medium to provide the program instructions and accompanying data to the computing system to execute the program. The computing system includes at least one or more memories and one or more processors that execute the program instructions.

[0070] It should be emphasized that the above embodiments are only non-limiting examples of implementations. Once the above disclosure is fully understood, numerous changes and modifications will become apparent to those skilled in the art. The appended claims are intended to be interpreted as covering all such changes and modifications.

Claims

1. A device comprising: Multiple queues, each queue is configured to store packets of a corresponding type; a plurality of queue arbiters, each queue arbiter configured to select a candidate packet from a corresponding queue of the plurality of queues; A packet arbiter comprising circuitry configured to: identifying a set of available queues from the plurality of queues, each available queue storing one or more candidate packets; In response to determining that a queue has a highest priority among the set of available queues, selecting the queue from the set of available queues; as well as In response to determining: There is available data storage space in the link packet to store the candidate packet from the selected queue; and The available data storage space in the link packet is aligned with a given boundary of the candidate packet, and the candidate packet from the selected queue is inserted into a link packet that includes space for multiple candidate packets.

2. The apparatus of claim 1, wherein in response to determining that the candidate packet from the selected queue does not satisfy an insertion condition, the packet arbitrator is further configured to notify a queue arbitrator corresponding to the selected queue to select a different candidate packet.

3. The apparatus of claim 1 , wherein in response to determining that the candidate packet from the selected queue does not satisfy an insertion condition, the packet arbiter is further configured to reduce the priority of the selected queue until one of the following is detected: The candidate packet is inserted into the link packet; and The link packet is sent.

4. The device of claim 1, wherein: One or more queues of the plurality of queues have a corresponding limit on a number of packets that can be inserted from the queue into the link packet before the link packet is sent on an available link.

5. The apparatus of claim 1 , wherein determining that the candidate packet from the selected queue satisfies an insertion condition comprises: A determination is made as to whether available data storage space exists at the receiver to store the candidate packet from the selected queue.

6. The apparatus of claim 1 , wherein in response to determining to send a link packet to a receiver over a link, the packetization arbiter is further configured to reset each of: a first count of enabled network periods, each enabled network period allowing a next link packet to use the link; and A second count of the number of candidate packets of a given type is inserted into the link packet.

7. The apparatus of claim 6, wherein in response to determining that the link is available for sending a next link packet to the receiver over the link, the packetization arbiter is further configured to increment the first count.

8. The apparatus of claim 7, wherein in response to determining that there is a candidate packet of the given type that satisfies the condition for insertion into the next link packet, the packetization arbiter is further configured to determine a sum by adding: The second count, one corresponding to the candidate packet, and The minimum number of cycles between link packets received by the receiver to avoid data collisions in the receive queue storing packets of the given type.

9. The apparatus of claim 2, wherein the set of available queues comprises one or more queues, the one or more queues: storing at least one valid packet; and Has not been selected a given number of times in a row.

10. A method comprising: Storing packets in a plurality of queues, each queue being configured to store packets of a corresponding type; selecting, by a plurality of queue arbiters, a candidate packet from each of the plurality of queues; identifying, by circuitry of the packet arbiter, a set of available queues from the plurality of queues, each available queue storing one or more candidate packets; In response to determining, by the circuitry, that a queue has a highest priority among the set of available queues, selecting, by the circuitry, the queue from the set of available queues; as well as In response to determining: There is available data storage space in the link packet to store the candidate packet from the selected queue; and The available data storage space in the link packet is aligned with a given boundary of the candidate packet, and the candidate packet from the selected queue is inserted by the circuit into a link packet that includes space for multiple candidate packets.

11. The method of claim 10, wherein one or more of the plurality of queues have corresponding limits on the number of packets that can be inserted from the queue into the link packet before the link packet is sent on an available link.

12. The method of claim 10, wherein in response to determining that a link packet is to be sent to a receiver over a link, the method further comprises resetting each of: a first count of enabled network periods, each enabled network period allowing a next link packet to use the link; and A second count of the number of candidate packets of a given type is inserted into the link packet.

13. The method of claim 12, wherein in response to determining that the link is available for sending a next link packet to the receiver via the link, the method further comprises: The first count is incremented.

14. The method of claim 13, wherein in response to determining that there is a candidate packet of the given type that satisfies the condition for insertion into the next link packet, the method further comprises: Determine the sum by adding: The second count, one corresponding to the candidate packet, and The minimum number of cycles between link packets received by the receiver to avoid data collisions in the receive queue storing packets of the given type.

15. A non-transitory computer-readable storage medium storing program instructions, wherein the program instructions are executable by a processor to perform the steps in the method of any one of claims 10 to 14.

Images