Scalable messaging (network-like) endpoint for PCIe switched networks based on PCIe address routing
Patent Information
- Application Number
- CN202580023695.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-04-09
- Filing Date
- 2025-04-07
- Publication Date
- 2026-11-17
AI Technical Summary
更进一步,对等端的出现和消失(上电/断电)未得到良好处理
Smart Images

Figure CN123125585A_ABST
Abstract
Description
[0001] Cross-references to related applications
[0002] This application claims the benefit and priority of U.S. Patent Application No. 18 / 630,349, filed April 9, 2024, entitled “Scalable Messaging (Similar Networking) Endpoint for PCIe Switched Networks (fabric) Based on PCIe Address Routing,” the entire contents of which are incorporated herein by reference. Technical Field
[0003] This application generally relates to scalable messaging (similar networking) endpoints for PCIe switched networks based on PCIe address routing. Background Technology
[0004] As understood in this paper, current PCIe switching network technologies lack support for peer-to-peer messaging or networking applications with a simple, scalable hardware and software stack, unlike Ethernet networks. As also understood in this paper, attempts to address this issue using non-transparent bridging memory windows to implement higher-level messaging protocols have several drawbacks, including a complex software stack and limited networking capabilities; the inability to support multicast or broadcast options in hardware because messages must be copied by each client software; and poor scalability to more than eight or sixteen systems. Furthermore, even with out-of-band / in-band management software, implementing higher-level messaging protocols using non-transparent bridging memory windows is difficult to configure. Moreover, doorbells / interrupts are not automatic and must be triggered by the sender as a separate transaction, which has several limitations (the number of available doorbells / interrupts, the number of interrupts per second at the destination). Additionally, security exposure is a concern because the local doorbell must be exposed to the switching network. Furthermore, the doorbell is not associated with other resources (including memory) shared between the two peers, but rather as a separate entity. This requires the management application to take several steps to connect the two peers in order to manage the doorbell for each queue shared between them. Moreover, the appearance and disappearance (power-on / power-off) of the peers are not handled well. Summary of the Invention
[0005] This principle addresses some or all of the aforementioned problems through a simple hardware solution that offloads peer-to-peer message traffic, including unicast, multicast, or broadcast messaging, to switching network hardware connected to a PCIe connection. This simple hardware solution leverages the PCIe and PCIe NTB features to present a networking DMA interface on a PCIe switching network. This improves performance, response time, and simplifies the software stack, thereby presenting a simple networking (NIC) interface to the local host.
[0006] An example environment in which this principle can be used is a computer game streaming server.
[0007] Therefore, one method includes: mapping a window segment address in a base address register (BAR) to a corresponding peer network ID to establish a mapping, and storing the mapping. The method also includes using the mapping to update the peer-to-peer network, where the peers are configured to communicate with each other via a peripheral computer high-speed interconnect (PCIe). Furthermore, the method includes: for a message to be sent from a sending peer to a destination peer, using the mapping to convert the network ID in the message to a corresponding destination window segment address. The method further includes writing the message to the corresponding destination window segment address, while at the destination peer, the message from the corresponding destination window segment address is placed in a receive-first-out (FIFO) storage device for further processing.
[0008] In some embodiments, the method includes identifying the next receive queue pointer and performing a direct mapping access (DMA) of messages in the FIFO storage device to the receive queue. This may also include generating an interrupt event for receiving messages.
[0009] In the non-restrictive examples, BAR includes BAR2.
[0010] In the example implementation, the mapping is stored in the hardware of the sending peer's switched network interface endpoint. The transformation can be performed during the processing of the send queue descriptor.
[0011] On the other hand, a computer-simulated streaming system includes at least a sending peer and a destination peer, configured to communicate with each other via a Peripheral Computer Interconnect (PCIe) switching network through corresponding Direct Memory Access (DMA) endpoints. The sending peer endpoint includes at least one Base Address Register (BAR) memory-mapped register space, at least one corresponding Send Descriptor Queue (TXQ), at least one segmented memory window in the BAR, and at least one DMA engine configured to control messages passing through the endpoint. The endpoint also includes at least one data structure stored in the endpoint's hardware to map Media Access Code (MAC) addresses to corresponding PCI memory windows on the destination peer.
[0012] The endpoint of the destination peer may include at least one receive buffer queue (RXQ), at least one segmented memory window in the BAR, and at least one DMA engine configured to control messages passing through the endpoint. The DMA engine of the sending peer endpoint is configured to use data structures stored in hardware to translate the destination peer address into a destination window segment address in order to write at least a first message to the destination window segment address over the PCIe switching network.
[0013] In another aspect, a device includes at least a first computerized peer and at least a first Direct Mapping Address (DMA) endpoint, the first DMA endpoint communicatively coupling the first computerized peer to at least one peripheral computer high-speed interconnect (PCIe) switching network. The DMA endpoint includes at least one host-side network interface and at least one switching network-side network interface, and has at least one DMA engine between them. The DMA endpoint is configured to perform switching based on a PCIe memory address translated from a network address associated with a destination endpoint, rather than based on the corresponding networking address of the destination endpoint.
[0014] The structural and operational details of this application are best understood with reference to the accompanying drawings, in which the same reference numerals refer to the same parts, and wherein: Attached Figure Description
[0015] Figure 1 This is a block diagram of an example system based on this principle; Figure 2 An example environment is shown; Figure 3 The high-level architecture is shown; Figure 4 The first example endpoint is shown; Figure 5 Further explanation of the endpoints is provided; Figure 6 The example logic is shown in the format of an example flowchart; Figure 7 Example endpoints are shown in multicast and broadcast message scenarios; and Figure 8 An example game data center is shown. Detailed Implementation
[0016] This disclosure generally relates to a computer ecosystem including various aspects of consumer electronics (CE) device networks, such as, but not limited to, computer gaming networks. Systems described herein may include server components and client components that can be networked to enable data exchange between the client and server components. Client components may include one or more computing devices, including game consoles such as Sony PlayStation®, or those manufactured by Microsoft, Nintendo, or other manufacturers; extended reality (XR) headsets such as virtual reality (VR) headsets, augmented reality (AR) headsets; portable televisions (e.g., smart TVs, internet-enabled televisions); portable computers such as laptops and tablets; and other mobile devices including smartphones and other examples discussed below. Client devices may also include servers. These client devices may operate in a variety of operating environments. For example, by way of example, some client computers may run on Linux, Microsoft's operating system, Unix, or an operating system manufactured by Apple or Google, or Berkeley Software Distribution or Berkeley Standard Distribution (BSD) OS (including BSD derivatives). These operating environments can be used to execute one or more browsing programs, such as browsers made by Microsoft, Google, or Mozilla, or other browser programs capable of accessing websites hosted by the internet servers discussed below. Furthermore, operating environments based on this principle can be used to execute one or more computer game programs.
[0017] A server and / or gateway may be used, which may include one or more processors executing instructions that configure the server to receive and send data over a network such as the Internet. Alternatively, the client and server may connect via a local intranet or virtual private network. The server or controller may be instantiated from a game console such as a Sony PlayStation®, a personal computer, etc.
[0018] Information can be exchanged between clients and servers over a network. For this purpose, and for security, servers and / or clients may include firewalls, load balancers, temporary storage devices, and proxies, as well as other network infrastructure for reliability and security. One or more servers may form a device for implementing methods to provide network members with secure communities such as online social networking sites or gaming networks.
[0019] A processor can be a single-chip or multi-chip processor capable of executing logic using various lines such as address lines, data lines, control lines, registers, and shift registers. A processor including a digital signal processor (DSP) can be an embodiment of circuitry. A processor system may include one or more processors.
[0020] Components included in one embodiment may be used in any suitable combination in other embodiments. For example, any of the various components described herein and / or shown in the accompanying drawings may be combined with, interchanged with, or excluded from other embodiments.
[0021] "A system having at least one of A, B and C" (similarly, "a system having at least one of A, B or C" and "a system having at least one of A, B, and C") includes systems having only A, only B, only C, both A and B, both A and C, both B and C, and / or both A, B and C.
[0022] Now refer to Figure 1 An example system 10 is illustrated, which may include one or more of the example devices described above according to these principles, and is further described below. The first example device included in system 10 is a consumer electronics (CE) device, such as an audio-visual device (AVD) 12, such as, but not limited to, a projector-based cinema display system, or an internet-enabled television with a television tuner (equivalently, a set-top box controlling a television). Alternatively, the AVD 12 may also be a computerized internet-enabled (“smart”) phone, tablet computer, laptop computer, head-mounted device (HMD) and / or a head-mounted device such as smart glasses or VR headsets, another wearable computerized device, a computerized internet-enabled music player, a computerized internet-enabled headset, a computerized internet-enabled implantable device such as an implantable skin device, etc. In any case, it should be understood that the AVD 12 is configured to implement these principles (e.g., to communicate with other CE devices to implement these principles, perform the logic described herein, and perform any other functions and / or operations described herein).
[0023] Therefore, to implement this principle, the AVD 12 can be composed of some or all of the components shown. For example, the AVD 12 may include one or more touch-enabled displays 14, which may be implemented using a high-definition or ultra-high-definition "4K" or higher resolution flat panel screen. The touch-enabled displays 14 may include, for example, capacitive or resistive touch sensing layers with an electrode grid for touch sensing conforming to this principle.
[0024] AVD 12 may also include one or more speakers 16 for outputting audio according to these principles, and at least one additional input device 18, such as an audio receiver / microphone, for inputting audible commands to control AVD 12. Example AVD 12 may also include one or more network interfaces 20 for communicating over at least one network 22 (such as the Internet, WAN, LAN, etc.) under the control of one or more processors 24. Thus, interface 20 may be, but is not limited to, a Wi-Fi transceiver, which is an example of a wireless computer network interface, such as, but not limited to, a mesh network transceiver. It should be understood that processor 24 controls AVD 12 to implement these principles, including controlling other elements of the AVD 12 described herein, such as controlling display 14 to present images thereon and receiving input from it. Furthermore, it should be noted that network interface 20 may be a wired or wireless modem or router, or other suitable interfaces such as a wireless telephone transceiver or the aforementioned Wi-Fi transceiver.
[0025] In addition to the foregoing, the AVD 12 may also include one or more input and / or output ports 26, such as an HDMI port or a USB port, for physical connection to another CE device, and / or include a headphone port for connecting headphones to the AVD 12 to present audio from the AVD 12 to the user via headphones. For example, input port 26 may be wired or wirelessly connected to a cable television or satellite source 26a containing audio and video content. Therefore, source 26a may be a standalone or integrated set-top box, or a satellite receiver. Alternatively, source 26a may be a game console or disc player containing content. When implemented as a game console, source 26a may include some or all of the components described below with respect to CE device 48.
[0026] AVD 12 may also include one or more computer memory / computer-readable storage media 28, such as disk-based or solid-state storage devices that are not transient signals, implemented in some cases as a stand-alone device within the AVD chassis, or as a personal video recording device (PVR) for playing back AV programs, or a video disk player located inside or outside the AVD chassis, or as removable storage media or a server described below. Furthermore, in some embodiments, AVD 12 may include a location or positioning receiver, such as, but not limited to, a cellular phone receiver, a GPS receiver, and / or an altimeter 30, configured to receive geographic location information from a satellite or cellular phone base station and provide that information to the processor 24, and / or in conjunction with the processor 24 to determine the altitude of the AVD 12.
[0027] Continuing with the description of AVD 12, in some embodiments, AVD 12 may include one or more cameras 32, which may be a thermal imaging camera, a digital camera such as a webcam, an IR sensor, an event-based sensor, and / or a camera integrated into AVD 12 and capable of being controlled by processor 24 to acquire pictures / images and / or videos in accordance with these principles. AVD 12 may also include a Bluetooth® transceiver 34 and other near-field communication (NFC) elements 36 for communicating with other devices using Bluetooth and / or NFC technologies, respectively. An example NFC element may be a radio frequency identification (RFID) element.
[0028] Furthermore, the AVD 12 may include one or more auxiliary sensors 38 that provide input to the processor 24. For example, the one or more auxiliary sensors 38 may include one or more pressure sensors forming a layer of the touch-enabled display 14 itself, and may be, but are not limited to, piezoelectric pressure sensors, capacitive pressure sensors, piezoresistive strain gauges, optical pressure sensors, electromagnetic pressure sensors, etc. Other sensor examples include pressure sensors; motion sensors such as accelerometers, gyroscopes, rangefinders, or magnetic sensors; infrared (IR) sensors; optical sensors; speed and / or rhythm sensors; event-based sensors; and gesture sensors (e.g., for sensing gesture commands). Thus, the sensor 38 may be implemented by one or more motion sensors, such as individual accelerometers, gyroscopes, and magnetometers, and / or an inertial measurement unit (IMU), which typically includes a combination of accelerometers, gyroscopes, and magnetometers to determine the position and orientation of the AVD 12 in three-dimensional space; or by an event-based sensor such as an event detection sensor (EDS). An EDS conforming to this disclosure provides an output indicating a change in light intensity sensed by at least one pixel of the light-sensing array. For example, if the light sensed by the pixel is decreasing, the EDS output can be -1; if the light is increasing, the EDS output can be +1. No change in light intensity below a certain threshold can be indicated by an output binary signal of 0.
[0029] AVD 12 may also include an over-the-air (OTA) television broadcast port 40 for receiving OTA television broadcasts that provide input to processor 24. In addition to the foregoing, it should be noted that AVD 12 may also include an infrared (IR) transmitter and / or an IR receiver and / or an IR transceiver 42, such as an IR Data Association (IRDA) device. A battery (not shown) may be provided to power AVD 12, and a kinetic energy harvester may be provided to convert kinetic energy into electrical energy to charge the battery and / or power AVD 12. A graphics processing unit (GPU) 44 and a field-programmable gate array 46 may also be included. One or more tactile / vibration generators 47 may be provided to generate tactile signals that can be sensed by a person holding or in contact with the device. Thus, the tactile generator 47 may use a motor connected to an eccentric and / or unbalanced counterweight via a rotatable shaft of a motor to vibrate all or part of AVD 12, such that the shaft can rotate under the control of the motor (and thus under the control of a processor such as processor 24) to produce vibrations of various frequencies and / or amplitudes, as well as force simulations in various directions.
[0030] It may also include light sources, such as projectors, for example, infrared (IR) projectors.
[0031] In addition to AVD 12, system 10 may include one or more other CE device types. In one example, the first CE device 48 may be a computer game console that can be used to send computer game audio and video to AVD 12 via commands sent directly to AVD 12 and / or via a server described below, while the second CE device 50 may include components similar to the first CE device 48. In the illustrated example, the second CE device 50 may be configured as a computer game controller operated by a player or a head-mounted display (HMD) worn by a player. The HMD may include a head-up transparent display or a non-transparent display to present AR / MR content or VR content (more generally, extended reality (XR) content), respectively. The HMD may be configured as a glasses-type display or as a bulky VR-type display sold by a computer game device manufacturer.
[0032] In the example shown, only two CE devices are illustrated; it should be understood that fewer or more devices may be used. The devices described herein may implement some or all of the components shown for AVD 12. Any components shown in the following figures may be combined with some or all of the components shown for AVD 12.
[0033] Referring now to at least one server 52 described above, it includes at least one server processor 54, at least one tangible computer-readable storage medium 56 (such as a disk-based or solid-state storage device), and at least one network interface 58, which, under the control of the server processor 54, allows communication with other illustrated devices via network 22 and, in practice, facilitates communication between the server and client devices according to this principle. It should be noted that the network interface 58 may be, for example, a wired or wireless modem or router, a Wi-Fi transceiver, or other suitable interface such as, for example, a wireless telephone transceiver.
[0034] Therefore, in some embodiments, server 52 may be an internet server or an entire server "group" and may include and perform "cloud" functionality, such that, in the example embodiment, devices of system 10 can access a "cloud" environment via server 52 for use, for example, online gaming applications. Alternatively, server 52 may be implemented by one or more game consoles or other computers located in the same room or vicinity as the other devices shown.
[0035] The components shown in the following figures may include some or all of the components shown herein. Any user interface (UI) described herein may be combined and / or extended, and UI elements may be mixed and matched between UIs.
[0036] This principle can be applied to various machine learning models, including deep learning models. Machine learning models conforming to this principle can be trained using a variety of algorithms, including supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, feature learning, self-learning, and other forms of learning. Examples of such algorithms that can be implemented by computer circuits include one or more neural networks, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and a class of RNNs known as long short-term memory (LSTM) networks. Generative pre-trained transformers (GPTTs) can also be used. Support vector machines (SVMs) and Bayesian networks can also be considered examples of machine learning models. In addition to the network types mentioned above, the models in this paper can be implemented using classifiers.
[0037] As understood in this paper, performing machine learning can therefore involve accessing training data and subsequently training a model on that training data so that the model can process further data to make inferences. Therefore, an artificial neural network / AI model trained via machine learning can include an input layer, an output layer, and multiple hidden layers in between, which are configured and weighted to infer appropriate outputs.
[0038] Figure 2An example environment in which this principle can be used is illustrated. A computer emulation server system 200, such as a computer game server or multiple servers, communicates with one or more end-user game consoles 202 and / or displays 204 to present a computer emulation, such as a computer game, on the displays, if necessary, under the control of a player-operated end-user computer game controller 206. This principle can be used to implement server system 200.
[0039] Now refer to Figure 3 This illustrates a rack-level network of computer servers, referred to herein as “hosts” 300, each host 300 having a corresponding Peripheral Computer Interconnect (PCIe) Messaging Direct Memory Access (DMA) endpoint 302 connected to a PCIe switching network 304 via a corresponding PCIe link 306. Link 306 can appear as a non-transparent bridge, to be enumerated, configured, and managed by a management host 308 connected to the same PCIe switching network 304 as the hosts 300. Alternatively, the switching network side can appear as a networking endpoint instead of a non-transparent bridge, affecting only the software stack on the management host but providing the same functionality. Therefore, this principle is based on a rack / row-level PCIe switching network where each host connects to the switching network using a non-transparent bridge. When each host sees the messaging / networking endpoint on its PCIe bus, data between peers is transferred through a non-transparent memory window on the PCIe switching network side, exchanged based on PCIe memory addresses rather than the endpoint's networking ID.
[0040] This principle aims to leverage the Ethernet principle in PCIe to enable the transmission of messages, including “doorbell” messages, between hosts 300 via a switching network 304 using various protocols, while minimizing the number of messages the software needs to send to all peer hosts by better scaling the hardware described herein.
[0041] Figure 4 It shows Figure 3 Details of the DMA endpoint 302 are as follows. To implement this principle, up to six 32-bit addresses or three 64-bit addresses (base address registers (BAR0, BAR2, BAR4)) are used as windows to enter the device on both the host side and the switching network side of endpoint 302, exposing the Ethernet network interface 400 (host side) and 402 (switching network side). Each BAR typically describes the memory location of a memory region or window (base address + width) used for data transfer. As discussed further below, a table in the hardware maps Media Access Control (MAC) addresses to corresponding PCI memory windows, and Figure 3Multiple hosts 300 can write to a single window, with writes queued in a first-in-first-out (FIFO) buffer 404. Endpoint 302 may also include receive queues 406 on both the host and switching network sides, and send queues 408 on both sides. A processor-implemented DMA engine 410 can coordinate message transfers.
[0042] Figure 5 The diagram illustrates two host peers 300 communicating with each other via corresponding message passing DMA endpoints 304. As shown, each host 300 may include a corresponding system memory 500 and a message passing / network protocol stack 502 that interfaces with the corresponding endpoint 304.
[0043] exist Figure 5 The image shows the sending component of the left endpoint (the host peer writing data) and the receiving component of the right endpoint (the target peer).
[0044] exist Figure 5 In the example, the upstream portion of each endpoint 304 includes the BAR0 memory-mapped register space 504. A set of MSI-x interrupts (error, receive, transmit complete, etc.) can also be provided.
[0045] The transmitting component of each endpoint also includes a corresponding set of transmit descriptor queues (TXQ) 506, while the receiving component includes a set of receive buffer ptr queues (RXQ) 508. These queues can be implemented in hardware or software. Each networking endpoint is associated with a unique corresponding networking ID in the switched network (locally managed or hard-coded). As described above, each endpoint includes a table 510 in hardware, such as a table in a solid-state storage device, that maps Media Access Code (MAC) addresses to the corresponding PCI memory windows of the receiver.
[0046] The switching network side of each endpoint includes a corresponding segmented memory window in BAR2 512 (in Figure 5 This is referred to as a "segmented receive frame buffer," similar to a non-transparent bridge endpoint. Memory segments can be used. Memory segments can be allocated in multiples of 4KB, powers of two. The size of the memory window segment can be selected such that it should accommodate the maximum possible frame size of a single message. PCIe alignment and typical computer system memory page sizes require a minimum window size of 4KB, even if the expected message delivery frame is less than 4KB.
[0047] Figure 3 The software in the management host 308 shown maps window segments so that each peer's network ID corresponds to one segment. The management software is also responsible for updating all peers using these mappings. Each peer host has a window segment for that peer in another peer participating in the PCIe switching network.
[0048] Each endpoint's DMA engine 410 can process the transmit queue descriptor, translate the destination peer address to the destination window address, and write it to the PCIe switching network address (using Transport Layer Packet (TLP) for PCIe writes, where the message is the TLP payload). This translation can be done according to the mapping in the corresponding lookup table 510 of the endpoint. The receiving peer's DMA engine 410 then stores the message passing frame arriving at the window from the remote peer into RXFIFO 514. The DMA engine 410 retrieves the next RXQ pointer of the message passing frame from RXQ 508 and performs DMA on the FIFO entry to transfer it to the RXQ buffer in the system memory of the upstream local host 300, while generating an interrupt event for receiving data. Note that peer-to-peer messages are transmitted via PCIe switching network 516.
[0049] It should also be noted that Figure 3 The management host 308 can always send packets to and receive packets from the other host through its transmit (TXQ) queue and receive (RXQ) queue without address translation of the destination network ID mentioned above, because the TXQ / RXQ of the management host side endpoint is a dedicated queue between the management host and the connected client host.
[0050] Figure 6 The above is illustrated in flowchart format. Starting with state 600, a unique network ID (locally managed or hard-coded) is assigned to each network endpoint within the switched network. This ID can be, for example, a MAC address. A window segment in BAR2 is mapped, for example, by the management software to the corresponding network ID of each peer, and this mapping is, for example, stored in... Figure 5 In the hardware at position 510.
[0051] Status 602 indicates that, in one example, the management software uses the mapping from status 600 to update all peers. Therefore, each peer host has a window segment for that peer host among the other peers participating in the PCIe switching network.
[0052] Moving to state 604, for messages sent from the sending peer to the destination peer, the logic moves to state 606 to translate the destination peer address into the corresponding destination window address identified according to the mapping at state 600, thereby effectively translating Ethernet-type MAC addresses into PCIe switched network window addresses. The logic at state 606 can be executed by the sending peer's DMA engine 410 when it processes the send queue descriptor.
[0053] At state 608, for example, the DMA engine writes the message to the PCIe switched network window address. State 610 indicates that, at endpoint 304 of the destination peer, the DMA engine of the destination peer places a message passing frame arriving from the remote peer into RX FIFO 514. As part of this, the DMA engine identifies the next RXQ pointer and performs a direct mapping access (DMA) from the entry in FIFO 514 to the RXQ buffer 508 in the upstream local host, while generating an interrupt event for reception.
[0054] Figure 5 This illustrates unicast packet delivery from one source peer to one destination peer system, while Figure 7 This illustrates multicast or broadcast packet delivery from one peer to multiple destination peers in a PCIe switching network. It should be noted that for broadcast rather than multicast, essentially the same principle applies by adding all destinations to a multicast group and specifically designating that multicast group as a broadcast group.
[0055] exist Figure 7 In the middle, the components 304 of the sending endpoint and the destination endpoint are... Figure 5 The components are largely the same, with the following exceptions. Data structures such as lookup table 700 associate (or map) network multicast / broadcast addresses with corresponding PCIe multicast addresses. This can be done by management software, which in turn keeps the client host updated on this mapping. The address of a multicast / broadcast message is translated into an equivalent PCIe multicast address, for example, by the sending-side DMA engine using this mapping. To join a multicast group, the endpoint's receive window is added to the PCIe multicast range of that multicast group.
[0056] Furthermore, the PCIe switching network 702 through which messages are transmitted from one peer to another includes one or more switches 704, which distribute multicast packets to their members according to a membership table in the switch, thereby enabling... Figure 7 The diagram shows the delivery of a copy of the packet to each of endpoints 706 and 708 in the multicast group. The PCIe switch implements this packet replication, and the provision of copies to members is done in accordance with the PCI-SIG specification for PCIe. The PCIe switching network executes the PCIe standard to ultimately deliver packets to their destination.
[0057] Figure 8An example gaming data center is shown. Each server rack includes compute servers, accelerator units (GPUs, DPUs, FPGAs, video processing, AI inference / ML hardware), and storage service / servers. All of these are connected via multiple switching networks (typically one, two, or three; as an example, one Ethernet switching network, one PCIe switching network, and one storage networking switching network), and most switching networks connect multiple racks, even connecting to the outside world via the Internet.
[0058] While specific embodiments have been shown and described in detail herein, it should be understood that the subject matter covered by the invention is limited only by the claims.
Claims
1. A method comprising: Map the window segment address in the base address register (BAR) to the corresponding peer network ID to establish a mapping; Store the mapping; The mapping is used to update the peers in the peer network, which are configured to communicate with each other via PCIe (Peripheral Computer Interconnect). For a message to be sent from the sending peer to the destination peer, the mapping is used to convert the network ID in the message into the corresponding destination window address. Write the message to the corresponding destination window segment address; At the destination peer, the message from the corresponding destination window address is placed in a receive-first-out (FIFO) storage device for further processing.
2. The method of claim 1, further comprising identifying a next receive queue pointer and performing a direct mapping access (DMA) from the message in the FIFO storage device to the receive queue.
3. The method of claim 2, further comprising generating an interruption event for receiving the message.
4. The method according to claim 1, wherein, The BAR includes BAR2.
5. The method according to claim 1, wherein, The mapping is stored in the hardware of the switching network interface endpoint of the sending peer.
6. The method according to claim 1, wherein, The conversion is performed during the processing of the send queue descriptor.
7. A computer-simulated streaming system, comprising: At least a sending peer and a destination peer are configured to communicate with each other via a Peripheral Computer Interconnection (PCIe) switching network through corresponding Message Passing Direct Memory Access (DMA) endpoints. The endpoint of the sending peer includes at least one base address register (BAR) memory-mapped register space, at least one corresponding transmit descriptor queue (TXQ), at least one segmented memory window in the BAR, and at least one direct mapped access (DMA) engine configured to control messages passing through the endpoint, and at least one data structure stored in the hardware of the endpoint to map media access code (MAC) addresses to corresponding PCI memory windows of the destination peer. The endpoint of the destination peer includes at least one receive buffer queue (RXQ), at least one segmented memory window in the BAR, and at least one DMA engine configured to control messages passing through the endpoint; wherein, The DMA engine of the endpoint of the sending peer is configured to use the data structure stored in hardware to convert the destination peer address into a destination window address in order to write at least a first message to the destination window address via the PCIe switching network.
8. The system according to claim 7, wherein, The DMA engine is configured to use transport layer packets (TLP) to write the first message to the destination window address.
9. The system according to claim 7, wherein, The DMA engine of the endpoint of the destination peer is configured to store the first message in a receive-first-out (Rx FIFO) manner.
10. The system according to claim 9, wherein, The DMA engine of the endpoint of the destination peer is configured to use a receive queue (RXQ) pointer to perform DMA to move the first message from the Rx FIFO to a buffer identified by the RXQ pointer.
11. The system according to claim 10, wherein, The DMA engine of the endpoint of the destination peer is configured to generate an interrupt event for receiving the first message.
12. An apparatus comprising: At least the first computerized peer; At least a first direct mapped address (DMA) endpoint, the first direct mapped address (DMA) endpoint communicatively coupling the first computerized peer to at least one peripheral computer high-speed interconnect (PCIe) switching network, the DMA endpoint including at least one host-side network interface and at least one switching network-side network interface, and having at least one DMA engine between the two; The DMA endpoint is configured to exchange based on a PCIe memory address derived from a network address and associated with the destination endpoint, rather than based on the corresponding network address of the destination endpoint.
13. The device according to claim 12, wherein, At least one of the network interfaces includes an Ethernet interface.
14. The device according to claim 12, wherein, At least one of the network interfaces includes a base address register (BAR) that serves as a window into the endpoint.
15. The device according to claim 14, wherein, The BAR includes BAR0.
16. The device according to claim 14, wherein, The BAR includes BAR2.
17. The device according to claim 14, wherein, The BAR includes BAR4.
18. The device according to claim 12, wherein, The endpoint includes at least one data structure in the hardware that associates a Media Access Code (MAC) address with a corresponding PCIe memory window.
19. The device according to claim 18, wherein, Multiple hosts can write to a single PCIe memory window, where writes are queued in at least one first-in-first-out (FIFO) buffer.
