A bus system, a communication method, and related devices

Abstract

The embodiment of the application discloses a bus system, a communication method and related equipment, the system is a graph structure composed of a plurality of master nodes, a plurality of switches and a plurality of slave nodes;Any multi-path bus subsystem in the graph structure includes a first master node, a first slave node and N switches;N switches include N1 first switches adjacent to the first master node, N2 second switches, N3 third switches adjacent to the first slave node;Any first switch is adjacent to any third switch or connected through a second switch;The first master node is used for sending enumeration messages to the first slave node multiple times, and determining a plurality of routing paths between the first master node and the first slave node;Each routing path at least sequentially passes through one or more of N1 first switches, S of N2 second switches and one or more of N3 third switches. The embodiment of the application can support multi-path access to devices on the bus.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a bus system, communication method and related equipment. Background Technology

[0002] With the rapid development of information technology, data-intensive and computationally intensive application scenarios such as artificial intelligence (AI), autonomous driving computing, and cloud computing are becoming increasingly widespread. The entire computing system will become more and more complex, and various proprietary computing devices (such as graphics processing units (GPUs) and tensor processing units (TPUs)) will be widely integrated and applied.

[0003] This will inevitably lead to increasingly higher requirements for interconnect buses, making high-bandwidth, low-latency, low-power, and easy-to-implement interconnect buses increasingly important. Future bus interconnects will be unordered, point-to-point graph structures. A node on the bus may be reachable by multiple paths (multipathing), and a node on the bus can be managed as a device within a central processing unit (CPU) node. The bus may also have multiple hosts (multi-host). Current industry interconnects are following this direction. For example, Compute Express Link (CXL) 2.0 supports CXL's multi-host functionality by defining special interconnect devices (such as CXL switches) and interconnect management mechanisms. Furthermore, future evolutions of CXL will further emphasize point-to-point computing and multi-path node interconnects. Therefore, from an industry trend perspective, bus device management supporting multi-host, multi-path connections is an inevitable trend.

[0004] However, the Peripheral Component Interconnect Express (PCIe), currently the most popular interconnect bus in the industry, is a bus with a strictly top-down tree topology and does not natively support multi-host and multipathing. For CXL 2.0, a special CXL switch and a specific interconnect topology (e.g., a Type 3 CXL device connected to a CXL switch) are required to achieve multi-host functionality. Furthermore, future CXL evolutions (e.g., CXL 3.0) may support multipathing, but this will certainly still require a special CXL switch and a specific interconnect topology.

[0005] Therefore, most existing technologies rely on special intermediate connection devices and specific interconnection topologies to "deceive" multiple hosts, making each host believe that it has "exclusive" access to these node devices. This cannot truly support multiple hosts and multiple paths, and it will undoubtedly greatly increase the latency, cost, and topology limitations of device interconnection. Summary of the Invention

[0006] This application provides a bus system, communication method, and related devices that can support multipath access to devices on the bus.

[0007] In a first aspect, embodiments of this application provide a bus system, which is a graphical structure composed of multiple master nodes, multiple switches, and multiple slave nodes connected by a bus. Any multi-path bus subsystem in the graphical structure includes a first master node, a first slave node, and N switches. The N switches include N1 first switches adjacent to the first master node, N2 second switches, and N3 third switches adjacent to the first slave node. Any first switch is adjacent to any third switch, or connected through one or more second switches. N1, N2, and N3 are all positive integers less than or equal to N. The first master node is used to send multiple whistleblower messages to the first slave node and determine multiple routing paths between the first master node and the first slave node based on at least one switch. The at least one switch is the switch that each whistleblower message passes through from the first master node to the first slave node. Each of the multiple routing paths passes at least sequentially through one or more of the N1 first switches, S of the N2 second switches, and one or more of the N3 third switches. S is a natural number less than or equal to N3.

[0008] Conventional PCIe bus interconnects have a tree topology with a strict top-down hierarchy. Nodes at each level can only be associated with one node in the layer above them, but may be associated with multiple nodes in the layer below. This results in a single, top-down path from the host to any node device (e.g., GPU, TPU, or network card). This severely limits the topology's scalability and cannot meet the demands of increasingly complex computing systems with ever-growing volumes of data. In this embodiment, the existing top-down tree structure of the PCIe bus is broken. The host (i.e., master node), node devices (i.e., slave nodes), and switches are connected to form a flattened graphical structure. In this graphical structure, any node (which can be a master node, a slave node such as a GPU or TPU, or a switch, etc.) can be associated with any other node. Based on this, multiple switches act as intermediary devices connecting master and slave nodes, and through their interconnections, multiple physical links can be constructed from a master node (e.g., the first master node) to a slave node (e.g., the first slave node). In the process of the master node discovering all devices in the entire topology by sending swarm reports, each swarm report sent by the master node may enumerate a slave node via a certain physical link. The master node can record the routing path from the master node to the slave node based on which port of the master node the swarm report originates from, which port of the switch it passes through, etc., within that physical link. Furthermore, as mentioned above, in this application, under the premise of constructing multiple physical links between a master node and a slave node through multiple switches (e.g., a first switch, a second switch, and a third switch), multiple swarm reports sent by the host may reach the same slave node via multiple different paths. Thus, the master node can record multiple different routing paths between the master node and the slave node by sending swarm reports multiple times. Therefore, compared with the tree structure in the prior art, the bus interconnection topology implemented in this application is a graph structure, which can support multipathing access to node devices. That is, a node device can be reached by multiple paths, which makes bus interconnection simpler, provides more design space, makes it more scalable, and has a larger capacity, thereby meeting the increasingly complex and massive computing needs of users.

[0009] It should be noted that in this embodiment, the master node can be a master processing chip, and the slave nodes can be slave processing chips, memory, or dedicated hardware processing units, etc. Specifically, the master node can be a host including multiple central processing units, and the slave nodes can be computing units such as GPUs and TPUs, or storage devices such as solid-state drives, etc. The master node can access the corresponding slave nodes through the aforementioned multiple routing paths, and then call upon the computing resources or data resources in the slave nodes to perform a series of computational processes, etc.

[0010] Furthermore, the master node described in the embodiments of this application can also be called a host, the slave node can also be called a node device, and correspondingly, the management master node can also be called a management host, the management master node information register can also be called a management host information register, the management master node signature can also be called a management host signature, and so on. These will not be explained repeatedly in the future.

[0011] In some possible implementations, the first master node is further configured to query the visible bit in the routing status register of the first slave node based on the sent spoofing message; if the visible bit is 0, then assign a corresponding device number to the first slave node; wherein, the visible bit being 0 indicates that the first slave node has not yet been enumerated and discovered; the first slave node is configured to save the device number assigned by the first master node to the routing status register and set the visible bit to 1; wherein, the visible bit being 1 indicates that the first slave node has already been enumerated and discovered.

[0012] In this embodiment, as described above, since a node device is reachable by multiple paths, a node device can be enumerated by the host via multiple paths during the enumeration process. When a node device is first discovered by the host via the first routing path, the visible bit in the node device's routing status register can be set to 1 to indicate that the node device has been enumerated. Thus, when the host sends an enumeration report to the node device via other routing paths, it can determine that the node device has been enumerated based on the visible bit being 1, without needing to assign a device number; it only needs to record the new routing path used to send the enumeration report. In this way, enumeration under multiple paths can be implemented by recording historical discovery states, avoiding repeated enumeration and infinite loops under multiple paths, and completing enumeration efficiently and accurately.

[0013] In some possible implementations, the first master node is further configured to send a first configuration message to the first slave node to obtain management authority over the first slave node; the first configuration message carries the master node password and master node number of the first master node; the first slave node is further configured to receive the first configuration message and, based on the first configuration message, set the signature bit in the management master node information register of the first slave node to 1; wherein, the signature bit being 1 is used to indicate that the first slave node currently has management master node status, and other master nodes in the system cannot obtain management authority over the first slave node; the first slave node is further configured to save the master node password and master node number of the first master node to the management master node information register.

[0014] In this embodiment, a host can obtain management rights over a node device by sending a first configuration message to the node device, setting the signature bit in the node device's management host information register to 1, and writing its own host password (i.e., master node password) and host number (i.e., master node number) into the node device. It should be noted that after the signature bit is set to 1, meaning that a host has obtained management rights, other hosts will be unable to obtain management rights over the node device, thus ensuring that the node device has a unique management host (i.e., master node) for a considerable period, ensuring clarity on the device management plane. In some possible embodiments, a host can obtain management rights over multiple different node devices, and when obtaining management rights for different node devices, the host password assigned to each node device can be different. The host password can serve as an important credential for subsequent verification of the management host's identity, thereby ensuring security on the device management plane. This management right includes, but is not limited to, arbitrating resource contention between node devices, handling abnormal errors of node devices, and managing and configuring basic characteristics of node devices (such as the maximum supported message length), etc. In some possible embodiments, a mechanism is also defined to separate the management and use of node devices. As mentioned above, a node device can only be managed by one host, but can be accessed and used by multiple hosts (e.g., including the use of data resources and computing resources, etc.). This application does not specifically limit this.

[0015] In some possible implementations, the first master node is further configured to send a second configuration message to the first slave node to revoke its management privileges over the first slave node; the second configuration message carries the master node password and master node number of the first master node; the first slave node is further configured to receive the second configuration message, and if the master node password and master node number carried in the second configuration message are consistent with the master node password and master node number stored in the management master node information register, then the signature bit in the management master node information register is set to 0; wherein, the signature bit being 0 is used to indicate that the first slave node currently does not have a management master node.

[0016] In this embodiment, after obtaining management access to a node device, the host can, based on actual needs (e.g., based on requests from other hosts or fault reports from the current host), send a second configuration message to the node device to reset the signature bit in the node device's management host information register (i.e., the management master node information register) to 0, thereby canceling management access to the node device. This improves the flexibility of the device management mechanism and meets actual user needs. Furthermore, when the node device receives the second configuration message, it needs to verify the host password and host number carried in the message. Only if they match the host password and host number saved during the initial management host signature is the host allowed to cancel management access. In other words, this embodiment also defines a verification mechanism for the node device's management host, ensuring that the node device's management host cannot be forged, further ensuring clear and reliable device management.

[0017] In some possible implementations, the first master node is further configured to, after acquiring management authority over the first slave node, respond to a query message sent by the first slave node by sending an in-place message to the first slave node; or, send the in-place message to the first slave node at a first time interval.

[0018] In this embodiment, a presence confirmation process for the management host is also defined to ensure the robustness of the bus system. After the host obtains management access to a node device, it can respond to a query message sent by the node device by sending a presence message to that node device, or proactively send a presence message to the node device at certain time intervals, thereby notifying the node device that its management host is currently functioning correctly and ensuring the real-time presence of the management host. Furthermore, under normal circumstances, the frequency of presence confirmation interactions between the host and node devices can be very low, and the impact on bus bandwidth is almost negligible.

[0019] In some possible implementations, the first slave node is further configured to set the signature bit in the management master node information register to 0, thereby canceling the first master node's management authority over the first slave node, under preset conditions; wherein, the preset conditions include: after the first master node acquires management authority over the first slave node, the first slave node does not receive the presence message sent by the first master node within a preset time, or, after the first slave node sends K query messages to the first master node, it does not receive the presence message sent by the first master node; K is an integer greater than or equal to 1.

[0020] In this embodiment, the node device may also consider the management host to be in an abnormal state if it does not receive an in-place message from its management host (e.g., the first master node) for a long time, or if it sends multiple query messages to its management host without receiving a response. It may then cancel its management privileges so that it can subsequently acquire a new management host, thereby ensuring the reliable operation of the entire bus system.

[0021] In some possible implementations, the first slave node is further configured to send a broadcast message to at least one master node in the system; the broadcast message is configured to indicate that the first slave node does not currently have a managing master node; the at least one master node is configured to receive the broadcast message and, based on the broadcast message, send the first configuration message to the first slave node to obtain management authority over the first slave node.

[0022] In this embodiment, when a node device determines that the current management host (e.g., the first master node) is abnormal and cancels its management privileges, it can also send a broadcast message to multiple hosts in the bus system to notify the multiple hosts to obtain management privileges for the node device, so that the node device can have a new management host, thereby ensuring the reliable operation of the entire bus system.

[0023] In some possible implementations, the multipath bus subsystem further includes a second master node; the N switches further include N4 fourth switches adjacent to the second master node; wherein any fourth switch is adjacent to any third switch, or connected through one or more second switches; N4 is a positive integer less than or equal to N; the second master node is used to send multiple whistleblower messages to the first slave node, and determine multiple routing paths between the second master node and the first slave node based on at least one switch; the at least one switch is the switch that each whistleblower message passes through from the second master node to the first slave node; wherein each of the multiple routing paths passes through at least one or more of the N4 fourth switches, S of the N2 second switches, and one or more of the N3 third switches in sequence.

[0024] In this embodiment, a node device (e.g., a first slave node) can be discovered by multiple different hosts. That is, other hosts (e.g., a second master node) can send multiple report messages to the node device via multiple switches connected to it, thereby recording multiple routing paths with the node device, and so on. This greatly expands the scope of use of node devices, meeting the needs of various hosts for a large number of node devices, allowing any host to access node devices under other hosts, and thus calling upon more computing resources to perform more complex computational processing. Furthermore, as mentioned above, although a node device can be used by multiple hosts, it can only be managed by one host. This single-host management mechanism makes management clearer.

[0025] In some possible implementations, the multipath bus subsystem includes a first node domain and a second node domain; the first node domain includes a first master node and multiple first slave nodes, and the second node domain includes a second master node and multiple second slave nodes; the second master node is specifically configured to: after all first slave nodes in the first node domain are assigned device numbers by the first master node, and after all second slave nodes in the second node domain are assigned device numbers by the second master node, send multiple report messages to the first slave nodes, and determine multiple routing paths between the second master node and the first slave nodes based on at least one switch.

[0026] In some possible implementations, the N switches further include a first cross-node domain switch belonging to a first node domain and a second cross-node domain switch belonging to a second node domain; the first port of the first cross-node domain switch is connected to the second port of the second cross-node domain switch; if some or all of the first slave nodes in the first node domain have not yet been assigned device numbers by the first master node, and / or some or all of the second slave nodes in the second node domain have not yet been assigned device numbers by the second master node, then the data link between the first port and the second port is closed; if all the first slave nodes in the first node domain have been assigned device numbers by the first master node, and all the second slave nodes in the second node domain have been assigned device numbers by the second master node, then the data link between the first port and the second port is open, so that the second master node sends multiple report messages to the first slave nodes through the second port and the first port, and determines multiple routing paths between the second master node and the first slave node based on at least one switch.

[0027] In this embodiment, the bus system can be divided into multiple node domains. Thus, a host can enumerate and discover node devices in its current node domain before proceeding to enumerate and discover node devices in other node domains, thereby recording multiple routing paths between node devices in other node domains. In short, the host's enumeration and discovery process begins within its current node domain. Before completing the enumeration and discovery of node devices within its own node domain, this node domain may not establish data flow connections with any node devices in other node domains. This supports device enumeration and discovery across multiple host nodes (i.e., across networks or different system management software) and ensures that different enumeration software does not assign multiple device IDs to the same node device, or that the same device ID is assigned to different node devices, preventing conflict scenarios.

[0028] In some possible implementations, the second master node is a remote master node that is network-connected to the first master node via a switch; the second master node is specifically used to: access the first slave node within the domain of the first node via a network connection, so as to call the computing resources in the first slave node or read the stored data in the first slave node.

[0029] In some possible implementations, the first master node is the central processing unit (CPU) in the first terminal; the first master node is also used to call the computing resources of at least one second slave node or read stored data from the at least one second slave node via a network connection; the second slave node is a graphics processing unit (GPU), solid-state drive (SSD), accelerator, network interface card (NIC), or tensor processing unit (TPU) in the second terminal. Optionally, the first terminal and the second terminal can be smartphones, tablets, desktop computers, computers, servers, etc., and this application embodiment does not specifically limit them.

[0030] In this embodiment, based on a graphical structure, a node device on the bus can be accessed and used by multiple hosts. Furthermore, besides being accessed and used by a nearby host connected via a wired connection, a node device can also be accessed and used by other hosts connected remotely via a wireless connection. This allows hosts to access remote computing resources and store data, thus better meeting the host's demand for massive computing resources during complex computational processing. Network connectivity undoubtedly greatly enhances the scalability of the entire bus system, further breaking through the limitations of existing PCIe bus connections.

[0031] In some possible implementations, the slave node is any one of an image processor, solid-state drive, accelerator, network card, tensor processing unit, embedded neural network processor (NPU), digital signal processor (DSP), image signal processor (ISP), or switch.

[0032] In this embodiment, the node device can be an image processor, solid-state drive, accelerator, network interface card (NIC), tensor processing unit, embedded neural network processor (NPU), digital signal processor (DSP), image signal processor (ISP), and switch, etc., and this embodiment does not specifically limit it. Based on the more scalable and design-space-flexible graphical structure in this embodiment, various node devices can be added to the structure, allowing the host on the bus to access the corresponding node device according to actual needs to use its computing resources or data resources, thereby meeting increasingly complex and massive computing demands.

[0033] In some possible implementations, the master node includes one or more central processing units (CPUs).

[0034] In this application embodiment, the host can be a computing system with one or more central processing units. In some possible embodiments, the host may also include main memory, cache, internal interconnect bus, input / output (I / O) interfaces, etc., which are not specifically limited in this application embodiment. Based on the more scalable and design-space-flexible graphics architecture in this application embodiment, the host can use the computing resources (such as computing units in the GPU) and data resources of each node device to perform computing processing through multiple different routing paths, thereby meeting increasingly complex and massive computing demands.

[0035] Secondly, embodiments of this application provide a communication method applied to a bus system, wherein the bus system is a graphical structure composed of multiple master nodes, multiple switches, and multiple slave nodes connected by a bus; any multi-path bus subsystem in the graphical structure includes a first master node, a first slave node, and N switches; the N switches include N1 first switches adjacent to the first master node, N2 second switches, and N3 third switches adjacent to the first slave node; wherein any first switch is adjacent to any third switch, or connected through one or more second switches; N1, N2, and N3 are all positive integers less than or equal to N; the method includes: sending multiple whistleblower messages to the first slave node through the first master node, and determining multiple routing paths between the first master node and the first slave node based on at least one switch; the at least one switch is the switch through which each whistleblower message travels from the first master node to the first slave node; wherein each routing path passes at least sequentially through one or more of the N1 first switches, S of the N2 second switches, and one or more of the N3 third switches; S is a natural number less than or equal to N3.

[0036] In some possible implementations, the method further includes: querying the visible bit in the routing status register of the first slave node based on the sent spoofing message through the first master node; if the visible bit is 0, then assigning a corresponding device number to the first slave node; wherein, the visible bit being 0 indicates that the first slave node has not yet been enumerated and discovered; through the first slave node, saving the device number assigned by the first master node to the routing status register, and setting the visible bit to 1; wherein, the visible bit being 1 indicates that the first slave node has now been enumerated and discovered.

[0037] In some possible implementations, the method further includes: sending a first configuration message to the first slave node through the first master node to obtain management authority over the first slave node; the first configuration message carries the master node password and master node number of the first master node; receiving the first configuration message through the first slave node, and setting the signature bit in the management master node information register of the first slave node to 1 based on the first configuration message; wherein, the signature bit being 1 is used to indicate that the first slave node currently has management master node status, and other master nodes in the system cannot obtain management authority over the first slave node; and saving the master node password and master node number of the first master node to the management master node information register through the first slave node.

[0038] In some possible implementations, the method further includes: sending a second configuration message to the first slave node through the first master node to revoke management privileges on the first slave node; the second configuration message carries the master node password and master node number of the first master node; receiving the second configuration message through the first slave node, and if the master node password and master node number of the first master node carried in the second configuration message are consistent with the master node password and master node number stored in the management master node information register, then setting the signature bit in the management master node information register to 0; wherein, the signature bit being 0 is used to indicate that the first slave node currently does not have a management master node.

[0039] In some possible implementations, the method further includes: after obtaining management authority over the first slave node, the first master node, in response to a query message sent by the first slave node, sending an presence message to the first slave node; or, sending the presence message to the first slave node at a first time interval.

[0040] In some possible implementations, the method further includes: setting the signature bit in the management master node information register to 0 through the first slave node, under the condition that a preset condition is met, to cancel the management authority of the first master node over the first slave node; wherein the preset condition includes: after the first master node obtains management authority over the first slave node, the first slave node does not receive the presence message sent by the first master node within a preset time, or the first slave node does not receive the presence message sent by the first master node after sending K query messages to the first master node; K is an integer greater than or equal to 1.

[0041] In some possible implementations, the method further includes: sending a broadcast message to at least one master node in the system through the first slave node; the broadcast message indicating that the first slave node does not currently have a management master node; receiving the broadcast message through the at least one master node, and sending the first configuration message to the first slave node based on the broadcast message to obtain management rights over the first slave node.

[0042] In some possible implementations, the multipath bus subsystem further includes a second master node; the N switches further include N4 fourth switches adjacent to the second master node; wherein any fourth switch is adjacent to any third switch, or connected through one or more second switches; N4 is a positive integer less than or equal to N; the method further includes: sending multiple whistleblower messages to the first slave node through the second master node, and determining multiple routing paths between the second master node and the first slave node based on at least one switch; the at least one switch is the switch through which each whistleblower message travels from the second master node to the first slave node; wherein each of the multiple routing paths passes at least sequentially through one or more of the N4 fourth switches, S of the N2 second switches, and one or more of the N3 third switches.

[0043] In some possible implementations, the multipath bus subsystem includes a first node domain and a second node domain; the first node domain includes a first master node and multiple first slave nodes, and the second node domain includes a second master node and multiple second slave nodes; the step of sending multiple report messages to the first slave nodes through the second master node and determining multiple routing paths between the second master node and the first slave nodes based on at least one switch includes: after all the first slave nodes in the first node domain are assigned device numbers by the first master node, and after all the second slave nodes in the second node domain are assigned device numbers by the second master node, sending multiple report messages to the first slave nodes through the second master node, and determining multiple routing paths between the second master node and the first slave nodes based on at least one switch.

[0044] In some possible implementations, the N switches further include a first cross-node domain switch belonging to a first node domain and a second cross-node domain switch belonging to a second node domain; the first port of the first cross-node domain switch is connected to the second port of the second cross-node domain switch; if some or all of the first slave nodes in the first node domain have not yet been assigned device numbers by the first master node, and / or some or all of the second slave nodes in the second node domain have not yet been assigned device numbers by the second master node, then the data link between the first port and the second port is closed; if all the first slave nodes in the first node domain have been assigned device numbers by the first master node, and all the second slave nodes in the second node domain have been assigned device numbers by the second master node, then the data link between the first port and the second port is open, so that the second master node sends multiple report messages to the first slave nodes through the second port and the first port, and determines multiple routing paths between the second master node and the first slave node based on at least one switch.

[0045] In some possible implementations, the second master node is a remote master node that is network-connected to the first master node via a switch; the method further includes: accessing the first slave node within the first node domain through the second master node via a network connection, in order to invoke computing resources in the first slave node or read stored data in the first slave node.

[0046] Thirdly, embodiments of this application provide a master node, which includes a processor configured to support the master node in performing corresponding functions in any of the communication methods provided in the second aspect. The master node may also include a memory coupled to the processor, which stores necessary program instructions and data for the master node. The master node may also include a communication interface for communicating with other devices or communication networks.

[0047] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the communication method flow described in any one of the second aspects above.

[0048] Fifthly, embodiments of this application provide a computer program that includes instructions that, when executed by a computer, enable the computer to perform the communication method flow described in any of the second aspects above.

[0049] In a sixth aspect, embodiments of this application provide a chip including a processor and a communication interface. The processor is used to call and execute instructions from the communication interface. When the processor executes the instructions, the chip performs the communication method flow described in the second aspect above.

[0050] In a seventh aspect, embodiments of this application provide a chip system including the bus system described in any one of the first aspects above, for implementing the functions involved in the communication method flow described in any one of the second aspects above. In one possible design, the chip system further includes a memory for storing program instructions and data necessary for the communication method. This chip system may be composed of chips or may include chips and other discrete devices. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of an interconnection structure based on CXL2.0.

[0052] Figure 2 This is a schematic diagram of a bus system provided in an embodiment of this application.

[0053] Figure 3 This is a schematic diagram of the structure of a multipath bus subsystem provided in an embodiment of this application.

[0054] Figure 4a This is a schematic diagram of another multipath bus subsystem provided in this application.

[0055] Figure 4b This is a schematic diagram of another multipath bus subsystem provided in this application.

[0056] Figure 4c This is a schematic diagram of another multipath bus subsystem provided in this application.

[0057] Figure 4d This is a schematic diagram of another multipath bus subsystem provided in this application.

[0058] Figure 5 This is a schematic diagram of another bus system provided in an embodiment of this application.

[0059] Figure 6 This is a schematic diagram of a management host signature process provided in an embodiment of this application.

[0060] Figure 7 This is a schematic diagram of a process for canceling management privileges on a host, provided in an embodiment of this application.

[0061] Figure 8a This is a schematic diagram of a management host presence confirmation process provided in an embodiment of this application.

[0062] Figure 8b This is a schematic diagram of another management host presence confirmation process provided in an embodiment of this application.

[0063] Figure 9 This is a schematic diagram of another multipath bus subsystem provided in the embodiments of this application.

[0064] Figures 10a-10c This is a schematic diagram of the structure of a multi-path, multi-master bus system provided in an embodiment of this application.

[0065] Figures 11a-11b This is a schematic diagram of an enumeration process under a multi-path, multi-host architecture provided in an embodiment of this application.

[0066] Figure 12 This is a flowchart illustrating a communication method provided in an embodiment of this application. Detailed Implementation

[0067] The embodiments of this application will now be described with reference to the accompanying drawings.

[0068] The terms "first" and "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses. It should be noted that when an element is referred to as being "coupled" or "connected" to one or more other elements, it can be a direct connection of one element to the other one or more elements, or an indirect connection.

[0069] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0070] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will understand, explicitly and implicitly, that the embodiments described herein can be combined with other embodiments.

[0071] The terms “component,” “module,” “system,” etc., used in this specification are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, both an application running on a processor and the processor itself can be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0072] First, some terms used in this application will be explained to facilitate understanding by those skilled in the art.

[0073] (1) Topology. The topology of a computer network is based on the method of studying the relationships between points and lines that are independent of size and shape, as used in topology. Computers and communication devices in the network are abstracted as points, and the transmission medium is abstracted as a line. The geometric figure composed of points and lines is the topology of the computer network. The most common topologies of computer networks include bus topology, ring topology, tree topology, star topology, hybrid topology, and mesh topology. For example, PCI uses a bus topology, while PCIe uses a tree topology, and so on. Furthermore, hosts in the topology can use algorithms to search the entire network topology, enumerating and discovering all connected devices. Enumeration and discovery simply means traversing the physically connected devices under each port, reading the device's configuration space, assigning it a corresponding device number, and recording the routing path taken when enumerating to that device. Ultimately, the host can know the number of devices in the entire topology and the connection relationships between them, etc. Obviously, as mentioned above, if a device has been assigned a device number, it means that the device has been enumerated and discovered. In some possible implementations of this application, the topology can be a flat graph structure, where there are multiple paths from a host (i.e., master node) on the bus to a node device (i.e., slave node). At the same time, a node device can be accessed by multiple hosts on the bus, and so on, which will not be described in detail here.

[0074] (2) A switch is used to implement bus interconnection and routing. It includes multiple ports, each of which can correspond to a physical link to connect master nodes, slave nodes, or other switches within the bus system. In the embodiments of this application, multiple switches are specifically used to construct multiple physical links between master nodes and slave nodes through their interconnections, thereby enabling multi-path access from master nodes to slave nodes. In some possible embodiments, some ports of the switch can also have network interface card (NIC) functions for remote network connections, and some special ports can be used to control the enumeration and discovery process of master nodes within their respective node domains, thereby avoiding interference during the enumeration process of multiple master nodes, etc., which will not be elaborated in detail here.

[0075] (3) Graph structure, also known as graph structure, is a more complex nonlinear structure than tree structure. In tree structure, nodes have a hierarchical relationship, and a node at each level can only be related to one node in the level above, but may be related to multiple nodes in the level below. In graph structure, any node can have one or more predecessor and successor adjacent nodes, breaking the strict top-down hierarchical relationship in tree structure. In other words, any two nodes in graph structure may be related, that is, the adjacency relationship between nodes can be arbitrary.

[0076] First, to facilitate understanding of the embodiments of this application, the specific technical problems to be solved by this application are further analyzed and proposed. In the prior art, there are various solutions for bus interconnection technology. The following are some of the more common CXL2.0 solutions.

[0077] Please see Figure 1 , Figure 1 This is a schematic diagram of an interconnection structure based on CXL2.0. (For example...) Figure 1 As shown, this structure can include a CXL switch, a bus manager (FM), multiple hosts, and multiple devices. Specifically, it can include host 0, host 1, and type3 device 0, type3 device 1, and type3 device 2. Clearly, as... Figure 1 As shown, these multiple hosts and devices must be interconnected through a CXL switch. That is, the host cannot directly interconnect with the device, but is indirectly interconnected with the device through the CXL switch. For a single host and device, it is still a top-down PCIe tree structure.

[0078] Furthermore, such as Figure 1 As shown, the CXL switch includes a fabric manager endpoint (FM EP) connected to the FM, and two virtual CXL switches (VCS), such as VCS-0 and VCS-1. Each VCS may also include multiple virtual PCI-to-PCI bridges (vPPBs), such as... Figure 1 As shown, VCS-0 may include vPPB-01, vPPB-02, and vPPB-03, while VCS-1 may include vPPB-11, vPPB-12, and vPPB-13. Furthermore, the CXL switch also includes multiple physical PCI-to-PCI bridges (PPBs), such as PPB-0, PPB-1, and PPB-2.

[0079] It's important to note that in the CXL2.0 solution, the CXL switch first requires initialization by the FM (Factor Management Unit). Furthermore, the CXL switch's downstream port (DP) is not bound to the virtual CXL switch; it belongs solely to the FM. The FM can initialize the CXL switch using vendor-defined mechanisms, pre-binding the CXL switch's vPPB (Virtual Power Branch Bank) to a specific physical PPB (Power Branch Bank). Multiple vPPBs can be bound to the same PPB. For example... Figure 1 As shown, both vPPB-03 and vPPB-12 can be bound to PPB-1.

[0080] Once the CXL switch completes initialization, host 0 and host 1 can enumerate the CXL switch and the interconnected devices behind it (such as type3 device 0, type3 device 1, and type3 device 2) according to the standard PCIe enumeration process, configuring the switch's address window, bus number window, etc. Therefore, both host 0 and host 1 see a complete, exclusive PCIe tree. The CXL switch must store these routing configurations during the enumeration process and ensure that subsequent downlink data flows are accurately routed to the actual physical downlink ports. Similarly, uplink data flows must also be routed correctly to the actual destination host. Therefore, the design of the CXL switch must comply with these CXL protocol specifications, making it more complex and costly than a regular switch.

[0081] In conclusion, Figure 1 The CXL2.0 scheme shown has the following drawbacks:

[0082] (1) The host and the device must be interconnected by a specially designed CXL switch as an intermediate interconnection device. This specially designed CXL switch will undoubtedly increase the interconnection latency and, to some extent, limit the topology of the entire interconnection, increasing the complexity of the interconnection design and additional costs.

[0083] (2) Node devices still need to be specially designed, such as supporting type 3 devices with multi logical device functionality.

[0084] (3) The entire interconnection topology interconnection initialization process has relatively strict requirements. The bus manager needs to participate in the initialization of the interconnection in advance (e.g., initializing CXL switch, initializing type 3 devices, etc.). The devices need to recognize these special management messages, making the process complex and cumbersome.

[0085] Therefore, to address the shortcomings of current bus interconnect technologies in meeting practical needs, this application aims to solve the following technical problems: breaking the limitations of the existing top-down tree structure of PCIe and implementing a graphical interconnect topology based on existing conventional switches. For a node in the structure, multiple paths can reach it, and it can be accessed by multiple hosts, thereby enabling the interconnect bus protocol to natively and cost-effectively support multi-path and multi-host connectivity.

[0086] Please see Figure 2 , Figure 2 This is a schematic diagram of a bus system provided in an embodiment of this application. The technical solution of this embodiment can be... Figure 2 Specific implementations are shown in the example system architecture or similar system architectures. For example... Figure 2As shown, the bus system 10 may include multiple master nodes, multiple switches, and multiple slave nodes. Specifically, it may include master nodes 100a, 100b, and 100c, switches 300a, 300b, and 300c, and slave nodes 200a, 200b, and 200c. These master nodes, switches, and slave nodes can be connected via a bus (e.g., a network on chip, or any other possible bus, such as an amba bus) to form a graphical structure. In other words, the topology of the computer network composed of these master nodes, switches, and slave nodes is a graphical structure, rather than a conventional tree structure. It should be noted that any master node, switch, or slave node can be a node in this graphical structure. In the graphical structure, any two nodes may be related. For example, master node 100a can be adjacent to switch 300a and switch 300b respectively (or directly connected, that is, there are no other devices on the physical link between master node 100a and switch 300a). Switch 300a, switch 300b, and switch 300c can be adjacent to each other. Switch 300a can also be adjacent to slave node 200a and slave node 200b. Furthermore, if slave node 200a has multiple ports, slave node 200a can also be adjacent to master node 100a, master node 100c, etc., etc. This application embodiment does not make specific limitations in this regard. It should be understood that the embodiments of this application are designed to address the increasing prevalence of applications with ever-growing computing demands, such as AI and autonomous driving. By boldly breaking through the limitations of the traditional tree structure in existing PCIe bus interconnect topologies, a graphical structure is adopted. This allows for more arbitrary connections between master nodes, switches, and slave nodes, resulting in greater scalability of the entire topology. Consequently, numerous proprietary computing devices (such as GPUs and TPUs) can be continuously added to the bus as slave nodes within this graphical structure. Furthermore, any master node within this graphical structure can access and utilize any slave node within the structure through multiple paths formed by connections between multiple switches, maximizing the unrestricted use of various computing and data resources by the master node.

[0087] In summary, master nodes 100a, 100b, and 100c may each include one or more CPUs. Optionally, they may also include main memory, cache memory (e.g., cache), internal interconnect bus, I / O interface, etc. This application embodiment does not specifically limit these components. Optionally, a master node can also be considered a computing system with the above-mentioned components. Switches 300a, 300b, and 300c may include multiple ports. In some other possible embodiments, the switches may also have corresponding congestion control and quality of service (QoS) functions. Slave nodes 200a, 200b, and 200c may be general-purpose computing devices such as GPUs, TPUs, or XPU processor units, or storage devices such as solid-state drives (SSDs), or accelerators with specific computing functions, smart network cards, or even switches (e.g., network switches), etc. This application embodiment does not specifically limit these components.

[0088] Further, please refer to Figure 3 , Figure 3 This is a schematic diagram of a multipath bus subsystem provided in an embodiment of this application. The bus system 10 with the above graphical structure may include one or more multipath bus subsystems, and any one of these subsystems may include a master node, slave nodes, and a switch. For example, as... Figure 3 As shown, the multipath bus subsystem 10a may include a master node 100a (i.e., the first master node), a slave node 200a (i.e., the first slave node), and N switches, where N can be an integer greater than or equal to 1. Among the N switches, N1 first switches (e.g., [missing information]) adjacent to the first master node 100a may be included. Figure 3 The first switch 11, the first switch 12, etc.), and the N3 third switches (e.g., the first slave node 200a) adjacent to the first slave node 200a. Figure 3 The third switch 31, the third switch 32, etc.), and N2 second switches (e.g. Figure 3 (Second switch 21, second switch 22, second switch 23, etc.).

[0089] In this embodiment, any first switch can be adjacent to any third switch, or connected through one or more of the N2 second switches. For example, first switch 11 can be adjacent to third switch 31; or first switch 11 can be connected to third switch 31 through second switch 21, in which case second switch 21 is adjacent to both first switch 11 and third switch 31; or first switch 11 can be connected to third switch 31 through second switches 22 and 23 in sequence, in which case second switch 22 is adjacent to both first switch 11 and second switch 23, and second switch 23 is also adjacent to third switch 31, and so on. This embodiment does not specifically limit these connections. N1, N2, and N3 can all be positive integers less than or equal to N.

[0090] Specifically, master node 100a can send multiple report messages to slave node 200a, and determine multiple routing paths between master node 100a and slave node 200a based on the switches traversed by each report message from master node 100a to slave node 200a. Each routing path can sequentially traverse at least one or more of N1 first switches, S of N2 second switches, and one or more of N3 third switches. This achieves multi-path access to a slave node, or in other words, allows a master node to access the same slave node through multiple routing paths. Here, S is a natural number less than or equal to N3, meaning S can be equal to 0, in which case the routing path between master node 100a and slave node 200a can traverse only the first and third switches. In some possible embodiments, the routing path between master node 100a and slave node 200a can also traverse only one or more of N1 first switches, or only one or more of N3 third switches, etc., and this application does not specifically limit this.

[0091] It should be noted that the embodiments of this application aim to achieve multi-path access to slave nodes based on a graphical structure, and do not specifically limit the specific connection between master nodes, slave nodes, and switches. The technical solutions provided by the embodiments of this application will be described in detail below through examples of several possible connection scenarios. The connection scenarios in this application may include, but are not limited to, the following examples.

[0092] Optionally, please refer to Figure 4a , Figure 4a This is a schematic diagram of another multipath bus subsystem provided in this application. For example... Figure 4aAs shown, the multipath bus subsystem 10a may specifically include a master node 100a, a slave node 200a, a first switch 11, a first switch 12, a second switch 21, a third switch 31, and a third switch 32. The first switches 11 and 12 are adjacent to the master node 100a, and the third switches 31 and 32 are adjacent to the slave node 200a. The first switch 11 and the third switch 32 are connected through the second switch 21. The first switch 11 is also adjacent to the third switch 31, and the third switch 31 is also adjacent to the master node 100a.

[0093] Based on this, after the master node 100a sends multiple enumeration messages from its own port through the enumeration software, it can discover and record multiple routing paths between itself and the slave node 200a. Subsequently, the master node 100a can access the slave node 200a and use its computing resources or manage its functional configuration based on any of these multiple routing paths, and so on. Figure 4a As shown, the multiple routing paths can include: (1) Master node 100a → First switch 11 → Second switch 21 → Third switch 32 → Slave node 200a; (2) Master node 100a → First switch 11 → Third switch 31 → Slave node 200a; (3) Master node 100a → Third switch 31 → Slave node 200a, which can be the above; (4) Master node 100a → First switch 12 → Third switch 32 → Slave node 200a.

[0094] Optionally, please refer to Figure 4b , Figure 4b This is a schematic diagram of another multipath bus subsystem provided in this application. For example... Figure 4a As shown, the multipath bus subsystem 10a may specifically include a master node 100a, a slave node 200a, a first switch 11, a first switch 12, a third switch 31, and a third switch 32. The first switches 11 and 12 are adjacent to the master node 100a, and the third switches 31 and 32 are adjacent to the slave node 200a. The first switch 11 is also adjacent to both the first switch 12 and the third switch, and the third switch 32 is also adjacent to both the first switch 12 and the third switch 31.

[0095] Based on this, after the master node 100a sends out multiple enumeration reports from its own port through the enumeration software, it can discover and record multiple routing paths between itself and the slave node 200a, which may include: (1) Master node 100a → First switch 11 → Third switch 31 → Slave node 200a; (2) Master node 100a → First switch 11 → Third switch 31 → Third switch 32 → Slave node 200a; (3) Master node 100a → First switch 11 → First switch 12 → Third switch 32 → Slave node 200a; (4) Master node 100a → First switch 11 → First switch 12 → Third switch 32 → Third switch 31 → Slave node 200a; (5) Master node 100a → First switch 12 → Third switch 32 → Slave node 200a; (6) Master node 100a → First switch 12 → Third switch 32 → Third switch 31 → Slave node 200a; (7) Master node 100a → First switch 12 → First switch 11 → Third switch 31 → Slave node 200a; (8) Master node 100a → First switch 12 → First switch 11 → Third switch 31 → Third switch 32 → Slave node 200a.

[0096] Optionally, please refer to Figure 4c , Figure 4c This is a schematic diagram of another multipath bus subsystem provided in this application. For example... Figure 4a As shown, the multipath bus subsystem 10a may specifically include a master node 100a, a slave node 200a, a first switch 11, and a first switch 12. The first switch 11 and the first switch 12 are respectively adjacent to the master node 100a and the slave node 200a, and the first switch 11 is also adjacent to the first switch 12.

[0097] Based on this, after the master node 100a sends out multiple enumeration reports from its own port through the enumeration software, it can discover and record multiple routing paths between itself and the slave node 200a, which may include: (1) master node 100a → first switch 11 → slave node 200a; (2) master node 100a → first switch 11 → first switch 12 → slave node 200a; (3) master node 100a → first switch 12 → slave node 200a; (4) master node 100a → first switch 12 → first switch 11 → slave node 200a.

[0098] Optionally, please refer to Figure 4d , Figure 4d This is a schematic diagram of another multipath bus subsystem provided in this application. For example... Figure 4aAs shown, the multipath bus subsystem 10a may specifically include a master node 100a, a slave node 200a, and a first switch 11. The first switch 11 is adjacent to both the master node 100a and the slave node 200a, and the slave node 200a is also directly adjacent to the master node 100a.

[0099] Based on this, after the master node 100a sends out multiple enumeration reports from its own port through the enumeration software, it can discover and record multiple routing paths between itself and the slave node 200a, which may include: (1) master node 100a → first switch 11 → slave node 200a; (2) master node 100a → slave node 200a.

[0100] In summary, it should be noted that the terms "first," "second," and "third" used to describe switches in the embodiments of this application do not specifically refer to any one switch, but rather are used to describe different connection situations between the switch and the master node and slave node. For example, as Figure 4a As shown, the third switch 31 is adjacent to both the master node 100a and the slave node 200a. Based on the aforementioned discussion, it can also be referred to as the first switch. For example, as... Figure 4c As shown, the first switch 11 and the first switch 12 are adjacent to the master node 100a and the slave node 200a, respectively. Based on the foregoing discussion, they can also be called the third switch, and so on. Generally, because switches have many ports, their adjacency relationships can be quite arbitrary and complex based on the graphical structure. For example... Figure 4a Other ports of the second switch 21 can also be connected to the master node 100a and the third switch 31 or the slave node 200a, etc., respectively. This application embodiment does not specifically limit this.

[0101] Furthermore, based on the descriptions of the various possible multipath bus subsystems mentioned above, the enumeration and discovery process under multipath will be elaborated in detail below.

[0102] Optionally, the master node 100a can query the visited bit in the routing status register of the slave node 200a based on the enumeration report sent to the slave node 200a. If the visited bit is 0, the master node 100a can assign the corresponding device number to the slave node 200a. The visited bit being 0 can be used to indicate that the slave node 200a has not been enumerated and discovered.

[0103] Correspondingly, slave node 200a can save the device number assigned by master node 100a to the routing status register and set the visibility bit to 1. The visibility bit being 1 indicates that slave node 200a has been enumerated and discovered. Essentially, when master node 100a first enumerates slave node 200a via a certain routing path, it can set its previously 0 visibility bit to 1 and assign it a device number. This allows master node 100a to determine that slave node 200a has been enumerated and discovered based on its visibility bit being 1 after sending an enumeration report to slave node 200a via other routing paths. Therefore, it does not need to assign a device number again; it only needs to record the new routing path used to send the enumeration report. In this way, enumeration under multiple paths can be implemented by recording historical discovery status, avoiding repeated enumeration and infinite loops under multiple paths, thus completing enumeration efficiently and accurately.

[0104] Optionally, the master node 100a may have corresponding enumeration software installed and running, which can be used to enumerate and discover devices in the entire interconnect bus topology. Additionally, each slave node in this embodiment, like a conventional PCIe device, has its own configuration space, allowing software to access this space to obtain device-related declaration information and manage the device, etc. For example, please refer to... Figure 5 , Figure 5 This is a schematic diagram of another bus system provided in an embodiment of this application. For example... Figure 5 As shown, the bus system includes master node 100a, switch A, switch B, device C, and device D. Device C and device D can be the aforementioned slave nodes 200a, 200b, or 200c, etc., specifically, they can be GPUs, TPUs, SSDs, accelerators, etc. Figure 5 As shown, master node 100a includes two ports: port A1 is connected to switch A, and port A2 is connected to switch B. Switch A is connected to both switch B and device C. Switch B is also connected to both device C and device D. Clearly, Figure 5 The bus system shown may include two multipath bus subsystems: one including master node 100a, switch A, switch B and device C, and the other including master node 100a, switch A, switch B and device D. These will not be described in detail here.

[0105] The following is based on Figure 5 Taking this as an example, the enumeration discovery process under multiple paths will be further elaborated. This enumeration discovery process can specifically include the following steps:

[0106] Step 1: The enumeration software running on master node 100a can start from port A1 of master node 100a and perform a breadth-first search algorithm on the graph. Optionally, the enumeration software can check whether the physical link corresponding to the current port A1 has been established and whether data transmission is possible. If the enumeration software determines that the physical link corresponding to port A1 has been established and data transmission is possible, it can send an enumeration request (or enumeration access) from port A1 to the device directly physically connected to port A1. Figure 5 Switch A in the middle.

[0107] Optionally, before sending the enumeration report, the basic input / output system (BIOS) can report local bus interface information, such as the number of local bus ports and their corresponding access addresses, to the enumeration software in the master node 100a through a specific interface based on an agreed-upon description. Figure 5 How many ports are there in total for switch A, switch B, device C, and device D?

[0108] Step 2: Switch A receives the enumeration report sent by master node 100a and returns a response to the enumeration report. The enumeration software can determine from this response that switch A is a legitimate device and is a switch device.

[0109] Step 3: The enumeration software performs a management host signature (i.e., management master node signature) on switch A according to the management host signature process (i.e., management master node signature process). Optionally, such as... Figure 5 As shown, the signature process may include setting the signature bit in the management master node information register (i.e., management host information register) in the configuration space A of switch A to 1, and setting the master node number of master node 100a (i.e., ... Figure 5 The management host ID (component identity document, CID) and the host key used for this signature are written into the management master node information register, thereby completing the acquisition of management authority for switch A by master node 100a, that is, confirming that master node 100a becomes the management master node of switch A. Optionally, step 3 can be referred to the following: Figure 6 The descriptions in the corresponding embodiments will not be repeated here.

[0110] Step 4: The enumeration software continues to send enumeration reports through port A1 to read the routing status register in the configuration space A of switch A and check whether the visible bits of the routing status register of switch A are 0.

[0111] Step 5: Switch A returns a visible bit of its routing status register to 0, thus the enumeration software determines that switch A has not yet been discovered through enumeration. For example... Figure 5 As shown, at this time, the enumeration software can assign device number cid1 to switch A, and switch A can directly write the device number (i.e., cid1) into the CID value bit field corresponding to the routing status register. At the same time, it can also set the visible bit that was originally 0 to 1.

[0112] Step 6, as mentioned above, since the enumeration software can know that switch A is a switch device, the enumeration software can also read the relevant port status registers of switch A, thereby knowing which ports in switch A have been physically linked.

[0113] Step 7: The enumeration software further sends an enumeration report from a physical link established on switch A to device C, and determines that device C is a legitimate device by the response returned by device C.

[0114] Step 8: The enumeration software follows the path from port A1 → switch A → device C. Referring to the process in steps 3 to 5, it performs master node signature management on device C and assigns device C a device number that has not been assigned to other devices. Figure 5 (Taking cid2 as an example).

[0115] Step 9: The enumeration software, following the path from port A1 to switch A to switch B, and referring to the procedures in steps 7 and 8, completes the enumeration and discovery of switch B, performs management master node signature on switch B, and assigns the corresponding device number to switch B. Figure 5 (Taking cid3 as an example).

[0116] Step 10: For switch B, which is also a switch device, the enumeration software follows the process from steps 6 to 8. It accesses device C through the path A1 → switch A → switch B → device C. At this point, the enumeration software finds that the visible bit of the routing status register of device C has been set (i.e., the visible bit has been set to 1), and it also finds that the signature register of the management master node of device C has been signed (i.e., the signature bit has been set to 1). Therefore, the enumeration software no longer performs actions such as assigning a device number to device C, but only records the new routing path between master node 100a and device C.

[0117] Step 11, referring to the process in steps 7 and 8, the enumeration software completes the enumeration and discovery of device D through the path of port A1 → switch A → switch B → device D, performs management master node signature on device D, and assigns the corresponding device number to device D. Figure 5 (Taking cid4 as an example).

[0118] Step 12: The enumeration software starts from port A2 of master node 100a and enumerates and discovers switches B, C, D, A, and C respectively through the paths: port A2 → switch B, port A2 → switch B → device C, port A2 → switch B → device D, port A2 → switch B → switch A, and port A2 → switch B → switch A → device C. Clearly, based on the previous steps, the enumeration software has already completed the enumeration and discovery of these devices through port A1. Therefore, in this enumeration and discovery process, no further enumeration and discovery actions are performed on these devices; that is, device numbers are not reassigned. Only the new routing paths between master node 100a and switches A, B, C, and D are recorded.

[0119] Step 13: At this point, master node 100a has completed the enumeration process using the corresponding enumeration software. Figure 5 The discovery enumeration of all devices and topologies in the bus system shown.

[0120] As described above, when the master node discovers a device (which may include switches and slave nodes) through enumeration, it can sign its management host to obtain management privileges for that device. These management privileges include, but are not limited to, arbitrating resource contention between devices, handling device anomalies, managing basic device characteristics (such as maximum supported packet length), and managing device functions, such as allowing or disallowing the use of certain functions (e.g., physical link-related functions), etc. This embodiment of the application does not specifically limit these privileges.

[0121] Optionally, please refer to Figure 6 , Figure 6 This is a schematic diagram of a management host signature process provided in an embodiment of this application. For example... Figure 6 As shown, the management host signing process may include the following steps.

[0122] S11, the master node sends a configuration message to the device, attempting to acquire management rights for the device. The master node is, for example, the aforementioned master node 100a, and the device is, for example, the aforementioned slave node 200a, or... Figure 5The configuration message (e.g., the first configuration message) is used to set the signature bit in the management master node information register of the configuration space to 1. This configuration message can also carry the master node's master node number and password. The signature bit being 1 indicates that the device has completed signing, meaning it now has management master node access, and other master nodes cannot acquire management rights for that device. This ensures that each device has a unique management master node for a considerable period, guaranteeing the clarity and manageability of the entire topology. It should be noted that a master node can acquire management rights for multiple devices. Furthermore, the master node password carried in the configuration messages sent by the master node when acquiring management rights for different devices can be different.

[0123] S12, the device receives a configuration message sent by the master node and determines that the configuration message is used to write the signature bit in its management master node information register to 1.

[0124] S13: The device checks if the signature bit in the management master node information storage is 0. If it is, proceed to S14; otherwise, proceed to S15. A signature bit of 0 indicates that the device has not yet completed signing, meaning it does not have a management master node. All master nodes in the system have the opportunity to acquire management rights over the device.

[0125] S14, the device sets the signature bit in the management master node information register to 1 and saves the master node password and master node number carried in the configuration message to the management master node information register. At this point, the master node has completed the management master node signature for the device and obtained management privileges. Optionally, the device can ensure that the master node password saved in this signature cannot be read or modified by other unverified master nodes. Optionally, all subsequent operations performed by the device's management master node on the device must first verify whether the master node password and master node number carried in the operation are consistent with those saved by the device. If they are inconsistent, the device can assume that the operation was not issued by the device's management master node and can choose not to respond, thereby ensuring the security of device management.

[0126] S15, the device verifies the master node sending the configuration message to determine if it is the device's management master node. If so, proceed to S16; otherwise, proceed to S17. Specifically, if the ID information of the current requester (or source) (i.e., the master node number carried in the configuration message) matches the master node number stored in the device's management master node information register, and the master node password carried in the configuration message matches the master node password stored in the management master node information register, then the master node sending the configuration message can be determined to be the device's management master node. Conversely, if any one or more of the following conditions are met—inconsistent master node number, inconsistent master node password, or absence of the master node password—then the master node sending the configuration message can be determined to be not the device's management master node.

[0127] S16, the device does not modify the signature bit in the management master node information register, that is, it keeps the signature bit at 1 and returns a configuration success response.

[0128] S17, the device does not modify the signature bit in the management master node information register, that is, it keeps the signature bit at 1, and returns a configuration failure response. Optionally, the device may also directly discard the configuration message and not respond.

[0129] In summary, this application defines a mechanism where whoever successfully signs first becomes the management master node. The system may include multiple master nodes (e.g., ...). Figure 2 As shown in the examples of master nodes 100a, 100b, and 100c, device management becomes clearer. Simultaneously, the device ensures that the master node password configured during master node signing cannot be read or modified by other non-master nodes. Furthermore, after master node signing is completed, the device will verify every subsequent master node management operation, greatly enhancing the security of device management.

[0130] Optionally, after acquiring management permissions for a device, the master node can proactively revoke those permissions. For example, the master node might receive requests from other master nodes seeking management permissions for the device, or it might detect a fault in itself. To ensure the device has a new, functioning master node for management and maintains efficient management, the master node can proactively revoke its management permissions. Please refer to [link to relevant documentation]. Figure 7 , Figure 7 This is a schematic diagram illustrating a process for revoking management privileges on a host, as provided in an embodiment of this application. Figure 7 As shown, the process for revokeing management permissions may include the following steps.

[0131] S21, the master node sends a configuration message to the device, attempting to revoke its management privileges over the device. Here, the master node is, for example, the aforementioned master node 100a, and the device is, for example, the aforementioned slave node 200a, or... Figure 5 The configuration message (e.g., the second configuration message) is intended to set the signature bit in the management master node information register of the configuration space to 0, and may also carry the master node number and master node password.

[0132] S22, the device receives a configuration message sent by the master node and determines that the configuration message is used to write the signature bit in its management master node information register to 0.

[0133] S23, the device verifies the master node that sent the configuration message to determine whether it is the management master node of the device. If it is, then proceed to S24; otherwise, proceed to S25.

[0134] S24, the device sets the signature bit in the management master node information register to 0 and returns a configuration success response. It should be understood that if the master node is the management master node of the device, it obviously means that the device already has a management master node, and the signature bit in the management master node information register of the device is generally 1.

[0135] S25, the device does not modify the signature bits in the management master node information register and returns a configuration failure response. Optionally, the device may also discard the configuration message directly without responding. It should be understood that if the master node is not the management master node of the device, the signature bits in the management master node information register of the device can be 1 or 0.

[0136] It should be noted that the embodiments of this application refer to the above. Figure 6 and Figure 7 The execution order of the steps shown is not specifically limited.

[0137] Furthermore, after the master node obtains management rights over the device, in order to ensure the real-time presence of the management master node and the robustness of the entire system (or bus topology), this application embodiment also defines a master node presence confirmation mechanism.

[0138] Optionally, please refer to Figure 8a , Figure 8a This is a schematic diagram of a management host presence confirmation process provided in an embodiment of this application. This management host presence confirmation process can be applied to the above-mentioned... Figures 2-5 In the system structure shown, Figure 8a The master node involved can be, for example, the aforementioned master node 100a (i.e., the first master node), and the involved device can be, for example, the aforementioned slave node 200a (i.e., the first slave node), or... Figure 5The switches in the diagram are A, B, C, or D.

[0139] like Figure 8a As shown, the device can be configured with a timer 1. When a master node acquires management rights over the device (for example, when the device's signature bit is set to 1), timer 1 can start counting down. Figure 8a As shown, when the value of timer 1 equals X (e.g., 200ms, 3s, 5s, or 8s), the device can send a query message to its management master node to attempt to obtain the presence status of its management master node. That is, the device can actively send query messages to its management master node at preset time intervals (or preset frequencies), and the management master node receives the query message accordingly. Optionally, while the device sends the query message, timer 2 inside the device can start counting. If, when the value of timer 2 equals Y (e.g., 300ms, 1s, 5s, or 7s), the device still has not received the presence message from its management master node, a timeout can be counted using a counter within the device. Thus, as... Figure 8a As shown, when the number of timeouts reaches K times (K is an integer greater than or equal to 1, such as 3, 5, or 7 times), meaning the device has sent K query messages to its management master node and has not received a presence message from the management master node each time, the device can consider its management master node to be abnormal and not present. Furthermore, the device can reset the signature bit in its management master node information register to 0. It can also clear the master node number and password stored in its management master node register, thereby revoking its original management master node privileges. At this point, the device is in a state of unsuccessful signature and does not possess a management master node. Correspondingly, as... Figure 8a As shown, if the device receives an in-place message from its management master node during this period, it can be determined that the management master node is in place normally. At the same time, the device can clear the current values ​​of timer 2 and counter, that is, clear timer 2 and counter to zero (or reset).

[0140] Optionally, please refer to Figure 8b , Figure 8b This is a schematic diagram of another management host presence verification process provided in an embodiment of this application. This management host presence verification process can be applied to the above... Figures 2-5 In the system structure shown, Figure 8b The master node involved can be, for example, the aforementioned master node 100a (i.e., the first master node), and the device involved can be, for example, the aforementioned slave node 200a (i.e., the first slave node), or... Figure 5 The switches in the diagram are A, B, C, or D.

[0141] like Figure 8bAs shown, the master node can be configured with timer 3. When the master node obtains management permissions for the device, timer 3 can start counting down. Figure 8b As shown, when the value of timer 3 equals W (e.g., 200ms, 3s, or 10s), the master node can send a presence message to the devices it manages. That is, the master node can proactively send presence messages to the devices it manages at preset time intervals (e.g., the first time interval). Correspondingly, the device can receive the presence message sent by the master node and determine whether the message originates from its master node. If so, the device can confirm that its master node is normally present; otherwise, it can discard the message without any response.

[0142] Optionally, the device can also be configured with a corresponding timer. This timer can start counting when the master node acquires management rights over the device. When the timer in the device equals a preset time (e.g., W), and the device still has not received a presence message actively sent by its master node, the device can consider its master node to be abnormal and not in place. Furthermore, the device can reset the signature bit in its master node information register to 0, and further, it can clear the master node number and password stored in its master node register, thereby canceling its original management rights. Conversely, if the device receives a presence message from its master node during this period, it can determine that the master node is normally in place, and the device can reset the locally maintained timer.

[0143] Optionally, the device can also be configured with corresponding timers and counters. Similarly, the timer can start counting when the master node gains management access to the device. When the timer in the device equals a preset time (e.g., W), and the device still hasn't received a presence message actively sent by its master node, a timeout can be recorded using the device's counter. When the number of timeouts reaches a preset value (e.g., 5, 7, or 10 times), meaning the master node hasn't actively sent a presence message for an extended period, the device can consider its master node abnormal and not in position. Furthermore, the device can reset the signature bit in its master node information register to 0, and further, clear the master node number and password stored in its master node register, thereby revoking its original management access. Conversely, if the device successfully receives a presence message actively sent by its master node during this period, it can confirm that the master node is normally in position. Simultaneously, the device can also reset the locally maintained timers and counters.

[0144] Optionally, the values ​​of X, Y, K, and W mentioned above can all be configured by the management master node, and a more suitable value can be selected according to actual needs.

[0145] Furthermore, as mentioned above, if a device fails to receive a presence message from its management master node for an extended period, thus determining that its management master node is abnormal and revoking its management privileges, in order to maintain the robustness of the entire bus topology and ensure that the device has a properly present management master node, the device can also send a presence message to at least one master node in the system (e.g., Figure 2 The master nodes 100a, 100b, and 100c (e.g., in the bus system 10 shown) send broadcast messages. These broadcast messages can indicate that the device currently does not have a managing master node. Optionally, the at least one master node may also include its original managing master node. Accordingly, the at least one master node receives the broadcast message and can, based on the broadcast message, sign off on the device to attempt to acquire management rights and become the new managing master node for the device. It should be understood that, according to the above discussion, the master node that completes the signing first acquires management rights for the device; the specific signing process can be referred to the above. Figure 6 The description of the corresponding embodiments will not be repeated here.

[0146] It should be noted that, generally speaking, the above Figure 8a and Figure 8b The frequency of interaction between the management master node and its device presence confirmation shown can be very low, so the impact on bus bandwidth is almost negligible. That is, the embodiments of this application can ensure that the management master node is in place in real time without affecting the data transmission and computing efficiency of the entire bus, thereby ensuring the robustness of the bus topology.

[0147] Optionally, please refer to Figure 9 , Figure 9 This is a schematic diagram of another multipath bus subsystem provided in an embodiment of this application. For example... Figure 9 As shown, the multipath bus subsystem 10a may also include a master node 100b (i.e., a second master node) and N4 fourth switches (e.g., ...) adjacent to the master node 100b. Figure 9 (e.g., fourth switch 41, fourth switch 42, etc.). Any fourth switch can be adjacent to any third switch, or connected through one or more second switches; where N4 is a positive integer less than or equal to N. For example, fourth switch 41 can be adjacent to third switch 31; or fourth switch 41 can be connected to third switch 31 through second switch 22, etc. This application does not specifically limit these connections.

[0148] Correspondingly, master node 100b can also send multiple report messages to slave node 200a, and determine multiple routing paths between master node 100b and slave node 200a based on the switches traversed by each report message from master node 100b to slave node 200a. Each routing path can sequentially traverse at least one or more of the N4 fourth switches, S of the N2 second switches, and one or more of the N3 third switches. This further enables multi-path access from multiple master nodes to a single slave node; that is, a slave node can be discovered by multiple master nodes, and subsequently accessed and its resources can be used by these multiple master nodes based on their respective multiple routing paths, etc. Correspondingly, the routing path between master node 100b and slave node 200a can traverse only the fourth and third switches. In some possible embodiments, it can traverse only one or more of the N4 fourth switches, or only one or more of the N3 third switches, etc. This application embodiment does not specifically limit this.

[0149] Optionally, the process by which master node 100b discovers slave node 200a through enumeration of multiple routing paths can be found in the above description. Figure 5 The corresponding descriptions in the text will not be repeated here.

[0150] Optionally, master node 100a and slave node 200a may belong to a first node domain, and master node 100b may belong to a second node domain. This second node domain further includes one or more second slave nodes (e.g., ...). Figure 2 (e.g., slave node 200b). Optionally, the first node domain may also include multiple switches connected to the master node 100a (e.g., the N1 first switches), and the second node domain may also include multiple switches connected to the master node 100b (e.g., the N4 fourth switches). This application embodiment does not specifically limit this. It should be noted that different node domains may belong to different sub-networks, or the system management software of different node domains may be different (e.g., different enumeration software, different operating systems, etc.).

[0151] Optionally, after assigning device numbers to devices (such as slave nodes and switches) within its own second node domain, i.e. after completing the enumeration discovery within its own second node domain, the master node 100b can send enumeration reports to devices (such as slave nodes 200a) in other node domains, thereby completing the enumeration discovery of multiple master nodes across node domains.

[0152] As described above, in a bus system where multiple node domains and corresponding master nodes exist, the enumeration and discovery process of a master node begins within its current node domain. Before completing the enumeration and discovery of devices within its own node domain, this node domain may not establish data flow connections with any devices in other node domains. This supports device enumeration and discovery across multiple master nodes (i.e., across networks or different system management software) and ensures that different enumeration software does not assign the same device number multiple times, or that the same device number is assigned to different devices, preventing conflicting scenarios.

[0153] Furthermore, based on the concept that different master nodes can belong to different node domains, when the aforementioned device (e.g., slave node 200a) cancels the management authority of the original managing master node (e.g., master node 100a) and thus sends a broadcast message to at least one master node in the system, it can first send a first-level broadcast message to the master node in its own node domain (e.g., master node 100a). If the master node in its own node domain does not respond, it can send a second-level broadcast message (i.e., a cross-node domain broadcast message) to the master nodes in other node domains (e.g., master node 100b and master node 100c), and so on.

[0154] Optionally, please refer to Figures 10a-10c , Figures 10a-10c This is a schematic diagram of a multi-path, multi-master bus system provided in an embodiment of this application.

[0155] like Figure 10a As shown, the bus system may include multiple hosts (e.g., master node 100a, master node 100b) and multiple switches (e.g. Figure 10a (Switch 1, Switch 2, Switch 3, etc.) and multiple slave nodes (e.g. Figure 10a (Including accelerators, SSDs, smart network cards, GPUs, and TPUs, etc.). Master node 100a can include ports A1 and A2, and master node 100b can include ports B1 and B2. Obviously, as... Figure 10a As shown, compared to the traditional tree structure, the implementation in this embodiment is a flattened graphical structure, where multiple switches can be arranged in a matrix and connected vertically and horizontally via a bus. Clearly, this structure can support multi-path, multi-master node access to any slave node, and this access may include initial enumeration and discovery, as well as subsequent resource usage and function management.

[0156] Optionally, master node 100a and switches 1, 2, 3, and 4 connected to master node 100a, as well as the XPU, accelerator, SSD, and smart NIC connected to switches 1, 3, and 4 respectively, can belong to the first node domain; master node 100b and switches 5, 6, 7, and 8 connected to master node 100b, as well as the TPU, SSD, GPU, and accelerator connected to switches 6, 7, and 8 respectively, can belong to the second node domain.

[0157] Optionally, in order to ensure that the enumeration processes of multiple master nodes (i.e., multiple hosts) do not interfere with each other, this embodiment of the application guarantees that before a master node completes the enumeration and discovery within its own node domain, it does not establish data stream connections with any devices in other node domains, that is, it does not enumerate and discover devices in other node domains, while simultaneously ensuring that devices in its own node domain cannot be discovered by master nodes in other node domains. Figure 10a As shown, this application embodiment also provides a switch with a special port, which is used to connect adjacent to other switches in the same node domain that also have special ports. For example, Figure 10a Switches 2, 4, 5, and 7 in the diagram, the special ports are, for example, the ports marked in black in switches 2, 4, 5, and 7. Figure 10a As shown, under the default settings (or when the master node has not completed the enumeration discovery within its own node domain), when the system enumeration software scans a port of this type, it will no longer continue the enumeration discovery based on this port. For example, when master node 100a starts enumerating from port A2 to the special port marked in black in switch 2, it will no longer enumerate and discover the device connected to that port (i.e., switch 5), but will continue to enumerate and discover the device within the first node domain through other ports; as another example, when master node 100b starts enumerating from port B1 to the special port marked in black in switch 5, it will no longer enumerate and discover the device connected to that port (i.e., switch 2), but will continue to enumerate and discover the device within the second node domain through other ports. Correspondingly, after master nodes 100a and 100b complete the enumeration and discovery of all devices in their respective node domains (for example, all devices in their respective node domains have been assigned the corresponding device number), the special ports of switches 2 and 5 can be "opened," and master nodes 100a and 100b can then perform cross-node domain enumeration and discovery through these special ports.

[0158] For example, please see Figures 11a-11b , Figures 11a-11b This is a schematic diagram of an enumeration process under a multi-path, multi-host architecture provided in an embodiment of this application. For example... Figure 11aAs shown, master node 100a and master node 100b have different operating systems, namely OS 1 and OS 2. Master node 100a includes CPU-A0 and CPU-A1, and corresponding ports A1, A2, A3 and A4; master node 100b includes CPU-B0 and CPU-B1, and corresponding ports B1, B2, B3 and B4.

[0159] like Figure 11a As shown, both master nodes 100a and 100b have performed some enumeration and discovery. Some devices (such as GPUs and SSDs in the first node domain, and accelerators in the second node domain) have not yet been assigned device numbers. Specifically, the first node domain where master node 100a resides includes switch E with device number cid4. Its port A is a special port that crosses node domains. After master node 100a's enumeration software (e.g., OS1) discovers port A of switch E, it stops enumerating the topology behind port A because it hasn't finished enumerating other physically linked ports that are not cross-node domains. Similarly, the second node domain where master node 100b resides includes switch F with device number cid2. Its port B is a special port that crosses node domains. After master node 100b's enumeration software (e.g., OS2) discovers port B of switch F, it also stops enumerating the topology behind port B because it hasn't finished enumerating other physically linked ports that are not cross-node domains.

[0160] like Figure 11b As shown, at this point, both master nodes 100a and 100b have completed the enumeration and discovery of the bus topology and all devices within their respective node domains, and all devices have been assigned device numbers. Therefore, the system software of master nodes 100a and 100b can respectively open the data flow switches on special port A of switch E and special port B of switch F, thereby enumerating and discovering devices in other node domains outside their own node domain. It should be noted that at this point, the devices in each node domain have completed the signature of the management master node, and it can be known that the master node in its own node domain has obtained the management master node authority for the devices in its own node domain. For example, master node 100a is the management master node for TPU, GPU, and SSD in the first node domain; master node 100b is the management master node for GPU and accelerator in the second node domain. Therefore, master node 100a only has data plane access rights for the devices in the second node domain, and does not have management master node authority on the management plane. Similarly, master node 100b only has data plane access rights for the devices in the first node domain, and does not have management master node authority on the management plane.

[0161] Optionally, after the system software in each node domain has completed the discovery of the bus topology and enumeration of all devices within its own node domain, the master node 100a can directly report the enumeration and discovery of devices in the second node domain to the system software (e.g., OS1) on the master node 100a through the system software in the second node domain (e.g., OS2) based on the agreed description structure and interface. In this way, the system software on the master node 100a can directly obtain the topology information of all devices in the node domain where the master node 100b is located. Correspondingly, the discovery and enumeration of devices under the first node domain where the master node 100a is located by the master node 100b is similar, and will not be elaborated here.

[0162] As described above, the special port provided in this application embodiment allows each master node to independently stop enumerating devices / topologies behind this port at this port, and cease breadth / depth-first search enumeration from this port. Only after each master node has completed enumeration and discovery within its own node domain in its system software will it open the data path switch of this special port within its respective domain to perform cross-master node and cross-node domain enumeration.

[0163] like Figure 10b As shown, this bus system can also include master node 100a, master node 100b, multiple switches, and multiple slave nodes, etc. Figure 10b The above introduction can be referenced. Figure 10a The description of the corresponding embodiments will not be repeated here.

[0164] Optionally, such as Figure 10b As shown, both switch 2 under master node 100a and switch 5 under master node 100b include a port capable of accessing the internet, or a port with network card functionality (e.g., Figure 10b (The ports are marked in gray). Thus, master nodes 100a and 100b can be network-connected via switches 2 and 5, meaning master node 100b can be a remote device of master node 100a. This allows a device to be accessed and used not only by a nearby master node connected via a wired connection, but also by other remote master nodes connected wirelessly. This enables the host to access remote computing resources and storage data, greatly satisfying the host's demand for massive computing resources during complex calculations, and so on. This undoubtedly greatly enhances the scalability of the entire bus system, further breaking through the limitations of existing PCIe bus connections.

[0165] like Figure 10cAs shown, the bus system may include master node 100a, master node 100b, master node 100c, multiple switches, and multiple slave nodes. The bus system may include switches with special ports or switches used for network connections. Optionally, regarding... Figure 10c The above introduction can be referenced. Figure 10a and Figure 10b The description of the corresponding embodiments will not be repeated here. Figure 10c As shown, this bus system can be a typical graphical bus system provided by the embodiments of this application, which greatly enables the bus interconnection to be more scalable, has a larger capacity, and a larger design space.

[0166] Optionally, Figures 10a-10c The special ports shown, as well as ports with network card functionality, can be directly indicated through the relevant function register groups in the configuration space of the switch device (i.e., the switch) to inform the system enumeration software.

[0167] It should be noted that, Figures 10a-10c This is merely an illustrative example and does not constitute a specific limitation on the multi-path, multi-master bus system of this application embodiment.

[0168] In summary, this application provides a bus system based on the concept of a graphical architecture. This system breaks away from the top-down tree structure limitations of existing PCIe bus interconnect topologies, achieving a flattened graphical interconnect topology. Thus, no additional special bus interconnect devices / components (such as those mentioned above) are required. Figure 1 The use of CXL switches and type 3 devices, along with complex bus topology discovery and enumeration processes, enables the interconnect bus protocol to natively and cost-effectively support multi-host and multipathing. Based on this, embodiments of this application further provide a series of management host signing processes, management host presence confirmation processes, and multi-host enumeration processes across node domains. This ensures not only multi-host and multipathing but also the clear manageability, security, and reliability of the entire multi-host and multipath bus system (or the entire graphical bus topology).

[0169] Furthermore, it is emphasized again that the embodiments of this application aim to break the tree structure with strict top-down hierarchical relationship in the prior art based on the concept of graph structure, so as to realize the above-mentioned multi-host and multipathing. The specific connection between each node is not specifically limited. In the graph structure of the embodiments of this application, not all nodes need to be adjacent. Even when a node device (such as a GPU) has multiple input ports, the node device can be directly adjacent to multiple hosts without going through a switch, so as to more conveniently realize multi-host and multipathing, etc.

[0170] It should be noted that the technical solutions provided in this application embodiment can be applied not only to network on chip interconnect, but also to ultra-large-scale network interconnect, such as the Internet, device management of interconnection between large-scale servers / computing nodes in data centers, etc. This application embodiment does not make specific limitations in this regard.

[0171] Please see Figure 12 , Figure 12 This is a flowchart illustrating a communication method provided in an embodiment of this application. This communication method can be applied to bus systems (e.g.,...). Figure 2 , Figure 5 or Figures 10a-10c The bus system shown in the example can include multiple master nodes, multiple switches, and multiple slave nodes; these master nodes, switches, and slave nodes can form a graphical structure through a bus; wherein, any multipath bus subsystem in this graphical structure (e.g.) Figure 3 , Figures 4a-4d The multipath bus subsystem 10a) illustrated in the example may include a first master node, a first slave node, and N switches; the N switches include N1 first switches adjacent to the first master node, N2 second switches, and N3 third switches adjacent to the first slave node; wherein any first switch is adjacent to any third switch, or connected through one or more second switches; N1, N2, and N3 are all positive integers less than or equal to N. The communication method may include the following step S401.

[0172] Step S401: Send multiple report messages to the first node device through the first host.

[0173] Step S402: Based on at least one switch, determine multiple routing paths between the first host and the first node device; the at least one switch is the switch that each report message passes through from the first host to the first node device; wherein, each routing path passes through at least one or more of the N1 first switches, S of the N2 second switches, and one or more of the N3 third switches in sequence; S is a natural number less than or equal to N3.

[0174] In some possible implementations, the multipath bus subsystem further includes a second master node; the N switches further include N4 fourth switches adjacent to the second master node; wherein any fourth switch is adjacent to any third switch, or connected through one or more second switches; N4 is a positive integer less than or equal to N; the method further includes: sending multiple whistleblower messages to the first slave node through the second master node, and determining multiple routing paths between the second master node and the first slave node based on at least one switch; the at least one switch is the switch through which each whistleblower message travels from the second master node to the first slave node; wherein each of the multiple routing paths passes at least sequentially through one or more of the N4 fourth switches, S of the N2 second switches, and one or more of the N3 third switches.

[0175] In some possible implementations, the multipath bus subsystem includes a first node domain and a second node domain; the first node domain includes a first master node and multiple first slave nodes, and the second node domain includes a second master node and multiple second slave nodes; the step of sending multiple report messages to the first slave nodes through the second master node and determining multiple routing paths between the second master node and the first slave nodes based on at least one switch includes: after all the first slave nodes in the first node domain are assigned device numbers by the first master node, and after all the second slave nodes in the second node domain are assigned device numbers by the second master node, sending multiple report messages to the first slave nodes through the second master node, and determining multiple routing paths between the second master node and the first slave nodes based on at least one switch.

[0176] In some possible implementations, the N switches further include a first cross-node domain switch belonging to a first node domain and a second cross-node domain switch belonging to a second node domain; the first port of the first cross-node domain switch is connected to the second port of the second cross-node domain switch; if some or all of the first slave nodes in the first node domain have not yet been assigned device numbers by the first master node, and / or some or all of the second slave nodes in the second node domain have not yet been assigned device numbers by the second master node, then the data link between the first port and the second port is closed; if all the first slave nodes in the first node domain have been assigned device numbers by the first master node, and all the second slave nodes in the second node domain have been assigned device numbers by the second master node, then the data link between the first port and the second port is open, so that the second master node sends multiple report messages to the first slave nodes through the second port and the first port, and determines multiple routing paths between the second master node and the first slave node based on at least one switch.

[0177] In some possible implementations, the second master node is a remote master node that is network-connected to the first master node via a switch; the method further includes: accessing the first slave node within the first node domain through the second master node via a network connection, in order to invoke computing resources in the first slave node or read stored data in the first slave node.

[0178] Alternatively, the specific details of this communication method can be found in the above description. Figures 2-11b The description of the corresponding embodiments will not be repeated here.

[0179] Optionally, the various method flows in the communication method described in the embodiments of this application can be implemented based on components, hardware, or a combination thereof. Hardware implementation may include logic circuits, algorithm circuits, or analog circuits, etc. Software implementation may include program instructions, which can be considered a software product, stored in memory, and executed by a processor to implement related functions.

[0180] This application also provides a computer-readable storage medium, wherein the computer-readable storage medium may store a program, which, when executed by a processor, enables the processor to perform some or all of the steps described in any of the above method embodiments.

[0181] This application also provides a computer program that includes instructions that, when executed by a multi-core processor, enable the processor to perform some or all of the steps described in the above method embodiments.

[0182] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not described in detail in a particular embodiment can be found in the relevant descriptions of other embodiments. It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0183] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0184] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0185] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0186] If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which can be a personal computer, server, or network device, specifically a processor in the computer device) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium may include various media capable of storing program code, such as a USB flash drive, portable hard drive, magnetic disk, optical disk, read-only memory (ROM), double data rate synchronous dynamic random access memory (DDR), flash memory, or random access memory (RAM).

[0187] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A bus system, characterized in that, The system is a graphical structure consisting of multiple master nodes, multiple switches, and multiple slave nodes connected by a bus. Any multipath bus subsystem within this graphical structure includes a first master node, a first slave node, and N switches. The N switches include N1 first switches adjacent to the first master node, N2 second switches, and N3 third switches adjacent to the first slave node. Any first switch is adjacent to any third switch, or connected through one or more second switches. N1, N2, and N3 are all positive integers less than or equal to N. The first master node is configured to send multiple whistleblower messages to the first slave node and determine multiple routing paths between the first master node and the first slave node based on at least one switch; the at least one switch is the switch through which each whistleblower message travels from the first master node to the first slave node; wherein each of the multiple routing paths passes through at least one or more of the N1 first switches, S of the N2 second switches, and one or more of the N3 third switches in sequence; S is a natural number less than or equal to N3.

2. The system according to claim 1, characterized in that, The first master node is further configured to query the visible bit in the routing status register of the first slave node based on the sent report message; if the visible bit is 0, then assign a corresponding device number to the first slave node; wherein, the visible bit being 0 is used to indicate that the first slave node has not been enumerated and discovered at present. The first slave node is used to save the device number assigned by the first master node to the routing status register and set the visibility bit to 1; wherein, the visibility bit being 1 is used to indicate that the first slave node has been enumerated and discovered.

3. The system according to claim 1, characterized in that, The first master node is further configured to send a first configuration message to the first slave node to obtain management authority over the first slave node; the first configuration message carries the master node password and master node number of the first master node; The first slave node is further configured to receive the first configuration message and, based on the first configuration message, set the signature bit in the management master node information register of the first slave node to 1; wherein, the signature bit being 1 is used to indicate that the first slave node currently has a management master node, and other master nodes in the system cannot obtain management rights over the first slave node. The first slave node is also used to save the master node password and master node number of the first master node to the management master node information register.

4. The system according to claim 3, characterized in that, The first master node is also configured to send a second configuration message to the first slave node to revoke its management privileges over the first slave node; the second configuration message carries the master node password and master node number of the first master node. The first slave node is further configured to receive the second configuration message. If the master node password and master node number of the first master node carried in the second configuration message are consistent with the master node password and master node number stored in the management master node information register, then the signature bit in the management master node information register is set to 0. The signature bit being 0 is used to indicate that the first slave node does not currently have a management master node.

5. The system according to claim 3, characterized in that, The first master node is further configured to, after obtaining management authority over the first slave node, respond to a query message sent by the first slave node by sending an in-place message to the first slave node; or, send the in-place message to the first slave node at a first time interval.

6. The system according to claim 5, characterized in that, The first slave node is further configured to, under preset conditions, set the signature bit in the management master node information register to 0, thereby canceling the first master node's management authority over the first slave node; wherein, The preset conditions include: after the first master node obtains management authority over the first slave node, the first slave node does not receive the presence message sent by the first master node within a preset time, or the first slave node does not receive the presence message sent by the first master node after sending K query messages to the first master node; K is an integer greater than or equal to 1.

7. The system according to claim 6, characterized in that, The first slave node is also configured to send a broadcast message to at least one master node in the system; the broadcast message is used to indicate that the first slave node does not currently have a managing master node; The at least one master node is configured to receive the broadcast message and send the first configuration message to the first slave node based on the broadcast message in order to obtain management authority over the first slave node.

8. The system according to any one of claims 1-7, characterized in that, The multipath bus subsystem further includes a second master node; the N switches further include N4 fourth switches adjacent to the second master node; wherein any fourth switch is adjacent to any third switch, or connected through one or more second switches; N4 is a positive integer less than or equal to N; The second master node is used to send multiple whistleblower messages to the first slave node and determine multiple routing paths between the second master node and the first slave node based on at least one switch; the at least one switch is the switch that each whistleblower message passes through from the second master node to the first slave node; wherein, each of the multiple routing paths passes through at least one or more of the N4 fourth switches, S of the N2 second switches, and one or more of the N3 third switches in sequence.

9. The system according to claim 8, characterized in that, The multipath bus subsystem includes a first node domain and a second node domain; the first node domain includes a first master node and multiple first slave nodes, and the second node domain includes a second master node and multiple second slave nodes; the second master node is specifically used for: After all first slave nodes in the first node domain are assigned device numbers by the first master node, and all second slave nodes in the second node domain are assigned device numbers by the second master node, multiple report messages are sent to the first slave nodes, and multiple routing paths between the second master node and the first slave nodes are determined based on at least one switch.

10. The system according to claim 9, characterized in that, The N switches also include a first cross-node domain switch belonging to the first node domain and a second cross-node domain switch belonging to the second node domain; the first port of the first cross-node domain switch is connected to the second port of the second cross-node domain switch. If there is a first slave node in the first node domain that has not been assigned a device number by the first master node, and / or there is a second slave node in the second node domain that has not been assigned a device number by the second master node, then the data link between the first port and the second port is closed. If all first slave nodes in the first node domain are assigned device numbers by the first master node, and all second slave nodes in the second node domain are assigned device numbers by the second master node, then the data link between the first port and the second port is open, so that the second master node can send multiple report messages to the first slave nodes through the second port and the first port, and determine multiple routing paths between the second master node and the first slave node based on at least one switch.

11. The system according to any one of claims 9-10, characterized in that, The second master node is a remote master node that is connected to the first master node via a network switch; the second master node is specifically used to: access the first slave node within the domain of the first node via a network connection, so as to call the computing resources in the first slave node or read the stored data in the first slave node.

12. The system according to any one of claims 1-7 and 9-10, characterized in that, The first master node is the central processing unit (CPU) in the first terminal; The first master node is also used to call the computing resources of at least one second slave node or read the stored data in the at least one second slave node through a network connection; the second slave node is an image processor GPU, solid-state drive, accelerator, network card or tensor processing unit TPU in the second terminal.

13. The system according to any one of claims 1-7 and 9-10, characterized in that, The slave node can be any one of the following: GPU, solid-state drive, accelerator, network card, TPU, embedded neural network processor (NPU), digital signal processor (DSP), image signal processor (ISP), or switch.

14. The system according to any one of claims 1-7 and 9-10, characterized in that, The master node includes one or more central processing units (CPUs).

15. A communication method, characterized in that, This method is applied to a bus system, which is a graphical structure composed of multiple master nodes, multiple switches, and multiple slave nodes connected by a bus. Any multipath bus subsystem in this graphical structure includes a first master node, a first slave node, and N switches. The N switches include N1 first switches adjacent to the first master node, N2 second switches, and N3 third switches adjacent to the first slave node. Any first switch is adjacent to any third switch, or connected through one or more second switches. N1, N2, and N3 are all positive integers less than or equal to N. The method includes: The first master node sends multiple whistleblower messages to the first slave node, and determines multiple routing paths between the first master node and the first slave node based on at least one switch; the at least one switch is the switch that each whistleblower message passes through from the first master node to the first slave node; wherein, each routing path passes through at least one or more of the N1 first switches, S of the N2 second switches, and one or more of the N3 third switches in sequence; S is a natural number less than or equal to N3.

16. The method according to claim 15, characterized in that, The method further includes: Based on the sent report message, the first master node queries the visible bit in the routing status register of the first slave node. If the visible bit is 0, the corresponding device number is assigned to the first slave node. The visible bit being 0 indicates that the first slave node has not yet been enumerated. The first slave node saves the device number assigned by the first master node to the routing status register and sets the visibility bit to 1; wherein, the visibility bit being 1 is used to indicate that the first slave node has been enumerated and discovered.

17. The method according to claim 15, characterized in that, The method further includes: The first master node sends a first configuration message to the first slave node to obtain management privileges over the first slave node; the first configuration message carries the master node password and master node number of the first master node. The first slave node receives the first configuration message and, based on the first configuration message, sets the signature bit in the management master node information register of the first slave node to 1; wherein, the signature bit being 1 is used to indicate that the first slave node currently has a management master node, and other master nodes in the system cannot obtain management rights over the first slave node. The first slave node saves the master node password and master node number of the first master node to the management master node information register.

18. The method according to claim 17, characterized in that, The method further includes: The first master node sends a second configuration message to the first slave node to revoke management privileges on the first slave node; the second configuration message carries the master node password and master node number of the first master node. If the master node receives the second configuration message through the first slave node, and the master node password and master node number of the first master node carried in the second configuration message are consistent with the master node password and master node number stored in the management master node information register, then the signature bit in the management master node information register is set to 0; wherein, the signature bit being 0 is used to indicate that the first slave node does not currently have a management master node.

19. The method according to claim 17, characterized in that, The method further includes: After obtaining management authority over the first slave node, the first master node responds to the query message sent by the first slave node by sending an in-place message to the first slave node; or, sends the in-place message to the first slave node at a first time interval.

20. The method according to claim 19, characterized in that, The method further includes: By having the first slave node set the signature bit in the management master node information register to 0 under preset conditions, the first master node's management authority over the first slave node is revoked; wherein, The preset conditions include: after the first master node obtains management authority over the first slave node, the first slave node does not receive the presence message sent by the first master node within a preset time, or the first slave node does not receive the presence message sent by the first master node after sending K query messages to the first master node; K is an integer greater than or equal to 1.

21. The method according to claim 20, characterized in that, The method further includes: The first slave node sends a broadcast message to at least one master node in the system; the broadcast message is used to indicate that the first slave node does not currently have a managing master node. The master node receives the broadcast message and sends the first configuration message to the first slave node based on the broadcast message to obtain management authority over the first slave node.

22. The method according to any one of claims 15-21, characterized in that, The multipath bus subsystem further includes a second master node; the N switches further include N4 fourth switches adjacent to the second master node; wherein any fourth switch is adjacent to any third switch, or connected through one or more second switches; N4 is a positive integer less than or equal to N; the method further includes: The second master node sends multiple report messages to the first slave node, and determines multiple routing paths between the second master node and the first slave node based on at least one switch; the at least one switch is the switch through which each report message passes from the second master node to the first slave node; wherein, each of the multiple routing paths passes at least sequentially through one or more of the N4 fourth switches, S of the N2 second switches, and one or more of the N3 third switches.

23. The method according to claim 22, characterized in that, The multi-path bus subsystem includes a first node domain and a second node domain; the first node domain includes a first master node and multiple first slave nodes, and the second node domain includes a second master node and multiple second slave nodes; the step of sending multiple report messages to the first slave nodes through the second master node, and determining multiple routing paths between the second master node and the first slave nodes based on at least one switch, includes: After all first slave nodes in the first node domain are assigned device numbers by the first master node, and all second slave nodes in the second node domain are assigned device numbers by the second master node, the second master node sends multiple report messages to the first slave nodes, and determines multiple routing paths between the second master node and the first slave nodes based on at least one switch.

24. The method according to claim 23, characterized in that, The N switches also include a first cross-node domain switch belonging to the first node domain and a second cross-node domain switch belonging to the second node domain; the first port of the first cross-node domain switch is connected to the second port of the second cross-node domain switch. If some or all of the first slave nodes in the first node domain have not yet been assigned device numbers by the first master node, and / or some or all of the second slave nodes in the second node domain have not yet been assigned device numbers by the second master node, then the data link between the first port and the second port is closed. If all first slave nodes in the first node domain are assigned device numbers by the first master node, and all second slave nodes in the second node domain are assigned device numbers by the second master node, then the data link between the first port and the second port is open, so that the second master node can send multiple report messages to the first slave nodes through the second port and the first port, and determine multiple routing paths between the second master node and the first slave node based on at least one switch.

25. The method according to any one of claims 23-24, characterized in that, The second master node is a remote master node that is connected to the first master node via a network switch; the method further includes: accessing the first slave node in the first node domain through the second master node via a network connection, so as to call the computing resources in the first slave node or read the stored data in the first slave node.

26. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a computer or processor, implements the method described in claims 15-25.

27. A computer program product, characterized in that, The computer program product includes instructions that, when executed by a computer or processor, cause the computer or processor to perform the method as described in claims 15-25.

Images