Method for carrying out inter-node routing based on multiple nodes and related product
By using routing tuples and resident Kernel in non-full-connect scenarios, efficient communication between any two points is achieved, high hardware cost and programming complexity are solved, user needs are met and computing resource consumption is reduced.
Patent Information
- Application Number
- CN202310526090.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-10
- Publication Date
- 2024-11-12
AI Technical Summary
In the non-full connection scenario, it is difficult for the prior art to realize efficient transmission and reception communication between any two points, and it is difficult for users to communicate without perception under different product forms, resulting in high hardware costs and increased programming complexity.
Using a routing method based on multiple nodes, routing between nodes is realized by constructing and receiving routing tuples, including session identifiers, data storage space information and control value storage space information, and routing between nodes is achieved, and resident Kernel is used to reduce computing core consumption, priority is given Scheduling conflict communication.
Avoid the hardware cost of introducing expensive switching chips and switches, realize the maximum theoretical bandwidth when there is no conflict in the session, and ensure bandwidth according to priority when multiple sessions communicate at the same time, meet user needs, and reduce the consumption of computing resources. .
Smart Images

Figure CN118939596A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of chips, and more specifically, to the field of inter-chip communication. Background Art
[0002] Artificial Intelligence (AI) chips have dedicated communication modules and links for high-speed interconnection between chips, such as NVLINK and MLULINK. High-speed interconnection links have different heterogeneous topologies in different product forms, such as ring, chain, tree, and full connection. At the same time, users need to perform send-receive (Send-Recv) communication between any two cards, and high-speed communication connections such as NVLINK and MLULINK are required between the two cards. However, there are the following contradictions between product forms and user needs: First, from the perspective of product cost, full connection is a relatively expensive form, and most product forms are not fully connected; 2. From the perspective of programming ease, users need to be able to perform Send-Recv communication between any two cards without feeling it, even if there is no MLULINK connection. Summary of the invention
[0003] The purpose of the present disclosure is to provide a method for performing inter-node routing based on multiple nodes to solve the sending and receiving communication operations between any two points in a non-fully connected scenario.
[0004] According to a first aspect of the present disclosure, a method for performing inter-node routing based on multiple nodes is provided, comprising: constructing a routing tuple, the routing tuple comprising a session identifier, local data storage space information, remote data storage space information and control value storage space information; wherein the session identifier is used to distinguish sessions, the local data storage space information is used to indicate the address of a storage space for storing data in a local node, the remote storage space information is used to indicate the address of a storage space for storing data in a downstream node, and the control value storage space information is used to indicate the address of a storage space for storing control values in a local node and an upstream node and / or a downstream node; and sending the routing tuple to multiple nodes on a session path so that the nodes receiving the routing tuple can perform inter-node routing according to the routing tuple.
[0005] According to a second aspect of the present disclosure, a method for performing inter-node routing based on multiple nodes is provided, comprising: receiving a routing tuple, the routing tuple comprising a session identifier, local data storage space information, remote data storage space information and control value storage space information; wherein the session identifier is used to distinguish sessions, the local data storage space information is used to indicate an address of a storage space for storing data in a local node, the remote storage space information is used to indicate an address of a storage space for storing data in a downstream node, and the control value storage space information is used to indicate an address of a storage space for storing control values in a local node and an upstream node and / or a downstream node; and performing inter-node routing on a session path according to the routing tuple.
[0006] According to a third aspect of the present disclosure, a method for performing inter-node routing based on multiple nodes is provided, comprising: on a host side, constructing a routing tuple, the routing tuple comprising a session identifier, local data storage space information, remote data storage space information and control value storage space information; wherein the session identifier is used to distinguish sessions, the local data storage space information is used to indicate the address of a storage space for storing data in a local node, the remote storage space information is used to indicate the address of a storage space for storing data in a downstream node, and the control value storage space information is used to indicate the address of a storage space for storing control values in a local node and an upstream node and / or a downstream node; sending the routing tuple to multiple nodes on a session path; and on a device side including multiple nodes, receiving the routing tuple; the node receiving the routing tuple performs inter-node routing according to the routing tuple.
[0007] According to a fourth aspect of the present disclosure, there is provided a host for performing inter-node routing based on multiple nodes, on which a resident agent program runs, and the resident agent program is used to execute the method as described above.
[0008] According to a fifth aspect of the present disclosure, a device for performing inter-node routing based on multiple nodes is provided, the device comprising multiple nodes, each node having a kernel running therein, the kernel being used to execute the method described above.
[0009] According to a sixth aspect of the present disclosure, a system for performing inter-node routing based on multiple nodes is provided, the system comprising the host as described above and the device as described above.
[0010] According to a seventh aspect of the present disclosure, a computer-readable storage medium is provided, comprising computer-executable instructions, and when the computer-executable instructions are executed by one or more processors, the method described above is executed.
[0011] According to an embodiment of the present disclosure, compared with the prior art, the technical solution of the present disclosure avoids the hardware cost caused by introducing expensive switching chips and switches.
[0012] According to an embodiment of the present disclosure, in the present disclosure, a pure software method is adopted, and the maximum theoretical bandwidth of the port can be achieved when there is no conflict in the session.
[0013] According to an embodiment of the present disclosure, when multiple sessions are communicating in conflict at the same time, the priorities of different sessions can be guaranteed according to the priorities, thereby satisfying user needs to the greatest extent.
[0014] According to an embodiment of the present disclosure, the consumption of the number of computing cores is minimized by using a resident Kernel. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:
[0016] Figure 1 A schematic diagram showing the structure of a board according to an embodiment of the present disclosure is shown;
[0017] Figure 2 is a schematic diagram showing a combined processing device of this embodiment;
[0018] Figure 3 A schematic diagram of the internal structure of a computing device is shown;
[0019] Figure 4 shows the internal architecture of the processing core;
[0020] Figure 5a and Figure 5b Different topological structures among multiple nodes are exemplarily shown;
[0021] Figure 6 A flow chart of a method for performing inter-node routing based on multiple nodes according to an embodiment of the present disclosure is shown;
[0022] Figure 7 A schematic block diagram of a system applying the inter-node routing method of the present disclosure according to one embodiment of the present disclosure is shown;
[0023] Figure 8 A schematic diagram of a communication system including a transmitting end and a receiving end according to an embodiment of the present disclosure is shown;
[0024] Fig. 9A schematic diagram showing that sessions with different priorities occupy different bandwidths when communicating;
[0025] Fig.10 A flow chart showing the establishment of a session path before building a routing tuple;
[0026] Fig.11 A schematic diagram showing a session path established between two reachable nodes. DETAILED DESCRIPTION
[0027] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.
[0028] It should be understood that the terms "first", "second", "third", "fourth", etc. in the claims, specifications and drawings of the present disclosure are used to distinguish different objects rather than to describe a specific order. "First", "second", "third", "fourth", etc. do not just mean one, but may also mean multiple. The terms "include" and "comprise" used in the specification and claims of the first disclosure indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.
[0029] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. As used in this disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.
[0030] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.
[0031] The specific implementation of the present disclosure is described in detail below with reference to the accompanying drawings.
[0032] Today's semiconductor manufacturing process starts with a complete wafer. Wafers are circular sheets made of pure silicon, generally divided into 6-inch, 8-inch, 12-inch and other specifications. Wafers are cut into small pieces, which are called dies. Each die is mounted with a chip and wired to achieve specific electrical functions. Then the die is packaged into a particle. The purpose of packaging is to place, fix, seal, protect the chip and enhance the electrical and thermal performance. At the same time, the contacts of the chip are connected to the pins of the package shell with wires, and a chip package structure is completed.
[0033] The memory is used to temporarily store the computing data required by the system on chip and the data exchanged with the external memory. In this embodiment, the memory can be a high bandwidth memory (HBM), which is a high-performance DRAM made based on a 3D stacking process and is suitable for applications with high memory bandwidth requirements, such as graphics processors, network switching and forwarding equipment (such as routers, switches), etc.
[0034] System on chip (SoC) refers to a technology that integrates a complete system on a single chip and packages all or part of the necessary electronic circuits. In this embodiment, the system on chip is assembled on a board. Figure 1 FIG. 1 is a schematic diagram showing the structure of a board 10 according to an embodiment of the present disclosure. Figure 1 As shown, the board 10 includes a combined processing device 101, which is an artificial intelligence computing unit to support various deep learning and machine learning algorithms to meet the intelligent processing requirements in complex scenarios in the fields of computer vision, speech, natural language processing, data mining, etc. In particular, deep learning technology is widely used in the field of cloud intelligence. A significant feature of cloud intelligence applications is the large amount of input data, which places high demands on the storage capacity and computing power of the platform. The board 10 of this embodiment is suitable for cloud intelligence applications and has huge off-chip storage, on-chip storage and a large amount of computing power.
[0035] The combined processing device 101 is connected to an external device 103 via an external interface device 102. The external device 103 is, for example, a server, a computer, a camera, a display, a mouse, a keyboard, a network card, or a wifi interface. The data to be processed can be transmitted from the external device 103 to the combined processing device 101 via the external interface device 102. The calculation result of the combined processing device 101 can be transmitted back to the external device 103 via the external interface device 102. According to different application scenarios, the external interface device 102 can have different interface forms, such as a PCIe interface.
[0036] The board 10 also includes an external memory 104 for storing data, which includes one or more storage units 105. The external memory 104 is connected to the control device 106 and the combined processing device 101 through a bus and transmits data. The control device 106 in the board 10 is configured to control the state of the combined processing device 101. To this end, in an application scenario, the control device 106 may include a single-chip microcomputer (Micro Controller Unit, MCU).
[0037] Figure 2 Schematic diagram showing the combination processing device 101 of this embodiment. Figure 2 As shown in , the combined processing device 101 includes a computing device 201, an interface device 202, a processing device 203, and a DRAM 204. In one application scenario, the computing device 201, the interface device 202, and the processing device 203 are integrated into the aforementioned system on chip. In another application scenario, the computing device 201 itself is the aforementioned system on chip.
[0038] The computing device 201 is configured to execute user-specified operations, and is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.
[0039] The interface device 202 is used to transmit data and control instructions between the computing device 201 and the processing device 203. For example, the computing device 201 can obtain input data from the processing device 203 via the interface device 202 and write it into the storage device on the computing device 201 chip. Further, the computing device 201 can obtain control instructions from the processing device 203 via the interface device 202 and write them into the control cache on the computing device 201 chip. Alternatively or optionally, the interface device 202 can also read data in the storage device of the computing device 201 and transmit it to the processing device 203.
[0040] The processing device 203, as a general processing device, performs basic controls including but not limited to data handling, starting and / or stopping the computing device 201. Depending on the implementation, the processing device 203 can be a central processing unit, a graphics processing unit or other general and / or special processors. One or more types of processors, these processors include but are not limited to digital signal processors (digital signal processor, DSP), application specific integrated circuits (application specific integrated circuit, ASIC), field programmable gate arrays (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, only with respect to the computing device 201 disclosed in the present invention, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 201 and the processing device 203 are integrated and considered together, the two are regarded as forming a heterogeneous multi-core structure.
[0041] DRAM 204 is the aforementioned high-bandwidth memory, which is used to store data to be processed. The size is usually 16G or larger, and is used to save data of the computing device 201 and / or the processing device 203.
[0042] Figure 3 The figure shows a schematic diagram of the internal structure of a computing device 201. The computing device 201 is used to process input data such as computer vision, speech, natural language, and data mining. The computing device 201 in the figure adopts a multi-core hierarchical structure design, which includes an external storage controller 301, a peripheral communication module 302, an on-chip interconnect module 303, a synchronization module 304, and multiple clusters 305.
[0043] There may be multiple external storage controllers 301, two of which are shown in the figure as an example, and are used to respond to access requests issued by the processor core and access external storage devices, such as Figure 2DRAM 204 in the chip, thereby reading data from outside the chip or writing data. The peripheral communication module 302 is used to receive control signals from the processing device 203 through the interface device 202 to start the computing device 201 to perform tasks. The on-chip interconnect module 303 connects the external storage controller 301, the peripheral communication module 302 and multiple clusters 305 to transmit data and control signals between each module. The synchronization module 304 is a global synchronization barrier controller (GBC) used to coordinate the work progress of each cluster and ensure information synchronization. Multiple clusters 305 are the computing cores of the computing device 201. Four are shown as examples in the figure. With the development of hardware, the computing device 201 disclosed in the present invention can also include 8, 16, 64, or even more clusters 305. Clusters 305 are used to efficiently execute deep learning algorithms.
[0044] Each cluster 305 includes a plurality of processor cores (IPU cores) 306 and a memory core (MEM core) 307 .
[0045] The figure shows four processor cores 306 as an example, but the present disclosure does not limit the number of processor cores 306. Figure 4 Each processor core 306 includes three modules: a control module 41 , a computing module 42 and a storage module 43 .
[0046] The control module 41 is used to coordinate and control the operation of the computing module 42 and the storage module 43 to complete the deep learning task, and includes an instruction fetch unit (IFU) 411 and an instruction decode unit (IDU) 412. The instruction fetch unit 411 is used to obtain instructions from the processing device 203, and the instruction decode unit 412 decodes the obtained instructions and sends the decoding results to the computing module 42 and the storage module 43 as control information.
[0047] The operation module 42 includes a vector operation unit 421 and a matrix operation unit 422. The vector operation unit 421 is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformation; the matrix operation unit 422 is responsible for the core calculation of the deep learning algorithm, namely matrix multiplication and convolution.
[0048] The storage module 43 is used to store or move related data, including a neuron RAM (NRAM) 431, a weight RAM (WRAM) 432, an input / output direct memory access module (IODMA) 433, and a move direct memory access module (MVDMA) 434. NRAM 431 is used to store input and output data and intermediate results for calculation by the processor core 306; WRAM 432 is used to store the weights of the deep learning network; IODMA 433 controls the memory access between NRAM 431 / WRAM 432 and DRAM 204 through the broadcast bus 309; MVDMA 434 is used to control the memory access between NRAM 431 / WRAM 432 and SRAM 308.
[0049] Back to Figure 3 The storage core 307 is mainly used for storage and communication, that is, to store shared data or intermediate results between the processor cores 306, and to perform communication between the cluster 305 and the DRAM 204, communication between the clusters 305, and communication between the processor cores 306. In other embodiments, the storage core 307 has the capability of scalar operation and is used to perform scalar operation.
[0050] The storage core 307 includes a shared memory unit (SRAM) 308, a broadcast bus 309, a cluster direct memory access module (cluster direct memory access, CDMA) 310 and a global direct memory access module (global direct memory access, GDMA) 311. The SRAM 308 plays the role of a high-performance data transfer station. The data reused between different processor cores 306 in the same cluster 305 does not need to be obtained from the DRAM 204 by each processor core 306, but is transferred between the processor cores 306 via the SRAM 308. The storage core 307 only needs to quickly distribute the reused data from the SRAM 308 to multiple processor cores 306, so as to improve the efficiency of inter-core communication and greatly reduce on-chip and off-chip input / output access.
[0051] The broadcast bus 309, CDMA 310 and GDMA 311 are respectively used to perform communication between processor cores 306, communication between clusters 305 and data transmission between clusters 305 and DRAM 204. They will be described below.
[0052] The broadcast bus 309 is used to complete high-speed communication between the processor cores 306 in the cluster 305. The broadcast bus 309 of this embodiment supports inter-core communication modes including unicast, multicast and broadcast. Unicast refers to point-to-point (i.e., single processor core to single processor core) data transmission, multicast is a communication mode of transmitting a copy of data from SRAM 308 to specific processor cores 306, and broadcast is a communication mode of transmitting a copy of data from SRAM 308 to all processor cores 306, which is a special case of multicast.
[0053] CDMA 310 is used to control memory access of SRAM 308 between different clusters 305 in the same computing device 201. GDMA 311 cooperates with external memory controller 301 to control memory access from SRAM 308 of cluster 305 to DRAM 204, or read data from DRAM 204 to SRAM 308.
[0054] The present disclosure relates to inter-node communication. A node is a processing point on a communication line, which can receive information from an upstream node, and after local processing or storage, it can be sent to a downstream node. Inter-node communication can also be called inter-chip communication, and the "inter-chip" described in the present disclosure includes multiple meanings. First, "machine" usually refers to a server computing node host, and "inter-machine communication" can refer to communication between multiple computing node hosts. "Card" usually refers to a dedicated AI computing device installed on a server computing node, and the "card" has one or more chips, such as MLU, GPU. A "machine" usually has multiple "cards", and a distributed training may involve multiple "machines" and multiple "cards" and "chips". There are inter-chip high-speed interconnection communication devices between multiple machines and multiple cards, such as an inter-chip communication network built based on Serdes (serial-deserialization) and a host-level network based on Infiniband. In the present disclosure, inter-chip communication includes communication between different chips between multiple hosts, communication between different chips on the same "card", and communication between different chips in multiple cards on the same host.
[0055] Cards or nodes can communicate with each other through high-speed communication interconnection ports, and a chip can have multiple high-speed communication interconnection ports.
[0056] Communication between cards or nodes can be carried out through RDMA (Remote DMA), that is, node A can asynchronously write / read data to the memory of node B without any operation of node B. After node B receives the data sent by node A, that is, the corresponding received data is written into DRAM from the M port of node B, when performing store-and-forward, the data is read out from DRAM and sent from the N port of node B to other nodes.
[0057] Figure 5a and Figure 5b Different topological structures between a plurality of nodes (node A to node D) are exemplarily shown. Figure 5a is a fully connected structure, and Figure 5b It is a non-fully connected structure. When node A needs to communicate with node B, Figure 5a and Figure 5b Nodes A and B can communicate directly, and when nodes A and C need to communicate, Figure 5a In the case of , nodes A and C can still communicate directly, while in Figure 5b In the case of a full connection, it is necessary to communicate through the path of A->B->C or A->D->C. Although the full connection method helps to improve communication efficiency and shorten the communication path, this full connection requires high hardware costs and is almost impossible to achieve in a complex network structure. Therefore, the present disclosure provides a method for node-to-node routing based on multiple nodes, which is based on the software routing algorithm and protocol of the AI chip, solves the sending (Send) and receiving (Recv) communication between any two points in a non-fully connected scenario, and provides a unified programming interface for upper-level users.
[0058] The technical solution of the present disclosure is described in more detail below in conjunction with the accompanying drawings.
[0059] Figure 6 A flow chart of a method for performing inter-node routing based on multiple nodes according to an embodiment of the present disclosure is shown, comprising: on the host side, in operation S610, constructing a routing tuple, the routing tuple comprising a session identifier, local data storage space information, remote data storage space information and control value storage space information; wherein the session identifier is used to distinguish sessions, the local data storage space information is used to indicate the address of a storage space for storing data in a local node, the remote storage space information is used to indicate the address of a storage space for storing data in a downstream node, and the control value storage space information is used to indicate the address of a storage space for storing control values in a local node and an upstream node and / or a downstream node; in operation S612, sending the routing tuple to multiple nodes on a session path; and on the device side including multiple nodes, in operation 620, receiving the routing tuple; and in operation S622, the node receiving the routing tuple performs inter-node routing according to the routing tuple.
[0060] Figure 7 A schematic block diagram of a system applying the inter-node routing method of the present disclosure according to an embodiment of the present disclosure is shown.
[0061] like Figure 7As shown, the system can be generally divided into a host side and a device side. The host side may include a host and clients 1 and 2 that can access the host. A resident agent program runs on the host, and the resident agent program may include a server process management module and / or a session management module. The server process management module can respond to requests from one or more client processes, and the session management module can create, revoke or manage sessions based on the requests received by the server process management module; alternatively, the resident agent program may only include a session management module, and the session management module can respond to requests from one or more client processes, and create, revoke or manage sessions based on the received requests. It should be understood that the above functions can be implemented in one module, or can be split into more separate modules, and are not limited to the following. Figure 7 As described.
[0062] In the present application, a session may refer to a group of communications. In one embodiment of the present application, the establishment of a session may be unidirectional and independent of each other. If there are multiple groups of communications occurring simultaneously, different sessions need to be used. One-way means, for example, a session from node A->node B can only be sent from node A and received by node B. If reverse communication from node B->node A is required, it is necessary to establish another session from node B->node A. The advantage of this is that if the reverse path is not used, there is no need to establish the hardware resources required for the reverse path, thereby avoiding waste of resources. Mutual independence means that different hardware resources (such as memory) are used between sessions and they are isolated from each other. The beneficial effect of this is that the design logic can be simplified to avoid the operation of one session interfering with the operation of another session.
[0063] It should be noted that the term "session path" in the above text refers to a path in which a session is established. In other words, there may be multiple paths from one node to another node, but in the solution of the present disclosure, only the path in which a session is established is called a "session path".
[0064] It should also be pointed out that in Figure 7 In the figure, although the client is shown as being separate from the host, the client can be a hardware entity separate from the host, or it can be a client process in the host, and the user can access the host through the client. Figure 7 In the example, assuming that client 1 needs to send information and client 2 needs to receive information, the following operations can be performed.
[0065] exist Figure 7When the system shown is started, the resident agent can first send the resident kernel to multiple nodes, so that when the node participates in a session, that is, when the node is on the session path, these intermediate nodes on the session path run the routing tuple through the resident kernel. The resident kernel will execute the above routing tuple at each intermediate node on the session path, thereby reducing or eliminating the need for host participation when performing communication routing.
[0066] Furthermore, after constructing the routing tuple or sending the routing tuple information to the device side, in order to set the sending node and the receiving node, a temporary sending Kernel can be sent to the starting node on the session path, and a temporary receiving Kernel can be sent to the receiving node on the session path. It should be pointed out that the signals between the various components may be bidirectional, for example, the host or the client needs to perceive the execution status of the sending node and the receiving node.
[0067] According to an embodiment of the present disclosure, the client 1 can directly transmit or send a Kernel to the sending node so that the sending node can perform a sending operation (such as Figure 7 The sending Kernel may include relevant sending information, such as the address of the data to be sent, the size of the data to be sent, etc. If the client 2 needs to receive information, it may directly transmit or send the receiving Kernel (Recv Kernel) to the receiving node so that the receiving node can perform the receiving operation (such as Figure 7 The receiving Kernel may include relevant receiving information, and the receiving information may include the address of the data to be received, the size of the data to be received, etc.
[0068] According to another embodiment of the present disclosure, a user may transmit a sending request and related sending information to a host through a client 1, and similarly, the sending information may include the address of the data to be sent, the size of the data to be sent, etc.; a user may transmit a receiving request and related receiving information to a host through a client 2, and similarly, the receiving information may include the address of the data to be received, the size of the data to be received, etc. The resident agent program running on the host may transmit a sending Kernel including the sending information to a sending node, and may transmit a receiving Kernel including the receiving information to a receiving node.
[0069] More specifically, the resident agent is used to provide the following functions: as a socket interface, it allows the client to access the host through the socket interface and provides the client with the necessary protocol information required for routing. The functions of the resident agent include but are not limited to: searching the topology of the current host; querying whether there is a reachable path between any two nodes; creating and closing sessions and the memory resources of the nodes involved; sending a resident kernel to each node; and sending a temporary send kernel to the sending node and a temporary recv kernel to the receiving node.
[0070] Next, the execution of operations S610 and S612 by the resident agent will be described in detail. The resident agent may include a server process, and the client process may exist alone on other clients, or may be in an entity of the host together with the server process. According to one embodiment of the present disclosure, the host may send a routing tuple to a corresponding node on a corresponding session path through the resident agent in response to a request from the client process. After the node receives the routing tuple, the routing tuple will be executed through the received Kernel, and the data will be routed according to the instructions of the routing tuple.
[0071] The routing tuple can distinguish different sessions by a session identifier (Session_ID). It should be understood that communications on different communication paths belong to different sessions, and even if the communication paths are exactly the same, communications occurring at different times also belong to different sessions.
[0072] In addition to the session identifier, the routing tuple also includes local data storage space information (ptr_local_buffer) and remote data storage space information (ptr_remote_buffer). When a node sends data to a downstream node, it can extract data from the storage space indicated by the local data storage space information and send the extracted data to the downstream node. In addition, the remote data storage space information here is used to indicate the space where the data sent by the current node will be stored in the downstream node. As described above, the inter-node communication in the present disclosure can adopt the RDMA method. If the current node knows the data storage space of the downstream node, the current node can directly store the data remotely in the downstream node without the downstream node performing any operation.
[0073] The control value storage space information is used to indicate the address of the storage space for storing the control value in the local node and the upstream node and / or the downstream node. The control value can control the amount of data sent or received between the upstream and downstream nodes. Figure 8 To describe the control values involved in inter-node communication.
[0074] Figure 8 A schematic diagram of a communication system including a transmitting end and a receiving end according to an embodiment of the present disclosure is shown.
[0075] The communication system can be divided into components according to functions, and is divided into a transmitting end 1 and a receiving end 2. In the present application, the transmitting end 1 can be an upstream node, and the receiving end 2 can be a downstream node. The transmitting end 1 can include a first storage module 11, a first instruction execution module 12, a first sending module 13, a first receiving module 14 and a first control module 15, and the receiving end 2 includes a second storage module 21, a second instruction execution module 22, a second sending module 23, a second receiving module 24 and a second control module 25.
[0076] For the transmitter 1, the function of each module is:
[0077] The first storage module 11 is used to store data and instructions. The data source may be local original data, data sent from an upstream device (in case of multi-level transmission), or result data after calculation between data sent from an upstream device and local data.
[0078] The first instruction execution module 12 is used to execute the instructions in the first storage module 11 and is responsible for completing the execution processing of the application layer logic. In this communication system, the sending request is initiated by the instruction, that is, the communication behavior is initiated by the application layer. In addition, the first instruction execution module 12 can also complete local operations according to the instructions.
[0079] First sending module 13: The sending end 1 is directly connected to the physical link through the first sending module 13. The first sending module 13 is used to send data and control values. The first sending module 13 can count the communication packets sent and receive the response to each packet. If NACK (negative acknowledgment, meaning abnormal response) is received or ACK (Acknowledgement, meaning positive feedback, indicating that the receiving end has correctly received the corresponding packet) is not received for a long time, the abnormal state is recorded and the interrupt is reported to the first control module 15. If ACK is received within the specified time, ACK can be replied to the ACK.
[0080] First receiving module 14: The transmitting end can be directly connected to the physical link through the first receiving module 14. The first receiving module 14 on the transmitting end 1 is only used to receive the control value.
[0081] The first control module 15 is used to issue tasks, receive the interruption of the first sending module 13, suspend the operation of the first instruction execution module 12, and reissue tasks to let the first instruction execution module 12 retransmit.
[0082] For the receiving end 2, the function of each module is:
[0083] The second storage module 21 is used to store data and instructions. The data source may be local original data, data sent from an upstream sending end, or result data calculated from data sent from an upstream sending end and local data.
[0084] The second instruction execution module 22 is used to execute the instructions in the second storage module 21 and is responsible for completing the execution processing of the application layer logic. In this communication system, the sending request is initiated by the instruction, that is, the communication behavior is initiated by the application layer. In addition, the second instruction execution module 22 can also complete local operations according to the instructions.
[0085] Second receiving module 24: The receiving end 2 is directly connected to the physical link through the second receiving module 24. The second receiving module 24 receives the received data packet and completes the memory synchronization operation according to the type of the data packet. The received packet is checked and corrected. If the error cannot be corrected, a NACK is replied to the sending module of the sending end. An ACK is replied for each correct reception of each packet. If the ACK sent by the second receiving module 24 does not receive an ACK response, the abnormal state is recorded and an interrupt is reported to the second control module 25.
[0086] Second sending module 23: The receiving end is directly connected to the physical link through the second sending module 23. The second sending module 23 is only used to send the control value on the receiving end.
[0087] The second control module 25 is used to issue the task, and according to the interruption sent by the second receiving module 24, the operation of the second instruction execution module 22 is suspended, and the task is reissued to let the second instruction execution module 22 retransmit.
[0088] The following is a detailed description of the various specifications of the application layer when performing data sending tasks.
[0089] The control value at the sending end 1 means: the number of data fragments that the sending end 1 has currently sent; and the control value at the receiving end 2 means the number of data fragments that the receiving end 2 has currently processed.
[0090] Referring to the data structure of the queue, the control value of the sender can be recorded as Tail, and the control value of the receiver 2 can be recorded as Head. It can be seen that if the Tail value of the upstream node is the same as the Head value of the downstream node, it means that the data sent by the upstream node is received by the downstream node, and if the head value is less than the tail value, it means that the upstream node has sent the data, but the downstream node has not received it yet. The tail value and the head value can be used to control the continued transmission and termination of data.
[0091] Furthermore, in both the upstream node and the downstream node, there are the original and the copy of the corresponding tail value, as well as the original and the copy of the head value.
[0092] Head and Tail need to meet the following constraints:
[0093] 1. Both Head and Tail are monotonically increasing, and each increase is the number of Data fragments sent or received this time.
[0094] 2. At any time, the Head is not greater than the Tail.
[0095] 3. The destination address of the Head and Tail remains unchanged, and the Head and Tail sent later will overwrite the original values in the device storage space. In other words, multiple Head values or multiple Tail values are not allowed, but only the previous Head value can be overwritten by the new Head value, or the previous Tail value can be overwritten by the new Tail value.
[0096] The above generally introduces the concepts and constraints of Tail value and Head value. In the present disclosure, except for the sending node and the receiving node, each current node is both the downstream node of its upstream node and the upstream node of its downstream node, so its Tail value and Head need to be carefully designed.
[0097] According to one embodiment of the present disclosure, in addition to the sending node and the receiving node at the end, the control value storage space information for other nodes may include: a local tail value storage space (local_rx_tail), used to indicate the storage space of the tail value of the upstream node in the local node; a local head value storage space (local_tx_head), used to indicate the storage space of the head value fed back by the downstream node in the local node; an upstream head value storage space (remote_rx_head), used to indicate the storage space of the head value fed back by the local node in the upstream node; and / or a downstream tail value storage space (remote_tx_tail), used to indicate the storage space of the tail value sent by the local node in the downstream node.
[0098] rx and tx represent receiving from upstream and sending to downstream respectively. For the convenience of expression, local and remote tags are introduced here to indicate whether it is located at the current node or the remote node. If it is remote_rx_head, it represents the upstream node address, and local_rx_head means that it is a local copy of remote_rx_head. If it is remote_tx_tail, it represents the downstream node address, and local_tx_tail means that it is a local copy of remote_tx_tail. It should be noted that the upstream and downstream expressions can be converted: the remote_tx_tail address seen at the current node is actually the local_rx_tail address seen by the downstream node. Similarly, the remote_rx_head address seen at the current node is actually the local_tx_head address seen by the upstream node. Figure 7 For example, if node 3 is the local node, node 2 is the upstream node of the local node, and node 4 is the downstream node of the local node. It should also be understood that the above-mentioned symbolic notation, such as local_rx_head, refers to the address, not the value or content stored in the address. For ease of understanding, if the content stored in the address is involved, it is expressed in the form of "address.value", for example, local_rx_head.value represents the value in the address local_rx_head, remote_tx_tail.value represents the value in the address remote_tx_tail, and so on.
[0099] For node 3, the local tail value storage space local_rx_tail is used to indicate the storage space of the tail value of node 2 in node 3, that is, the storage space of the tail value corresponding to the data sent by node 2 in node 3. The value in this storage space can be expressed as local_rx_tail.value.
[0100] For node 3, the local head value storage space local_tx_head is used to indicate the storage space in node 3 of the head value fed back by node 4 after node 3 sends data to node 4. The value in the storage space can be expressed as local_tx_head.value.
[0101] For node 3, the upstream head value storage space remote_rx_head is used to indicate the storage space in node 2 of the head value fed back by node 3 after receiving data from node 2. The value in the storage space can be expressed as remote_rx_head.value.
[0102] For node 3, the downstream tail value storage space remote_tx_tail is used to indicate the storage space of the tail value sent by node 3 in node 4 after node 3 sends data to node 4. The value in the storage space can be expressed as remote_tx_tail.value.
[0103] Thus, in addition to the starting sending node and the ending receiving node, other intermediate nodes may include the above four elements, namely local_rx_tail, local_tx_head, remote_rx_head, and remote_tx_tail; and for the sending node, since it has no upstream node, the tuple sent to the sending node may not include the elements local_rx_tail and remote_rx_head, or the elements local_rx_tail and remote_rx_head may be empty; similarly, for the receiving node, since it has no downstream node, the tuple sent to the receiving node may not include the elements local_tx_head and remote_tx_tail, or the elements local_tx_head and remote_tx_tail may be empty.
[0104] Therefore, the basic structure of each route tuple is: <session_id,ptr_local_buffer,local_rx_tail,remote_rx_head,ptr_remote_buffer,remote_tx_tail,local_tx_head> .
[0105] According to one embodiment of the present disclosure, the amount of data that still needs to be sent to the downstream node is determined based on the difference between the local tail value (stored in local_rx_tail) and the upstream head value (stored in remote_rx_head) (ie, the difference between local_rx_tail.value and remote_rx_head.value).
[0106] As described above, the local tail value (local_rx_tail.value) in the storage space indicated by the element local_rx_tail is used to reflect how much data the local node has sent to the downstream node, and the upstream head value (remote_rx_head.value) in the storage space indicated by the element remote_rx_head is used to reflect how much data the local node has received from the upstream node. If there is a difference between the two, for example, the upstream head value is greater than the local tail value, it means that the data received by the local node from the upstream is more than the data sent to the downstream, which further indicates that the local node has not yet forwarded all the data received from the upstream node to the downstream node, so it is necessary to continue to send the data that has not yet been sent to the downstream node. Through the local tail value and the upstream head value, the data transmission and reception between the upstream and downstream nodes can be accurately controlled.
[0107] According to an embodiment of the present disclosure, the routing tuple further includes a priority element priority, which is used to indicate the priority of the session. Therefore, preferably, the basic structure of each routing tuple is: <session_id,priority,ptr_local_buffer,local_rx_tail,remote_rx_head,ptr_remote_buffer,remote_tx_tail,local_tx_head> .
[0108] In an embodiment of the present disclosure, each node supports not only a single session, but also multiple sessions. In this case, multiple sessions may exist at each node, for example, they may be formed as a session list session_list. The resident Kernel sent to the node may scan the session list to distinguish each session. In multiple sessions, a priority may be set, which indicates the priority of each session in the session list, and its value may vary between 1 and MAX_PRIORITY (maximum priority). The higher the priority of a session, the higher the bandwidth allocated to the session when multiple sessions communicate simultaneously.
[0109] According to one embodiment of the present disclosure, sessions with different priorities can occupy different bandwidths in the following manner: splitting the data involved in the session so that the data is split into data slices; sending the data slices of sessions with different priorities in each cycle, wherein the higher the priority of the session, the more copies of the corresponding data slices are sent in each cycle, so that the higher the priority of the session, the higher the bandwidth allocated to the session.
[0110] Fig. 9A schematic diagram showing that sessions with different priorities occupy different bandwidths when communicating.
[0111] like Fig. 9 As shown, it is assumed that there are three sessions, namely, sessions S1, S2 and S3, wherein the priority of session S1 is 4, the priority of session S2 is 2, and the priority of session S3 is 1. In order to make the sessions with higher priorities occupy higher bandwidth, more data slices can be transmitted per unit time according to the value of the priority. In this embodiment, the larger the value of the priority, the higher the priority. Fig. 9 As shown, for example, session S1 can transmit 4 data slices in a unit time, occupying 4 communication time slots (t1~t4), session S2 can transmit 2 data slices in a unit time, occupying 2 communication time slots (t5~t6), and session S3 can transmit 1 data slice in a unit time, occupying 1 communication time slot (t7). Therefore, in each unit time, the session with higher priority occupies more bandwidth.
[0112] It is important to understand that Fig. 9 This is just an example, and other methods may be used to preferentially execute sessions with higher priorities. For example, all low-priority sessions may be kept waiting until higher-priority sessions are completed, and then the corresponding sessions may be executed in descending order of priority.
[0113] According to a preferred embodiment of the present disclosure, the routing tuple further includes storage space size information, which is used to indicate the maximum size of the storage space, buffer_size, wherein the maximum number of data fragments sent to the downstream node is an integer obtained by dividing the maximum size of the storage space by the size of each data fragment. Therefore, preferably, the basic structure of each routing tuple is: <session_id,priority,ptr_local_buffer,local_rx_tail,remote_rx_head,ptr_remote_buffer,remote_tx_tail,local_tx_head,buffer_size> .
[0114] It should be understood that the storage space size information may be a default one, and therefore the element is not necessarily included in the routing tuple.
[0115] The following takes a certain intermediate node as a local node and a single session as an example to describe in detail the node's data reception, storage, and forwarding process. The node that sends data to the local node is the upstream node, and the node to which the local node's data is sent is the downstream node. It should be understood that the description of the sending node and the receiving node as the end is omitted in this example, and only the intermediate node that forwards the data is described. More specifically, the data reception, storage, and forwarding process in this example can be performed by the resident kernel of the intermediate node.
[0116] (1) If the local_rx_tail.value of the intermediate node is increased by 1, it means that the local memory ptr_local_buffer has received a data slice of length, for example, Slice_size, and its storage position at the local node starts at ptr_local_buffer+local_rx_head.value*Slice_size. Thus, the data slice is cached to the corresponding position of the on-chip storage area of the current node.
[0117] (2) Next, the data slice of length Slice_size needs to be sent to the ptr_remote_buffer of the downstream node shown in the routing tuple via RDMA, and its position starts at ptr_remote_buffer+local_tx_tail.value*Slice_size.
[0118] (3) After ensuring that the above data slices are written to the downstream node via RDMA, update the following parameters:
[0119] local_tx_tail.value=(local_tx_tail.value+1)%max_slice
[0120] local_rx_head.value=(local_rx_head.value+1)%max_slice;
[0121] The % symbol is a remainder symbol, which is used to prevent address overflow due to exceeding max_slice.
[0122] Next, the tail value local_tx_tail.value in the address local_tx_tail and the head value local_rx_head.value in the address local_rx_head are written to the corresponding remote remote_tx_tail and remote_rx_head respectively through RDMA, thereby completing the update of the tail value local_tx_tail.value and the head value local_rx_head.value. Where max_slice = buffer_size / Slice_size. It can be seen that by determining the integer part of the value of buffer_size, the maximum number of data slices to be transmitted can be determined.
[0123] This application can reduce the consumption of computing cores or computing resources as much as possible through the resident kernel, thereby improving resource utilization. Specifically, after starting the resident agent program, the resident kernel is sent to multiple nodes through the resident agent program, and the resident kernel can continuously perform the following scanning operations:
[0124] (1) Scan the session list session_list to obtain all session identifiers session_id and their corresponding priorities. Next, in order to provide different bandwidths or different execution orders for each session of different priorities, a session identifier queue session_id_array is generated that is extended according to the priority value, so that different numbers of data slices can be transmitted in one unit time for different sessions.
[0125] like Fig. 9 As shown, the data slices transmitted in time slots t1-t4 belong to the same session and therefore have the same session identifier, which is equivalent to expanding the session identifier into a session identifier array session_id_array according to the priority value.
[0126] (2) Next, the session identifier session_id in the session identifier queue session_id_array is traversed to form the following Fig. 9 The communication method shown.
[0127] (3) Repeat step (1) until the user actively terminates the resident agent program.
[0128] It should be understood that when no routing tuple information is sent or the sending kernel and receiving kernel are not issued, session_list is empty and the resident kernel is idle. When the routing tuple information is sent or the sending kernel and receiving kernel are issued, the resident kernel of the intermediate node performs store-and-forward according to the received routing tuple.
[0129] The above introduces the formation of routing tuples and the process of routing data through routing tuples. Fig.10 FIG. 4 shows a flow chart of establishing a session path before constructing a routing tuple. Fig.10 As shown, according to one embodiment of the present disclosure, before constructing a routing tuple, a session path is established, and the establishment of the session path includes: in operation S1010, searching a topological structure of multiple nodes; in operation S1020, determining whether any two nodes are reachable based on the topological structure; in operation S1030, if the two nodes are reachable, establishing a session path through the nodes between the two reachable nodes.
[0130] Before building a routing tuple, the host can first search the topology structure formed by all nodes to determine whether there is a direct or indirect connection relationship between all nodes, that is, it can determine whether any two nodes are "reachable" based on the topology structure. "Reachable" means that there is a direct or indirect physical connection between any two nodes, that is, in the topology structure, one node can always reach another node without building a new hardware connection.
[0131] Fig.11 A schematic diagram showing a session path established between two reachable nodes.
[0132] like Fig.11 As shown, for example, there are two tasks, Task1 and Task2. In Task1, data needs to be sent from node A to node D, while in Task2, data needs to be sent from node A to node F. After searching, nodes A and D are reachable, and it is necessary to pass through nodes B and C to reach node D from node A; nodes A and F are also reachable, and it is necessary to pass through nodes B and E to reach node F from node A. Therefore, a session path 1 can be established for Task1, i.e., node A->node B->node C->node D. In addition, since node D is directly connected to node F, a session path 1' can also be established for Task1, i.e., node A->node B->node E->node F->node D; and a session path 2 can be established for Task2, i.e., node A->node B->node E->node F. In addition, a session path 2' can also be established for Task2, i.e., node A->node B->node C->node D->node F.
[0133] After the session path is established, different resources, such as cache space, etc., are allocated to different sessions (eg, different sessions corresponding to different tasks) at each node.
[0134] from Fig.11 It can be seen that different session paths can be established for the same task. In different application scenarios, different session paths can be established for different performance requirements. According to one embodiment of the present disclosure, the session path can be selected as the shortest session path, the idlest session path, or the highest bandwidth session path.
[0135] The shortest session path refers to the session path that passes through the least number of nodes. Generally, the shortest session path involves the least number of nodes, so the transmission path is shorter and the delay is lower. At the same time, the possibility of nodes being occupied by other sessions is also lower, and the transmission speed may be faster.
[0136] The most idle session path refers to the session path least occupied by other sessions. Since nodes need to allocate corresponding resources for sessions, nodes in the most idle session path will have more resources to allocate to participating sessions.
[0137] The highest bandwidth session path refers to a session path with the highest transmission speed during communication or a session path with the least bandwidth occupied by other sessions.
[0138] The above describes the data receiving and forwarding process of the node. The following will introduce the detailed execution process on the host side and the device side.
[0139] On the host side:
[0140] (1) Two clients responsible for sending and receiving requests, e.g. Figure 7 Client 1 and client 2 in the example access the resident agent in the host;
[0141] (2) Use the resident agent to query whether the sending node and the receiving node are reachable;
[0142] (3) If it is reachable, apply to create a session path between the sending node and the receiving node, and obtain or create a routing tuple. Otherwise, exit the execution.
[0143] On the device side:
[0144] (1) Start a Send Kernel in the computing core of the sending node. The computing core can be a processor unit in the node with scalar or vector computing capabilities. The computing core can be Figure 2 The computing device 201 or processing device 203 shown, or may be Figure 3The processor core 306 or cluster in the Figure 4 The SendKernel performs the following sending process:
[0145] (a) Split the size of the data to be sent according to the size of the data slice Slice_size;
[0146] (b) Write a slice of data of size Slice_size to the position ptr_remote_buffer+local_tx_tail.value*Slice_size of the remote_buffer of the downstream node through RDMA;
[0147] (c) Update local_tx_tail.value=(local_tx_tail.value+1)%max_step, and write the latest value downstream to the address remote_tx_tail via RDMA.
[0148] (d) Repeat steps (b) and (c) until there are no more data pieces to send, then exit the Send Kernel, and the host side can sense that the sending process is completed.
[0149] (2) A Recv Kernel is started in the computing core of the receiving node. The Recv Kernel performs the following receiving process:
[0150] (a) Read the tail value local_rx_tail.value in local_rx_tail and determine whether the tail value local_rx_tail.value is equal to the number of data slices to be received. Read the address ptr_local_buffer+local_rx_head.value*Slice_size corresponding to (local_rx_tail.value–local_rx_head.value) SLICE_size.
[0151] (b) Repeat step (a). If the tail value in local_rx_tail is equal to the number of data set slices, the current Recv process is exited. Correspondingly, the host side will also sense that the Recv process is completed.
[0152] The present disclosure further provides a host for performing inter-node routing based on multiple nodes, on which a resident agent program runs, and the resident agent program is used to execute the method described in the above operation S610 and / or S612.
[0153] The present disclosure also provides a device for performing inter-node routing based on multiple nodes, the device comprising multiple nodes, each node having a kernel running therein, the kernel being used to perform the method described in the above operations S620 and / or S622. It should be noted that the host can send a resident kernel to each node, when the node acts as an intermediate node, the resident kernel can perform data routing or storage and forwarding according to the routing tuple, and when the node acts as a sending node or a receiving node, the node can receive a sending kernel (Send Kernel) or a receiving kernel (Recv Kernel) sent from the host or the client, so as to perform the function of the sending node by sending the kernel, and to perform the function of the receiving node by receiving the kernel.
[0154] The present disclosure also provides a system for performing inter-node routing based on multiple nodes, wherein the system includes the host as described above and the device as described above.
[0155] The present disclosure also provides a computer-readable storage medium, comprising computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the method described above is executed.
[0156] The present disclosure also provides an electronic device, including: one or more processors; and a memory, wherein the memory stores computer executable instructions, and when the computer executable instructions are executed by the one or more processors, the electronic device executes the method described above.
[0157] The following table provides a comparison between the technical solution of the present disclosure and the prior art.
[0158]
[0159] Combined with the above table, the differences between the present invention and the prior art are mainly reflected in the following aspects:
[0160] In the first solution, that is, the solution using switches and switching chips, a full connection method is usually adopted, and the data transfer involved is implemented by hardware such as hardware switches or chips. Therefore, there is no need for software-based communication primitives, and there is no need for resident agent programs at the nodes; in addition, since it is fully hardware-connected, there is no need for transfer nodes.
[0161] In the second solution, that is, in the collective communication software solution, data transfer is done by software, requiring users to explicitly determine the transfer / intermediate nodes in the software, and these transfer / intermediate nodes need to use communication primitives such as Ring Broadcast to forward data, which is different from the solution using switches and switching chips. In addition, the combined communication software solution does not require a fully interconnected topology, which effectively reduces the cost and complexity of the system.
[0162] For the solution of the present application, its data transfer adopts software storage and forwarding instead of hardware storage and forwarding of the first solution. Unlike the second solution, the solution of the present application adopts implicit forwarding, that is, the user does not need to know which specific intermediate nodes are used, but only needs to know or set the sending node and the receiving node. In other words, in the solution of the present application, the user does not need to explicitly select the transfer / intermediate node, while the second solution requires the user to make an explicit selection. The second solution will lead to the following problems: First, the original transfer / intermediate node does not need to participate in the user programming process, but in this solution, in order to allow the transfer / intermediate node to participate, the user needs to change the programming model; second, the user does not know which node should be selected as the transfer / intermediate node. In addition, compared with the first solution, the technical solution of the present application does not require a fully interconnected topology, which effectively reduces the cost and complexity of the system.
[0163] Furthermore, in the second solution, when multiple sending nodes call the Ring Broadcast primitive at the same time, the intermediate node is usually in a busy state, but the intermediate node cannot adjust the bandwidth of multiple groups of communications, so that faster communications occupy a larger bandwidth, and load balancing of multiple groups of communications cannot be achieved. However, the technical solution of the present disclosure can still achieve load balancing when implemented in software, and can even adjust the bandwidth occupied by each group of communications according to priority.
[0164] In general, compared with the prior art, the technical solution of the present disclosure avoids the hardware cost of introducing expensive switching chips and switches in the sending and receiving communication operations in non-fully connected scenarios. In addition, in the present disclosure, a pure software method is adopted, and the maximum theoretical bandwidth of the communication port can be achieved when there is no conflict in the session. Furthermore, according to an embodiment of the present disclosure, when multiple sessions are communicating in conflict at the same time, the priority of different sessions can be guaranteed according to the priority, so as to meet the user needs to the greatest extent. In addition, by means of a resident Kernel, the consumption of the number of computing cores is minimized.
[0165] According to different application scenarios, the electronic equipment or device disclosed herein may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, PC devices, IoT terminals, mobile terminals, mobile phones, driving recorders, navigators, sensors, cameras, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, automatic driving terminals, transportation, household appliances, and / or medical equipment. The transportation includes airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, and range hoods; the medical equipment includes magnetic resonance imaging, ultrasound machines and / or electrocardiographs. The electronic equipment or device disclosed herein may also be applied to the Internet, IoT, data centers, energy, transportation, public administration, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, and medical fields. Further, the electronic equipment or device disclosed herein may also be used in cloud, edge, and terminal applications related to artificial intelligence, big data, and / or cloud computing. In one or more embodiments, electronic devices or devices with high computing power according to the disclosed solution can be applied to cloud devices (such as cloud servers), while electronic devices or devices with low power consumption can be applied to terminal devices and / or edge devices (such as smart phones or cameras). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and / or edge device are compatible with each other, so that according to the hardware information of the terminal device and / or edge device, appropriate hardware resources can be matched from the hardware resources of the cloud device to simulate the hardware resources of the terminal device and / or edge device, so as to complete the unified management, scheduling and collaborative work of end-to-end or cloud-edge-to-end.
[0166] The embodiments of the present disclosure are described in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method and its core idea of the present disclosure. At the same time, changes or deformations made by those skilled in the art based on the ideas of the present disclosure, the specific implementation methods and the scope of application of the present disclosure, all fall within the scope of protection of the present disclosure. In summary, the content of this specification should not be understood as a limitation on the present disclosure.
[0167] The following clauses can better understand the technical solution of the present disclosure.
[0168] Clause 1. A method for performing inter-node routing based on multiple nodes, comprising:
[0169] Constructing a routing tuple, the routing tuple comprising a session identifier, local data storage space information, remote data storage space information, and control value storage space information; wherein the session identifier is used to distinguish a session, the local data storage space information is used to indicate an address of a storage space for storing data in a local node, the remote storage space information is used to indicate an address of a storage space for storing data in a downstream node, and the control value storage space information is used to indicate an address of a storage space for storing a control value in a local node and an upstream node and / or a downstream node;
[0170] The routing tuple is sent to multiple nodes on a session path, so that the nodes receiving the routing tuple can perform inter-node routing according to the routing tuple.
[0171] Clause 2. The method of clause 1, wherein the control value storage space information comprises:
[0172] Local tail value storage space (local_rx_tail), used to indicate the storage space of the tail value of the upstream node in the local node;
[0173] Local head value storage space (local_tx_head), used to indicate the storage space of the head value fed back by the downstream node in the local node;
[0174] Upstream head value storage space (remote_rx_head), used to indicate the storage space of the head value fed back by the local node in the upstream node; and / or
[0175] The downstream tail value storage space (remote_tx_tail) is used to indicate the storage space of the tail value sent by the local node in the downstream node.
[0176] Clause 3. The method according to clause 1 or 2, wherein the routing tuple further comprises a priority to indicate a priority of the session.
[0177] Clause 4. The method of clause 3, wherein the higher the priority of a session, the higher the bandwidth allocated to the session.
[0178] Clause 5. The method according to any one of clauses 1-4, wherein the routing tuple further includes storage space size information for indicating a maximum size of the storage space.
[0179] Clause 6. The method according to any one of clauses 1 to 5, further comprising, before constructing the routing tuple, establishing a session path, wherein establishing the session path comprises:
[0180] Search the topology of multiple nodes;
[0181] Determine whether any two nodes are reachable from each other according to the topological structure;
[0182] If two nodes are reachable, a session path is established through the nodes between the two reachable nodes.
[0183] Clause 7. The method according to any one of clauses 1-6, wherein the session path is the shortest session path, the idlest session path or the highest bandwidth session path.
[0184] Clause 8. The method of any one of clauses 1-7, wherein the session path is unidirectional.
[0185] Clause 9. The method of any one of clauses 1-8, wherein the session path is session independent.
[0186] Clause 10. The method according to any one of clauses 1-9, wherein the method is performed by a resident agent on a host, and the resident agent sends a routing tuple to a node on a corresponding session path in response to a request from a client.
[0187] Clause 11. A method according to any one of clauses 1-9, wherein the method is performed by a resident agent and a client on a host, and wherein the client sends a routing tuple to a sending node and a receiving node on a corresponding session path, and wherein the resident agent sends a routing tuple to other nodes on the corresponding session path in response to a request from the client.
[0188] Clause 12. The method according to any one of clauses 1-11 further includes: sending a resident kernel to multiple nodes so that the intermediate nodes on the session path run the routing tuple through the resident kernel; and / or sending a resident kernel to multiple nodes so that the multiple nodes scan the session list through the resident kernel.
[0189] Clause 13. The method according to clause 12 further comprises: sending a temporary sending kernel to the starting node; and sending a temporary receiving kernel to the receiving node.
[0190] Clause 14. A method for performing inter-node routing based on a plurality of nodes, comprising:
[0191] Receive a routing tuple, the routing tuple including a session identifier, local data storage space information, remote data storage space information, and control value storage space information; wherein the session identifier is used to distinguish a session, the local data storage space information is used to indicate an address of a storage space for storing data in a local node, the remote storage space information is used to indicate an address of a storage space for storing data in a downstream node, and the control value storage space information is used to indicate an address of a storage space for storing a control value in a local node and an upstream node and / or a downstream node;
[0192] Inter-node routing is performed on the session path according to the routing tuple.
[0193] Clause 15. The method of clause 14, wherein the control value storage space information comprises:
[0194] Local tail value storage space (local_rx_tail), used to indicate the storage space of the tail value of the upstream node in the local node;
[0195] Local head value storage space (local_tx_head), used to indicate the storage space of the head value fed back by the downstream node in the local node;
[0196] Upstream head value storage space (remote_rx_head), used to indicate the storage space of the head value fed back by the local node in the upstream node; and / or
[0197] The downstream tail value storage space (remote_tx_tail) is used to indicate the storage space of the tail value sent by the local node in the downstream node.
[0198] Clause 16. The method of clause 15, wherein the amount of data sent to the downstream node is determined based on a difference between a local tail value in the local tail value storage space and an upstream head value in the upstream head value storage space.
[0199] Clause 17. The method of any one of clauses 14-16, wherein the routing tuple further comprises a priority to indicate a priority of the session.
[0200] Clause 18. The method according to clause 17, comprising:
[0201] Splitting the data involved in the session so that the data is split into data fragments;
[0202] In each cycle, data slices of sessions with different priorities are sent, wherein the higher the priority of a session, the more copies of the corresponding data slices are sent in each cycle, so that the higher the priority of a session, the higher the bandwidth allocated to the session.
[0203] Clause 19. A method according to Clause 18, wherein the routing tuple further includes storage space size information for indicating the maximum size of the storage space, wherein the maximum number of data shards sent to the downstream node is an integer obtained by dividing the maximum size of the storage space by the size of each data shard.
[0204] Clause 20. The method of any one of clauses 14-19, wherein the session path is the shortest session path, the idlest session path, or the highest bandwidth session path.
[0205] Clause 21. The method of any one of clauses 14-20, wherein the session path is unidirectional.
[0206] Clause 22. The method of any one of clauses 14-21, wherein the session path is session independent.
[0207] Clause 23. A method according to any one of clauses 14-22, wherein the routing tuple is run by a resident kernel running at an intermediate node of the session path; and / or the session list is scanned by a resident kernel running at multiple nodes of the session path.
[0208] Clause 24. The method according to Clause 23 further comprises: a starting node on the session path receives a temporary sending Kernel; and a receiving node on the session path receives a temporary receiving Kernel.
[0209] Clause 25. A method for performing inter-node routing based on a plurality of nodes, comprising:
[0210] On the host side,
[0211] Constructing a routing tuple, the routing tuple comprising a session identifier, local data storage space information, remote data storage space information, and control value storage space information; wherein the session identifier is used to distinguish a session, the local data storage space information is used to indicate an address of a storage space for storing data in a local node, the remote storage space information is used to indicate an address of a storage space for storing data in a downstream node, and the control value storage space information is used to indicate an address of a storage space for storing a control value in a local node and an upstream node and / or a downstream node;
[0212] sending the routing tuple to a plurality of nodes on a session path; and
[0213] On the device side including multiple nodes,
[0214] receiving the routing tuple;
[0215] The node receiving the routing tuple performs inter-node routing according to the routing tuple.
[0216] Clause 26. A host for performing inter-node routing based on multiple nodes, on which a resident agent program runs, wherein the resident agent program is used to execute the method as described in any one of Clauses 1-13.
[0217] Clause 27. A device for performing inter-node routing based on multiple nodes, the device comprising multiple nodes, each node running a kernel, the kernel being used to execute the method as described in any one of clauses 14-24.
[0218] Clause 28. A system for performing inter-node routing based on multiple nodes, the system comprising the host as described in Clause 26 and the device as described in Clause 27.
[0219] Clause 29. A computer-readable storage medium comprising computer-executable instructions, which, when executed by one or more processors, perform the method as described in any one of Clauses 1-24.
Claims
1. A method for performing inter-node routing based on multiple nodes, comprising: Constructing a routing tuple, the routing tuple comprising a session identifier, local data storage space information, remote data storage space information, and control value storage space information; wherein the session identifier is used to distinguish a session, the local data storage space information is used to indicate an address of a storage space for storing data in a local node, the remote storage space information is used to indicate an address of a storage space for storing data in a downstream node, and the control value storage space information is used to indicate an address of a storage space for storing a control value in a local node and an upstream node and / or a downstream node; The routing tuple is sent to multiple nodes on a session path, so that the nodes receiving the routing tuple can perform inter-node routing according to the routing tuple.
2. The method according to claim 1, wherein: The control value storage space information includes: Local tail value storage space (local_rx_tail), used to indicate the storage space of the tail value of the upstream node in the local node; Local head value storage space (local_tx_head), used to indicate the storage space of the head value fed back by the downstream node in the local node; Upstream head value storage space (remote_rx_head), used to indicate the storage space of the head value fed back by the local node in the upstream node; and / or The downstream tail value storage space (remote_tx_tail) is used to indicate the storage space of the tail value sent by the local node in the downstream node.
3. The method according to claim 1 or 2, wherein: The routing tuple further includes a priority, which is used to indicate the priority of the session.
4. The method according to claim 3, wherein: The higher the priority of a session, the higher the bandwidth allocated to the session.
5. The method according to any one of claims 1 to 4, wherein: The routing tuple further includes storage space size information, which is used to indicate the maximum size of the storage space.
6. The method according to any one of claims 1 to 5, further comprising, before constructing the routing tuple, establishing a session path, wherein establishing the session path comprises: Search the topology of multiple nodes; Determine whether any two nodes are reachable from each other according to the topological structure; If two nodes are reachable, a session path is established through the nodes between the two reachable nodes.
7. The method according to any one of claims 1 to 6, wherein: The session path is the shortest session path, the idlest session path or the highest bandwidth session path.
8. The method according to any one of claims 1 to 7, wherein: The session path is unidirectional.
9. The method according to any one of claims 1 to 8, wherein: The session path is session independent.
10. The method according to any one of claims 1 to 9, wherein: The method is executed by a resident agent on a host, and the resident agent sends a routing tuple to a node on a corresponding session path in response to a request from a client.
11. The method according to any one of claims 1 to 9, wherein: The method is performed by a resident agent and a client on a host, wherein the client sends a routing tuple to a sending node and a receiving node on a corresponding session path, and the resident agent sends a routing tuple to other nodes on the corresponding session path in response to a request from the client.
12. The method according to any one of claims 1 to 11, further comprising: Sending a resident kernel to multiple nodes so that the intermediate nodes on the session path run the routing tuple through the resident kernel; And / or, a resident Kernel is sent to multiple nodes, so that the multiple nodes scan the session list through the resident Kernel.
13. The method according to claim 12, further comprising: A temporary sending Kernel is sent to the starting node; and a temporary receiving Kernel is sent to the receiving node.
14. A method for performing inter-node routing based on multiple nodes, comprising: Receive a routing tuple, the routing tuple including a session identifier, local data storage space information, remote data storage space information, and control value storage space information; wherein the session identifier is used to distinguish a session, the local data storage space information is used to indicate an address of a storage space for storing data in a local node, the remote storage space information is used to indicate an address of a storage space for storing data in a downstream node, and the control value storage space information is used to indicate an address of a storage space for storing a control value in a local node and an upstream node and / or a downstream node; Inter-node routing is performed on the session path according to the routing tuple.
15. The method according to claim 14, wherein: The control value storage space information includes: Local tail value storage space (local_rx_tail), used to indicate the storage space of the tail value of the upstream node in the local node; Local head value storage space (local_tx_head), used to indicate the storage space of the head value fed back by the downstream node in the local node; Upstream head value storage space (remote_rx_head), used to indicate the storage space of the head value fed back by the local node in the upstream node; and / or The downstream tail value storage space (remote_tx_tail) is used to indicate the storage space of the tail value sent by the local node in the downstream node.
16. The method according to claim 15, wherein: The amount of data sent to the downstream node is determined according to the difference between the local tail value in the local tail value storage space and the upstream head value in the upstream head value storage space.
17. The method according to any one of claims 14 to 16, wherein: The routing tuple further includes a priority, which is used to indicate the priority of the session.
18. The method according to claim 17, comprising: Splitting the data involved in the session so that the data is split into data fragments; In each cycle, data slices of sessions with different priorities are sent, wherein the higher the priority of a session, the more copies of the corresponding data slices are sent in each cycle, so that the higher the priority of a session, the higher the bandwidth allocated to the session.
19. The method according to claim 18, wherein: The routing tuple further includes storage space size information for indicating a maximum size of the storage space, wherein the maximum number of data slices sent to the downstream node is an integer obtained by dividing the maximum size of the storage space by the size of each data slice.
20. The method according to any one of claims 14 to 19, wherein: The session path is the shortest session path, the idlest session path or the highest bandwidth session path.
21. The method according to any one of claims 14 to 20, wherein: The session path is unidirectional.
22. The method according to any one of claims 14 to 21, wherein: The session path is session independent.
23. The method according to any one of claims 14 to 22, wherein: The routing tuple is run by a resident kernel running at an intermediate node of the session path; and / or a session list is scanned by a resident kernel running at a plurality of nodes of the session path.
24. The method according to claim 23, further comprising: The starting node on the session path receives a temporary sending Kernel; and the receiving node on the session path receives a temporary receiving Kernel.
25. A method for performing inter-node routing based on multiple nodes, comprising: On the host side, Constructing a routing tuple, the routing tuple comprising a session identifier, local data storage space information, remote data storage space information, and control value storage space information; wherein the session identifier is used to distinguish a session, the local data storage space information is used to indicate an address of a storage space for storing data in a local node, the remote storage space information is used to indicate an address of a storage space for storing data in a downstream node, and the control value storage space information is used to indicate an address of a storage space for storing a control value in a local node and an upstream node and / or a downstream node; sending the routing tuple to a plurality of nodes on a session path; and On the device side including multiple nodes, receiving the routing tuple; The node receiving the routing tuple performs inter-node routing according to the routing tuple.
26. A host for performing inter-node routing based on multiple nodes, on which a resident agent program runs, wherein the resident agent program is used to execute the method according to any one of claims 1 to 13.
27. A device for performing inter-node routing based on multiple nodes, the device comprising multiple nodes, each node running a Kernel, the Kernel being used to execute the method according to any one of claims 14 to 24.
28. A system for performing inter-node routing based on multiple nodes, the system comprising the host according to claim 26 and the device according to claim 27.
29. A computer-readable storage medium comprising computer-executable instructions, which, when executed by one or more processors, perform the method according to any one of claims 1 to 24.
