Data transmission method and computing node

Abstract

The embodiment of the invention provides a data transmission method and a computing node. The method is applied to a first computing node of a computer cluster, the computer cluster comprises a plurality of computing nodes, a storage node and computing interconnection CXL equipment, the plurality of computing nodes are connected with the storage node through the CXL equipment, and the plurality of computing nodes are further connected through Internet links. The method comprises the steps that a first computing node stores target data in a target storage position of a target storage node through CXL equipment, the first computing node is any computing node in a plurality of computing nodes, and the storage nodes comprise the target storage node; the first computing node sends a notification message to the second computing node through the internet link, the notification message comprises the node information of the target storage node and the target storage position, and the second computing node is any computing node except the first computing node in the plurality of computing nodes. Through the method, the data transmission efficiency in the computer cluster can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of server technology, and in particular to a data transmission method and a computing node. Background Art

[0002] A computer cluster is a computer system consisting of a group of computing nodes. The computing nodes in a computer cluster can transmit data to each other. For example, a computer cluster can be a training cluster for artificial intelligence (AI).

[0003] Currently, data can be transmitted between different computing nodes in a computer cluster via the Internet. However, when the data to be transmitted is large, insufficient bandwidth resources on the Internet will affect the data transmission efficiency, resulting in low data transmission efficiency and large transmission delay between computing nodes, leading to low data transmission efficiency in the computer cluster. Summary of the invention

[0004] The embodiments of the present application provide a data transmission method and a computing node, which can improve the data transmission efficiency in a computer cluster.

[0005] In a first aspect, an embodiment of the present application provides a data transmission method, which is applied to a first computing node of a computer cluster, wherein the computer cluster includes multiple computing nodes, a storage node, and a CXL device, wherein the multiple computing nodes are connected to the storage node via the CXL device, and the multiple computing nodes are also connected via an Internet link, wherein the method includes:

[0006] The first computing node stores the target data in the target storage location of the target storage node through the CXL device, wherein the first computing node is any computing node among the plurality of computing nodes, and the storage nodes include the target storage node;

[0007] The first computing node sends a notification message to the second computing node through the Internet link, the notification message including the node information of the target storage node and the target storage location, the notification message being used to inform the second computing node that the target data is stored in the target storage location of the target storage node, wherein the second computing node is any computing node among the multiple computing nodes except the first computing node.

[0008] In the above scheme, the first computing node can store the target data in the target storage location of the target storage node through the CXL device, which can realize the rapid transmission of the target data, and can send a notification message to the second computing node through the Internet link to inform the second computing node of the storage location of the target data, so that the second computing device can quickly obtain the target data from the target storage location as needed. Through the above method, the decoupling of the target data and the notification message is realized, and the data transmission efficiency in the computer cluster is improved.

[0009] In a possible implementation, the CXL device is a multi-host CXL device, and the plurality of computing nodes and the storage node are respectively connected to the multi-host CXL device;

[0010] The first computing node stores the target data in the target storage location of the target storage node through the CXL device, including:

[0011] The first computing node sends a first storage request to the multi-host CXL device, the first storage request including node information of the target storage node, the target storage location, and the target data;

[0012] The first storage request is used to request the multi-host CXL device to store the target data in the target storage location of the target storage node.

[0013] In the above solution, the CXL device may be a multi-host CXL device, and the first computing node may store the target data through the multi-host CXL device, thereby improving data storage efficiency.

[0014] In a possible implementation, the CXL device is a CXL switching device, and the plurality of computing nodes are connected to the storage node via the CXL switching device;

[0015] The first computing node stores the target data in the target storage location of the target storage node through the CXL device, including:

[0016] The first computing node sends a second storage request to the CXL switch device, where the second storage request includes node information of the target storage node, the target storage location, and the target data;

[0017] The second storage request is used to instruct the CXL switch device to store the target data in the target storage location of the target storage node.

[0018] In the above solution, the CXL device may be a CXL switching device, and the first computing node may store the target data through the CXL switching device, thereby improving data storage efficiency.

[0019] In a possible implementation, the method further includes, by the first computing node, determining a target storage node and a target storage location in the target storage node.

[0020] In the above solution, the first computing node can determine the target storage location in the storage node, thereby achieving the purpose of determining the target storage location.

[0021] In a possible implementation, the first computing node determines a target storage node and a target storage location in the target storage node, including:

[0022] The first computing node acquires storage configuration information corresponding to the first computing node, the storage configuration information including node information of a candidate storage node corresponding to the first computing node and a candidate storage area in the candidate storage node;

[0023] The first computing node determines the target storage node among the candidate storage nodes based on the data size of the target data and the storage configuration information, and determines the target storage location in the candidate storage area of ​​the target storage node, wherein the remaining storage space of the candidate storage area of ​​the target storage node is greater than the data size of the target data.

[0024] In the above scheme, the first computing node can determine the target storage node among the candidate storage nodes, and determine the target storage location in the candidate storage area of ​​the target storage node, thereby avoiding the first computing node determining the target storage node and the target storage location among all storage nodes, so that the first computing node determines the target storage node and the target storage location faster, further improving the storage speed of the first computing node.

[0025] In a second aspect, an embodiment of the present application provides a data transmission method, which is applied to a second computing node of a computer cluster, wherein the computer cluster includes multiple computing nodes, a storage node, and a CXL device, wherein the multiple computing nodes are connected to the storage node via the CXL device, and the multiple computing nodes are also connected via an Internet link, wherein the method includes:

[0026] The second computing node obtains a notification message from the first computing node through the Internet link, the notification message including node information and a target storage location of a target storage node, the multiple computing nodes including the second computing node and the first computing node, and the storage nodes including the target storage node;

[0027] The second computing node reads the target data from the target storage location of the target storage node through the CXL device according to the notification message.

[0028] In the above scheme, the second computing node can receive the notification message sent by the first computing node through the Internet link, and can quickly read the target data at the target storage location of the target storage node through the CXL device according to the notification message. Through the above method, the decoupling of the target data and the notification message is achieved, and the data transmission efficiency in the computer cluster is improved.

[0029] In a possible implementation, the method further includes:

[0030] The second computing node sends a response message to the first computing node, where the response message includes a read result of the target data, where the read result is a read success or a read failure.

[0031] In the above solution, the second computing node can send the read result of the target data to the first computing node, thereby enabling the first computing node to obtain the read result of the target data.

[0032] In one possible implementation,

[0033] The CXL device is a multi-host CXL device, and the plurality of computing nodes and the storage node are respectively connected to the multi-host CXL device; or,

[0034] The CXL device is a CXL switching device, and the plurality of computing nodes are connected to the storage node via the CXL switching device.

[0035] In the above solution, data transmission can be performed through multiple host CXL devices / CXL switching devices, thereby improving data transmission efficiency.

[0036] In a third aspect, an embodiment of the present application provides a computer cluster, comprising a plurality of computing nodes, a storage node, and a CXL device, wherein the computing nodes are used to execute the method described in any one of the first aspect or the second aspect.

[0037] In the above scheme, computing nodes can transmit target data through CXL devices and send notification messages through Internet links, which decouples target data and notification messages, improves the reading and writing efficiency of target data, and improves the data transmission efficiency in computer clusters.

[0038] In a possible implementation, the CXL device is a multi-host CXL device, and the plurality of computing nodes and the storage node are respectively connected to the multi-host CXL device; or,

[0039] The CXL device is a CXL switching device, and the plurality of computing nodes are connected to the storage node via the CXL switching device.

[0040] In the above solution, data can be transmitted between computing nodes in a computer cluster through a multi-host CXL device / CXL switching device, thereby improving data transmission efficiency.

[0041] In a fourth aspect, an embodiment of the present application provides a data transmission device, which is applied to a computer cluster, wherein the computer cluster includes a plurality of computing nodes, a storage node, and a CXL device, wherein the plurality of computing nodes are connected to the storage node via the CXL device, and the plurality of computing nodes are also connected via an Internet link, and any one of the plurality of computing nodes may include the data transmission device, and the data transmission device may include a processing module and a sending module, wherein:

[0042] The processing module is used for storing the target data in the target storage location of the target storage node through the CXL device;

[0043] The sending module is used to send a notification message to the second computing node through the Internet link, the notification message including the node information of the target storage node and the target storage location, the notification message is used to inform the second computing node that the target data is stored in the target storage location of the target storage node, wherein the second computing node is any computing node among the multiple computing nodes except the computing node to which the data transmission device belongs.

[0044] In the above solution, the data transmission device can store the target data in the target storage location of the target storage node through the CXL device, and can send a notification message to the second computing node through the Internet link to inform the second computing node of the storage location of the target data. Through the above solution, the decoupling of the target data and the notification message is achieved, and the data transmission efficiency in the computer cluster is improved.

[0045] In a possible implementation, the CXL device is a multi-host CXL device, the plurality of computing nodes and the storage node are respectively connected to the multi-host CXL device; the processing module is specifically configured to:

[0046] sending a first storage request to the multi-host CXL device, the first storage request including node information of the target storage node, the target storage location, and the target data;

[0047] The first storage request is used to request the multi-host CXL device to store the target data in the target storage location of the target storage node.

[0048] In the above solution, the CXL device may be a multi-host CXL device, and the storage module may store the target data through the multi-host CXL device, thereby improving data storage efficiency.

[0049] In a possible implementation, the CXL device is a CXL switching device, and the plurality of computing nodes are connected to the storage node via the CXL switching device; and the processing module is specifically configured to:

[0050] sending a second storage request to the CXL switch device, wherein the second storage request includes node information of the target storage node, the target storage location, and the target data;

[0051] The second storage request is used to instruct the CXL switch device to store the target data in the target storage location of the target storage node.

[0052] In the above solution, the CXL device may be a CXL switching device, and the storage module may store the target data through the CXL switching device, thereby improving data storage efficiency.

[0053] In a possible implementation, the processing module is further configured to:

[0054] A target storage node and a target storage location in the target storage node are determined.

[0055] In the above solution, the processing module can determine the target storage location in the storage node, thereby achieving the purpose of determining the target storage location.

[0056] In a possible implementation, the processing module is specifically configured to:

[0057] Acquire storage configuration information corresponding to the first computing node, the storage configuration information including node information of a candidate storage node corresponding to the first computing node and a candidate storage area in the candidate storage node;

[0058] According to the data size of the target data and the storage configuration information, the target storage node is determined among the candidate storage nodes, and the target storage location is determined in the candidate storage area of ​​the target storage node, and the remaining storage space of the candidate storage area of ​​the target storage node is greater than or equal to the data size of the target data.

[0059] In the above scheme, the processing module can determine the target storage node in at least one candidate storage node, and determine the target storage location in the candidate storage area of ​​the target storage node, thereby avoiding determining the target storage node and the target storage location in all storage nodes, so that the processing module determines the target storage node and the target storage location faster, further improving the data storage speed.

[0060] In a fifth aspect, an embodiment of the present application provides a data transmission device, which is applied to a computer cluster, wherein the computer cluster includes a plurality of computing nodes, a storage node, and a CXL device, wherein the plurality of computing nodes are connected to the storage node via the CXL device, and the plurality of computing nodes are also connected via an Internet link, and any one of the plurality of computing nodes may include the data transmission device, and the data transmission device may include an acquisition module, a processing module, and a sending module, wherein:

[0061] The acquisition module is used to acquire a notification message from a first computing node through the Internet link, the notification message including node information and a target storage location of a target storage node, the multiple computing nodes including the first computing node, and the storage nodes including the target storage node;

[0062] The processing module is configured to read the target data from the target storage location of the target storage node through the CXL device according to the notification message.

[0063] The sending module is used to send a response message to the first computing node through the Internet link, and the response message includes a reading result of the target data, and the reading result is a reading success or a reading failure.

[0064] In the above scheme, the data transmission device can receive the notification message sent by the first computing node, and can read the target data at the target storage location of the target storage node according to the notification message. Through the above scheme, the decoupling of the target data and the notification message is achieved, and the data transmission efficiency in the computer cluster is improved.

[0065] In one possible implementation,

[0066] The CXL device is a multi-host CXL device, and the plurality of computing nodes and the storage node are respectively connected to the multi-host CXL device; or,

[0067] The CXL device is a CXL switching device, and the plurality of computing nodes are connected to the storage nodes via the CXL switching device.

[0068] In the above solution, data transmission can be performed through multiple host CXL devices / CXL switching devices, thereby improving data transmission efficiency.

[0069] In a sixth aspect, an embodiment of the present application provides a computing node, comprising: a processor, and a memory communicatively connected to the processor;

[0070] The memory stores computer-executable instructions;

[0071] The processor executes the computer-executable instructions stored in the memory to implement the method described in any one of the first aspect or the second aspect.

[0072] In the above scheme, computing nodes can transmit target data through CXL devices and send notification messages through Internet links, which decouples target data and notification messages, improves the reading and writing efficiency of target data, and improves the data transmission efficiency in computer clusters.

[0073] In a possible implementation, a network card is further included, and the processor is communicatively connected to the network card;

[0074] The processor is communicatively connected to an Internet link via the network card.

[0075] In the above solution, the computing node can be connected to the Internet link through the network card, thereby achieving the purpose of connecting the computing node to the Internet link.

[0076] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method as described in any one of the first aspect or the second aspect.

[0077] In the above scheme, computing nodes can transmit target data through CXL devices and send notification messages through Internet links, which decouples target data and notification messages, improves the reading and writing efficiency of target data, and improves the data transmission efficiency in computer clusters.

[0078] In an eighth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a computer, implements the method described in any one of the first aspect or the second aspect.

[0079] In the above scheme, computing nodes can transmit target data through CXL devices and send notification messages through Internet links, which decouples target data and notification messages, improves the reading and writing efficiency of target data, and improves the data transmission efficiency in computer clusters. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0081] Figure 1A schematic diagram of a cluster architecture provided for an embodiment of the present application;

[0082] Figure 2 A schematic diagram of connections between computing nodes in a cluster provided in an embodiment of the present application;

[0083] Figure 3 A schematic diagram of the architecture of a computer cluster provided in an embodiment of the present application;

[0084] Figure 4 A schematic diagram of the speed of reading and writing data through a CXL device provided in an embodiment of the present application;

[0085] Figure 5 A flowchart of a data transmission method provided in an embodiment of the present application;

[0086] Figure 6 A schematic diagram of the structure of a computer cluster provided in an embodiment of the present application;

[0087] Figure 7 A flowchart of another data transmission method provided in an embodiment of the present application;

[0088] Figure 8 A schematic diagram of a data transmission process provided in an embodiment of the present application;

[0089] Fig. 9 A schematic diagram of the structure of another computer cluster provided in an embodiment of the present application;

[0090] Fig.10 A flowchart of another data transmission method provided in an embodiment of the present application;

[0091] Fig.11 Another data transmission process diagram provided in an embodiment of the present application;

[0092] Fig.12 A schematic diagram of the structure of a data transmission device provided in an embodiment of the present application;

[0093] Fig.13 A schematic diagram of the structure of another data transmission device provided in an embodiment of the present application;

[0094] Fig.14 A schematic diagram of the structure of a computing node provided in an embodiment of the present application. DETAILED DESCRIPTION

[0095] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0096] To facilitate understanding, first combine Figure 1 , taking the AI ​​training cluster as an example, the application scenarios involved in the embodiments of the present application are explained.

[0097] Figure 1 A schematic diagram of a cluster architecture provided in an embodiment of the present application. Figure 1 As shown, the cluster can be, for example, an AI training cluster, or, of course, other functional clusters. The following description takes the cluster as an AI training cluster as an example. The AI ​​training cluster 100 can include multiple computing nodes, and the multiple computing nodes can be, for example, computing node A101, computing node B102, computing node C103, computing node D104, and computing node E105.

[0098] The computing node A 101 , the computing node B 102 , the computing node C 103 , the computing node D 104 , and the computing node E 105 may be connected to each other via wired or wireless means.

[0099] Any computing node in the AI ​​training cluster 100 can exchange information with the remaining computing nodes. For example, computing node A101 can exchange information with computing node B102, computing node C103, computing node D104, or computing node E105. The information can be data or messages.

[0100] The data may be the job data that needs to be transmitted between computing nodes. For example, in an AI training cluster, the data may be the training data transmitted between computing nodes.

[0101] The message may be a message corresponding to the data, and the message may describe the storage address of the data, etc. For example, the data is training data transmitted between computing nodes, and the message may include the storage address of the data, etc. The message may be, for example, a notification message or a response message.

[0102] The computing node may be a computing device that can perform data read and write operations. For example, the computing node may be a server, a laptop computer, a desktop computer, a tablet computer, a single-chip microcomputer, a multimedia player, an artificial intelligence device, a wearable device, an Internet of Things device, or a virtual reality / augmented reality / mixed reality device.

[0103] Currently, data and messages between computing nodes in a cluster are transmitted through the Internet. Figure 2 As shown, Figure 2 A schematic diagram of the connection between computing nodes in a cluster provided in an embodiment of the present application. Figure 2 , the AI ​​training cluster 100 may include computing node A 101 , computing node B 102 , computing node C 103 , computing node D 104 , computing node E 105 , and an Internet link 106 .

[0104] It should be noted that the Internet link 106 may be in wired or wireless form. Figure 2 The Internet link 106 is only described as a wired form, which does not constitute a limitation to the embodiments of the present application.

[0105] Computing node A101, computing node B102, computing node C103, computing node D104, and computing node E105 can all be connected to Internet link 106. In other words, computing node A101, computing node B102, computing node C103, computing node D104, and computing node E105 can be connected via Internet link 106. Data can be transmitted between computing nodes via Internet link 106, and messages can be transmitted via Internet link 106.

[0106] Illustratively, computing node A101 may transmit data to computing node B102, computing node C103, computing node D104, or computing node E105 via Internet link 106, and computing node A101 may also transmit messages to computing node B102, computing node C103, computing node D104, or computing node E105 via Internet link 106.

[0107] However, during the above information interaction process, if the data transmitted between computing nodes is large, insufficient bandwidth resources of the Internet will affect the data transmission efficiency, resulting in low data transmission efficiency and large transmission delay between computing nodes, leading to low data transmission efficiency in the computer cluster.

[0108] In view of this, an embodiment of the present application provides a computer cluster and a data transmission method applied to the computer cluster to solve the problems of low data transmission efficiency and large transmission delay between computing nodes.

[0109] like Figure 3 As shown, Figure 3 A schematic diagram of the architecture of a computer cluster provided in an embodiment of the present application. Figure 3The computer cluster may include multiple computing nodes, storage nodes, CXL links, and Internet links. The multiple computing nodes may be computing node 1, computing node 2, computing node 3, ..., and computing node N. The number of storage nodes may be one or more. The multiple may be two or more. Figure 3 Taking multiple storage nodes as an example for description, the multiple storage nodes may be storage node 1, storage node 2, ... and storage node M. Among them, M and N may be the same, or M and N may be different, which is not limited in the embodiment of the present application.

[0110] Multiple computing nodes and multiple storage nodes can be connected to the CXL link, that is, multiple computing nodes and multiple storage nodes can be connected through the CXL link. Multiple computing nodes can also be connected through an Internet link.

[0111] The computer cluster may be a computer cluster that needs to perform large data transmission. For example, the computer cluster may be an AI training cluster, a distributed database, or other types of clusters.

[0112] The computing node may be a computing device that can perform data read and write operations. For example, the computing node includes but is not limited to a server, a laptop computer, a desktop computer, a tablet computer, a single-chip microcomputer, a multimedia player, an artificial intelligence device, a wearable device, an Internet of Things device, or a virtual reality / augmented reality / mixed reality device, a mobile phone, and other computing devices.

[0113] The storage node may be a device with storage function. For example, the storage node may include dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), solid state disk (SSD), and persistent memory (PMEM). The storage node may also be a network card, a graphics processing unit (GPU) card, a field programmable gate array (FPGA) card, or other devices with storage function and supporting CXL technology.

[0114] The CXL link may include at least one CXL device, which may be a multi-host CXL device or a CXL switch device.

[0115] In some embodiments, when the CXL device may be a multi-host CXL device, the storage node may include a DRAM, SDRAM, SSD, PMEM, or a Dual Inline Memory Modules (DIMM) memory module.

[0116] When the CXL device can be a CXL switching device, the storage node can include a controller and a memory, wherein the controller can include a memory expansion (CXL memory expander) chip, a CXL memory controller (CXL memory expandercontroller), or a CXL memory pooling (CXL memory expander pooling) chip, and the memory can include DRAM, SDRAM, SSD, PMEM or DIMM memory stick. It can be understood that the present application does not limit the specific structure of the storage node, as long as the computing node can use the storage space of the storage node through the CXL device.

[0117] The multi-host CXL device may include multiple compute node interfaces and one or more storage node interfaces. For any one of the compute node interfaces, the multi-host CXL device may be connected to a compute node via the compute node interface. For any one of the storage node interfaces, the multi-host CXL device may be connected to a storage node via the storage node interface.

[0118] The multi-host CXL device may include, for example, a multi-host CXL memory expander chip, a multi-host CXL memory expander controller, or a CXL memory expander pooling chip.

[0119] The CXL switch device may include one or more computing node interfaces and / or one or more storage node interfaces. For any computing node interface, the multi-host CXL device may be connected to a computing node via the computing node interface. For any storage node interface, the multi-host CXL device may be connected to a storage node via the storage node interface.

[0120] Optionally, the CXL switching device may further include one or more device interfaces. For any device interface, the CXL switching device may be connected to another CXL switching device via the device interface.

[0121] The CXL switching device may be, for example, a CXL switch, which may include a CXL switch chip.

[0122] The Internet link may include a network card connection with a computing node, a line and a connection device for connecting the computing node to the Internet.

[0123] In the above computer cluster, the CXL device reads and writes data at a relatively fast speed. Figure 4 As shown, Figure 4 A schematic diagram of the speed of reading and writing data through a CXL device provided in an embodiment of the present application. Figure 4 , Figure 4 The horizontal axis represents the operation mode of the data, which can be read and write; Figure 4 The vertical axis represents the speed of operations on data.

[0124] like Figure 4 As shown in Figure 1, the speed of reading data through CXL devices can reach 23.8GB / S, and the speed of writing data can reach 12GB / S. Currently, the speed of reading and writing data through the Internet is about 3GB / S. It can be seen that the speed of reading and writing data through CXL devices is much higher than the speed of reading and writing data through the Internet.

[0125] It should be noted that Figure 4 The minimum read and write speeds that can be achieved when reading and writing data through the CXL device are only exemplified. In actual implementation, the speed of reading data and the speed of writing data of the CXL device may be faster when the type of the CXL device is different. Figure 4 It does not limit the speed at which the CXL device can read or write data.

[0126] In the data transmission method for the computing nodes of the above-mentioned computer cluster provided in the embodiment of the present application, the computing nodes in the computer cluster can store the data to be transmitted to another computing node in the CXL device and transmit the message corresponding to the data through the Internet link. Since the size of the data is large, and the size of the message corresponding to the data is generally small. Therefore, in the data transmission method provided in the embodiment of the present application, data can be quickly transmitted through the CXL device, avoiding the limitation of the Internet bandwidth on data transmission, making the data transmission rate faster, improving the data transmission efficiency in the computer cluster, and can further improve the working efficiency of the computer cluster. For example, taking the computer cluster as an AI training cluster as an example, when the data transmission efficiency of the AI ​​training cluster is improved, the training speed of the AI ​​training cluster can also be improved.

[0127] The technical solutions of the embodiments of the present application are described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0128] Figure 5 A flowchart of a data transmission method provided in an embodiment of the present application. Figure 5 , the method may include:

[0129] S501 . A first computing node stores target data in a target storage location of a target storage node through a CXL device.

[0130] The data transmission method provided in this embodiment can be applied to Figure 3 The computer cluster shown is a computing node of the computer cluster, and the computer cluster includes multiple computing nodes, storage nodes, and a CXL link. The CXL link may include one or more CXL devices.

[0131] Multiple computing nodes are connected to storage nodes via CXL devices, and multiple computing nodes are also connected via Internet links.

[0132] The first computing node may be any computing node in the computer cluster.

[0133] In this embodiment, the number of storage nodes may be one or more, wherein a plurality means two or more than two.

[0134] In a possible implementation, the number of the storage node may be one.

[0135] In this method, a storage node in the computer cluster is the target storage node, and the target storage location is the storage location of the target storage node.

[0136] In another possible implementation, there may be multiple storage nodes.

[0137] In this manner, the target storage node may be one or more (two or more) storage nodes among the multiple storage nodes.

[0138] If the target storage node is one storage node among the multiple storage nodes, the target storage location is the storage location of the one target storage node.

[0139] Exemplarily, it is assumed that the number of storage nodes is 3, and the 3 storage nodes are storage node 1, storage node 2, and storage node 3. It is assumed that the target storage node is storage node 2, and the target storage location is the storage location of storage node 2.

[0140] If the target storage node is a plurality of storage nodes among the plurality of storage nodes, the target storage location may be a storage location of one target storage node, or the target storage location may also be a storage location of a plurality of (two or more) target storage nodes.

[0141] Exemplarily, it is assumed that the number of storage nodes is 3, and the 3 storage nodes are storage node 1, storage node 2, and storage node 3. It is assumed that the target storage nodes are storage node 2 and storage node 3, then the target storage location may be the storage location of storage node 2, or the target storage location may also be the storage location of storage node 3, or the target storage location may also be the storage location of storage node 2 and the storage location of storage node 3.

[0142] In this embodiment, the first computing node may use a storage instruction (Store) to store the target data in the target storage location.

[0143] Optionally, before storing the target data in the target storage location of the target storage node, the first computing node may also determine a target storage node and a target storage location in the target storage node in the storage node.

[0144] The first computing node may store the target data in the target storage location of the target storage node. Thus, the first computing node may determine the target storage node and the target storage location according to the data size of the target data. The first computing node determines the target storage node and determines the target storage location in the target storage node in at least the following two ways:

[0145] In a possible implementation, the first computing node may determine a target storage node and a target storage position in the target storage node from among a plurality of storage nodes according to the data size of the target data.

[0146] Specifically, the first computing node may determine an available storage node among all storage nodes in the computer cluster, and determine a target storage node among the available storage nodes. The available storage node may be a storage node whose remaining storage space is greater than or equal to the data size of the target data. The storage space of the target storage location is equal to the data size of the target data.

[0147] Exemplarily, assuming that the computer cluster includes three storage nodes, and only one storage node has a remaining storage space greater than or equal to the data size of the target data, then the storage node may be an available storage node, and the first computing node may determine the storage node as the target storage node.

[0148] It should be noted that, if there are multiple available storage nodes, the first computing node may determine a storage node as a target storage node from the multiple available storage nodes according to a first preset method.

[0149] The first preset method can be set according to actual needs, and this embodiment does not limit this. For example, the first preset method can be to determine the storage node with the smallest remaining storage space among multiple available storage nodes as the target storage node, so that the size of the remaining storage space of the target storage node is closest to the size of the target data, and the available storage nodes can be reasonably used to avoid wasting the available storage nodes with larger remaining storage space; or, determine the storage node closest to the first computing node among multiple available storage nodes as the target storage node, so that the storage path of the target data can be the shortest and the storage speed can be the fastest.

[0150] After determining the target storage node, the first computing node may determine the target storage location in the remaining storage space of the target storage node according to the method for determining the target storage location.

[0151] It should be noted that the method for determining the target storage location can be set according to actual needs, and this embodiment does not limit this. For example, the first computing node can determine the continuous storage space starting from the minimum storage address in the remaining storage space and having a storage space size equal to the data size of the target data as the target storage location.

[0152] In the above manner, any available storage node in the computer cluster can be used as a target storage node, so that the utilization rate of the storage nodes in the computer cluster is high.

[0153] In another possible implementation, the first computing node obtains storage configuration information corresponding to the first computing node, the storage configuration information including node information of a candidate storage node corresponding to the first computing node, and a candidate storage area in the candidate storage node; the first computing node determines a target storage node among the candidate storage nodes and a target storage location in the candidate storage area of ​​the target storage node based on the data size of the target data and the storage configuration information, and the remaining storage space of the candidate storage area of ​​the target storage node is greater than or equal to the data size of the target data.

[0154] The storage area to be selected in the storage node to be selected may be a storage area allocated to the first computing node among the plurality of storage nodes. The first computing node may store data in the storage area to be selected in the storage node to be selected.

[0155] The node information of the storage node to be selected may include: the node identifier of the storage node to be selected, etc.

[0156] It should be noted that, for any candidate storage node, the candidate storage area in the candidate storage node may be the storage area allocated to the first computing node by the computer cluster during the initial configuration process; or, the candidate storage area in the candidate storage node may be the storage area allocated to the first computing node by the computer cluster when the first computing node stores data in the storage node for the first time.

[0157] In this manner, the first computing node may determine, as the target storage node, a candidate storage node whose remaining candidate storage space is greater than or equal to the data size of the target data.

[0158] It should be noted that if there are multiple candidate storage nodes whose remaining candidate storage space is greater than or equal to the data size of the target data, the first computing node can determine a candidate storage node as the target storage node from multiple qualified candidate storage nodes according to the second preset method.

[0159] The second preset method can be set according to actual needs, and this embodiment does not limit this. For example, the second preset method can be to determine the storage node with the smallest remaining storage space among multiple qualified storage nodes to be selected as the target storage node; or to determine the storage node closest to the first computing node among multiple qualified storage nodes to be selected as the target storage node, etc.

[0160] After determining the target storage node, the first computing node may determine the target storage location in the remaining storage space to be selected of the target storage node according to the method for determining the target storage location.

[0161] It should be noted that the method for determining the target storage location can be set according to actual needs, and this embodiment does not limit this. For example, the first computing node can determine the continuous storage space starting from the smallest storage address in the remaining storage space to be selected and whose storage space size is equal to the data size of the target data as the target storage location.

[0162] In the above method, a corresponding storage area can be allocated to the first computing node, and the first computing node can determine the target storage node and the target storage location in the corresponding storage area, avoiding the first computing node from determining the target storage node and the target storage location among all storage nodes, so that the first computing node determines the target storage node and the target storage location faster, further improving the storage speed of the first computing node.

[0163] S502: The first computing node sends a notification message to the second computing node via an Internet link.

[0164] Correspondingly, the second computing node obtains the notification message from the first computing node via the Internet link.

[0165] The notification message is used to inform the second computing node that the target data is stored in the target storage location of the target storage node.

[0166] The second computing node may be any computing node in the computer cluster except the first computing node.

[0167] It should be noted that the number of the second computing nodes may be one or more, wherein the number may be two or more.

[0168] In this embodiment, in at least the following two cases, the first computing node stores the target data in the target storage location of the target storage node and sends a notification message to the second computing node:

[0169] Case 1: The second computing node requests to obtain target data.

[0170] In this case, the second computing node may send a request message to the first computing node. The request message is used to request to obtain the target data. After receiving the request message, the first computing node may store the target data in the target storage location of the target storage node through the CXL device according to the request message, and may inform the second computing node that the target data is stored in the target storage location of the target storage node through a notification message.

[0171] In this case, the number of the second computing node may be one.

[0172] Case 2: The first computing node actively stores the target data.

[0173] In this case, data can be shared between the first computing node and the second computing node. If the first computing node needs to update all or part of the shared data, the updated shared data (target data) can be stored in the target storage location of the target storage node, and the second computing node can be informed through a notification message that the target data is stored in the target storage location of the target storage node.

[0174] In this case, the number of the second computing nodes may be one or more.

[0175] The notification message includes node information of the target storage node and the target storage location.

[0176] The node information of the target storage node may include: a node identifier of the target storage node, etc.

[0177] Optionally, the node information of the target storage node may also include the location of the target storage node in the computer cluster, etc. In this way, if the target storage node fails, the first computing node, the second computing node, or other computing nodes in the computer cluster can quickly determine the target storage node in the computer cluster based on the location of the target storage node in the computer cluster.

[0178] The position of the target storage node in the computer cluster may be a connection position of the target storage node in the CXL link.

[0179] The target storage location may be a storage space represented by a start storage address and an end storage address; or, the target storage location may be a storage space represented by a start storage address and a data size of target data.

[0180] In a possible implementation, the content of the notification message may be as shown in Table 1-1:

[0181] Table 1-1

[0182]

[0183] In another possible implementation, the notification message may further include more content, and the content included in the notification message may be as shown in Table 1-2:

[0184] Table 1-2

[0185]

[0186]

[0187] As shown in Table 1-2, based on Table 1-1, the notification message may also include one or more of the location of the target storage node in the computer cluster, the node identifier of the source computing node, the node identifier of the target computing node, the timeout configuration, the timestamp or the message type.

[0188] The node identifier of the source computing node may be the node identifier of the notification message sender, that is, the node identifier of the source computing node may be the node identifier of the first computing node. The node identifier of the source computing node may be used to indicate the source of the message.

[0189] The node identifier of the target computing node may be the node identifier of the notification message recipient, that is, the node identifier of the target computing node may be the node identifier of the second computing node. The node identifier of the target computing node may be used to indicate the computing node receiving the message.

[0190] The timestamp may be the time at which the first computing node sends the notification message.

[0191] The timeout configuration can be used to indicate the waiting time when reading the target data corresponding to the notification message.

[0192] It should be noted that when the second computing node reads the target data corresponding to the notification message according to the notification message, if the target data is not successfully read within the time indicated by the timeout configuration, the second computing node can stop reading the target data and return a response message of read failure to the first computing node.

[0193] Exemplarily, assuming that the waiting time indicated by the timeout configuration is 10s, if the second computing node successfully reads the target data within 10s, it can return a response message of successful reading to the first computing node; if the second computing node fails to successfully read the target data within 10s, it can return a response message of failed reading to the first computing node.

[0194] The message type can be a notification type message or a response type message.

[0195] Among them, the notification type message can be used to inform the computing node that receives the notification message of the storage location of the target data; the response type message can be used to reply the reading result of the target data to the computing node that sends the notification message.

[0196] It should be understood that the notification message sent by the first computing node to the second computing node may be a notification type message.

[0197] It is understandable that the first computing node can store the target data in the target storage location of the target storage node through the CXL device, which can achieve rapid transmission of the target data, and can send a notification message to the second computing node through the Internet link to inform the second computing node of the storage location of the target data, so that the second computing device can quickly obtain the target data from the target storage location as needed. Through the above method, the decoupling of the target data and the notification message is achieved, and the data transmission efficiency in the computer cluster is improved.

[0198] S503 : The second computing node reads target data from a target storage location of a target storage node through a CXL device according to the notification message.

[0199] After receiving the notification message, the second computing node may parse the content of the notification message and read the target data at the target storage location of the target storage node according to the content of the notification message.

[0200] Optionally, the second computing node may read an instruction (Load) to read target data at a target storage location of a target storage node.

[0201] It should be noted that if the first computing node sends target data to the second computing node via an Internet link, the target data transmission process can be as follows: first, the first computing node can transfer the target data from the hard disk of the first computing node to the memory of the first computing node; second, the first computing node can transfer the target data from the memory of the first computing node to the network card of the first computing node.

[0202] Correspondingly, the second computing node can read the target data from the network card of the first computing node through the network card of the second computing node; secondly, the second computing node can transfer the target data from the network card of the second computing node to the memory of the second computing node, and can read and transfer the target data from the memory of the second computing node to the hard disk of the second computing node.

[0203] It can be seen that the target data needs to be transmitted multiple times from the first computing node to the second computing node, and each transmission requires the target data to be read and written. Therefore, when transmitting data (target data) through the Internet link, on the one hand, the data needs to be read and written multiple times, resulting in a low data transmission rate; on the other hand, the data reading and writing speed is limited by the Internet bandwidth resources. If the bandwidth is small, the data transmission efficiency will be low.

[0204] In the data transmission method provided in this embodiment, the first computing node can use the storage instruction (Store) to store the local target data of the first computing node to the target storage location of the target storage node through the CXL device. The second computing node can use the read instruction (Load) to directly read the target data from the target storage location of the target storage node through the CXL device. On the one hand, there is no need to perform data reading and writing operations during the transmission of data (target data) from the first computing node to the second computing node, which improves the data transmission efficiency and reduces the data reading and writing costs; on the other hand, the data reading and writing speed through the CXL device is fast and is not limited by the Internet bandwidth, which further improves the data transmission efficiency.

[0205] It is understandable that the second computing node can complete the corresponding cluster task according to the read target data.

[0206] It is understandable that the second computing node can receive the notification message sent by the first computing node through the Internet link, and can quickly read the target data at the target storage location of the target storage node through the CXL device according to the notification message. Through the above method, the decoupling of the target data and the notification message is achieved, and the data transmission efficiency in the computer cluster is improved.

[0207] S504. The second computing node sends a response message to the first computing node through the Internet link. The response message includes a reading result of the target data. The reading result is a reading success or a reading failure.

[0208] Correspondingly, the first computing node can receive the response message sent by the second computing node through the Internet link.

[0209] In this embodiment, the second computing node may read the target data according to the notification message, and may feed back the reading result to the first computing node through a response message.

[0210] Optionally, if the read result is a read failure, the response message may further include the reason for the read failure.

[0211] It should be understood that the message type of the response message may be a response-type message.

[0212] In the data transmission method provided in this embodiment, the first computing node can store the target data in the target storage location of the target storage node through the CXL device, and can send a notification message to the second computing node through the Internet link. The second computing node can read the target data at the target storage location of the target storage node through the CXL device according to the notification message, and can send a response message to the first computing node through the Internet link. In the above method, the first computing node can quickly transmit the target data to the second computing node through the CXL device, and can send a notification message through the Internet link to notify the second computing node to read the target data, thereby realizing the decoupling of data (target data) and messages (notification messages or response messages), and improving the data transmission efficiency in the computer cluster.

[0213] It is understandable that in other embodiments, the data transmission method may also include only Figure 5 Alternatively, the data transmission method may also include only Figure 5 S503 and S504 in.

[0214] The following will expand the CXL link, introduce various implementations of the CXL link, and based on the various implementations of the CXL link, introduce data transmission methods corresponding to the various implementations.

[0215] For example, the CXL link may include at least one CXL device. The CXL device may be a multi-host CXL device, or the CXL device may be a CXL switch device.

[0216] Next, combine Figure 6-Figure 11 , respectively, a data transmission method in a computer cluster when the CXL device is a multi-host CXL device and when the CXL device is a CXL switching device is described.

[0217] Next, combine Figure 6-Figure 8 , a case where the CXL device is a multi-host CXL device is described.

[0218] First, combine Figure 6 , when the CXL device is a multi-host CXL device, the structure of the computer cluster is described.

[0219] Figure 6 A schematic diagram of the structure of a computer cluster provided in an embodiment of the present application. Figure 6 , the computer cluster may include multiple storage nodes, multiple computing nodes, multiple host CXL devices, and an Internet link. The multiple storage nodes may be storage node 1, storage node 2, ..., and storage node M. The multiple computing nodes may be computing node 1, computing node 2, ..., and computing node N. Among them, M and N may be the same, or M and N may be different, which is not limited in this embodiment. Of course, in other embodiments, the number of storage nodes may be one, and the number of computing nodes may be two or more.

[0220] The number of the multi-host CXL devices may be one or more, and the multiple computing nodes are connected to the multiple storage nodes via the one or more multi-host CXL devices.

[0221] When there are multiple multi-host CXL devices, the multi-host CXL devices may be directly or indirectly connected to each other. For example, for any two multi-host CXL devices, the two multi-host CXL devices may be directly connected via connection interfaces on the CXL devices; or the two multi-host CXL devices may be respectively connected to the same computing node to achieve indirect connection between the two multi-host CXL devices.

[0222] When the number of multi-host CXL devices is one, such as Figure 6 As shown, Figure 6 For example, a multi-host CXL device is used for description. Each storage node can be connected to the multi-host CXL device, and each computing node can be connected to the multi-host CXL device. In other words, multiple storage nodes and multiple computing nodes can be connected via the multi-host CXL device.

[0223] For any computing node, the computing node may communicate with one or more other computing nodes in the computer cluster through the multi-host CXL device; or the computing node may communicate with a storage node in the computer cluster through the multi-host CXL device.

[0224] Multiple computing nodes can also be connected via Internet links. Figure 6 Examples in Figure 3 Based on the embodiment, the case where the CXL device is a multi-host CXL device is further described. Figure 6 For other information about compute nodes, storage nodes, and CXL links, see Figure 3 , which will not be elaborated here.

[0225] Secondly, combined Figure 7 , a data transmission method in a computer cluster is described when the CXL device is a multi-host CXL device.

[0226] Figure 7 A flowchart of another data transmission method provided in an embodiment of the present application. Figure 7 , the method may include:

[0227] S701. A first computing node determines a target storage node and a target storage location in the target storage node among multiple storage nodes.

[0228] In this embodiment, the first computing node may be Figure 6 Any computing node in, that is, the first computing node can be any computing node among computing node 1, computing node 2, . . . and computing node N.

[0229] It should be noted that the specific implementation method of S701 can refer to S501 and will not be repeated here.

[0230] S702: The first computing node stores target data in a target storage location of a target storage node through a multi-host CXL device.

[0231] In this embodiment, the first computing node sends a first storage request to the multi-host CXL device, the first storage request including node information of a target storage node, a target storage location, and target data; wherein the first storage request is used to request the multi-host CXL device to store the target data in the target storage location of the target storage node.

[0232] Specifically, the first computing node may request the multi-host CXL device to store the target data in the target storage location of the target storage node.

[0233] It should be noted that since the speed of reading and writing data through a multi-host CXL device is much faster than the speed of reading and writing data through an Internet link, and the target data is generally large-scale big data, this embodiment avoids storing the target data through an Internet link, thereby avoiding the bandwidth resources of the Internet link from affecting the transmission efficiency of the target data.

[0234] S703. The first computing node sends a notification message to the second computing node through an Internet link.

[0235] Since the size of the notification message is small, the Internet bandwidth resources required to transmit the notification message through the Internet link are small. At present, the Internet bandwidth resources can meet the transmission speed requirements of the notification message, so in this embodiment, the notification message can be sent through the Internet link.

[0236] In this embodiment, the network card of the first computing node can be connected to the network card of the second computing node through an Internet link. When the first computing node sends a notification message to the second computing node, the network card of the first computing node can send the notification message to the network card of the second computing node through the Internet link, so that the second computing node can obtain the notification message.

[0237] In this embodiment, the second computing node may be Figure 6 Any one or more computing nodes except the first computing node.

[0238] For example, Figure 6 For example, assuming that the first computing node is computing node 1, the second computing node can be any one or more computing nodes among computing node 2, computing node 3, ... and computing node N.

[0239] For information related to the notification message, please refer to S502, which will not be repeated here.

[0240] S704 : The second computing node reads target data from a target storage location of a target storage node through the multi-host CXL device according to the notification message.

[0241] In this embodiment, the second computing node may parse the content of the notification message to obtain the target storage node and the target storage location of the target data, and may read the target data at the target storage location of the target storage node through the multi-host CXL device.

[0242] Optionally, the second computing node may send a read request to the multi-host CXL device, and the read request may include a target storage node and a target storage location. The multi-host CXL device may obtain the target data from the target storage location of the target storage node according to the read request. After the multi-host CXL device obtains the target data, the second computing node may read the target data from the multi-host CXL device, and may process a corresponding cluster task in the computer cluster according to the target data.

[0243] S705. The second computing node sends a response message to the second computing node through the Internet link. The response message includes a reading result of the target data. The reading result is a reading success or a reading failure.

[0244] The size of the response message is relatively small. Less bandwidth resources are required in the process of transmitting the corresponding message via the Internet link. Therefore, in this embodiment, the response message can also be sent via the Internet link.

[0245] The data transmission method provided in this embodiment is that the first computing node can store the target data in the target storage location of the target storage node through the CXL device, and can send a notification message to the second computing node through the Internet link. The second computing node reads the target data at the target storage location of the target storage node through the CXL device according to the notification message, and can send a response message to the second computing node through the Internet link. In the above method, data can be transmitted between the first computing node and the second computing node through the CXL device, and messages (such as notification messages or response messages) can be transmitted through the Internet link. Through the above method, the decoupling of data (target data) and messages can be achieved. In addition, the speed of reading and writing data through the CXL device is high, and the reading and writing data through the CXL device is not limited by the Internet bandwidth. Therefore, the data transmission method provided in the embodiment of the present application can improve the data transmission efficiency in the computer cluster.

[0246] Finally, combine Figure 8 , through a specific example, the data transmission process in the computer cluster when the CXL device is a multi-host CXL device is explained.

[0247] Figure 8 A data transmission flow diagram provided in this embodiment of the present application. Figure 8 The computer cluster may include storage node 1, storage node 2, storage node 3, a multi-host CXL device, computing node 1, computing node 2, computing node 3, computing node 4, and an Internet link.

[0248] Storage node 1, storage node 2, storage node 3, compute node 1, compute node 2, compute node 3, and compute node 4 can all be connected to the multi-host CXL device. Compute node 1, compute node 2, compute node 3, and compute node 4 can also be connected to an Internet link.

[0249] Assume that the first computing node is computing node 1, the second computing node is computing node 4, the target storage node is storage node 2, and the target storage location is storage location 1. Then the process of target data transmission can be:

[0250] First, computing node 1 can store the target data in storage location 1 of storage node 2 through the multi-host CXL device. Second, computing node 1 can send a notification message corresponding to the target data to computing node 4 through the Internet link. Finally, computing node 4 can parse the notification message to obtain the target storage node and the target storage location, and can read the target data from storage location 1 of storage node 2 through the multi-host CXL device.

[0251] Optionally, in other embodiments, the number of the second computing nodes may be multiple. If the number of the second computing nodes is multiple, the first computing node may broadcast a notification message to the multiple second computing nodes via an Internet link, so that each second computing node may obtain target data through the multi-host CXL device according to the notification message.

[0252] Next, combine Figure 9-11 , a case where a CXL link includes at least one CXL device, and the CXL device is a CXL switching device is described.

[0253] First, combine Fig. 9 , the structure of the computer cluster is described when the CXL device is a CXL switching device.

[0254] Fig. 9 This is a schematic diagram of another computer cluster structure provided in the embodiment of the present application. Fig. 9 The computer cluster may include multiple computing nodes, multiple CXL switching devices, and multiple storage nodes. For example, the multiple computing nodes may be H1, H2, H3, H4, H5, H6, H7, H8, and H9. The multiple storage nodes may be D1, D2, D3, D4, D5, D6, and D7. The multiple CXL switching devices may be CXL switching device 1, CXL switching device 2, CXL switching device 3, CXL switching device 4, and CXL switching device 5.

[0255] The number of CXL switching devices may be one or more. When the number of CXL switching devices is multiple, all CXL switching devices in the computer cluster may be interconnected. The interconnection may be a direct connection, or the interconnection may be an indirect connection. For example, CXL switching device 1 may be directly connected to CXL switching device 4, and CXL switching device 1 may also be indirectly connected to CXL switching device 5 through CXL switching device 4.

[0256] Each computing node can be connected to a CXL switch device. For example, H1 can be connected to CXL switch device 1.

[0257] Each storage node may be connected to a CXL switch device. For example, D1 may be connected to CXL switch device 4.

[0258] Each CXL switch device may be connected to at least one computing node and / or storage node.

[0259] In addition, all computing nodes can also be connected through the Internet link ( Fig. 9 The Internet link may be a wired link, or the Internet link may be a wireless link.

[0260] It should be noted that Fig. 9 The structure of the computer cluster is only exemplified and does not constitute a limitation on the structure of the computer cluster. The position of each computing node and each storage node in the computer cluster can be set according to actual needs, the number of storage devices and CXL switching devices in the computer cluster can be one or more, and the number of computing nodes / storage nodes connected to each CXL switching device can also be one or more, and each computing node can also be connected to one or more CXL switching devices, which is not limited in this embodiment.

[0261] For any computing node, the computing node can communicate with one or more other computing nodes in the computer cluster through the CXL switching device; or the computing node can communicate with a storage node in the computer cluster through the CXL switching device.

[0262] It should be noted that Fig. 9 Examples in Figure 3 Based on the embodiment, the case where the CXL device is a CXL switching device is further described. Fig. 9 For other information about compute nodes, storage nodes, and CXL links, see Figure 3 , which will not be elaborated here.

[0263] Secondly, combined Fig.10 , a data transmission method in a computer cluster is described when the CXL device is a CXL switching device.

[0264] Fig.10 A flowchart of another data transmission method provided in an embodiment of the present application. Fig.10 , the method may include:

[0265] S1001. A first computing node determines a target storage node and a target storage location in the target storage node among multiple storage nodes.

[0266] In this embodiment, the first computing node may be Fig. 9 Any computing node, that is, the first computing node can be any computing node among H1, H2, H3, H4, H5, H6, H7, H8 and H9.

[0267] It should be noted that the specific implementation method of S1001 can refer to S501 and will not be repeated here.

[0268] S1002: The first computing node stores target data in a target storage location of a target storage node through a CXL switching device.

[0269] In this embodiment, the first computing node may send a second storage request to the CXL switch device, the second storage request including node information of the target storage node, the target storage location and the target data. The second storage request is used to instruct the CXL switch device to store the target data in the target storage location of the target storage node.

[0270] Optionally, the first computing node may send the second storage request to a first CXL switching device to which the first computing node is directly connected.

[0271] In this embodiment, the target storage node may be directly connected to the first CXL switch device; or, the target storage node may not be directly connected to the first CXL switch device.

[0272] In a possible implementation, if the target storage node is directly connected to the first CXL switch device, the first CXL switch device may store the target data in the target storage location of the target storage node according to the second storage request.

[0273] In another possible implementation, if the target storage node is not directly connected to the first CXL switch device, but the CXL switch device directly connected to the target storage node is directly connected to the first CXL switch device, in this implementation, the first CXL switch device may directly forward the second storage request to the CXL switch device directly connected to the target storage node, so that the CXL switch device directly connected to the target storage node stores the target data in the target storage location of the target storage node.

[0274] In another possible implementation, if the target storage node is not directly connected to the first CXL switch device, and the CXL switch device directly connected to the target storage node is not directly connected to the first CXL switch device, in this embodiment, the first CXL switch device may forward the second storage request to the CXL switch device directly connected to the target storage node through at least one other CXL switch device, so that the CXL switch device directly connected to the target storage node stores the target data in the target storage location of the target storage node.

[0275] It should be noted that since the speed of reading and writing data through the CXL switching device is much faster than the speed of reading and writing data through the Internet link, and the target data is generally large-scale big data, this embodiment avoids storing the target data through the Internet link, thereby avoiding the bandwidth resources of the Internet link from affecting the transmission efficiency of the target data.

[0276] S1003. The first computing node sends a notification message to the second computing node via an Internet link.

[0277] The second computing node may be any computing node in the computer cluster except the first computing node.

[0278] For example, Fig. 9 For example, the second computing node can be Fig. 9 Any one or more computing nodes except the first computing node. Assuming that the first computing node is H1, the second computing node can be any one or more computing nodes among H1, H2, H3, H4, H5, H6, H7, H8 and H9.

[0279] For information related to the notification message, please refer to S502, which will not be repeated here.

[0280] Since the size of the notification message is small, the Internet bandwidth resources required to transmit the notification message through the Internet link are small. At present, the Internet bandwidth resources can meet the transmission speed requirements of the notification message, so in this embodiment, the notification message can be sent through the Internet link.

[0281] In this embodiment, the network card of the first computing node can be connected to the network card of the second computing node through an Internet link. When the first computing node sends a notification message to the second computing node, the network card of the first computing node can send the notification message to the network card of the second computing node through the Internet link, so that the second computing node can obtain the notification message.

[0282] S1004: The second computing node reads target data from a target storage location of a target storage node through a CXL switch device according to the notification message.

[0283] In this embodiment, the second computing node may parse the content of the notification message to obtain the target storage node and the target storage location of the target data, and may read the target data at the target storage location of the target storage node through the CXL switch device.

[0284] Optionally, the second computing node may send a read request to a CXL switch device directly connected to the second computing node, and the read request may include a target storage node and a target storage location. The CXL switch device directly connected to the second computing node may directly or indirectly obtain the target data from the target storage location of the target storage node according to the read request. After the CXL switch device directly connected to the second computing node obtains the target data, the second computing node may read the target data from the CXL switch device directly connected to the second computing node, and may process corresponding cluster tasks in the computer cluster according to the target data.

[0285] S1005. The second computing node sends a response message to the second computing node through the Internet link. The response message includes a reading result of the target data. The reading result is a reading success or a reading failure.

[0286] In this embodiment, the response message may also be sent via an Internet link.

[0287] In the data transmission method provided in this embodiment, the first computing node can store the target data to the corresponding target storage node through the CXL switching device, and send a notification message to the second computing node through the Internet link. The second computing node reads the target data at the target storage location of the target storage node through the CXL switching device according to the notification message, and can send a response message to the second computing node through the Internet link. In the above method, data can be transmitted between the first computing node and the second computing node through the CXL switching device, and messages (such as notification messages or response messages) can be transmitted through the Internet link. It can be understood that data has a large demand for bandwidth. If the bandwidth is small, it will affect the data transmission efficiency. Through the above method, the decoupling of data and messages can be achieved. The speed of reading and writing data through the CXL switching device is high, and the reading and writing of data through the CXL switching device is not limited by the Internet bandwidth, avoiding the problem of low efficiency of data transmission through the Internet, and can improve the data transmission efficiency in the computer cluster.

[0288] Finally, combine Fig.11 , through a specific example, the data transmission process in the computer cluster when the CXL device is a CXL switching device is explained.

[0289] Fig.11 Another data transmission flow diagram provided in this embodiment of the present application. Fig.11 ,exist Fig. 9 Based on the above, it is assumed that the first computing node is H1, the second computing node is H8, the target storage node is D7, and the target storage location is storage location 1 on D7. Then the first CXL switch device is CXL switch device 1.

[0290] The target data transmission process can be:

[0291] First, H1 can send a second storage request to CXL switch device 1. CXL switch device 1 can forward the second storage request to CXL switch device 3 through CXL switch device 2, so that CXL switch device 3 can store the target data in storage location 1 of D2 according to the second storage request. Second, H1 can send a notification message corresponding to the target data to H8 through an Internet link. Finally, H8 can parse the notification message to obtain the target storage node and the target storage location, and can read the target data from storage location 1 of D7 through CXL switch device 3.

[0292] Optionally, in other embodiments, the number of the second computing nodes may be multiple. If the number of the second computing nodes is multiple, the first computing node may broadcast a notification message to the multiple second computing nodes via an Internet link, so that each second computing node may obtain target data through the CXL switching device according to the notification message.

[0293] Fig.12 The data transmission device 10 is applied to a computing node of a computer cluster, such as a computer cluster. Figure 3 / Figure 6 / Fig. 9 The computer cluster shown. The computer cluster includes multiple computing nodes, storage nodes and CXL devices, the multiple computing nodes are connected to the storage nodes through the CXL devices, and the multiple computing nodes are also connected through Internet links. Any computing node among the multiple computing nodes may include the data transmission device 10, and the data transmission device 10 may include a processing module 11 and a sending module 12, wherein:

[0294] The processing module 11 is used for storing target data in a target storage location of a target storage node through the CXL device;

[0295] The sending module 12 is used to send a notification message to the second computing node through the Internet link, the notification message including the node information of the target storage node and the target storage location, the notification message is used to inform the second computing node that the target data is stored in the target storage location of the target storage node, wherein the second computing node is any computing node among the multiple computing nodes except the computing node to which the data transmission device belongs.

[0296] The data transmission device provided in this embodiment can be used to execute the data transmission method performed by the first computing node in the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail here.

[0297] In a possible implementation, the CXL device is a multi-host CXL device, and the plurality of computing nodes and the storage node are respectively connected to the multi-host CXL device; the processing module 11 is specifically configured to:

[0298] sending a first storage request to the multi-host CXL device, the first storage request including node information of the target storage node, the target storage location, and the target data;

[0299] The first storage request is used to request the multi-host CXL device to store the target data in the target storage location of the target storage node.

[0300] In a possible implementation, the CXL device is a CXL switching device, and the plurality of computing nodes are connected to the storage node via the CXL switching device; the processing module 11 is specifically configured to:

[0301] sending a second storage request to the CXL switch device, wherein the second storage request includes node information of the target storage node, the target storage location, and the target data;

[0302] The second storage request is used to instruct the CXL switch device to store the target data in the target storage location of the target storage node.

[0303] In a possible implementation, the processing module 11 is further configured to determine a target storage node and a target storage location in the target storage node.

[0304] In a possible implementation, the processing module 11 is specifically used to:

[0305] Acquire storage configuration information corresponding to the first computing node, the storage configuration information including node information of a candidate storage node corresponding to the first computing node and a candidate storage area in the candidate storage node;

[0306] According to the data size of the target data and the storage configuration information, the target storage node is determined among the candidate storage nodes, and the target storage location is determined in the candidate storage area of ​​the target storage node, and the remaining storage space of the candidate storage area of ​​the target storage node is greater than or equal to the data size of the target data.

[0307] The data transmission device provided in this embodiment can be used to execute the data transmission method performed by the first computing node in the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail here.

[0308] Fig.13A schematic diagram of another data transmission device provided in an embodiment of the present application. The data transmission device 20 is applied to a computer cluster, the computer cluster includes multiple computing nodes, storage nodes and CXL devices, the multiple computing nodes are connected to the storage nodes via the CXL devices, and the multiple computing nodes are also connected via an Internet link, any one of the multiple computing nodes may include the data transmission device 20, the data transmission device 20 may include an acquisition module 21, a processing module 22 and a sending module 23, wherein:

[0309] The acquisition module 21 is used to acquire a notification message from a first computing node through the Internet link, the notification message including node information and a target storage location of a target storage node, the multiple computing nodes including the first computing node, and the multiple storage nodes including the target storage node;

[0310] The processing module 22 is configured to read the target data from the target storage location of the target storage node through the CXL device according to the notification message.

[0311] The sending module 23 is used to send a response message to the first computing node through the Internet link, and the response message includes a reading result of the target data, and the reading result is a reading success or a reading failure.

[0312] The data transmission device provided in this embodiment can be used to execute the data transmission method executed by the second computing node in the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail here.

[0313] In one possible implementation,

[0314] The CXL device is a multi-host CXL device, and the plurality of computing nodes and the storage node are respectively connected to the multi-host CXL device; or,

[0315] The CXL device is a CXL switch device connected to one or more computing nodes and / or at least one storage node.

[0316] The data transmission device provided in this embodiment can be used to execute the data transmission method executed by the second computing node in the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail here.

[0317] An embodiment of the present application provides a computer cluster, which includes multiple computing nodes, storage nodes, and CXL devices, wherein the computing nodes are used to execute the data transmission method as described in any of the above embodiments.

[0318] Optionally, the CXL device is a multi-host CXL device, and the plurality of computing nodes and the storage node are respectively connected to the multi-host CXL device; or,

[0319] The CXL device is a CXL switching device, and the plurality of computing nodes are connected to the storage nodes via the CXL switching device.

[0320] The computing nodes in the computer cluster provided in this embodiment can be used to execute the data transmission method in the above method embodiment. The implementation principle and technical effect are similar and will not be described in detail here.

[0321] Fig.14 Schematic diagram of a computing node provided in an embodiment of the present application. The computing node may be a first computing node. The computing node may also be a second computing node. Fig.14 As shown, the computing node 30 may include: a processor 31 and a memory 32, wherein the processor 31 and the memory 32 can communicate; exemplarily, the processor 31 and the memory 32 communicate via a communication bus 33, the memory 32 is used to store computer execution instructions, and the processor 31 is used to call the computer execution instructions in the memory to execute the data transmission method shown in any of the above method embodiments.

[0322] Optional, Fig.12 The processing module 11, the sending module 12, and Fig.13 The acquisition module 21 , the processing module 22 , and the sending module 23 may be implemented by the processor 31 .

[0323] Optionally, the computing node 30 may further include a communication interface, which may include a transmitter and / or a receiver.

[0324] Optional, Fig.12 The sending module 12, and Fig.13 The information sent by the sending module 23 can be sent to the external network through the transmitter; Fig.13 The information received by the acquisition module 21 may be first received from an external network by a receiver and then transmitted to the acquisition module 21 .

[0325] Optionally, the computing node 30 may further include a network card, and the processor is communicatively connected to the network card; the processor is communicatively connected to an Internet link via the network card.

[0326] Optionally, the processor may be a central processing unit (CPU), a graphics processing unit (GPU), other general-purpose processors, a digital signal processor (DSP), or an application-specific integrated circuit (ASIC). A general-purpose processor may be a microprocessor or the processor may be any conventional processor. The steps of the method disclosed in the embodiments of the present application may be directly implemented as a hardware processor, or may be implemented as a combination of hardware and software modules in the processor.

[0327] An embodiment of the present application provides a computer-readable storage medium, on which computer-executable instructions are stored; the computer-executable instructions are used to implement the data transmission method described in any of the above embodiments.

[0328] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the above-mentioned data transmission method.

[0329] All or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a readable memory. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned memory (storage medium) includes: read-only memory (English: read-only memory, abbreviated: ROM), RAM, flash memory, hard disk, solid state drive, magnetic tape (English: magnetic tape), floppy disk (English: floppy disk), optical disc (English: optical disc) and any combination thereof.

[0330] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable terminal device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0331] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0332] These computer program instructions can also be loaded onto a computer or other programmable terminal device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0333] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the spirit and scope of the embodiments of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the embodiments of the present application are also intended to include these modifications and variations.

[0334] In the embodiments of the present application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the existence of other identical elements in the process, method, article or device including the element. The term "or" and its variations may refer to "and / or". In the embodiments of the present application, the terms "first", "second", etc. are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. In the embodiments of the present application, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, A and / or B, which may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0335] After considering the invention disclosed in the specification and practice, those skilled in the art will easily think of other embodiments of the present application. The embodiments of the present application are intended to cover any variation, use or adaptive change of the embodiments of the present application, which follows the general principles of the embodiments of the present application and includes common knowledge or conventional technical means in the technical field that are not disclosed in the embodiments of the present application.

Claims

1. A data transmission method, characterized in that: The method is applied to a first computing node of a computer cluster, wherein the computer cluster includes a plurality of computing nodes, a storage node, and a computing interconnect CXL device, wherein the plurality of computing nodes are connected to the storage node via the CXL device, and the plurality of computing nodes are also connected to each other via an Internet link, and wherein the method includes: The first computing node stores the target data in the target storage location of the target storage node through the CXL device, wherein the first computing node is any computing node among the plurality of computing nodes, and the storage nodes include the target storage node; The first computing node sends a notification message to the second computing node via the Internet link, the notification message including the node information of the target storage node and the target storage location, the notification message being used to inform the second computing node that the target data is stored in the target storage location of the target storage node, wherein the second computing node is any computing node among the multiple computing nodes except the first computing node.

2. The method according to claim 1, characterized in that The CXL device is a multi-host CXL device, and the plurality of computing nodes and the storage node are respectively connected to the multi-host CXL device; The first computing node stores the target data in the target storage location of the target storage node through the CXL device, including: The first computing node sends a first storage request to the multi-host CXL device, the first storage request including node information of the target storage node, the target storage location, and the target data; The first storage request is used to request the multi-host CXL device to store the target data in the target storage location of the target storage node.

3. The method according to claim 1, characterized in that The CXL device is a CXL switching device, and the plurality of computing nodes are connected to the storage node via the CXL switching device; The first computing node stores the target data in the target storage location of the target storage node through the CXL device, including: The first computing node sends a second storage request to the CXL switch device, where the second storage request includes node information of the target storage node, the target storage location, and the target data; The second storage request is used to instruct the CXL switch device to store the target data in the target storage location of the target storage node.

4. The method according to any one of claims 1 to 3, characterized in that: The method also includes the first computing node determining a target storage node and a target storage location in the target storage node.

5. The method according to claim 4, characterized in that The first computing node determines a target storage node and a target storage location in the target storage node, including: The first computing node acquires storage configuration information corresponding to the first computing node, the storage configuration information including node information of a candidate storage node corresponding to the first computing node and a candidate storage area in the candidate storage node; The first computing node determines the target storage node among the candidate storage nodes based on the data size of the target data and the storage configuration information, and determines the target storage location in the candidate storage area of ​​the target storage node, wherein the remaining storage space of the candidate storage area of ​​the target storage node is greater than the data size of the target data.

6. A data transmission method, characterized in that: A method for applying to a second computing node of a computer cluster, wherein the computer cluster includes a plurality of computing nodes, a storage node, and a CXL device, wherein the plurality of computing nodes are connected to the storage node via the CXL device, and the plurality of computing nodes are also connected to each other via an Internet link, and wherein the method includes: The second computing node obtains a notification message from the first computing node through the Internet link, the notification message including node information and a target storage location of a target storage node, the multiple computing nodes including the second computing node and the first computing node, and the storage nodes including the target storage node; The second computing node reads the target data from the target storage location of the target storage node through the CXL device according to the notification message.

7. The method according to claim 6, characterized in that The method further comprises: The second computing node sends a response message to the first computing node through the Internet link, and the response message includes a reading result of the target data, and the reading result is a reading success or a reading failure.

8. The method according to claim 6 or 7, characterized in that: The CXL device is a multi-host CXL device, and the plurality of computing nodes and the storage node are respectively connected to the multi-host CXL device; or, The CXL device is a CXL switching device, and the plurality of computing nodes are connected to the storage node via the CXL switching device.

9. A computing node, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 8.

10. The computing node according to claim 9, characterized in that: Also comprising a network card, the processor being communicatively connected to the network card; The processor is communicatively connected to an Internet link via the network card.

Images