Computing cluster, memory access method, and related device

Abstract

A computing cluster, a memory access method, and a related device, relating to the technical field of computers. A plurality of computing nodes in a computing cluster are connected by using the same bus, and a processor in each computing node comprises a DMA engine. A first computing node in the computing cluster generates a memory access request, and uses the DMA engine to send the memory access request to a second computing node on the basis of the bus, the memory access request being used for requesting access to a memory in the second computing node; and the second computing node receives and responds to the memory access request. In this way, the first computing node can access the memory in the second computing node by using the DMA engine by means of the bus, which can effectively reduce a network stack that needs to be executed by the first computing node. Moreover, a P2P forwarding process does not need to be executed by using another computing node, which can effectively improve performance in accessing a memory across nodes in the computing cluster. In addition, a single accelerator can access the bus as an independent computing node without relying on a CPU (host) to access the bus.

Description

Computing cluster, memory access method and related device

[0001] This application claims priority from the Chinese patent application No. 202411356890.X filed with the State Intellectual Property Office of China on September 26, 2024 and entitled "Computing cluster, memory access method and related device", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of computers, and in particular to a computing cluster, a memory access method and related devices. BACKGROUND

[0003] With the development of computer technology, computing clusters are widely used in scenarios such as artificial intelligence (AI) model training or inference. As shown in FIG. 1, a computing cluster usually includes multiple nodes, each of which can be composed of a central processing unit (CPU) and multiple accelerators, which can be graphics processing units (GPUs) or the like. The multiple nodes interact with each other through a communication network. In actual application scenarios, some nodes may need to access the memory of the CPU or the accelerator in other nodes. As shown in FIG. 1, the accelerator 101 in node 1 can access the memory of the accelerator 201 in node 2. For example, in the process of distributed training of an AI model using node 1 and node 2, the accelerator 101 may access the memory of the accelerators in each node to obtain the gradient information generated by the accelerators in each node, so that the accelerator 101 can aggregate the gradient information generated by the accelerators and use the aggregated gradient to update the parameter values in the AI model.

[0004] However, in a computing cluster, the performance of accessing the memory in other nodes based on the communication network is usually low, which may not meet the business requirements in some scenarios. For example, in the scenario of distributed training of an AI model, the performance of accessing the memory between multiple nodes is low, which affects the efficiency of data interaction (such as gradient data) between different nodes, i.e., the efficiency of data interaction between the accelerators in multiple nodes, thereby affecting the overall training efficiency of the AI model, resulting in a long training time of the AI model. SUMMARY

[0005] The present application provides a computing cluster to improve the performance of accessing the memory across nodes in the computing cluster. In addition, the present application also provides a corresponding memory access method, a computing node, a computer readable storage medium and a computer program product.

[0006] In a first aspect, the present application provides a memory access method, which can be applied to a computing cluster. The computing cluster includes a plurality of computing nodes, and the plurality of computing nodes are connected through a same bus, which can be a UB bus or other types of bus, etc. Each of the plurality of computing nodes in the computing cluster includes a processor and a memory. The processor can be a CPU or an accelerator (such as a GPU, NPU, etc.), and includes a DMA (Direct Memory Access) engine. Taking the first computing node in the computing cluster accessing the memory of the second computing node in the computing cluster as an example, the first computing node is configured to generate a memory access request, and send the memory access request to the second computing node based on the bus by using the built-in DMA engine. The sent memory access request is used to request to access the memory in the second computing node. The second computing node is configured to receive the memory access request from the bus, and respond to the memory access request, such as writing the data carried in the memory access request into the memory, or reading the data from the memory and sending the data to the first computing node, etc.

[0007] Since the first computing node can access the memory in the second computing node by using the DMA engine through the bus, the network stack required to be executed by the first computing node can be effectively reduced, so that the efficiency of the first computing node accessing the memory can be effectively improved. Moreover, in the computing cluster, a single processor can also access the same bus as an independent computing node, so that the first computing node can directly access the memory of the second computing node by using the DMA engine based on the bus, without the need to perform a P2P forwarding process through other computing nodes. This can effectively reduce the overhead generated by the first computing node accessing the memory across nodes, and also effectively improve the efficiency of the first computing node accessing the memory. In this way, the performance of accessing the memory across nodes in the computing cluster can be effectively improved. In addition, in the computing cluster, the first computing node can only need to connect the bus based on a group of ports to realize the communication with other computing nodes, without the need to configure the first computing node with multiple groups of ports. This can effectively reduce the hardware cost of the first computing node and the difficulty of the hardware manufacturing process. At the same time, a single accelerator in the computing cluster can access the bus as an independent computing node, so that the number of accelerators accessing the bus in the computing cluster can be flexibly configured according to the actual application requirements, without the need to rely on the CPU (host) to access the bus.

[0008] In a possible implementation, the first computing node is specifically configured to generate an access request descriptor, and generate, by using the DMA engine, a memory access request according to the access request descriptor, where the access request descriptor comprises a UBA (Uniform Bus Address) used to indicate a storage space in the memory of the second computing node that can be accessed by the first computing node, and the generated memory access request comprises the UBA. In this way, the first computing node generates an asynchronous memory access request based on the access request descriptor, so as to realize asynchronous access to the memory of the second computing node.

[0009] In a possible implementation, the first computing node is specifically configured to add the access request descriptor to a submission queue in the memory of the first computing node, and generate, by using the DMA engine, a memory access request according to the access request descriptor in the submission queue; for example, the first computing node can ring a doorbell of the DMA engine to trigger the DMA engine to read the access request descriptor from the submission queue and generate the memory access request accordingly. Then, the first computing node can also be configured to, after sending the memory access request, generate a completion notification message corresponding to the access request descriptor, and submit the completion notification message to a completion queue in the memory of the first computing node. In this way, the first computing node can realize asynchronous access to the memory of the second computing node by submitting the access request descriptor to the submission queue and submitting the completion notification message to the completion queue, so as to realize efficient memory access to the second computing node.

[0010] In a possible implementation, the processor in the first computing node comprises a first bus controller, and the first bus controller comprises the DMA engine, so that the first computing node can generate the memory access request by using the DMA engine in the first bus controller.

[0011] In a possible implementation, the processor in the second computing node comprises a second bus controller, and the memory access request sent by the first computing node comprises an identifier of the first bus controller and an identifier of the second bus controller. In this way, in the process of transmitting the memory access request based on the bus, the first computing node that sends the memory access request and the second computing node that receives the memory access request can be determined based on the identifiers of the bus controllers, so as to realize successful access to the memory of the second computing node. For example, the identifier of the bus controller can be an EID of the bus controller, or can be a CNA of the bus controller, or can comprise both the EID and the CNA.

[0012] In a possible implementation, the access request descriptor can further comprise an identifier of the second bus controller, for example, an EID of the second bus controller. Further, the access request descriptor can further comprise an identifier of the first bus controller.

[0013] In a possible implementation, the first computing node comprises a transmission connection table item and a routing table item, wherein the transmission connection table item is configured to record a mapping relationship between a target transmission identifier and a network address of the second computing node, the target transmission identifier being used to indicate a communication path between the first computing node and the second computing node, and the routing table item is configured to record a mapping relationship between a target port of the first computing node and the network address of the second computing node. Then, the first computing node is specifically configured to query the transmission connection table item to obtain the network address of the second computing node, and query the routing table item according to the network address of the second computing node to obtain the target port, so that the first computing node sends a memory access request to the second computing node based on the bus by using the DMA engine and the target port. In this way, the first computing node can guide the transmission of the memory access request between the first computing node and the second computing node by using the target transmission identifier, so as to shield the perception of the software of the first computing node to the physical link, which can effectively reduce the influence of the change of the physical link on the software. That is, the software in the first computing node can consider that the transmission connection between the first computing node and other computing nodes is stable and unchanged, when the target transmission identifier remains unchanged. In this way, when the physical link used to transmit data between the first computing node and the second computing node changes, such as the change of the physical channel or the number of channels, the software of the first computing node can not perceive, so as to reduce the influence of the change of the physical link on the software.

[0014] In a possible implementation, the memory access request comprises a plurality of sub-requests, and the target port comprises a plurality of ports. Then, the first computing node is specifically configured to send the plurality of sub-requests in parallel based on the bus by using the plurality of ports. In this way, the first computing node can send the to-be-written data to the second computing node in parallel based on a plurality of physical links (each port corresponds to a physical link), or receive a plurality of response messages fed back by the second computing node based on the plurality of sub-requests based on the plurality of physical links, which can effectively improve the transmission efficiency of data between the first computing node and the second computing node, thereby further improving the performance of accessing the memory across nodes in the computing cluster.

[0015] In a possible implementation, each of the plurality of sub-requests is used to request to read data in the memory of the second computing node. Then, the first computing node is further configured to receive a plurality of response messages, the plurality of response messages being messages generated by the second computing node in response to the plurality of sub-requests respectively. In this way, when the first computing node requests to read data in the memory of the second computing node, the first computing node can improve the efficiency of feeding back the required data of the first computing node by generating and sending the plurality of sub-requests, thereby achieving the improvement of the performance of accessing the memory across nodes in the computing cluster.

[0016] In a second aspect, the present application provides a memory access method, which is applied to a first computing node, and the method comprises: generating, by the first computing node, a memory access request, the first computing node comprising a processor, and the processor in the first computing node comprising a DMA (Direct Memory Access) engine, wherein the first computing node is connected to a second computing node through a bus, the second computing node comprising a processor and a memory; and sending, by the first computing node, the memory access request to the second computing node based on the bus by using the DMA engine, wherein the memory access request is used to request to access the memory in the second computing node.

[0017] In a possible implementation, the second computing node receives the memory access request from the bus; and the second computing node responds to the memory access request, the second computing node and the first computing node being two computing nodes in a plurality of computing nodes included in a computing cluster, the plurality of computing nodes being connected through the bus.

[0018] In a possible implementation, the first computing node generates the memory access request, comprising: generating, by the first computing node, an access request descriptor, the access request descriptor comprising a UBA (Uniform Bus Address), the UBA being used to indicate a storage space in the memory of the second computing node that can be accessed by the first computing node; and generating, by the first computing node, the memory access request based on the access request descriptor by using the DMA engine, the memory access request comprising the UBA.

[0019] In a possible implementation, the first computing node generates the memory access request based on the access request descriptor by using the DMA engine, comprising: adding, by the first computing node, the access request descriptor to a submission queue in the memory of the first computing node; generating, by the first computing node, the memory access request based on the access request descriptor in the submission queue by using the DMA engine; and the method further comprises: after sending the memory access request, generating, by the first computing node, a completion notification message corresponding to the access request descriptor; and submitting, by the first computing node, the completion notification message to a completion queue in the memory of the first computing node.

[0020] In a possible implementation, the processor in the first computing node comprises a first bus controller, and the first bus controller comprises the DMA engine; and the first computing node generates the memory access request, comprising: generating, by the first computing node, the memory access request by using the DMA engine in the first bus controller.

[0021] In a possible implementation, the processor in the second computing node comprises a second bus controller, and the memory access request comprises an identifier of the first bus controller and an identifier of the second bus controller.

[0022] In a possible implementation, the access request descriptor can further include an identifier of the second bus controller (e.g., an EID of the second bus controller).

[0023] In a possible implementation, the first computing node includes a transmission connection table item and a routing table item, the transmission connection table item is used to record a mapping relationship between a target transmission identifier and a network address of the second computing node, the target transmission identifier is used to indicate a communication path between the first computing node and the second computing node, and the routing table item is used to record a mapping relationship between a target port of the first computing node and the network address of the second computing node; and the first computing node sends, by using the DMA engine, the memory access request to the second computing node based on the bus, including: the first computing node queries the transmission connection table item to obtain the network address of the second computing node; the first computing node queries the routing table item according to the network address of the second computing node to obtain the target port; and the first computing node sends, by using the DMA engine and the target port, the memory access request to the second computing node based on the bus.

[0024] In a possible implementation, the memory access request includes a plurality of sub-requests, and the target port includes a plurality of ports; and the first computing node sends, by using the DMA engine and the target port, the memory access request to the second computing node based on the bus, including: the first computing node sends, by using the DMA engine and the plurality of ports, the plurality of sub-requests to the second computing node in parallel based on the bus.

[0025] In a possible implementation, each of the plurality of sub-requests is used to request to read data in the memory of the second computing node; and the method further includes: the first computing node receives a plurality of response messages, the plurality of response messages being generated by the second computing node in response to the plurality of sub-requests respectively.

[0026] The memory access method provided in the second aspect corresponds to the computing cluster provided in the first aspect, and therefore the technical effects of any one of the implementation manners of the memory access method provided in the second aspect can be referred to the technical effects of the corresponding implementation manners in the first aspect, which will not be described herein.

[0027] In the third aspect, the present application provides a computing node, which is connected to other computing nodes through the same bus, and includes a processor and a memory, where the processor includes a direct memory access (DMA) engine; the computing node is configured to generate a memory access request and send, by using the DMA engine, the memory access request to the other computing nodes based on the bus, the memory access request being used to request to access the memory in the other computing nodes. The computing node provided in the third aspect corresponds to the first computing node in the computing cluster provided in the first aspect, and therefore the technical effects of the computing node provided in the third aspect can be referred to the technical effects of the corresponding implementation manners in the first aspect, which will not be described herein.

[0028] In a fourth aspect, the present application provides a computer readable storage medium having instructions stored therein, which when executed on a computing device, cause the computing device to perform the operation steps of the memory access method of the second aspect or any of the implementation forms of the second aspect.

[0029] In a fifth aspect, the present application provides a computer program product having instructions, which when executed on a computing device, cause the computing device to perform the operation steps of the memory access method of the second aspect or any of the implementation forms of the second aspect.

[0030] On the basis of the implementation forms of the aspects described above, the present application can be further combined to provide more implementation forms. BRIEF DESCRIPTION OF DRAWINGS

[0031] Fig. 1 is a structural schematic diagram of a computing cluster for data interaction based on a communication network;

[0032] Fig. 2 is a structural schematic diagram of an exemplary computing cluster for data interaction based on a bus provided by the present application;

[0033] Fig. 3 is a structural schematic diagram of an exemplary bus controller 213 and bus controller 223;

[0034] Fig. 4 is a flow schematic diagram of a memory access method (for writing data into memory) provided by the present application;

[0035] Fig. 5a is a flow schematic diagram of generating a memory access request for applying data writing and processing an acknowledgement message in the computing node 210;

[0036] Fig. 5b is a flow schematic diagram of processing a memory access request for applying data writing in the computing node 220;

[0037] Fig. 6 is a format schematic diagram of an exemplary memory access request;

[0038] Fig. 7 is a flow schematic diagram of another memory access method (for reading data in memory) provided by the present application;

[0039] Fig. 8a is a flow schematic diagram of generating a memory access request for applying data reading and processing a response message in the computing node 210;

[0040] Fig. 8b is a flow schematic diagram of processing a memory access request for applying data reading in the computing node 220. DETAILED DESCRIPTION

[0041] To improve the performance of cross-node memory access in a computing cluster, this application provides a computing cluster comprising multiple computing nodes, each implemented by a processor. The different computing nodes can be architecturally equivalent and may not have any dependencies. The multiple computing nodes in the computing cluster are all connected to the same bus. This allows the processor-implemented computing node to directly access the memory of other computing nodes implemented by other processors based on this bus, without executing a heavy network stack or relaying memory access requests through other computing nodes, thereby improving the performance of cross-node memory access in the computing cluster.

[0042] To facilitate understanding of the technical solution of this application, the relevant technical terms involved in this application will be explained below.

[0043] The network stack refers to the software portion of an operating system (OS) that implements network protocols, along with dedicated hardware (such as network interface cards). It is a crucial component of communication networks. The network stack typically comprises multiple network protocol layers, each responsible for handling specific tasks. Through layer-by-layer processing, data encapsulation, transmission, and decapsulation are achieved, thereby enabling reliable and efficient network communication.

[0044] Direct memory access (DMA) is a feature of computer systems that allows certain hardware subsystems to access system memory independently of the CPU cores in the computer.

[0045] A DMA engine is an engine that implements asynchronous memory access based on DMA technology.

[0046] Remote direct memory access (RDMA) refers to remotely accessing the memory of other computing devices over a network without the intervention of the CPU cores in those other computing devices.

[0047] The memory management unit (MMU), sometimes called the paging memory management unit, is a piece of computer hardware responsible for handling the CPU's memory access requests. Its functions include virtual address to physical address translation and memory access permission control.

[0048] A bus memory management unit (MMU) is an MMU adapted to a bus that can process bus-based memory access requests, including translating virtual addresses in the request into physical addresses in memory. The bus can be a unified bus (UB or UBUS) or other types of bus; there is no limitation on this.

[0049] A page table is a data structure used in a virtual memory system in a computer operating system, which stores the mapping of virtual addresses (also referred to as logical addresses) to physical addresses. When data is queried, a computing node can query the page table according to a virtual address to obtain a physical address corresponding to the virtual address, so as to access a storage area indicated by the physical address.

[0050] The technical solutions in the present application will be described below in combination with the accompanying drawings provided by the present application.

[0051] In the computing cluster shown in FIG. 1, different computing nodes interact with each other through an RDMA communication network. When the accelerator 101 in the node 1 needs to access the memory of the accelerator 201 in the node 2 across nodes, the accelerator 101 generates a communication task and notifies the network card 103 to execute. Taking an RDMA write operation as an example, the network card 103 reads data from the memory of the accelerator 101 through a bus, encapsulates the data into a network packet, and sends the network packet to the network card 203 of the node 2. After receiving the network packet, the network card 203 performs decapsulation processing, and the network card 203 writes the data into the memory of the accelerator 201 through a bus. In this process, the node 1 and the node 2 need to perform a large amount of network stack processing, such as bus and network communication data conversion, packet encapsulation and transmission, etc. Therefore, the execution of a heavy network stack will affect the data communication efficiency between the node 1 and the node 2, that is, the efficiency of the accelerator 101 accessing the memory of the accelerator 201 across nodes. Moreover, in the process of accessing the memory across nodes, the network card 103 will use the CPU 10 to perform at least 2 times of peer-to-peer (P2P) forwarding of bus access, and the network card 203 will use the CPU 20 to perform at least 1 time of P2P forwarding of bus access. This not only generates a large communication overhead, but also affects the efficiency of the accelerator 101 accessing the memory of the accelerator 201 across nodes, and increases the cost of the computing node.

[0052] Based on this, the present application provides a computing cluster to improve the performance of accessing memory across nodes in the computing cluster. Referring to FIG. 2, a structural schematic diagram of a computing cluster is shown. As shown in FIG. 2, the computing cluster 200 can include a plurality of computing nodes, which are taken as examples of the computing node 210 to the computing node 240 in FIG. 2. Each computing node can be implemented by a processor, which can be a CPU or an accelerator. For example, the computing node 210 can be implemented by the accelerator 211, and the computing node 220 can be implemented by the CPU 221. Each computing node runs an operating system (OS), so in actual application scenarios, the computing nodes in the computing cluster 200 can also be referred to as OS nodes.

[0053] The plurality of computing nodes can be connected through a bus, which can be a UB, for example, or can be a bus of other types (such as a bus of a Peripheral Component Interconnect Express (PCIe) standard), and is not limited thereto. The bus connecting the plurality of computing nodes can include a fabric manager (FM), which can be used to monitor and manage resources of the bus, such as bandwidth, channels, etc. used for the plurality of computing nodes to transmit data, and can also manage the plurality of computing nodes accessing the bus, such as assigning identifiers to the plurality of computing nodes.

[0054] Exemplarily, the processor, in addition to being a CPU, can also be a CPU SoC (i.e., a CPU in the form of an SoC), or a general-purpose processor, or can be any one of an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), etc., or any combination thereof.

[0055] Alternatively, the processor can also be an accelerator. The accelerator can be any one of a GPU, a neural-network processing unit (NPU), a tensor processing unit (TPU), a deep-learning processing unit (DPU), a software-defined infrastructure (SDI) chip, an artificial intelligence (AI) chip, etc., or any combination thereof. In actual applications, the processor can be an accelerator in the form of an SoC.

[0056] As shown in FIG. 2, the CPU or accelerator in each computing node can include a bus controller, such as the accelerator 211 in the computing node 210 can include the bus controller 213. Also, each computing node can include a memory in addition to the CPU or accelerator, such as the computing node 210 further includes the memory 212, etc. Further, each computing node can also include other devices, such as each computing node can also include a hard disk (for persistent storage of data), or the computing node can also include a network card (which can be used to communicate data with other computing nodes over a communication network), without limitation.

[0057] The memory in each computing node can be, for example, a random access memory (RAM) or a read only memory (ROM). The random access memory can be, for example, a dynamic random access memory (DRAM), a high bandwidth memory (HBM), or a static random access memory (SRAM), or a storage class memory (SCM), etc. The read only memory can be, for example, a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), etc. In addition, the memory in the computing node can also be a dual in-line memory module (DIMM), i.e., a module composed of dynamic random access memory (DRAM); or the memory can also be a solid state disk (SSD). Furthermore, the memory in the computing node can also have a power retention function, where the power retention function refers to that when the memory is powered off and then powered on again, the data stored in the memory will not be lost due to power off.

[0058] The bus controller integrated in the accelerator or the CPU can be a controller for enabling data communication with other computing nodes through a bus, and the bus controller can include a DMA engine for enabling access to the memory in other computing nodes. For example, as shown in FIG. 3, the bus controller 213 in the computing node 210 can include a DMA engine 2131, and the bus controller 223 in the computing node 220 can include a DMA engine 2231. Further, the bus controller in each computing node can also include a bus memory management unit and a data transceiver. For example, as shown in FIG. 3, the bus controller 213 can further include a bus memory management unit 2132 and a data transceiver 2133, and the bus controller 223 can further include a bus memory management unit 2232 and a data transceiver 2233. The DMA engine, the bus memory management unit, and the data transceiver (as well as other hardware) in the bus controller can be connected through an internal bus. The bus memory management unit, for example, can be a unified bus memory management unit (UMMU) capable of adapting to the UB bus and processing memory access requests based on UB bus transmission. In actual applications, the bus memory management unit can also be a memory management unit capable of adapting to other types of buses. The data transceiver is responsible for transmitting or receiving data based on the bus, and can include at least one port, such as the ports P0 to P n For example, the data transceiver 2133 can include a transmitter (for transmitting data), a receiver (for receiving data), and a router (for determining the route of transmitting data), etc., for enabling data transmission and reception. Further, the data transceiver can also be used to read data in the memory according to the physical address or write data to the memory, etc. It can be understood that the bus controller shown in FIG. 3 is only used for illustration by way of example, and in actual applications, the bus controller 213 and the bus controller 223 can include more hardware (not shown in FIG. 3), such as an encoder or a decoder, etc., wherein the encoder is used to convert data into a digital signal (for facilitating subsequent transmission of the digital signal), and the decoder is used to convert the received digital signal into data; or the bus controller 213 and the bus controller 223 can include other necessary constituent hardware, which will not be expanded here.

[0059] In each computing node, different devices can be connected through an internal bus of the chip. The internal bus can be, for example, an Open Core Protocol (OCP) bus, an advanced microcontroller bus architecture (AMBA) bus, or the like, and is not limited in this regard. The chip can be a system on chip (SoC), or the like.

[0060] In the computing cluster 200 shown in FIG. 2, each processor can access the bus as an independent computing node (when the processor is specifically an accelerator, each accelerator can access the bus as a host), and each computing node can directly access the memory in other computing nodes through the bus. When implementing, taking access of the computing node 220 by the computing node 210 as an example, the computing node 210 can generate a memory access request, and send the memory access request to the computing node 220 based on the bus by using a DMA engine 2131 in a bus controller 213.

[0061] Correspondingly, the computing node 220 receives the memory access request from the bus and responds to the memory access request. As an implementation example, when the memory access request applies to write data into the memory of the computing node 220, the computing node 220 can receive the memory access request from the bus by using a data transceiver 223, parse the data to be written from the memory access request, and determine the physical address of the storage space in the memory 222 to be accessed by the computing node 210 by using a bus memory management unit 2232, so that the data transceiver 223 can write the data into the memory 222 based on the physical address. When the memory access request applies to read data in the memory of the computing node 220, the computing node 220 can receive the memory access request from the bus by using the data transceiver 223, and determine the physical address of the storage space in the memory 222 to be accessed by the computing node 210 by using the bus memory management unit 2232; then, the data transceiver 223 can read the data required by the computing node 210 from the memory 222 based on the physical address, and send the data to the computing node 210 through the bus.

[0062] Since the computing node 210 is capable of directly accessing the memory in the computing node 220 based on the bus by using the DMA engine 2131, the network stack required to be executed by the computing node 210 can be effectively reduced, and the efficiency of the computing node 210 in accessing the memory can be effectively improved. Moreover, the computing node 210 and the computing node 220 access the same bus, and thus the computing node 210 is capable of directly sending a memory access request to the computing node 220 through the bus by using the DMA engine 2131 in the process of accessing the memory in the computing node 220, without forwarding the memory access request through other computing nodes, i.e., without performing a P2P forwarding process through other computing nodes, which can effectively reduce the overhead generated by the computing node 210 in accessing the memory across nodes, and can effectively improve the efficiency of the computing node 210 in accessing the memory. In this way, by separately connecting the CPU and the accelerator as computing nodes and connecting them to the bus, the performance of the processor in accessing the memory across nodes in the computing cluster 200 can be effectively improved.

[0063] In addition, in the computing cluster 200, the accelerator 211 in the computing node 210 only needs to be connected to the bus based on a set of ports to access the memory in other computing nodes, without the need for the accelerator 211 to be configured with multiple sets of ports, which can effectively reduce the hardware cost of the accelerator 211 and the difficulty of the hardware manufacturing process for the accelerator 211. At the same time, a single accelerator is capable of being connected to the bus as an independent computing node, which makes it possible for a user to flexibly configure the number of accelerators connected to the bus in the computing cluster 200 according to the actual application requirements, without the need for relying on the CPU (host) to be connected to the bus.

[0064] It is worth noting that the computing cluster 200 shown in FIG. 2 is only illustrative and is not intended to be limiting. For example, in other possible computing clusters, the bus memory management unit can also be deployed in the processor independently of the bus controller, i.e., the bus controller does not include the bus memory management unit, and in this case, the bus controller and the bus memory management unit can be connected through an internal bus. Alternatively, in other possible computing clusters, the number of processors included in each computing node can also be other numbers. Alternatively, in other possible computing clusters, on the basis of the computing cluster 200 shown in FIG. 2, other types of nodes can also be included, such as nodes for persistently storing data (such as storage servers), nodes for managing and controlling multiple computing nodes, and the like. Alternatively, in other possible computing clusters, the number of computing nodes can be other numbers, or each computing node can also include other hardware, or in addition to being connected through a bus, multiple computing nodes can also be connected through a communication network. Alternatively, in other possible computing clusters, a single computing node can include multiple processors, and the multiple processors can be processors of the same type, such as a computing node including multiple CPUs or including multiple accelerators, and the like. Alternatively, the multiple processors can be processors of different types, such as a computing node including both CPUs and accelerators, and the like.

[0065] For ease of understanding, embodiments of the memory access method provided in the present application are described below with reference to the accompanying drawings.

[0066] Referring to FIG. 4, FIG. 4 is a flowchart of a memory access method provided in an embodiment of the present application. The method can be applied to the computing cluster 200 shown in FIG. 2, or can be applied to other applicable computing clusters. For ease of illustration, in this embodiment, the method is exemplarily described by taking the case of being applied to the computing cluster 200 shown in FIG. 2 and writing data into the memory in the computing node 220 by the computing node 210.

[0067] Specifically, the memory access method shown in FIG. 4 can include the following steps.

[0068] S401: The computing node 220 sends a notification message to the computing node 210, and the notification message is used to indicate a storage space in the memory 222 of the computing node 220 that can be accessed by the computing node 210.

[0069] In this embodiment, the computing node 220 can allocate a storage space from the memory 222 to the computing node 210 to authorize the computing node 210 to access the storage space in the memory 222. Wherein, the computing node 210 accesses the memory 222, including writing data into the memory 222, or reading data from the memory 222.

[0070] In a possible implementation, the computing node 220 can include software, which can be an operating system in the computing node 220 or an application, and the software can run on the CPU 221 (specifically, a core in the CPU 221) in the computing node 220, and when running, the software can instruct the computing node 220 to authorize a certain size of storage space in the memory 222 to be accessed by the computing node 210, for example, the software can apply for a piece of storage space in the memory 222 from the operating system in the computing node 220 and authorize the computing node 210 to access the piece of storage space. For example, in a scenario of training an AI model by using the computing node 210 and the computing node 220, the software running in the computing node 220 can be an application for training the AI model, and the application can estimate the size of storage space required for gradient information for updating parameter values in the AI model, and according to the size of the storage space, the application can apply for a piece of storage space in the memory 222 from the operating system in the computing node 220 and authorize the computing node 210 to access the piece of storage space, so that the computing node 210 can subsequently obtain the gradient information for updating the parameter values by accessing the piece of storage space.

[0071] After determining the memory space (that is, a piece of storage space in the memory 222) authorized to be accessed by the computing node 210, the computing node 220 can register the memory space to the bus memory management unit 2232, for example, a new page table can be created according to a physical address of the memory space, and the page table records a mapping relationship between a virtual address and a physical address, the virtual address can be a unified bus address (UBA) for example, and the physical address can be a host physical address (HPA) for example. In other implementation examples, the bus memory management unit 2232 can also share a page table with the MMU when registering the memory, and in this case, the UBA is equal to a virtual address (VA). Further, the bus memory management unit 2232 can also create a token for the memory space, so that when the memory space is accessed by other computing nodes subsequently, the token (second token in the following) can be used to verify whether the other computing nodes have the permission to access the memory space.

[0072] Then, compute node 220 can generate a notification message based on the virtual address corresponding to the memory space and the token, and send the notification message to compute node 210. Compute node 220 can send the notification message to compute node 210 via a bus, or it can send it via a communication network. For example, compute node 220 can send the notification message to compute node 210 via a network interface card (NIC); correspondingly, compute node 210 can receive the notification message using its NIC. In this way, compute node 210 can determine the storage space in memory 222 of compute node 220 that it can access based on the received notification message.

[0073] Specifically, regarding the virtual address carried in the notification message, compute node 210 does not need to create a page table based on the UBA after receiving the notification message. That is, compute node 210 does not need to create a new page table to map the storage space indicated by the virtual address to a segment of local memory, thus eliminating the need for unified addressing of the storage location indicated by the virtual address in local memory. Consequently, during subsequent requests by compute node 210 to access the storage space in memory 222 indicated by the virtual address, there is no need to perform address translation locally for the virtual address, reducing one page table lookup process. This reduces the resource overhead of cross-node memory access and improves memory access efficiency.

[0074] S402: Compute node 210 generates a memory access request for writing data into memory 222 of compute node 220.

[0075] In practical applications, compute node 210 may write data from its local memory 212 to the memory 222 of compute node 220. For example, compute node 210 may synchronize gradient information generated during model training to compute node 220. In this case, compute node 210 can generate a memory access request containing the data to be written, so that the data can be written to compute node 220 subsequently through this memory access request.

[0076] To facilitate understanding, the implementation process of generating memory access requests in compute node 210 will be described below with reference to Figure 5a.

[0077] In the first implementation, as shown in FIG. 5a, the accelerator 211 can run software (running through a core in the accelerator 211) and a driver of the DMA engine 2131, and the software can invoke the driver of the DMA engine 2131 during running to submit an access request descriptor to a submission queue (SQ) of the DMA engine 2131, where the access request descriptor is used to indicate writing data to the memory 222 in the computing node 220. The SQ queue of the DMA engine 2131 is located in the memory 212, that is, the SQ queue can be constructed through a storage space in the memory 212. The access request descriptor submitted to the SQ queue can include an operation code (write operation), a virtual address of the data to be written in the memory 212 (that is, a virtual address of the data in the source memory), a first token (used for permission verification when accessing the memory 212), a length of the data to be written, a UBA of the memory 222 (that is, an input / output virtual address (IOVA) corresponding to a storage space in the memory 222), and a second token (used for permission authentication when accessing the memory 222). In actual application, before submitting the access request descriptor, the software can register the bus memory management unit 2132 according to the virtual address and the length of the data to be written in the memory 212 to obtain the first token for accessing the data to be written in the memory 212.

[0078] Then, as shown in FIG. 5a, the software can invoke the driver of the DMA engine 2131 to press the doorbell corresponding to the SQ queue to trigger the DMA engine 2131 to read the access request descriptor from the SQ queue. The doorbell is a trigger mode. The software pressing the doorbell of the SQ queue specifically means that the software invokes the driver of the DMA engine 2131 to write data to the doorbell register corresponding to the SQ queue, so that the DMA engine 2131 can trigger the operation of reading the access request descriptor from the SQ queue according to the value of the doorbell register, as shown in FIG. 5a.

[0079] Then, the DMA engine 2131 parses the read access request descriptor, determines the virtual address, the first token and the length of the data to be written in the memory 212, and requests the bus memory management unit 2132 to query the page table according to the virtual address and the first token to obtain the physical address of the data to be written in the memory 212, as shown in FIG. 5a. In the process of requesting the physical address, the bus memory management unit 2132 can verify whether the first token provided by the DMA engine 2131 is a legal token, which can specifically be judging whether the first token provided by the DMA engine 2131 is consistent with the first token generated when being registered in the bus memory management unit 2132 in advance, and determining that the first token provided by the DMA engine 2131 is a legal token when the two tokens are consistent. Then, the bus memory management unit 2132 queries the page table according to the virtual address to obtain the physical address corresponding to the virtual address, and feeds back the queried physical address to the DMA engine 2131. Thus, the DMA engine 2131 can access the memory 212 according to the physical address to obtain the data to be written, as shown in FIG. 5a.

[0080] Finally, the DMA engine 2131 generates a memory access request according to the accessed data to be written. The data field of the memory access request can carry the data. In addition, the generated memory access request can also include operation code, length of the data to be written, UBA of the memory 222, second token and other information.

[0081] Further, the memory access request can further include an identifier of the computing node 210 and an identifier of the computing node 220. For example, the identifier of the computing node 210 can be an identifier of the bus controller 213, the identifier of the computing node 220 can be an identifier of the bus controller 223, and the like. In actual application, in the process of the computing node 210 and the computing node 220 accessing the bus, the FM (structure manager) in the bus can enumerate each computing node accessing the bus, and uniformly issue configuration information to each computing node. The configuration information can include an entity identifier (EID) allocated to the bus controller in the computing node. The EID can uniquely identify the bus controller, and indirectly identify the computing node configured with the bus controller. In the scenario of the computing node 210 accessing the memory in the computing node 220, the memory access request generated in the computing node 210 can carry the EID of the bus controller 213 and the EID of the bus controller 223. The EID of the bus controller 213 is the source EID (SEID), and the EID of the bus controller 223 is the destination EID (DEID). The DMA engine 2131 can parse the DEID from the access request descriptor, and generate a memory access request including the DEID.

[0082] In addition, the memory access request can further include a network address of the computing node 220 (i.e., a destination network address), so as to transmit the memory access request to the computing node 220 according to the network address when transmitting the memory access request based on the bus. In actual application, the memory access request can further include a network address of the computing node 210 (i.e., a source network address). For example, in the process of the computing node 210 and the computing node 220 accessing the bus, the FM in the bus can configure a network address for each computing node. The network address can be a clan network address (CNA), or can be another type of network address, which is not limited. In the scenario of the computing node 210 accessing the memory in the computing node 220, the network address of the computing node 210 is the source network address, such as a source CNA (SCNA). For example, the network address of the computing node 210 can be a network address of an out port of the computing node 210, and the like. The network address of the computing node 220 is the destination network address, such as a destination CNA (DCNA).

[0083] In actual application, before accessing the memory in other computing nodes, the computing node 210 can establish a transmission connection with other computing nodes in advance, which can be a compact transmission (CTP) connection or the like, and establish a corresponding transmission connection entry for the transmission connection with each other computing node, which is used to record the mapping relationship between a transport identifier (TPID) and the network address of the other computing node, and the transport identifier is used to indicate the communication path between the computing node 210 and the other computing node. For the sake of distinction and description, the transport identifier used to indicate the communication path between the computing node 210 and the computing node 220 is referred to as a target transport identifier in this embodiment. The transmission connection entry can be stored in the DMA engine 2131, so that in the process of generating the memory access request, the DMA engine 2131 can parse the target transport identifier from the access request descriptor, and query the transmission connection entry according to the target transport identifier, so that the DMA engine 2131 can generate the memory access request according to the network address of the computing node 220 corresponding to the target transport identifier obtained by querying.

[0084] The transmission connection established by the computing node 210 can be a unilateral transmission connection, that is, it is not required to request other computing nodes to establish a transmission connection with the computing node 210. In addition, the transmission connection can be a stateless transmission connection, which can not have multiple states for indicating the quality of data transmission. In addition, the computing node 210 establishes the transmission connection entry according to the transport identifier, which can effectively reduce the influence of the change of the physical link on the software for shielding the perception of the computing node 210 to the physical link used to implement the communication path. That is, for the software in the computing node 210, whether the transmission connection with other computing nodes changes can be determined according to the transport identifier. In the case that the transport identifier remains unchanged, the software in the computing node 210 can consider that the transmission connection with other computing nodes is stable and unchanged. In this way, when the physical link used to transmit data between the computing node 210 and other computing nodes (such as the computing node 220) changes, such as the change of the physical channel or the number of channels, the software of the computing node 210 can not perceive, so as to reduce the influence of the change of the physical link on the software.

[0085] Exemplarily, the format of the memory access request generated by the DMA engine 2131 can be as shown in FIG. 6, including an UBL field, an NTH field, a TAH field, and a data field, etc. The UBL field refers to a unified bus link (UBL) field. The NTH field refers to a network header field, which can be used to carry the SCNA and the DCNA. The TAH field refers to a transaction header field, which can be used to carry the operation code, the length of the data to be written, the UBA of the memory 222, the second token, and other information. The data field is used to carry the data to be written to the computing node 220. In addition, the memory access request shown in FIG. 6 also carries the SEID and the DEID. It should be noted that in the first implementation mode described above, the memory access request is taken as an example that simultaneously includes the SEID, the DEID, the SCNA, and the DCNA. In actual application, the memory access request can also not carry the SEID and the DEID. At this time, the memory access request can carry the network address (such as the CNA) of the computing node 220, and can take the network address of the computing node 220 as the identifier of the computing node 220. In addition, the memory access request can also carry the network address of the computing node 210, and can take the network address as the identifier of the computing node 210. At the same time, the network address of the computing node 220 carried in the memory access request can also indicate the transmission of the memory access request in the bus (i.e., which computing node the memory access request is transmitted to). Alternatively, the computing node 220 can also be identified by the CNA and the EID, and the like, which is not limited herein.

[0086] It should be noted that the software described in this embodiment performs certain operations, which can be performed by the core in the accelerator 211 where the software runs.

[0087] In addition to the above implementation examples of generating the memory access request, other ways of generating the memory access request can also be used in the computing node 210.

[0088] For example, in the first implementation manner, the DMA engine 2131 is taken as an example for description, which parses the target transmission identifier and the EID of the destination from the access request descriptor. In the second implementation manner, the EID of the computing node 220 and the target transmission identifier can be bound to the SQ queue. In this way, after the DMA engine 2131 reads the access request descriptor from the SQ queue, the DMA engine 2131 can determine the EID and the target transmission identifier bound to the SQ queue according to the SQ queue, for example, reading the EID and the target transmission identifier from the context of the SQ queue. Thus, subsequently, the DMA engine 2131 can determine the network address (that is, the network address of the computing node 220) of the destination according to the target transmission identifier, and generate the memory access request including the EID and the network address of the computing node 220.

[0089] For another example, in the first implementation manner, the access request descriptor includes the virtual address of the data to be written in the memory 212 (and the first token is taken as an example for description. In the third implementation manner, the access request descriptor can also directly include the physical address (for example, the HPA) of the data to be written in the memory 212, so that the DMA engine 2131 can directly access the data to be written in the memory 212 based on the physical address, without the need for the memory 212 to perform the permission authentication process.

[0090] For another example, in the first implementation manner, the DMA engine 2131 is taken as an example for description, which queries the transmission connection table item to obtain the computing node 220 corresponding to the target transmission identifier. In the fourth implementation manner, the DMA engine 2131 can also not store the transmission connection table item, but can calculate the network address of the computing node 220 corresponding to the target transmission identifier based on a preset mapping rule according to the target transmission identifier parsed from the access request descriptor, so that the DMA engine 2131 can generate the memory access request according to the calculated network address of the computing node 220.

[0091] It should be noted that, in the second implementation manner to the fourth implementation manner, in addition to the differences in the implementation manners of the described part of the steps, the technical implementation of other part of the steps in the process of generating the memory access request is similar to that in the first implementation manner, and specific descriptions can be made with reference to the related descriptions in the first implementation manner, and details are not described herein.

[0092] S403: The computing node 210 sends the memory access request to the computing node 220 based on the bus by using the DMA engine 2131.

[0093] Since the computing node 210 and the computing node 220 are both connected to the same bus, after the computing node 210 generates the memory access request by using the DMA engine 2131, the memory access request can be sent to the data transceiver 2133, and the data transceiver 2133 can send the memory access request to the computing node 220 based on the bus.

[0094] In a possible implementation, the computing node 210 can be configured with multiple ports, such as the ports P0 to P n , and the computing node 210 can communicate data with different computing nodes based on different ports. For this purpose, the computing node 210 can be configured with a routing table, and the computing node 210 can select a corresponding port from the multiple ports to send the memory access request by looking up the routing table.

[0095] In a specific implementation, the bus controller 213 in the computing node 210 can obtain the network address of the destination, such as by querying the transport connection table item. Then, the bus controller 213 can query the routing table according to the network address of the destination, and the routing table can be used to record the mapping relationship between each network address and at least one port of the ports P0 to P n . In this way, the bus controller 213 can obtain at least one port that is mapped to the network address of the destination based on the routing table. Then, the bus controller 213 can select a port from the at least one port to send the memory access request, and the selected port is referred to as the target port (at this time, the target port can be one port) for the convenience of distinguishing and describing. Thus, the bus controller 213 can send the memory access request to the bus through the target port, so as to transmit the memory access request to the computing node 220 through the bus. The routing table in the bus controller 213 can be preconfigured by the FM in the bus, such as the configuration information issued by the FM to the computing node 210 can include the routing table (also including EID, network address, and other information). In this process, the operations performed by the bus controller 213 can be performed by the DMA engine 2131 and the data transceiver 2133 in the bus controller 213 cooperatively, or can be performed by the data transceiver 2133 alone, which is not limited.

[0096] S404: The computing node 220 receives the memory access request from the bus, and writes the to-be-written data carried in the memory access request into the memory 222.

[0097] In this embodiment, the computing device 220 can receive the memory access request from the bus, and write data into the memory 222 in response to the received memory access request, to achieve access to the memory 222.

[0098] In a possible implementation, as shown in FIG. 5b, in the bus controller 223 in the computing node 220, the data transceiver 2233 receives the memory access request transmitted through the bus through a port and parses the memory access request to obtain the operation code (write operation), data to be written, the UBA of the memory 222, the second token, the data length, the SEID, the DEID, the SCNA, the DCNA, and the like. In addition, the data transceiver 2233 can also record the port used when the memory access request is received.

[0099] Then, the data transceiver 2233 can access the memory 222 according to the operation code, the UBA of the memory 222, and the second token. Specifically, in the bus controller 223, the data transceiver 2233 can first determine the page table entry to be queried according to the DEID, and then request the bus memory management unit 2232 to query the page table according to the UBA and the second token to obtain the physical address of the storage space indicated by the UBA in the memory 222. In the process of requesting the physical address, the bus memory management unit 2232 can determine whether the second token provided by the data transceiver 2233 is consistent with the token (that is, the token sent to the computing node 210) created in advance by the bus memory management unit 2232, and determine that the second token provided by the data transceiver 2233 is a legal token when the two tokens are consistent, as shown in FIG. 5b. Then, the bus memory management unit 2232 queries the page table according to the UBA to obtain the physical address corresponding to the UBA, and feeds back the physical address obtained by the query to the data transceiver 2233. In this way, the data transceiver 2233 can access the memory 222 according to the physical address and the data length, and write data into the storage space indicated by the physical address according to the write operation indicated by the operation code, as shown in FIG. 5b.

[0100] In this way, the computing node 210 can use the DMA engine 2131 to write data into the memory in the computing node 220.

[0101] S405: The computing node 220 feeds back an acknowledgement message for the memory access request to the computing node 210 based on the bus.

[0102] As shown in FIG. 5b, after the computing node 220 successfully writes the data in the memory access request into the memory 222, the computing node 220 can feed back an acknowledgement message to the computing node 210, for example, the acknowledgement message can be sent through the port receiving the memory access request, and the like, to notify the computing node 210 that the data writing is successful.

[0103] As an implementation example, after completing the data write, the bus controller 223 (the data transceiver 2233 therein) can generate an acknowledgement message including a flag bit, the SEID, the DEID, the SCNA, and the DCNA. The flag bit is used to identify that the message is an acknowledgement message. The DEID in the acknowledgement message is the SEID in the memory access request (i.e., the EID of the bus controller 213), that is, the bus controller 223 takes the SEID in the memory access request as the DEID in the acknowledgement message. Similarly, the bus controller 223 takes the DEID in the memory access request as the SEID in the acknowledgement message, takes the SCNA in the memory access request as the DCNA in the acknowledgement message, and takes the DCNA in the memory access request as the DCNA in the acknowledgement message.

[0104] In other embodiments, after successfully writing the data to the memory 222, the computing node 220 can also not need to feed back the acknowledgement message to the computing node 210. In actual application scenarios, when the operation code in the memory access request indicates that the computing node 220 needs to feed back the acknowledgement message, the computing node 220 can feed back the acknowledgement message to the computing node 210 after completing the data write; and when the operation code in the memory access request indicates that no acknowledgement message needs to be fed back, the computing node 220 can not generate and send the acknowledgement message after completing the data write.

[0105] Further, in the case where the computing node 210 receives the acknowledgement message, the DMA engine 2131 can generate a completion notification message and write the completion notification message to the completion queue (CQ) of the DMA engine 2131, as shown in FIG. 5a. The CQ of the DMA engine 2131 is located in the memory 212, that is, the CQ can be constructed by using a storage space in the memory 212. Then, the DMA engine 2131 can notify the software to process the completion notification message in the CQ, for example, the DMA engine 2131 can generate an interrupt signal to trigger the software (or a hardware module) running on the accelerator 211 to process the completion notification message in the CQ, for example, the software can request to release the storage space occupied by the data in the memory 212, and the like. Alternatively, the software can periodically poll the CQ, so as to process the polled completion notification message when the CQ stores the completion notification message.

[0106] In other embodiments, in the case where the operation code in the memory access request indicates that no acknowledgement message needs to be waited for, the DMA engine 2131 can directly write the completion notification message to the CQ after sending the memory access request.

[0107] Thus, in the computing cluster 200, the computing node 210 can utilize the DMA engine 2131 to directly access the memory of the computing node 220 based on the bus to implement data writing, without performing a P2P forwarding process through other computing nodes, and can also effectively reduce the network stack required to be executed by the computing node 210, which can effectively reduce the overhead generated by the computing node 210 in accessing the memory across nodes, and can also effectively improve the efficiency of the computing node 210 in accessing the memory, thereby effectively improving the performance of accessing the memory across nodes in the computing cluster 200. Moreover, the accelerator in the computing cluster 200 can independently initiate an asynchronous memory write request without relying on the CPU, which can improve the flexibility of configuring the accelerator resources in the computing cluster 200.

[0108] It is worth noting that in the embodiment, the accelerator 211 accessing the memory of the CPU 221 is taken as an example for description (that is, the computing node 210 accessing the memory of the computing node 220). In an actual application scenario, in the computing cluster 200, the CPU 221 can also utilize the DMA engine 2231 to access the memory of the accelerator 211, or the accelerator 211 accessing the memory of another accelerator (as a separate computing node, such as the accelerator 231 in the computing node 230), or the CPU 221 accessing the memory of another CPU (as a separate computing node, such as the CPU 241 in the computing node 240), and the specific implementation manner is similar to the implementation manner of the accelerator 211 accessing the memory of the CPU 221, and specific reference can be made to the description of the related part above, and no further description is made herein.

[0109] Moreover, in the embodiment, the computing node 210 is taken as an example for description in sending the memory access request to the computing node 220 based on one port (that is, a target port). In other embodiments, the computing node 210 can also utilize multiple ports to send the to-be-written data in the memory access request to the computing node 210 in parallel.

[0110] As an implementation example, the memory access request generated by the DMA engine 2131 can include multiple sub-requests. For example, when the length of the data to be written exceeds the maximum length of data that can be carried by a single packet, the DMA engine 2131 can divide the data to be written into multiple parts, and generate a sub-request based on each data part, each sub-request carrying part of the data to be written, and different sub-requests carrying different parts of the data. In each sub-request, in addition to carrying the data, an operation code (write operation), UBA of the memory 222, a second token, data length, SEID, DEID, SCNA, DCNA, and the like can also be carried. Since different sub-requests carry different parts of the data, the UBA used to indicate the data write position in different sub-requests is different. For the implementation of the DMA engine 2131 generating each sub-request, refer to the description of the related part of the DMA engine 2131 generating the memory access request in step S402 above, which will not be repeated here.

[0111] After generating the multiple sub-requests, the DMA engine 2131 can determine the target port for the multiple sub-requests, at this time, the determined target port includes multiple ports, so that the DMA engine 2131 can use the multiple ports to send the multiple sub-requests in parallel based on the bus. In this way, the data to be written can be sent in parallel to the computing node 220 based on multiple physical links (each port corresponds to a physical link), such as sending multiple sub-requests through multiple ports, etc., which can effectively improve the transmission efficiency of data between the computing node 210 and the computing node 220, thereby further improving the performance of cross-node access to the memory in the computing cluster 200.

[0112] It is worth noting that in this embodiment, the access request descriptor includes the UBA used to indicate a storage region in the memory 222 as an example, and in other embodiments, the access request descriptor can also include other types of virtual addresses (which are used to indicate a storage region in the memory 222 that can be accessed by the computing node 210), such as a VA generated by the computing node 210 for uniform addressing of a storage region in the memory 222, etc., which is not limited.

[0113] In the embodiment shown in FIG. 4 above, the implementation process of the computing node 210 writing data into the memory of the computing node 220 is introduced. Next, in combination with FIG. 7, the process of the computing node 210 reading data from the memory of the computing node 220 is introduced.

[0114] Referring to FIG. 7, a flowchart of another memory access method is shown. As shown in FIG. 7, the method specifically includes the following steps.

[0115] S701: The computing node 220 sends a notification message to the computing node 210, the notification message being used to indicate the storage space in the memory 222 of the computing node 220 that can be accessed by the computing node 210.

[0116] Before the computing node 210 accesses the memory 222 of the computing node 220 for the first time, the computing node 220 can notify the computing node 210 of the storage space in the memory 222 that is allowed to be accessed by the computing node 210. In this way, the computing node 210 does not need to be additionally notified by the computing node 220 of the storage space in the memory 222 that is allowed to be accessed each time the computing node 210 requests to access the memory 222.

[0117] The specific implementation of step S701 is similar to the implementation of step S401 described above. For details, refer to the description of step S401 above, which will not be repeated here.

[0118] S702: The computing node 210 generates a memory access request, the memory access request being used to request to read data in the memory 222 of the computing node 220.

[0119] As an implementation example, when the software on the accelerator 211 needs to read the memory data in other computing nodes, the software (through the core in the accelerator 211) can call the driver of the DMA engine 2131 to submit an access request descriptor to the SQ queue of the DMA engine 2131, the access request descriptor being used to indicate to read data in the memory 222 of the computing node 220, as shown in FIG. 8a. The access request descriptor can include an operation code (read operation), the length of the data to be read, the UBA of the data to be read in the memory 222 (i.e., the IOVA corresponding to the memory space in the memory 222 where the data is stored), and a token (used for permission authentication when accessing the memory 222). Then, the software can call the driver of the DMA engine 2131 to ring the doorbell corresponding to the SQ queue, so as to trigger the DMA engine 2131 to read the access request descriptor from the SQ queue, as shown in FIG. 8a. Next, the DMA engine 2131 generates a memory access request according to the access request descriptor. Specifically, the DMA engine 2131 can first parse the access request descriptor, and then generate a memory access request including the operation code, the length of the data, the UBA of the data in the memory 222, and the token obtained by parsing.

[0120] Further, the memory access request can further include an identifier of the computing node 210 and an identifier of the computing node 220, such as an SEID and a DEID, etc., wherein the SEID is the EID of the bus controller 213 and the DEID is the EID of the bus controller 223. In addition, the memory access request can further include a network address of the computing node 220 and a network address of the computing node 210, such as an SCNA and a DCNA, etc., wherein the SCNA is the network address of the computing node 210 and the DCNA is the network address of the computing node 220. The implementation of the DMA engine 2131 generating the memory access request according to the access request descriptor is similar to the above-mentioned implementation, and details are described above and will not be repeated here.

[0121] In other embodiments, the memory access request generated by the computing node 210 can not include the EID, but can include a network address (such as CNA, etc.) of the computing node 220. At this time, the network address of the computing node 220 can be used not only to indicate that the memory access request is transmitted to the computing node 220 in the bus, but also as an identifier of the computing node 220. In addition, the memory access request can further include a network address of the computing node 210, and the network address can be used as an identifier of the computing node 210.

[0122] S703: The computing node 210 sends the memory access request to the computing node 220 based on the bus by using the DMA engine 2131.

[0123] Since the computing node 210 and the computing node 220 are both connected to the same bus, after the computing node 210 generates the memory access request, the computing node 210 can send the memory access request to the computing node 220 through the bus by using the DMA engine 2131.

[0124] In a first possible implementation, the computing node 210 can be configured with multiple ports, such as ports P0 to P n The computing node 210 can communicate data with different computing nodes based on different ports. In the computing node 210, the DMA engine 2131 can send the memory access request to the data transceiver 2133, and the data transceiver 2133 can query a pre-configured routing table to select a corresponding port from the multiple ports to send the memory access request. The routing table is used to record each network address and the port P0 to P nThe data transceiver 2133 can obtain a DCNA of the computing node 220, and query the routing table according to the DCNA to obtain a target port (which can be one port) that has a mapping relationship with the DCNA. Thus, the data transceiver 2133 can send the memory access request to the bus through the target port, so as to transmit the memory access request to the computing node 220 through the bus. The routing table in the computing node 220 can be preconfigured by the FM in the bus, for example, the FM pre-configures the computing node 210 with configuration information including the routing table (and can also include EID, network address, etc.). In other implementation manners, the DMA engine 2131 can query the routing table and determine the target port, etc., which is not limited herein.

[0125] In the second possible implementation manner, the memory access request generated by the DMA engine 2131 can include multiple sub-requests. For example, when the length of the data to be read is greater than the maximum length allowed by single reading, multiple sub-requests can be generated for the data to be read. Each sub-request is used to request to access part of the data in the memory 222, and the total amount of data accessed by the multiple sub-requests is the total data requested to be read by the software. Each sub-request can carry the operation code (read operation), UBA of the data in the memory 222, token, data length, SEID, DEID, SCNA, DCNA, etc. After the DMA engine 2131 generates multiple sub-requests, the data transceiver 2133 can determine the target port for the multiple sub-requests, and the determined target port includes multiple ports. Thus, the data transceiver 2133 can use the multiple ports to send multiple sub-requests in parallel based on the bus, for example, can send multiple sub-requests through multiple port spraying, etc.

[0126] S704: The computing node 220 receives the memory access request from the bus, and reads the data in the memory 222 according to the memory access request.

[0127] In this embodiment, the computing device 220 can receive the memory access request from the bus, and read the data in the memory 222 in response to the received memory access request, to realize the access to the memory 222.

[0128] In a possible implementation, as shown in FIG. 8b, in the bus controller 223 in the computing node 220, the data transceiver 2233 can receive the memory access request through a port, and parse the memory access request to obtain the operation code (read operation), the UBA of the data requested to be read in the memory 222, the token, the data length, the SEID, the DEID, the SCNA, the DCNA, and the like. In addition, the data transceiver 2233 can also record the port used when the memory access request is received.

[0129] Then, the data transceiver 2233 can access the memory 222 according to the operation code, the UBA of the data in the memory 222, and the token. Specifically, as shown in FIG. 8b, in the bus controller 223, the data transceiver 2233 can request the bus memory management unit 2232 to query the page table according to the UBA and the token, to obtain the physical address of the storage space where the data is stored in the memory 222. In the process of requesting the physical address, the bus memory management unit 2232 can determine whether the token provided by the data transceiver 2233 is consistent with the token (that is, the token previously sent to the computing node 210 in step S701) previously created by the bus memory management unit 2232, and determine that the second token provided by the data transceiver 2233 is a legal token when the two tokens are consistent. Then, the bus memory management unit 2232 queries the page table according to the UBA to obtain the physical address corresponding to the UBA, and feeds back the physical address obtained by querying to the data transceiver 2233. In this way, the data transceiver 2233 can access the memory 222 according to the physical address and the data length, and read the data (that is, the data required by the software) indicated by the physical address and the data length according to the read operation indicated by the operation code, as shown in FIG. 8b.

[0130] S705: The computing node 220 feeds back a response message for the memory access request to the computing node 210 based on the bus.

[0131] When the memory access request is one request, the computing node 220 can generate at least one response message, and the at least one response message includes all the data requested to be read by the software. When the memory access request includes multiple sub-requests, the computing node 220 can read part of the data in the memory 222 based on each sub-request, and generate a response message according to the part of the data, so as to generate multiple response messages based on the multiple sub-requests.

[0132] As an implementation example, after reading the data in the memory 222, the data transceiver 2233 can generate a response packet based on the read data, the response packet including a flag bit, the read data, the SEID, the DEID, the SCNA, and the DCNA. The value of the flag bit is used to identify that the packet is a response packet. The DEID in the response packet is the SEID in the memory access request (i.e., the EID of the bus controller 213), that is, the data transceiver 2233 will take the SEID in the memory access request as the DEID in the confirmation packet. Similarly, the data transceiver 2233 will take the DEID in the memory access request as the SEID in the response packet, take the SCNA in the memory access request as the DCNA in the response packet, and take the DCNA in the memory access request as the SCNA in the response packet. For example, the SCNA in the response packet can be the network address of the out port of the computing node 220, etc. Then, the data transceiver 2233 can send the response packet to the computing node 210 through the port receiving the memory access request, as shown in FIG. 8b. Accordingly, the computing node 210 can receive the response packet based on the target port.

[0133] Also, when the memory access request includes multiple sub-requests, the data transceiver 2233 can receive the multiple sub-requests through multiple ports and send multiple response packets to the computing node 210 through the multiple ports in parallel, each sub-request corresponding to at least one response packet. In this way, the computing node 210 can trigger the computing node 220 to send multiple response packets in parallel using multiple ports by generating multiple sub-requests, that is, trigger the computing node 220 to send data required by the software in parallel using multiple ports, thereby effectively improving the efficiency of data transmission from the computing node 220 to the computing node 210, and thus improving the performance of cross-node memory access in the computing cluster 200.

[0134] Further, in the case that the computing node 210 receives the response message, the DMA engine 2131 in the computing node 210 can request a physical address in the memory 222 from the bus memory management unit 2132. Wherein, the implementation manner that the DMA engine 2131 requests the physical address is similar to the foregoing implementation manner of requesting the physical address, and details can be referred to the description in the related part, and details are not described herein. Then, the DMA engine 2131 can write the data carried in the response message into the storage position indicated by the physical address in the memory 212 according to the physical address. And after writing the data into the memory 212, the DMA engine 2131 can generate a completion notification message and write the completion notification message into the CQ queue of the DMA engine 2131, as shown in FIG. 8a. Then, the DMA engine 2131 can notify the software (or the hardware module) to process the completion notification message in the CQ queue, for example, the software can read the required data from the memory 212. Or, the software can periodically poll the CQ queue, so as to process the polled completion notification message when the completion notification message is stored in the CQ queue.

[0135] Thus, in the computing cluster 200, the computing node 210 can directly access the memory of the computing node 220 based on the bus by using the DMA engine 2131 to implement data reading, without performing the P2P forwarding process through other computing nodes, and can effectively reduce the network stack required to be executed by the computing node 210, which can effectively reduce the overhead generated by the computing node 210 in accessing the memory across nodes, and can effectively improve the efficiency of the computing node 210 in accessing the memory, thereby effectively improving the performance of accessing the memory across nodes in the computing cluster 200.

[0136] It should be noted that, according to the above description, other reasonable combinations of steps that can be thought of by those skilled in the art also belong to the protection scope of the present application. Secondly, those skilled in the art should be familiar that the embodiments described in the specification all belong to preferred embodiments, and the actions involved are not necessarily all required by the present application.

[0137] The embodiments of the present application also provide a computer readable storage medium. The computer readable storage medium can be any available medium or data storage device including one or more available media that at least one computing device can store. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk) and the like. The computer readable storage medium includes instructions indicating at least one computing device to execute the foregoing memory access method.

[0138] The embodiments of the present application further provide a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on at least one computing device, the processes or functions described in the embodiments of the present application are wholly or partially produced.

[0139] The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website site, computer or data center to another website site, computer or data center through wired (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.).

[0140] The computer program product can be a software installation package, which can be downloaded and executed on at least one computing device in any method requiring the aforementioned memory access method.

[0141] The above embodiments can be wholly or partially implemented by software, hardware, firmware or any combination thereof. When implemented by software, the above embodiments can be wholly or partially implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are wholly or partially produced. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center through wired (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.). The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center and the like containing a set of one or more available media. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD) or a semiconductor medium. The semiconductor medium can be a solid state disk.

[0142] The terminology used in the above description merely describes specific embodiments and is not intended to limit the application. As used in this application and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. As used herein, the term "if' can be construed to mean "when" or "upon" or "in response to the occurrence of" unless context indicates otherwise. Stated differently, the term "if' can be construed to simply mean "during" or "while" or "when" or "in response to the occurrence of" unless context indicates otherwise. As used herein, the terms "first," "second," and the like, do not imply any particular order, but are used for naming purposes only. Moreover, the use of the terms first and second to describe a particular object or action must not be construed to imply that one is ordered in time before the other, but merely that they are two different objects or actions. It is to be understood that the terms so used are interchangeable under appropriate circumstances and embodiments.

[0143] Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," and the like in the specification do not necessarily all refer to the same group of embodiments, unless otherwise indicated. The terms "including," "comprising," "having," and variations thereof, mean "including but not limited to," unless expressly specified otherwise.

[0144] The above description is embodied in the form of specific examples, but the scope of the application is not limited thereto. Any skilled person in the art can easily conceive various equivalent modifications or replacements within the technical scope disclosed in the application, and these modifications or replacements should be included in the scope of the application. Therefore, the scope of the application should be determined by the scope of the claims.

Claims

1. A computing cluster, characterized by, The computing cluster comprises a plurality of computing nodes connected through a same bus, each of the plurality of computing nodes comprising a processor and a memory, the processor comprising a direct memory access (DMA) engine, the plurality of computing nodes comprising a first computing node and a second computing node; The first computing node is configured to generate a memory access request and send the memory access request to the second computing node based on the bus by using the DMA engine, the memory access request being configured to request access to a memory in the second computing node; The second computing node is configured to receive the memory access request from the bus and respond to the memory access request.

2. The computing cluster of claim 1, wherein, The first computing node is configured to: generate an access request descriptor, the access request descriptor comprising a unified bus address (UBA), the UBA being configured to indicate a storage space in the memory of the second computing node that can be accessed by the first computing node; and generate the memory access request by using the DMA engine according to the access request descriptor, the memory access request comprising the UBA.

3. The computing cluster of claim 2, wherein, The first computing node is configured to: add the access request descriptor to a submission queue in the memory of the first computing node; and generate the memory access request by using the DMA engine according to the access request descriptor in the submission queue. The first computing node is further configured to: generate a completion notification message corresponding to the access request descriptor after sending the memory access request; and submit the completion notification message to a completion queue in the memory of the first computing node.

4. The computing cluster according to claim 2 or 3, c h a r a c t e r i z e d in that, The processor in the first computing node comprises a first bus controller, the first bus controller comprising the DMA engine, and the first computing node generates the memory access request by using the DMA engine in the first bus controller.

5. The computing cluster of claim 4, wherein, The processor in the second computing node comprises a second bus controller, and the memory access request comprises an identifier of the first bus controller and an identifier of the second bus controller.

6. The computing cluster of claim 5, wherein, The access request descriptor further comprises the identifier of the second bus controller.

7. The computing cluster according to any of claims 1 to 6, characterized in that, The first computing node comprises a transmission connection table item and a routing table item, the transmission connection table item being configured to record a mapping relationship between a target transmission identifier and a network address of the second computing node, the target transmission identifier being configured to indicate a communication path between the first computing node and the second computing node, and the routing table item being configured to record a mapping relationship between a target port of the first computing node and the network address of the second computing node; The first computing node is configured to query the transmission connection table item to obtain the network address of the second computing node; and query the routing table item according to the network address of the second computing node to obtain the target port. The first computing node is configured to send the memory access request to the second computing node based on the bus by using the DMA engine and the target port.

8. The computing cluster of claim 7, wherein, The memory access request comprises a plurality of sub-requests, and the target port comprises a plurality of ports. The first computing node is configured to send the plurality of sub-requests in parallel based on the bus by using the plurality of ports.

9. The computing cluster of claim 8, wherein, Each of the plurality of sub-requests is configured to request reading data in the memory of the second computing node. The first computing node is further configured to receive a plurality of response messages, which are generated by the second computing node in response to the plurality of sub-requests respectively.

10. A memory access method, characterized by, The method comprises: The first computing node generates a memory access request, the first computing node comprises a processor, the processor in the first computing node comprises a direct memory access (DMA) engine, the first computing node is connected to a second computing node through a bus, and the second computing node comprises a processor and a memory. The first computing node sends the memory access request to the second computing node based on the bus by using the DMA engine, and the memory access request is configured to request accessing the memory in the second computing node.

11. The method of claim 10, wherein, The method further comprises: The second computing node receives the memory access request from the bus. The second computing node responds to the memory access request, and the second computing node and the first computing node are two computing nodes in a plurality of computing nodes included in a computing cluster, and the plurality of computing nodes are connected through the bus.

12. The method according to claim 10 or 11, characterized in that, The first computing node generates a memory access request, comprising: The first computing node generates an access request descriptor, and the access request descriptor comprises a unified bus address (UBA), and the UBA is configured to indicate a storage space in the memory of the second computing node that can be accessed by the first computing node. The first computing node generates the memory access request by using the DMA engine according to the access request descriptor, and the memory access request comprises the UBA.

13. The method of claim 12, wherein, The first computing node generates the memory access request according to the access request descriptor by using the DMA engine, comprising: The first computing node adds the access request descriptor to a submission queue in the memory of the first computing node. The first computing node generates the memory access request according to the access request descriptor in the submission queue by using the DMA engine. The method further comprises: After sending the memory access request, the first computing node generates a completion notification message corresponding to the access request descriptor; The first computing node submits the completion notification message to a completion queue in the memory of the first computing node.

14. The method according to claim 12 or 13, characterized in that, The processor in the first computing node comprises a first bus controller, and the first bus controller comprises the DMA engine. The first computing node generates a memory access request, comprising: The first computing node generates the memory access request by using the DMA engine in the first bus controller.

15. The method of claim 14, wherein, The processor in the second computing node comprises a second bus controller, and the memory access request comprises an identifier of the first bus controller and an identifier of the second bus controller.

16. The method of claim 15, wherein, The access request descriptor further comprises the identifier of the second bus controller.

17. The method according to any one of claims 10 to 16, characterized in that, The first computing node comprises a transmission connection table item and a routing table item, the transmission connection table item is used to record a mapping relationship between a target transmission identifier and a network address of the second computing node, the target transmission identifier is used to indicate a communication path between the first computing node and the second computing node, and the routing table item is used to record a mapping relationship between a target port of the first computing node and the network address of the second computing node. The first computing node sends the memory access request to the second computing node based on the bus by using the DMA engine, and the method comprises the steps that: The first computing node queries the transmission connection table item to obtain the network address of the second computing node; The first computing node queries the routing table item according to the network address of the second computing node to obtain the target port; The first computing node sends the memory access request to the second computing node based on the bus by using the DMA engine and the target port.

18. The method of claim 17, wherein, The memory access request comprises a plurality of sub-requests, and the target port comprises a plurality of ports. The first computing node sends the memory access request to the second computing node based on the bus by using the DMA engine and the target port, and the method comprises the steps that: The first computing node sends the plurality of sub-requests based on the bus by using the DMA engine and the plurality of ports.

19. The method of claim 18, wherein, Each of the plurality of sub-requests is used to request reading data in the memory of the second computing node. The method further comprises the steps that: The first computing node receives a plurality of response packets, and the plurality of response packets are packets generated by the second computing node in response to the plurality of sub-requests.

20. A computing node, characterized in that, The computing node is connected with other computing nodes through the same bus, and the computing node comprises a processor and a memory, and the processor comprises a direct memory access (DMA) engine. The computing node is used to generate a memory access request and send the memory access request to the other computing nodes based on the bus by using the DMA engine, and the memory access request is used to request accessing the memory in the other computing nodes.

21. A computer-readable storage medium, characterized in that, The instructions, when executed on at least one computing device, cause the at least one computing device to perform the steps of the method of any one of claims 10 to 19.

22. A computer program product comprising instructions, wherein: The instructions, when executed on at least one computing device, cause the at least one computing device to perform the steps of the method of any one of claims 10 to 19.

Images