Remote Direct Memory Access System, Device, and Cluster

Abstract

The present application provides a remote direct memory access system, device, and cluster. The system includes: when there is no first physical address corresponding to the first virtual address, the remote direct memory access device sends a first address conversion request to the host; the host queries the first physical address corresponding to the first virtual address in the host's memory according to the first address conversion request, and when the first physical address corresponding to the first virtual address cannot be queried, returns a conversion failure response to the remote direct memory access device; the remote direct memory access device sends a page request to the host based on the conversion failure response; the host migrates and locks the data corresponding to the first virtual address from the host's hard disk into the host's memory based on the page request, and returns a page request response; the remote direct memory access device stores the correspondence between the first virtual address and the first physical address in the address conversion list based on the page request response.

Description

Technical Field

[0001] This application relates to the field of computers, and particularly to a remote direct memory access system, device, and cluster. Background Art

[0002] A remote direct memory access device can access the entire physical address space of a host through remote direct memory access. For example, it can access memory areas such as the code segment and data segment of an operating system. To protect the system and achieve isolation in the address space, an input / output memory management unit (IOMMU) can be used to restrict the remote direct memory access device to only access specified memory areas and to aggregate multiple scattered DMA operations into continuous DMA operations. However, performing address translation through the IOMMU increases the access time of DMA operations. Summary of the Invention

[0003] This application provides a remote direct memory access system, device, and cluster that can avoid frequently loading data from the host's hard disk to the host's memory and unloading it from the memory to the hard disk due to the need for address translation.

[0004] In a first aspect, a remote direct memory access system is provided, including:

[0005] A remote direct memory access device, configured to query whether there is a first physical address corresponding to the first virtual address in an address translation list according to the first virtual address carried in a first remote direct memory access request. In the case where there is a first physical address corresponding to the first virtual address, replace the first virtual address in the first remote direct memory access request with the first physical address. In the case where there is no first physical address corresponding to the first virtual address, send a first address conversion request to the host, where the first address conversion request carries the first virtual address;

[0006] A host, configured to receive the first address conversion request, query the first physical address corresponding to the first virtual address in the input / output memory management unit of the host according to the first address conversion request, and return a conversion failure response to the remote direct memory access device in the case where the first physical address corresponding to the first virtual address cannot be queried;

[0007] A remote direct memory access device, configured to receive the conversion failure response and send a page request to the host based on the conversion failure response, where the page request includes the first virtual address;

[0008] A host, configured to receive the page request. Based on the page request, the operating system of the host migrates the data corresponding to the first virtual address from the hard disk of the host to the memory of the host, and returns a page request response, where the page request response includes the first physical address;

[0009] A Remote Direct Memory Access (RDMA) device, configured to receive the page request response, and store the correspondence between the first virtual address and the first physical address in the address translation list based on the page request response.

[0010] In the above solution, when the RDMA device sends an address translation request to the host, if the address is not in the memory but in the hard disk, the host will return a conversion failure response to the RDMA device. After receiving the conversion failure response sent by the host, the RDMA device will send a page request to the host again. The operating system of the host migrates the data corresponding to the first virtual address from the hard disk of the host to the memory of the host based on the page request. The above method ensures that the page request is responded to by the operating system of the host, avoiding the situation that when the RDMA device sends an address translation request to the host, if the address is not in the memory but in the hard disk, the driver of the host migrates the data corresponding to the first virtual address from the hard disk of the host to the memory of the host through a specific function, thereby locking the data in the memory. Once the user forgets to release the data, it will cause the memory to be occupied all the time. In this application, the operating system swaps the required physical address into the physical memory and updates the page table of the IOMMU after receiving the request, but does not fix these physical addresses, and may still swap them out to the hard disk later. In scenarios such as Graphics Processing Unit (GPU) / Artificial Intelligence (AI) that require access to dynamic memory, such usage increases the flexibility of memory management. The operating system only ensures that the page exists when responding and uses memory on demand, without the overhead caused by fixing physical addresses.

[0011] In some possible embodiments, a host, configured to send an address invalidation request to the RDMA device, where the address invalidation request includes the first virtual address;

[0012] A Remote Direct Memory Access (RDMA) device, configured to delete the correspondence between the first virtual address and the first physical address in the address translation list according to the address invalidation request, and send an invalidation completion response to the host.

[0013] In some possible embodiments, a Remote Direct Memory Access (RDMA) device is configured to send a second RDMA request to the host between receiving the address invalidation request and the invalidation completion response, where the second RDMA request includes a second virtual address.

[0014] In the above solution, when any address in the address translation list needs to be invalidated, address translation is no longer performed. Instead, the second RDMA request is directly passed through to the host, thus avoiding errors caused by misusing the invalidated address.

[0015] In some possible embodiments, a Remote Direct Memory Access (RDMA) device is configured to, between receiving the address invalidation request and the invalidation completion response, query the address translation list to determine whether there is a second physical address corresponding to the second virtual address carried in the second RDMA request. If there is a second physical address corresponding to the second virtual address, the second RDMA request is sent to the host.

[0016] In the above solution, when any address in the address translation list needs to be invalidated, even if another address is to be queried and the address exists in the address translation list, address translation is no longer performed. Instead, the second RDMA request is directly passed through to the host, thus avoiding errors caused by misusing the invalidated address.

[0017] In some possible embodiments, a Remote Direct Memory Access (RDMA) device is configured to merge the second RDMA request and the invalidation completion response into the same message and send the message to the host.

[0018] In the above solution, the invalidation completion response is preferentially scheduled, and the second RDMA request and the invalidation completion response are merged into the same message, thereby reducing the amount of data to be transmitted.

[0019] In some possible embodiments, a Remote Direct Memory Access (RDMA) device is configured to store the first address translation request in the cache of the RDMA device after sending the first address translation request to the host, query the address translation list to determine whether there is a first physical address corresponding to the first virtual address carried in the third RDMA request. If there is no first physical address corresponding to the first virtual address and the first address translation request is stored in the cache of the RDMA device, generation of the current address translation request is prohibited.

[0020] In the above solution, when the first address translation request is stored in the cache of the remote direct memory access device, the first address translation request will not be sent again, thus avoiding issuing multiple address translation requests for the same address and causing unnecessary bandwidth waste.

[0021] In some possible embodiments, the remote direct memory access device is used to store the page request in a first-in, first-out queue.

[0022] In some possible embodiments, the remote direct memory access device is used to store the page request in the first-in, first-out queue when the first-in, first-out queue is not full, and discard the page request when the first-in, first-out queue is full.

[0023] In the above solution, if the first-in, first-out queue is not full, the page request is stored in the first-in, first-out queue, and if the first-in, first-out queue is full, the page request is discarded, thus avoiding occupying the communication resources of the remote direct memory access request due to excessive page requests.

[0024] In a second aspect, a computing device is provided, including a remote direct memory access system and a storage unit, where communication can be performed between the remote direct memory access system and the storage unit, and the remote direct memory access system is the system according to any one of the first aspect.

[0025] In a third aspect, a computing device cluster is provided, including a plurality of computing devices, at least one of which includes a remote direct memory access system and a storage unit, where communication can be performed between the remote direct memory access system and the storage unit, and the remote direct memory access system is the system according to any one of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the background art, the drawings required for use in the embodiments of the present invention or the background art will be described below.

[0027] Figure 1 It is a schematic structural diagram of a remote direct memory access system provided by this application;

[0028] Figure 2 It is a schematic structural diagram of another remote direct memory access system provided by this application;

[0029] Figure 3 It is a schematic flowchart of a remote direct memory access method provided by this application;

[0030] Figure 4 It is a schematic structural diagram of a computing device provided by this application. Detailed Implementation Manner

[0031] The embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention. The terms used in the embodiments of the present invention are only used to explain the specific embodiments of the present invention, rather than intended to limit the present invention.

[0032] Refer to Figure 1 , Figure 1 which is a schematic structural diagram of a remote direct memory access system provided by this application. As Figure 1 shown, the remote direct memory access system of this application includes: a host 110 and a remote direct memory access device 120.

[0033] The remote direct memory access device 120 can be a device that has the ability to access the memory of the host through remote direct memory access. Remote direct memory access (English: remote direct memory access, RDMA) is a technology that bypasses the operating system kernel of the remote host to access the data in its memory. Since it does not go through the operating system, it not only saves a large amount of CPU resources, but also improves the system throughput and reduces the network communication latency of the system. It is especially suitable for wide application in large-scale parallel computer clusters. RDMA can transmit the data in the storage space of the storage device as a message in the form of an RDMA packet to the storage space of the host through the front-end network, or transmit the data in the storage space of the host as a message in the form of an RDMA packet to the storage space of the storage device through the front-end network. These transmission and copying operations are mainly performed by the network interface cards of the storage device and the host (for example, RDMA network interface card (RDMA network interface card, RNIC)), and do not require the processor to execute, so it does not affect the performance of the host client. Currently, the hardware implementation methods of RDMA include InfiniBand (IB), RDMA over converged ethernet (RoCE) (including RoCEv1 and RoCEv2), internet wide area RDMA protocol (iWARP), etc. Among them, InfiniBand is a network designed specifically for RDMA, which ensures reliable transmission at the hardware level. Both RoCE and iWARP are RDMA technologies based on Ethernet, support the corresponding Verbs interface, and RoCE also supports reliable transmission guaranteed at the hardware level. The remote direct memory access device can include a network card, a storage device, an accelerator, a gateway, and so on.

[0034] The host 110 may refer to a computing device or system that can provide resources, services, or run applications. The host can include servers, desktop computers, laptops, tablets, smartphones, and so on. It includes memory, a memory controller, and multiple processing units. Among them, the memory, the memory controller, and the multiple processing units are interconnected through an on-chip bus. The processing unit can be a central processing unit (CPU) for processing instructions and data stored in the memory. The processing units on a node can quickly access the local memory of that node through the on-chip bus and can also access the memory of other nodes through a high-speed interconnect network. For example, the processing unit 0 of node 0 can access the memory of node 0 or the memory of node 1. Compared with accessing the memory of the same node, it takes more time for the processor to access the memory of other nodes. Therefore, for a Non-Uniform Memory Access (NUMA) system, the memory within the same node as the processing unit should ideally contain the information most relevant to that processing unit. In one implementation, the processing unit can be a multi-core chip, that is, a chip containing multiple processing cores. In another implementation, the processing unit can be a chip with one processing core. Additionally, in some embodiments, multiple nodes of the NUMA system are located on the same chip. Memory refers to the internal memory that directly exchanges data with the processing unit. It can read and write data at any time and is very fast, serving as the temporary data storage for the operating system or other running programs. Memory includes at least two types of memories. For example, memory can be either random access memory or read-only memory (ROM). For example, the random access memory is dynamic random access memory (DRAM) or storage class memory (SCM). DRAM is a semiconductor memory and, like most random access memories (RAM), belongs to a volatile memory device. SCM is a composite storage technology that combines the characteristics of traditional storage devices and memories. Storage class memory can provide faster read and write speeds than hard disks, but its access speed is slower than that of DRAM, and its cost is also cheaper than that of DRAM. However, DRAM and SCM are only exemplary descriptions in this embodiment, and memory can also include other random access memories, such as static random access memory (SRAM), etc.For read-only memory, for example, it can be Programmable Read Only Memory (PROM), Erasable Programmable Read Only Memory (EPROM), etc. In addition, the memory can also be a Dual In-line Memory Module (DIMM), that is, a module composed of Dynamic Random Access Memory (DRAM), or it can be a Solid State Disk (SSD). In practical applications, multiple memories and different types of memories can be configured in the NUMA system. The number and type of memories are not limited in this embodiment. In addition, the memory can be configured to have a power retention function. The power retention function means that when the system loses power and then powers on again, the data stored in the memory will not be lost. The memory with the power retention function is called non-volatile memory. The memory controller is used to manage and plan the data transfer from the memory to the processing unit, and it can be a separate chip or integrated in the chip of the processing unit.

[0035] As Figure 2 shown, the Remote Direct Memory Access device can include: a service module and an address translation module. Among them, the address translation module includes a look-up table module, a cache management module, a invalidate request processing module, an address translation information cache module, and an address translation request initiation module.

[0036] The address translation information cache module is used to store the address translation list, and one or more corresponding relationships can be stored in the address translation list. Each corresponding relationship is used to indicate the corresponding relationship between a pair of virtual addresses and physical addresses. Among them, the virtual address here refers to the virtual address of the host's memory, and the physical address here refers to the physical address of the host's memory. The virtual address space of the host's memory is divided into multiple virtual pages, and the physical address space of the host's memory is also divided into multiple physical pages. Then, the mapping relationship between the virtual page and the physical page can be recorded through the page table. For example, the address translation list can include the corresponding relationship between the first virtual address and the first physical address, the corresponding relationship between the second virtual address and the second physical address, and so on.

[0037] The service module sends a first Remote Direct Memory Access (RDMA) request to the look-up table module. The first virtual address carried by the first RDMA request is the virtual address of the memory of the host to be accessed. After receiving the first RDMA request sent by the service module, the look-up table module obtains the first virtual address from the first RDMA request and sends a look-up request for the first virtual address to the cache management module. After receiving the look-up request, the cache management module sends a cache read enable to the address translation information cache module, enabling the address translation information cache module to be in a state where it can be read. Then, the cache management module queries the address translation list based on the first virtual address.

[0038] If the cache management module queries and finds that the first physical address corresponding to the first virtual address is stored in the address translation information cache module, after obtaining the first physical address, the cache management module sends the first physical address to the look-up table module through the look-up result. After receiving the first physical address, the look-up table module replaces the first virtual address in the first RDMA request with the first physical address, and then sends the replaced first RDMA request to the host. After receiving the first RDMA request, the host obtains the first physical address from the first RDMA request and uses the first physical address to access the host's memory.

[0039] If the cache management module queries and finds that the first physical address corresponding to the first virtual address is not stored in the address translation information cache module, the cache management module sends the first virtual address to the address translation request initiation module. The address translation request initiation module sends a first address translation request to the host, where the first virtual address is carried in the first address translation request. After receiving the first virtual address, the host can find the corresponding first physical address in the memory through the Input / Output Memory Management Unit (IOMMU).

[0040] If the host finds the first physical address corresponding to the first virtual address, it indicates that the data corresponding to the first virtual address has been relocated from the hard disk to the physical memory of the host. The host sends a conversion success response to the address conversion request initiation module in the remote direct memory access device. The conversion success response includes the first physical address. After receiving the conversion success response, the address conversion request initiation module in the remote direct memory access device obtains the first physical address from the conversion success response and sends the first physical address to the cache management module. The cache management module sends a cache write enable to the address conversion information cache module to enable the address conversion information cache module to process the write-allowed state. Then, the cache management module writes the correspondence between the first virtual address and the first physical address into the address conversion information cache module.

[0041] If the host fails to find the first physical address corresponding to the first virtual address, it indicates that the data corresponding to the first virtual address has not been relocated from the hard disk to the physical memory of the host. The host sends a conversion failure response to the address conversion request initiation module in the remote direct memory access device. The address conversion request initiation module in the remote direct memory access device sends the first virtual address to the page request initiation module. The page request initiation module generates a page request based on the first virtual address and checks whether the first-in-first-out queue is full. If the first-in-first-out queue is not full, the page request is stored in the first-in-first-out queue. If the first-in-first-out queue is full, the page request is discarded. The page request initiation module sends the page request to the host. After receiving the page request, the host relocates the data corresponding to the first virtual address from the hard disk to the memory and determines the first physical address corresponding to the first virtual address. Then, the host sends a page request response to the page request initiation module in the remote direct memory access device. The page request response includes the first physical address. The page request initiation module in the remote direct memory access device sends the first physical address to the address conversion request initiation module. The address conversion request initiation module sends the first physical address to the cache management module. The cache management module sends a cache write enable to the address conversion information cache module to enable the address conversion information cache module to process the write-allowed state. Then, the cache management module writes the correspondence between the first virtual address and the first physical address into the address conversion information cache module.

[0042] Optionally, the host may send an invalidation request to the invalidation request processing module of the remote direct memory access device. The address invalidation request includes a first virtual address. The invalidation request processing module of the remote direct memory access device sends an invalidation request to the cache management module. After receiving the invalidation request, the cache management module sends a cache write enable to the address translation information cache module, so that the address translation information cache module is in a state where it can be written. The corresponding relationship between the first virtual address and the first physical address is found in the address translation table of the address translation information cache module according to the first virtual address, and the corresponding relationship between the first virtual address and the first physical address is deleted. Then, the cache management module sends an invalidation completion response to the lookup module. Between the time when the cache management module receives the invalidation request and the cache management module sends the invalidation completion response, the cache management module does not respond to any lookup requests received. Since when invalidating a certain corresponding relationship, if the management module is allowed to query the address translation table from each other, then it is possible to just query the corresponding relationship that needs to be invalidated, thus obtaining an incorrect physical address. Therefore, between the time when the cache management module receives the invalidation request and the cache management module sends the invalidation completion response, if the service module sends a second remote direct memory access request (including a second virtual address) to the lookup module, then even if the address translation information cache module stores the corresponding relationship between the second virtual address and the second physical address, the lookup module will not send a lookup request to the cache management module, but directly send the second remote direct memory access request (including the second virtual address) to the host. Since the invalidation completion response can be scheduled preferentially, the lookup module receives the second remote direct memory access request first, and then receives the invalidation completion response. However, the invalidation completion response can be sent to the host first, and then the second remote direct memory access request can be sent. Or, the invalidation completion response and the second remote direct memory access request can be combined into the same message and sent to the host.

[0043] Optionally, after the address translation request initiation module of the remote direct memory access device sends the first address translation request to the host, it stores the first address translation request in the cache of the remote direct memory access device. The service request module sends a third remote direct memory access request to the table lookup module, where the third remote direct memory access request carries a first virtual address. After the table lookup module receives the third remote direct memory access request sent by the service module, the table lookup module obtains the first virtual address from the third remote direct memory access request and sends a table lookup request for the first virtual address to the cache management module. After the cache management module receives the table lookup request, it sends a cache read enable to the address translation information cache module, enabling the address translation information cache module to be in a state where it can be read. Then, the cache management module queries the address translation list based on the first virtual address. Since the cache management module queries that the address translation information cache module does not store the first physical address corresponding to the first virtual address, the cache management module sends the first virtual address to the address translation request initiation module. The address translation request initiation module determines that the first address translation request in the cache is for querying the first physical address corresponding to the first virtual address. Therefore, it prohibits the generation of this address translation request and only needs to wait for the result of the first address translation request.

[0044] See Figure 3 , Figure 3 is a schematic flowchart of a remote direct memory access method provided by this application. As Figure 3 shown, the remote direct memory access method of this application includes:

[0045] S101: Query whether there is a first physical address corresponding to the first virtual address in the address translation list according to the first virtual address carried in the first remote direct memory access request. If there is a first physical address corresponding to the first virtual address, replace the first virtual address in the first remote direct memory access request with the first physical address. If there is no first physical address corresponding to the first virtual address, send a first address translation request to the host.

[0046] The service module sends a first remote direct memory access request to the table lookup module. Among them, the first virtual address carried in the first remote direct memory access request is the virtual address of the memory of the host to be accessed. After the table lookup module receives the first remote direct memory access request sent by the service module, the table lookup module obtains the first virtual address from the first remote direct memory access request and sends a table lookup request for the first virtual address to the cache management module. After the cache management module receives the table lookup request, it sends a cache read enable to the address translation information cache module, enabling the address translation information cache module to be in a state where it can be read. Then, the cache management module queries the address translation list based on the first virtual address.

[0047] If the cache management module queries and finds that the first physical address corresponding to the first virtual address is stored in the address translation information cache module, then after the cache management module obtains the first physical address, it sends the first physical address to the look-up table module through the look-up table result. After receiving the first physical address, the look-up table module replaces the first virtual address in the first remote direct memory access request with the first physical address, and then sends the replaced first remote direct memory access request to the host. After receiving the first remote direct memory access request, the host obtains the first physical address from the first remote direct memory access request and accesses the memory of the host using the first physical address.

[0048] If the cache management module queries and finds that the first physical address corresponding to the first virtual address is not stored in the address translation information cache module, then the cache management module sends the first virtual address to the address translation request initiation module. The address translation request initiation module sends a first address translation request to the host, where the first address translation request carries the first virtual address.

[0049] S102: The host receives the first address translation request, queries for the first physical address corresponding to the first virtual address in the input / output memory management unit of the host. In the case where the first physical address corresponding to the first virtual address cannot be found, a conversion failure response is returned to the remote direct memory access device.

[0050] If the host finds the first physical address corresponding to the first virtual address, it indicates that the data corresponding to the first virtual address has been relocated from the hard disk to the physical memory of the host. The host sends a conversion success response to the address translation request initiation module in the remote direct memory access device. Among them, the conversion success response includes the first physical address. After receiving the conversion success response, the address translation request initiation module in the remote direct memory access device obtains the first physical address from the conversion success response and sends the first physical address to the cache management module. The cache management module sends a cache write enable to the address translation information cache module to enable the address translation information cache module to process the state of allowing writes. Then, the cache management module writes the correspondence between the first virtual address and the first physical address into the address translation information cache module.

[0051] If the host finds the first physical address corresponding to the first virtual address, it indicates that the data corresponding to the first virtual address has not been relocated from the hard disk to the physical memory of the host. The host sends a conversion failure response to the address translation request initiation module in the remote direct memory access device.

[0052] S103: The remote direct memory access device receives the conversion failure response and sends a page request to the host based on the conversion failure response.

[0053] The page request includes a first virtual address. The address translation request initiation module in the remote direct memory access device sends the first virtual address to the page request initiation module. The page request initiation module generates a page request based on the first virtual address, and determines whether the first-in first-out queue is full. If the first-in first-out queue is not full, the page request is stored in the first-in first-out queue. If the first-in first-out queue is full, the page request is discarded. The page request initiation module sends the page request to the host.

[0054] S104: The host receives the page request, migrates the data corresponding to the first virtual address from the hard disk of the host to the memory of the host based on the page request, and returns a page request response.

[0055] The page request response includes a first physical address. After receiving the page request, the host migrates the data corresponding to the first virtual address from the hard disk to the memory, and determines the first physical address corresponding to the first virtual address.

[0056] S105: The remote direct memory access device receives the page request response, and stores the correspondence between the first virtual address and the first physical address in the address translation list based on the page request response.

[0057] The page request initiation module of the remote direct memory access device sends the first physical address to the address translation request initiation module. The address translation request initiation module sends the first physical address to the cache management module. The cache management module sends a cache write enable to the address translation information cache module to enable the address translation information cache module to process the state allowing writing. Then, the cache management module writes the correspondence between the first virtual address and the first physical address into the address translation information cache module.

[0058] See Figure 4 , Figure 4 is a schematic structural diagram of a computing device provided by the present application. The computing device 400 includes: one or more processing units 410, a communication interface 420, and a storage unit 430.

[0059] The processing unit 410, the communication interface 420, and the storage unit 430 are interconnected via a bus 440. Optionally, the computing device 400 may further include an input / output interface 450, which is connected to input / output devices for receiving parameters set by the user, etc. The computing device 400 can be used to implement some or all of the functions of the device embodiment or the system embodiment in the embodiments of the present application described above; the processing unit 410 can also be used to implement some or all of the operation steps of the method embodiment in the embodiments of the present application described above. For example, the specific implementation of the computing device 400 performing various operations can refer to the specific details in the above embodiments. For example, the processing unit 410 is used to execute some or all of the steps or some or all of the operations in the above method embodiment. For another example, in the embodiments of the present application, the computing device 400 can be used to implement some or all of the functions of one or more components in the above device embodiment. In addition, the communication interface 420 can specifically be used for communication functions necessary to implement the functions of these devices and components, etc., and the processing unit 410 can specifically be used for processing functions necessary to implement the functions of these devices and components, etc.

[0060] Figure 4 The computing device 400 may include one or more processing units 410, and the multiple processing units 410 may cooperate to provide processing capabilities in a parallel connection mode, a serial connection mode, a serial-parallel connection mode, or any connection mode. Or the multiple processing units 410 may form a processor sequence or a processor array, or the multiple processing units 410 may be divided into a main processor and an auxiliary processor, or the multiple processing units 410 may have different architectures, such as using a heterogeneous computing architecture. Additionally, Figure 4 For the computing device 400 shown, the related structural descriptions and functional descriptions are exemplary and non-limiting. In some exemplary embodiments, the computing device 400 may include more or fewer components than Figure 4 shown, or combine certain components, or split certain components, or have a different component layout.

[0061] The processing unit 410 can have various specific implementation forms. For example, the processing unit 410 can include one or a combination of a central processing unit (CPU), a graphic processing unit (GPU), a neural-network processing unit (NPU), a tensor processing unit (TPU), or a data processing unit (DPU), etc., and the embodiments of the present application do not make specific limitations. The processing unit 410 can also be a single-core processor or a multi-core processor. The processing unit 410 can be a combination of a CPU and a hardware chip. The above-mentioned hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The above-mentioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. The processing unit 410 can also be implemented by a logic device with built-in processing logic alone, such as an FPGA or a digital signal processor (DSP), etc. The communication interface 420 can be a wired interface or a wireless interface for communicating with other modules or devices. The wired interface can be an Ethernet interface, a local interconnect network (LIN), etc., and the wireless interface can be a cellular network interface or a wireless local area network interface, etc.

[0062] The storage unit 430 can be a non-volatile memory, for example, a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory. The storage unit 430 can also be a volatile memory, and the volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM). The storage unit 430 can also be used to store program code and data, so that the processing unit 410 can call the program code stored in the storage unit 430 to execute some or all of the operation steps in the above method embodiments, or execute the corresponding functions in the above device embodiments. In addition, the computing device 400 may include more or fewer components than Figure 4 shown, or have a different component configuration.

[0063] The bus 440 can be a Peripheral Component Interconnect Express (PCIe) bus, or an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (Ubus or UB), a Compute Express Link (CXL), a Cache Coherent Interconnect for Accelerators (CCIX), etc. The bus 440 can be divided into an address bus, a data bus, a control bus, etc. In addition to the data bus, the bus 440 can also include a power bus, a control bus, and a status signal bus, etc. However, for the sake of clarity,Figure 4 It is represented only by a thick line, but it does not mean that there is only one bus or one type of bus.

[0064] The embodiment of the present application also provides a system, which includes a plurality of computing devices. The structure of each computing device can refer to the structure of the above-mentioned computing device. The functions or operations that the system can implement can refer to the specific implementation steps in the above method embodiment and / or the specific functions described in the above device embodiment, which will not be elaborated here.

[0065] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one network site, computer, server or data center to another network site, computer, server or data center in a wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, microwave, etc.) manner. The computer-readable storage medium can be any available medium that the computer can access, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape, etc.), an optical medium (such as a DVD, etc.), or a semiconductor medium (such as a solid-state drive), etc. In the above embodiments, the descriptions of the respective embodiments have their own emphases. For the parts not elaborated in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

Claims

1. A remote direct memory access system, characterized in that Including: A Remote Direct Memory Access (RDMA) device, configured to query an address translation list to determine whether a first physical address corresponding to a first virtual address carried in a first RDMA request exists. If a first physical address corresponding to the first virtual address exists, the first virtual address in the first RDMA request is replaced with the first physical address. If no first physical address corresponding to the first virtual address exists, a first address translation request carrying the first virtual address is sent to a host. The host is configured to receive the first address translation request, query the first physical address corresponding to the first virtual address in an Input / Output Memory Management Unit (IOMMU) of the host according to the first address translation request, and return a conversion failure response to the RDMA device if the first physical address corresponding to the first virtual address cannot be found. The RDMA device is configured to receive the conversion failure response and send a page request to the host based on the conversion failure response, where the page request includes the first virtual address. The host is configured to receive the page request, and an operating system of the host migrates data corresponding to the first virtual address from a hard disk of the host to a memory of the host based on the page request, and returns a page request response, where the page request response includes the first physical address, and the page request does not fix the first physical address in the memory. The RDMA device is configured to receive the page request response and store a correspondence between the first virtual address and the first physical address in the address translation list based on the page request response.

2. The system according to claim 1, wherein: The host is configured to send an address invalidation request to the RDMA device, where the address invalidation request includes the first virtual address. The RDMA device is configured to delete a correspondence between the first virtual address and the first physical address in the address translation list according to the address invalidation request and send an invalidation completion response to the host.

3. The system according to claim 2, wherein: The RDMA device is configured to send a second RDMA request to the host between receiving the address invalidation request and the invalidation completion response, where the second RDMA request includes a second virtual address.

4. The system according to claim 3, wherein: The RDMA device is configured to query the address translation list to determine whether a second physical address corresponding to a second virtual address carried in the second RDMA request exists between receiving the address invalidation request and the invalidation completion response. If a second physical address corresponding to the second virtual address exists, the second RDMA request is sent to the host.

5. The system according to claim 4, wherein a Remote Direct Memory Access (RDMA) device, configured to merge the second RDMA request and the invalidate completion response into the same message queue, and send all messages in the message queue to the host.

6. The system according to claim 1, wherein a Remote Direct Memory Access (RDMA) device, configured to store the first address translation request in the cache of the RDMA device after sending the first address translation request to the host, query whether there is a first physical address corresponding to the first virtual address in the address translation list according to the first virtual address carried in the third RDMA request, and prohibit the generation of the current address translation request when there is no first physical address corresponding to the first virtual address and the first address translation request is stored in the cache of the RDMA device.

7. The system according to any one of claims 1 to 6, wherein a Remote Direct Memory Access (RDMA) device, configured to store the page request into a first-in-first-out queue.

8. The system according to claim 7, wherein a Remote Direct Memory Access (RDMA) device, configured to store the page request into the first-in-first-out queue when the first-in-first-out queue is not full, and discard the page request when the first-in-first-out queue is full.

9. A computing device, characterized in that, Comprising a Remote Direct Memory Access (RDMA) system and a storage unit, wherein the RDMA system and the storage unit can communicate with each other, and the RDMA system is the system according to any one of claims 1 - 8.

10. A cluster of computing devices, characterized in that, Comprising a plurality of computing devices, at least one of the computing devices comprises a Remote Direct Memory Access (RDMA) system and a storage unit, wherein the RDMA system and the storage unit can communicate with each other, and the RDMA system is the system according to any one of claims 1 - 8.

Images