Data transmission method, device, storage medium, and program product

Abstract

Embodiments of the present disclosure provide a data transmission method, a device, a storage medium, and a program product. The embodiments of the present disclosure provide a software-implemented RDMA memory page fault solution: when acquiring data from a remote memory, a remote end maps a page-faulted virtual memory into a specific physical memory region, allows for the occurrence of a memory page fault in an RDMA process, and adds a memory page fault processing flow between a local end and the remote end to implement memory page fault processing; and when writing data into the remote memory, the remote end provides a shared virtual memory and a private virtual memory and exposes the shared virtual memory to the local end, the local end writes data into the shared virtual memory, and then a remote control object migrates the data to the private virtual memory, wherein memory page faults are allowed to occur and be processed during the migration. Since memory page faults are allowed to occur and page fault processing is supported, dynamic allocation of physical memory resources is supported during the RDMA process, thereby improving the utilization of memory resources while leveraging the advantages of RDMA.

Description

Data transmission methods, devices, storage media and software products Technical Field

[0001] This disclosure relates to the field of cloud computing technology, and in particular to a data transmission method, device, storage medium, and program product. Background Technology

[0002] Remote Direct Memory Access (RDMA) is actually a high-speed remote memory access technology that is fully optimized by intelligent network interface cards (NICs) and software architecture. It achieves high-performance remote direct data access by embedding the RDMA protocol on the NIC hardware and supporting zero-copy network technology and kernel memory bypass technology, without the need for time-consuming processing by the central processing unit (CPU). Therefore, it is widely used.

[0003] Before the local application uses RDMA to access the remote application's memory, the remote application can request a contiguous block of virtual memory through a Memory Region (MR) registration process. This virtual memory is then mapped to physical memory to store the data that needs to be read and written during RDMA. Simultaneously, this physical memory is pinned to prevent pages from being swapped out. While RDMA technology can improve communication performance, it also reduces the amount of memory available for over-division on the remote end, thus decreasing memory resource utilization. Summary of the Invention

[0004] This disclosure provides a data transmission method, apparatus, storage medium, and program product to support dynamic allocation of physical memory resources during RDMA processes, thereby leveraging the advantages of RDMA while improving the memory resource utilization of electronic devices.

[0005] This disclosure provides a data transmission method applied to a first application on a first electronic device, and a second application on a second electronic device registered with a target virtual address. The method includes: reading first data from a first physical memory region mapped to the target virtual address via a first remote direct memory access operation, wherein the second electronic device includes a specific physical memory region for mapping a virtual address that the second application requests to register and that experiences a page fault; upon identifying that the first data originates from the specific physical memory region, sending a page fault request to the second application via a second remote direct memory access operation, wherein the page fault handling is used to request that the target virtual address be mapped from the specific physical memory region to the second physical memory region; and upon receiving a page fault handling completion message returned by the second application, rereading the second data from the second physical memory region mapped to the target virtual address via the first remote direct memory access operation.

[0006] This disclosure also provides a data transmission method applied to a second memory access device on a second electronic device. A second application on the second electronic device has registered a target virtual address, and a specific physical memory region on the second electronic device is used to map the virtual address that the second application requests to register and where a memory page fault occurs. The method includes: responding to a first remote direct memory access operation initiated by a first application on a first electronic device; sending first data to the first application, the first data being data in the first physical memory region to which the target virtual address is mapped; receiving a page fault request sent by the first application through a second remote direct memory access operation, the page fault request being sent by the first application after recognizing that the first data originates from a specific physical memory region of the second electronic device; providing the page fault request to the second application for page fault processing to map the target virtual address from the specific physical memory region to the second physical memory region; and sending a page fault processing completion message to the first application, the page fault processing completion message being used to trigger the first application to reread the second data from the second physical memory region through the first remote direct memory access operation.

[0007] This disclosure also provides a data transmission method applied to a second application on a second electronic device, wherein a specific physical memory region on the second electronic device is used to map a virtual address that the second application requests to register and where a memory page fault occurs; the method includes: receiving a page fault request from a first application on a first electronic device, the page fault request being sent by the first application via a second remote direct memory access operation after recognizing that first data originates from the specific physical memory region, the first data being data in the first physical memory region mapped to a target virtual address registered by the second application; performing page fault processing on the target virtual address according to the page fault request to map the target virtual address from the specific physical memory region to the second physical memory region; and sending a page fault processing completion message to the first application via a third remote direct memory access operation, so that the first application can reread the second data from the second physical memory region via a first remote direct memory access operation.

[0008] This disclosure also provides a data transmission method applied to a first memory access device on a first electronic device, wherein a first application runs on the first electronic device. The method includes: receiving a data transmission request issued by the first application, the data transmission request including a first virtual address and identification information of a second application running on a second electronic device; the second memory access device of the second electronic device pre-registering a shared virtual memory, wherein the first virtual address belongs to the shared virtual memory; sending target data to be sent by the first application, the first virtual address, and the identification information of the second application to the second memory access device, so that the second memory access device writes the target data and the identification information of the second application into the shared virtual memory based on the first virtual address; wherein the shared virtual memory is associated with a management object, the management object being used to migrate the target data from the shared virtual memory to a private virtual memory registered by the second application according to the identification information of the second application.

[0009] This disclosure also provides a data transmission method applied to a second memory access device of a second electronic device, the second electronic device running a second application. The method includes: receiving target data, a first virtual address, and identification information of the second application sent by a first memory access device of a first electronic device, wherein the first virtual address belongs to a shared virtual memory registered by the second memory access device; and writing the target data and the identification information of the second application into the shared virtual memory based on the first virtual address, so that a management object associated with the shared virtual memory can migrate the target data from the shared virtual memory to a private virtual memory registered by the second application according to the identification information of the second application.

[0010] This disclosure also provides a data transmission method applied to a controlled object on a second electronic device. The second electronic device runs a second application and includes a second memory access device registered with shared virtual memory. The method includes: monitoring the second memory access device writing target data and the identification information of the second application to a first virtual address in the shared virtual memory; determining a second virtual address from a private virtual memory registered by the second application based on the identification information of the second application; if a page fault occurs at the second virtual address, performing page fault processing on the second virtual address to map the second virtual address to a second physical address; and migrating the target data from the shared virtual memory to the private virtual memory based on the second physical address mapped to the second virtual address.

[0011] This disclosure also provides a memory access device, which includes a memory and a processor. The memory is used to store a computer program, and the processor is coupled to the memory and is used to execute the computer program to implement the steps in the data transmission method performed by a first network card or a second network card provided in this disclosure.

[0012] This disclosure also provides an electronic device, including: a memory access device provided in this disclosure.

[0013] This disclosure also provides an electronic device, including: a memory and a processor; the memory is used to store a computer program, and the processor is coupled to the memory and used to execute the computer program for steps in a data transmission method provided in this disclosure, which is executed by a first application, a second application, or a controlled object.

[0014] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps in the data transmission methods provided in various embodiments of this disclosure.

[0015] This disclosure also provides a computer program product, including a computer program / instructions, which, when executed by a processor, cause the processor to implement the steps in the data transmission methods provided in various embodiments of this disclosure.

[0016] In this disclosure, a software-implemented RDMA scheme supporting memory page faults is proposed:

[0017] To address the scenario of retrieving data from remote memory, a specific physical memory region is provided remotely. During the MapReduce (MR) registration process, the page fault virtual memory is mapped to this specific physical memory region. This allows data to be read from the specific physical memory region when the RDMA operation accesses the page fault virtual memory, ensuring that the RDMA operation does not encounter errors and simultaneously causing a memory page fault during the RDMA process. Furthermore, a memory page fault handling process is added between the local application and the remote application to handle memory page faults, enabling the local application to successfully retrieve data from the remote memory after the memory page fault has been handled.

[0018] For writing data to remote memory, the remote end provides both shared virtual memory and private virtual memory. The shared virtual memory is exposed to the local application. The local application writes data to the shared virtual memory, and the remote control object migrates the data from the shared virtual memory to the private virtual memory. This migration process allows for and handles page faults, ensuring that the local application can write data to the remote memory while also supporting page fault handling. Because it allows for and handles page faults, RDMA enables dynamic allocation of physical memory resources, leveraging the advantages of RDMA while improving the memory resource utilization of electronic devices. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this disclosure, illustrate exemplary embodiments of the present disclosure and are used to explain the disclosure, but do not constitute an undue limitation of the disclosure. In the drawings:

[0020] Figure 1a is a schematic diagram of the interaction flow of the data transmission system 1 at each stage provided in an exemplary embodiment of the present disclosure;

[0021] Figure 1b is a schematic diagram of the interaction flow of the data transmission system 1 at each stage provided in another exemplary embodiment of this disclosure;

[0022] Figure 1c is a schematic diagram of the interaction flow of the data transmission system 1 at each stage provided in another exemplary embodiment of this disclosure;

[0023] Figure 2 is a schematic diagram of another interaction flow of the data transmission system 1 provided in an exemplary embodiment of the present disclosure;

[0024] Figure 3 is a flowchart illustrating a data transmission method provided in an exemplary embodiment of this disclosure;

[0025] Figure 4 is a flowchart illustrating a data transmission method provided in another exemplary embodiment of this disclosure;

[0026] Figure 5 is a flowchart illustrating a data transmission method provided in another exemplary embodiment of this disclosure;

[0027] Figure 6a is a schematic diagram of the interaction flow of the data transmission system 1 provided in another exemplary embodiment of the present disclosure when executing another interaction logic;

[0028] Figure 6b is a flowchart illustrating a data transmission method provided in an exemplary embodiment of this disclosure;

[0029] Figure 6c is a flowchart illustrating a data transmission method provided in another exemplary embodiment of this disclosure;

[0030] Figure 6d is a flowchart illustrating a data transmission method provided in another exemplary embodiment of this disclosure;

[0031] Figure 7 is a schematic diagram of the structure of a data transmission device provided in an exemplary embodiment of the present disclosure;

[0032] Figure 8 is a schematic diagram of the structure of a data transmission apparatus provided in another exemplary embodiment of the present disclosure;

[0033] Figure 9 is a schematic diagram of the structure of a data transmission apparatus provided in another exemplary embodiment of the present disclosure;

[0034] Figure 10 is a schematic diagram of the structure of an electronic device provided in an exemplary embodiment of the present disclosure;

[0035] Figure 11 is a schematic diagram of the structure of a data transmission apparatus provided in another exemplary embodiment of the present disclosure;

[0036] Figure 12 is a schematic diagram of the structure of a data transmission apparatus provided in another exemplary embodiment of the present disclosure;

[0037] Figure 13 is a schematic diagram of the structure of a data transmission device provided in another exemplary embodiment of the present disclosure. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this disclosure clearer, the technical solutions of this disclosure will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0039] It should be noted that, in the cases involving user information in the embodiments of this disclosure, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse. In addition, the various models involved in this disclosure (including but not limited to language models or large models) comply with relevant laws and standards.

[0040] This disclosure relates to RDMA technology. RDMA technology is a high-speed, direct remote memory access technology with fully optimized memory access devices and software architecture. In this embodiment, the memory access device may be, but is not limited to, a network interface card (NIC). In the following embodiments, a NIC is used as an example, but the implementation is not limited to this. This disclosure does not limit the implementation method of RDMA technology at the protocol level; for example, it may include, but is not limited to, the following:

[0041] InfiniBand protocol: RDMA technology based on the InfiniBand architecture, proposed by IBTA (InfiniBand Trade Association), abbreviated as IB. Building an RDMA network based on IB technology requires dedicated IB network interface cards (NICs) and IB switches. In terms of performance, InfiniBand networks are superior, but the NICs and switches are relatively expensive.

[0042] iWARP stands for Internet Wide Area RDMA Protocol. It's an RDMA technology based on the Transmission Control Protocol / Internet Protocol (TCP / IP), defined by the Internet Engineering Task Force (IETF). iWARP supports the use of RDMA technology on standard Ethernet infrastructure without requiring switches to support lossless Ethernet transmission, but servers need to use iWARP-enabled network interface cards (NICs).

[0043] RoCE protocol: The full English abbreviation is RDMA over Converged Ethernet, which is also proposed by IBTA. RoCE supports the use of RDMA technology on standard Ethernet infrastructure, but requires the switch to support lossless Ethernet transmission and the server to use RoCE network interface card devices. Its performance is comparable to IB.

[0044] Different RDMA technologies using different protocols will have different hardware implementations, specifically different physical and data link layers. However, regardless of the protocol or hardware implementation used, RDMA uses the same set of Application Programming Interfaces (APIs) for applications. These APIs, also known as Verbs, are the software transport interface between the application and the network interface card (NIC) device. In other words, RDMA provides a series of Verbs interfaces in the application's user space, allowing the application to directly operate the NIC device, bypassing the kernel and directly accessing the application's user space.

[0045] From a development perspective, the applications in this disclosure refer to applications implemented based on the API programming interface (i.e., the Verbs interface) provided by RDMA, which are applications capable of directly reading and writing to remote memory through RDMA technology. From a functional perspective, the applications in this disclosure can be, but are not limited to: video processing applications, instant messaging applications, e-commerce applications, cloud gaming applications, load balancing applications, gateway applications, and various cloud service applications.

[0046] The network interface card (NIC) device involved in the embodiments of this disclosure can be any NIC device that supports the RDMA protocol, such as a smart NIC, a network interface card (NIC), a cloud infrastructure processing unit (CIPU) or an infrastructure processing unit (IPU) chip or module that integrates a NIC module that supports the RDMA protocol.

[0047] The traditional Verbs interface includes two types: Memory verbs and Messaging verbs. Based on these two Verbs interfaces, RDMA can provide two basic operations for applications: one-sided operations and two-sided operations.

[0048] Memory verbs include RDMA read, RDMA write, and RDMA atomic operations. These are one-sided operations, meaning that the local application only needs to specify the source and destination addresses of the data to be transmitted. The remote application does not need to be aware of this communication. Data reading or writing is completed between the remote network card device and the remote application's memory using DMA (Direct Memory Access) technology, without the remote application's awareness or participation.

[0049] RDMA Read: This is the process of directly reading data from the memory of a remote application (referred to as remote memory) using RDMA technology. Before performing an RDMA read operation, the remote application needs to grant the local application appropriate permissions to access the remote memory. Once the permissions are set, the RDMA read operation can be performed without notifying the remote application. In other words, the local network interface card (NIC) device uses RDMA technology to read data directly from the remote memory as if it were reading data from local memory, without the remote application being aware of it.

[0050] RDMA Write: Similar to RDMA Read, it's the process of writing data to the remote application's memory (referred to as remote memory) using RDMA technology. Before performing an RDMA write operation, the remote application needs to grant the local application appropriate permissions to access the remote memory. Once the permissions are set, the RDMA write operation can be performed without notifying the remote application. In other words, the local network interface card (NIC) uses RDMA technology to write data directly to the remote memory as if it were writing data to local memory, without the remote application being aware of it.

[0051] RDMA Atomic includes atomic take, atomic add, atomic comparison, and atomic exchange, and is an extension of RDMA atomic operations. This disclosure does not focus on these operations in detail.

[0052] Messaging verbs include RDMA send and RDMA receive operations. These are bilateral operations, meaning that data transmission and reception operations require the remote application to be aware of and participate in them to be completed.

[0053] RDMA Send: This is the process of sending data from the local application's memory (referred to as local memory) to the remote application's memory (referred to as remote memory) using RDMA technology. Specifically, the local application places a Work Queue Element (WQE) representing a send task into the send queue (SQ) of a queue pair (QP). Similarly, the remote application sends a WQE representing a receive task to the receive queue (RQ) of the remote QP. Then, the local network interface card (NIC) retrieves the send task from the SQ, obtains the data from the local memory according to the send task, and sends it to the remote NIC via the physical link. When the remote network interface card (NIC) receives data, verifies it, and replies with an acknowledgment message (ACK) to the sending end, it places the data in the remote memory location specified in the WQE (Wide Queue Element). Then, it generates a completion queue element (CQE) for the receive task and places it in the completion queue (CQ). The remote application then obtains the task completion information. Upon receiving the ACK, the local NIC similarly generates a CQE for the send task and places it in the CQ. The local application then obtains the task completion message. This entire process requires the awareness and participation of both the local and remote applications.

[0054] RDMA Receive: This is the process of receiving data from the remote application's memory (referred to as remote memory) to the local application's memory (referred to as local memory) via RDMA technology. Specifically, the remote application sends a Send Qualification (WQE) indicating a send task to the remote QP's Send Qualification (SQ); the local application places a receive Qualification (WQE) indicating a receive task into the QP's Receiving Qualification (RQ). Then, the remote network interface card (NIC) retrieves the send task from the SQ, obtains data from the remote memory according to the send task, and sends it to the local NIC via the physical link. Upon receiving the data, the local NIC verifies it and replies with an Acknowledgment (ACK) message to the remote NIC, placing the data in the location specified in the WQE in its local memory. It then generates a Receive Qualification (CQE) for the receive task and places it in the CQ. The local application then obtains the task completion information. Upon receiving the ACK, the remote NIC similarly generates a send Qualification (CQE) for the send task and places it in the CQ. The remote application then obtains the task completion message. The entire process requires the awareness and participation of both the local and remote applications.

[0055] Before accessing remote memory using RDMA technology, the remote application pre-allocates remote memory through the MapReduce (MR) registration process and provides this remote memory information to the local application. When using virtual memory technology, the memory resources of each application are virtual memory allocated by the operating system. The remote memory pre-allocated by the remote application is virtual memory, which is mapped to the physical memory on the electronic device where the remote application resides. The address mapping relationship between this virtual memory and physical memory is maintained on the remote network interface card (NIC) so that the remote NIC can directly access the physical memory. Since the RDMA process bypasses the kernel, to prevent page faults from causing anomalies in the RDMA access process, the physical memory mapped to the remote memory needs to be pinned. This means keeping the pages in that physical memory resident in physical memory and preventing them from being swapped out. This pinned portion of physical memory cannot participate in memory allocation again, thus reducing the available memory range for the remote electronic device and decreasing memory resource utilization. This is especially problematic when RDMA operations require a large amount of memory, severely reducing memory resource utilization. In this context, "over-allocation" refers to a memory allocation method where the total allocated virtual memory exceeds the actual available physical memory. The available over-allocation memory range refers to the actual available physical memory on the electronic device; the larger the physical memory occupied by the pinned virtual memory area, the smaller the available physical memory for over-allocation.

[0056] In this embodiment of the disclosure, an RDMA scheme that supports memory page faults is provided. That is, the virtual memory accessed by RDMA supports memory page faults, and the physical memory mapped to the virtual memory does not need to be pinned. This allows the physical memory to be allocated to other objects for use, and then allocated to RDMA operations when needed. For electronic devices that support RDMA operations, this enables memory over-allocation within a larger physical memory range. While leveraging the advantages of RDMA technology, it can improve the memory resource utilization of electronic devices.

[0057] In one implementation provided in this disclosure, the hardware behavior of the network interface card (NIC) device can be modified to enable the NIC device to support memory page fault handling when performing RDMA operations, thereby realizing an RDMA scheme that supports memory page faults. However, solutions relying on hardware modifications are not applicable to existing NIC devices, necessitating the replacement of NIC devices. Large-scale replacement of NIC devices is not only time-consuming but also costly.

[0058] Compared to hardware upgrades, software solutions have a natural advantage in rapid, large-scale deployment. Therefore, in another implementation provided in this disclosure, a software-based RDMA scheme supporting memory page faults is proposed. This software implementation can be completed by applications developed based on the programming interface provided by RDMA, network card device drivers, and the operating system of the electronic device in which the network card device resides, without changing the hardware logic of the RDMA network card device. The specific implementation of this software-based RDMA scheme supporting memory page faults will vary depending on the specific RDMA operation, as described below.

[0059] Scenario 1: RDMA operation where the local application retrieves data from the remote application's remote memory: This RDMA operation can be an RDMA Read operation initiated by the local application to the remote application, or an RDMA Receive operation initiated by the local application to the remote application. These two operations are collectively referred to as RDMA data retrieval operations. In this case, the remote application provides a specific physical memory region and keeps this region from being swapped out. During the MapReduce (MR) registration process, page fault virtual memory can be mapped to this specific physical memory region. This specific physical memory region allows page faults to occur during the RDMA process and enables the identification of page faults. Furthermore, a page fault handling process is added between the local and remote applications to resolve page fault issues. Ultimately, this ensures that the local application can successfully retrieve data from the remote memory after page fault handling, while also supporting the occurrence of page faults, i.e., supporting dynamic allocation of physical memory resources. This leverages the advantages of RDMA technology and improves the memory resource utilization of electronic devices.

[0060] Scenario 2: RDMA operation where the local application writes data to the remote application's remote memory: This RDMA operation can be an RDMA Write operation initiated by the local application to the remote application, or an RDMA Send operation initiated by the local application to the remote application. These two operations are collectively referred to as RDMA data provision operations. In this case, the remote application simultaneously provides shared virtual memory and private virtual memory. The shared virtual memory is kept free from page faults and exposed to the local application so that it can write data to it. Then, the remote application uses a managed object to migrate the data from the shared virtual memory to the private virtual memory. During this process, page faults are allowed and handled, enabling the identification and resolution of page fault problems. Ultimately, this ensures that the local application can successfully write data to the remote memory after page fault handling, while also supporting the occurrence of page faults, i.e., supporting dynamic allocation of physical memory resources. This leverages the advantages of RDMA technology and improves the memory resource utilization of electronic devices.

[0061] It should be noted that the RDMA scheme supporting memory page faults implemented in software according to the embodiments of this disclosure is applicable to both one-sided and two-sided RDMA operations. In the following embodiments of this disclosure, the technical solution of this disclosure will be described in detail using one-sided RDMA operation as an example.

[0062] Furthermore, the application scenarios of the RDMA scheme provided in this disclosure are not limited to virtualized environments; it can be applied in both virtualized and non-virtualized environments. A virtualized environment refers to a scenario where at least one virtualized instance is created on an electronic device using virtualization technology, and these virtualized instances provide independent and isolated runtime environments, with various applications deployed within them. Virtualized instances can be, but are not limited to, VMs (Virtual Machines), containers, or various cloud computing instances. In contrast, a non-virtualized environment refers to a scenario where various applications are directly deployed on the electronic device without using virtualization technology. In a virtualized environment, RDMA technology can improve the communication performance between virtualized instances and increase the communication efficiency of the entire virtualization system. That is, the local application (i.e., the first application hereinafter) and the remote application (i.e., the second application hereinafter) involved in the embodiments of this disclosure can be applications deployed in virtualized instances (e.g., VMs) or applications deployed in the runtime environment directly provided by the electronic device. Regardless of the runtime environment, these applications are applications developed based on the programming interface provided by RDMA.

[0063] The technical solutions provided by the embodiments of this disclosure are described in detail below with reference to the accompanying drawings, specifically for situation 1 described above.

[0064] Figures 1a-1c are schematic diagrams of the architecture of a data transmission system 1 provided in an exemplary embodiment of this disclosure. Figure 2 is a schematic diagram of the interaction flow of a data transmission method provided based on the data transmission system 1 shown in Figures 1a-1c. As shown in Figure 2, the embodiment of this disclosure divides the data transmission process based on the data transmission system 1 into a first stage, a second stage, and a third stage. Steps ①-⑤ shown in Figure 1a correspond to steps ①-⑤ in the first stage shown in Figure 2; steps ⑥-... Similar to step ⑥ in the second stage shown in Figure 2. Correspondingly; the steps shown in Figure 1c The steps in the third stage shown in Figure 2 Correspondingly.

[0065] As shown in Figures 1a-1c, the data transmission system 1 may include a first electronic device 11 and a second electronic device 12. The first electronic device 11 and the second electronic device 12 can be understood as a local device and a remote device. The first electronic device 11 may be a terminal device such as a smartphone, tablet, or computer, or it may be a server device; the second electronic device 12 may be a terminal device such as a smartphone, tablet, or computer, or it may be a server device; this embodiment does not impose any limitations. Both the first electronic device 11 and the second electronic device 12 include network interface cards (NICs) supporting RDMA technology and run RDMA applications. Specifically, the first electronic device 11 may run a first application 111, and its hardware resources include at least a first NIC 112; the second electronic device 12 may run a second application 121, and its hardware resources include at least a second NIC 122. The use of "first" and "second" to distinguish between the electronic devices, NICs, and applications does not limit the number, type, or composition of the electronic devices, NICs, and applications.

[0066] In the data transmission system 1 shown in Figures 1a-1c, the first application 111 and the second application 121 have completed memory registration and established RDMA connections through the first network interface card (NIC) device 112 and the second NIC device 122. Interactions between the first NIC device 112 and the first application 111, and between the second NIC device 122 and the second application 121, can be performed based on local queue pairs. The queue pairs, including the send queue, receive queue, and completion queue, have been described in detail in the preceding embodiments and will not be elaborated upon here. Furthermore, the process of establishing an RDMA connection and performing memory registration between the first application 111 and the second application 121 can be found in the description of subsequent embodiments. This embodiment describes the data transmission process between the two applications based on the established RDMA connection and completed memory registration.

[0067] In Figure 2, steps ①-⑤ describe the first stage of the data transmission process, namely, the process by which the first application can read the first data from the first physical memory region mapped to the target virtual address registered by the second application through the first RDMA operation.

[0068] Specifically, referring to step ① in Figure 2, the first application 111 can send a data acquisition request to the first network interface card (NIC) device 112, and correspondingly, the first NIC device 112 can receive the data acquisition request sent by the first application 111. This data acquisition request is used to acquire data from the peer's memory. This embodiment does not limit the type of data acquisition request. The data acquisition request can be a read request corresponding to a read operation in a one-sided RDMA operation, or a receive request corresponding to a receive operation in a two-sided RDMA operation. This embodiment does not impose any restrictions.

[0069] Optionally, the first application 111 can write a data read task to the send queue between the first application and the first network interface card (NIC) device. The data read task may include a data acquisition request; in other words, the data acquisition request can be encapsulated within the data read task, and then sent to the first NIC device 112 via the send queue. Correspondingly, the first NIC device 112 can retrieve the data read task from the send queue and parse the data acquisition request from it. The data read task can be implemented as an SQE (Send Queue Entry). By exchanging information through the send queue, the first application and the first NIC device can bypass the operating system and directly perform data interaction, which improves the efficiency of their interaction.

[0070] Referring to step ② in Figure 2, when the first network interface card (NIC) device 112 receives a data acquisition request, it can send the data acquisition request to the second NIC device 122; correspondingly, the second NIC device 122 receives the data acquisition request sent by the first NIC device 112.

[0071] Optionally, the first network interface card (NIC) can encapsulate the data acquisition request in a request packet and send the request packet to the second NIC device 122 via a physical link. Correspondingly, the second NIC device 122 can receive the request packet sent by the first NIC device 112 and parse the data acquisition request from the request packet.

[0072] In this embodiment, the data acquisition request includes a target virtual address, which is the address information in the virtual memory pre-registered by the second application during the RDMA connection establishment process. During RDMA connection establishment, the second application provides the relevant information of its pre-registered virtual memory to the first application, allowing the first application to directly perform read and write operations on the virtual memory via the RDMA connection. Upon receiving the data acquisition request, the second network interface card (NIC) parses the target virtual address from it and, according to the pre-maintained target page table, reads data from the first physical memory region mapped to the target virtual address and returns it to the first application via the first NIC. The process of the first application performing read and write operations on the virtual memory registered by the second application via the RDMA connection is also the process of the first application performing read and write operations on the virtual memory registered by the second application through both the first and second NICs.

[0073] During the virtual memory registration process of the second application 121, the mapping relationship between virtual memory and physical memory can be configured on the second network interface card (NIC) device 122. The second NIC device 122 maintains a target page table, which records the mapping relationship between each virtual address and physical address in the virtual memory registered by the second application 121. In this embodiment, page faults are allowed in the virtual memory registered by the second application 121; that is, virtual addresses that have experienced page faults can exist in the virtual memory. As shown in Figures 1a-1c, the virtual memory can include one or more virtual memory regions, each virtual memory region includes one or more virtual addresses, and each virtual memory region can be valid or invalid virtual memory. Any virtual address can be address information within a valid or invalid virtual memory region. An invalid virtual memory region refers to a virtual memory region where page faults have occurred, meaning that this virtual memory region does not have a corresponding physical memory mapping; a valid virtual memory region refers to a virtual memory region where page faults have not occurred, meaning that this virtual memory region has a corresponding physical memory mapping. Figures 1a-1c show valid virtual memory region 1, valid virtual memory region 2, invalid virtual memory region 3, invalid virtual memory region 4, physical memory region 1, physical memory region 2, and specific physical memory regions. These are for illustrative purposes only and do not limit the number or arrangement of valid virtual memory regions, invalid virtual memory regions, or physical memory regions, nor do they impose any restrictions on the hardware architecture of the second electronic device.

[0074] In this embodiment, page faults are allowed in the virtual memory registered by the second application 121. If the first application 111 accesses the virtual address where a page fault has occurred while directly accessing the virtual memory via an RDMA connection, it will be unable to retrieve data from the physical memory of the second electronic device, leading to an RDMA operation error. To ensure that the first application 111 can successfully read data from the virtual address where a page fault has occurred via an RDMA connection, a specific physical memory region can be pre-allocated before the second application 121 registers the virtual memory. This specific physical memory region needs to be pinned, meaning it cannot be swapped out. It should be noted that this embodiment does not limit the size of the specific physical memory region, as long as it can complete the mapping of the virtual address where a page fault has occurred. Preferably, the smaller the specific physical memory region, the fewer the pinned physical memory resources, the more physical memory resources the second electronic device can allocate, and the higher the memory resource utilization rate. For example, the size of the page fault virtual memory region is 256M, while the size of the specific physical memory region is 4M, and other physical memory resources besides the specific physical memory region can be allocated.

[0075] During the virtual memory registration process of the second application 121, if a virtual address with a memory page fault exists, this virtual address is mapped to a specific physical memory region. The mapping relationship between the virtual address with the memory page fault and the physical address of the specific physical memory region is configured on the second network interface card (NIC) device 122. Thus, when the first application 111 accesses the virtual address with the memory page fault via an RDMA connection, the second NIC device 122 can directly read data from the specific physical memory region based on the maintained mapping relationship and return it to the first application 111 via the first NIC device 112, ensuring that the RDMA process does not encounter exceptions due to memory page faults. The mapping relationship between the virtual address with the memory page fault and the physical address of the specific physical memory region is also stored in the target page table. As shown in Figure 1a, invalid virtual memory regions 3 and 4 are both mapped to specific physical memory regions.

[0076] For ease of distinction and description, in this embodiment, the virtual address included in the data acquisition request is referred to as the target virtual address. Optionally, the data acquisition request may also include: remote memory index information (remote key, rkey), local memory index information (local key, lkey), and queue pair number (Queue Pair Number, QPN), etc. rkey is the remote memory index information of the virtual memory registered by the second application 121, used by the second network interface device 122 to perform permission verification on the first application 111. QPN identifies the queue pair of the second application 121. The second network interface device 122 can provide RDMA communication services for multiple RDMA applications. Different RDMA applications have their own queue pair and other resource information, and the queue pairs of different RDMA applications can be distinguished through QPN. The second network interface device 122 can perform permission verification based on rkey. If the permission verification is successful, the target page table corresponding to the second application 121 is determined, and then the target virtual address is converted to a physical address based on the target page table, that is, the first physical memory region mapped to the target virtual address is determined. Next, referring to step ③ in Figure 2, the second physical network card reads the first data from the first physical memory region. Here, the first physical memory region refers to the physical memory region corresponding to the physical address mapped by the target virtual address. It should be noted that the target virtual address may or may not experience a page fault; if a page fault occurs, the first physical memory region is the specific physical memory region mentioned above; if no page fault occurs, the first physical memory region is a physical memory region outside the specific physical memory region.

[0077] Referring to step ④ in Figure 2, after reading the first data from the first physical memory area, the second network interface card (NIC) device 122 can send the first data to the first NIC device 112. Correspondingly, the first NIC device 112 receives the first data sent by the second NIC device 122.

[0078] Optionally, the second network interface card (NIC) device 122 can encapsulate the first data in a data packet and send the data packet to the first NIC device 112 via a physical link. Correspondingly, the first NIC device 112 can receive the data packet encapsulated with the first data sent by the second NIC device 122 and parse the data packet to obtain the first data.

[0079] Referring to step ⑤ in Figure 2, after receiving the first data, the first network interface card (NIC) device 112 can report the first data to the first application 111. Optionally, the first NIC device 112 can write the first data into the physical memory area mapped to the local virtual memory area specified in the data acquisition request (or SQE mentioned above) to provide the first data to the first application 111. Further, the first NIC device 112 can also generate a completion queue element (CQE) and write the CQE into the completion queue to notify the first application 111 to read the first data from the local virtual memory area. The first application 111 then reads the first data from the physical memory area mapped to the specified local virtual memory area, thus ending the first stage of the data transmission process. Similarly, data interaction between the first application and the first NIC device via the completion queue can bypass the operating system, which is beneficial for improving data interaction efficiency.

[0080] Among them, step ⑥ shown in Figure 2 The second stage of the data transfer process is described, which is the process of sending a page fault request to the second application through a second RDMA operation after it is identified that the first data comes from a specific physical memory region.

[0081] Specifically, referring to step ⑥ in Figure 2, the first application 111 identifies the source of the first data, that is, identifies whether the first data comes from a specific physical memory region in the second electronic device. In this embodiment, the method of identifying whether the first data comes from a specific physical memory region is not limited; examples of identification methods can be found in the descriptions of subsequent embodiments.

[0082] Referring to step ⑦ in Figure 2, when the first application 111 identifies that the first data originates from a specific physical memory region, it can send a page fault request to the first network interface card (NIC) device 112; correspondingly, the first NIC device 112 receives the page fault request sent by the first application 111. The page fault request is sent to the second application 121 to request the second application 121 to perform page fault processing on the target virtual address where the memory page fault occurred.

[0083] Optionally, the first application 111 can send a page fault request to the second application 121 via the RDMA Send operation in the RDMA bilateral operation. Specifically, the first application 111 can write a first data transmission task into the transmission queue between the first application and the first network interface card (NIC) device. This first data transmission task includes the page fault request. The first data transmission task refers to a transmission task encapsulating the page fault request, which can be implemented as an SQE in the RDMA Send operation. Correspondingly, the first NIC device 112 retrieves the first data transmission task from the transmission queue, parses the first data transmission task, and extracts the page fault request from it.

[0084] Referring to steps ⑧-⑨ in Figure 2, upon receiving a page fault request, the first network interface card (NIC) device 112 sends the page fault request to the second NIC device 122, which then reports it to the second application 121. Optionally, the first NIC device 112 can encapsulate the page fault request in a request packet and send it to the second NIC device 122 via a physical link; the second NIC device 122 receives the request packet sent by the first NIC device 112 and parses the page fault request from the request packet.

[0085] Optionally, when sending a page fault request to the second application 121 using an RDMA Send operation, the second application 121 pre-issues a data reception task to its receive queue. This data reception task can be implemented as an RQE, and includes a local virtual address specified by the second application 121 for receiving data. Based on this, the second network interface device 122 can obtain the data reception task from the receive queue, parse the local virtual address from the data reception task, write the page fault request into the local virtual address specified in the data reception task, and write a CQE to the completion queue of the second application 121 to notify the second application 121 to read the page fault request from the physical memory mapped to the local virtual address. The second application 121 then reads the page fault request from the physical memory region mapped to the local virtual address. In this embodiment, the page fault request is implemented as data sent by the first application 111 to the second application 121 during the RDMA Send operation.

[0086] Referring to step 10 in Figure 2, after obtaining a page fault request, the second application 121 can perform page fault processing on the target virtual address according to the page fault request, so as to map the target virtual address from a specific physical memory region to a second physical address. In this embodiment, the specific implementation of page fault processing based on the page fault request is not limited. In some optional embodiments, the second application 121 can notify the driver of the second network interface card device 122 to perform page fault processing on the target virtual address. The driver can call the memory management module of the second electronic device to load the physical page (i.e., page) corresponding to the target virtual address from the persistent storage medium (e.g., disk, solid-state drive, etc.) to the second physical address in the second physical memory region, and update the mapping relationship between the target virtual address and the physical address in the specific physical memory region stored on the second network interface card device 122 to the mapping relationship between the target virtual address and the second physical address in the second physical memory region.

[0087] The memory management module, a functional module in the operating system, is responsible for managing and allocating the computer system's memory resources, ensuring that processes can effectively utilize these resources. In this embodiment, the memory management module can perform page fault handling on the target virtual address based on the following process: The memory management module checks a preset target page table to confirm whether the physical data corresponding to the target virtual address is in physical memory, i.e., whether the target virtual address is invalid virtual memory. If the physical data corresponding to the target virtual address is not in physical memory, a free physical page can be selected in physical memory, or a single physical data can be selected from multiple physical data in physical memory and swapped to persistent storage media (such as a disk) to free up the physical memory area; then, the physical data corresponding to the target virtual address is loaded from the persistent storage media into the selected free physical page or the freed physical memory area. Optionally, a page replacement algorithm can be used for page scheduling. This algorithm determines existing pages that should be swapped out to persistent storage media (such as a disk) to make room for new pages, such as First-In-First-Out (FIFO) or Least Recently Used (LRU) algorithms. The First-In-First-Out (FIFO) replacement algorithm works by swapping out the page that was first entered into memory, while the Least Recently Used (LRU) algorithm works by swapping out the page that was least recently used.

[0088] Furthermore, after the driver detects that the memory management module has completed the page fault handling, it can generate a page fault handling completion message and notify the second application 121. Further, see the steps in Figure 2. As shown, the second application 121 can send a page fault completion message to the second network interface card (NIC) device 122, which then provides the message to the first application 111 via the first NIC device 112. The first application 111 receives the page fault completion message via the first NIC device 112.

[0089] Optionally, the second application 121 can use the RDMA Send operation in RDMA bilateral operation to send a page fault completion message to the first application 111. Specifically, the second application 121 writes a second data transmission task into the transmission queue, the second data transmission task including the page fault completion message, to send the page fault completion message to the second network interface card device 122. The second data transmission task can be implemented as SQE in the RDMA Send operation. Correspondingly, the second network interface card device 122 retrieves the second data transmission task from the transmission queue, parses the page fault completion message from it, and transmits it to the first network interface card device 112 via the physical link. When the second application 121 sends a page fault completion message to the first application 111 using the RDMA Send operation, the first application 111 pre-issues a data reception task to its reception queue. This data reception task can be implemented as RQE, and the data reception task includes the local virtual address specified by the first application 111 for the received data. Based on this, when the first network interface card device 112 receives a page fault completion notification message, it retrieves a data reception task from the reception queue, parses the local virtual address from the data reception task, writes the page fault completion message into the physical memory region corresponding to the local virtual address, and writes a CQE to the completion queue to notify the first application 111 to read the page fault completion message from the physical memory region mapped to the local virtual address. The first application 111 reads the page fault completion message from the physical memory region mapped to the local virtual address according to the CQE, and thus knows that the page fault has been resolved, thereby entering the third stage of the data transmission process.

[0090] The steps are shown in Figure 2. The third stage of the data transmission process is described as follows: upon receiving a page fault completion message from the second application, the second data is reread from the second physical memory region mapped to the target virtual address via the first RDMA operation.

[0091] Specifically, see the steps in Figure 2. As shown, specifically, the first application 111 can resend a data acquisition request to the first network interface card (NIC) device 112, so that it can be sent to the second NIC device 122 via the first NIC device 112. This data acquisition request includes a target virtual address. Correspondingly, the second NIC device 122 receives the data acquisition request and parses the target virtual address from it. For a detailed implementation of this process, please refer to the corresponding steps in the aforementioned first stage, which will not be repeated here.

[0092] The second network interface card (NIC) device 122 can perform virtual address to physical address translation based on the maintained target page table, that is, map the target virtual address to the second physical memory region. The second physical memory region refers to the physical memory region corresponding to the physical address mapped by the target virtual address. Since the address mapping relationship between virtual and physical addresses changes after page fault handling, the term "second" is used here to define the physical memory and distinguish it from the "first physical memory region" in the aforementioned embodiment. Further, refer to the steps in Figure 2. The second network interface card (NIC) device 122 reads the second data from the second physical memory region mapped to the target virtual address. This second data is the data actually requested in the data acquisition request. Then, refer to the steps in Figure 2. As shown, the second network interface card (NIC) device 122 can send the second data to the first NIC device 112, which then reports it to the first application 111. For a detailed implementation of this process, please refer to the corresponding steps in the first stage described above, which will not be repeated here.

[0093] This embodiment proposes a software-implemented RDMA scheme supporting memory page faults: For data retrieval from remote memory, a specific physical memory region is provided remotely. During MR registration, the page fault virtual memory is mapped to this specific physical memory region. Thus, when the RDMA operation accesses the page fault virtual memory, data can be read from the specific physical memory region, ensuring error-free RDMA operations while simultaneously causing memory page faults during the RDMA process. Furthermore, a memory page fault handling process is added between the local and remote applications to handle memory page faults, enabling the local application to successfully retrieve data from the remote memory after the memory page fault handling. Because memory page faults are allowed and supported, the RDMA process supports dynamic allocation of physical memory resources, improving the memory resource utilization of electronic devices while leveraging the advantages of RDMA.

[0094] This disclosure does not limit the specific implementation of identifying whether the first data originates from a specific physical memory region. In some optional embodiments, identifying whether the first data originates from a specific physical memory region may employ, but is not limited to, the following implementation methods:

[0095] Implementation Method 1: Specific data may be stored in a specific physical memory region, and the first application 111 knows this specific data in advance. For example, during the RDMA connection establishment process, the second application 121 may provide the specific data to the first application 111 based on its control plane connection with the first application. The specific data sent by the first application and the second application are stored locally. Based on this, the first application 111 can compare the first data with the locally stored specific data, and if the first data matches the specific data, it determines that the first data comes from the specific physical memory region.

[0096] The specific data can be data with a specific format and / or specific content to distinguish it from data stored in a non-specific physical memory area. The specific format can be a custom format or an unconventional data format; the specific content can be any preset content, such as tagging information or text information, etc. The tagging information can be implemented as a special identifier, a user-defined identifier, etc., and this embodiment does not impose any limitations. If the specific data is implemented as data with a specific format, the first application 111 can identify that the specific data has a specific format, compare the format of the first data with the specific data to determine whether the format of the first data is the specific format. If yes, the first data matches the specific data, indicating that the first data comes from a specific physical memory area; if no, the first data does not match the specific data, indicating that the first data does not come from a specific physical memory area. If specific data is implemented as data with specific content, the first application 111 can identify that the specific data has specific content, compare the first data with the specific data to determine whether the first data contains specific content. If yes, the first data matches the specific data, indicating that the first data comes from a specific physical memory area; if no, the first data does not match the specific data, indicating that the first data does not come from a specific physical memory area.

[0097] Implementation Method 2: A specific physical memory region has a physical address, and the first application 111 knows this physical address in advance. For example, during the RDMA connection establishment process, the second application 121 can provide the physical address of the specific physical memory region to the first application 111. Based on this, when the second network card device 122 returns the first data, it can simultaneously return the physical address from which the first data comes. The first application 111 can determine whether the physical address from which the first data comes is the physical address of the specific physical memory region. If it is, then the first data comes from the specific physical memory region; if it is not, then the first data does not come from the specific physical memory region.

[0098] Implementation Method 3: A specific physical memory region corresponds to specific identification information, such as "SS" or "AA," and the first application 111 knows this specific identification information in advance. For example, during the RDMA connection establishment process, the second application 121 can provide the specific identification information to the first application 111. Based on this, when the second network interface card (NIC) device 122 returns the first data, if the first data comes from the specific physical memory region, it also returns the specific identification information. The first application 111 can determine whether the second NIC device 122 has returned the specific identification information. If yes, the first data comes from the specific physical memory region; if no, the first data does not come from the specific physical memory region. The method by which the second NIC device 122 returns the specific identification information is not limited. For example, it can add the specific identification information to the first data, return the specific identification information as a suffix to the first data, return the specific identification information as header information of the first data, and so on. It should be noted that the first application 111 can identify the information regardless of the return method.

[0099] Through the above optional implementation methods, the data source of the first data can be identified more accurately to determine whether the first data comes from a specific physical memory area.

[0100] This disclosure does not limit the specific implementation of identifying the RDMA connection and performing memory registration between the first application 111 and the second application 121. The RDMA connection establishment process and memory registration process are described below by way of example:

[0101] RDMA Driver Initialization: The first network interface card (NIC) device 112 and the second NIC device 122 can each perform initialization, i.e., RDMA driver initialization. In this embodiment, the RDMA driver initialization process of the first NIC device 112 is not considered; a conventional RDMA driver initialization process can be referenced. In this embodiment, the RDMA driver initialization process of the second NIC device 122 is improved. During this initialization process, the RDMA driver can request a specific physical memory region from the operating system of the second electronic device. Specifically, the second NIC device 122 requests the operating system to create or allocate a physical memory region as a specific physical memory region for the RDMA driver and keeps this specific physical memory region from being swapped out. Its function is to map the virtual address of the memory page fault that occurs during the memory region registration process of the second application 121.

[0102] Memory Registration Process: In this embodiment, the memory registration process of the first application 111 is not discussed; a traditional memory registration process can be referenced. In this embodiment, the memory registration process of the second application 121 is improved, primarily in the allocation of a specific physical memory region and the mapping between that specific physical memory region and the virtual address where a page fault occurs. Specifically, the second application 121 can initiate a memory registration request to the RDMA driver of the second network interface card device 122, which requests the registration of a target virtual memory. The RDMA driver of the second network interface card device 122 can respond to the memory registration request of the second application 121 and register the target virtual memory with the operating system of the second electronic device (specifically, the memory management module within the operating system). Registering the target virtual memory with the operating system of the second electronic device can be understood as creating or allocating a virtual memory region as the target virtual memory through the operating system. During this process, a global page table maintained by the operating system stores the address mapping relationship between virtual memory and physical memory on the second electronic device. By determining whether the virtual address of the target virtual memory exists in this global page table, it can be determined whether there is a virtual address in the target virtual memory that is experiencing a page fault. Virtual addresses not present in the global page table are considered to be page fault addresses. If a page fault occurs in the target virtual memory, the RDMA driver maps the page fault address to a physical address in a specific physical memory region. Upon successful registration of the target virtual memory, the mapping relationship between each virtual address and physical address in the target virtual memory is configured on the second physical network interface card (NIC), so that the second physical NIC maintains this mapping relationship through the target page table. The target page table maintained by the second physical NIC is different from the global page table maintained by the operating system.

[0103] RDMA Connection Establishment and Information Exchange: The first application 111 and the second application 121 establish an RDMA connection through a control plane connection, and exchange the context information required for RDMA communication through this control plane connection. The context information includes, but is not limited to, the target virtual address, length information, rkey, QPN, etc., and may further include the physical address of a specific physical memory region, specific data, or specific identification information. In this embodiment, the implementation method of the control plane connection between the first application 111 and the second application 121 is not limited. For example, it can be a Transmission Control Protocol (TCP) connection or an Open Systems Interconnection (UDP) connection, and there is no limitation on this. Furthermore, the control plane connection between the first application 111 and the second application 121 can be established based on a socket API or a communication manager API (CM API), and this disclosure does not limit this.

[0104] Based on the above steps, the first application 111 and the second application 121 can complete memory registration, establish an RDMA connection, and exchange the context information required for RDMA communication, providing the basic conditions for subsequent data transmission and support for memory page faults.

[0105] Following the above embodiments, the data transmission method provided by the embodiments of this disclosure will be described below from the perspectives of a first application, a second network interface card device, and a second application, respectively. Figure 3 is a schematic flowchart of a data transmission method provided by an exemplary embodiment of this disclosure. As shown in Figure 3, the data transmission method described from the perspective of a first application includes:

[0106] Step 31: Read the first data from the first physical memory region mapped to the target virtual address registered by the second application through the first RDMA operation. The specific physical memory region on the second electronic device where the second application is located is used to map the virtual address where the second application requests registration and a memory page fault occurs.

[0107] Step 32: If the first data is identified as originating from a specific physical memory region, a page fault request is sent to the second application via a second RDMA operation. The page fault processing is used to request that the target virtual address be mapped from the specific physical memory region to the second physical memory region.

[0108] Step 33: Upon receiving a page fault completion message from the second application, read the second data again from the second physical memory region mapped to the target virtual address via the first RDMA operation.

[0109] In this embodiment, the first application reads first data from the first physical memory region mapped to the target virtual address registered by the second application through a first RDMA operation. The first application determines whether a page fault has occurred during the data reading process by judging whether the first data originates from a specific physical memory region. If a page fault occurs, it indicates that the target virtual address is mapped to a specific physical memory region. The first application then sends a page fault request to the second application through a second RDMA operation to request page fault handling. The result of page fault handling is that the target virtual address is remapped from the specific physical memory region to the second physical memory region. After completing the page fault handling, the second application sends a page fault handling completion message to the first application. Upon confirming that the page fault handling is complete, the first application re-reads the second data from the first physical memory region mapped to the target virtual address through the first RDMA operation. The second data is the actual data that the first application needs to read. This process ensures the successful execution of the RDMA data reading process while supporting page faults, achieving on-demand memory allocation without pinning physical memory, thus improving memory utilization.

[0110] In some optional embodiments, specific data is stored in a specific physical memory region, and the specific data is locally stored in the first application. Based on this, identifying that the first data originates from the specific physical memory region includes: comparing the first data with the specific data, and determining that the first data originates from the specific physical memory region if the first data matches the specific data.

[0111] Further optionally, comparing the first data with the locally stored specific data includes: identifying that the specific data has a specific format, comparing the format of the first data with the specific format; if the format of the first data is the same as the specific format, determining that the first data matches the specific data; or, identifying that the specific data has specific content, comparing the content of the first data with the specific content; if the content of the first data is the same as the specific content, determining that the first data matches the specific data.

[0112] Further optionally, the method further includes: receiving the specific data sent by the second application and storing it locally based on the control plane connection between the first application and the second application.

[0113] In some optional embodiments, the first electronic device includes a memory access device supporting RDMA operation. For ease of distinction and description, the memory access device on the first electronic device is referred to as the first memory access device. The first memory access device can be various network interface cards (NICs) or other device forms supporting RDMA operation that will emerge in the future. Based on this, reading first data from the first physical memory region mapped to the target virtual address via the first RDMA operation includes: writing a data read task to a transmission queue between the first application and the first memory access device. The data read task includes a data acquisition request, which is used to request data to be acquired from the target virtual address by the first NIC. The data acquisition request is used to request data to be acquired from the target virtual address. Receiving the first data reported by the first memory access device, wherein the first data is read by the first memory access device from the first physical memory region mapped to the target virtual address according to the data acquisition request.

[0114] In some optional embodiments, sending a page fault request to a second application via a second RDMA operation includes: writing a first data transmission task, which includes a page fault request, into a transmission queue between the first application and the first memory access device, so as to obtain the page fault request from the transmission queue via a first network interface card device and send it to the second application via a second network interface card device.

[0115] In some optional embodiments, the data transmission method further includes: writing a data receiving task to a receiving queue between the first application and the first memory access device, the data receiving task including a local virtual address, for the first memory access device to write a page fault completion message; reading the page fault completion message written by the first network interface card device from a third physical memory region mapped to the local virtual address, the page fault completion message being parsed from a second data transmission task sent by the second network interface card device after the second application completes page fault processing.

[0116] The detailed implementation methods and beneficial effects of each step in this embodiment have been described in detail in the foregoing embodiments, and will not be elaborated here.

[0117] Following the above embodiments, this disclosure also provides a data transmission method applicable to a memory access device on a second electronic device. For ease of distinction, the memory access device on the second electronic device is referred to as a second memory access device. The second memory access device can be various network interface cards (NICs) supporting RDMA operations, or other device forms supporting RDMA operations that will emerge in the future. The second NIC is registered with a specific physical memory region, which is used to map the virtual address of a second application requesting registration and experiencing a memory page fault. Figure 4 is a flowchart illustrating a data transmission method provided in another exemplary embodiment of this disclosure. As shown in Figure 4, the method includes the following steps:

[0118] Step 41: In response to the first RDMA operation initiated by the first application on the first electronic device, send the first data to the first application. The first data is the data in the first physical memory region to which the target virtual address registered by the second application is mapped.

[0119] Step 42: Receive a page fault request sent by the first application through the second RDMA operation. The page fault request is sent by the first application after recognizing that the first data comes from a specific physical memory region of the second electronic device.

[0120] Step 43: Provide the page fault request to the second application so that the second application can perform page fault processing to map the target virtual address from the specific physical memory region to the second physical memory region.

[0121] Step 44: Send a page fault completion message to the first application. The page fault completion message is used to trigger the first application to reread the second data from the second physical memory region through the first RDMA operation.

[0122] In this embodiment, a second memory access device on a second electronic device responds to a first RDMA operation initiated by a first application, reads first data from the first physical memory region mapped to the target virtual address registered by the second application, and sends it to the first application. The first application determines whether a page fault has occurred during the data reading process by judging whether the first data originates from a specific physical memory region. If a page fault occurs, it indicates that the target virtual address is mapped to a specific physical memory region. The first application then sends a page fault request to the second application via a second RDMA operation. The page fault request is sent to the second application via the second memory access device. The second application performs page fault processing based on the request, resulting in the remapping of the target virtual address from the specific physical memory region to the second physical memory region. The second application sends a page fault processing completion message to the first application. Upon confirming that the page fault processing is complete, the first application rereads the second data from the first physical memory region mapped to the target virtual address via the first RDMA operation. The second data is the actual data that the first application needs to read. This process ensures the successful execution of the RDMA data reading process while supporting memory page faults, enabling on-demand memory allocation without pinning physical memory, thus improving memory utilization.

[0123] In some optional embodiments, the data transmission method further includes: registering a specific physical memory region with the operating system of the second electronic device during the initialization process of the second memory access device; registering a target virtual memory with the operating system of the second electronic device in response to a memory registration request from the second application; and mapping the virtual address to the specific physical memory region if a virtual address in the target virtual memory is present and a page fault occurs.

[0124] Following the above embodiments, this disclosure also provides a data transmission method applicable to a second application on a second electronic device. The second electronic device includes a second memory access device, and a specific physical memory region on the second electronic device is used to map the virtual address of a second application requesting registration and experiencing a memory page fault. Figure 5 is a flowchart illustrating a data transmission method provided in another exemplary embodiment of this disclosure. As shown in Figure 5, the method includes:

[0125] Step 51: Receive a page fault request from a first application on a first electronic device. The page fault request is sent by the first application through a first RDMA operation after recognizing that the first data comes from a specific physical memory region. The first data is data in the first physical memory region to which the target virtual address registered by the second application is mapped.

[0126] Step 52: Perform page fault processing on the target virtual address according to the page fault request, so as to map the target virtual address from a specific physical memory region to a second physical address.

[0127] Step 53: Send a page fault completion message to the first application through the third RDMA operation, so that the first application can read the second data from the second physical memory area again through the first RDMA operation.

[0128] In this embodiment, the first application reads first data from the first physical memory region mapped to the target virtual address registered by the second application through a first RDMA operation. The first application determines whether a page fault has occurred during the data reading process by judging whether the first data originates from a specific physical memory region. If a page fault occurs, it indicates that the target virtual address is mapped to a specific physical memory region. The first application then sends a page fault request to the second application through a second RDMA operation to request page fault handling. The result of page fault handling is that the target virtual address is remapped from the specific physical memory region to the second physical memory region. After completing the page fault handling, the second application sends a page fault handling completion message to the first application. Upon confirming that the page fault handling is complete, the first application re-reads the second data from the first physical memory region mapped to the target virtual address through the first RDMA operation. The second data is the actual data that the first application needs to read. This process ensures the successful execution of the RDMA data reading process while supporting page faults, achieving on-demand memory allocation without pinning physical memory, thus improving memory utilization.

[0129] In one optional embodiment, the second electronic device includes a second memory access device, which can be various memory access devices supporting RDMA operation, or other device forms supporting RDMA operation that will emerge in the future. Based on this, sending a page fault completion message to the first application via a third RDMA operation includes: writing a second data transmission task into a transmission queue between the second application and the second memory access device, the second data transmission task including the page fault completion message, so that the second memory access device can retrieve the page fault completion message from the transmission queue and send it to the first application.

[0130] In some optional embodiments, page fault processing is performed on the target virtual address according to the page fault request to map the target virtual address from a specific physical memory region to a second physical address. This includes: notifying the driver of the second memory access device to perform page fault processing on the target virtual address, and receiving a page fault processing completion message returned by the driver. The page fault processing performed by the driver on the target virtual address includes: calling the memory management module of the second electronic device to load the physical page corresponding to the target virtual address from the persistent storage medium into the second physical memory region and updating the address mapping relationship of the target virtual address.

[0131] In one optional embodiment, specific data is stored in a specific physical memory region; before receiving a page fault request reported by the second memory access device, the method further includes: sending the specific data to the first application based on the control plane connection between the first application and the second application.

[0132] This embodiment proposes a software-implemented RDMA scheme supporting memory page faults: For data retrieval from remote memory, a specific physical memory region is provided remotely. During MR registration, the page fault virtual memory is mapped to this specific physical memory region. Thus, when the RDMA operation accesses the page fault virtual memory, data can be read from the specific physical memory region, ensuring error-free RDMA operations while simultaneously causing memory page faults during the RDMA process. Furthermore, a memory page fault handling process is added between the local and remote applications to handle memory page faults, enabling the local application to successfully retrieve data from the remote memory after the memory page fault handling. Because memory page faults are allowed and supported, the RDMA process supports dynamic allocation of physical memory resources, improving the memory resource utilization of electronic devices while leveraging the advantages of RDMA.

[0133] The above description addresses situation 1. The following, in conjunction with the accompanying drawings, will detail the technical solutions provided by the various embodiments of this disclosure for situation 2. A general description of the technical solutions provided for situation 2 is provided below. In the following embodiments, a network interface card (NIC) is used as an example to illustrate the memory access device, but the description is not limited to this.

[0134] The second electronic device can run one or more RDMA applications, including the second application 121. Shared virtual memory and private virtual memory are provided for each of these RDMA applications. That is, these RDMA applications can share the same shared virtual memory and can register their own private virtual memory. Furthermore, there is a mapping relationship between the private virtual memory of each RDMA application and the identification information of the RDMA application. The identification information of the RDMA application can uniquely identify a private virtual memory. The specific implementation of the identification information of the RDMA application is not limited. It can be unique information allocated by the controlled object 123 to each RDMA application, such as a space number or identity (ID), or it can reuse the original identification information of the RDMA application, such as the RDMA application's ID, name, and IP (Internet Protocol) address.

[0135] To prevent page faults in the shared virtual memory, the physical memory mapped to the shared virtual memory needs to be pinned to prevent it from being swapped out. For ease of description and distinction, the physical memory mapped to the shared virtual memory is referred to as shared physical memory. Then, the shared virtual memory is exposed to the RDMA applications on the first electronic device (e.g., first application 111) that communicate with various RDMA applications on the second electronic device. Taking the example of first application 111 writing data to the private virtual memory registered by second application 121, first application 111 writes the data to the shared virtual memory, and then migrates the data from the shared virtual memory to the private virtual memory of second application 121, completing the RDMA operation of writing data to remote memory.

[0136] During the aforementioned data migration process, page faults are allowed. Through page fault handling, physical memory is allocated as needed to the virtual addresses in the private virtual memory where page faults occur. In other words, the physical memory mapped to the private virtual memory is allowed to participate in memory over-allocation. Although keeping the shared virtual memory free from page faults and pinned shared physical memory prevents participation in memory over-allocation, the fact that each RDMA's private virtual memory allows page faults overall helps increase the size of the physical memory space participating in memory over-allocation, thus improving the utilization of physical memory resources. Further, optionally, if the managed object 123 can promptly detect data writing to the shared virtual memory and migrate the data from the shared virtual memory to the private virtual memory, the shared virtual memory doesn't need to be too large. This means the pinned shared physical memory doesn't need to be too large, allowing more physical memory to participate in over-allocation and further improving physical memory utilization.

[0137] The solution provided in this embodiment for situation 2 is also applicable to the architecture of the data transmission system 1 described above. The difference is that a control object 123 is added to the second electronic device to perform data migration between shared virtual memory and various private virtual memory locations, and to solve the memory page fault problem during the data migration process. This embodiment does not limit the specific implementation of the control object 123. In one exemplary embodiment, the control object 123 may be a thread or process in the operating system of the second electronic device. In another exemplary embodiment, the control object 123 is also implemented as a functional module in the driver of the second network card device 122. This embodiment does not impose any limitations. For a related introduction to the data transmission system 1, please refer to the foregoing embodiments. Based on the data transmission system 1, this embodiment provides another data transmission method. Figure 6a is a schematic diagram of the interaction flow of the data transmission system 1 provided in another exemplary embodiment of this disclosure when executing another interaction logic. As shown in Figure 6a, the interaction flow of this data transmission method will be described in detail below with reference to Figure 6a.

[0138] It should be noted that the data transmission method provided in this embodiment is implemented based on the fact that the first application 111 and the second application 121 have completed memory registration and established an RDMA connection through the first network interface card (NIC) device 112 and the second NIC device 122. The interaction between the first NIC device 112 and the first application 111, and between the second NIC device 122 and the second application 121, can be based on local queue pairs. For details regarding queue pairs and the sending queue, receiving queue, and completion queue within the queue pairs, please refer to the detailed descriptions in the foregoing embodiments, which will not be repeated here. Furthermore, the process of establishing an RDMA connection and performing memory registration between the first application 111 and the second application 121 can be found in the descriptions of subsequent embodiments. This embodiment describes the data transmission process between the two applications based on the fact that an RDMA connection has been established and memory registration has been completed.

[0139] Referring to step ① in Figure 6a, the first application 111 can send a data transmission request to the first network interface card (NIC) device 112; correspondingly, the first NIC device 112 receives the data transmission request sent by the first application 111. This data transmission request may include a first virtual address and the identification information of the second application 121 running on the second electronic device. The first virtual address refers to the virtual address on the second electronic device used to store the target data. The target data refers to the data that the first application 111 needs to send to the second electronic device, which can be any type of data, such as audio data or text data, etc., and this embodiment does not impose any restrictions. In this embodiment, the identification information of the second application 121 is used by the control object 123 to identify the data to be migrated from shared virtual memory to the private virtual memory corresponding to the second application 121, rather than the private virtual memory corresponding to other applications. The identification information of the second application 121 can be unique identification information such as the space number allocated by the control object 123 during the registration of the private virtual memory of the second application 121, or it can be reused original identification information of the second application 121, such as the name, IP address, or ID of the second application 121, etc., and this embodiment does not impose any restrictions.

[0140] This embodiment does not limit the method by which the first application 111 obtains the identification information of the second application 121. In an exemplary embodiment, the first application 111 may receive the identification information of the second application 121 sent by the second application 121 during the process of establishing an RDMA connection with the second application 121, based on the control plane connection between the first application 111 and the second application 121. In this embodiment, the implementation method of the control plane connection between the first application 111 and the second application 121 is not limited; for example, it can be a TCP connection or a UDP connection, and there is no limitation in this regard. Furthermore, the control plane connection between the first application 111 and the second application 121 can be established based on a socket API or a CM API, and this disclosure does not limit it in this way. The process of establishing an RDMA connection can be found in the description of the following embodiments, and will not be detailed here.

[0141] Optionally, the first application 111 can write a data write task to the sending queue. The data write task may include a data sending request; in other words, the data sending request can be encapsulated in the data write task, thereby sending the data sending request to the first network interface card device 112 via the sending queue. Correspondingly, the first network interface card device 112 can obtain the data write task from the sending queue and parse the data sending request from it. The data write task can be implemented as an SQE (System Queued Execution Environment).

[0142] Referring to step ② in Figure 6a, when the first network interface card device 112 receives a data transmission request, it retrieves the target data to be sent by the first application 111 from the physical memory mapped to the local virtual address specified by the first application 111, and sends the target data to be sent by the first application 111, the first virtual address, and the identification information of the second application 121 to the second network interface card device 122.

[0143] Optionally, the first network interface card (NIC) device 112 may encapsulate the target data, the first virtual address, and the identification information of the second application 121 in a request packet, and send the request packet to the second NIC device 122 via a physical link. Correspondingly, the second NIC device 122 may receive the request packet sent by the first NIC device 112 and parse the target data, the first virtual address, and the identification information of the second application 121 from the request packet.

[0144] Referring to step ③ in Figure 6a, when the second network interface card (NIC) device 122 receives the target data, the first virtual address, and the identification information of the second application 121, it can write the target data and the identification information of the second application 121 into the shared virtual memory based on the first virtual address. The second NIC device 122 pre-maintains an address mapping relationship between the shared virtual memory and the shared physical memory, for example, through a page table. The first virtual address belongs to the shared virtual memory. Therefore, the process of writing the target data and the identification information of the second application 121 into the shared virtual memory based on the first virtual address is as follows: the second NIC device 122, based on the maintained address mapping relationship between the shared virtual memory and the shared physical memory, converts the first virtual address to its mapped first physical address, and writes the target data and the identification information of the second application 121 into the first physical address mapped to the first virtual address. The first physical address is a physical address in the shared physical memory.

[0145] In this embodiment, the shared virtual memory is associated with a control object 123. Referring to step ④ in Figure 6a, the control object 123 can monitor the data writing process between the second network interface card device 122 and the shared virtual memory. When it detects that the second network interface card device 122 writes target data and the identification information of the second application 121 to the first virtual address in the shared virtual memory, referring to step ⑤ in Figure 6a, the control object 123 can determine the second virtual address from the private virtual memory registered by the second application 121 based on the identification information of the second application 121. If no page fault occurs at the second virtual address, the second physical address mapped to the second virtual address in the non-shared physical memory can be determined. Based on the second physical address mapped to the second virtual address, the target data is directly migrated from the shared virtual memory to the private virtual memory of the second application 121. The migration process specifically involves reading the target data from the first physical location in the shared physical memory and writing the target data to the second physical address mapped to the second virtual address in the non-shared physical memory to complete the migration of the target data from the shared virtual memory to the private virtual memory of the second application 121. Non-shared physical memory is relative to shared physical memory. Non-shared physical memory can participate in memory over-allocation and does not need to be pinned.

[0146] Referring to step ⑥ in Figure 6a, if a page fault occurs at the second virtual address, the control object 123 can perform page fault handling on the second virtual address to map the second virtual address to the second physical address. Referring to step ⑦ in Figure 6a, the control object 123 can migrate the target data from shared virtual memory to private virtual memory according to the second physical address mapped to the second virtual address. The migration process is as follows: read the target data from the first physical location in the shared physical memory and write the target data to the second physical address mapped to the second virtual address in the non-shared physical memory to complete the migration of the target data from shared virtual memory to the private virtual memory of the second application 121.

[0147] In this embodiment, the specific implementation of page fault handling is not limited. In an exemplary embodiment, the memory management module of the second electronic device can be invoked to load the physical page corresponding to the second virtual address from the persistent storage medium to the second physical address in the non-shared physical memory of the second electronic device, establish a mapping relationship between the second virtual address and the second physical address, and provide the second physical address to the control object 123. The control object 123 can establish a mapping relationship between the second virtual address and the second physical address in the target page table.

[0148] In this embodiment, the memory management module can perform page fault handling on the second virtual address based on the following process: The memory management module can check a preset target page table to confirm whether the data corresponding to the second virtual address is in physical memory, that is, to confirm whether the second virtual address is invalid virtual memory. If the data corresponding to the second virtual address is not in physical memory, a free physical page can be selected in physical memory, or a physical page can be selected from multiple physical pages in physical memory and swapped to persistent storage medium (such as a disk) to release the physical memory area; then, the data corresponding to the second virtual address is loaded from the persistent storage medium to the second physical address in the non-shared physical memory of the second electronic device; wherein the selected free physical page or the released physical memory area belongs to non-shared physical memory and has a second physical address. Optionally, a page replacement algorithm can be used for page scheduling. The page replacement algorithm is used to determine the physical page that should be swapped out to persistent storage medium (such as a disk) to make room for the new page, such as FIFO or LRU, etc. The First-In-First-Out (FIFO) replacement algorithm works by selecting the earliest page to enter memory to replace the physical page corresponding to the second virtual address, while the Least Recently Used (LRU) algorithm works by selecting the least recently used page to replace the physical page corresponding to the second virtual address.

[0149] In some optional embodiments, the first application 111 may send a transmission completion message to the first network interface card (NIC) device 112 after completing data transmission. The first NIC device 112 may receive the transmission completion message and send it to the second application 121 via the second NIC device 122. The second application 121 may notify the management object 123 that the first application 111 has completed data transmission based on the transmission completion message, so that the management object 123 can determine whether the data migration is complete after the first application 111 has completed data transmission. Specifically, when the management object 123 receives the transmission completion message, it may iterate through the shared virtual memory to see if the identification information of the second application 121 still exists. If it still exists, it means that the target data migration has not yet been completed, and the target data migration continues while synchronously waiting for the target data migration to end; if it does not exist, it means that the target data migration has been completed, and the data migration process ends.

[0150] In this embodiment, the transmission completion message is sent to the second application 121 so that the second application 121 can notify the management object 123 to determine whether the target data sent by the first application 111 has been migrated. This embodiment does not limit the method of sending the transmission completion message to the second application 121. Optionally, the first application 111 can send the transmission completion message to the second application 121 through the RDMA Send operation in RDMA bilateral operation. The first application 111 can write a third data transmission task to the transmission queue, and the third data transmission task includes the transmission completion message. The third data transmission task refers to a transmission task encapsulated with the transmission completion message, which can be implemented as an SQE in the RDMA Send operation. Correspondingly, the first network interface card (NIC) device 112 obtains the third data transmission task from the transmission queue, parses the third data transmission task, and obtains the transmission completion message from it. The first NIC device 112 encapsulates the transmission completion message in a request packet and sends it to the second NIC device 122 via the physical link.

[0151] Optionally, when sending the aforementioned transmission completion message to the second application 121 using an RDMA Send operation, the second application 121 pre-issues a data reception task to its receive queue. This data reception task can be implemented as an RQE, and includes a local virtual address specified by the second application 121 for receiving data. Based on this, the second network interface device 122 can obtain the data reception task from the receive queue, parse the local virtual address from the data reception task, write the aforementioned transmission completion message into the local virtual address specified in the data reception task, and write a CQE to the completion queue of the second application 121 to notify the second application 121 to read the aforementioned transmission completion message from the physical memory mapped to the local virtual address. The second application 121 then reads the aforementioned transmission completion message from the physical memory region mapped to the local virtual address. In this embodiment, the aforementioned transmission completion message is implemented as data sent by the first application 111 to the second application 121 in the RDMA Send operation.

[0152] In some optional embodiments, the second application 121 may send a migration completion message to the second network interface card (NIC) device 122 after the managed object 123 has completed the target data migration. The second NIC device 122 may send the migration completion message to the first NIC device 112, which will then send it to the first application 111.

[0153] In this embodiment, the migration completion message is sent to the first application 111 to notify it that the migration of the target data has been completed. This embodiment does not limit the method of sending the migration completion message to the first application 111. Optionally, the second application 121 can send the migration completion message to the first application 111 via an RDMA Send operation in an RDMA bilateral operation. The second application 121 can write a fourth data sending task to the sending queue, and the fourth data sending task includes the migration completion message. The fourth data sending task refers to a sending task encapsulated with the migration completion message, which can be implemented as an SQE in an RDMA Send operation. Correspondingly, the second network interface card (NIC) device 122 retrieves the fourth data sending task from the sending queue, parses the fourth data sending task, and obtains the migration completion message from it. The second NIC device 122 encapsulates the migration completion message in a request packet and sends it to the first NIC device 112 via the physical link.

[0154] Optionally, when sending the migration completion message to the first application 111 using an RDMA Send operation, the first application 111 pre-issues a data reception task to its receive queue. This data reception task can be implemented as an RQE, and includes a local virtual address specified by the first application 111 for receiving data. Based on this, the first network interface card device 112 can obtain the data reception task from the receive queue, parse the local virtual address from the data reception task, write the migration completion message into the local virtual address specified in the data reception task, and write a CQE to the completion queue of the first application 111 to notify the first application 111 to read the migration completion message from the physical memory mapped to the local virtual address. The first application 111 then reads the migration completion message from the physical memory region mapped to the local virtual address. In this embodiment, the migration completion message is implemented as data sent by the second application 121 to the first application 111 during the RDMA Send operation.

[0155] In this embodiment, the specific implementation method for registering shared virtual memory and private virtual memory is not limited. The RDMA connection establishment process and memory registration process are illustrated below:

[0156] RDMA Driver Initialization: The first network interface card (NIC) device 112 and the second NIC device 122 can each perform initialization, i.e., RDMA driver initialization. In this embodiment, the RDMA driver initialization process of the first NIC device 112 is not discussed; a traditional RDMA driver initialization process can be referenced. In this embodiment, the RDMA driver initialization process of the second NIC device 122 is improved. During the RDMA driver initialization process of the second NIC device 122, shared virtual memory is requested from the operating system of the second electronic device, and page faults do not occur in the shared virtual memory. The shared virtual memory is mapped to shared physical memory, which is not allowed to be swapped out; that is, the shared physical memory is in a pinned state, ensuring that page faults do not occur in the shared virtual memory.

[0157] Memory registration process: The driver for the second network interface card (NIC) device 122 can launch a target thread or process in the operating system, acting as a managed object 123 for shared virtual memory. During the registration of private virtual memory by the second application 121, the private virtual memory is registered with the managed object 123. In this embodiment, the memory registration process of the first application 111 is not considered; a traditional memory registration process can be referenced. In this embodiment, the memory registration process of the second application 121 is improved, mainly in that: private virtual memory is registered and registered with the managed object 123, and shared virtual memory is exposed to the first application instead of private virtual memory.

[0158] The registration process for private virtual memory includes: The second application 121 initiates a memory registration request to the RDMA driver of the second network interface card (NIC) device 122, requesting the registration of private virtual memory. The RDMA driver of the second NIC device 122 responds to the memory registration request of the second application 121 and registers the private virtual memory with the operating system of the second electronic device (specifically, the memory management module within the operating system). Registering private virtual memory with the operating system of the second electronic device can be understood as creating or allocating a virtual memory region as private virtual memory through the operating system. The RDMA driver of the second NIC device 122 registers the private virtual memory registered by the second application 121 with the managed object 123. During the registration process of the private virtual memory by the second application 121, the managed object 123 receives the registration request for the private virtual memory and, based on the registration request, records the correspondence between the identification information of the second application 121 and the private virtual memory. Optionally, the controlled object 123 may assign identification information to the second application and provide the identification information of the second application to the second application so that the second application can provide it to the first application through the control plane connection. The shared virtual memory is registered with the second network interface card device 122 so that the second network interface card device 122 can use the shared virtual memory instead of the private virtual memory. That is, the second network interface card device 122 will record the address mapping relationship between the shared virtual memory and the shared physical memory, instead of the address mapping relationship between the private virtual memory and the non-shared physical memory, so that the shared virtual memory can be exposed to the first application instead of the private virtual memory.

[0159] RDMA Connection Establishment and Information Exchange: The first application 111 and the second application 121 establish an RDMA connection through a control plane connection, and exchange the context information required for RDMA communication through this control plane connection. The context information includes, but is not limited to, the first virtual address in the shared virtual memory, length information, rkey, QPN, etc., and may further include the identification information of the second application. In this embodiment, the implementation method of the control plane connection between the first application 111 and the second application 121 is not limited. For example, it can be a TCP connection or a UDP connection, and there is no limitation in this regard. Furthermore, the control plane connection between the first application 111 and the second application 121 can be established based on the Socket API or the CM API, and this disclosure does not impose any restrictions on this.

[0160] Based on the above steps, the first application 111 and the second application 121 can complete memory registration, establish an RDMA connection, and exchange the context information required for RDMA communication, providing the basic conditions for subsequent data transmission and support for memory page faults.

[0161] Following the above embodiments, this disclosure also provides a data transmission method, which can be applied to a first memory access device on a first electronic device. The first memory access device can be various network interface cards (NICs) supporting RDMA operation, or other device forms supporting RDMA operation that will emerge in the future, and is not limited thereto. Figure 6b is a flowchart illustrating a data transmission method provided in an exemplary embodiment of this disclosure. As shown in Figure 6b, it may include:

[0162] Step 71: Receive a data transmission request from the first application. The data transmission request includes a first virtual address and the identification information of the second application running on the second electronic device. The second memory access device of the second electronic device pre-registers a shared virtual memory, and the first virtual address belongs to the shared virtual memory.

[0163] Step 72: Send the target data to be sent by the first application, the first virtual address, and the identification information of the second application to the second memory access device, so that the second memory access device can write the target data and the identification information of the second application into the shared virtual memory based on the first virtual address; wherein, the shared virtual memory is associated with a management object, and the management object is used to migrate the target data from the shared virtual memory to the private virtual memory registered by the second application according to the identification information of the second application.

[0164] In this embodiment, the second electronic device provides both shared virtual memory and a private virtual memory for the second application. The shared virtual memory is exposed to the first application. The first application on the first electronic device sends a data transmission request to the first memory access device, causing the first memory access device to send the target data to be sent by the first application, the first virtual address, and the identification information of the second application to the second memory access device. The second memory access device writes the target data and the identification information of the second application into the shared virtual memory based on the first virtual address. Then, the shared virtual memory, associated with a managed object, migrates the target data from the shared virtual memory to the private virtual memory registered by the second application based on the identification information of the second application, completing the RDMA write operation of this embodiment. This migration process allows for and handles memory page faults, ensuring that the local application can write data to the remote memory while supporting the occurrence of memory page faults. Because it allows for and supports memory page fault handling, the RDMA process supports dynamic allocation of physical memory resources, improving the memory resource utilization of the electronic device while leveraging the advantages of RDMA.

[0165] Further optionally, the method further includes: receiving the identification information of the second application based on the control plane connection between the first application and the second application, and reporting the identification information of the second application to the first application.

[0166] Further optionally, the method further includes: receiving a sending completion message issued by the first application and sending it to the second application via the second memory access device, so that the second application can notify the managed object that the first application has completed data sending; and / or receiving a migration completion message sent by the second memory access device and reporting it to the first application, wherein the migration completion message is issued by the second application to the second memory access device after the managed object has completed data migration.

[0167] Following the above embodiments, this disclosure also provides a data transmission method, which can be applied to a second memory access device. Figure 6c is a schematic flowchart of a data transmission method provided in another exemplary embodiment of this disclosure. As shown in Figure 6c, it may include:

[0168] Step 81: Receive target data, a first virtual address, and identification information of a second application sent by the first memory access device of the first electronic device. The first virtual address belongs to the shared virtual memory registered by the second memory access device.

[0169] Step 82: Based on the first virtual address, write the target data and the identification information of the second application into the shared virtual memory, so that the management object associated with the shared virtual memory can migrate the target data from the shared virtual memory to the private virtual memory registered by the second application according to the identification information of the second application.

[0170] In this embodiment, the second electronic device provides both shared virtual memory and a private virtual memory for the second application. The shared virtual memory is exposed to the first application. The first application on the first electronic device sends a data transmission request to the first memory access device, causing the first memory access device to send the target data to be sent by the first application, the first virtual address, and the identification information of the second application to the second memory access device. The second memory access device writes the target data and the identification information of the second application into the shared virtual memory based on the first virtual address. Then, the shared virtual memory, associated with a managed object, migrates the target data from the shared virtual memory to the private virtual memory registered by the second application based on the identification information of the second application, completing the RDMA write operation of this embodiment. This migration process allows for and handles memory page faults, ensuring that the local application can write data to the remote memory while supporting the occurrence of memory page faults. Because it allows for and supports memory page fault handling, the RDMA process supports dynamic allocation of physical memory resources, improving the memory resource utilization of the electronic device while leveraging the advantages of RDMA.

[0171] Further optionally, the method further includes: receiving a transmission completion message sent by a first memory access device, reporting the transmission completion message to a second application so that the second application can notify the managed object that the first application has completed data transmission; and / or receiving a migration completion message issued by the second application and sending it to the first application via the first memory access device, wherein the migration completion message is issued by the second application after the managed object has completed data migration.

[0172] Further optionally, the method further includes: during the initialization process of the second memory access device, requesting shared virtual memory from the operating system of the second electronic device, wherein the shared virtual memory does not experience page faults; starting a target thread or target process in the operating system as the management object of the shared virtual memory; and during the registration of private virtual memory by the second application, registering the private virtual memory with the management object so that the management object can record the correspondence between the identification information of the second application and the private virtual memory and register the shared virtual memory with the second memory access device, so that the second memory access device can use the shared virtual memory instead of the private virtual memory.

[0173] Following the above embodiments, this disclosure also provides a data transmission method, which can be applied to a controlled object on a second electronic device. Figure 6d is a schematic flowchart of a data transmission method provided in another exemplary embodiment of this disclosure. As shown in Figure 6d, it may include:

[0174] Step 91: Monitor the second memory access device as it writes target data and the identification information of the second application to the first virtual address in the shared virtual memory.

[0175] Step 92: Determine the second virtual address from the private virtual memory registered by the second application based on the identification information of the second application.

[0176] Step 93: If a page fault occurs at the second virtual address, perform page fault handling on the second virtual address to map the second virtual address to the second physical address.

[0177] Step 94: Based on the second physical address mapped to the second virtual address, migrate the target data from shared virtual memory to private virtual memory.

[0178] In this embodiment, the second electronic device provides both shared virtual memory and a private virtual memory for the second application. The shared virtual memory is exposed to the first application. The first application on the first electronic device sends a data transmission request to the first memory access device, causing the first memory access device to send the target data to be sent by the first application, the first virtual address, and the identification information of the second application to the second memory access device. The second memory access device writes the target data and the identification information of the second application into the shared virtual memory based on the first virtual address. Then, the shared virtual memory, associated with a managed object, migrates the target data from the shared virtual memory to the private virtual memory registered by the second application based on the identification information of the second application, completing the RDMA write operation of this embodiment. This migration process allows for and handles memory page faults, ensuring that the local application can write data to the remote memory while supporting the occurrence of memory page faults. Because it allows for and supports memory page fault handling, the RDMA process supports dynamic allocation of physical memory resources, improving the memory resource utilization of the electronic device while leveraging the advantages of RDMA.

[0179] Further optionally, if a memory page fault occurs at the second virtual address, page fault handling is performed on the second virtual address to map the first virtual address to the second physical address, including: calling the memory management module of the second electronic device so that the memory management module loads the memory page corresponding to the second virtual address from the persistent storage medium to the second physical address in the non-shared physical memory of the second electronic device; and establishing a mapping relationship between the second virtual address and the second physical address.

[0180] Further optionally, the method further includes: during the process of registering private virtual memory for the second application, receiving a registration request for the private virtual memory; recording the correspondence between the identification information of the second application and the private virtual memory according to the registration request; and registering the shared virtual memory with the second memory access device so that the second memory access device can use the shared virtual memory instead of the private virtual memory.

[0181] This embodiment proposes a software-implemented RDMA scheme supporting memory page faults: For writing data to remote memory, the remote end provides both shared virtual memory and private virtual memory. The shared virtual memory is exposed to the local application, which writes data to it. A remote control object then migrates the data from the shared virtual memory to the private virtual memory. This migration process allows and handles memory page faults, ensuring both data writing from the local application to the remote memory and support for memory page faults. Because it allows and handles memory page faults, the RDMA process supports dynamic allocation of physical memory resources, improving the memory resource utilization of electronic devices while leveraging the advantages of RDMA.

[0182] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 31 to 33 can be device A; or the execution subject of steps 31 and 32 can be device A, and the execution subject of step 33 can be device B; and so on.

[0183] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 31, 32, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0184] Figure 7 is a schematic diagram of the structure of a data transmission apparatus provided in an exemplary embodiment of this disclosure. The data transmission apparatus is applied to a first application on a first electronic device, which includes a first memory access device. As shown in Figure 7, the apparatus includes: a read module 703, configured to read first data from a first physical memory region mapped to a target virtual address registered by a second application via a first RDMA operation; the second electronic device includes a specific physical memory region for mapping a virtual address that the second application requests to register and where a page fault occurs; a send module 702, configured to send a page fault request to the second application via a second RDMA operation when the first data is identified as originating from the specific physical memory region; the page fault processing is used to request mapping the target virtual address from the specific physical memory region to the second physical memory region; and a read module 703, configured to reread the second data from the second physical memory region via the first RDMA operation upon receiving a page fault processing completion message returned by the second application.

[0185] Optionally, specific data is stored in a specific physical memory region, and the first application locally stores the specific data. The device further includes an identification module for identifying that the first data originates from the specific physical memory region. Specifically, the identification module is configured to compare the first data with the locally stored specific data, and if the first data matches the specific data, determine that the first data originates from the specific physical memory region.

[0186] Further optionally, the identification module compares the first data with locally stored specific data, including: identifying that the specific data has a specific format, comparing the format of the first data with the specific format; if the format of the first data is the same as the specific format, determining that the first data matches the specific data; or, identifying that the specific data has specific content, comparing the content of the first data with the specific content; if the content of the first data is the same as the specific content, determining that the first data matches the specific data.

[0187] Further optionally, the device further includes: a receiving module 701, configured to receive specific data sent by the second application and store it locally based on the control plane connection between the first application and the second application before reading the first data from the first physical memory region mapped from the target virtual address.

[0188] Optionally, the first electronic device includes a first memory access device. The read module 703 reads first data via a first RDMA operation, including: writing a data read task to a send queue between the first application and the first memory access device, the data read task including a data acquisition request for being read from the send queue by the first memory access device, the data acquisition request being used to request data to be acquired from a target virtual address; and receiving first data reported by the first memory access device, the first data being read by the first memory access device from a first physical memory region mapped to the target virtual address according to the data acquisition request.

[0189] Optionally, when the sending module 702 sends a page fault request to the second application through the second RDMA operation, it is specifically configured to: write a first data sending task into the sending queue between the first application and the first memory access device, the first data sending task including the page fault request, so as to obtain the page fault request from the sending queue through the first memory access device and send it to the second application.

[0190] Optionally, the reading module 703 is further configured to: write a data receiving task to the receiving queue between the first application and the first memory access device, the data receiving task including a local virtual address, so that the first memory access device can write a page fault completion message; and read the page fault completion message from the third physical memory region mapped to the local virtual address, the page fault completion message being parsed from the second data sending task sent after the second application completes page fault handling.

[0191] Figure 8 is a schematic diagram of the structure of a data transmission device provided in another exemplary embodiment of this disclosure. This data transmission device is applied to a second memory access device on a second electronic device. A second application runs on the second electronic device, and a specific physical memory region on the second memory access device is used to map the virtual address where the second application requests registration and a memory page fault occurs.

[0192] As shown in Figure 8, the device includes: a sending module 801, configured to send first data to the first application in response to a first remote direct memory access operation initiated by a first application on a first electronic device; the first data being data in a first physical memory region mapped to a target virtual address registered by a second application; a receiving module 803, configured to receive a page fault request sent by the first application through a second RDMA operation; the page fault request being sent by the first application after recognizing that the first data originates from a specific physical memory region; a providing module 802, configured to provide the page fault request to the second application for page fault processing to map the target virtual address from the specific physical memory region to the second physical memory region; and a sending module 801, configured to send a page fault processing completion message to the first application; the page fault processing completion message triggers the first application to reread the second data from the second physical memory region through the first RDMA operation.

[0193] Optionally, the sending module 801 is further configured to: register a specific physical memory region with the operating system of the second electronic device during the initialization process of the second memory access device; register a target virtual memory with the operating system of the second electronic device in response to the memory registration request of the second application; and map the virtual address to the specific physical memory region if a virtual address in the target virtual memory is present and a memory page fault occurs.

[0194] Figure 9 is a schematic diagram of the structure of a data transmission apparatus provided in another exemplary embodiment of this disclosure. This apparatus is applied to a second application on a second electronic device. The second electronic device includes a second memory access device, and a specific physical memory region on the second electronic device is used to map the virtual address of a second application requesting registration and experiencing a memory page fault. As shown in Figure 9, the apparatus includes:

[0195] The receiving module 901 is configured to receive a page fault request from a first application on a first electronic device. The page fault request is sent by the first application on the first electronic device through a first RDMA operation after recognizing that the first data comes from a specific physical memory region. The first data is data in the first physical memory region mapped to a target virtual address registered by a second application. The page fault processing module 902 is configured to perform page fault processing on the target virtual address according to the page fault request, so as to map the target virtual address from the specific physical memory region to a second physical address. The sending module 903 is configured to send a page fault processing completion message to the first application through a third remote direct memory access operation, so that the first application can reread the second data from the second physical memory region through the first remote direct memory access operation.

[0196] Optionally, when the page fault handling module 902 performs page fault handling on the target virtual address according to the page fault request to map the target virtual address from a specific physical memory region to a second physical address, it is specifically configured to: notify the driver of the second memory access device to perform page fault handling on the target virtual address, and receive a page fault handling completion message returned by the driver; wherein, the driver performs page fault handling on the target virtual address by: calling the memory management module of the second electronic device to load the physical page corresponding to the target virtual address from the persistent storage medium into the second physical memory region and update the address mapping relationship of the target virtual address.

[0197] In an optional embodiment, specific data is stored in a specific physical memory region; the sending module 903 is further configured to send the specific data to the first application based on the control plane connection between the first application and the second application before receiving a page fault request reported by the second memory access device.

[0198] This embodiment proposes a software-implemented RDMA scheme supporting memory page faults: For data retrieval from remote memory, a specific physical memory region is provided remotely. During MR registration, the page fault virtual memory is mapped to this specific physical memory region. Thus, when the RDMA operation accesses the page fault virtual memory, data can be read from the specific physical memory region, ensuring error-free RDMA operations while simultaneously causing memory page faults during the RDMA process. Furthermore, a memory page fault handling process is added between the local and remote applications to handle memory page faults, enabling the local application to successfully retrieve data from the remote memory after the memory page fault handling. Because memory page faults are allowed and supported, the RDMA process supports dynamic allocation of physical memory resources, improving the memory resource utilization of electronic devices while leveraging the advantages of RDMA.

[0199] The internal functions and structure of the data transmission device have been described above. Figure 10 is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present disclosure. As shown in Figure 10, in practice, the data transmission device can be implemented as an electronic device, including: a memory 1001, a processor 1002, and a communication component 1003.

[0200] Memory 1001 is used to store computer programs and can be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0201] The memory 1001 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0202] In one exemplary embodiment, a processor 1002, coupled to a memory 1001, is configured to execute a computer program in the memory 1001 for: reading first data from a first physical memory region mapped to a target virtual address registered by a second application via a first RDMA operation, the second electronic device including a specific physical memory region for mapping a virtual address that the second application requests to register and where a page fault occurs; upon recognizing that the first data originates from the specific physical memory region, sending a page fault request to the second application via a second RDMA operation, the page fault handling being used to request mapping the target virtual address from the specific physical memory region to the second physical memory region; and upon receiving a page fault handling completion message returned by the second application, rereading the second data from the second physical memory region via the first RDMA operation.

[0203] Optionally, specific data is stored in a specific physical memory region, and the first application locally stores the specific data. The processor 1002 is further configured to: identify that the first data comes from the specific physical memory region. Specifically, the processor 1002 is configured to compare the first data with the locally stored specific data, and if the first data matches the specific data, determine that the first data comes from the specific physical memory region.

[0204] Further optionally, the processor 1002 compares the first data with locally stored specific data, including: identifying that the specific data has a specific format, comparing the format of the first data with the specific format; if the format of the first data is the same as the specific format, determining that the first data matches the specific data; or, identifying that the specific data has specific content, comparing the content of the first data with the specific content; if the content of the first data is the same as the specific content, determining that the first data matches the specific data.

[0205] Further optionally, the processor 1002 is also configured to receive specific data sent by the second application and store it locally based on the control plane connection between the first application and the second application before reading the first data from the first physical memory region mapped from the target virtual address.

[0206] Optionally, the first electronic device includes a first memory access device. The processor 1002 reads first data via a first RDMA operation, including: writing a data read task to a transmission queue between the first application and the first memory access device, the data read task including a data acquisition request for being read from the transmission queue by the first memory access device, the data acquisition request being used to request data to be acquired from a target virtual address; and receiving first data reported by the first memory access device, the first data being read by the first memory access device from a first physical memory region mapped to the target virtual address according to the data acquisition request.

[0207] Optionally, when the processor 1002 sends a page fault request to the second application through the second RDMA operation, it is specifically used to: write a first data transmission task into the transmission queue between the first application and the first memory access device, the first data transmission task including the page fault request, so as to obtain the page fault request from the transmission queue through the first memory access device and send it to the second application.

[0208] Optionally, the processor 1002 is further configured to: write a data receiving task to a receiving queue between the first application and the first memory access device, the data receiving task including a local virtual address, for the first memory access device to write a page fault completion message; and read the page fault completion message from a third physical memory region mapped to the local virtual address, the page fault completion message being parsed from a second data sending task sent after the second application completes page fault handling.

[0209] Furthermore, as shown in Figure 10, the memory access device also includes other components such as a power supply component 1004. Figure 10 only schematically shows some of the components and does not imply that the memory access device includes only the components shown in Figure 10.

[0210] This disclosure provides a memory access device, including a memory and a processor, the processor being coupled to the memory, for executing a computer program in the memory, for: responding to a first remote direct memory access operation initiated by a first application on a first electronic device; sending first data to the first application, the first data being data in a first physical memory region to which a target virtual address registered by a second application is mapped; receiving a page fault request sent by the first application via a second RDMA operation, the page fault request being sent by the first application upon recognizing that the first data originates from a specific physical memory region; providing the page fault request to the second application for page fault processing to map the target virtual address from the specific physical memory region to a second physical memory region; and sending a page fault processing completion message to the first application, the page fault processing completion message being used to trigger the first application to reread the second data from the second physical memory region via the first RDMA operation.

[0211] Optionally, the processor is further configured to: register a specific physical memory region with the operating system of the second electronic device during the initialization process of the second memory access device; register a target virtual memory with the operating system of the second electronic device in response to a memory registration request from the second application; and map the virtual address to the specific physical memory region if a virtual address in the target virtual memory is present and a page fault occurs.

[0212] This disclosure also provides a memory access device, including a processor and a memory. The processor is coupled to the memory and is configured to execute a computer program in the memory for: receiving a page fault request from a first application on a first electronic device, the page fault request being sent by the first application on the first electronic device via a first RDMA operation after recognizing that first data originates from a specific physical memory region; the first data being data in the first physical memory region mapped to a target virtual address registered by a second application; performing page fault processing on the target virtual address according to the page fault request to map the target virtual address from the specific physical memory region to a second physical address; and sending a page fault processing completion message to the first application via a third remote direct memory access operation, so that the first application can reread the second data from the second physical memory region via the first remote direct memory access operation.

[0213] Optionally, when the processor performs page fault handling on the target virtual address according to the page fault request to map the target virtual address from a specific physical memory region to a second physical address, it is specifically used to: notify the driver of the second memory access device to perform page fault handling on the target virtual address, and receive a page fault handling completion message returned by the driver; wherein, the driver performs page fault handling on the target virtual address by: calling the memory management module of the second electronic device to load the physical page corresponding to the target virtual address from the persistent storage medium into the second physical memory region and update the address mapping relationship of the target virtual address.

[0214] Further optionally, specific data is stored in a specific physical memory region; the processor is also configured to: send the specific data to the first application based on the control plane connection between the first application and the second application before receiving a page fault request reported by the second memory access device.

[0215] This disclosure proposes a software-implemented RDMA scheme supporting memory page faults. For data retrieval from remote memory, a specific physical memory region is provided remotely. During MR registration, the page fault virtual memory is mapped to this specific physical memory region. This allows data to be read from the specific physical memory region when the RDMA operation accesses the page fault virtual memory, ensuring error-free RDMA operations while simultaneously causing a memory page fault during the RDMA process. Furthermore, a memory page fault handling process is added between the local and remote applications to handle memory page faults, enabling the local application to successfully retrieve data from the remote memory after the memory page fault handling. Because memory page faults are allowed and supported, the RDMA process supports dynamic allocation of physical memory resources, improving the memory resource utilization of electronic devices while leveraging the advantages of RDMA.

[0216] Figure 11 is a schematic diagram of the structure of a data transmission device provided in another exemplary embodiment of this disclosure. This data transmission device is applied to a first memory access device on a first electronic device, and a first application runs on the first electronic device. As shown in Figure 11, the device includes: a receiving module 1101, configured to receive a data transmission request issued by the first application, the data transmission request including a first virtual address and identification information of a second application running on a second electronic device; the second memory access device of the second electronic device pre-registers a shared virtual memory, and the first virtual address belongs to the shared virtual memory; a sending module 1102, configured to send the target data to be sent by the first application, the first virtual address, and the identification information of the second application to the second memory access device, so that the second memory access device writes the target data and the identification information of the second application into the shared virtual memory based on the first virtual address; wherein, the shared virtual memory is associated with a management object, and the management object is used to migrate the target data from the shared virtual memory to a private virtual memory registered by the second application according to the identification information of the second application.

[0217] Optionally, the receiving module 1101 is further configured to receive the identification information of the second application based on the control plane connection between the first application and the second application, and report the identification information of the second application to the first application.

[0218] Optionally, the receiving module 1101 is further configured to: receive a sending completion message sent by the first application and send it to the second application via the second memory access device, so that the second application can notify the managed object that the first application has completed data sending; and / or receive a migration completion message sent by the second memory access device and report it to the first application, wherein the migration completion message is sent by the second application to the second memory access device after the managed object has completed data migration.

[0219] Figure 12 is a schematic diagram of the structure of a data transmission device provided in another exemplary embodiment of the present disclosure. As shown in Figure 12, the data transmission device is applied to a second memory access device of a second electronic device, the second electronic device running a second application, and the data transmission device includes:

[0220] The receiving module 1201 is configured to receive target data, a first virtual address, and identification information of a second application sent by a first memory access device of a first electronic device. The first virtual address belongs to the shared virtual memory registered by the second memory access device. The writing module 1202 is configured to write the target data and the identification information of the second application into the shared virtual memory based on the first virtual address, so that the management object associated with the shared virtual memory can migrate the target data from the shared virtual memory to the private virtual memory registered by the second application according to the identification information of the second application.

[0221] Optionally, the receiving module 1201 is further configured to: receive a transmission completion message sent by the first memory access device, report the transmission completion message to the second application, the transmission completion message indicating that the first application has completed data transmission; and / or receive a migration completion message issued by the second application, and send it to the first application via the first memory access device, the migration completion message being issued by the second application after the managed object has completed data migration.

[0222] Optionally, the writing module 1202 is further configured to: request shared virtual memory from the operating system of the second electronic device during the initialization process of the second memory access device, and the shared virtual memory does not experience page faults; start a target thread or target process in the operating system as the management object of the shared virtual memory; and register the private virtual memory with the management object during the registration process of the second application's private virtual memory, so that the management object can record the correspondence between the identification information of the second application and the private virtual memory and register the shared virtual memory with the second memory access device, and the shared virtual memory is used to replace the private virtual memory.

[0223] Figure 13 is a schematic diagram of a data transmission device provided in another exemplary embodiment of the present disclosure. As shown in Figure 13, the data transmission device is applied to a controlled object on a second electronic device. The second electronic device runs a second application and includes a second memory access device. The second memory access device is registered with shared virtual memory. The data transmission device includes:

[0224] The monitoring module 1301 is configured to monitor the second memory access device writing target data and the identification information of the second application to the first virtual address in the shared virtual memory; the determination module 1302 is configured to determine the second virtual address from the private virtual memory registered by the second application based on the identification information of the second application; the page fault handling module 1303 is configured to perform page fault handling on the second virtual address if a memory page fault occurs at the second virtual address, so as to map the second virtual address to the second physical address; the migration module 1304 is configured to migrate the target data from the shared virtual memory to the private virtual memory based on the second physical address mapped to the second virtual address.

[0225] Optionally, if a memory page fault occurs at the second virtual address, the page fault handling module 1303 performs page fault handling on the second virtual address to map the first virtual address to the second physical address. Specifically, this is configured to: call the memory management module of the second electronic device so that the memory management module loads the memory page corresponding to the second virtual address from the persistent storage medium to the second physical address in the non-shared physical memory of the second electronic device; and establish a mapping relationship between the second virtual address and the second physical address.

[0226] Optionally, the monitoring module 1301 is further configured to: receive a registration request for the private virtual memory during the process of registering the private virtual memory for the second application; record the correspondence between the identification information of the second application and the private virtual memory according to the registration request; and register the shared virtual memory with the second memory access device so that the second memory access device can use the shared virtual memory instead of the private virtual memory.

[0227] This embodiment proposes a software-implemented RDMA scheme supporting memory page faults: For writing data to remote memory, the remote end provides both shared virtual memory and private virtual memory. The shared virtual memory is exposed to the local application, which writes data to it. A remote control object then migrates the data from the shared virtual memory to the private virtual memory. This migration process allows and handles memory page faults, ensuring both data writing from the local application to the remote memory and support for memory page faults. Because it allows and handles memory page faults, the RDMA process supports dynamic allocation of physical memory resources, improving the memory resource utilization of electronic devices while leveraging the advantages of RDMA.

[0228] The internal functions and structure of the data transmission device have been described above. In practice, this data transmission device can be implemented as a memory access device, which includes a memory and a processor. The processor is coupled to the memory and executes a computer program in the memory to: receive a data transmission request from a first application, the data transmission request including a first virtual address and identification information of a second application running on a second electronic device; the second memory access device of the second electronic device pre-registers a shared virtual memory, the first virtual address belonging to the shared virtual memory; sends the target data to be transmitted by the first application, the first virtual address, and the identification information of the second application to the second memory access device, so that the second memory access device can write the target data and the identification information of the second application into the shared virtual memory based on the first virtual address; wherein the shared virtual memory is associated with a control object, the control object being used to migrate the target data from the shared virtual memory to a private virtual memory registered by the second application according to the identification information of the second application.

[0229] Optionally, the processor is also configured to receive the identification information of the second application based on the control plane connection between the first application and the second application, and report the identification information of the second application to the first application.

[0230] Optionally, the processor is further configured to: receive a transmission completion message from the first application and send it to the second application via the second memory access device, so that the second application can notify the managed object that the first application has completed data transmission; and / or receive a migration completion message from the second memory access device and report it to the first application, wherein the migration completion message is sent by the second application to the second memory access device after the managed object has completed data migration.

[0231] This disclosure also provides a memory access device, including a memory and a processor. The processor is coupled to the memory and is used to execute a computer program in the memory for: receiving target data, a first virtual address, and identification information of a second application sent by a first memory access device of a first electronic device, wherein the first virtual address belongs to a shared virtual memory registered by the second memory access device; and writing the target data and the identification information of the second application into the shared virtual memory based on the first virtual address, so that a control object associated with the shared virtual memory can migrate the target data from the shared virtual memory to a private virtual memory registered by the second application according to the identification information of the second application.

[0232] Optionally, the processor is further configured to: receive a transmission completion message sent by the first memory access device, report the transmission completion message to the second application, the transmission completion message indicating that the first application has completed data transmission; and / or receive a migration completion message issued by the second application, and send it to the first application via the first memory access device, the migration completion message being issued by the second application after the managed object has completed data migration.

[0233] Optionally, the processor is further configured to: request shared virtual memory from the operating system of the second electronic device during the initialization process of the second memory access device, wherein the shared virtual memory does not experience page faults; start a target thread or target process in the operating system as the management object of the shared virtual memory; and register the private virtual memory with the management object during the registration process of the second application's private virtual memory, so that the management object can record the correspondence between the identification information of the second application and the private virtual memory and register the shared virtual memory with the second memory access device, wherein the shared virtual memory is used to replace the private virtual memory.

[0234] This disclosure also provides an electronic device including a memory and a processor. The processor is coupled to the memory and is configured to execute a computer program in the memory for: monitoring a second memory access device writing target data and identification information of a second application to a first virtual address in shared virtual memory; determining a second virtual address from a private virtual memory registered by the second application based on the identification information of the second application; performing page fault handling on the second virtual address if a page fault occurs at the second virtual address to map the second virtual address to a second physical address; and migrating the target data from the shared virtual memory to the private virtual memory based on the second physical address mapped to the second virtual address.

[0235] Optionally, if a memory page fault occurs at the second virtual address, and the processor performs page fault handling on the second virtual address to map the first virtual address to the second physical address, the specific steps are as follows: calling the memory management module of the second electronic device so that the memory management module loads the memory page corresponding to the second virtual address from the persistent storage medium to the second physical address in the non-shared physical memory of the second electronic device; and establishing a mapping relationship between the second virtual address and the second physical address.

[0236] Optionally, the processor is further configured to: receive a registration request for the private virtual memory during the process of registering the private virtual memory for the second application; record the correspondence between the identification information of the second application and the private virtual memory according to the registration request; and register the shared virtual memory with the second memory access device so that the second memory access device can use the shared virtual memory instead of the private virtual memory.

[0237] This embodiment proposes a software-implemented RDMA scheme supporting memory page faults: For writing data to remote memory, the remote end provides both shared virtual memory and private virtual memory. The shared virtual memory is exposed to the local application, which writes data to it. A remote control object then migrates the data from the shared virtual memory to the private virtual memory. This migration process allows and handles memory page faults, ensuring both data writing from the local application to the remote memory and support for memory page faults. Because it allows and handles memory page faults, the RDMA process supports dynamic allocation of physical memory resources, improving the memory resource utilization of electronic devices while leveraging the advantages of RDMA.

[0238] Accordingly, this disclosure also provides an electronic device, including: the memory access device provided in the above embodiments.

[0239] Accordingly, this disclosure also provides a computer-readable storage medium storing a computer program, which, when executed, can perform the steps that can be executed by an electronic device in the above method embodiments.

[0240] Accordingly, this disclosure also provides a computer program product, including a computer program / instructions, which, when executed by a processor, performs the steps in the data transmission method.

[0241] The aforementioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0242] The aforementioned communication components are configured to facilitate wired or wireless communication between the device containing the communication components and other devices. The device containing the communication components can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication components receive broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication components also include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wide Band (UWB), Bluetooth (BT), and other technologies.

[0243] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0244] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, compact disc read-only memory (CD-ROM), optical storage, etc.) containing computer-usable program code.

[0245] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0246] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0247] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0248] In a typical configuration, a computing device includes one or more processors (Central Processing Units, CPUs), input / output interfaces, network interfaces, and memory.

[0249] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0250] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0251] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0252] The above are merely embodiments of this disclosure and are not intended to limit the scope of this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of the claims of this disclosure.

Claims

1. A data transmission method, applied to a first application on a first electronic device, wherein a second application on a second electronic device is registered with a target virtual address; the method includes: The first data is read from the first physical memory region mapped to the target virtual address through a first remote direct memory access operation. The second electronic device includes a specific physical memory region for mapping the virtual address of the second application request registration and the occurrence of a memory page fault. If the first data is identified as originating from the specific physical memory region, a page fault request is sent to the second application via a second remote direct memory access operation. The page fault processing is used to request that the target virtual address be mapped from the specific physical memory region to the second physical memory region. Upon receiving a page fault completion message from the second application, the second data is reread from the second physical memory region mapped to the target virtual address via the first remote direct memory access operation.

2. The method according to claim 1, wherein, The specific physical memory region stores specific data, and the first application locally stores the specific data; Identifying that the first data originates from the specific physical memory region includes: comparing the first data with the specific data stored locally, and determining that the first data originates from the specific physical memory region if the first data matches the specific data.

3. The method according to claim 2, wherein, Comparing the first data with the specific data stored locally includes: The specific data is identified to have a specific format, and the format of the first data is compared with the specific format; if the format of the first data is the same as the specific format, it is determined that the first data matches the specific data. and / or The specific data is identified to have specific content, and the content of the first data is compared with the specific content; if the content of the first data is the same as the specific content, it is determined that the first data matches the specific data.

4. The method according to claim 2, wherein, Before reading the first data from the first physical memory region mapped to the target virtual address, the process also includes: Based on the control plane connection between the first application and the second application, the specific data sent by the second application is received and stored locally.

5. The method according to claim 1, wherein, The first electronic device includes a first memory access device, which reads first data from a first physical memory region mapped to the target virtual address via a first remote memory direct access operation, including: A data read task is written to a send queue between the first application and the first memory access device. The data read task includes a data retrieval request, which is used to be read by the first memory access device from the send queue. The data retrieval request is used to request data to be retrieved from a target virtual address. The system receives the first data reported by the first memory access device, wherein the first data is read by the first memory access device from the first physical memory region mapped to the target virtual address according to the data acquisition request.

6. The method according to any one of claims 1-5, wherein, The first electronic device includes a first memory access device, which sends a page fault request to the second application via a second remote direct memory access operation, including: A first data sending task is written into the sending queue between the first application and the first memory access device. The first data sending task includes the page fault request, so that the first memory access device can retrieve the page fault request from the sending queue and send it to the second application.

7. The method according to claim 6, wherein, Also includes: A data receiving task is written to the receive queue between the first application and the first memory access device. The data receiving task includes a local virtual address so that the first memory access device can write a page fault completion message. The page fault completion message is read from the third physical memory region mapped to the local virtual address. The page fault completion message is parsed from the second data transmission task sent after the second application completes page fault processing.

8. A data transmission method applied to a second memory access device on a second electronic device, wherein a second application on the second electronic device is registered with a target virtual address, and a specific physical memory region on the second electronic device is used to map the virtual address to which the second application requests registration and a memory page fault occurs; the method includes: In response to a first remote direct memory access operation initiated by a first application on a first electronic device, first data is sent to the first application, wherein the first data is data in the first physical memory region to which the target virtual address is mapped; Receive a page fault request sent by the first application through a second remote direct memory access operation, wherein the page fault request is sent by the first application after recognizing that the first data comes from a specific physical memory region of the second electronic device; The page fault request is provided to the second application so that the second application can perform page fault processing to map the target virtual address from the specific physical memory region to the second physical memory region; A page fault completion message is sent to the first application, which triggers the first application to reread the second data from the second physical memory region through the first remote direct memory access operation.

9. The method according to claim 8, wherein, Also includes: During the initialization process of the second memory access device, the specific physical memory region is registered with the operating system of the second electronic device; In response to the memory registration request of the second application, the target virtual memory is registered with the operating system of the second electronic device; If a virtual address where a page fault occurs exists in the target virtual memory, the virtual address is mapped to the specific physical memory region.

10. A data transmission method applied to a second application on a second electronic device, wherein a specific physical memory region on the second electronic device is used to map a virtual address where the second application requests registration and a memory page fault occurs; the method includes: Receive a page fault request from a first application on a first electronic device. The page fault request is sent by the first application through a second remote direct memory access operation after recognizing that the first data comes from the specific physical memory region. The first data is data in the first physical memory region to which the target virtual address registered by the second application is mapped. Page fault processing is performed on the target virtual address according to the page fault request, so as to map the target virtual address from the specific physical memory region to the second physical memory region; A page fault completion message is sent to the first application via a third remote direct memory access operation, so that the first application can reread the second data from the second physical memory region via the first remote direct memory access operation.

11. The method according to claim 10, wherein, Page fault processing is performed on the target virtual address according to the page fault request to map the target virtual address from the specific physical memory region to a second physical address, including: The second network interface card device driver is notified to perform page fault handling for the target virtual address, and a page fault handling completion message is received from the driver. The page fault handling for the target virtual address by the driver includes: calling the memory management module of the second electronic device to load the physical page corresponding to the target virtual address from the persistent storage medium into the second physical memory region and updating the address mapping relationship of the target virtual address.

12. A data transmission method applied to a first memory access device on a first electronic device, the first electronic device running a first application, the method comprising: Receive a data transmission request sent by the first application, the data transmission request including a first virtual address and the identification information of the second application running on the second electronic device; The second memory access device of the second electronic device pre-registers a shared virtual memory, and the first virtual address belongs to the shared virtual memory; The first application sends the target data to be sent, the first virtual address, and the identification information of the second application to the second memory access device, so that the second memory access device can write the target data and the identification information of the second application into the shared virtual memory based on the first virtual address; The shared virtual memory is associated with a management object, which is used to migrate the target data from the shared virtual memory to a private virtual memory registered by the second application based on the identification information of the second application.

13. The method according to claim 12, wherein, Also includes: Based on the control plane connection between the first application and the second application, the identification information of the second application is received, and the identification information of the second application is reported to the first application.

14. The method according to claim 12 or 13, wherein, Also includes: The system receives a completion message from the first application and sends it to the second application via the second memory access device, so that the second application can notify the controlled object that the first application has completed data transmission. And / or, The system receives a migration completion message from the second memory access device and reports it to the first application. The migration completion message is sent by the second application to the second memory access device after the managed object has completed the data migration.

15. A data transmission method applied to a second memory access device of a second electronic device, the second electronic device running a second application, the method comprising: The device receives target data, a first virtual address, and identification information of the second application sent by a first memory access device of a first electronic device, wherein the first virtual address belongs to the shared virtual memory registered by the second memory access device; Based on the first virtual address, the target data and the identification information of the second application are written into the shared virtual memory, so that the management object associated with the shared virtual memory can migrate the target data from the shared virtual memory to the private virtual memory registered by the second application according to the identification information of the second application.

16. The method according to claim 15, wherein, Also includes: The first application receives a transmission completion message from the first memory access device and reports the transmission completion message to the second application. The transmission completion message indicates that the first application has completed data transmission. And / or, The system receives a migration completion message from the second application and sends it to the first application via the first memory access device. The migration completion message is sent by the second application after the managed object has completed the data migration.

17. The method according to claim 15 or 16, wherein, Also includes: During the initialization process of the second memory access device, the shared virtual memory is requested from the operating system of the second electronic device, and no memory page fault occurs in the shared virtual memory; The target thread or target process is started in the operating system as the object to be managed by the shared virtual memory; During the process of registering the private virtual memory in the second application, the private virtual memory is registered with the management object so that the management object can record the correspondence between the identification information of the second application and the private virtual memory and register the shared virtual memory with the second memory access device. The shared virtual memory is used to replace the private virtual memory.

18. A data transmission method applied to a controlled object on a second electronic device, the second electronic device running a second application and including a second memory access device, the second memory access device being registered with shared virtual memory, the method comprising: Monitor the second memory access device as it writes target data and the identification information of the second application to a first virtual address in the shared virtual memory; Based on the identification information of the second application, the second virtual address is determined from the private virtual memory registered by the second application; If a memory page fault occurs at the second virtual address, page fault handling is performed on the second virtual address to map the second virtual address to the second physical address; Based on the second physical address mapped to the second virtual address, the target data is migrated from the shared virtual memory to the private virtual memory.

19. The method according to claim 18, wherein, If a page fault occurs at the second virtual address, page fault handling is performed on the second virtual address to map the first virtual address to the second physical address, including: The memory management module of the second electronic device is invoked so that the memory management module loads the memory page corresponding to the second virtual address from the persistent storage medium to the second physical address in the non-shared physical memory of the second electronic device; Establish a mapping relationship between the second virtual address and the second physical address.

20. The method according to claim 18 or 19, wherein, Also includes: During the registration of the private virtual memory in the second application, a registration request for the private virtual memory is received; Based on the registration request, record the correspondence between the identification information of the second application and the private virtual memory; The shared virtual memory is registered with the second memory access device so that the second memory access device can use the shared virtual memory instead of the private virtual memory.

21. A memory access device, the memory access device comprising: A memory and a processor, the memory for storing a computer program, the processor coupled to the memory for executing the computer program to implement the steps of the method of any one of claims 7-8, 12-14, and 15-17.

22. An electronic device, comprising: The memory access device as described in claim 19.

23. An electronic device, comprising: Memory and processor; The memory is used to store a computer program, and the processor is coupled to the memory to execute the computer program for: implementing the steps of the method according to any one of claims 1-6, 9-11, and 18-20.

24. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method according to any one of claims 1-20.

25. A computer program product comprising a computer program / instructions that, when executed by a processor, perform the steps of the method according to any one of claims 1-20.

Images