Memory access method and device, server, storage medium and computer program product

Abstract

The invention discloses a memory access method and device, a server, a storage medium and a computer program product, and the method comprises the steps: a first server sends a first message to a second server under the condition of obtaining page error exception information; the first server receives a second message sent by the second server; the first server creates a shadow page table in a kernel space of the first server based on the second message; and the first server accesses the memory of the second server based on the shadow page table. And the second server adds a Type3 device based on the CXL protocol, wherein the Type3 device is used for obtaining the memory data. Wherein the page error exception information represents information triggered when the virtual machine on the first server maps the physical address of the second server for the first time, and the first message carries the page error exception information; a mapping relation between the virtual machine and the physical address of the second server is recorded in the shadow page table.

Description

Technical Field

[0001] The present application relates to the field of virtualization technology, and in particular to a method, device, server, storage medium and computer program product for memory access. Background Art

[0002] In the related art, the execution efficiency of the virtual machine depends on the specification configuration of the host server, that is, the virtual machine is dependent on the physical memory of the host. The number and quality of virtual machines on the same physical server are limited to a certain extent, and it is impossible to maximize the utilization of resources. Summary of the invention

[0003] To solve related technical problems, embodiments of the present application provide a method, device, server, storage medium and computer program product for memory access.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] An embodiment of the present application provides a method for memory access, which is applied to a first server, and the method includes:

[0006] In the case of acquiring page fault exception information, sending a first message to the second server; the page fault exception information represents information triggered when the virtual machine on the first server maps the physical address of the second server for the first time, and the first message carries the page fault exception information;

[0007] receiving a second message sent by the second server; the second message carries page information and address information of the second server;

[0008] Based on the second message, a shadow page table is created in the kernel space of the first server; the shadow page table records a mapping relationship between the virtual machine and the physical address of the second server;

[0009] Based on the shadow page table, the memory of the second server is accessed; wherein the second server obtains memory data from the Type 3 device based on the Compute Express Link (CXL) protocol.

[0010] In the above solution, accessing the memory of the second server based on the shadow page table includes:

[0011] Based on the shadow page table, converting the guest virtual address (GVA) of the virtual machine into a first host physical address (HPA) of the first server;

[0012] An access request is sent to the second server, where the access request is used to request access to the memory of the second server, and the access request carries the first HPA of the first server.

[0013] In the above solution, accessing the memory of the second server based on the shadow page table includes:

[0014] Based on the shadow page table, converting the client virtual address GVA of the virtual machine into the first host physical address HPA of the first server;

[0015] An access request is sent to the second server, where the access request is used to request access to the memory of the second server, and the access request carries the first HPA of the first server.

[0016] In the above scheme, the method further includes:

[0017] Monitoring page fault exception information of the kernel space in the user space, and creating a new node in the least recently used LRU linked list when the page fault exception information is monitored; and / or,

[0018] Based on the page access frequency, create an LRU linked list in user space; and / or,

[0019] Update the LRU linked list according to the page fault exception information and / or the page information and / or the address information; and / or,

[0020] The page information and / or address information in the LRU linked list is sent from the user space to the kernel space, or the page information and / or address information in the LRU linked list is sent to the second server.

[0021] In the above solution, monitoring the page error exception information of the kernel space in the user space includes:

[0022] In user space, monitor the page fault exception information of the userfaultfd module in the user page fault file description in kernel space;

[0023] The sending of the page information and / or address information in the LRU linked list from the user space to the kernel space includes:

[0024] The page information and / or address information in the LRU linked list is sent from the user space to a kernel-based virtual machine KVM.

[0025] In the above scheme, the method further includes:

[0026] Based on the FreeBSD protocol stack and the DPDK protocol stack, memory data is exchanged with the second server through the user space.

[0027] The embodiment of the present application also provides a method for memory access, which is applied to a second server, and the method includes:

[0028] Add Type3 device based on CXL protocol, and the Type3 device is used to obtain memory data;

[0029] Receive a first message sent by a first server; the first message includes page fault exception information; the page fault exception information represents information triggered when the virtual machine on the first server maps the physical address of the second server for the first time;

[0030] A second message is sent to the first server; the second message carries page information and address information of the memory of the second server; the second message is used by the first server to create a shadow page table in the kernel space; the shadow page table records the mapping relationship between the virtual machine and the physical address of the second server; the shadow page table is used by the first server to access the memory of the second server.

[0031] In the above scheme, the method further comprises:

[0032] receiving an access request sent by a first server, where the access request is used to request access to a memory of the second server, and the access request carries a first HPA of the first server;

[0033] The first HPA is converted into a device physical address (DPA) of the Type 3 device, so as to read and write memory data through the Type 3 device.

[0034] In the above solution, converting the first HPA into the DPA of the Type 3 device includes:

[0035] Converting a first HPA of the first server to a first host virtual address (HVA) of the second server;

[0036] converting the first HVA into a second HPA of the second server;

[0037] The second HPA is converted into the DPA of the Type 3 device.

[0038] In the above scheme, the method further comprises:

[0039] Creating multiple memory areas; the address of each memory area corresponds to a physical address of the Type 3 device;

[0040] Based on the CXL dynamic link library interface, one or more memory areas are allocated to the virtual machine user program according to the requirements of the virtual machine user program.

[0041] In the above scheme, the method further comprises:

[0042] Based on the FreeBSD protocol stack and the DPDK protocol stack, memory data is exchanged with the first server through the user space.

[0043] The present application also provides a memory access device, including:

[0044] A first sending unit is configured to send a first message to a second server when page fault exception information is obtained; the page fault exception information represents information triggered when a virtual machine on the first server maps a physical address of the second server for the first time, and the first message carries the page fault exception information;

[0045] A first receiving unit, configured to receive a second message sent by the second server; the second message carries page information and address information of the second server;

[0046] A first creation unit, configured to create a shadow page table in a kernel space of the first server based on the second message; the shadow page table records a mapping relationship between a virtual machine and a physical address of the second server;

[0047] An access unit is used to access the memory of the second server based on the shadow page table; wherein the second server obtains memory data from the Type 3 device based on the CXL protocol.

[0048] The embodiment of the present application also provides another memory access device, including:

[0049] An adding unit, used for adding a Type 3 device based on the CXL protocol, wherein the Type 3 device is used for obtaining memory data;

[0050] A second receiving unit is used to receive a first message sent by a first server; the first message includes page fault exception information; the page fault exception information represents information triggered when the virtual machine on the first server maps the physical address of the second server for the first time;

[0051] The second sending unit is used to send a second message to the first server; the second message carries page information and address information of the memory of the second server; the second message is used by the first server to create a shadow page table in the kernel space; the shadow page table records the mapping relationship between the virtual machine and the physical address of the second server; the shadow page table is used by the first server to access the memory of the second server.

[0052] An embodiment of the present application further provides a server, including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor is used to execute any of the above-described methods for memory access.

[0053] An embodiment of the present application further provides a storage medium on which a computer program is stored, wherein the computer program implements the steps of any of the above methods when executed by a processor.

[0054] An embodiment of the present application further provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0055] The embodiment of the present application provides a method, device, server, storage medium and computer program product for memory access. First, when the first server obtains page fault exception information, the first server sends a first message to the second server, receives a second message sent by the second server, and creates a shadow page table in the kernel space of the first server based on the second message, wherein the page fault exception information represents the information triggered when the virtual machine on the first server maps the physical address of the second server for the first time, the first message carries the page fault exception information, the second message carries the page information and address information of the second server, and the shadow page table records the mapping relationship between the virtual machine and the physical address of the second server; secondly, a Type3 device is added to the second server based on the CXL protocol, the first message sent by the first server is received, and the second message is sent to the first server, wherein the Type3 device is used for the second server to obtain memory data. After the above steps, the virtual machine in the first server can access the remote memory of the second server based on the mapping relationship of the physical address, so as to expand the memory of the virtual machine, thereby reducing the dependence of the virtual machine on the physical memory of the host machine, and greatly improving the number and quality of virtual machines on the same physical server. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A flowchart of a memory access method according to an embodiment of the present application;

[0057] Figure 2 A flowchart of another memory access method according to an embodiment of the present application;

[0058] Figure 3 A schematic diagram of the structure of another memory access system according to an embodiment of the present application;

[0059] Figure 4A schematic diagram of a flow chart of a third memory access method according to an embodiment of the present application;

[0060] Figure 5 A schematic diagram of a fourth memory access method according to an embodiment of the present application;

[0061] Figure 6 A flowchart of a fifth memory access method according to an embodiment of the present application;

[0062] Figure 7 A flowchart of a sixth memory access method according to an embodiment of the present application;

[0063] Figure 8 A flowchart of a seventh memory access method according to an embodiment of the present application;

[0064] Fig. 9 This is a flowchart of an eighth memory access method according to an embodiment of the present application;

[0065] Fig.10 A schematic diagram of the structure of a memory access device according to an embodiment of the present application;

[0066] Fig.11 A schematic diagram of the structure of another memory access device according to an embodiment of the present application;

[0067] Fig.12 A schematic diagram of a server according to an embodiment of the present application;

[0068] Fig.13 A schematic diagram of another server according to an embodiment of the present application. DETAILED DESCRIPTION

[0069] In the field of virtual machines and remote data access, a server may host multiple virtual machines, and virtual machines are dependent on the physical memory of the host machine, and the number and quality of virtual machines on the same physical server are limited. Specifically, memory is a key component of cloud computing virtual machines. In essence, the host machine allocates memory from physical memory based on the kernel virtual machine. Therefore, when the number of virtual machines on the host machine reaches the upper limit and the virtual machine user program has a high demand for memory, it will affect the physical memory performance of the host machine, thereby reducing the execution efficiency of the virtual machine.

[0070] In the related technology, the communication between the server core and the memory is based on the PCIE protocol. The remote direct access device obtains data from the processor core and then sends it out through a message. The whole process involves asynchronous event notifications and multiple data copies, and the local user program has a large delay in accessing the remote memory. Moreover, it lacks an address translation management mechanism, is not suitable for scenarios where virtual machines remotely access memory, and cannot expand memory for virtual machines, so it has great limitations in the field of cloud computing.

[0071] The embodiment of the present application provides a method, device, server, storage medium and computer program product for memory access. First, when the first server obtains page fault exception information, the first server sends a first message to the second server, receives a second message sent by the second server, and creates a shadow page table in the kernel space of the first server based on the second message, wherein the page fault exception information represents the information triggered when the virtual machine on the first server maps the physical address of the second server for the first time, the first message carries the page fault exception information, the second message carries the page information and address information of the second server, and the shadow page table records the mapping relationship between the virtual machine and the physical address of the second server; secondly, a Type3 device is added to the second server based on the CXL protocol, the first message sent by the first server is received, and the second message is sent to the first server, wherein the Type3 device is used for the second server to obtain memory data. After the above steps, the virtual machine in the first server can access the remote memory of the second server based on the mapping relationship of the physical address, so as to expand the memory of the virtual machine, thereby reducing the dependence of the virtual machine on the physical memory of the host machine, and greatly improving the number and quality of virtual machines on the same physical server.

[0072] The present application embodiment provides a memory access method, which is applied to a first server, referring to Figure 1 , the method comprising:

[0073] Step 101: When page error exception information is obtained, a first message is sent to a second server.

[0074] Here, the page fault exception information represents information triggered when the virtual machine on the first server maps the physical address of the second server for the first time, and the first message carries the page fault exception information.

[0075] Step 102: Receive a second message sent by the second server.

[0076] Here, the second message carries the page information and address information of the second server.

[0077] Step 103: Based on the second message, create a shadow page table in the kernel space of the first server.

[0078] Here, the shadow page table records the mapping relationship between the virtual machine and the physical address of the second server.

[0079] Step 104: Based on the shadow page table, access the memory of the second server.

[0080] Here, the second server obtains memory data from the Type 3 device based on the open fast interconnect CXL protocol.

[0081] In the embodiment of the present application, since the memory in the virtual machine of the first server does not occupy the memory space of the host machine, but is bound to the memory of the remote Type 3 device, when the virtual machine user program of the first server accesses the remote memory of the second server for the first time, that is, when the virtual machine user program is mapped to the address space range for the first time, the guest virtual machine page table and the host shadow page table are inconsistent, and the virtual machine cannot map from GVA to the host physical address HPA, which will trigger a page fault exception. Specifically, the page fault exception can be called a page fault exception.

[0082] In the embodiment of the present application, Type 3 device refers to a memory buffer storage device using the CXL.io protocol and the CXL.mem protocol, allowing the CPU to directly access the memory of the Type 3 device without being transferred through the host memory. Among them, the CXL.io protocol is used for device enumeration and configuration components, and the CXL.mem protocol is used for the CPU to directly access the device memory.

[0083] In the embodiment of the present application, the first server includes a user space and a kernel space. The user space is the part of the operating system where the user process runs, and it is at a lower privilege level; the kernel space is the part where the operating system kernel runs, and it is at a higher privilege level. The kernel is responsible for managing hardware resources, providing system call interfaces, maintaining system security and stability, etc. Exemplarily, the user space may include a Quick Emulator (QEMU) module, and QEMU can run independently without hardware virtualization support; the kernel space may include a kernel-based virtual machine (KVM) module. Furthermore, the first server may also include a user program module for the virtual machine.

[0084] In the embodiment of the present application, the first server sends a first message to the second server, the first message includes page fault exception information, and after receiving the first message, the second server sends a second message including page information and address information of the memory of the second server to the first server. Afterwards, the first server can create a shadow page table in the kernel space of the first server based on the second message.

[0085] In the embodiment of the present application, the shadow page table created in the kernel space of the first server is used to establish a physical address mapping relationship between the first server and the second server. Only by establishing the physical address mapping relationship can the first server access the remote memory of the second server. Specifically, the shadow page table is a page table maintained by the virtual machine monitor, which is used to directly map the GVA to the HPA to achieve the mapping of the virtual machine to the physical machine. In actual application, when allocating memory to the virtual machine, it is non-continuous memory, but QEMU must present continuous memory to the virtual machine, so a conversion is required to complete the conversion from the virtual address of the internal process of the virtual machine to the real physical address.

[0086] In an embodiment of the present application, based on the physical address mapping relationship, the remote memory of the second server is accessed, and the second server obtains memory data from the Type 3 device based on the CXL protocol. The remote memory may be memory data obtained by the second server from the Type 3 device based on the CXL protocol.

[0087] In one embodiment, the accessing the memory of the second server based on the shadow page table includes:

[0088] Based on the shadow page table, converting the GVA of the virtual machine into a first HPA of the first server;

[0089] An access request is sent to the second server, where the access request is used to request access to the memory of the second server, and the access request carries the first HPA of the first server.

[0090] In an embodiment of the present application, address conversion in the second server and sending an access request carrying the first HPA to the second server facilitates establishing a physical address mapping relationship between the first server and the second server so that the first server can access the remote memory of the second server.

[0091] In one embodiment, the method further comprises:

[0092] Monitoring page fault exception information of the kernel space in the user space, and creating a new node in the least recently used LRU linked list when the page fault exception information is monitored; and / or,

[0093] Based on the page access frequency, create an LRU linked list in user space; and / or,

[0094] Update the LRU linked list according to the page fault exception information and / or the page information and / or the address information; and / or,

[0095] The page information and / or address information in the LRU linked list is sent from the user space to the kernel space, or the page information and / or address information in the LRU linked list is sent to the second server.

[0096] In an embodiment of the present application, the first server monitors the page fault exception information of the kernel space in the user space. Exemplarily, the QEMU module in the user space includes a local agent module, and the kernel space includes a KVM module and a Userfaultfd module. The local agent module can capture or monitor the page fault exception information from the Userfaultfd module in the Linux kernel space. The page fault exception triggered when the first server maps the physical address of the second server for the first time and the subsequent page access will be transferred to the local agent module of the QEMU process in the user space through the interface of the Userfaultfd module.

[0097] In the embodiment of the present application, an LRU linked list can be created in the user space based on the page access frequency, and the LRU linked list can be updated according to the page fault exception information and / or page information and / or address information. When a page is accessed, a large amount of data will be copied between the user space and the kernel space and the second server.

[0098] In addition, there are page reads and writes in a fixed address range. In actual application, an LRU linked list can be created through HashMap and a bidirectional linked list, and the linked list nodes can be flexibly moved to indicate the frequency of access, thereby improving copy efficiency and reducing data cache. Furthermore, when a page fault exception occurs, a new node can be created in the LRU linked list based on the page fault exception information; when no page fault exception occurs and the linked list contains page information, data can be obtained from the LRU linked list or the LRU linked list can be updated based on the page information and / or address information.

[0099] In an embodiment of the present application, the first server can send the page information and / or address information in the LRU linked list from the user space to the kernel space, and the first server can also send the page information and / or address information in the LRU linked list to the second server, thereby realizing the joint control and implementation of the reading and writing of the remote Type 3 device memory by the user space and the kernel space.

[0100] In an embodiment of the present application, by monitoring page error exception information of kernel space in user space, a least recently used LRU linked list is created in user space based on page access frequency, and the LRU linked list is updated according to the page error exception information and / or page information and / or address information, and the page information and / or address information in the LRU linked list is sent from user space to kernel space, or the page information and / or address information in the LRU linked list is sent to the second server, the data copying efficiency between the user space and the kernel space in the first server and between the first server and the second server can be improved, and data caching during the data copying process can be reduced.

[0101] In one embodiment, monitoring page error exception information of kernel space in user space includes:

[0102] In user space, monitor the page fault exception information of the userfaultfd module in the user page fault file description in kernel space;

[0103] The sending of the page information and / or address information in the LRU linked list from the user space to the kernel space includes:

[0104] The page information and / or address information in the LRU linked list is sent from the user space to a kernel-based virtual machine KVM.

[0105] In an embodiment of the present application, the kernel space may include a KVM module and a Userfaultfd module. The KVM module is used to receive page information and / or address information sent by the local agent module of the user space; the Userfaultfd module is used to capture page fault exception information from the virtual machine or type3 device, and pass it to the local agent module of the user space for processing.

[0106] In an embodiment of the present application, by further limiting the kernel space, that is, the KVM of the kernel space receives the page information and / or address information sent by the user space, and the page fault exception information is monitored by the local agent module of the user space from the Userfaultfd module, the virtual machine's access to the memory can be transferred from the kernel state to the user state, which helps the virtual machine to further access the remote memory of the second server.

[0107] In one embodiment, the method further comprises:

[0108] Based on the FreeBSD protocol stack and the DPDK protocol stack, memory data is exchanged with the second server through the user space.

[0109] In an embodiment of the present application, through the FreeBSD protocol stack and the DPDK protocol stack, the first server can exchange memory data with the second server through the local proxy module in the user space, thereby reducing the delay in sending and receiving data through the kernel protocol stack, and preventing the kernel protocol stack from processing messages and causing an increase in transmission delay, thereby further improving the efficiency of the virtual machine accessing the remote Type 3 device memory.

[0110] The present application embodiment also provides a memory access method, which is applied to the second server. Figure 2 , the method comprising:

[0111] Step 201: Add a Type 3 device based on the CXL protocol, where the Type 3 device is used to obtain memory data.

[0112] Step 202: Receive a first message sent by a first server.

[0113] Here, the first message includes page fault exception information; the page fault exception information represents information triggered when the virtual machine on the first server maps the physical address of the second server for the first time.

[0114] Step 203: Send a second message to the first server.

[0115] Here, the second message carries the page information and address information of the memory of the second server; the second message is used by the first server to create a shadow page table in the kernel space; the shadow page table records the mapping relationship between the virtual machine and the physical address of the second server; the shadow page table is used by the first server to access the memory of the second server.

[0116] In the embodiment of the present application, the second server adds a Type 3 device, exemplarily, such as Figure 3As shown, first, the CPU enumerates the device with the help of the Linux kernel CXL driver and the CXL.io protocol, and registers the device on the CXL bus; then it obtains the basic information of the device and configures the key registers, and establishes the device topology from the Host Bridge (a bridge connecting the CPU and the PCI system) to the Root Complex (an interface between the CPU and the PCIe topology), from the Root Port (a downstream port, connecting the Root Complex and other devices, such as the Switch) to the Switch (providing expansion or aggregation capabilities and allowing more devices to be connected to a PCle port), and finally to Type3; finally, based on the device topology, the address mapping relationship from Host to Type3 is completed through various parsers, the link of the CXL.mem protocol is completed through device binding in the user space, and the spatial range of accessing the device memory and the interweaving relationship of the address space are determined through the fabric memory window (CFMW, CXL Fabric Memory Window).

[0117] In the embodiment of the present application, after adding the Type 3 device, the second server can map the managed memory data to the memory address space of the Type 3 device through the CXL driver. The second server can access the Type 3 device through a remote agent module to eliminate the possibility of program failure due to insufficient memory of the virtual machine.

[0118] In an embodiment of the present application, a first message sent by a first server is received, the first message includes page fault exception information, and the page fault exception information represents information triggered when the first server maps the physical address of the second server for the first time, and then a second message is sent to the first server; the second message includes memory page and memory address information; the second message is used for the first server to create a shadow page table in the kernel space; the shadow page table is used to establish a physical address mapping relationship with the second server; the physical address mapping relationship is used for the first server to access the remote memory of the second server, and can accept the memory access of the first server, thereby realizing the expansion of the memory of the first server, thereby reducing the dependence of the virtual machine on the physical memory of the host machine, and greatly improving the number and quality of virtual machines of the first server.

[0119] In one embodiment, the method further comprises:

[0120] Creating multiple memory areas; the address of each memory area corresponds to a physical address of the Type 3 device;

[0121] Based on the CXL dynamic link library interface, one or more memory areas are allocated to the virtual machine user program according to the requirements of the virtual machine user program.

[0122] In the embodiment of the present application, similar to the QEMU process in the first server creating a memory area to manage the memory of the virtual machine, the second server can also create a uniformly managed address space (AS, AddressSpace) and multiple memory spaces (MR, MemoryRegion), and these MRs will be dynamically mapped to the memory address space of CXL Type3 through the CXL driver.

[0123] In the embodiment of the present application, the defined MR size is not equal to the actual occupied Type 3 memory area size, and the occupied Type 3 memory area can be dynamically adjusted according to the size of the read-write area, that is, one or more MRs can be dynamically allocated according to user program requirements based on the CXL dynamic link library interface.

[0124] In the embodiment of the present application, when accessing the required page from CXL, in addition to extracting the original requested page, the redundant pages nearby can also be cached, thereby reducing the delay in obtaining data from the Type 3 device.

[0125] In an embodiment of the present application, the second server creates a user-mode address space and multiple memory areas, the address of each memory area corresponds to the physical address of the Type 3 device, and based on the CXL dynamic link library interface, dynamically allocates one or more memory areas according to user program requirements, thereby realizing flexible allocation and release of the memory of the Type 3 device.

[0126] In one embodiment, the method further comprises:

[0127] receiving an access request sent by a first server, where the access request is used to request access to a memory of the second server, and the access request carries a first HPA of the first server;

[0128] The first HPA is converted into a device physical address DPA of the Type 3 device, so as to read and write memory data through the Type 3 device.

[0129] In an embodiment of the present application, an access request sent by a first server is received by a second server, and the access request carries a first HPA of the first server. The second server can convert the first HPA into a device physical address DPA of the Type 3 device to read and write memory data through the Type 3 device, which helps to establish a physical address mapping relationship between the first server and the second server, so that the first server can access the remote memory of the second server.

[0130] In one embodiment, converting the first HPA into a device physical address DPA of the Type 3 device includes:

[0131] Converting a first HPA of the first server to a first host virtual address HVA of the second server;

[0132] converting the first HVA into a second HPA of the second server;

[0133] The second HPA is converted into a device physical address DPA of the Type 3 device.

[0134] In an embodiment of the present application, the shadow page table of the first server can realize the conversion between the client virtual address GVA of the first server and the HPA of the first server; thereafter, the conversion between the HPA of the first server and the HVA of the second server can be realized through the local proxy module of the first server and the remote proxy module of the second server; the conversion between HVA and HPA can be realized through the CXL bus and multiple parsers; the conversion between HPA and DPA can be realized through registers on the Type3 device itself.

[0135] In the embodiment of the present application, through address conversion between the first server and the second server, and in the second server, a physical address mapping relationship can be established between the second server and the first server, thereby achieving accurate remote memory access of the first server.

[0136] In one embodiment, the method further comprises:

[0137] Based on the FreeBSD protocol stack and the DPDK protocol stack, memory data is exchanged with the first server through the user space.

[0138] In an embodiment of the present application, through the FreeBSD protocol stack and the DPDK protocol stack, the second server can exchange memory data with the first server through the remote proxy module in the user space, thereby reducing the delay in sending and receiving data through the kernel protocol stack, and preventing the kernel protocol stack from processing messages and causing an increase in transmission delay, thereby further improving the efficiency of the virtual machine in the first server accessing the remote Type3 device memory.

[0139] The present application is further described in detail below in conjunction with application examples.

[0140] This application proposes an application example 1, such as Figure 3 and Figure 4 As shown, Figure 4 The left side of the dotted line in FIG. 1 shows how to access the host physical memory in the related art. Figure 4 The dashed line on the right side shows how the present application accesses the remote CXL Type3 memory, that is, through the local Agent module of the local server and the remote Agent module of the remote server, the virtual machine hosted by the local server accesses the Type3 memory space of the remote server.

[0141] Specifically, the memory access system of the application embodiment 1 proposed in this application is as follows: Figure 3 As shown in the figure, the creation of shadow page table and access to remote memory are realized through local Agent module and remote Agent module. The local Agent module extends and references the function of Linux kernel Userfaultfd, and maps the remote CXL Type3 to the physical memory space of the virtual machine and isolates it separately. The shadow page table corresponding to the virtual machine page table is specially processed. The page fault exception triggered by the first access and the subsequent page access will be transferred to the local Agent module of the QEMU process through the Userfaultfd interface; the remote Agent module relies on the CXL driver, and with the help of the CXL protocol, it can save the conversion from virtual address to physical address, and can quickly apply for address space or directly obtain memory data in Type3 devices. Furthermore, in order to meet the concurrent business of data, it can also be based on the thread pool, taking advantage of the concurrency and short delay between threads, batch write data and perform multi-read operations.

[0142] In the above application embodiment, the CXL protocol is used to access the remote Type 3 memory to indirectly expand the virtual machine memory, thereby reducing the virtual machine's dependence on the host machine's physical memory, which can greatly improve the number and quality of virtual machines on the same physical server and maximize resource utilization.

[0143] This application also proposes a second application example. Figure 5 As shown in the figure, the local Agent and the remote Agent start the DPDK service through the fstack interface. The two send data directly through DPDK, which reduces the latency of sending and receiving data through the kernel protocol stack and further improves the efficiency of virtual machines accessing remote Type3 memory.

[0144] The virtual machine needs to access the remote CXL Type3 memory multiple times for address mapping and data transmission, which is bound to have a higher latency than accessing the local memory. In order to reduce the access latency, the local Agent and the remote Agent use the UDP protocol for interaction, and this application uses the FreeBSD processing protocol stack and DPDK to forward messages to prevent the kernel protocol stack from processing messages and causing increased transmission latency. The interaction information between these two modules is mainly divided into three categories: 1. CXL Type3 registration; 2. Type3 read operations; 3. Type3 write operations. Taking the read operation as an example, Figure 5 As shown in the flow chart of the interaction between the local agent and the remote agent, the local agent sends the read operation, relative address and size as the payload of the message to the remote agent; the remote agent receives the message and performs the corresponding operation, and converts the memory data into bytes and returns it to the local agent as the message payload.

[0145] This application also proposes a third application embodiment, which is applied to a local server, such as Figure 6 As shown in the figure, the local Agent module borrows the Linux kernel epoll mechanism, calls userfaultfd to create the monitored file descriptor, and customizes the event data of remote memory read and write access, mainly including address information and page fault exception reasons, and initializes some functions through the interface UFFDIO_API, and verifies the version number; the local Agent module creates a global variable LRU linked list for local Agent and remote Agent to exchange data, creates a global variable addr and allocates a virtual address range from the user space, and then maps the virtual address space to the kernel space through the interface UFFDIO_REGISTER; the kernel triggers the corresponding event by accessing the address space, and generates event data containing page fault exceptions, which wakes up the loop mechanism of monitoring events. The Agent will block the loop to monitor the file descriptor indicating the page fault exception. If an event is triggered, it will first determine the cause of the event triggering, then extract the address information and the page fault exception reason, and make a response operation.

[0146] There will be a lot of data copying between the local agent in user space and the KVM and remote agent in kernel space, and there will be page read and write in a fixed address range. In order to improve the copy efficiency and reduce the data cache, a data structure with LRU function can be created through HashMap and bidirectional linked list (i.e. Figure 6 The LRU linked list in the table is used to access the address range (key) to the linked list node (val). The capacity of the bidirectional linked list is fixed, and the linked list node can be flexibly moved to indicate the frequency of access. If the accessed page node is found in the hash table, the node is moved to the head of the linked list and the data of the node is updated or obtained; if it is not found in the hash table, a new node is created and placed at the head, and then data is obtained from the remote agent; if the number of linked list nodes is exceeded, the tail node is deleted.

[0147] In actual application, if the virtual machine user program accesses the physical memory mapped to the virtual machine by remote Type3 for the first time, it will cause a page fault exception, and then fall into the KVM of the kernel space, and then call userfaultfd to access the registered virtual address space and generate a page fault exception. The Agent module will monitor and extract the page fault cause and address of the event, then encapsulate it into an Ethernet message and pass it to the remote server. After receiving the reply, it will use the address information and memory size as the zero page and send it through Figure 6The ioctl interface in the client notifies KVM to create a shadow page table and perform hash mapping with the virtual machine page table. If the remote memory is read or written, the remote end is also notified through Ethernet messages. The remote agent needs to do address adaptation again, and then obtain the data corresponding to the address space as the payload of the Ethernet message, and return the result to the local agent. The agent copies the data to the LRU linked list, and uses the UFFDIO_COPY method to copy the data from the linked list to the virtual page corresponding to the page error, passing the data from the user state to the kernel state, and finally KVM passes the data to the user program of the virtual machine.

[0148] This application also proposes a fourth application embodiment, which is applied to a remote server, such as Figure 7 As shown, the remote agent will also create a uniformly managed AS and several MRs. Since the address space managed by the remote agent is not consistent with the address space ultimately obtained by the virtual machine through the shadow page table, the host and the remote agent need to do an address mapping, that is, each request access requires an address conversion. Among them, each MR has three main parameters: Addr, Size, and Data. Addr represents the starting address of the MR, Size represents the address space size of the MR, Addr and Size together represent a continuous address space, and these address spaces are ultimately mapped to the host KVM shadow page table. We will evenly divide this continuous area into several parts, each of which is represented by a pointer ptr, and organize these local MRs through a linked list, and Data represents the initial address of the linked list. When the remote Agent receives a write request for a certain address segment, it first converts the address, determines which MR the address belongs to based on the address space as, then traverses the linked list Data to find which local areas the address segment belongs to, and finally determines whether the pointer ptr of these local areas is NULL. If it is NULL, the CXL driver interface is called to initialize the memory area and assign it to ptr. If it is not NULL, the data is directly copied to the corresponding area. If it is a read request for a certain address segment, similar to the above operation, the data in the area covered by the address space is extracted, and the address is converted again and encapsulated into an Ethernet message and sent to the host Agent.

[0149] This application also proposes an application embodiment 5, which is applied to a remote server, such as Figure 8As shown in the figure, the remote server adds a Type3 device. First, the CPU enumerates the device with the help of the Linux kernel CXL driver and CXL.io protocol, and registers the device on the CXL bus. Then, the basic information of the device is obtained and the key registers are configured to establish the device topology from Host Bridge to Root Complex, from Root Port to Switch, and finally to Type3. Finally, based on the device topology, the address mapping relationship from Host to Type3 is completed through various parsers, and the link of CXL.mem protocol is completed in the user space through device binding, and the space range for accessing device memory and the interweaving relationship of address space are determined through CFMW. The remote agent initializes various resources about Type3 through the CXL dynamic link library interface, and obtains the accessible address space from CFMW, but this is the mapping of Type3 device (DPA) to host address space (HPA), and the two still need to be converted by various components of the CXL bus. The agent extracts part of the address space and binds it to the user-mode address space AddressSpace, but it will not allocate all the memory contained in CFMW at one time, but dynamically allocates memory according to the needs of user programs. If the local mr needs to read and write Type3 memory, use the CXL interface to allocate memory and point the mr pointer ptr to that part of the memory.

[0150] This application also proposes an application embodiment 6, taking write access to remote memory as an example, such as Fig. 9 As shown, four address translations are required: the first is to translate from the virtual machine user program virtual address GVA to the shadow page table physical address HPA, the second is that the local Agent and the remote Agent translate HPA to HVA in user mode, the third is to implement the HVA to HPA translation through the CXL bus and multiple parsers, and the fourth is to implement the HPA to DPA translation through registers on the Type3 device itself.

[0151] In the above application embodiment, the remote server dynamically allocates the memory of the Type3 device on demand based on the CXL protocol, and establishes a mapping relationship with the physical address of the local server shadow page table, transfers the page error exception received in the kernel state to the user state, and realizes remote memory access by processing the page error exception in the kernel state in the user state; the local Agent module dynamically stores page data through the LRU linked list according to the page access frequency, and the remote Agent module creates multiple logical memory areas to finely manage the Type3 memory, realize flexible allocation and release of Type3 memory, and also ensure local and remote data consistency. Moreover, the two modules communicate through DPDK to reduce the delay of data copying and message kernel processing.

[0152] In order to implement the memory access method of the embodiment of the present application, the embodiment of the present application also provides a memory access device, such as Fig.10 As shown, the device comprises:

[0153] The first sending unit 1001 is used to send a first message to the second server when page fault exception information is obtained; the page fault exception information represents information triggered when the virtual machine on the first server maps the physical address of the second server for the first time, and the first message carries the page fault exception information;

[0154] The first receiving unit 1002 is configured to receive a second message sent by the second server; the second message carries page information and address information of the second server;

[0155] A first creating unit 1003 is used to create a shadow page table in the kernel space of the first server based on the second message; the shadow page table records a mapping relationship between the virtual machine and the physical address of the second server;

[0156] The access unit 1004 is used to access the memory of the second server based on the shadow page table; wherein the second server obtains memory data from the Type 3 device based on the CXL protocol.

[0157] In one embodiment, the accessing the memory of the second server based on the shadow page table includes:

[0158] Based on the shadow page table, converting the client virtual address GVA of the virtual machine into the first host physical address HPA of the first server;

[0159] An access request is sent to the second server, where the access request is used to request access to the memory of the second server, and the access request carries the first HPA of the first server.

[0160] In one embodiment, the device further comprises:

[0161] The monitoring unit is used to monitor the page fault exception information of the kernel space in the user space, and create a new node in the least recently used LRU linked list when the page fault exception information is monitored.

[0162] and / or a second creation unit, configured to create an LRU linked list in a user space based on a page access frequency;

[0163] and / or an updating unit, configured to update the LRU linked list according to the page fault exception information and / or the page information and / or the address information.

[0164] And / or a third sending unit, used to send the page information and / or address information in the LRU list from the user space to the kernel space, or send the page information and / or address information in the LRU list to the second server.

[0165] In one embodiment, monitoring page error exception information of kernel space in user space includes:

[0166] In user space, monitor the page fault exception information of the userfaultfd module in the user page fault file description in kernel space;

[0167] The sending of the page information and / or address information in the LRU linked list from the user space to the kernel space includes:

[0168] The page information and / or address information in the LRU linked list is sent from the user space to a kernel-based virtual machine KVM.

[0169] In one embodiment, the device further comprises:

[0170] The first data exchange unit is used to exchange memory data with the second server through the user space based on the FreeBSD protocol stack and the DPDK protocol stack.

[0171] In order to implement the memory access method of the embodiment of the present application, the embodiment of the present application also provides a memory access device, such as Fig.11 As shown, the device comprises:

[0172] An adding unit 1101 is used to add a Type 3 device based on the CXL protocol, where the Type 3 device is used to obtain memory data;

[0173] The second receiving unit 1102 is used to receive a first message sent by the first server; the first message includes page fault exception information; the page fault exception information represents information triggered when the virtual machine on the first server maps the physical address of the second server for the first time;

[0174] The second sending unit 1103 is used to send a second message to the first server; the second message carries page information and address information of the memory of the second server; the second message is used by the first server to create a shadow page table in the kernel space; the shadow page table records the mapping relationship between the virtual machine and the physical address of the second server; the shadow page table is used by the first server to access the memory of the second server.

[0175] In one embodiment, the device further comprises:

[0176] The third receiving unit is configured to receive an access request sent by the first server, where the access request is used to request access to the memory of the second server, and the access request carries the first HPA of the first server.

[0177] A conversion unit is used to convert the first HPA into a device physical address DPA of the Type 3 device, so as to read and write memory data through the Type 3 device.

[0178] In one embodiment, converting the first HPA into a device physical address DPA of the Type 3 device includes:

[0179] Converting a first HPA of the first server to a first host virtual address HVA of the second server;

[0180] converting the first HVA into a second HPA of the second server;

[0181] The second HPA is converted into a device physical address DPA of the Type 3 device.

[0182] In one embodiment, the device further comprises:

[0183] A second creation unit is used to create a plurality of memory areas; the address of each of the memory areas corresponds to a physical address of the Type 3 device;

[0184] The allocation unit is used to allocate one or more memory areas to the virtual machine user program based on the CXL dynamic link library interface and according to the requirements of the virtual machine user program.

[0185] In one embodiment, the device further comprises:

[0186] The second data exchange unit is used to exchange memory data with the first server through the user space based on the FreeBSD protocol stack and the DPDK protocol stack.

[0187] In actual application, the above-mentioned units can be implemented by a processor or a communication interface in the memory access device.

[0188] It should be noted that the memory access device provided in the above embodiment is only illustrated by the division of the above program modules. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the above-described processing. In addition, the memory access device provided in the above embodiment and the memory access method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0189] Based on the hardware implementation of the above program modules, the embodiment of the present application also provides a server, such as Fig.12 As shown, the first server 1200 includes:

[0190] The first communication interface 1201 is capable of exchanging information with other network nodes;

[0191] The first processor 1202 is connected to the first communication interface 1201 to implement information exchange with other network nodes, and is used to execute the method provided by one or more technical solutions when running a computer program. The computer program is stored in the first memory 1203.

[0192] When the first processor 1202 runs a computer program to execute the above one or more memory access methods, specifically:

[0193] The first communication interface 1201 is used to send a first message to the second server when page fault exception information is obtained; the page fault exception information represents information triggered when the virtual machine on the first server maps the physical address of the second server for the first time, and the first message carries the page fault exception information;

[0194] The first communication interface 1201 is further used to receive a second message sent by the second server; the second message carries page information and address information of the second server.

[0195] The first processor 1202 is used to create a shadow page table in the kernel space of the first server based on the second message; the shadow page table records the mapping relationship between the virtual machine and the physical address of the second server.

[0196] The first communication interface 1201 is further used to access the memory of the second server based on the shadow page table; wherein the second server obtains memory data from the Type 3 device based on the CXL protocol.

[0197] In one embodiment, the accessing the memory of the second server based on the shadow page table includes:

[0198] Based on the shadow page table, converting the GVA of the virtual machine into a first HPA of the first server;

[0199] An access request is sent to the second server, where the access request is used to request access to the memory of the second server, and the access request carries the first HPA of the first server.

[0200] In one embodiment, the first processor 1202 and the first communication interface 1201 are further used to monitor page error exception information of the kernel space in the user space, and create a new node in the least recently used LRU linked list when the page error exception information is monitored.

[0201] The first processor 1202 is further configured to create an LRU linked list in the user space based on the page access frequency, and / or update the LRU linked list according to the page fault exception information and / or the page information and / or the address information.

[0202] The first communication interface 1201 is further used to send the page information and / or address information in the LRU list from the user space to the kernel space, or to send the page information and / or address information in the LRU list to the second server.

[0203] In one embodiment, monitoring page error exception information of kernel space in user space includes:

[0204] In user space, monitor the page fault exception information of the userfaultfd module in the user page fault file description in kernel space;

[0205] The sending of the page information and / or address information in the LRU linked list from the user space to the kernel space includes:

[0206] The page information and / or address information in the LRU linked list is sent from the user space to the KVM.

[0207] In one embodiment, the first communication interface 1201 is further used to exchange memory data with the second server through the user space based on the FreeBSD protocol stack and the DPDK protocol stack.

[0208] It should be noted that the specific processing process of the first processor 1202 and the first communication interface 1201 can be understood by referring to the above method.

[0209] Of course, in actual application, the various components in the first server 1200 are coupled together through the bus system 1204. It can be understood that the bus system 1204 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 1204 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Fig.12 Various buses are labeled as bus system 1204.

[0210] The first memory 1203 in the embodiment of the present application is used to store various types of data to support the operation of the first server 1200. Examples of such data include: any computer program used to operate on the first server 1200.

[0211] The method disclosed in the above embodiment of the present application can be applied to the first processor 1202, or implemented by the first processor 1202. The first processor 1202 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of the hardware in the first processor 1202 or an instruction in the form of software. The above-mentioned first processor 1202 may be a general-purpose processor, a digital signal processor (DSP, DigitalSignal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The first processor 1202 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in the first memory 1203, and the first processor 1202 reads the information in the first memory 1203, and completes the steps of the above method in combination with its hardware.

[0212] In an exemplary embodiment, the first server 1200 can be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to execute the aforementioned method.

[0213] Based on the hardware implementation of the above program modules, the embodiment of the present application also provides an electronic device, such as Fig.13 As shown, the second server 1300 includes:

[0214] The second communication interface 1301 is capable of exchanging information with other network nodes;

[0215] The second processor 1302 is connected to the second communication interface 1301 to implement information exchange with other network nodes, and is used to execute the method provided by one or more technical solutions when running a computer program. The computer program is stored in the second memory 1303.

[0216] When the second processor 1302 runs the computer program to execute the above one or more memory access methods, specifically:

[0217] The second processor 1302 is configured to add a Type 3 device based on the CXL protocol, where the Type 3 device is used to obtain memory data.

[0218] The second communication interface 1301 is used to receive a first message sent by a first server; the first message includes page fault exception information; the page fault exception information represents information triggered when the virtual machine on the first server maps the physical address of the second server for the first time.

[0219] The second communication interface 1301 is also used to send a second message to the first server; the second message carries the page information and address information of the memory of the second server; the second message is used by the first server to create a shadow page table in the kernel space; the shadow page table records the mapping relationship between the virtual machine and the physical address of the second server; the shadow page table is used by the first server to access the memory of the second server.

[0220] In one embodiment, the second communication interface 1301 is further used to receive an access request sent by the first server, where the access request is used to request access to the memory of the second server, and the access request carries the first HPA of the first server.

[0221] The second processor 1302 is further configured to convert the first HPA into a device physical address DPA of the Type 3 device, so as to read and write memory data through the Type 3 device.

[0222] In one embodiment, converting the first HPA into a device physical address DPA of the Type 3 device includes:

[0223] Converting a first HPA of the first server to a first host virtual address HVA of the second server;

[0224] converting the first HVA into a second HPA of the second server;

[0225] The second HPA is converted into a device physical address DPA of the Type 3 device.

[0226] In one embodiment, the second processor 1302 is further configured to create a plurality of memory areas; an address of each of the memory areas corresponds to a physical address of the Type 3 device.

[0227] The second processor 1302 is further configured to allocate one or more memory areas to the virtual machine user program based on the CXL dynamic link library interface and according to the requirements of the virtual machine user program.

[0228] The second communication interface 1301 is also used to exchange memory data with the first server through the user space based on the FreeBSD protocol stack and the DPDK protocol stack.

[0229] Of course, in actual application, the various components in the second server 1300 are coupled together through the bus system 1304. It can be understood that the bus system 1304 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 1304 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Fig.13 Various buses are labeled as bus system 1304 .

[0230] The second memory 1303 in the embodiment of the present application is used to store various types of data to support the operation of the second server 1300. Examples of such data include: any computer program used to operate on the second server 1300.

[0231] The method disclosed in the above embodiment of the present application can be applied to the second processor 1302, or implemented by the second processor 1302. The second processor 1302 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the second processor 1302. The above-mentioned second processor 1302 may be a general-purpose processor, DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The second processor 1302 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in the second memory 1303, and the second processor 1302 reads the information in the second memory 1303 and completes the steps of the above method in combination with its hardware.

[0232] In an exemplary embodiment, the second server 1300 may be implemented by one or more ASICs, DSPs, PLDs, CPLDs, FPGAs, general purpose processors, controllers, MCUs, Microprocessors, or other electronic components to perform the aforementioned methods.

[0233] It can be understood that the memory (first memory 1203, second memory 1303) of the embodiment of the present application can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a ferromagnetic random access memory, a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memories described in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memories.

[0234] In an exemplary embodiment, the embodiment of the present application further provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, including a first memory 1203 storing a computer program, and the computer program can be executed by the first processor 1202 of the first server 1200 to complete the steps of the first server-side method. For another example, a second memory 1303 storing a computer program can be executed by the second processor 1302 of the second server 1300 to complete the steps of the second server-side method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disk, or CD-ROM.

[0235] Exemplarily, the embodiment of the present application further provides a computer program product, including a computer program, which can be executed by the first processor 1202 of the first server 1200 to complete the steps of the first server-side method. Alternatively, the computer program can be executed by the second processor 1302 of the second server 1300 to complete the steps of the second server-side method.

[0236] It should be noted that: "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0237] The term "one or more" herein refers to any combination of at least two of any one or more of a plurality, for example, including at least one of A, B, and C may refer to any one or more elements selected from the set consisting of A, B, and C. The term "one or more" herein refers to any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, and C may refer to any one or more elements selected from the set consisting of A, B, and C.

[0238] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict. The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. A memory access method, characterized in that: Applied to a first server, the method comprises: In the case of acquiring page fault exception information, sending a first message to the second server; the page fault exception information represents information triggered when the virtual machine on the first server maps the physical address of the second server for the first time, and the first message carries the page fault exception information; receiving a second message sent by the second server; the second message carries page information and address information of the second server; Based on the second message, a shadow page table is created in the kernel space of the first server; the shadow page table records a mapping relationship between the virtual machine and the physical address of the second server; Based on the shadow page table, the memory of the second server is accessed; wherein the second server obtains memory data from the Type 3 device based on the open fast interconnect CXL protocol.

2. The method according to claim 1, characterized in that The accessing the memory of the second server based on the shadow page table includes: Based on the shadow page table, converting the client virtual address GVA of the virtual machine into the first host physical address HPA of the first server; An access request is sent to the second server, where the access request is used to request access to the memory of the second server, and the access request carries the first HPA of the first server.

3. The method according to claim 1, characterized in that The method further comprises: Monitoring page fault exception information of the kernel space in the user space, and creating a new node in the least recently used LRU linked list when the page fault exception information is monitored; and / or, Based on the page access frequency, create an LRU linked list in user space; and / or, Update the LRU linked list according to the page fault exception information and / or the page information and / or the address information; and / or, The page information and / or address information in the LRU linked list is sent from the user space to the kernel space, or the page information and / or address information in the LRU linked list is sent to the second server.

4. The method according to claim 3, characterized in that The monitoring of page error exception information of the kernel space in the user space includes: In user space, monitor the page fault exception information of the userfaultfd module in the user page fault file description in kernel space; The sending of the page information and / or address information in the LRU linked list from the user space to the kernel space includes: The page information and / or address information in the LRU linked list is sent from the user space to a kernel-based virtual machine KVM.

5. The method according to claim 1, characterized in that The method further comprises: Based on the FreeBSD protocol stack and the DPDK protocol stack, memory data is exchanged with the second server through the user space.

6. A memory access method, characterized in that: Applied to the second server, the method comprises: Add Type3 device based on CXL protocol, and the Type3 device is used to obtain memory data; Receive a first message sent by a first server; the first message includes page fault exception information; the page fault exception information represents information triggered when the virtual machine on the first server maps the physical address of the second server for the first time; A second message is sent to the first server; the second message carries page information and address information of the memory of the second server; the second message is used by the first server to create a shadow page table in the kernel space; the shadow page table records the mapping relationship between the virtual machine and the physical address of the second server; the shadow page table is used by the first server to access the memory of the second server.

7. The method according to claim 6, characterized in that The method further comprises: receiving an access request sent by a first server, where the access request is used to request access to a memory of the second server, and the access request carries a first HPA of the first server; The first HPA is converted into a device physical address DPA of the Type 3 device, so as to read and write memory data through the Type 3 device.

8. The method according to claim 7, characterized in that The converting the first HPA into a device physical address DPA of the Type 3 device includes: Converting a first HPA of the first server to a first host virtual address HVA of the second server; converting the first HVA into a second HPA of the second server; The second HPA is converted into a device physical address DPA of the Type 3 device.

9. The method according to claim 7, characterized in that: The method further comprises: Creating multiple memory areas; the address of each memory area corresponds to a physical address of the Type 3 device; Based on the CXL dynamic link library interface, one or more memory areas are allocated to the virtual machine user program according to the requirements of the virtual machine user program.

10. The method according to claim 6, characterized in that The method further comprises: Based on the FreeBSD protocol stack and the DPDK protocol stack, memory data is exchanged with the first server through the user space.

11. A memory access device, characterized in that: include: A first sending unit, configured to send a first message to a second server when page error exception information is obtained; The page fault exception information represents information triggered when the virtual machine on the first server maps the physical address of the second server for the first time, and the first message carries the page fault exception information; A first receiving unit, configured to receive a second message sent by the second server; The second message carries page information and address information of the second server; A first creation unit, configured to create a shadow page table in a kernel space of the first server based on the second message; the shadow page table records a mapping relationship between a virtual machine and a physical address of the second server; An access unit is used to access the memory of the second server based on the shadow page table; wherein the second server obtains memory data from the Type 3 device based on the CXL protocol.

12. A memory access device, characterized in that: include: An adding unit, used for adding a Type 3 device based on the CXL protocol, wherein the Type 3 device is used for obtaining memory data; A second receiving unit, configured to receive a first message sent by the first server; The first message includes page error exception information; The page fault exception information represents information triggered when the virtual machine on the first server maps the physical address of the second server for the first time; A second sending unit, configured to send a second message to the first server; The second message carries the page information and address information of the memory of the second server; the second message is used by the first server to create a shadow page table in the kernel space; the shadow page table records the mapping relationship between the virtual machine and the physical address of the second server; the shadow page table is used by the first server to access the memory of the second server.

13. A server, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor is used to execute the method for memory access described in any one of claims 1 to 5, or to execute the method for memory access described in any one of claims 6 to 10.

14. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 5, or is used to execute the steps of the method according to any one of claims 6 to 10.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 5, or is used to execute the steps of the method according to any one of claims 6 to 10.

Images