Computing system, memory page fault processing method and storage medium

Abstract

The embodiment of the present application provides a computing system, a memory page fault processing method and a storage medium. In the embodiment of the present application, a page fault request service component is added to the pass-through device side, and a page fault request driver is added to the virtual machine management node. The page fault request service component and the page fault request driver cooperate with each other to realize the asynchronous recovery of large-block IO memory page faults corresponding to large-block data, and the pass-through device can asynchronously access the recovered memory pages. Compared with the existing synchronous page fault request recovery method, there is no need to wait for all memory pages corresponding to large-block data to be recovered, which helps to reduce the IO memory page fault recovery time, thereby reducing the delay of the pass-through device accessing memory.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a computing system, a memory page fault processing method, and a storage medium. Background Art

[0002] In the computing field, direct pass-through technology introduces the Input / Output Memory Management Unit (IOMMU). By sacrificing the device's shared capabilities, it dedicates the device to a single client, achieving full device functionality and optimal performance. The IOMMU connects the Direct Memory Access (DMA) I / O bus with the host's memory. The IOMMU translates virtual addresses accessed by pass-through devices into physical addresses, enabling pass-through devices to access host memory.

[0003] In actual applications, the Virtual Machine Manager (VMM) reclaims physical memory, stores the data stored in physical memory on the hard disk, and releases the physical memory pages for use by other processes. When the process of the passthrough device needs memory again, an IO memory page fault occurs.

[0004] The Page Request Interface (PRI) is a standard page fault request protocol for PCI peripherals defined by the PCI-SIG organization. The PRI specification defines a model where a root complex provides PRI services. The PRI services provided by the root complex can recover from I / O memory page faults. However, the inventors discovered through actual testing that the PRI services provided by the root complex take a long time to recover from I / O memory page faults, resulting in significant latency in memory access by direct devices. Summary of the Invention

[0005] Multiple aspects of the present application provide a computing system, a memory page fault processing method and a storage medium to reduce the IO memory page fault recovery time and reduce the delay of direct device access to memory.

[0006] The embodiment of the present application provides a computing system, comprising: a host machine and a virtual machine management node; the host machine is deployed with a virtual machine and has a pass-through device for the virtual machine to pass through mounted thereon; the operating system of the virtual machine includes a memory management module;

[0007] The pass-through device includes: a page fault request service component; the virtual machine management node includes: a page fault request driver;

[0008] The page fault request service component is configured to issue a page fault request to the page fault request driver when a memory page accessed by the pass-through device is missing;

[0009] The page fault request driver is used to asynchronously request the memory management module to recover the missing memory page according to the page fault request; and asynchronously return the identifier of the recovered memory page to the page fault request service component;

[0010] The pass-through device asynchronously accesses the restored memory page according to the identifier of the restored memory page.

[0011] The present application also provides a method for processing a memory page fault, including:

[0012] Obtaining a page fault request issued by a page fault request service component in a pass-through device;

[0013] According to the page missing request, asynchronously requesting the memory management module to recover the missing memory page;

[0014] The identifier of the restored memory page is asynchronously returned to the page fault request service component, so that the pass-through device can asynchronously access the restored memory page according to the identifier of the restored memory page.

[0015] An embodiment of the present application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the above-mentioned memory page fault processing method.

[0016] In an embodiment of the present application, a page fault request service component is added to the pass-through device side, and a page fault request driver is added to the virtual machine management node. The page fault request service component and the page fault request driver work together to achieve asynchronous recovery of large-block IO memory page faults corresponding to large blocks of data. The pass-through device can asynchronously access the recovered memory pages. Compared with existing synchronous page fault request recovery methods, there is no need to wait for all memory pages corresponding to large blocks of data to be fully recovered, which helps to reduce the IO memory page fault recovery time and thus reduce the memory access latency of the pass-through device. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 A schematic diagram of the structure of a computing system provided in an embodiment of the present application;

[0019] Figure 2 for Figure 1 A diagram showing how the computing system handles IO memory page faults.

[0020] Figure 3 A schematic diagram of the structure of another computing system provided in an embodiment of the present application;

[0021] Figure 4 for Figure 3 A diagram showing how the computing system handles IO memory page faults.

[0022] Figure 5 for Figure 3 A schematic diagram of another way a computing system handles IO memory page faults is provided;

[0023] Figure 6 A flowchart of a memory page fault processing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0025] To address the technical issue that the PRI service provided by existing root complexes takes a long time to recover I / O memory page faults, resulting in significant latency in memory access for pass-through devices, some embodiments of the present application add a page fault request service component to the pass-through device side and a page fault request driver to the virtual machine management node. The page fault request service component and the page fault request driver work together to achieve asynchronous recovery of large-block I / O memory page faults corresponding to large blocks of data, allowing pass-through device 102 to asynchronously access recovered memory pages. Compared to existing synchronous page fault request recovery methods, this eliminates the need to wait for all memory pages corresponding to large blocks of data to be fully recovered, helping to reduce I / O memory page fault recovery time and, in turn, memory access latency for pass-through devices.

[0026] The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0027] It should be noted that the same reference numerals denote the same objects in the following figures and embodiments, and therefore, once an object is defined in one figure or embodiment, it does not need to be further discussed in the subsequent figures and embodiments.

[0028] Figure 1 This is a schematic diagram of the structure of a computing system provided in an embodiment of the present application. Figure 1As shown, the computing system includes: a host machine 10 and a virtual machine management node (Virtual Machine Manager, VMM) 20.

[0029] In this embodiment, the host machine 10 refers to a computer device with computing, storage, and communication functions, such as a server-side device. For example, the host machine 10 can be a single server device or a cloud-based server array. Of course, the host machine 10 can also be a terminal device such as a mobile phone, tablet computer, personal computer, or wearable device.

[0030] In this embodiment, a virtual machine (VM) 101 is deployed on the host machine 10. The virtual machine 101 can also be called the guest of the host machine 10. The VM 101 has an independent CPU, memory, network, and disk. The CPU corresponding to the virtual machine 101 can also be called the virtual machine CPU, i.e. Figure 1 vCPU 101a is shown.

[0031] In this embodiment, the VMM 20 refers to a logical node for managing virtual machines, and can be deployed on the host machine 10 or on other physical machines that communicate with the host machine 10. For the embodiment in which the VMM 20 is deployed on the host machine 10, the VMM 20 can run on the processing unit ( Figure 1 (not shown). VMM 20 can perform task scheduling, load balancing, status monitoring, etc. on VM 101. The processing unit of host machine 10 can be a CPU, GPU, etc.

[0032] In the pass-through technology, VM 101 can directly access the I / O hardware mounted on the host machine 10 through VMM 20, so that the I / O operation path of VM 101 is almost the same as the I / O path in a non-virtualized environment. Among them, the I / O hardware is the pass-through device 102. The pass-through device 102 refers to the I / O hardware device mounted on the host machine 10, which may include: a network card, a storage medium, etc. Among them, the storage medium may be a persistent storage medium such as a disk or a hard disk. The pass-through device 102 can be connected to the processing unit of the host machine 10 through a communication interface. Depending on the form of the communication interface, the pass-through device 102 can be implemented as a PCI device, a PCIe device, etc.

[0033] Direct memory access (DMA) is used by the passthrough device 102 to access the host machine 10's memory, namely, the host machine memory 103. DMA provides a data path between the passthrough device 102 and the host machine memory 103, enabling direct data transfer between the passthrough device 102 and the host machine memory 103. The host machine 10's CPU is not required to participate in the data transfer process between the passthrough device 102 and the host machine memory 103.

[0034] In virtualization technology, the operating system (guest OS) running on VM 101 usually does not know the physical memory address of the host machine it accesses, which can also be called the host physical address (HPA), or simply the physical address. If a DMA operation is to be performed, it is possible to damage the memory. Because the pass-through device 102 can only obtain the physical memory address of the virtual machine, which is simply called the virtual address (VA), or the guest physical address (GPA), the pass-through device 102 does not know the mapping relationship between the GPA and the HPA. In the embodiment of the present application, therefore, the IOMMU 104 is introduced in the pass-through technology.

[0035] The IOMMU 104 can be communicatively connected between the passthrough device 102 and the host memory 103. Optionally, the IOMMU 104 can be communicatively connected to the passthrough device 102 and the host memory 103 via a communication interface bus. The serial interface bus can be a PCI interface bus, a PCIe interface bus, etc. The IOMMU 104 can ensure that the passthrough device 102 can access the host memory 103 when performing DMA operations.

[0036] For a host machine without an IOMMU 104, the pass-through device 102 can access the entire memory address space of the host machine 10 through DMA. For a host machine equipped with an IOMMU 104, the virtual machine's CPU 101a can allocate a GPA accessible to the pass-through device 102, and the VMM 20 can allocate a corresponding HPA for the GPA. The IOMMU 104 can maintain a mapping table between the GPA and the HPA, which can also be called a page table. The page table records the correspondence between the GPA and the HPA. When the pass-through device 102 accesses the memory of the host machine 10 through DMA, the VMM 20 can capture the DMA request issued by the pass-through device 102 and transparently pass the DMA request to the IOMMU 104. The IOMMU 104 can obtain the GPA to be accessed from the DMA request; then, the GPA to be accessed is matched in the page table stored in the IOMMU 104 to obtain the HPA corresponding to the GPA to be accessed. For the pass-through device 102 , the memory space corresponding to the host memory 103 can be accessed through the HPA corresponding to the GPA to be accessed.

[0037] In actual application, VMM 20 will reclaim physical memory, store the data stored in physical memory in hard disk, and release physical memory pages for use by other processes. When the process of pass-through device 102 needs memory again, IO memory page fault will occur.

[0038] PRI is an industry-standard PCI bus protocol defined by the PCI-SIG organization. It is used to directly provide page fault requests to the operating system (OS) running on the CPU through devices, thereby improving memory utilization and flexibility. In virtualized cloud computing scenarios, hardware that supports PRI capabilities can elevate cloud computing's memory resource utilization to a new level, significantly reducing the waste of hardware resources and energy. The PRI specification defines a model for providing PRI services using a root complex 30. The root complex is typically encapsulated within the processing unit of the host computer 10.

[0039] In a PCI Express (PCIe) system, a root complex connects the processing unit and memory subsystem to the PCIe switch fabric, which consists of one or more switch devices. Root complex functionality can be implemented as a discrete device or integrated into the processing unit.

[0040] In the embodiments of this application, Figure 1 As shown, the IOMMU 104 may be located in the root complex 30. The PRI specification also provides an IO page fault recovery method using the root complex. Figure 2The PRI specification provided in the embodiment of the present application provides a schematic diagram of the process of providing IO page fault recovery with the Root Complex. Figure 2 As shown in the figure, the process of providing IO page fault recovery based on the Root Complex provided by the PRI specification mainly includes the following steps:

[0041] Step 1: When a PRI-supported passthrough device 102 initiates DMA, it first queries the I / O Translation Lookaside Buffer (IOTLB). If the IOTLB does not cache the physical address corresponding to the virtual address (VA) contained in the DMA, the passthrough device 102 sends an address translation request to the IOMMU 104 in the root complex 30. The address translation request includes the virtual address contained in the DMA.

[0042] Step 2: IOMMU 104 receives the address translation request and queries the recorded correspondence between virtual addresses and physical addresses. In this correspondence, the virtual address refers to the GPA, and the physical address refers to the HPA. If IOMMU 104 does not find the physical address corresponding to the virtual address to be translated in the correspondence between virtual addresses and physical addresses, it determines that the memory is in a page fault state and returns the address translation result to the pass-through device 102. The address translation result in step 2 reflects the memory page fault.

[0043] Step 3: In response to the address translation result reflecting the memory page fault, the pass-through device 102 initiates a page fault request to the root complex 30. The page fault request includes the virtual address corresponding to the page fault memory and the process space identifier of the pass-through device 102.

[0044] Step 4: The IOMMU 104 in the Root Complex 30 receives the data through the memory queue (i.e. Figure 2 The memory queue refers to the message queue in the host memory 103, submits a page request to the operating system, and sends an interrupt to the operating system.

[0045] Step 5: After receiving the interrupt, the operating system software (usually the kernel driver) in the CPU reads the page fault request from the memory queue.

[0046] Step 6: The operating system software in the CPU restores the single memory page by calling the memory management module, which is a software function module in the operating system.

[0047] Step 7: The operating system software in the CPU passes another memory queue (i.e. Figure 2The page fault response queue in the Root Complex 30 sends a page fault response (Page Response) to the Root Complex 30. The page fault response may include the physical address of the physical memory page allocated for the page fault memory.

[0048] Step 8: The IOMMU 104 hardware in the root complex 30 receives and processes the page fault response. Specifically, the IOMMU 104 writes the correspondence between the virtual address corresponding to the page fault and the physical address allocated for the memory page into the virtual address space corresponding to the process space identifier corresponding to the pass-through device 102.

[0049] Step 9: Root Complex 30 sends a page fault response to pass-through device 102. The page fault response is used to indicate that the faulted memory page has been recovered.

[0050] Step 10: The direct device 102 receives the page fault response, determines that the page fault has been recovered, and continues to perform DMA to obtain the data to be accessed. Specifically, the direct device 102 sends an address translation request to the IOMMU 104, and the address translation request includes a virtual address in DMA.

[0051] Step 11: The IOMMU 104 receives the address translation request, queries the recorded correspondence between the virtual address and the physical address, obtains the physical address corresponding to the virtual address in the DMA, and then returns the address translation result to the pass-through device 102. The address translation result includes the physical address corresponding to the virtual address in the DMA.

[0052] Step 12: The pass-through device 102 directly accesses the memory space corresponding to the physical address based on the physical address corresponding to the virtual address in the DMA to obtain the data to be accessed.

[0053] Steps 1-12 above are the primary steps for recovering a memory page using a root complex IO page fault, as provided by the PRI specification. However, through actual testing, the inventors of this application discovered that, when a large number of page faults occur, using root complex IO page fault recovery can severely degrade IO performance. For example, when processing a 1GB continuous page fault in user memory, the root complex IO page fault recovery method provided by the PRI specification would trigger 262,144 page fault interrupts, requiring steps 1-12 to be repeated 262,144 times.

[0054] Root Complex I / O page fault recovery is 3-80 times slower than CPU page fault recovery, and when an I / O page fault occurs, the throughput of the entire I / O device (i.e., the pass-through device) decreases by 30%-40%. This significant performance degradation is due to the lengthy processing chain for root Complex I / O page fault recovery, and the fact that each page fault request can only be processed synchronously for one memory page. A memory page is typically 4KB in size, which undoubtedly results in longer root Complex I / O page fault recovery times and higher latency.

[0055] In order to reduce the time required to recover from an IO page fault and improve the efficiency of IO memory page fault recovery, the present invention provides a new computing system architecture and a new IO memory page fault processing method based on the computing system architecture.

[0056] Figure 3 This is a schematic diagram of another computing system provided in an embodiment of the present application. Figure 3 As shown, the computing system includes a host machine 10 and a virtual machine manager (VMM) 20. In this embodiment, the host machine 10 is deployed with a virtual machine (VM) 101. For a description of the host machine 10, VMM 20, and VM 101, please refer to the relevant content of the above embodiment and will not be repeated here.

[0057] In the pass-through technology, VM 101 can directly access the pass-through device 102 mounted on the host machine 10 through VMM 20. The description of the pass-through device 102 can be found in the relevant content of the above embodiment and will not be repeated here.

[0058] In this embodiment, to reduce the duration of IO page fault recovery and improve the efficiency of IO memory page fault recovery, a page fault request service component 102a is added to the pass-through device 102; and a page fault request driver 201 corresponding to the page fault request service component 102a is added to the VMM 20. The page fault request service component 102a can be implemented as a hardware module, for example, constructed using hardware functional circuits or programmable devices. The programmable device can be an FPGA, ASIC chip, or the like. In this embodiment of the present application, the specific implementation form of the page fault request service component 102a is not limited.

[0059] In this embodiment, the page fault request service component 102a can reuse the communication interface of the direct device 102 to communicate with the processing unit (such as CPU) of the host machine 10. The VMM 20 can be deployed on the processing unit of the host machine 10 and run on the processing unit of the host machine 10, thereby realizing the communication between the page fault request service component 102a and the processing unit (such as CPU) of the host machine 10. Figure 3 Communication connection not shown).

[0060] In this embodiment, the VMM 20 may add a page fault request driver 201 corresponding to the page fault request service component 102a. The page fault request driver may be a software function module running on a processing unit (such as a CPU) of the host machine 10, and is used to respond to page fault requests from the page fault request service component 102a.

[0061] The following combination Figure 4 and Figure 5 The IO page fault processing method provided in the embodiment of the present application is exemplified. Figure 4 As shown, the page fault request service component 102a can issue a page fault request to the page fault request driver when the memory page accessed by the pass-through device 102 is missing. In the embodiment of the present application, the memory page accessed by the pass-through device 102 is missing specifically refers to the physical address corresponding to the virtual address accessed by the pass-through device 102 not being stored in the page table of the IOMMU 104. The missing memory page is the memory page pointed to by the physical address corresponding to the virtual address accessed by the pass-through device 102.

[0062] like Figure 4 As shown, the pass-through device 102 initiates a memory access request, i.e., a DMA request. The pass-through device 102 first queries the IO Translation Lookaside Buffer (IOTLB). If the IOTLB does not cache the physical address corresponding to the virtual address (VA) included in the DMA request, the pass-through device 102 sends an address translation request to the IOMMU 104 in the root complex 30. The address translation request includes the virtual address included in the DMA request. The virtual address included in the memory access request can be a single virtual address or a continuous virtual address space including multiple continuous virtual addresses.

[0063] IOMMU 104 receives the address translation request and queries the page table for the correspondence between the virtual address and the physical address. If IOMMU 104 does not find the physical address corresponding to the virtual address to be translated in the correspondence between the virtual address and the physical address, it determines that the memory is in a page fault state and returns the address translation result to the pass-through device 102. The address translation result is a memory page fault.

[0064] The pass-through device 102 determines the IO memory page fault in response to the address translation result of the memory page fault, that is, the physical address corresponding to the virtual address included in the memory access request does not exist in the page table of the IOMMU 104. Figure 4 As shown, the pass-through device 102 may trigger the page fault request service component 102a to issue a page fault request to the page fault request driver 201 when the physical address corresponding to the virtual address included in the memory access request does not exist in the page table of the IOMMU 104. The physical address corresponding to the virtual address included in the memory access request does not exist in the page table of the IOMMU 104, indicating that the memory page corresponding to the virtual address included in the memory access request is missing. The above-mentioned page fault request includes the virtual address in the memory access request and the process space identifier of the pass-through device 102. The virtual address in the memory access request included in the page fault request may be all the virtual addresses included in the memory access request, or may be part of the virtual addresses included in the memory access request, specifically referring to the virtual address for which the corresponding physical address does not exist in the page table of the IOMMU 104.

[0065] In summary, the pass-through device 102 may trigger the page fault request service component 102a to issue a page fault request to the page fault request driver 201 when the physical address corresponding to the virtual address included in the memory access request does not exist in the page table of the IOMMU 104 .

[0066] For the pass-through device 102 , when the physical address corresponding to the virtual address included in the memory access request does not exist in the page table of the IOMMU 104 , the starting virtual address of the missing memory page, the number of missing memory pages, and the process space identifier can be determined.

[0067] Specifically, the pass-through device 102 initiates a memory access request; first queries the IOTLB; if the IOTLB does not cache the physical address corresponding to the virtual address (VA) included in the memory access request, the pass-through device 102 sends an address translation request to the IOMMU 104 in the root complex 30. The address translation request includes the virtual address included in the memory access request. For the IOMMU 104 to obtain the address translation request, the correspondence between the virtual address and the physical address recorded in the page table can be queried. If the physical address corresponding to the virtual address included in the memory access request is queried, the address translation result reflecting the successful translation is returned to the pass-through device 102. If the physical address corresponding to the virtual address included in the memory access request is not queried in the page table, the address translation result reflecting the memory page fault is returned to the pass-through device 102.

[0068] When the pass-through device 102 receives an address translation result reflecting a memory page fault, it can determine that the physical address corresponding to the virtual address included in the memory access request does not exist in the page table of the IOMMU 104, and determine the virtual address included in the address translation result as the starting virtual address of the missing memory page. Furthermore, the pass-through device 102 can also determine the number of missing memory pages based on the virtual address included in the memory access request and the starting virtual address of the missing memory page. Specifically, the pass-through device 102 can determine the virtual address of the last memory page included in the memory access request based on the virtual address included in the memory access request; and determine the starting virtual address of the missing memory page and the virtual address of the last memory page included in the memory access request to determine the number of missing memory pages, that is, the number of memory pages from the starting virtual address of the missing memory page to the virtual address of the last memory page included in the memory access request, which is the number of missing memory pages. In the embodiment of the present application, the specific value of the number of missing memory pages is not limited and is determined by the actual situation. The number of missing memory pages can be one or more. Multiple refers to two or more.

[0069] Further, the pass-through device 102 may provide the starting virtual address of the missing memory page, the number of missing memory pages and the process space identifier to the page fault request service component 102a. The page fault request service component 102a may generate a page fault request based on the starting virtual address of the missing memory page, the number of missing memory pages and the process space identifier. The page fault request may include the starting virtual address of the missing memory page, the number of missing memory pages and the process space identifier. Further, the page fault request service component 102a may provide the page fault request to the page fault request driver 201 (such as Figure 5 As shown in step 1). In the embodiment of the present application, the page fault request may be an asynchronous page fault request (ASYNC_PAGE_REQUEST, APR).

[0070] In the embodiments of this application, Figure 5 As shown, the page fault request service component 102a can wake up the page fault request driver 201 (such as Figure 5As shown in step 2). For an embodiment in which the VMM 20 runs on the processing unit of the host machine 10, the page fault request driver 201 also runs on the processing unit of the host machine 10 and is communicatively connected to the page fault request service component 102a. Specifically, the page fault request service component 102a can be communicatively connected to the processing unit of the host machine 10 through the communication interface of the pass-through device 102. Accordingly, the page fault request service component 102a can send an interrupt signal to the processing unit through the communication link between the processing unit. In the embodiment of the present application, the implementation form of the interrupt signal is not limited. In some embodiments, the interrupt signal can be a pulse signal. Specifically, the page fault request service component 102a can send an interrupt signal to the processing unit through the communication link between the processing unit and the pass-through device 102 when the memory page accessed by the pass-through device 102 is missing. The processing unit wakes up the page fault request driver 201 in response to the interrupt signal.

[0071] The page fault request driver 201 is awakened and, in response to the page fault request, returns an asynchronous response start message (ASYNC_PAGE_RESPONSE_START, APRS) to the page fault request service component 102a (eg, Figure 5 As shown in step 3). APRS indicates that the asynchronous memory page fault request begins processing.

[0072] The page fault request driver 201 can asynchronously request the memory management module 40 of the operating system (OS) of the virtual machine 101 to restore the memory page of the page fault according to the page fault request (corresponding to Figure 5 (as shown in step 4-x4). In this embodiment, the operating system of the virtual machine 101 can run on the CPU of the virtual machine. The memory management module 40 is a software functional module that manages and allocates memory.

[0073] In the embodiment of the present application, the specific implementation method of the page fault request driver 201 asynchronously requesting the memory management module 40 of the operating system (OS) of the virtual machine 101 to recover the missing memory pages is not limited. In some embodiments, the page fault request driver 201 can asynchronously request the memory management module 40 to recover the missing memory pages in batches.

[0074] Specifically, the page fault request driver 201 can obtain the number of missing memory pages and the process space identifier corresponding to the pass-through device from the page fault request; and loop through the following steps until the number of restored memory pages equals the number of missing memory pages, that is, the missing memory pages are restored. The looping steps include:

[0075] The current batch of virtual addresses to be restored is determined; the asynchronous request memory management module 40 allocates a target memory page for the virtual address to be restored; and the correspondence between the physical address of the target memory page and the current batch of virtual addresses to be restored is written into the page table corresponding to the process space identifier in the IOMMU 104. The page table corresponding to the process space identifier is the process space identifier of the pass-through device 102 included in the page fault request.

[0076] In some embodiments, the current batch may be the first batch. For the first batch, the page fault request driver 201 may obtain the starting virtual address of the missing memory page from the page fault request; and determine the virtual address to be restored in the first batch based on the starting virtual address. In an embodiment of the present application, the number of memory pages restored in each batch is not limited. In some embodiments, the memory page restored in each batch is 1 memory page, then the starting virtual address of the missing memory page can be determined as the virtual address to be restored in the first batch. In other embodiments, the memory pages restored in each batch are M memory pages. Wherein, M≥2 and is an integer. Accordingly, the page fault request driver may start from the starting virtual address of the missing memory page and select M virtual addresses in sequence as the virtual addresses to be restored in the first batch. Figure 5 The number of memory pages restored in each batch is shown as one, but this does not constitute a limitation.

[0077] In other embodiments, the current batch may be any batch other than the first batch. For any batch other than the first batch, the page fault request driver 201 may determine the virtual addresses to be restored for any batch based on the virtual addresses corresponding to the memory pages restored in the previous batch of any batch. Specifically, the page fault request driver 201 may start from the next virtual address of the virtual address corresponding to the memory page restored in the previous batch of any batch and select N virtual addresses in sequence as the virtual addresses to be restored for the batch. Wherein, N is a positive integer. N refers to the number of memory pages restored in each batch.

[0078] For any batch, the page fault request driver 201 can asynchronously request the memory management module 40 to allocate target memory pages for the virtual addresses to be recovered. In the embodiment of the present application, the specific implementation method of the memory management module 40 allocating target memory pages for the virtual addresses to be recovered is not limited. Optionally, the memory management module 40 can select target memory pages from free physical memory pages. The number of target memory pages is the same as the number of virtual addresses to be recovered in the current batch. Further, the memory management module 40 can map the target memory page to the virtual address space corresponding to the process space identifier of the pass-through device 102. Specifically, the memory management module 40 can write the correspondence between the physical address of the target memory page and the virtual address to be recovered in the current batch into the page table corresponding to the process space identifier of the pass-through device 102 in the IOMMU 104 to realize page fault recovery of the current batch.

[0079] The missing page request driver 201 may asynchronously request the memory management module 40 to restore the missing memory page according to the above embodiment until the missing memory page is restored. Figure 4 As shown, the page fault request driver 201 can asynchronously return the identifier of the restored memory page to the page fault request service component 102a. In this way, the direct device 102 can asynchronously access the restored memory page according to the identifier of the restored memory page. Specifically, the direct device 102 can asynchronously access the restored memory page through DMA and obtain the stored data from the memory page (corresponding to Figure 5 Steps 7, 11…x4).

[0080] The computing system provided by the embodiments of the present application adds a page fault request service component on the pass-through device side and a page fault request driver on the virtual machine management node. The page fault request service component and the page fault request driver cooperate with each other to achieve asynchronous recovery of large-block IO memory page faults corresponding to large blocks of data. The pass-through device 102 can asynchronously access the recovered memory pages. Compared with the existing synchronous page fault request recovery method, there is no need to wait for all memory pages corresponding to the large block of data to be recovered, which helps to reduce the IO memory page fault recovery time and thus reduce the memory access latency of the pass-through device.

[0081] Moreover, the embodiment of the present application can realize asynchronous recovery of multiple memory pages with one interrupt, which can reduce the number of page fault interrupts and help improve IO performance.

[0082] For an embodiment in which the page missing request driver 201 asynchronously requests the memory management module 40 to recover the missing memory pages in batches, as shown in FIG. Figure 5 As shown, when the memory page corresponding to the virtual address to be restored in the current batch is restored, the page fault request driver 201 asynchronously returns the identifier of the memory page that has been restored in the current batch to the page fault request service component 102a. The pass-through device 102 can asynchronously access the memory page corresponding to the virtual address to be restored in the current batch based on the identifier of the memory page that has been restored in the current batch. Specifically, the pass-through device 102 can asynchronously access the memory page corresponding to the virtual address to be restored in the current batch (corresponding to the DMA address) in the DMA mode. Figure 5 (See steps 7, 11, ..., x4). Optionally, the pass-through device 102 can perform DMA access to the memory pages corresponding to the current batch of virtual addresses to be restored via the DMA controller 50. The DMA controller 50 is a unique peripheral device that transfers data within the system and can be considered a controller that connects internal and external memories to each DMA-capable peripheral device via a set of dedicated buses.

[0083] In the embodiment of the present application, the specific implementation form of the page fault request driver 201 asynchronously returning the identifier of the memory page that has been restored in the current batch to the page fault request service component 102a is not limited. In some embodiments, if the current batch is not the last batch, the page fault request driver 201 can determine that the virtual address to be restored in the current batch is the identifier of the memory page restored in the current batch after the memory page corresponding to the virtual address to be restored in the current batch is successfully restored; and asynchronously return a partial memory page recovery success message (PARTIAL_PAGE_RESPONSE, PPR) to the page fault request component (corresponding to Figure 5 Steps 6, 10, etc.). The partial memory page recovery success message includes: the current batch of virtual addresses to be recovered. Further, the direct device 102 can asynchronously access the memory pages corresponding to the current batch of virtual addresses to be recovered through DMA.

[0084] In some other embodiments, the current batch is the last batch, and the page fault request driver 201 can determine that the virtual address of the last batch to be restored is the identifier of the memory page restored in the current batch after the memory page corresponding to the virtual address to be restored in the last batch is successfully restored; and asynchronously return a message of success of restoring all memory pages (PAGE_RESPONSE_COMPLETION, PRC) to the page fault request service component 102a (corresponding to Figure 5 Step x3, etc.). The message that all memory pages have been successfully restored includes: determining the last batch of virtual addresses to be restored. The message that all memory pages have been successfully restored indicates that the asynchronous large block memory page fault recovery is complete. Further, the direct device 102 can asynchronously access the memory pages corresponding to the last batch of virtual addresses to be restored through DMA.

[0085] For large-block memory page fault recovery, asynchronous recovery of missing memory pages can be achieved. Compared with the existing synchronous page fault request recovery method, there is no need to wait for all memory pages corresponding to large blocks of data to be recovered. Asynchronous access to the memory pages recovered in the current batch can be achieved, which helps to reduce the IO memory page fault recovery time and thus reduce the memory access latency of direct-through devices.

[0086] In addition to the computing system provided in the above embodiment, the present invention also provides a method for handling a memory page fault. The following is an exemplary description of the method for handling a memory page fault provided in the present invention.

[0087] Figure 6 Schematic diagram of the process of processing memory page fault provided by the embodiment of the present application. Figure 6 As shown, the memory page fault processing method includes:

[0088] 601. Obtain a page fault request issued by a page fault request service component in a pass-through device.

[0089] 602. According to the page missing request, the memory management module is asynchronously requested to recover the missing memory page.

[0090] 603. Asynchronously return the identifier of the restored memory page to the page fault request service component, so that the pass-through device can asynchronously access the restored memory page according to the identifier of the restored memory page.

[0091] In this embodiment, to reduce the duration of IO page fault recovery and improve the efficiency of IO memory page fault recovery, a page fault request service component 102a is added to the pass-through device; and a page fault request driver corresponding to the page fault request service component is added to the VMM. This embodiment mainly describes the page fault request processing method provided in this embodiment of the application from the perspective of the page fault request driver. In this embodiment, the description of the page fault request service component and the page fault request driver can be found in the relevant content of the above-mentioned system embodiment and will not be repeated here.

[0092] In step 601 of this embodiment, a page fault request issued by a page fault request service component in the pass-through device may be obtained. The page fault request service component may issue a page fault request to a page fault request driver when a memory page accessed by the pass-through device is missing. This page fault request may be an asynchronous page fault request. For details on how to determine if a memory page accessed by the pass-through device is missing, please refer to the relevant content of the above-mentioned system embodiment and will not be repeated here.

[0093] Next, in step 602, the memory management module 40 may be asynchronously requested to recover the missing memory page according to the page fault request. The memory management module may be a memory management module of the operating system (OS) of the virtual machine, which may manage and allocate memory.

[0094] In some embodiments, based on the page fault request, the memory management module may be asynchronously requested to recover the missing memory pages in batches.

[0095] Specifically, the number of missing memory pages and the process space identifier corresponding to the pass-through device can be obtained from the page fault request; and the following steps are executed cyclically until the number of restored memory pages equals the number of missing memory pages, that is, the missing memory pages are recovered. The steps of the cyclic execution include:

[0096] The current batch of virtual addresses to be restored is determined; the asynchronous memory management module is requested to allocate target memory pages for the virtual addresses to be restored; and the correspondence between the physical addresses of the target memory pages and the current batch of virtual addresses to be restored is written into the page table corresponding to the process space identifier in the IOMMU. The page table corresponding to the process space identifier is the process space identifier of the pass-through device included in the page fault request.

[0097] In some embodiments, the current batch may be the first batch. For the first batch, the starting virtual address of the missing memory page may be obtained from the page fault request; and the virtual address to be restored in the first batch may be determined based on the starting virtual address.

[0098] In other embodiments, the current batch may be any batch other than the first batch. For any batch other than the first batch, the virtual address to be restored of any batch may be determined based on the virtual address corresponding to the memory page restored in the previous batch.

[0099] For any batch, the memory management module can be asynchronously requested to allocate target memory pages for the virtual addresses to be restored. For the specific implementation of the memory management module allocating target memory pages for the virtual addresses to be restored, please refer to the relevant content of the above system embodiment, which will not be repeated here.

[0100] According to the above-mentioned embodiment, the memory management module can be asynchronously requested to restore the missing memory page until the missing memory page is restored. Further, for the restored memory page, in step 603, the identifier of the restored memory page can be asynchronously returned to the missing page request service component. In this way, the direct device can asynchronously access the restored memory page based on the identifier of the restored memory page. Specifically, the direct device can asynchronously access the restored memory page through DMA and obtain the stored data from the memory page.

[0101] In this embodiment, a page fault request service component is added to the passthrough device side, and a page fault request driver is added to the virtual machine management node. The page fault request service component and the page fault request driver work together to achieve asynchronous recovery of large-block IO memory page faults corresponding to large blocks of data, allowing the passthrough device to asynchronously access the recovered memory pages. Compared to existing synchronous page fault request recovery methods, there is no need to wait for all memory pages corresponding to large blocks of data to be fully recovered, which helps to reduce the IO memory page fault recovery time and, in turn, the memory access latency of the passthrough device.

[0102] Moreover, the embodiment of the present application can realize asynchronous recovery of multiple memory pages with one interrupt, which can reduce the number of page fault interrupts and help improve IO performance.

[0103] In an embodiment of asynchronously requesting the memory management module to restore missing memory pages in batches, upon completion of the restoration of the memory pages corresponding to the virtual addresses to be restored in the current batch, the identifiers of the restored memory pages in the current batch may be asynchronously returned to the missing page request service component. Based on the identifiers of the restored memory pages in the current batch, the pass-through device may asynchronously access the memory pages corresponding to the virtual addresses to be restored in the current batch.

[0104] In the embodiments of the present application, the specific implementation form of asynchronously returning the identifiers of the restored memory pages of the current batch to the missing page request service component is not limited. In some embodiments, if the current batch is not the last batch, after the memory page corresponding to the virtual address to be restored in the current batch is successfully restored, the virtual address to be restored in the current batch can be determined to be the identifier of the memory page restored in the current batch; and a partial memory page recovery success message (PARTIAL_PAGE_RESPONSE, PPR) is asynchronously returned to the missing page request component. The partial memory page recovery success message; the partial memory page recovery success message includes: the virtual address to be restored in the current batch. Further, the pass-through device can asynchronously access the memory page corresponding to the virtual address to be restored in the current batch through DMA.

[0105] In other embodiments, the current batch is the last batch, and after the memory page corresponding to the virtual address to be recovered in the last batch is successfully recovered, the virtual address to be recovered in the last batch is determined to be the identifier of the memory page recovered in the current batch; and a message of successful recovery of all memory pages (PAGE_RESPONSE_COMPLETION, PRC) is asynchronously returned to the page fault request service component. The message of successful recovery of all memory pages includes: determining the virtual address to be recovered in the last batch. The message of successful recovery of all memory pages indicates that the asynchronous large-block memory page fault recovery is complete. Further, the pass-through device can asynchronously access the memory page corresponding to the virtual address to be recovered in the last batch through DMA.

[0106] For large-block memory page fault recovery, asynchronous recovery of missing memory pages can be achieved. Compared with the existing synchronous page fault request recovery method, there is no need to wait for all memory pages corresponding to large blocks of data to be recovered. Asynchronous access to the memory pages recovered in the current batch can be achieved, which helps to reduce the IO memory page fault recovery time and thus reduce the memory access latency of direct-through devices.

[0107] It should be noted that the execution entity of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution entity of steps 601 and 602 can be device A; for another example, the execution entity of step 601 can be device A, and the execution entity of step 602 can be device B; and so on.

[0108] In addition, some of the processes described in the above embodiments and the accompanying drawings include multiple operations that appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The sequence numbers of the operations, such as 602, 603, etc., are merely used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel.

[0109] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the above-mentioned memory page fault processing method.

[0110] In the embodiment of the present application, the processing unit can be any hardware processing device that can execute the logic of the above method. Optionally, the processor can be a central processing unit (CPU), a graphics processing unit (GPU) or a microcontroller unit (MCU); it can also be a programmable device such as a field programmable gate array (FPGA), a programmable array logic device (PAL), a general array logic device (GAL), a complex programmable logic device (CPLD); or an advanced reduced instruction set (RISC) processor (Advanced RISC Machines, ARM) or a system on chip (System on Chip, SOC), etc., but is not limited thereto.

[0111] It should be noted that the descriptions such as "first" and "second" in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit "first" and "second" to different types.

[0112] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0113] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0114] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0115] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0116] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0117] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0118] Computer storage media is readable storage media, also known as computer-readable media. Computer-readable storage media include both permanent and non-permanent, removable and non-removable media that can be implemented using any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0119] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0120] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A computing system, characterized in that: include: Host and virtual machine management nodes; The host machine is deployed with a virtual machine, and a pass-through device for the virtual machine is mounted; The operating system of the virtual machine includes a memory management module; The pass-through device includes: a page fault request service component; the page fault request service component is a hardware module; the virtual machine management node includes: a page fault request driver; the page fault request driver is a software function module; The page fault request service component is configured to issue a page fault request to the page fault request driver when a memory page accessed by the pass-through device is missing; The page fault request driver is used to asynchronously request the memory management module to recover the missing multiple memory pages according to the page fault request; and asynchronously return the identifier of the recovered memory page to the page fault request service component; The pass-through device asynchronously accesses the restored memory page according to the identifier of the restored memory page.

2. The system according to claim 1, wherein: When the page missing request driver asynchronously requests the memory management module to recover the missing memory pages, the missing page request driver is specifically configured to: According to the page missing request, the memory management module is asynchronously requested to recover the missing multiple memory pages in batches.

3. The system according to claim 2, characterized in that The host machine further includes an IOMMU; a page table stored in the IOMMU records a correspondence between a virtual address and a physical address; the physical address in the correspondence is a physical address of a memory page of the host machine's memory; the pass-through device accesses the host machine's memory based on the correspondence; When the page missing request driver asynchronously requests the memory management module to recover the missing multiple memory pages in batches, it is specifically used to: Obtaining, from the page fault request, the number of the missing memory pages and the process space identifier corresponding to the pass-through device; and looping through the following steps until the number of restored memory pages equals the number of the missing memory pages; wherein the looping step includes: Determining the virtual addresses to be restored in the current batch; asynchronously requesting the memory management module to allocate target memory pages for the virtual addresses to be restored; The correspondence between the physical address of the target memory page and the virtual addresses to be restored in the current batch is written into the page table corresponding to the process space identifier in the IOMMU.

4. The system according to claim 3, characterized in that The pass-through device initiates a memory access request; and when the page table does not contain a physical address corresponding to the virtual address included in the memory access request, triggers the page fault request service component to issue a page fault request to the page fault request driver.

5. The system according to claim 4, characterized in that The pass-through device queries the IOTLB based on the virtual address included in the memory access request; if the physical address corresponding to the virtual address included in the memory access request is not found in the IOTLB, the pass-through device sends an address translation request to the IOMMU; the address translation request includes the virtual address included in the memory access request; The IOMMU queries the correspondence between the virtual address and the physical address; if the physical address corresponding to the virtual address included in the memory access request is not found in the correspondence, returning an address translation result reflecting the memory page fault to the pass-through device; The pass-through device determines, based on the address translation result, that the page table does not contain a physical address corresponding to the virtual address included in the memory access request.

6. The system according to any one of claims 1 to 5, characterized in that: The page fault request driver runs on the processing unit of the host machine; the processing unit of the host machine is in communication with the page fault request service component; The page fault request service component sends an interrupt signal to the processing unit through a communication link between the processing unit and the processing unit; The processing unit wakes up the page fault request driver in response to the interrupt signal.

7. A memory page fault processing method, characterized in that: The method is applicable to a page fault request driver in a virtual machine management node; The page fault request driver is a software function module; it includes: Obtaining a page fault request issued by a page fault request service component in a pass-through device; the page fault request is issued by the page fault request service component when a memory page accessed by the pass-through device is missing; the page fault request service component is a hardware module; the pass-through device is mounted on a host machine and is a pass-through device of a virtual machine deployed on the host machine; According to the page missing request, asynchronously requesting the memory management module to recover the missing multiple memory pages; The identifier of the restored memory page is asynchronously returned to the page fault request service component, so that the pass-through device can asynchronously access the restored memory page according to the identifier of the restored memory page.

8. The method according to claim 7, characterized in that The asynchronously requesting the memory management module to recover the missing multiple memory pages according to the page missing request includes: According to the page missing request, the memory management module is asynchronously requested to recover the missing multiple memory pages in batches.

9. The method according to claim 8, characterized in that The asynchronously requesting the memory management module to recover the missing multiple memory pages in batches according to the page missing request includes: Obtaining, from the page fault request, the number of the missing memory pages and the process space identifier corresponding to the pass-through device; and looping through the following steps until the number of restored memory pages equals the number of the missing memory pages; wherein the looping step includes: Determining the virtual addresses to be restored in the current batch; asynchronously requesting the memory management module to allocate target memory pages for the virtual addresses to be restored; The correspondence between the physical address of the target memory page and the virtual addresses to be restored in the current batch is written into the page table corresponding to the process space identifier in the IOMMU.

10. The method according to claim 9, characterized in that Determining the virtual addresses to be restored in the current batch includes: For the first batch, obtaining the starting virtual addresses of the missing multiple memory pages from the page fault request; determining the starting virtual addresses as the first batch of virtual addresses to be restored; For any batch except the first batch, the virtual addresses to be restored of the batch are determined according to the virtual addresses corresponding to the memory pages restored in the batch before the batch.

11. The method according to claim 9, characterized in that The asynchronously returning the identifier of the recovered memory page to the page fault request service component includes: If the current batch is not the last batch, after the memory page corresponding to the virtual address to be restored in the current batch is successfully restored, determining that the virtual address to be restored in the current batch is an identifier of the memory page restored in the current batch; Asynchronously returning a partial memory page recovery success message to the page fault request component; the partial memory page recovery success message includes: the virtual addresses of the current batch to be recovered, so that the pass-through device can asynchronously access the memory pages corresponding to the virtual addresses of the current batch to be recovered through direct memory access according to the virtual addresses of the current batch to be recovered.

12. The method according to claim 9, characterized in that The asynchronously returning the identifier of the recovered memory page to the page fault request service component includes: If the current batch is the last batch, after the memory page corresponding to the virtual address to be recovered in the last batch is successfully recovered, the virtual address to be recovered in the last batch is determined to be the identifier of the memory page recovered in the current batch; and a message of successful recovery of all memory pages is asynchronously returned to the missing page request service component; the message of successful recovery of all memory pages includes: determining the virtual address to be recovered in the last batch, so that the pass-through device can asynchronously access the memory page corresponding to the virtual address to be recovered in the last batch through direct memory access according to the virtual address to be recovered in the last batch.

13. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed by one or more processors, the one or more processors are caused to perform the steps of the method according to any one of claims 7 to 12.

Images