A host memory resource processing method, device, equipment and storage medium for data processor virtualization scenario

Abstract

The present disclosure provides a host memory resource processing method, apparatus, device and storage medium for a data processor virtualization scenario, which receives a virtual machine creation request, the request being used to request the creation of a virtual machine that meets the memory configuration requirements of a target virtual machine; starts the target virtual machine according to the memory configuration requirements of the target virtual machine and creates two corresponding table items, including user-available table items and firmware-reserved table items; constructs a memory information table corresponding to the target virtual machine based on the two types of table items, and sends the memory information table to the target virtual machine, the memory information representing the physical memory mapping information of the target virtual machine. It can be seen that when creating table items for the memory information table of the virtual machine, the embodiment of the present disclosure can not only create table items of the user-available type, but also create table items of the firmware-reserved type, and then distribute the system reserved memory indicated by the firmware-reserved table items to each virtual machine, thereby improving the utilization rate of the host memory resources in the data processor virtualization scenario.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular to a host memory resource processing method, apparatus, device, and storage medium for a data processor virtualization scenario. Background Art

[0002] A DPU (Data Processing Unit) is a processor or processing unit used for data processing and accelerated computing tasks. It is typically used in data centers and cloud computing scenarios. It can accelerate data processing tasks through hardware design and optimized algorithms, thereby improving system performance and efficiency.

[0003] In the DPU virtualization scenario, the advantages of DPU hardware acceleration are utilized to offload some virtualization components to the DPU, reducing the resource usage of the physical machine (or host).

[0004] In DPU virtualization scenarios, physical machines are usually required to reserve some physical memory as system reserved memory. These resources cannot be allocated to virtual machines. Therefore, a more efficient memory resource processing method is needed in this scenario to improve resource utilization. Summary of the Invention

[0005] In order to solve the above technical problems, the present disclosure provides a host memory resource processing method, apparatus, device and storage medium in a data processor virtualization scenario, which can improve the utilization rate of host memory resources.

[0006] In a first aspect, the present disclosure provides a method for processing host memory resources in a data processor virtualization scenario, the method comprising:

[0007] Receive a virtual machine creation request, wherein the virtual machine creation request is used to request creation of a virtual machine that meets the memory configuration requirements of the target virtual machine;

[0008] Starting a target virtual machine according to the memory configuration requirement of the target virtual machine and creating two corresponding table entries, the two types of table entries including user available table entries and firmware reserved table entries, the memory address of the user available table entry indicating the available memory of the target virtual machine, the memory address of the firmware reserved table entry indicating the system reserved memory, and the memory size corresponding to the firmware reserved table entry being determined according to a preset first ratio and the memory size specified in the memory configuration requirement;

[0009] A memory information table corresponding to the target virtual machine is constructed based on the two types of table entries, and the memory information table is sent to the target virtual machine, where the memory information table represents physical memory mapping information of the target virtual machine.

[0010] In an optional implementation, the sending the memory information table to the target virtual machine includes:

[0011] Determining a target virtualization firmware that matches the operating system of the target virtual machine according to the operating system characteristics thereof, and if the target virtualization firmware is the target firmware, reading each table entry in the memory information table through the target virtualization firmware, and creating a firmware reserved memory interval for the firmware reserved table entry in the memory information table; wherein the firmware reserved memory interval is used to represent the memory address of the firmware reserved table entry indicating the system reserved memory;

[0012] The memory information table is transparently transmitted to the target virtual machine through the target virtualization firmware.

[0013] In an optional implementation, the method further includes:

[0014] If the target virtualization firmware is the target firmware, determine whether the memory address range corresponding to the user-available table entry in the memory information table belongs to the target address range, and create an available memory interval for the user-available table entry whose memory address range belongs to the target address range; wherein, the available memory interval is used to characterize that the memory address of the user-available table entry indicates the available memory of the target virtual machine.

[0015] In an optional implementation, starting the target virtual machine and creating two corresponding table entries according to the memory configuration requirements of the target virtual machine include:

[0016] Determine a first memory address range according to a target memory address range indicated by the memory configuration requirement of the target virtual machine, and create a user-available table entry based on the first memory address range;

[0017] A second memory address range is determined according to the memory address range indicated by the memory configuration requirement of the target virtual machine, and a firmware reservation table entry is created based on the second memory address range; the memory size of the second memory address range is in a first proportional relationship with the memory size of the target memory address range.

[0018] In an optional implementation, the target virtual machine runs on a physical machine connected to a data processor DPU, and the physical machine relies on the DPU to offload some functions.

[0019] In an optional implementation manner, before receiving the memory configuration requirement of the target virtual machine, the method further includes:

[0020] During the physical machine kernel startup phase, a physical memory of a preset memory size is reserved for the virtual machine; wherein the physical memory of the preset memory size is used for memory management through a target memory management mode.

[0021] In an optional implementation, the method further includes:

[0022] Creating a memory file in a mount directory of a file system corresponding to the target memory management mode, and mapping the memory file to a virtual address space of the target virtual machine;

[0023] If a page fault occurs when accessing the virtual address of the target virtual machine, memory is allocated for the target virtual machine based on the reserved physical memory of the preset memory size, and the allocated memory address is recorded in the memory file to persist the mapping relationship between the memory file and the physical memory of the preset memory size. The mapping relationship is used to access the corresponding physical memory through the memory address in the memory file.

[0024] In a second aspect, the present disclosure provides a host memory resource processing device for a data processor virtualization scenario, the device comprising:

[0025] A receiving module, configured to receive a virtual machine creation request, wherein the virtual machine creation request is used to request creation of a virtual machine that meets the memory configuration requirements of a target virtual machine;

[0026] a creation module, configured to start a target virtual machine according to the memory configuration requirement of the target virtual machine and create two corresponding table entries, the two types of table entries including user-available table entries and firmware-reserved table entries, the memory addresses of the user-available table entries indicating the available memory of the target virtual machine, the memory addresses of the firmware-reserved table entries indicating the system-reserved memory, and the memory size corresponding to the firmware-reserved table entries being determined according to a preset first ratio and the memory size specified in the memory configuration requirement;

[0027] The sending module is configured to construct a memory information table corresponding to the target virtual machine based on the two types of table entries, and send the memory information table to the target virtual machine, where the memory information table represents physical memory mapping information of the target virtual machine.

[0028] In a third aspect, the present disclosure provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method when executing the computer program.

[0029] In a fourth aspect, the present disclosure provides a computer program product, which includes a computer program / instructions, and the computer program / instructions implement the above method when executed by a processor.

[0030] The technical solution provided by the embodiments of the present disclosure has at least the following advantages compared with the prior art:

[0031] In the host memory resource processing method for a data processor virtualization scenario provided by an embodiment of the present disclosure, a virtual machine creation request is first received, where the request is used to request the creation of a virtual machine that meets the memory configuration requirements of a target virtual machine. Then, the target virtual machine is started according to the memory configuration requirements of the target virtual machine and two corresponding types of table items are created, wherein the two types of table items include user-available table items and firmware-reserved table items, the memory address of the user-available table items indicates the available memory of the target virtual machine, the memory address of the firmware-reserved table items indicates the system-reserved memory, and the memory size corresponding to the firmware-reserved table items is determined according to a preset first ratio and the memory size specified in the memory configuration requirements; then, a memory information table corresponding to the target virtual machine is constructed based on the two types of table items, and the memory information table is sent to the target virtual machine. The memory information table is used to represent the physical memory mapping information of the target virtual machine.

[0032] It can be seen that when creating table entries for the memory information table of a virtual machine, the embodiment of the present disclosure can not only create table entries of user-available type, but also create table entries of firmware reserved type, and then distribute the system reserved memory indicated by the firmware reserved table entries to each virtual machine, thereby improving the utilization rate of the host memory resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0034] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0035] Figure 1 A flowchart of a memory resource processing method provided in an embodiment of the present disclosure;

[0036] Figure 2 A schematic diagram of a virtual machine simulator, virtualization firmware, and target virtual machine provided in an embodiment of the present disclosure;

[0037] Figure 3 A schematic diagram of a user-available table entry and a firmware reserved table entry provided in an embodiment of the present disclosure;

[0038] Figure 4 A schematic diagram of the structure of a memory resource processing device provided in an embodiment of the present disclosure;

[0039] Figure 5 A schematic diagram of the structure of a memory resource processing device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0040] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0041] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0042] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0043] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0044] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0045] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0046] Some technical concepts or noun concepts involved in this article are explained:

[0047] A physical machine refers to a physical computer, including physical components such as processors, memory, and storage devices. A physical machine is used to provide a hardware environment for virtual machines, so a physical machine is sometimes also called a "host" or "host."

[0048] A virtual machine refers to a logical computer system created on a physical machine through virtualization technology. Multiple virtual machines can be created on a physical machine, and each virtual machine can run an independent operating system and application. A virtual machine is usually called a guest machine, and the physical computer where the virtual machine runs is called a host.

[0049] Virtualization components are the software and hardware components necessary to support the operation and management of a virtualized environment. They are used to manage and control various functions in a virtualized environment to ensure its stability and security.

[0050] A Data Processing Unit (DPU) is a processor or processing unit used for data processing and accelerated computing tasks, typically used in data centers and cloud computing. In DPU virtualization scenarios, the DPU's hardware acceleration can be leveraged to offload some virtualization components to the DPU, reducing resource usage on the physical machine (or host).

[0051] In DPU virtualization scenarios, physical machines are usually required to reserve some physical memory as system reserved memory. These resources cannot be allocated to virtual machines. Therefore, a more efficient memory resource processing method is needed in this scenario to improve resource utilization.

[0052] To this end, an embodiment of the present disclosure provides a host memory resource processing method for a data processor virtualization scenario. Specifically, a virtual machine creation request is first received, where the request is used to request the creation of a virtual machine that meets the memory configuration requirements of the target virtual machine. Then, the target virtual machine is started according to the memory configuration requirements of the target virtual machine and two corresponding types of table items are created. The two types of table items include user-available table items and firmware reserved table items. The memory address of the user-available table item indicates the available memory of the target virtual machine, and the memory address of the firmware reserved table item indicates the system reserved memory. The memory size corresponding to the firmware reserved table item is determined according to a preset first ratio and the memory size specified in the memory configuration requirements. Then, a memory information table corresponding to the target virtual machine is constructed based on the two types of table items, and the memory information table is sent to the target virtual machine. The memory information table is used to characterize the physical memory mapping information of the target virtual machine.

[0053] It can be seen that when creating table entries for the memory information table of a virtual machine, the embodiment of the present disclosure can not only create table entries of user-available type, but also create table entries of firmware reserved type, and then distribute the system reserved memory indicated by the firmware reserved table entries to each virtual machine, thereby improving the utilization rate of the host memory resources.

[0054] Based on this, the embodiment of the present disclosure provides a host memory resource processing method for a data processor virtualization scenario, referring to Figure 1, is a flowchart of a host memory resource processing method in a data processor virtualization scenario provided by an embodiment of the present disclosure, the method comprising:

[0055] S101: Receive a virtual machine creation request, where the virtual machine creation request is used to request creation of a virtual machine that meets memory configuration requirements of a target virtual machine.

[0056] The host memory resource processing method in the DPU virtualization scenario provided by the embodiments of the present disclosure can be applied to any physical machine in a DPU virtualization cluster or a DPU virtualization system.

[0057] In practice, a virtualization cluster or system can include one or more cloud controllers, each of which can be connected to multiple compute nodes. The cloud controller is primarily responsible for managing cloud resource usage, monitoring cloud environment performance, and ensuring cloud security, while the compute nodes are primarily responsible for executing computing tasks and running applications. In practice, a physical machine connected to a data processing unit (DPU) can be considered a compute node.

[0058] In an embodiment of the present disclosure, a virtual machine simulator may be run on a physical machine connected to a DPU. The virtual machine simulator is used to receive a virtual machine creation request, which is used to request the creation of a virtual machine that meets the memory configuration requirements of a target virtual machine.

[0059] In an optional implementation, the target virtual machine can run on a physical machine connected to a data processor DPU. The physical machine relies on the DPU to offload some functions. For example, some virtualization components on the physical machine can be offloaded to the DPU, and then the various functions in the virtualized environment can be managed and controlled based on the DPU to ensure the stability and security of the virtualized environment.

[0060] In actual applications, when cloud control receives a memory application request for a target virtual machine sent by a client, it first determines the memory configuration requirements of the target virtual machine based on the system's preset memory resource allocation method, and then sends a virtual machine creation request to the physical machine connected to the DPU based on the memory configuration requirements of the target virtual machine, requesting the physical machine connected to the DPU to create a virtual machine that meets the memory configuration requirements of the target virtual machine.

[0061] In one optional embodiment, the memory configuration requirement may include information about at least one NUMA (Non-Uniform Memory Access) node allocated to the target virtual machine, as well as a memory address range corresponding to each NUMA node. Information such as the name of the NUMA node, the memory size of the NUMA node, and the memory address range corresponding to the NUMA node can be understood as the memory configuration requirement of the target virtual machine in the disclosed embodiments.

[0062] For example, when cloud control receives a memory request for a 2G target virtual machine, it first determines the memory configuration requirements of the target virtual machine based on the system's preset memory resource allocation method. For example, it allocates two 1G numa nodes to the 2G target virtual machine, such as numa node 1 and numa node 2, where the memory address range of numa node 1 is A1-A2 and the memory size is 1G, and the memory address range of numa node 2 is B1-B2 and the memory size is 1G. After determining the memory configuration requirements of the target virtual machine, cloud control sends the memory configuration requirements of the target virtual machine to the computing node connected to cloud control, that is, the physical machine connected to the DPU, in the form of a virtual machine creation request.

[0063] S102: Start the target virtual machine according to the memory configuration requirement of the target virtual machine and create two types of corresponding table entries.

[0064] Among them, the two types of table entries include user-available table entries and firmware reserved table entries, the memory address of the user-available table entry indicates the available memory of the target virtual machine, the memory address of the firmware reserved table entry indicates the system reserved memory, and the memory size corresponding to the firmware reserved table entry is determined according to the preset first ratio and the memory size specified in the memory configuration requirement.

[0065] In the embodiment of the present disclosure, after receiving a virtual machine creation request, the physical machine connected to the DPU calls the virtual machine simulator to start the target virtual machine according to the memory configuration requirements of the target virtual machine and creates two corresponding table entries.

[0066] In an optional embodiment, the virtual machine simulator can create two types of table entries for the target virtual machine based on the memory address range of the NUMA node in the memory configuration requirements of the target virtual machine. The memory address ranges corresponding to the two types of table entries respectively belong to the memory address range of the NUMA node.

[0067] In the embodiment of the present disclosure, the two types of table entries include user-available table entries and firmware reserved table entries, wherein the memory address of the user-available table entry is used to indicate the available memory of the target virtual machine, that is, the address range of the available memory in the target virtual machine can be determined based on the memory address corresponding to the user-available table entry Ram, and the memory address of the firmware reserved table entry is used to indicate the system reserved memory, that is, the address range of the system reserved memory in the target virtual machine can be determined based on the memory address corresponding to the firmware reserved table entry Reserved.

[0068] For example, assuming that the target virtual machine's memory configuration requirements are used to create two types of table entries for numa node 1 and numa node 2, where the memory address range corresponding to numa node 1 is A1-A2 and the memory size is 1G, and the memory address range corresponding to numa node 2 is B1-B2 and the memory size is 1G, after the virtual machine simulator receives the above memory configuration requirements, it creates two types of table entries based on the memory address range of numa node 1, such as Entry0 (table entry 0) and Entry1 (table entry 1), and creates two types of table entries based on the memory address range of numa node 2, such as Entry2 (table entry 2) and Entry3 (table entry 3). Among them, Entry0 created based on numa node 1 is a user-available table entry, Entry1 is a firmware reserved table entry, Entry2 created based on numa node 2 is a user-available table entry, and Entry3 is a firmware reserved table entry.

[0069] In an embodiment of the present disclosure, the memory size corresponding to the firmware reservation table entry is determined according to a preset first ratio and the memory size specified in the memory configuration requirement, wherein the preset first ratio can be determined based on the memory resource size of the target virtual machine. For example, for a target virtual machine with a memory size of 64G, the corresponding preset first ratio can be 1 / 64. Accordingly, assuming that the memory size of the numa node specified in the memory configuration requirement is 2G, the memory size corresponding to the firmware reservation table entry is (1 / 64)*2G.

[0070] In an optional implementation, the memory address of the user-available table entry and the memory address of the firmware reserved table entry together constitute the memory address of the NUMA node specified in the memory configuration requirement.

[0071] In actual applications, when creating user-available table entries and firmware-reserved table entries according to the internal configuration requirements of the target virtual machine, the sum of the memory addresses of the created user-available table entries and the memory addresses of the firmware-reserved table entries is equal to the memory size of the NUMA node, that is, the memory addresses of the user-available table entries and the memory addresses of the firmware-reserved table entries together constitute the memory address of the NUMA node.

[0072] S103: Construct a memory information table corresponding to the target virtual machine based on the two types of table entries, and send the memory information table to the target virtual machine.

[0073] The memory information table is used to represent the physical memory mapping information of the target virtual machine.

[0074] In the embodiment of the present disclosure, after the virtual machine simulator creates two types of corresponding table entries according to the memory configuration requirements of the target virtual machine, it first constructs a memory information table corresponding to the target virtual machine based on the two types of table entries.

[0075] In the embodiment of the present disclosure, the memory information table is used to represent the physical memory mapping information of the target virtual machine, wherein the physical memory mapping information includes information such as the memory address range and memory attributes corresponding to each table entry. For example, it is assumed that the memory information table includes table entry Entry0 and table entry Entry1, wherein the memory address range corresponding to table entry Entry0 is E1-E2, and the memory attribute is available memory, indicating that the memory address segment E1-E2 is the available memory of the target virtual machine, and the memory address range corresponding to table entry Entry1 is F1-F2, and the memory attribute is system reserved memory, indicating that the memory address segment F1-F2 is the system reserved memory of the target virtual machine.

[0076] In an embodiment of the present disclosure, after the virtual machine simulator constructs a memory information table corresponding to the target virtual machine based on two types of table entries, the memory information table is sent to the target virtual machine so that the target virtual machine can understand the layout and availability of the memory in the system by reading the physical memory mapping information recorded in the memory information table, and use and manage memory resources based on the physical memory mapping information.

[0077] In the memory resource processing method provided by the embodiment of the present disclosure, a virtual machine creation request is first received, where the request is used to request the creation of a virtual machine that meets the memory configuration requirements of the target virtual machine. Then, the target virtual machine is started according to the memory configuration requirements of the target virtual machine and two corresponding types of table items are created, wherein the two types of table items include user-available table items and firmware-reserved table items. The memory address of the user-available table item indicates the available memory of the target virtual machine, the memory address of the firmware-reserved table item indicates the system-reserved memory, and the memory size corresponding to the firmware-reserved table item is determined according to a preset first ratio and the memory size specified in the memory configuration requirements. Then, a memory information table corresponding to the target virtual machine is constructed based on the two types of table items, and the memory information table is sent to the target virtual machine. The memory information table is used to characterize the physical memory mapping information of the target virtual machine.

[0078] It can be seen that when creating table entries for the memory information table of a virtual machine, the embodiment of the present disclosure can not only create table entries of user-available type, but also create table entries of firmware reserved type, and then distribute the system reserved memory indicated by the firmware reserved table entries to each virtual machine, thereby improving the utilization rate of the host memory resources.

[0079] In actual applications, virtualization firmware, such as SeaBIOS firmware and EDK2 / OVMF firmware, can also run on the physical machine connected to the DPU. Among them, SeaBIOS firmware is a firmware implemented based on the open source BIOS (Basic Input / Output System), which is usually used in virtualization platforms and embedded systems of the x86 platform, such as QEMU (Quick Emulator, virtual machine simulator), or kernel-based virtual machine KVM. EDK2 / OVMF firmware is a UFFI firmware for virtual machines. In a virtualization environment, OVFM can be used in conjunction with a virtual machine simulator (such as QEMU) to provide virtual machines with the function of starting and initializing firmware. The physical machine connected to the DPU in the embodiment of the present disclosure can support virtualization firmware SeaBIOS and EDK2 / OVMF at the same time.

[0080] Therefore, after constructing the memory information table corresponding to the target virtual machine based on the two types of table entries, the embodiment of the present disclosure can also send the memory information table to the virtualization firmware corresponding to the target virtual machine, and call the virtualization firmware to send the memory information table to the target virtual machine.

[0081] In an embodiment of the present disclosure, the virtualization firmware corresponding to the target virtual machine can be determined from multiple virtualization firmwares based on the operating system characteristics of the target virtual machine. Specifically, the target virtualization firmware that matches the target virtual machine is determined according to the operating system characteristics of the target virtual machine. For example, the virtualization firmware corresponding to the target virtual machine can be determined based on the image file of the operating system of the target virtual machine, such as SeaBIOS firmware or EDK2 / OVMF firmware.

[0082] like Figure 2 1 is a schematic diagram of a virtual machine simulator, virtualization firmware and target virtual machine provided by an embodiment of the present disclosure, specifically taking the 440fx platform where the virtual machine simulator QEMU is located as an example.

[0083] After cloud control determines the memory configuration requirements of the target virtual machine based on the system's preset memory resource allocation method, it sends a virtual machine creation request to the target physical machine connected to cloud control (i.e., the physical machine connected to the DPU). The virtual machine creation request is used to request the creation of a virtual machine that meets the memory configuration requirements of the target virtual machine.

[0084] After receiving the virtual machine creation request, the physical machine connected to the DPU calls the virtual machine emulator QEMU to start the target virtual machine according to the memory configuration requirements of the target virtual machine, create user available table entries and firmware reserved table entries, and build the E820 memory information table corresponding to the target virtual machine based on the user available table entries and firmware reserved table entries. The E820 memory information table is then sent to the target virtual machine through the virtualization firmware SeaBIOS or EDK2 / OVMF.

[0085] Among them, the E820 table includes table entries Entry0, Entry1, Entry2, Entry3...EntryN. Each table entry records the memory address and memory attributes. Specifically, Entry0 and Entry1 are created based on the memory address of NUMA node 1. The memory address range corresponding to Entry0 is E1-E2, and the memory attribute is available memory. The memory address range corresponding to Entry1 is F1-F2, and the memory attribute is system reserved memory. Entry2 and Entry3 are created based on the memory address range of NUMA node 2.

[0086] When it is determined that the target virtualization firmware matching the operating system characteristics of the target virtual machine is SeaBIOS, the virtual machine simulator QEMU first sends the memory information table to SeaBIOS, and then SeaBIOS directly sends the memory information table to the target virtual machine.

[0087] In actual applications, since SeaBIOS can directly read and process the E820 table reported by QEMU, when it is determined that the virtual machine firmware is SeaBIOS, SeaBIOS can be directly called to read and process the entries in the E820 table.

[0088] In an optional implementation, if the target virtualization firmware is the target firmware, each table entry in the memory information table is read through the target virtualization firmware, and a firmware reserved memory interval is created for the firmware reserved table entry in the memory information table; then, the memory information table is transparently transmitted to the target virtual machine through the target virtualization firmware.

[0089] The target firmware may include EDK2 / OVMF firmware. Since the embodiment of the present disclosure modifies the method of creating table entries, the modified method of creating table entries is not perceptible to EDK2 / OVMF. Therefore, when it is determined that the target virtualization firmware is EDK2 / OVMF, it is necessary to call EDK2 / OVMF to further process the E820 table.

[0090] like Figure 2As shown, when it is determined that the target virtualization firmware that matches the operating system characteristics of the target virtual machine is EDK2 / OVMF, the virtual machine simulator QEMU first sends the memory information table to EDK2 / OVMF, and then EDK2 / OVMF reads each table item in the memory information table and creates a firmware reserved memory interval for the firmware reserved table item in the memory information table. Furthermore, the memory information table is transparently transmitted to the target virtual machine through the EDK2 / OVMF firmware, so that the target virtual machine can access and manage memory resources by reading the physical memory mapping information recorded in the memory information table.

[0091] In the disclosed embodiments, a firmware reserved memory interval can be used to represent the memory address of a firmware reserved entry indicating system reserved memory. System reserved memory refers to the memory space required to maintain core system functions, drive devices, and store virtualized firmware. This memory space will not be released or allocated to user processes during system operation. After a firmware reserved memory interval is created for a firmware reserved entry in the memory information table, the memory address range corresponding to the firmware reserved entry will not be allocated to other devices or user processes.

[0092] In an optional implementation, if the target virtualization firmware is the target firmware, each table entry in the memory information table is read through the target virtualization firmware, and it is determined whether the memory address range corresponding to the user-available table entry in the memory information table belongs to the target address range, and an available memory interval is created for the user-available table entry whose memory address range belongs to the target address range.

[0093] The available memory area represents the available memory of the target virtual machine indicated by the memory address of the user-available table entry. Available memory refers to the memory space in the system that is not reserved by the system or occupied by other system components and can be used for tasks such as running applications and storing data. After creating an available memory interval for the user-available table entry whose memory address range falls within the target address range, user processes can allocate and use this memory space through the operating system memory management module in the target virtual machine.

[0094] In actual applications, since the target firmware EDK2 / OVMF can directly read user-available table entries above 4G, but cannot read user-available table entries below 4G, it is necessary to create an available memory range for user-available table entries below 4G, that is, the target address range can include the memory address range of 0-4G.

[0095] like Figure 2As shown, after the target firmware EDK2 / OVMF receives the E820 table sent from the virtual machine simulator, it first reads each table entry in the E820 table. When reading the user-available table entry, it first determines whether the memory address range corresponding to the user-available table entry is below 4G. If it is determined that the memory address range corresponding to the user-available table entry is below 4G, an available memory range is created for the user-available table entry.

[0096] In an optional implementation, user-available entries and firmware-reserved entries may also be created according to a memory address range indicated by the memory configuration requirement of the target virtual machine.

[0097] Specifically, a first memory address range is determined according to a target memory address range indicated by the memory configuration requirement of the target virtual machine, and a user-available table entry is created based on the first memory address range; and a second memory address range is determined according to the target memory address range indicated by the memory configuration requirement of the target virtual machine, and a firmware reserved table entry is created based on the second memory address range.

[0098] The target memory address range indicated by the memory configuration requirement of the target virtual machine may include the memory address range of the NUMA node allocated to the target virtual machine.

[0099] The memory size of the second memory address range is in a first proportional relationship with the target memory address range indicated by the memory configuration requirement of the target virtual machine. For example, the sum of the memory size corresponding to the first memory address range and the memory size corresponding to the second memory address range can be equal to the target memory address range indicated by the memory configuration requirement of the target virtual machine, that is, the memory size of the NUMA node allocated to the target virtual machine.

[0100] like Figure 3 The figure shows a schematic diagram of user-available table entries and firmware reserved table entries provided by an embodiment of the present disclosure. Specifically, taking the 440fx platform as an example, the E820 table is created using the following rules. The allocatable memory ranges in the 440fx platform include 0-H and above F, for example, 0-3G and above 4G. The memory configuration requirements of the target virtual machine include n NUMA nodes, where the memory address range of NUMA node n is A1-A2, the memory size is m, and the first proportional relationship is t.

[0101] If it is determined that the memory address range of the numa node n is above 0-H or F (that is, A1-A2 is less than H or A1-A2 is greater than F), the first memory address range is determined to be A1 to A1+m*t, and a user-available table entry Ram is created based on the first memory address range, and the second memory address range is determined to be A1+m*t to A2, and a firmware reserved table entry Reserved is created based on the second memory address range, that is, an available table entry and a firmware reserved table entry are created.

[0102] If it is determined that the memory address range of NUMA node n includes both 0-H and F and above, the following two cases are included:

[0103] In the first case, if A1+m*t is within 0-H (that is, the firmware reserved table entry spans HF), the first memory address range is determined to be A1 to A1+m*t, and the user-available table entry Ram is created based on the first memory address range, and the second memory address range is determined to be A1+m*t to H, F to A2, and the firmware reserved table entry Reserved1 and the firmware reserved table entry Reserved2 are created based on the second memory address range, that is, one available table entry and two firmware reserved table entries are created.

[0104] In the second case, if A1+m*t is above H (that is, the available table entry spans HF), the first memory address range is determined to be A1 to H, F to A2-m*(1 / t), and user available table entries Ram1 and user available table entries Ram2 are created based on the first memory address range, and the second memory address range is determined to be A2-m*(1 / t) to A2, and a firmware reserved table entry Reserved is created based on the second memory address range, that is, two available table entries and one firmware reserved table entry are created.

[0105] In an optional implementation, since the target metadata is used for memory management of the target virtual machine, it cannot be offloaded to the DPU, thus affecting physical memory resource utilization. Therefore, the disclosed embodiment can also optimize the physical memory resources required for managing virtual machine memory by deleting the target metadata during the target physical machine startup process.

[0106] In traditional memory management schemes, the physical machine kernel must maintain a large page structure to track the usage of each memory page in the virtual machine. These data structures are typically stored in the physical machine's memory space, occupying a certain amount of physical machine memory resources (i.e., the additional memory resources required for virtual machine memory management), which cannot be allocated to the virtual machine. Therefore, to improve the utilization of physical machine memory resources, target memory management can be used to further optimize the memory resources on the physical machine before receiving a virtual machine creation request.

[0107] Specifically, during the kernel startup phase of the physical machine, a preset memory size of physical memory is reserved for the virtual machine. Since this reserved physical memory is used for memory management through the target memory management method and will not be taken over by the memory management mechanism of the physical machine (that is, the traditional memory management solution will not be used for memory management), there is no need to reserve an additional portion of memory resources in the memory space of the physical machine for memory management. In this way, the memory resources originally used to store and manage virtual machines can be released from the memory resources of the physical machine and used for other purposes, such as allocation to virtual machines, thereby further improving the utilization rate of the physical machine's memory resources.

[0108] The target memory management method may include a memory management method such as DMEM (Direct Memory FS). In DMEM technology, memory resources on each NUMA node are generally managed through the dmem_node (also known as a dynamic memory node, or DMEM node) data structure. The DMEM node may include information such as the node identifier, memory address range, and memory usage of the corresponding NUMA node.

[0109] Each numa node contains a certain number of dmem_regions (also known as dynamic memory regions, or DMEM regions). DMEM regions can be used to manage memory regions with continuous addresses on numa nodes. Each DMEM region on a numa node can contain multiple dmem pages (also known as dynamic memory pages, or DMEM memory pages). A bitmap can usually be used to identify whether each DMEM memory page in the memory region is allocated. The DMEM region can include information such as the start and end addresses of the memory region with continuous addresses and the memory size. When DMEM supports 2MB and 1GB large pages, the metadata of each DMEM memory page only contains the reference count of the memory page.

[0110] In an optional implementation, during the physical machine kernel startup phase, DMEM memory management technology can reserve physical memory of a preset memory size for the virtual machine on each NUMA node through command line parameters, and establish a dmem_node node / dmem_region memory area / dmem page memory page data structure for the physical memory to utilize DMEM memory management technology for memory management.

[0111] Among them, the dmem_node node is used to manage the affinity of memory under the NUMA architecture, which can ensure that memory allocation is as close as possible to the processor that is executing the task, thereby reducing memory access latency; dmem_region is used to represent the continuous memory area on the specified NUMA node.

[0112] In actual applications, after the physical machine kernel is started, DMEM memory management technology can also support mounting the target as a file system and creating memory files under the mount directory.

[0113] Specifically, a memory file is created in the mount directory of the file system corresponding to the target memory management mode, and the memory file is mapped to the virtual address space of the target virtual machine.

[0114] In the embodiment of the present disclosure, the virtual machine simulator is first called to create a memory file under the mount directory of the DMEM file system during the startup initialization phase of the target virtual machine, and the memory file is mapped to the virtual address space of the target virtual machine to perform memory pre-allocation by actively accessing the virtual address.

[0115] In actual applications, during the process of memory pre-allocation by actively accessing a virtual address, if the memory page corresponding to the virtual address is not in the physical memory, a page fault exception will be triggered, and this exception needs to be handled.

[0116] In the disclosed embodiment, if a page fault occurs when accessing a virtual address of a target virtual machine, memory is allocated for the target virtual machine based on a reserved physical memory size of a preset memory size, and the allocated memory address is recorded in a memory file to persist a mapping relationship between the memory file and the physical memory size of the preset memory size. The mapping relationship is used to access the corresponding physical memory through the memory address in the memory file.

[0117] In actual applications, when a page fault occurs when accessing a virtual address, the page exception callback mechanism in DMEM will obtain an idle memory area in the DMEM node that meets the numa affinity according to the mapped numa affinity as the target DMEM area. After determining the target DMEM area, the allocation status of each DMEM memory page in the target DMEM area is determined according to the bitmap bitmap, and the DMEM memory page in the target DMEM area that is in an unallocated state is allocated to the target virtual machine, and the allocation status of the DMEM memory page is modified to an allocated state. Furthermore, the memory address of the DMEM memory page allocated to the target virtual machine is recorded in the memory file to persist the mapping relationship between the memory file and the physical memory of the reserved preset memory size.

[0118] For example, in a 64-bit operating system, the metadata data structure introduced by the physical machine kernel to track the status of each memory page requires approximately 64 bytes. In traditional memory resource management solutions, when managing a virtual machine with 320GB of memory, it is necessary to reserve approximately 5GB of physical memory on the physical machine when managing the metadata of the virtual machine. However, the disclosed embodiment reserves a preset memory size of physical memory for the virtual machine during the physical machine startup phase, for example, 320GB of memory, and manages the memory using the target memory management method. Since the target memory management method does not need to maintain metadata for the physical memory, it can save approximately 5GB of physical memory for subsequent allocation to the virtual machine.

[0119] It can be seen that the embodiment of the present disclosure can save the physical memory required for managing the virtual machine and improve the utilization rate of physical memory resources by reserving a preset memory size of physical memory for the virtual machine and managing the physical memory through the target memory management method.

[0120] Based on the same inventive concept as the above embodiment, the present disclosure also provides a host memory resource processing device for a data processor virtualization scenario, referring to Figure 4 , is a structural diagram of a host memory resource processing device for a data processor virtualization scenario provided by an embodiment of the present disclosure, the device comprising:

[0121] Receiving module 401, configured to receive a virtual machine creation request, wherein the virtual machine creation request is used to request creation of a virtual machine that meets the memory configuration requirements of a target virtual machine;

[0122] a creation module 402, configured to start a target virtual machine according to the memory configuration requirement of the target virtual machine and create two corresponding entries, the two entries including user-available entries and firmware-reserved entries, the memory addresses of the user-available entries indicating the available memory of the target virtual machine, the memory addresses of the firmware-reserved entries indicating the system-reserved memory, and the memory size corresponding to the firmware-reserved entries being determined according to a preset first ratio and the memory size specified in the memory configuration requirement;

[0123] The sending module 403 is configured to construct a memory information table corresponding to the target virtual machine based on the two types of table entries, and send the memory information table to the target virtual machine, where the memory information table represents physical memory mapping information of the target virtual machine.

[0124] In an optional implementation, the sending module includes:

[0125] A first creation submodule is configured to determine a target virtualization firmware that matches the target virtual machine's operating system characteristics, and if the target virtualization firmware is the target firmware, read each table entry in the memory information table through the target virtualization firmware, and create a firmware reserved memory interval for the firmware reserved table entry in the memory information table; wherein the firmware reserved memory interval is used to represent a memory address of the firmware reserved table entry indicating system reserved memory;

[0126] The transparent transmission submodule is used to transparently transmit the memory information table to the target virtual machine through the target virtualization firmware.

[0127] In an optional embodiment, the device further includes:

[0128] A determination module is configured to determine, if the target virtualization firmware is the target firmware, whether the memory address range corresponding to the user-available table entry in the memory information table belongs to the target address range, and to create an available memory interval for the user-available table entry whose memory address range belongs to the target address range; wherein the available memory interval is used to characterize that the memory address of the user-available table entry indicates the available memory of the target virtual machine.

[0129] In an optional implementation, the creation module includes:

[0130] A second creation submodule is configured to determine a first memory address range according to a target memory address range indicated by the memory configuration requirement of the target virtual machine, and to create a user-available table entry based on the first memory address range;

[0131] The third creation sub-module is used to determine a second memory address range according to the memory address range indicated by the memory configuration requirement of the target virtual machine, and create a firmware reservation table entry based on the second memory address range; the memory size of the second memory address range is in a first proportional relationship with the memory size of the target memory address range.

[0132] In an optional implementation, the target virtual machine runs on a physical machine connected to a data processor DPU, and the physical machine relies on the DPU to offload some functions.

[0133] In an optional embodiment, the device further includes:

[0134] The reservation module is used to reserve physical memory of a preset memory size for the virtual machine during the physical machine kernel startup phase; wherein the physical memory of the preset memory size is used for memory management through a target memory management mode.

[0135] In an optional embodiment, the device further includes:

[0136] A mapping module, configured to create a memory file under a mount directory of a file system corresponding to the target memory management mode, and map the memory file to a virtual address space of the target virtual machine;

[0137] A persistence module is used to allocate memory for the target virtual machine based on the reserved physical memory of the preset memory size if a page fault occurs in the virtual address access to the target virtual machine, and record the allocated memory address in the memory file to persist the mapping relationship between the memory file and the physical memory of the preset memory size, and the mapping relationship is used to access the corresponding physical memory through the memory address in the memory file.

[0138] In the host memory resource processing device of the data processor virtualization scenario provided by the embodiment of the present disclosure, a virtual machine creation request is first received, where the request is used to request the creation of a virtual machine that meets the memory configuration requirements of the target virtual machine. Then, the target virtual machine is started according to the memory configuration requirements of the target virtual machine and two corresponding types of table items are created, wherein the two types of table items include user-available table items and firmware-reserved table items, the memory address of the user-available table items indicates the available memory of the target virtual machine, the memory address of the firmware-reserved table items indicates the system-reserved memory, and the memory size corresponding to the firmware-reserved table items is determined according to a preset first ratio and the memory size specified in the memory configuration requirements; then, a memory information table corresponding to the target virtual machine is constructed based on the two types of table items, and the memory information table is sent to the target virtual machine. The memory information table is used to represent the physical memory mapping information of the target virtual machine.

[0139] It can be seen that when creating table entries for the memory information table of a virtual machine, the embodiment of the present disclosure can not only create table entries of user-available type, but also create table entries of firmware reserved type, and then distribute the system reserved memory indicated by the firmware reserved table entries to each virtual machine, thereby improving the utilization rate of the host memory resources.

[0140] The host memory resource processing in the data processor virtualization scenario provided by the embodiments of the present disclosure can execute the memory resource processing method provided by any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0141] In order to implement the above embodiments, the present disclosure further proposes a computer program product, including a computer program / instruction, which implements the host memory resource processing method in the above embodiments when executed by a processor.

[0142] In addition, the present disclosure also provides a host memory resource processing device for a data processor virtualization scenario, see Figure 5 As shown, this may include:

[0143] Processor 501, memory 502, input device 503 and output device 504. The number of processors 501 in the host memory resource processing device can be one or more. Figure 5 In some embodiments of the present disclosure, the processor 501, the memory 502, the input device 503 and the output device 504 may be connected via a bus or other means, wherein: Figure 5 The bus connection is taken as an example.

[0144] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing of the host memory resource processing device by running the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, at least one application required for a function, etc. In addition, the memory 502 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. The input device 503 may be used to receive input digital or character information, and to generate signal input related to user settings and function control of the host memory resource processing device.

[0145] Specifically in this embodiment, the processor 501 will load the executable files corresponding to the processes of one or more applications into the memory 502 according to the following instructions, and the processor 501 will run the applications stored in the memory 502, thereby realizing the various functions of the above-mentioned host memory resource processing device.

[0146] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0147] The foregoing description is intended only to provide specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments described herein, but rather to be construed in the broadest manner consistent with the principles and novel features disclosed herein.

Claims

1. A method for processing host memory resources in a data processor virtualization scenario, characterized in that: The method comprises: Receive a virtual machine creation request, wherein the virtual machine creation request is used to request creation of a virtual machine that meets the memory configuration requirements of the target virtual machine; Starting a target virtual machine according to the memory configuration requirements of the target virtual machine and creating two corresponding table entries, the two types of table entries including user-available table entries and firmware-reserved table entries, the memory address in the user-available table entry indicating the available memory of the target virtual machine, the memory address in the firmware-reserved table entry indicating the system-reserved memory, the memory size corresponding to the memory address in the firmware-reserved table entry being determined according to a preset first ratio and the memory size specified in the memory configuration requirements, the system-reserved memory referring to the memory space required for maintaining system functions, driving devices, and storing virtualized firmware; A memory information table corresponding to the target virtual machine is constructed based on the two types of table entries, and the memory information table is sent to the target virtual machine, where the memory information table represents physical memory mapping information of the target virtual machine.

2. The method according to claim 1, characterized in that The sending the memory information table to the target virtual machine includes: Determining a target virtualization firmware that matches the operating system of the target virtual machine according to the operating system characteristics thereof, and if the target virtualization firmware is the target firmware, reading each table entry in the memory information table through the target virtualization firmware, and creating a firmware reserved memory interval for the firmware reserved table entry in the memory information table; wherein the firmware reserved memory interval is used to represent the memory address of the firmware reserved table entry indicating the system reserved memory; The memory information table is transparently transmitted to the target virtual machine through the target virtualization firmware.

3. The method according to claim 2, characterized in that The method further comprises: If the target virtualization firmware is the target firmware, determine whether the memory address range corresponding to the user-available table entry in the memory information table belongs to the target address range, and create an available memory interval for the user-available table entry whose memory address range belongs to the target address range; wherein, the available memory interval is used to characterize that the memory address of the user-available table entry indicates the available memory of the target virtual machine.

4. The method according to claim 1, wherein The target virtual machine is started according to the memory configuration requirements of the target virtual machine and two types of corresponding entries are created, including: Determine a first memory address range according to a target memory address range indicated by the memory configuration requirement of the target virtual machine, and create a user-available table entry based on the first memory address range; A second memory address range is determined according to the target memory address range indicated by the memory configuration requirement of the target virtual machine, and a firmware reservation table entry is created based on the second memory address range; the memory size of the second memory address range is in a first proportional relationship with the memory size of the target memory address range.

5. The method according to claim 1, characterized in that The target virtual machine runs on a physical machine connected to a data processor DPU, and the physical machine relies on the DPU to offload some functions.

6. The method according to claim 1, characterized in that Before receiving the virtual machine creation request, the method further includes: During the physical machine kernel startup phase, a physical memory of a preset memory size is reserved for the virtual machine; wherein the physical memory of the preset memory size is used for memory management through a target memory management mode.

7. The method according to claim 6, characterized in that The method further comprises: Creating a memory file in a mount directory of a file system corresponding to the target memory management mode, and mapping the memory file to a virtual address space of the target virtual machine; If a page fault occurs when accessing the virtual address of the target virtual machine, memory is allocated for the target virtual machine based on the reserved physical memory of the preset memory size, and the allocated memory address is recorded in the memory file to persist the mapping relationship between the memory file and the physical memory of the preset memory size. The mapping relationship is used to access the corresponding physical memory through the memory address in the memory file.

8. A host memory resource processing device for a data processor virtualization scenario, characterized in that: The device comprises: A receiving module, configured to receive a virtual machine creation request, wherein the virtual machine creation request is used to request creation of a virtual machine that meets the memory configuration requirements of a target virtual machine; a creation module, configured to start a target virtual machine according to the memory configuration requirement of the target virtual machine and create two corresponding table entries, the two types of table entries including user-available table entries and firmware-reserved table entries, the memory address in the user-available table entry indicating the available memory of the target virtual machine, the memory address in the firmware-reserved table entry indicating the system-reserved memory, the memory size corresponding to the memory address in the firmware-reserved table entry being determined according to a preset first ratio and the memory size specified in the memory configuration requirement, the system-reserved memory being the memory space required for maintaining system functions, driving devices, and storing virtualized firmware; The sending module is configured to construct a memory information table corresponding to the target virtual machine based on the two types of table entries, and send the memory information table to the target virtual machine, where the memory information table represents physical memory mapping information of the target virtual machine.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device implements the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, characterized in that The computer program product comprises a computer program / instruction, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images