Host memory resource processing method and device for data processor virtualization scene, equipment and storage medium

Abstract

The invention provides a host memory resource processing method and device for a data processor virtualization scene, equipment and a storage medium, and the method comprises the steps: receiving a virtual machine creation request which is used for requesting to create a virtual machine meeting the memory configuration requirement of a target virtual machine; starting the target virtual machine according to the memory configuration requirement of the target virtual machine and creating two corresponding types of table items, including user available table items and firmware reserved table items; and constructing a memory information table corresponding to the target virtual machine based on the two types of table items, and sending the memory information table and the memory information to the target virtual machine to represent physical memory mapping information of the target virtual machine. Visibly, when the table item is created for the memory information table of the virtual machine, not only can the table item of a user available type be created, but also the table item of a firmware reservation type can be created, so that the system reserved memory indicated by the firmware reservation table item can be allocated to each virtual machine, and the user experience can be improved. Therefore, the utilization rate of host memory resources in a data processor virtualization scene is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular to a host memory resource processing method, device, equipment and storage medium in a data processor virtualization scenario. Background Art

[0002] DPU (Data Processing Unit) is a processor or processing unit used for data processing and accelerated computing tasks. It is usually used in data centers and cloud computing scenarios. It can accelerate data processing tasks and improve system performance and efficiency through hardware design and optimization algorithms.

[0003] In the DPU virtualization scenario, the advantages of DPU hardware acceleration are used to offload some virtualization components to the DPU, reducing the resource usage of the physical machine (or host).

[0004] In the DPU virtualization scenario, the physical machine is usually required to reserve part of the physical memory as system reserved memory, and this part of the resources cannot be allocated to the virtual machine for use. Therefore, in this scenario, a more efficient memory resource processing method is needed to improve resource utilization. Summary of the invention

[0005] In order to solve the above technical problems, the present disclosure provides a host memory resource processing method, device, equipment and storage medium in a data processor virtualization scenario, which can improve the utilization rate of the host memory resources.

[0006] In a first aspect, the present disclosure provides a host memory resource processing method in a data processor virtualization scenario, the method comprising:

[0007] Receive a virtual machine creation request, where the virtual machine creation request is used to request creation of a virtual machine that meets a memory configuration requirement of a target virtual machine;

[0008] Starting the target virtual machine according to the memory configuration requirement of the target virtual machine and creating two corresponding table items, the two table items including user available table items and firmware reserved table items, the memory address of the user available table items indicates the available memory of the target virtual machine, the memory address of the firmware reserved table items indicates the system reserved memory, and the memory size corresponding to the firmware reserved table items is determined according to the preset first ratio and the memory size specified in the memory configuration requirement;

[0009] A memory information table corresponding to the target virtual machine is constructed based on the two types of table entries, and the memory information table is sent to the target virtual machine, where the memory information table represents physical memory mapping information of the target virtual machine.

[0010] In an optional implementation manner, sending the memory information table to the target virtual machine includes:

[0011] Determine a target virtualization firmware that matches the target virtual machine according to the operating system characteristics of the target virtual machine; if the target virtualization firmware is the target firmware, read each table item in the memory information table through the target virtualization firmware, and create a firmware reserved memory interval for the firmware reserved table item in the memory information table; wherein the firmware reserved memory interval is used to characterize the memory address of the firmware reserved table item to indicate the system reserved memory;

[0012] The memory information table is transparently transmitted to the target virtual machine through the target virtualization firmware.

[0013] In an optional implementation manner, the method further includes:

[0014] If the target virtualization firmware is the target firmware, determine whether the memory address range corresponding to the user available table item in the memory information table belongs to the target address range, and create an available memory interval for the user available table item whose memory address range belongs to the target address range; wherein the available memory interval is used to characterize that the memory address of the user available table item indicates the available memory of the target virtual machine.

[0015] In an optional implementation, starting the target virtual machine and creating two types of corresponding table entries according to the memory configuration requirements of the target virtual machine include:

[0016] Determine a first memory address range according to a target memory address range indicated by a memory configuration requirement of the target virtual machine, and create a user-available table entry based on the first memory address range;

[0017] A second memory address range is determined according to the memory address range indicated by the memory configuration requirement of the target virtual machine, and a firmware reservation table entry is created based on the second memory address range; the memory size of the second memory address range is in a first proportional relationship with the memory size of the target memory address range.

[0018] In an optional implementation, the target virtual machine runs on a physical machine connected to a data processor DPU, and the physical machine relies on the DPU to offload some functions.

[0019] In an optional implementation manner, before receiving the memory configuration requirement of the target virtual machine, the method further includes:

[0020] During the physical machine kernel startup phase, a physical memory of a preset memory size is reserved for the virtual machine; wherein the physical memory of the preset memory size is used for memory management through a target memory management mode.

[0021] In an optional implementation manner, the method further includes:

[0022] Creating a memory file under the mount directory of the file system corresponding to the target memory management mode, and mapping the memory file to the virtual address space of the target virtual machine;

[0023] If a page fault occurs when accessing the virtual address of the target virtual machine, memory is allocated for the target virtual machine based on the reserved physical memory of the preset memory size, and the allocated memory address is recorded in the memory file to persist the mapping relationship between the memory file and the physical memory of the preset memory size. The mapping relationship is used to access the corresponding physical memory through the memory address in the memory file.

[0024] In a second aspect, the present disclosure provides a host memory resource processing device in a data processor virtualization scenario, the device comprising:

[0025] A receiving module, used for receiving a virtual machine creation request, wherein the virtual machine creation request is used for requesting to create a virtual machine that meets the memory configuration requirements of the target virtual machine;

[0026] a creation module, configured to start a target virtual machine according to a memory configuration requirement of the target virtual machine and create two corresponding table items, wherein the two table items include user available table items and firmware reserved table items, the memory address of the user available table items indicates the available memory of the target virtual machine, the memory address of the firmware reserved table items indicates the system reserved memory, and the memory size corresponding to the firmware reserved table items is determined according to a preset first ratio and the memory size specified in the memory configuration requirement;

[0027] The sending module is used to construct a memory information table corresponding to the target virtual machine based on the two types of table entries, and send the memory information table to the target virtual machine, where the memory information table represents physical memory mapping information of the target virtual machine.

[0028] In a third aspect, the present disclosure provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the computer program.

[0029] In a fourth aspect, the present disclosure provides a computer program product, wherein the computer program product comprises a computer program / instructions, and the computer program / instructions implement the above method when executed by a processor.

[0030] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has at least the following advantages:

[0031] In the host memory resource processing method for a data processor virtualization scenario provided by an embodiment of the present disclosure, a virtual machine creation request is first received, the request is used to request the creation of a virtual machine that meets the memory configuration requirements of a target virtual machine, and then the target virtual machine is started according to the memory configuration requirements of the target virtual machine and two corresponding types of table items are created, wherein the two types of table items include user available table items and firmware reserved table items, the memory address of the user available table items indicates the available memory of the target virtual machine, the memory address of the firmware reserved table items indicates the system reserved memory, and the memory size corresponding to the firmware reserved table items is determined according to a preset first ratio and the memory size specified in the memory configuration requirements; then a memory information table corresponding to the target virtual machine is constructed based on the two types of table items, and the memory information table is sent to the target virtual machine, the memory information table is used to characterize the physical memory mapping information of the target virtual machine.

[0032] It can be seen that when creating table entries for the memory information table of a virtual machine, the embodiment of the present disclosure can not only create table entries of user-available types, but also create table entries of firmware reserved types, and then distribute the system reserved memory indicated by the firmware reserved table entries to each virtual machine, thereby improving the utilization rate of the host memory resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0035] Figure 1 A flowchart of a memory resource processing method provided by an embodiment of the present disclosure;

[0036] Figure 2 A schematic diagram of a virtual machine simulator, virtualization firmware and target virtual machine provided in an embodiment of the present disclosure;

[0037] Figure 3 A schematic diagram of a user-available table item and a firmware reserved table item provided in an embodiment of the present disclosure;

[0038] Figure 4 A schematic diagram of the structure of a memory resource processing device provided in an embodiment of the present disclosure;

[0039] Figure 5 A schematic diagram of the structure of a memory resource processing device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0040] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0041] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0042] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0043] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0044] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0045] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0046] Some technical concepts or noun concepts involved in this article are explained:

[0047] A physical machine refers to a physical computer, including physical components such as processors, memory, and storage devices. A physical machine is used to provide a hardware environment for a virtual machine, so a physical machine is sometimes also called a "host" or "host."

[0048] A virtual machine refers to a logical computer system created on a physical machine through virtualization technology. Multiple virtual machines can be created on a physical machine. Each virtual machine can run an independent operating system and application programs. Virtual machines are usually called guest machines, and the physical computer where the virtual machine runs is called the host.

[0049] Virtualization components are software and hardware components necessary to support the operation and management of a virtualized environment. They are used to manage and control various functions in a virtualized environment to ensure the stability and security of the virtualized environment.

[0050] DPU (Data Processing Unit) is a processor or processing unit used for data processing and accelerated computing tasks, usually used in data centers and cloud computing scenarios. In the DPU virtualization scenario, the advantages of DPU hardware acceleration are used to offload some virtualization components to the DPU, reducing the resource usage of the physical machine (or host).

[0051] In the DPU virtualization scenario, the physical machine is usually required to reserve part of the physical memory as system reserved memory, and this part of the resources cannot be allocated to the virtual machine for use. Therefore, in this scenario, a more efficient memory resource processing method is needed to improve resource utilization.

[0052] To this end, an embodiment of the present disclosure provides a host memory resource processing method for a data processor virtualization scenario. Specifically, a virtual machine creation request is first received, where the request is used to request the creation of a virtual machine that meets the memory configuration requirements of a target virtual machine. Then, the target virtual machine is started according to the memory configuration requirements of the target virtual machine and two corresponding types of table items are created, wherein the two types of table items include user available table items and firmware reserved table items, the memory address of the user available table items indicates the available memory of the target virtual machine, the memory address of the firmware reserved table items indicates the system reserved memory, and the memory size corresponding to the firmware reserved table items is determined according to a preset first ratio and the memory size specified in the memory configuration requirements; then, a memory information table corresponding to the target virtual machine is constructed based on the two types of table items, and the memory information table is sent to the target virtual machine, where the memory information table is used to characterize the physical memory mapping information of the target virtual machine.

[0053] It can be seen that when creating table entries for the memory information table of a virtual machine, the embodiment of the present disclosure can not only create table entries of user-available types, but also create table entries of firmware reserved types, and then distribute the system reserved memory indicated by the firmware reserved table entries to each virtual machine, thereby improving the utilization rate of the host memory resources.

[0054] Based on this, the present disclosure provides a host memory resource processing method for a data processor virtualization scenario, referring to Figure 1, is a flowchart of a host memory resource processing method in a data processor virtualization scenario provided by an embodiment of the present disclosure, the method comprising:

[0055] S101: Receive a virtual machine creation request, where the virtual machine creation request is used to request the creation of a virtual machine that meets a memory configuration requirement of a target virtual machine.

[0056] The host memory resource processing method for the DPU virtualization scenario provided by the embodiment of the present disclosure can be applied to any physical machine in a DPU virtualization cluster or a DPU virtualization system.

[0057] In actual applications, a virtualization cluster or a virtualization system may include one or more cloud controllers, and a cloud controller may be connected to multiple computing nodes. The cloud controller is mainly responsible for managing the use of cloud resources, monitoring the performance of the cloud environment, and ensuring cloud security, while the computing nodes are mainly responsible for executing computing tasks and running applications. In actual applications, a physical machine connected to a data processing unit (DPU) can be regarded as a computing node.

[0058] In the disclosed embodiment, a virtual machine simulator may be run on a physical machine connected to the DPU, and the virtual machine simulator is used to receive a virtual machine creation request, where the virtual machine creation request is used to request the creation of a virtual machine that meets the memory configuration requirements of the target virtual machine.

[0059] In an optional implementation, the target virtual machine can run on a physical machine connected to a data processor DPU, which relies on the DPU to offload some functions. For example, some virtualization components on the physical machine can be offloaded to the DPU, and then various functions in the virtualized environment can be managed and controlled based on the DPU to ensure the stability and security of the virtualized environment.

[0060] In actual applications, when cloud control receives a memory application request for a target virtual machine sent from a client, it first determines the memory configuration requirements of the target virtual machine based on the system's preset memory resource allocation method, and then sends a virtual machine creation request to the physical machine connected to the DPU based on the memory configuration requirements of the target virtual machine, requesting the physical machine connected to the DPU to create a virtual machine that meets the memory configuration requirements of the target virtual machine.

[0061] In an optional implementation, the memory configuration requirement may include information about at least one numa (Non-Uniform Memory Access) node allocated to the target virtual machine, and specify a corresponding memory address range for each numa node. Among them, the name of the numa node, the memory size of the numa node, and the memory address range corresponding to the numa node can be understood as the memory configuration requirement of the target virtual machine in the embodiment of the present disclosure.

[0062] Exemplarily, when cloud control receives a memory application request for a 2G target virtual machine, it first determines the memory configuration requirements of the target virtual machine according to the system preset memory resource allocation method, for example, allocating two 1G numa nodes to the 2G target virtual machine, such as numa node 1 and numa node 2, where the memory address range of numa node 1 is A1-A2, the memory size is 1G, and the memory address range of numa node 2 is B1-B2, the memory size is 1G. After determining the memory configuration requirements of the target virtual machine, cloud control sends the memory configuration requirements of the target virtual machine to the computing node connected to cloud control in the form of a virtual machine creation request, that is, the physical machine connected to the DPU.

[0063] S102: Start the target virtual machine according to the memory configuration requirement of the target virtual machine and create two types of corresponding table entries.

[0064] Among them, the two types of table items include user-available table items and firmware reserved table items, the memory address of the user-available table items indicates the available memory of the target virtual machine, the memory address of the firmware reserved table items indicates the system reserved memory, and the memory size corresponding to the firmware reserved table items is determined according to a preset first ratio and the memory size specified in the memory configuration requirement.

[0065] In the disclosed embodiment, after receiving a virtual machine creation request, the physical machine connected to the DPU calls the virtual machine simulator to start the target virtual machine according to the memory configuration requirements of the target virtual machine and creates two corresponding table entries.

[0066] In an optional implementation, the virtual machine simulator may create two types of entries for the target virtual machine according to the memory address range of the numa node in the memory configuration requirements of the target virtual machine, wherein the memory address ranges corresponding to the two types of entries created respectively belong to the memory address range of the numa node.

[0067] In the disclosed embodiment, the two types of table entries include user available table entries and firmware reserved table entries, wherein the memory address of the user available table entry is used to indicate the available memory of the target virtual machine, that is, the address range of the available memory in the target virtual machine can be determined based on the memory address corresponding to the user available table entry Ram, and the memory address of the firmware reserved table entry is used to indicate the system reserved memory, that is, the address range of the system reserved memory in the target virtual machine can be determined based on the memory address corresponding to the firmware reserved table entry Reserved.

[0068] Exemplarily, assuming that the memory configuration requirements of the target virtual machine are used to create two types of table entries for numa node 1 and numa node 2, wherein the memory address range corresponding to numa node 1 is A1-A2, the memory size is 1G, and the memory address range corresponding to numa node 2 is B1-B2, the memory size is 1G. After receiving the above memory configuration requirements, the virtual machine simulator creates two types of table entries according to the memory address range of numa node 1, such as Entry0 (table entry 0) and Entry1 (table entry 1), and creates two types of table entries according to the memory address range of numa node 2, such as Entry2 (table entry 2) and Entry3 (table entry 3). Among them, Entry0 created based on numa node 1 is a user-available table entry, Entry1 is a firmware reserved table entry, Entry2 created based on numa node 2 is a user-available table entry, and Entry3 is a firmware reserved table entry.

[0069] In the embodiment of the present disclosure, the memory size corresponding to the firmware reservation table entry is determined according to a preset first ratio and a memory size specified in a memory configuration requirement, wherein the preset first ratio may be determined based on the memory resource size of the target virtual machine. For example, for a target virtual machine with a memory size of 64G, the corresponding preset first ratio may be 1 / 64. Accordingly, assuming that the memory size of the numa node specified in the memory configuration requirement is 2G, the memory size corresponding to the firmware reservation table entry is (1 / 64)*2G.

[0070] In an optional implementation, the memory address of the user available table entry and the memory address of the firmware reserved table entry together constitute the memory address of the NUMA node specified in the memory configuration requirement.

[0071] In actual applications, when user-available table items and firmware-reserved table items are created according to the internal configuration requirements of the target virtual machine, the sum of the memory addresses of the created user-available table items and the memory addresses of the firmware-reserved table items is equal to the memory size of the NUMA node, that is, the memory addresses of the user-available table items and the memory addresses of the firmware-reserved table items together constitute the memory address of the NUMA node.

[0072] S103: construct a memory information table corresponding to the target virtual machine based on the two types of table entries, and send the memory information table to the target virtual machine.

[0073] The memory information table is used to represent the physical memory mapping information of the target virtual machine.

[0074] In the disclosed embodiment, after the virtual machine simulator creates two types of corresponding table entries according to the memory configuration requirements of the target virtual machine, it first constructs a memory information table corresponding to the target virtual machine based on the two types of table entries.

[0075] In the disclosed embodiment, the memory information table is used to characterize the physical memory mapping information of the target virtual machine, wherein the physical memory mapping information includes information such as the memory address range and memory attributes corresponding to each table entry. For example, it is assumed that the memory information table includes table entry Entry0 and table entry Entry1, wherein the memory address range corresponding to table entry Entry0 is E1-E2, and the memory attribute is available memory, indicating that the memory address segment E1-E2 is the available memory of the target virtual machine, and the memory address range corresponding to table entry Entry1 is F1-F2, and the memory attribute is system reserved memory, indicating that the memory address segment F1-F2 is the system reserved memory of the target virtual machine.

[0076] In the disclosed embodiment, after the virtual machine simulator constructs a memory information table corresponding to the target virtual machine based on the two types of table items, the memory information table is sent to the target virtual machine so that the target virtual machine can understand the layout and availability of the memory in the system by reading the physical memory mapping information recorded in the memory information table, and use and manage memory resources based on the physical memory mapping information.

[0077] In the memory resource processing method provided by the embodiment of the present disclosure, a virtual machine creation request is first received, the request is used to request the creation of a virtual machine that meets the memory configuration requirements of the target virtual machine, and then the target virtual machine is started according to the memory configuration requirements of the target virtual machine and two corresponding types of table items are created, wherein the two types of table items include user available table items and firmware reserved table items, the memory address of the user available table items indicates the available memory of the target virtual machine, the memory address of the firmware reserved table items indicates the system reserved memory, and the memory size corresponding to the firmware reserved table items is determined according to a preset first ratio and the memory size specified in the memory configuration requirements; then a memory information table corresponding to the target virtual machine is constructed based on the two types of table items, and the memory information table is sent to the target virtual machine, and the memory information table is used to characterize the physical memory mapping information of the target virtual machine.

[0078] It can be seen that when creating table entries for the memory information table of a virtual machine, the embodiment of the present disclosure can not only create table entries of user-available types, but also create table entries of firmware reserved types, and then distribute the system reserved memory indicated by the firmware reserved table entries to each virtual machine, thereby improving the utilization rate of the host memory resources.

[0079] In actual applications, virtualization firmware, such as SeaBIOS firmware and EDK2 / OVMF firmware, can also run on the physical machine connected to the DPU. Among them, SeaBIOS firmware is a firmware implemented based on the open source BIOS (Basic Input / Output System), which is usually used in virtualization platforms and embedded systems of the x86 platform, such as QEMU (Quick Emulator) or kernel-based virtual machine KVM. EDK2 / OVMF firmware is a UFFI firmware for virtual machines. In a virtualization environment, OVFM can be used in conjunction with a virtual machine simulator (such as QEMU) to provide virtual machines with the function of starting and initializing firmware. The physical machine connected to the DPU in the disclosed embodiment can support virtualization firmware SeaBIOS and EDK2 / OVMF at the same time.

[0080] Therefore, after constructing the memory information table corresponding to the target virtual machine based on the two types of table entries, the embodiment of the present disclosure can also send the memory information table to the virtualization firmware corresponding to the target virtual machine, and call the virtualization firmware to send the memory information table to the target virtual machine.

[0081] In the disclosed embodiment, the virtualization firmware corresponding to the target virtual machine can be determined from multiple virtualization firmwares based on the operating system characteristics of the target virtual machine. Specifically, the target virtualization firmware that matches the target virtual machine is determined according to the operating system characteristics of the target virtual machine. For example, the virtualization firmware corresponding to the target virtual machine can be determined based on the image file of the operating system of the target virtual machine, such as SeaBIOS firmware or EDK2 / OVMF firmware.

[0082] like Figure 2 As shown, it is a schematic diagram of a virtual machine simulator, virtualization firmware and target virtual machine provided in an embodiment of the present disclosure, specifically taking the platform where the virtual machine simulator QEMU is located as the 440fx platform as an example.

[0083] After cloud management and control determines the memory configuration requirements of the target virtual machine according to the system's preset memory resource allocation method, it sends a virtual machine creation request to the target physical machine connected to cloud management and control (i.e., the physical machine connected to the DPU). The virtual machine creation request is used to request the creation of a virtual machine that meets the memory configuration requirements of the target virtual machine.

[0084] After receiving the virtual machine creation request, the physical machine connected to the DPU calls the virtual machine emulator QEMU to start the target virtual machine according to the memory configuration requirements of the target virtual machine, and creates user available table entries and firmware reserved table entries, and builds the E820 memory information table corresponding to the target virtual machine based on the user available table entries and the firmware reserved table entries, and then sends the E820 memory information table to the target virtual machine through the virtualization firmware SeaBIOS or EDK2 / OVMF.

[0085] Among them, the E820 table includes table entries Entry0, Entry1, Entry2, Entry3...EntryN, each of which records a memory address and a memory attribute. Specifically, Entry0 and Entry1 are created based on the memory address of numa node 1, the memory address range corresponding to Entry0 is E1-E2, and the memory attribute is available memory. The memory address range corresponding to Entry1 is F1-F2, and the memory attribute is system reserved memory. Entry2 and Entry3 are created based on the memory address range of numa node 2.

[0086] When it is determined that the target virtualization firmware matching the operating system characteristics of the target virtual machine is SeaBIOS, the virtual machine simulator QEMU first sends the memory information table to SeaBIOS, and then SeaBIOS directly sends the memory information table to the target virtual machine.

[0087] In actual applications, since SeaBIOS can directly read and process the E820 table reported by QEMU, when it is determined that the virtual machine firmware is SeaBIOS, SeaBIOS can be directly called to read and process the table items in the E820 table.

[0088] In an optional implementation, if the target virtualization firmware is the target firmware, each table item in the memory information table is read by the target virtualization firmware, and a firmware reserved memory interval is created for the firmware reserved table item in the memory information table; then, the memory information table is transparently transmitted to the target virtual machine through the target virtualization firmware.

[0089] The target firmware may include EDK2 / OVMF firmware. Since the embodiment of the present disclosure modifies the method of creating table entries, the modified method of creating table entries is not perceptible to EDK2 / OVMF. Therefore, when it is determined that the target virtualization firmware is EDK2 / OVMF, EDK2 / OVMF needs to be called to further process the E820 table.

[0090] like Figure 2As shown, when it is determined that the target virtualization firmware that matches the operating system characteristics of the target virtual machine is EDK2 / OVMF, the virtual machine simulator QEMU first sends the memory information table to EDK2 / OVMF, and then EDK2 / OVMF reads each table item in the memory information table, and creates a firmware reserved memory interval for the firmware reserved table item in the memory information table. Furthermore, the memory information table is transparently transmitted to the target virtual machine through the EDK2 / OVMF firmware, so that the target virtual machine can access and manage memory resources by reading the physical memory mapping information recorded in the memory information table.

[0091] In the disclosed embodiment, the firmware reserved memory interval can be used to characterize the memory address of the firmware reserved table entry to indicate the system reserved memory. The system reserved memory refers to the memory space required to maintain the core functions of the system, drive devices, and store virtualized firmware. This part of the memory space will not be released or allocated to user processes during system operation. After creating a firmware reserved memory interval for the firmware reserved table entry in the memory information table, the memory address range corresponding to the firmware reserved table entry will not be allocated to other devices or user processes.

[0092] In an optional implementation, if the target virtualization firmware is the target firmware, each table entry in the memory information table is read through the target virtualization firmware, and it is determined whether the memory address range corresponding to the user available table entry in the memory information table belongs to the target address range, and an available memory interval is created for the user available table entry whose memory address range belongs to the target address range.

[0093] The available memory area is used to represent the available memory of the target virtual machine indicated by the memory address of the user available table entry. The available memory refers to the memory space in the system that is not reserved by the system or occupied by other system components, which can be used to run applications, store data, and other tasks. After creating an available memory interval for the user available table entry whose memory address range belongs to the target address range, the user process can allocate and use this part of the memory space through the operating system memory management module in the target virtual machine.

[0094] In actual applications, since the target firmware EDK2 / OVMF can directly read user-available table items above 4G, but cannot read user-available table items below 4G, it is necessary to create an available memory range for user-available table items below 4G, that is, the target address range can include a memory address range of 0-4G.

[0095] like Figure 2As shown, after the target firmware EDK2 / OVMF receives the E820 table sent from the virtual machine simulator, it first reads each table entry in the E820 table. When reading the user available table entry, it first determines whether the memory address range corresponding to the user available table entry is below 4G. If it is determined that the memory address range corresponding to the user available table entry is below 4G, an available memory range is created for the user available table entry.

[0096] In an optional implementation, a user-available table entry and a firmware-reserved table entry may also be created according to a memory address range indicated by a memory configuration requirement of the target virtual machine.

[0097] Specifically, a first memory address range is determined according to a target memory address range indicated by the memory configuration requirements of the target virtual machine, and a user-available table entry is created based on the first memory address range; and a second memory address range is determined according to the target memory address range indicated by the memory configuration requirements of the target virtual machine, and a firmware reserved table entry is created based on the second memory address range.

[0098] The target memory address range indicated by the memory configuration requirement of the target virtual machine may include the memory address range of the NUMA node allocated to the target virtual machine.

[0099] The memory size of the second memory address range is in a first proportional relationship with the target memory address range indicated by the memory configuration requirement of the target virtual machine. For example, the sum of the memory size corresponding to the first memory address range and the memory size corresponding to the second memory address range can be equal to the target memory address range indicated by the memory configuration requirement of the target virtual machine, that is, the memory size of the NUMA node allocated to the target virtual machine.

[0100] like Figure 3 As shown, a schematic diagram of a user available table entry and a firmware reserved table entry provided by an embodiment of the present disclosure is shown. Specifically, taking the 440fx platform as an example, the E820 table is created using the following rules. Among them, the allocatable memory intervals in the 440fx platform include 0-H and above F, for example, 0-3G and above 4G, and the memory configuration requirements of the target virtual machine include n numa nodes, wherein the memory address range of numa node n is A1-A2, the memory size is m, and the first proportional relationship is t.

[0101] If it is determined that the memory address range of the numa node n is above 0-H or F (that is, A1-A2 is less than H or A1-A2 is greater than F), the first memory address range is determined to be A1 to A1+m*t, and a user available table entry Ram is created based on the first memory address range, and the second memory address range is determined to be A1+m*t to A2, and a firmware reserved table entry Reserved is created based on the second memory address range, that is, an available table entry and a firmware reserved table entry are created.

[0102] If it is determined that the memory address range of numa node n includes both 0-H and above F, the following two cases are included:

[0103] In the first case, if A1+m*t is within 0-H (that is, the firmware reserved entry spans HF), the first memory address range is determined to be A1 to A1+m*t, and a user-available entry Ram is created based on the first memory address range, and the second memory address range is determined to be A1+m*t to H, F to A2, and a firmware reserved entry Reserved1 and a firmware reserved entry Reserved2 are created based on the second memory address range, that is, an available entry and two firmware reserved entries are created.

[0104] In the second case, if A1+m*t is above H (that is, the available table entries span HF), the first memory address range is determined to be A1 to H, F to A2-m*(1 / t), and user available table entries Ram1 and user available table entries Ram2 are created based on the first memory address range, and the second memory address range is determined to be A2-m*(1 / t) to A2, and a firmware reserved table entry Reserved is created based on the second memory address range, that is, two available table entries and one firmware reserved table entry are created.

[0105] In an optional implementation, since the target metadata is used to perform memory management on the target virtual machine, the target metadata cannot be offloaded to the DPU, thereby affecting the utilization of physical memory resources. Therefore, the disclosed embodiment can also optimize the physical memory resources required for managing the virtual machine memory by deleting the target metadata during the startup of the target physical machine.

[0106] In traditional memory management schemes, the physical machine kernel needs to maintain a huge page structure to track the usage of each memory page in the virtual machine. These data structures are usually stored in the physical machine's memory space, which will occupy a certain amount of physical machine memory resources (that is, some additional memory resources required for managing virtual machine memory), and this part of memory resources cannot be allocated to virtual machines. Therefore, in order to improve the utilization of physical machine memory resources, before receiving a virtual machine creation request, the target memory management method can be used to further optimize the memory resources on the physical machine.

[0107] Specifically, during the kernel startup phase of the physical machine, physical memory of a preset memory size is reserved for the virtual machine. Since this reserved physical memory is used for memory management through the target memory management method and will not be taken over by the memory management mechanism of the physical machine (that is, the traditional memory management solution will not be used for memory management), there is no need to reserve an additional portion of memory resources in the memory space of the physical machine for memory management. In this way, the memory resources originally used to store and manage virtual machines can be released from the memory resources of the physical machine and used for other purposes, such as being allocated to virtual machines, thereby further improving the utilization rate of the memory resources of the physical machine.

[0108] The target memory management method may include memory management methods such as DMEM (Direct Memory FS). In DMEM technology, memory resources on each numa node are generally managed through a dmem_node (also known as a dynamic memory node, or DMEM node) data structure, wherein the DMEM node may include information such as the node identifier, memory address range, and memory usage of the corresponding numa node.

[0109] Each numa node contains a certain number of dmem_regions (also known as dynamic memory regions, or DMEM regions), which can be used to manage the memory regions with continuous addresses on the numa node. Each DMEM region on the numa node can contain multiple dmem pages (also known as dynamic memory pages, or DMEM memory pages). Usually, a bitmap can be used to identify whether each DMEM memory page in the memory region is allocated, wherein the DMEM region can include the start and end addresses of the memory region with continuous addresses, memory size and other information. When DMEM supports 2MB large pages and 1GB large pages, the metadata of each DMEM memory page only contains the reference count of the memory page.

[0110] In an optional implementation, during the physical machine kernel startup phase, DMEM memory management technology can reserve physical memory of a preset memory size for the virtual machine on each numa node through command line parameters, and establish a dmem_node node / dmem_region memory area / dmem page memory page data structure for the physical memory to utilize DMEM memory management technology for memory management.

[0111] Among them, the dmem_node node is used to manage the affinity of memory under the numa architecture, which can ensure that memory allocation is as close as possible to the processor that is executing the task, thereby reducing memory access latency; dmem_region is used to represent the continuous memory area on the specified numa node.

[0112] In actual applications, after the physical machine kernel is started, DMEM memory management technology can also support mounting the target in the form of a file system and creating memory files under the mount directory.

[0113] Specifically, a memory file is created under a mount directory of a file system corresponding to the target memory management mode, and the memory file is mapped to a virtual address space of the target virtual machine.

[0114] In the disclosed embodiment, the virtual machine simulator is first called during the startup initialization phase of the target virtual machine to create a memory file in the mount directory of the DMEM file system, and the memory file is mapped to the virtual address space of the target virtual machine to perform memory pre-allocation by actively accessing the virtual address.

[0115] In actual applications, during the process of pre-allocating memory by actively accessing a virtual address, if the memory page corresponding to the virtual address is not in the physical memory, a page fault exception will be triggered, and the exception needs to be handled.

[0116] In the disclosed embodiment, if a page fault occurs when accessing the virtual address of the target virtual machine, memory is allocated for the target virtual machine based on the reserved physical memory of the preset memory size, and the allocated memory address is recorded in the memory file to persist the mapping relationship between the memory file and the physical memory of the preset memory size. The mapping relationship is used to access the corresponding physical memory through the memory address in the memory file.

[0117] In actual applications, when a page fault occurs in a virtual address, the page exception callback mechanism in DMEM will obtain an idle memory area in a DMEM node that meets the numa affinity according to the mapped numa affinity as the target DMEM area. After determining the target DMEM area, the allocation status of each DMEM memory page in the target DMEM area is determined according to the bitmap bitmap, and the DMEM memory page in the target DMEM area that is in an unallocated state is allocated to the target virtual machine, and the allocation status of the DMEM memory page is modified to an allocated state. Furthermore, the memory address of the DMEM memory page allocated to the target virtual machine is recorded in a memory file to persist the mapping relationship between the memory file and the physical memory of the reserved preset memory size.

[0118] For example, in a 64-bit operating system, the metadata data structure introduced by the physical machine kernel to track the status of each memory page takes up about 64 bytes. In a traditional memory resource management solution, when managing a virtual machine with a memory size of 320G, when managing the metadata of the virtual machine, it is necessary to reserve about 5G of physical memory on the physical machine. However, in the embodiment of the present disclosure, a physical memory of a preset memory size is reserved for the virtual machine during the physical machine startup phase, for example, 320G of memory is reserved, and the memory is managed using the target memory management method. Since the target memory management method does not need to maintain metadata for the physical memory, about 5G of physical memory can be saved for subsequent allocation to the virtual machine.

[0119] It can be seen that the embodiments of the present disclosure can save the physical memory required for managing the virtual machine and improve the utilization rate of the physical memory resources by reserving a physical memory of a preset memory size for the virtual machine and managing the physical memory through the target memory management method.

[0120] Based on the same inventive concept as the above embodiments, the present disclosure also provides a host memory resource processing device in a data processor virtualization scenario, referring to Figure 4 , is a structural diagram of a host memory resource processing device for a data processor virtualization scenario provided by an embodiment of the present disclosure, the device comprising:

[0121] The receiving module 401 is used to receive a virtual machine creation request, where the virtual machine creation request is used to request the creation of a virtual machine that meets the memory configuration requirements of the target virtual machine;

[0122] A creation module 402 is used to start the target virtual machine according to the memory configuration requirement of the target virtual machine and create two corresponding table items, wherein the two types of table items include user available table items and firmware reserved table items, the memory address of the user available table items indicates the available memory of the target virtual machine, the memory address of the firmware reserved table items indicates the system reserved memory, and the memory size corresponding to the firmware reserved table items is determined according to the preset first ratio and the memory size specified in the memory configuration requirement;

[0123] The sending module 403 is used to construct a memory information table corresponding to the target virtual machine based on the two types of table entries, and send the memory information table to the target virtual machine, where the memory information table represents physical memory mapping information of the target virtual machine.

[0124] In an optional implementation manner, the sending module includes:

[0125] The first creation submodule is used to determine the target virtualization firmware matching the target virtual machine according to the operating system characteristics of the target virtual machine, and if the target virtualization firmware is the target firmware, read each table item in the memory information table through the target virtualization firmware, and create a firmware reserved memory interval for the firmware reserved table item in the memory information table; wherein the firmware reserved memory interval is used to characterize the memory address of the firmware reserved table item to indicate the system reserved memory;

[0126] The transparent transmission submodule is used to transparently transmit the memory information table to the target virtual machine through the target virtualization firmware.

[0127] In an optional implementation, the device further includes:

[0128] A determination module is used to determine whether the memory address range corresponding to the user available table item in the memory information table belongs to the target address range if the target virtualization firmware is the target firmware, and to create an available memory interval for the user available table item whose memory address range belongs to the target address range; wherein the available memory interval is used to characterize that the memory address of the user available table item indicates the available memory of the target virtual machine.

[0129] In an optional implementation manner, the creation module includes:

[0130] A second creation submodule, configured to determine a first memory address range according to a target memory address range indicated by a memory configuration requirement of the target virtual machine, and to create a user-available table entry based on the first memory address range;

[0131] The third creation submodule is used to determine a second memory address range according to the memory address range indicated by the memory configuration requirement of the target virtual machine, and create a firmware reservation table entry based on the second memory address range; the memory size of the second memory address range is in a first proportional relationship with the memory size of the target memory address range.

[0132] In an optional implementation, the target virtual machine runs on a physical machine connected to a data processor DPU, and the physical machine relies on the DPU to offload some functions.

[0133] In an optional implementation, the device further includes:

[0134] The reservation module is used to reserve physical memory of a preset memory size for the virtual machine during the physical machine kernel startup phase; wherein the physical memory of the preset memory size is used for memory management through a target memory management method.

[0135] In an optional implementation, the device further includes:

[0136] A mapping module, used to create a memory file under the mount directory of the file system corresponding to the target memory management mode, and map the memory file to the virtual address space of the target virtual machine;

[0137] A persistence module is used to allocate memory for the target virtual machine based on the reserved physical memory of the preset memory size if a page fault occurs in the virtual address access to the target virtual machine, and record the allocated memory address in the memory file to persist the mapping relationship between the memory file and the physical memory of the preset memory size, and the mapping relationship is used to access the corresponding physical memory through the memory address in the memory file.

[0138] In a host memory resource processing device of a data processor virtualization scenario provided by an embodiment of the present disclosure, a virtual machine creation request is first received, the request is used to request the creation of a virtual machine that meets the memory configuration requirements of a target virtual machine, and then the target virtual machine is started according to the memory configuration requirements of the target virtual machine and two corresponding types of table items are created, wherein the two types of table items include user available table items and firmware reserved table items, the memory address of the user available table items indicates the available memory of the target virtual machine, the memory address of the firmware reserved table items indicates the system reserved memory, and the memory size corresponding to the firmware reserved table items is determined according to a preset first ratio and the memory size specified in the memory configuration requirements; then a memory information table corresponding to the target virtual machine is constructed based on the two types of table items, and the memory information table is sent to the target virtual machine, the memory information table is used to characterize the physical memory mapping information of the target virtual machine.

[0139] It can be seen that when creating table entries for the memory information table of a virtual machine, the embodiment of the present disclosure can not only create table entries of user-available types, but also create table entries of firmware reserved types, and then distribute the system reserved memory indicated by the firmware reserved table entries to each virtual machine, thereby improving the utilization rate of the host memory resources.

[0140] The host memory resource processing of the data processor virtualization scenario provided by the embodiments of the present disclosure can execute the memory resource processing method provided by any embodiment of the present disclosure, and has the functional modules and beneficial effects corresponding to the execution method.

[0141] In order to implement the above embodiments, the present disclosure further proposes a computer program product, including a computer program / instruction, which implements the host memory resource processing method in the above embodiments when executed by a processor.

[0142] In addition, the present disclosure also provides a host memory resource processing device for a data processor virtualization scenario, see Figure 5 As shown, this may include:

[0143] Processor 501, memory 502, input device 503 and output device 504. The number of processors 501 in the host memory resource processing device can be one or more. Figure 5 In some embodiments of the present disclosure, the processor 501, the memory 502, the input device 503 and the output device 504 may be connected via a bus or other means, wherein: Figure 5 The example of connecting through bus is taken in the following.

[0144] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing of the host memory resource processing device by running the software programs and modules stored in the memory 502. The memory 502 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc. In addition, the memory 502 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. The input device 503 can be used to receive input digital or character information, and generate signal input related to user settings and function control of the host memory resource processing device.

[0145] Specifically in this embodiment, the processor 501 will load the executable files corresponding to the processes of one or more applications into the memory 502 according to the following instructions, and the processor 501 will run the applications stored in the memory 502, thereby realizing the various functions of the above-mentioned host memory resource processing device.

[0146] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0147] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for processing host memory resources in a data processor virtualization scenario, characterized in that: The method comprises: Receive a virtual machine creation request, where the virtual machine creation request is used to request creation of a virtual machine that meets a memory configuration requirement of a target virtual machine; Starting the target virtual machine according to the memory configuration requirement of the target virtual machine and creating two corresponding table items, the two table items including user available table items and firmware reserved table items, the memory address of the user available table items indicates the available memory of the target virtual machine, the memory address of the firmware reserved table items indicates the system reserved memory, and the memory size corresponding to the firmware reserved table items is determined according to the preset first ratio and the memory size specified in the memory configuration requirement; A memory information table corresponding to the target virtual machine is constructed based on the two types of table entries, and the memory information table is sent to the target virtual machine, where the memory information table represents physical memory mapping information of the target virtual machine.

2. The method according to claim 1, characterized in that The sending the memory information table to the target virtual machine includes: Determine a target virtualization firmware that matches the target virtual machine according to the operating system characteristics of the target virtual machine; if the target virtualization firmware is the target firmware, read each table item in the memory information table through the target virtualization firmware, and create a firmware reserved memory interval for the firmware reserved table item in the memory information table; wherein the firmware reserved memory interval is used to characterize the memory address of the firmware reserved table item to indicate the system reserved memory; The memory information table is transparently transmitted to the target virtual machine through the target virtualization firmware.

3. The method according to claim 2, characterized in that The method further comprises: If the target virtualization firmware is the target firmware, determine whether the memory address range corresponding to the user available table item in the memory information table belongs to the target address range, and create an available memory interval for the user available table item whose memory address range belongs to the target address range; wherein the available memory interval is used to characterize that the memory address of the user available table item indicates the available memory of the target virtual machine.

4. The method according to claim 1, characterized in that: The target virtual machine is started according to the memory configuration requirements of the target virtual machine and two types of corresponding table entries are created, including: Determine a first memory address range according to a target memory address range indicated by a memory configuration requirement of the target virtual machine, and create a user-available table entry based on the first memory address range; A second memory address range is determined according to the memory address range indicated by the memory configuration requirement of the target virtual machine, and a firmware reservation table entry is created based on the second memory address range; the memory size of the second memory address range is in a first proportional relationship with the memory size of the target memory address range.

5. The method according to claim 1, characterized in that The target virtual machine runs on a physical machine connected to a data processor DPU, and the physical machine relies on the DPU to offload some functions.

6. The method according to claim 1, characterized in that Before receiving the memory configuration requirement of the target virtual machine, the method further includes: During the physical machine kernel startup phase, a physical memory of a preset memory size is reserved for the virtual machine; wherein the physical memory of the preset memory size is used for memory management through a target memory management mode.

7. The method according to claim 6, characterized in that The method further comprises: Creating a memory file under the mount directory of the file system corresponding to the target memory management mode, and mapping the memory file to the virtual address space of the target virtual machine; If a page fault occurs when accessing the virtual address of the target virtual machine, memory is allocated for the target virtual machine based on the reserved physical memory of the preset memory size, and the allocated memory address is recorded in the memory file to persist the mapping relationship between the memory file and the physical memory of the preset memory size. The mapping relationship is used to access the corresponding physical memory through the memory address in the memory file.

8. A host memory resource processing device for a data processor virtualization scenario, characterized in that: The device comprises: A receiving module, used for receiving a virtual machine creation request, wherein the virtual machine creation request is used for requesting to create a virtual machine that meets the memory configuration requirements of the target virtual machine; a creation module, configured to start a target virtual machine according to a memory configuration requirement of the target virtual machine and create two corresponding table items, wherein the two table items include user available table items and firmware reserved table items, the memory address of the user available table items indicates the available memory of the target virtual machine, the memory address of the firmware reserved table items indicates the system reserved memory, and the memory size corresponding to the firmware reserved table items is determined according to a preset first ratio and the memory size specified in the memory configuration requirement; The sending module is used to construct a memory information table corresponding to the target virtual machine based on the two types of table entries, and send the memory information table to the target virtual machine, where the memory information table represents physical memory mapping information of the target virtual machine.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device implements the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, characterized in that The computer program product comprises a computer program / instruction, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images