Resource allocation, processing method, device, storage medium and program product
By introducing target resource allocation components into the container orchestration engine, pre-allocating large-page memory resources based on NUMA granularity solves the application deployment problem that the existing technology cannot support the NUMA mechanism, and realizes high-performance containerized deployment and large-page memory resource management.
Patent Information
- Application Number
- CN202411505990.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-24
- Publication Date
- 2025-03-11
- Estimated Expiration
- 2044-10-24
AI Technical Summary
The existing container orchestration engine has insufficient refinement in the allocation and management of large-page memory resources, and cannot support the deployment of applications that use the non-unified memory access (NUMA) mechanism to access large-page memory.
Provides a resource allocation method, by obtaining the resource allocation request of the target container unit, calling the target resource allocation component to pre-allocate large-page memory resources based on NUMA granularity, and allocating its directory to the target container unit, so that the target application can directly load and access these large-page memory files.
It realizes high-performance processing of target applications deployed in containerized form, meets the refined management requirements of large-page memory resources, and supports large-page memory access based on NUMA mechanism.
Smart Images

Figure CN119045952B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of virtualization technology, and in particular to a resource allocation and processing method, device, storage medium and program product. Background Art
[0002] Virtualization is a technology that allows multiple logical computing resources to be created and run on top of a single physical computing resource (such as a server, storage device, or network resource). With the rapid development of technologies such as cloud computing and big data, containerization has gradually become a mainstream lightweight alternative to virtualization. Compared with traditional virtual machines, containers share the operating system kernel of the host machine and can package applications and their dependencies together, allowing them to run consistently in various environments, thereby achieving a lighter-weight isolation environment.
[0003] The container orchestration engine is crucial for managing large-scale containerized environments. It provides a framework and tools for managing container lifecycles, deployment, scaling, and networking, reducing the complexity of handling large numbers of containers in production environments. Currently, container orchestration engines are increasingly managing container resources, and can already support containers' application and restriction of hugepage memory resources, allowing applications / services that rely heavily on hugepage memory resources to be containerized. Hugepages refer to memory pages that are larger than the conventional 4 KiB. For example, huge pages are usually 2MiB or 1GiB, which are used to reduce the number of page table entries, thereby improving memory access efficiency.
[0004] However, the existing container orchestration engines still have problems with the allocation and management of large page memory resources. For example, they still cannot support the deployment of some applications that use the Non-Uniform Memory Access (NUMA) mechanism to access large page memory. Therefore, a new solution is needed. Summary of the invention
[0005] Multiple aspects of the present application provide a resource allocation and processing method, device, storage medium and program product for pre-allocating large page memory resources based on NUMA granularity for target applications deployed in containerization.
[0006] An embodiment of the present application provides a resource allocation method, including: obtaining a resource allocation request of a target container unit, the target container unit is used to run a target application that accesses memory resources based on a non-uniform memory access mechanism, and the target node where the target container unit is located includes multiple non-uniform memory nodes; calling a target resource allocation component, pre-allocating at least one group of large page memory files according to the resource allocation request, the at least one group of large page memory files corresponding to at least one non-uniform memory node among the multiple non-uniform memory nodes; according to the directory of the at least one group of large page memory files, allocating the at least one group of large page memory files to the target container unit, the target application is used to directly load the at least one group of large page memory files, and use the non-uniform memory access mechanism to access the at least one group of large page memory files.
[0007] Optionally, calling a target resource allocation component to pre-allocate at least one group of large page memory files according to the resource allocation request, including: parsing the resource allocation request to obtain declaration information of the container storage interface; if the declaration information includes identification information of the target resource allocation component, calling the target resource allocation component to pre-allocate at least one group of large page memory files according to the resource allocation request.
[0008] Optionally, allocating the at least one group of large page memory files to the target container unit according to the directory of the at least one group of large page memory files includes: determining, within the target container unit, the large page memory directory corresponding to each of the at least one non-uniform memory nodes; calling a container storage interface, and mounting the directory of the at least one group of large page memory files to the large page memory directory corresponding to each of the at least one non-uniform memory nodes according to the correspondence between the at least one group of large page memory files and the at least one non-uniform memory node.
[0009] An embodiment of the present application also provides a resource allocation method, comprising: receiving a resource allocation request for a target container unit sent by a node agent, the target container unit being used to run a target application program that accesses memory resources based on a non-uniform memory access mechanism, and the target node where the target container unit is located includes multiple non-uniform memory nodes; pre-allocating at least one group of large page memory files according to the resource allocation request, the at least one group of large page memory files corresponding to at least one non-uniform memory node among the multiple non-uniform memory nodes; returning a directory of the at least one group of large page memory files to the node agent, the directory of the at least one group of large page memory files being used to enable the node agent to allocate the at least one group of large page memory files to the target container unit.
[0010] Optionally, according to the resource allocation request, at least one group of large page memory files is pre-allocated, including: according to the resource allocation request, determining the at least one non-uniform memory node to which the large page memory resources are to be allocated from the multiple non-uniform memory nodes; in the physical memory of the target node, pre-allocating at least one local physical memory corresponding to the at least one non-uniform memory node for the target container unit, and the address of any local physical memory is continuous; and creating large page memory files in the local physical memory pre-allocated to the at least one non-uniform memory node, respectively, to obtain the at least one group of large page memory files.
[0011] Optionally, it also includes: in the process of pre-allocating the at least one continuous local physical memory for the target container unit, if there is a resource allocation request of at least one other container unit to be processed, blocking the resource allocation request of the at least one other container unit, and the at least one other container unit is located on the target node.
[0012] An embodiment of the present application also provides a resource processing method, comprising: responding to a startup operation of a target application in a target container unit, determining a directory of at least one group of large page memory files corresponding to at least one non-uniform memory node allocated to the target container unit; wherein the at least one group of large page memory files is pre-allocated to the target container unit using the resource allocation method provided in an embodiment of the present application; according to the directory of the at least one group of large page memory files, loading the at least one group of large page memory files, and mapping the at least one group of large page memory files to a virtual address space corresponding to the target application, so as to access the at least one group of large page memory files in the virtual address space based on a non-uniform memory access mechanism.
[0013] Optionally, determining the directory of at least one group of large page memory files corresponding to multiple non-uniform memory nodes allocated to the target container unit includes: determining, within the target container unit, the large page memory directory corresponding to each of the at least one non-uniform memory nodes; and reading, according to the large page memory directory corresponding to each of the at least one non-uniform memory nodes, the directory of at least one group of large page memory files corresponding to the at least one non-uniform memory node.
[0014] The embodiment of the present application also provides a resource allocation system, including: a scheduler, a target node and a target resource allocation component; a node agent and a target container unit are running on the target node, and a target application is running in the target container unit; the target node includes multiple non-uniform memory nodes, and the target application accesses memory resources based on a non-uniform memory access mechanism; wherein the scheduler is used to: respond to a deployment request of the target container unit and schedule the target container unit to the target node; the node agent is used to: obtain a resource allocation request of the target container unit; call the target resource allocation component, pre-allocate at least one group of large page memory files according to the resource allocation request, and the at least one group of large page memory files corresponds to at least one non-uniform memory node among the multiple non-uniform memory nodes; and allocate at least one group of large page memory files according to the at least one group of large page memory files. The target container unit comprises a directory of at least one large page memory file and a directory of at least one large page memory file corresponding to the at least one non-uniform memory node; the target resource allocation component is used to: receive the resource allocation request, pre-allocate at least one large page memory file corresponding to the at least one non-uniform memory node, and return the directory of the at least one large page memory file to the node agent; the target application is used to: respond to the startup operation of the target application, determine the directory of at least one large page memory file corresponding to the at least one non-uniform memory node allocated to the target container unit; load the at least one large page memory file according to the directory of the at least one large page memory file, and map the at least one large page memory file to the virtual address space corresponding to the target application, so as to access the at least one large page memory file in the virtual address space based on the non-uniform memory access mechanism.
[0015] An embodiment of the present application also provides an electronic device, comprising: a memory and a processor; the memory is used to store one or more computer instructions; the processor is used to execute the one or more computer instructions to: execute the steps in the method provided in the embodiment of the present application.
[0016] The embodiment of the present application also provides a computer-readable storage medium storing a computer program, which can implement the steps in the method provided in the embodiment of the present application when the computer program is executed by a processor.
[0017] The embodiment of the present application also provides a computer program product, including: a computer program / instructions, which, when executed by a processor, can implement the steps in the method provided in the embodiment of the present application.
[0018] In an embodiment of the present application, after obtaining the resource allocation request of the target container unit, the target resource allocation component can be called to pre-allocate multiple groups of large page memory files corresponding to multiple non-uniform memory nodes on the target node for the target container unit, and the directory of the multiple groups of large page memory files can be returned to the node agent. The node agent can allocate the multiple groups of large page memory files to the target container unit so that the target application running in the target container unit can access at least one group of large page memory files on the target node based on the non-uniform memory access mechanism. Furthermore, in the case where the container orchestration engine does not support the allocation of large page memory resources to the target container unit according to the granularity of non-uniform memory nodes, the pre-allocation of large page memory resources based on the granularity of non-uniform memory nodes is realized with the help of the target resource allocation component, which facilitates the containerized deployment of the target application in the target container unit and meets the high performance processing requirements of the target application. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0020] Figure 1 It is a schematic diagram of allocating large page memory resources to DPDK programs in Pods based on control groups;
[0021] Figure 2 This is a schematic diagram of multiple Pods cross-applying for large page memory resources;
[0022] Figure 3 A schematic diagram of the structure of a resource allocation system provided by an exemplary embodiment of the present application;
[0023] Figure 4 A schematic diagram of the internal structure of a target node provided for an exemplary embodiment of the present application;
[0024] Figure 5 A schematic diagram of a flow chart of a resource allocation method provided by an exemplary embodiment of the present application when executed on a node agent side;
[0025] Figure 6 A schematic diagram of a flow chart of a resource allocation method provided by an exemplary embodiment of the present application when executed on the target resource allocation component side;
[0026] Figure 7 A schematic diagram of a flow chart of a resource processing method provided by an exemplary embodiment of the present application when executed on a target application side;
[0027] Figure 8 A schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION
[0028] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0029] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two, but does not exclude the inclusion of at least one.
[0030] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0031] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a product or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such a product or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the product or system including the element.
[0032] The container orchestration engine is used to implement the deployment, expansion and management of containerized applications. The container orchestration engine is deployed in a set of physical or virtual machines, which is called a cluster. The cluster consists of a control plane and nodes. Among them, the control plane of the cluster is responsible for managing the status and configuration of the entire cluster. The control plane includes at least an API (Application Programming Interface Server) server, a scheduler, and a key-value storage system. The API server is the front end of the control plane, which is used to provide an operation entry for the cluster and is responsible for verifying and configuring data. The scheduler is used to assign unassigned Pods to appropriate nodes. The key-value storage system is used to save all cluster data.
[0033] The nodes in the cluster are used to work together to run and manage containerized applications. Any node can be a physical server or a virtual machine. Each node runs a node agent, which is responsible for communicating with the API server and ensuring that the container on the node runs as expected. One or more container units Pod can run on a node. Pod is the smallest deployable and schedulable unit in the container orchestration engine. A Pod can contain one or more containers.
[0034] At present, the container orchestration engine has increasingly richer resource management for containers, and can already support the application and restriction of hugepage memory resources by containers, so that a large number of applications / services that rely on hugepage memory resources can be containerized. In the container orchestration engine, users can define the configuration data of the Pod through the configuration file of the container unit (i.e., Pod), and the configuration data includes the allocation description information of the Pod's memory resources. The size of the Hugepage resource is usually fixed. For example, a hugepage memory is usually 1G in size. In the configuration file, the application of the Hugepage resource of the Pod can be realized by describing the limits and requests fields of the Hugepage resource of the container orchestration engine. Among them, the requests field is used to define the minimum number of Hugepage resources required by the Pod, and the limits field defines the maximum number of Hugepage resources that the Pod can use.
[0035] Control group (Cgroup) is a kernel mechanism of Linux (an open source operating system) used to limit, describe and isolate the resource usage of a set of processes. The resources used by a set of processes may include but are not limited to CPU (Central Processing Unit), memory, disk I / O (Input / Output), etc. In the container orchestration engine, nodes can manage Hugepages based on Cgroup to perform fine-grained control over the allocation and use of Hugepage resources. A Hugepage resource allocation process based on the Cgroup mechanism is as follows:
[0036] 1. The user applies for Hugepage resources from the scheduler of the container orchestration engine through the Pod configuration file.
[0037] 2. After the scheduler schedules the Pod to the node, the node agent on the node can create a Cgroup of Hugepage resources for the Pod.
[0038] 3. After the Pod runs on the node, the process inside the Pod creates a Hugepage file through the MMAP (Memory-Mapped File) system call.
[0039] 4. Cgroup limits the hugepage memory used by the process.
[0040] The Hugepage resources allocated by the above method do not have NUMA attributes, which enables the container orchestration engine to control how many Hugepage resources the Pod uses, but cannot control which NUMA Hugpage resources the Pod uses, which makes it impossible to support the deployment of some applications that use the NUMA mechanism to access large page memory.
[0041] Secondly, in this method, the creation and mapping of Hugepage files are completed through standard Linux system calls. When the services running in the Pod need to apply for more Hugepages, it takes more time to perform the creation and mapping operations of Hugepage files. For some services, when the process is closed, all Hugepage files will be deleted, and the creation and mapping operations of Hugepage files will be re-executed when the process is restarted. As a result, the overall startup time of the services running in the Pod will be longer.
[0042] In addition, when multiple Pods on the same node are started at the same time, the service processes in multiple Pods may apply for Hugepage resources at the same time. In this case, the service processes in multiple Pods may apply for Hugepage resources alternately through MMAP system calls, which may cause the physical memory corresponding to the Hugepage resources of the same Pod to be alternating and discontinuous, such as Figure 2 In this case, the processing performance of the service process in the Pod may decrease.
[0043] In view of the above technical problems, a solution is provided in some embodiments of the present application. The technical solutions provided by the embodiments of the present application are described in detail below in conjunction with the accompanying drawings.
[0044] Figure 3 is a schematic diagram of the structure of a resource allocation system provided by an exemplary embodiment of the present application. The resource allocation system is built based on a container orchestration engine. Figure 3 As shown, the resource allocation system 30 includes: a scheduler 31, multiple nodes (such as a target node 32), and a target resource allocation component 33. The target node 32 can be any node in the container orchestration engine. Figure 3As shown, a node agent 34 and a target container unit 35 are running on the target node 32, and a target application 36 is running in the target container unit 35. The target node 32 includes multiple non-uniform memory nodes (hereinafter referred to as NUMA nodes), such as Figure 3 NUMA0 and NUMA1 are shown. The target application 36 can access the memory resources on the target node 32 based on the non-uniform memory access mechanism (hereinafter referred to as NUMA mechanism) to provide high-performance services.
[0045] Among them, the target node 32 is designed based on the NUMA architecture. In the NUMA architecture, the computer system is composed of multiple NUMA nodes, each NUMA node contains one or more processors and local memory, and can be connected to other NUMA nodes through a high-speed network or bus. The speed at which the processor in the NUMA node accesses its own local memory is greater than the speed at which the memory on another NUMA node is accessed through the network or bus. The number of NUMA nodes included in the target node 32 is usually fixed, and is determined by the motherboard design of the target node 32 and the number and type of CPUs installed. For example, when the target node 32 uses a multi-core CPU, its local memory can be allocated to multiple CPUs respectively to form multiple NUMA nodes. When the server motherboard design allows multiple CPUs to work together, local memory can be allocated to multiple CPUs working together to form a large NUMA node.
[0046] In this embodiment, when the target application 36 in the target container unit 35 accesses memory resources based on the NUMA mechanism, the scheduler 31 can respond to the deployment request of the target container unit 35 and schedule the target container unit 35 to the target node 32 containing multiple NUMA nodes to meet the requirements of the target application 36 for the NUMA architecture.
[0047] After the scheduler 31 schedules the target container unit 35 to the target node 32, the node agent 34 and the target resource allocation component 33 on the target node 32 can collaboratively allocate Hugepage resources with NUMA attributes to the target container unit 35. The Hugepage resource has the NUMA attribute, which means that the Hugepage resource is allocated to a specific NUMA node, thereby ensuring that the data requested and processed by the CPU in the NUMA node can be stored in the local memory of the CPU, thereby minimizing memory access latency.
[0048] Among them, the target resource allocation component 33 can be a plug-in that can communicate with the node agent 34 and the huge page file system in the kernel space of the target node 32, and can perform the pre-allocation operation of the Hugepage resource. The target resource allocation component 33 can run on the target node 32, or can run on other nodes in the resource allocation system 30, and the other nodes can be conventional servers or cloud servers, which are not limited in this embodiment. In some embodiments, the target resource allocation component 33 can be a CSI driver developed based on the container storage interface (Container Storage Interface, CSI) standard. In the container orchestration engine, CSI provides a standard method to connect an external storage system to the cluster of the container orchestration engine. CSI defines a set of APIs for enabling communication between the storage system and the cluster of the container orchestration engine to achieve lifecycle management of the storage volume (Volume). In this embodiment, the target resource allocation component 33 that complies with the CSI standard can create a huge page memory file in the physical memory of the target node 32, and enable the created huge page memory file to be used by the target container unit 35, so that the physical memory is provided to the containerized target application 36 in the form of a huge page memory file for use.
[0049] like Figure 4 As shown, the target resource allocation component 33 may include a CSI identification module and a large page memory management module. The CSI identification module is used to register the CSI driver with the node agent 34 based on the CSI identification, so that the node agent 34 can perceive the large page file system connected to the CSI driver. The large page memory management module is connected to the large page file system in the kernel space of the target node 32 to manage the large page memory based on the large page file system. Figure 4 , the resource allocation system 30 is further described in detail by taking the target node 32 as an example.
[0050] In the resource allocation system 30, the node agent 34 is used to: obtain the resource allocation request of the target container unit 35, call the target resource allocation component 33, and pre-allocate at least one group of large page memory files according to the resource allocation request. Among them, the resource allocation request can be initiated by the user through the API server of the container orchestration engine. In some embodiments, the user can submit a configuration file of the target container unit 35 through the API server, and the configuration file is used to describe the target container unit 35's demand information for Hugepage resources. For example, in the configuration file, the minimum number of Hugepage resources required by the target container unit 35 can be defined as 10G through the requests field, and the maximum number of Hugepage resources that the target container unit 35 can use can be defined as 10G using the limits field.
[0051] In the resource allocation system 30, the target resource allocation component 33 is used to: receive a resource allocation request of the target container unit 35 sent by the node agent 34, pre-allocate at least one group of large page memory files, and return the directory of the at least one group of large page memory files to the node agent 34. Among them, the at least one group of large page memory files corresponds to at least one NUMA node among the multiple NUMAs on the target node 32, each NUMA node may correspond to a group of large page memory files, and a group of large page memory files may include one or more large page memory files. The directory of any group of large page memory files refers to the storage address of the group of large page memory files on the target node 32. Among them, pre-allocation means that before the target application 36 is started, resources are allocated to the target application 36 in advance, thereby reducing the risk of affecting the processing performance of the target application 36 due to insufficient remaining allocable large page memory resources during the operation of the target application 36. Optionally, the target resource allocation component 33 that complies with the CSI standard can support node-published volumes (Node-Published Volume) and provide a Node-PublishedVolume method. Among them, the node-published volume is a local storage resource on the target node 32. When the node agent 34 sends a resource allocation request to the target resource allocation component 33, it can call the Node-Published Volume method of the target resource allocation component 33 through a remote procedure call (RPC), so that the method can create at least one large page memory resource pool corresponding to each NUMA node in the physical memory of the target node 32, and each large page memory resource pool contains a group of large page memory files.
[0052] Accordingly, the node agent 34 is used to receive the directory of at least one group of large page memory files returned by the resource allocation component 33 according to the resource allocation request, and allocate the at least one group of large page memory files to the target container unit 35 according to the directory of the at least one group of large page memory files.
[0053] The target application 36 is used to: respond to the startup operation of the target application 36, determine the directory of at least one group of large page memory files corresponding to the at least one NUMA node allocated to the target container unit 35, and map the at least one group of large page memory files to the virtual address space corresponding to the target application 36 according to the directory of the at least one group of large page memory files. Furthermore, when the target application 36 needs to access memory resources during operation, it can access the at least one group of large page memory files in the virtual address space based on the non-uniform memory access mechanism. Among them, the startup operation of the target application 36 can be a startup operation triggered by the user after the target application 36 completes the containerized deployment; or, it can be a timed startup operation of the target application 36; or, it can be a startup operation of the target application 36 driven by a specific script program; or, it can be a restart operation after the process of the target application 36 is stopped, etc. This embodiment includes but is not limited to this.
[0054] The virtual space address refers to a virtual memory area allocated by the operating system to the user space process (i.e., the process of the target application 36). At least one set of large page memory files is mapped to the virtual address space corresponding to the target application 36, so as to allocate the large page memory files to the user space process corresponding to the target application 36, so that the user space process can directly access these large page memory files. In the process of mapping the large page memory file to the virtual space address, it can be implemented by memory mapping. Based on memory mapping, the content in the large page memory file can be directly read and written in the memory without the need for traditional file I / O calls.
[0055] In this embodiment, the startup operation of the target application 36 includes the initial process startup operation after the target application 36 is deployed in the target container unit 35, and may also include the restart operation after the process of the target application 36 is stopped. In the restart scenario of the target application 36, the restarted target application 36 can directly load the directory of at least one group of large page memory files in the large page memory directory of the target container unit 35, and map at least one group of large page memory files to the virtual address space corresponding to the target application 36 according to the directory of at least one group of large page memory files. Furthermore, even if the target application 36 releases at least one group of large page memory files when the process stops, after the restart, the target application 36 does not need to re-apply for large page memory resources through system calls again, but can directly load the large page memory files in the large page memory directory and map them to re-acquire large page memory resources. In other words, even if the target application 36 is restarted, the required large page memory resources can be quickly allocated to the target application 36, thereby improving the pull-up speed of the target application 36.
[0056] Based on the above implementation, when the container orchestration engine does not support allocating large page memory resources to the Pod according to the granularity of the NUMA node, the pre-allocation of large page memory resources based on the granularity of the NUMA node is implemented with the help of the target resource allocation component 33, which facilitates the containerized deployment of the target application in the Pod and meets the high performance processing requirements of the target application.
[0057] In some optional embodiments, the configuration file of the target container unit 35 submitted by the user through the API server may include declaration information, which is used to declare the identifier of the target resource allocation component 33, so as to call the target resource allocation component 33. Optionally, the declaration information may be declaration information of the CSI, or may be declaration information of the API call of the target resource allocation component 33, or may be declaration information of a script file, which may call the target resource allocation component 33 at runtime, and this embodiment does not limit this.
[0058] Accordingly, when the node agent 34 calls the target resource allocation component 33 and pre-allocates at least one set of large page memory files according to the resource allocation request, it is specifically used to: parse the resource allocation request and obtain the declaration information of the container storage interface. If the declaration information includes the identification information of the target resource allocation component 33, the node agent 34 can call the target resource allocation component 33 and pre-allocate the at least one set of large page memory files according to the resource allocation request.
[0059] Taking the declaration information implemented as CSI declaration information as an example, in some embodiments, the identification information of the target resource allocation component 33 contained in the CSI declaration information may be the file name of the CSI driver, and the node agent 34 may call the CSI driver according to the file name of the CSI driver. If the declaration information of the configuration files of multiple different Pods submitted by the user all contain the identification of the target resource allocation component 33, then in the resource allocation system 30, the target resource allocation component 33 may be used as a unified entry for pre-allocating the large page memory resources corresponding to each of the multiple different Pods. In some embodiments, when pre-allocating the large page memory resources corresponding to each Pod, the target resource allocation component 33 may obtain the number of NUMA nodes of the node where each Pod is located, and allocate one or more groups of large page memory files to each Pod based on the number of NUMA nodes. In other embodiments, when pre-allocating the large page memory resources corresponding to each Pod, the target resource allocation component 33 may obtain the number of NUMA nodes specified in the resource allocation request, and pre-allocate one or more groups of large page memory files to each Pod based on the number of specified NUMA nodes. Based on this implementation, when the user has a need to allocate large page memory resources according to NUMA nodes, the user can customize the declaration information in the configuration file of the Pod, and add the identification information of the target resource allocation component 33 in the declaration information, and then instruct the node agent 34 to sink the allocation operation of the large page memory resources to the target resource allocation component 33, so as to call the target resource allocation component 33 to allocate large page memory resources with NUMA attributes to the Pod. Furthermore, in the case where the node agent 34 does not support the allocation of large page memory resources to the Pod according to NUMA nodes, the allocation operation of large page memory resources based on NUMA attributes is implemented with the help of the target resource allocation component 33.
[0060] In some exemplary embodiments, when the node agent 34 allocates at least one group of large page memory files to the target container unit 35 according to the directory of the at least one group of large page memory files, it may associate the at least one group of large page memory files with the Cgroup corresponding to the target container unit 35 to allocate the multiple groups of large page memory resources to the target container unit 35 for use.
[0061] In other exemplary embodiments, when the node agent 34 allocates the at least one group of large page memory files to the target container unit 35 according to the directory of the at least one group of large page memory files, the node agent 34 may mount the at least one group of large page memory files into the target container unit 35 in the form of a storage volume.
[0062] Optionally, the node agent 34 may determine the huge page memory directory corresponding to each of the at least one NUMA nodes in the target container unit 35, and call the container storage interface (i.e., CSI) to mount the directory of the at least one group of huge page memory files to the huge page memory directory corresponding to each of the at least one NUMA node according to the correspondence between the at least one group of huge page memory files and the at least one NUMA node. The huge page memory target is usually the / dev / hugepages directory.
[0063] On the target node 32, each NUMA node can mount a large page file system instance, such as a Hugetlbfs (a Linux file system that allows applications to use large pages) instance, and the large page memory directory corresponding to each NUMA node points to the Hugetlbfs instance corresponding to the NUMA node. Among them, the Hugetlbfs file system is a special file system in the Linux kernel for managing large page memory. Each NUMA node corresponds to a / dev / hugepages directory. For example, the target node 32 includes a NUMA 0 node and a NUMA 1 node, the NUMA0 node corresponds to the / dev / hugepages_node0 directory, and the NUMA1 node corresponds to the / dev / hugepages_node1 directory. The large page memory directories corresponding to the NUMA 0 node and the NUMA 1 node can be mounted as volumes in the target container unit 35 so that the containers in the target container unit 35 can use the large page memory files corresponding to the NUMA 0 node and the NUMA 1 node.
[0064] Based on this, after obtaining a set of huge page memory files corresponding to the NUMA0 node, this set of huge page memory files can be mounted to the / dev / hugepages_node0 directory; after obtaining a set of huge page memory files corresponding to the NUMA1 node, this set of huge page memory files can be mounted to the / dev / hugepages_node1 directory.
[0065] Based on the above implementation, the target application in the target container unit 35 can access the huge page memory file corresponding to the NUMA0 node through the / dev / hugepages_node0 directory, and access the huge page memory file corresponding to the NUMA1 node through the / dev / hugepages_node1 directory, thereby realizing access to the huge page memory resources based on the NUMA mechanism. In some high-performance processing scenarios, the target application can access the local memory corresponding to different CPUs through different CPUs. For any CPU, the speed of accessing the local memory is greater than the speed of accessing the memory on another NUMA node through the network or bus, which can effectively reduce access latency and improve processing efficiency.
[0066] For example, in a large-scale parallel processing scenario, a target application for executing a data processing task is deployed in a target container unit. Assume that the target container unit includes a NUMA1 node and a NUMA2 node, the NUMA1 node includes a CPU1 and its local memory R1, and the NUMA2 node includes a CPU2 and its local memory R2. The target application can use CPU1 and CPU2 to execute data processing tasks in parallel. Specifically, the target application can use CPU1 to read the data in its local memory R1, and store the intermediate data generated during the calculation process in the local memory R1 for subsequent use. Correspondingly, the target application can use CPU2 to read the data in its local memory R2, and store the intermediate data generated during the calculation process in the local memory R2 for subsequent use. In this embodiment, for CPU1 and CPU2, the data to be calculated and the intermediate result data are stored in their respective local memories, and a single CPU does not need to access the local memory in other NUMA nodes through a network or bus, thereby effectively improving the data processing efficiency.
[0067] For another example, in a graphics rendering scenario, in a multi-NUMA node architecture, different rendering tasks can be assigned to different NUMA nodes. For each rendering task, after determining the NUMA node to which it is assigned, the rendering thread corresponding to the rendering task is bound to the CPU in the NUMA node to ensure that the rendering thread can access the local memory of the CPU. Furthermore, when rendering threads in multiple NUMA nodes execute rendering tasks in parallel, they can mainly access the local memory of the CPU to which they are bound without accessing the local memory in other NUMA nodes, thereby reducing data transmission delay and improving rendering efficiency.
[0068] In some exemplary embodiments, when the target resource allocation component 33 pre-allocates at least one group of large page memory files corresponding to the at least one NUMA node for the target container unit 35, it can determine at least one NUMA node to which the large page memory resource is to be allocated from multiple NUMA nodes on the target node 32 according to the resource allocation request. Optionally, the number of at least one NUMA node to which the large page memory resource is to be allocated can be specified in the resource allocation request, or the identifier of at least one NUMA node to which the large page memory resource is to be allocated can be specified. After determining at least one NUMA node, the target resource allocation component 33 can pre-allocate multiple local physical memories corresponding to the at least one NUMA node for the target container unit 35 in the physical memory of the target node 32, and the address of any local physical memory is continuous. Among them, one NUMA node corresponds to a continuous local physical memory, and the continuous local physical memory means that the memory block corresponding to the NUMA node is continuous in the physical address space. Afterwards, large page memory files can be created in the local physical memory pre-allocated to at least one NUMA node respectively, and at least one group of large page memory files corresponding to the at least one NUMA node can be obtained.
[0069] In this implementation, the memory blocks corresponding to a single NUMA node are continuous, which is convenient for reducing memory access across NUMA nodes in multi-threaded or multi-process scenarios, thereby improving concurrent performance.
[0070] Optionally, in some embodiments, to ensure that continuous local physical memory corresponding to the NUMA node is pre-allocated for each container unit, the target resource allocation component 33 may be used as a unified large page memory resource allocation entry.
[0071] Continuing to take the target container unit 35 as an example, in the process of pre-allocating a plurality of continuous local physical memories corresponding to the at least one NUMA node for the target container unit 35, if there is a resource allocation request of at least one other container unit to be processed, the resource allocation request of the at least one other container unit is blocked, and the at least one other container unit is located on the target node 32. Further, it can be ensured that local physical memories are not cross-allocated to different container units on the target node 32 at the same time, thereby ensuring the continuity of the local physical memory of the same container unit.
[0072] Optionally, when the target resource allocation component 33 receives resource allocation requests from multiple container units, the resource allocation requests from the multiple container units may be processed serially to avoid cross-allocation of local physical memory to different container units, thereby ensuring the continuity of the local physical memory corresponding to a single container unit. For example, when receiving a resource allocation request from a target container unit 35, if there are other resource allocation requests for which resource pre-allocation has not yet been completed, the target resource allocation component 33 may add the resource allocation request from the target container unit 35 to the waiting queue. The target resource allocation component 33 may process the resource allocation requests added to the waiting queue one by one in a first-in-first-out order.
[0073] Figure 5 is a flow chart of a resource allocation method provided by an exemplary embodiment of the present application when it is executed on a node agent side. The method may include: Figure 5 Steps shown:
[0074] Step 501: Obtain a resource allocation request of a target container unit, where the target container unit is used to run a target application program that accesses memory resources based on a non-uniform memory access mechanism, and the target node where the target container unit is located includes multiple non-uniform memory nodes.
[0075] Step 502: Call the target resource allocation component to pre-allocate at least one group of large page memory files according to the resource allocation request, where the at least one group of large page memory files corresponds to at least one non-uniform memory node among the multiple non-uniform memory nodes.
[0076] Step 503: According to the directory of the at least one group of large page memory files, the at least one group of large page memory files is allocated to the target container unit, and the target application is used to directly load the at least one group of large page memory files and use the non-uniform memory access mechanism to access the at least one group of large page memory files.
[0077] Optionally, calling a target resource allocation component to pre-allocate at least one group of large page memory files according to the resource allocation request, including: parsing the resource allocation request to obtain declaration information of the container storage interface; if the declaration information includes identification information of the target resource allocation component, calling the target resource allocation component to pre-allocate at least one group of large page memory files according to the resource allocation request.
[0078] Optionally, allocating the at least one group of large page memory files to the target container unit according to the directory of the at least one group of large page memory files includes: determining, within the target container unit, the large page memory directory corresponding to each of the at least one non-uniform memory nodes; calling a container storage interface, and mounting the directory of the at least one group of large page memory files to the large page memory directory corresponding to each of the at least one non-uniform memory nodes according to the correspondence between the at least one group of large page memory files and the at least one non-uniform memory node.
[0079] In this embodiment, after the node agent obtains the resource allocation request of the Pod, the target resource allocation component can be called to pre-allocate at least one set of large page memory files corresponding to multiple NUMA nodes on the target node for the Pod. After obtaining the directory of at least one set of large page memory files, the node agent can allocate the at least one set of large page memory files to the Pod, so that the target application running in the Pod can access at least one set of large page memory files on the target node based on the NUMA access mechanism. Furthermore, in the case where the container orchestration engine does not support the allocation of large page memory resources to the Pod according to the granularity of the NUMA node, the pre-allocation of large page memory resources based on the granularity of the NUMA node is realized with the help of the target resource allocation component, which facilitates the containerized deployment of the target application in the Pod and meets the high-performance processing requirements of the target application.
[0080] Figure 6 is a flow chart of a resource allocation method provided by an exemplary embodiment of the present application when it is executed on the target resource allocation component side. The method may include: Figure 6 Steps shown:
[0081] Step 601: Receive a resource allocation request for a target container unit sent by a node agent. The target container unit is used to run a target application that accesses memory resources based on a non-uniform memory access mechanism. The target node where the target container unit is located includes multiple non-uniform memory nodes.
[0082] Step 602: pre-allocate at least one group of large page memory files according to the resource allocation request, wherein the at least one group of large page memory files corresponds to at least one non-uniform memory node among the multiple non-uniform memory nodes.
[0083] Step 603: Return the directory of the at least one group of large page memory files to the node agent, where the directory of the at least one group of large page memory files is used to enable the node agent to allocate the at least one group of large page memory files to the target container unit.
[0084] Optionally, according to the resource allocation request, at least one group of large page memory files is pre-allocated, including: according to the resource allocation request, determining the at least one non-uniform memory node to which the large page memory resources are to be allocated from the multiple non-uniform memory nodes; in the physical memory of the target node, pre-allocating at least one local physical memory corresponding to the at least one non-uniform memory node for the target container unit, and the address of any local physical memory is continuous; and creating large page memory files in the local physical memory pre-allocated to the at least one non-uniform memory node, respectively, to obtain the at least one group of large page memory files.
[0085] Optionally, it also includes: in the process of pre-allocating the at least one continuous local physical memory for the target container unit, if there is a resource allocation request of at least one other container unit to be processed, blocking the resource allocation request of the at least one other container unit, and the at least one other container unit is located on the target node.
[0086] In this embodiment, the target resource allocation component can establish a communication connection with the node agent and assist the node agent in pre-allocating at least one set of large page memory files corresponding to multiple NUMA nodes on the target node for the Pod. Furthermore, in the case where the container orchestration engine does not support the allocation of large page memory resources to the Pod according to the granularity of the NUMA node, the node agent is assisted in implementing the pre-allocation of large page memory resources based on the granularity of the NUMA node, which facilitates the containerized deployment of the target application in the Pod and meets the high-performance processing requirements of the target application.
[0087] Figure 7 is a flow chart of a resource processing method provided by an exemplary embodiment of the present application when it is executed on the target application side. The method may include: Figure 7 Steps shown:
[0088] Step 701, in response to a startup operation of a target application in a target container unit, determine a directory of at least one group of large page memory files allocated to the target container unit and corresponding to at least one non-uniform memory node; wherein the at least one group of large page memory files is pre-allocated to the target container unit by the node agent calling a target resource allocation component.
[0089] Step 702: According to the directory of the at least one group of large page memory files, load the at least one group of large page memory files, and map the at least one group of large page memory files to the virtual address space corresponding to the target application, so as to access the at least one group of large page memory files in the virtual address space based on a non-uniform memory access mechanism.
[0090] Optionally, determining the directory of at least one group of large page memory files corresponding to multiple non-uniform memory nodes allocated to the target container unit includes: determining, within the target container unit, the large page memory directory corresponding to each of the at least one non-uniform memory nodes; and reading, according to the large page memory directory corresponding to each of the at least one non-uniform memory nodes, the directory of at least one group of large page memory files corresponding to the at least one non-uniform memory node.
[0091] In this embodiment, after the target application is containerized and deployed, at least one group of large page memory files is determined directly by reading the directory of at least one group of large page memory files corresponding to multiple non-uniform memory nodes allocated to the target container unit, and at least one group of large page memory files is mapped to the virtual address space to use the large page memory resources. Relative to the traditional MMAP system call method, in the scheme provided by this embodiment, at least one group of large page memory files has been created, and the target application does not need to apply for and create large page memory resources by the MMAP system call method, but only needs to map the created large page memory files, thereby greatly improving the process pull-up speed of the target application. In particular, the more large page memory resources applied by the Pod, the more time-consuming the process pull-up of the target application is saved relative to the process pull-up time-consuming of the traditional scheme. In some embodiments, when the size multiple of the large page memory resources applied by the Pod increases, the process pull-up time in the Pod can also be kept within 1 second, which greatly improves the process pull-up efficiency. In some optional embodiments, relative to the large page memory resource allocation scheme based on the control group, the process pull-up efficiency in the Pod can be increased to about 60 times.
[0092] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 501 to 503 can be device A; for another example, the execution subject of steps 501 and 502 can be device A, and the execution subject of step 503 can be device B; and so on.
[0093] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and the sequence numbers of the operations, such as 501, 502, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.
[0094] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0095] Figure 8 A schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application is shown. Figure 8 As shown, the electronic device includes: a memory 801 , a processor 802 and a communication component 803 .
[0096] The memory 801 is used to store computer programs and can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application program or method operating on the electronic device.
[0097] In some exemplary embodiments, Figure 8 The electronic device shown can be used to execute a resource allocation method, wherein a processor 802 is coupled to a memory 801 and is used to execute a computer program corresponding to a node agent in the memory 801, so as to: obtain a resource allocation request of a target container unit, wherein the target container unit is used to run a target application program that accesses memory resources based on a non-uniform memory access mechanism, and the target node where the target container unit is located includes multiple non-uniform memory nodes; call a target resource allocation component, and pre-allocate at least one group of large page memory files according to the resource allocation request, wherein the at least one group of large page memory files corresponds to at least one non-uniform memory node among the multiple non-uniform memory nodes; and allocate the at least one group of large page memory files to the target container unit according to a directory of the at least one group of large page memory files, wherein the target application program is used to directly load the at least one group of large page memory files and access the at least one group of large page memory files using the non-uniform memory access mechanism.
[0098] Optionally, when calling the target resource allocation component and pre-allocating at least one group of large page memory files according to the resource allocation request, the processor 802 is specifically used to: parse the resource allocation request and obtain declaration information of the container storage interface; if the declaration information includes the identification information of the target resource allocation component, call the target resource allocation component and pre-allocate the at least one group of large page memory files according to the resource allocation request.
[0099] Optionally, when allocating the at least one group of large page memory files to the target container unit according to the directory of the at least one group of large page memory files, the processor 802 is specifically used to: determine, within the target container unit, the large page memory directory corresponding to each of the at least one non-uniform memory nodes; and call a container storage interface to mount the directory of the at least one group of large page memory files to the large page memory directory corresponding to each of the at least one non-uniform memory nodes according to the correspondence between the at least one group of large page memory files and the at least one non-uniform memory node.
[0100] In this embodiment, after the node agent obtains the resource allocation request of the Pod, it can call the target resource allocation component to pre-allocate at least one set of large page memory files corresponding to multiple NUMA nodes on the target node for the Pod. After obtaining the directory of at least one set of large page memory files, the node agent can allocate the at least one set of large page memory files to the Pod, so that the target application running in the Pod can access at least one set of large page memory files on the target node based on the NUMA access mechanism. Furthermore, in the case where the container orchestration engine does not support the allocation of large page memory resources to the Pod according to the granularity of the NUMA node, the pre-allocation of large page memory resources based on the granularity of the NUMA node is realized with the help of the target resource allocation component, which facilitates the containerized deployment of the target application in the Pod and meets the high-performance processing requirements of the target application.
[0101] In other exemplary embodiments, Figure 8 The electronic device shown can be used to execute a resource allocation method, wherein the processor 802 is coupled to the memory 801 and is used to execute a computer program corresponding to a target resource allocation component in the memory 801, so as to: receive a resource allocation request of a target container unit sent by a node agent, wherein the target container unit is used to run a target application program that accesses memory resources based on a non-uniform memory access mechanism, and the target node where the target container unit is located includes multiple non-uniform memory nodes; pre-allocate at least one group of large page memory files according to the resource allocation request, wherein the at least one group of large page memory files corresponds to at least one non-uniform memory node among the multiple non-uniform memory nodes; and return a directory of the at least one group of large page memory files to the node agent, wherein the directory of the at least one group of large page memory files is used to enable the node agent to allocate the at least one group of large page memory files to the target container unit.
[0102] Optionally, when the processor 802 pre-allocates at least one group of large page memory files according to the resource allocation request, it is specifically used to: determine, according to the resource allocation request, the at least one non-uniform memory node to which the large page memory resources are to be allocated from the multiple non-uniform memory nodes; pre-allocate at least one local physical memory corresponding to the at least one non-uniform memory node for the target container unit in the physical memory of the target node, and the address of any local physical memory is continuous; and create large page memory files in the local physical memory pre-allocated to the at least one non-uniform memory node, respectively, to obtain the at least one group of large page memory files.
[0103] Optionally, the processor 802 is further used to: in the process of pre-allocating the at least one continuous local physical memory for the target container unit, if there is a resource allocation request of at least one other container unit to be processed, block the resource allocation request of the at least one other container unit, and the at least one other container unit is located on the target node.
[0104] In this implementation, the target resource allocation component can establish a communication connection with the node agent, and assist the node agent in pre-allocating at least one set of large page memory files corresponding to multiple NUMA nodes on the target node for the Pod. Furthermore, in the case where the container orchestration engine does not support allocating large page memory resources to the Pod according to the granularity of the NUMA node, the node agent is assisted in implementing the pre-allocation of large page memory resources based on the granularity of the NUMA node, which facilitates the containerized deployment of the target application in the Pod and meets the high-performance processing requirements of the target application.
[0105] In some further exemplary embodiments, Figure 8 The electronic device shown can be used to execute a resource processing method, and the processor 802 is coupled to the memory 801, and is used to execute a computer program corresponding to the target application in the memory 801, so as to: respond to the startup operation of the target application in the target container unit, and determine the directory of at least one group of large page memory files corresponding to at least one non-uniform memory node allocated to the target container unit; wherein the at least one group of large page memory files is pre-allocated to the target container unit using the resource allocation method provided in an embodiment of the present application; according to the directory of the at least one group of large page memory files, load the at least one group of large page memory files, and map the at least one group of large page memory files to the virtual address space corresponding to the target application, so as to access the at least one group of large page memory files in the virtual address space based on the non-uniform memory access mechanism.
[0106] Optionally, when determining the directory of at least one group of large page memory files corresponding to multiple non-uniform memory nodes allocated to the target container unit, the processor 802 is specifically used to: determine, within the target container unit, the large page memory directory corresponding to each of the at least one non-uniform memory nodes; and read the directory of at least one group of large page memory files corresponding to the at least one non-uniform memory node respectively according to the large page memory directory corresponding to each of the at least one non-uniform memory nodes.
[0107] In this embodiment, after the target application is containerized and deployed, at least one group of large page memory files is determined directly by reading the directory of at least one group of large page memory files corresponding to multiple non-uniform memory nodes assigned to the target container unit, and at least one group of large page memory files is mapped to the virtual address space to use large page memory resources. Relative to the traditional MMAP system call method, in the scheme provided by this embodiment, at least one group of large page memory files has been created, and the target application does not need to apply for and create large page memory resources by means of MMAP system calls. It only needs to map the created large page memory files, thereby greatly improving the process pull-up speed of the target application. In particular, the more large page memory resources applied by the Pod, the more time-consuming the process pull-up of the target application is saved relative to the process pull-up time-consuming of the traditional solution. When the size multiples of the large page memory resources applied by the Pod increase, the process pull-up time in the Pod can also be kept within 1 second, which greatly improves the process pull-up efficiency.
[0108] Further, if Figure 8 As shown, the electronic device also includes other components such as a power supply component 804 , a display component 805 , and an audio component 806 . Figure 8 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 8 Components shown. Figure 8 Components in the dashed box are optional components, not mandatory components, and may depend on the product form of the electronic device. The electronic device of this embodiment may be implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, or an IOT device, or may be a server device such as a conventional server, a cloud server, or a server array. If the electronic device of this embodiment is implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, etc., it may include Figure 8 If the electronic device of this embodiment is implemented as a server device such as a conventional server, a cloud server or a server array, it may not include Figure 8 Components within the dashed box.
[0109] Among them, the memory 801 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.
[0110] The communication component 803 is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as Wi-Fi (wireless network communication technology), 2G (such as Global System for Mobile Communications (GSM)), 3G (such as Wideband Code Division Multiple Access (WCDMA), 4G (such as Long Term Evolution (LTE)), 4G+ (such as upgraded Long Term Evolution (LTE-Advanced, LTE-A)) or 5G (5th Generation Mobile Communication Technology), or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component can be implemented based on Near Field Communication (NFC) technology, Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.
[0111] The power supply component 804 is used to provide power to various components of the device where the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing and distributing power to the device where the power supply component is located.
[0112] The display assembly includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.
[0113] The audio component may be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal may be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal.
[0114] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement each step that can be executed by an electronic device in the above method embodiment.
[0115] The embodiment of the present application also provides a computer program product, including: a computer program / instructions, which, when executed by a processor, can implement the steps in the method provided in the embodiment of the present application.
[0116] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM (Compact Disc Read-Only Memory), optical storage, etc.) containing computer-usable program codes.
[0117] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0118] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0119] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0120] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interface, network interface and memory.
[0121] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.
[0122] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, Parallel Random Access Machine (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0123] It should also be noted that the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, product or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, product or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, product or device including the element.
[0124] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A resource allocation method, characterized in that: include: Obtaining a resource allocation request of a target container unit, wherein the target container unit is used to run a target application program that accesses memory resources based on a non-uniform memory access mechanism, and the target node where the target container unit is located includes multiple non-uniform memory nodes; Calling a target resource allocation component to pre-allocate at least one group of large page memory files according to the resource allocation request, wherein the at least one group of large page memory files corresponds to at least one non-uniform memory node among the multiple non-uniform memory nodes; According to the directory of the at least one group of large page memory files, the at least one group of large page memory files is allocated to the target container unit, and the target application is used to directly load the at least one group of large page memory files and use the non-uniform memory access mechanism to access the at least one group of large page memory files; Among them, the resource allocation request is used to enable the target resource allocation component to determine at least one non-uniform memory node from the multiple non-uniform memory nodes, and create at least one group of large page memory files corresponding to the at least one non-uniform memory node in multiple local physical memories on the target node.
2. The method according to claim 1, characterized in that Calling a target resource allocation component to pre-allocate at least one set of large page memory files according to the resource allocation request includes: Parsing the resource allocation request to obtain declaration information of the container storage interface; If the declaration information includes the identification information of the target resource allocation component, the target resource allocation component is called to pre-allocate the at least one group of large page memory files according to the resource allocation request.
3. The method according to claim 1, characterized in that Allocating the at least one group of large page memory files to the target container unit according to the directory of the at least one group of large page memory files includes: Determine, within the target container unit, a large page memory directory corresponding to each of the at least one non-uniform memory nodes; Call the container storage interface, and mount the directory of the at least one group of large page memory files to the large page memory directory corresponding to each of the at least one non-uniform memory nodes according to the correspondence between the at least one group of large page memory files and the at least one non-uniform memory node.
4. A resource allocation method, characterized in that: include: receiving a resource allocation request of a target container unit sent by a node agent, wherein the target container unit is used to run a target application program that accesses memory resources based on a non-uniform memory access mechanism, and a target node where the target container unit is located includes a plurality of non-uniform memory nodes; Pre-allocating at least one group of large page memory files according to the resource allocation request, wherein the at least one group of large page memory files corresponds to at least one non-uniform memory node among the multiple non-uniform memory nodes; Returning a directory of the at least one set of large page memory files to the node agent, wherein the directory of the at least one set of large page memory files is used to enable the node agent to allocate the at least one set of large page memory files to the target container unit; Among them, pre-allocating at least one group of large page memory files according to the resource allocation request includes: determining at least one non-uniform memory node from the multiple non-uniform memory nodes according to the resource allocation request, and creating at least one group of large page memory files corresponding to the at least one non-uniform memory node in multiple local physical memories on the target node.
5. The method according to claim 4, characterized in that Pre-allocating at least one set of large page memory files according to the resource allocation request, including: Determine, according to the resource allocation request, the at least one non-uniform memory node to which large page memory resources are to be allocated from the multiple non-uniform memory nodes; In the physical memory of the target node, at least one local physical memory corresponding to the at least one non-uniform memory node is pre-allocated for the target container unit, and the address of any local physical memory is continuous; Large page memory files are respectively created in the local physical memory pre-allocated to the at least one non-uniform memory node to obtain the at least one group of large page memory files.
6. The method according to claim 5, characterized in that Also includes: In the process of pre-allocating the at least one continuous local physical memory for the target container unit, if there is a resource allocation request of at least one other container unit to be processed, the resource allocation request of the at least one other container unit is blocked, and the at least one other container unit is located on the target node.
7. A resource processing method, characterized in that: include: In response to a start-up operation of a target application in a target container unit, determining a directory of at least one group of large page memory files corresponding to at least one non-uniform memory node allocated to the target container unit; wherein the at least one group of large page memory files is pre-allocated to the target container unit using the method described in any one of claims 1 to 6; According to the directory of the at least one group of large page memory files, load the at least one group of large page memory files, and map the at least one group of large page memory files to the virtual address space corresponding to the target application, so as to access the at least one group of large page memory files in the virtual address space based on a non-uniform memory access mechanism.
8. The method according to claim 7, characterized in that Determining a directory of at least one group of large page memory files corresponding to a plurality of non-uniform memory nodes and allocated to the target container unit comprises: Determine, within the target container unit, a large page memory directory corresponding to each of the at least one non-uniform memory nodes; According to the large page memory directory corresponding to each of the at least one non-uniform memory nodes, the directory of at least one group of large page memory files corresponding to the at least one non-uniform memory node is read respectively.
9. A resource allocation system, characterized in that: include: Scheduler, target node, and target resource allocation components; A node agent and a target container unit are running on the target node, and a target application is running in the target container unit; The target node includes a plurality of non-uniform memory nodes, and the target application accesses memory resources based on a non-uniform memory access mechanism; The scheduler is used to: respond to the deployment request of the target container unit and schedule the target container unit to the target node; The node agent is used to: obtain a resource allocation request of the target container unit; call the target resource allocation component to pre-allocate at least one group of large page memory files according to the resource allocation request, wherein the at least one group of large page memory files corresponds to at least one non-uniform memory node among the multiple non-uniform memory nodes; and allocate the at least one group of large page memory files to the target container unit according to a directory of the at least one group of large page memory files; The target resource allocation component is used to: receive the resource allocation request, pre-allocate at least one group of large page memory files corresponding to the at least one non-uniform memory node, including: determining the at least one non-uniform memory node from the multiple non-uniform memory nodes according to the resource allocation request, and creating the at least one group of large page memory files corresponding to the at least one non-uniform memory node in multiple local physical memories on the target node; and returning the directory of the at least one group of large page memory files to the node agent; The target application is used to: respond to the startup operation of the target application, determine the directory of at least one group of large page memory files allocated to the target container unit and corresponding to the at least one non-uniform memory node; load the at least one group of large page memory files according to the directory of the at least one group of large page memory files, and map the at least one group of large page memory files to the virtual address space corresponding to the target application, so as to access the at least one group of large page memory files in the virtual address space based on the non-uniform memory access mechanism.
10. An electronic device, characterized in that: include: Memory and processor; The memory is used to store one or more computer instructions; The processor is configured to execute the one or more computer instructions to: perform the steps in the method according to any one of claims 1-8.
11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps in the method described in any one of claims 1 to 8 can be implemented.
12. A computer program product, characterized in that include: A computer program / instruction, which, when executed by a processor, can implement the steps of the method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Memory management method used in Linux system
CN105893269A
Method and device for dynamically distributing large page for cloud native application
CN117251286A