Application starting method and device, electronic equipment and computer readable storage medium

Abstract

Embodiments of the present application provide an application starting method and device, electronic equipment and computer readable storage medium, and relate to the fields of cloud technology and games. The method is applied to an application system, the application system comprising an application management server and an application server comprising at least two NUMA nodes, each NUMA node corresponding to a respective storage resource and creating at least one container, and the method comprising the following operation performed by the application management server: receiving an application starting request for a target application sent by a user terminal; determining a target container based on an application identifier in the application starting request; and sending the application starting request to a target NUMA node to which the target container belongs, so that the target NUMA node controls the target container to start the target application based on a target container image file read from the corresponding storage resource. The embodiments of the present application accelerate the data reading speed and the starting speed of the target application, reduce the load pressure of a single NUMA node and the waiting time required for starting multiple target applications.

Description

Technical Field

[0001] This application relates to the fields of cloud technology and gaming, and more specifically, to an application launching method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] With the development of internet technology and the increasing demands of applications, the utilization rate of network and storage resources, as well as the requirements for resource management, are becoming increasingly stringent. Under these demands, container technology has emerged and developed rapidly. For example, different applications can be launched using different containers during the application startup process.

[0003] When launching an application via a container, the process typically involves three steps: starting the container, launching the operating system supporting the application, and launching the application within the container. All three steps require disk read / write operations. However, in related technologies, within the same container management system, each NUMA node usually corresponds to one CPU, but only one NUMA node has a dedicated disk. During disk read / write operations, for NUMA nodes without a dedicated disk, disk read / write can only be performed across NUMA nodes, resulting in low disk read / write efficiency and slow container and application startup speeds, which requires improvement. Summary of the Invention

[0004] This application provides an application startup method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can improve disk read / write efficiency and speed up container startup and application startup.

[0005] According to one aspect of the embodiments of this application, an application startup method is provided. The method is applied to an application system, which includes an application management server and an application server. The application server includes at least two non-uniform memory access NUMA nodes, each NUMA node having its own storage resources, and each NUMA node having at least one container. The method is executed by the application management server and includes:

[0006] Receive an application launch request for a target application sent by a user terminal. The application launch request includes the application identifier of the target application.

[0007] Based on the application identifier, the target container corresponding to the application startup request is determined;

[0008] Send an application start request to the target NUMA node to which the target container belongs, so that the target NUMA node controls the target container to start the target application based on the target application's target container image file. The target container image file is read by the target NUMA node from the storage resources corresponding to the target NUMA node, and the target container image file includes the first image file for running the target application.

[0009] According to another aspect of the embodiments of this application, an application launching device is provided. This device is applied to an application system, which includes an application management server and an application server. The application server includes at least two non-uniform memory access NUMA nodes, each NUMA node having its own storage resources, and each NUMA node having at least one container. The device is implemented through the application management server and includes:

[0010] The application launch request receiving module is used to receive an application launch request for a target application sent by a user terminal. The application launch request includes the application identifier of the target application.

[0011] The target container determination module is used to determine the target container corresponding to the application startup request based on the application identifier;

[0012] The target application startup module is used to send an application startup request to the target NUMA node to which the target container belongs, so that the target NUMA node controls the target container to start the target application based on the target container image file of the target application. The target container image file is read by the target NUMA node from the storage resources corresponding to the target NUMA node, and the target container image file includes the first image file for running the target application.

[0013] According to another aspect of the present application, an electronic device is provided, the electronic device including a memory, a processor and a computer program stored in the memory, the processor executing the computer program to implement the steps of the above method.

[0014] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-described method.

[0015] According to another aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method.

[0016] The beneficial effects of the technical solutions provided in this application are:

[0017] This application provides an application startup method, apparatus, electronic device, computer-readable storage medium, and computer program product. The application startup method provided in this application is applied to an application system including an application management server and an application server. The application server includes at least two NUMA nodes, each NUMA node having at least one container created within it, and each NUMA node having its own corresponding storage resources. This allows the container image file corresponding to the container created by each NUMA node to be read from the storage resources corresponding to that NUMA node, eliminating the need to read the corresponding container image file from other NUMA nodes. This reduces the time overhead of reading data across NUMA nodes and accelerates data reading speed.

[0018] By receiving application startup requests for a target application sent by a user terminal, the target container corresponding to the application startup request can be determined based on the application identifier of the target application in the application startup request. The application startup request is then sent to the target NUMA node to which the target container belongs. This allows the target NUMA node to control the target container to start the target application based on the target container image file corresponding to the target container, which is directly read from its corresponding storage resources. This can speed up the startup of the target application and avoid problems such as slow data reading speed caused by reading data across NUMA nodes, which in turn slows down the application startup speed.

[0019] Furthermore, by setting up at least two NUMA nodes, at least two target applications in the application startup request can be started independently and in parallel by each NUMA node, further accelerating the application startup speed. This avoids the high load caused by frequent scheduling to obtain target container image files when starting at least two target applications in the application startup request sequentially through a single NUMA node, and reduces the waiting time required to start other target applications when a single NUMA node is processing any one of the at least two target applications. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0021] Figure 1 A schematic diagram of the cloud gaming startup process provided in an embodiment of this application is shown;

[0022] Figure 2 A schematic diagram of a dual-C dual-NUMA architecture in related technologies is shown;

[0023] Figure 3 It shows the basis Figure 2 The diagram shown illustrates an application startup system implemented using a dual-C dual-NUMA architecture.

[0024] Figure 4 A schematic diagram of the dual-C dual-NUMA architecture provided in an embodiment of this application is shown;

[0025] Figure 5 This illustration shows a schematic diagram of an application startup system based on a multi-C, multi-NUMA architecture provided in an embodiment of this application.

[0026] Figure 6 A schematic diagram of the initial container creation process provided in an embodiment of this application is shown;

[0027] Figure 7 A schematic diagram of a container startup process based on an embodiment of this application is shown;

[0028] Figure 8 A flowchart of an application startup method provided in an embodiment of this application is shown;

[0029] Figure 9 A flowchart of an application startup system provided in an embodiment of this application is shown;

[0030] Figure 10 A schematic diagram of an application startup process provided in an embodiment of this application is shown;

[0031] Figure 11 A schematic diagram of another application startup process provided in an embodiment of this application is shown;

[0032] Figure 12 This illustration shows a schematic diagram of another application startup system based on a multi-C multi-NUMA architecture provided in an embodiment of this application;

[0033] Figure 13 A schematic diagram illustrating the process of launching a cloud game, as shown in an embodiment of this application, is presented.

[0034] Figure 14 A schematic diagram of the application launch device provided in an embodiment of this application is shown;

[0035] Figure 15 A schematic diagram of the structure of an electronic device to which this application embodiment applies is shown. Detailed Implementation

[0036] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0037] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” can be implemented as “A,” or as “B,” or as “A and B.”

[0038] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0039] The technical terms used in the embodiments of this application are explained as follows:

[0040] Cloud gaming (Game Streaming, GS): A brand-new cloud gaming platform built on container and technology, system and kernel technologies, and combined with audio and video technologies, network optimization, computing resource management, and other aspects, with a significant investment of technical resources.

[0041] SMP (Symmetrical Multi-Processing): In an SMP system, there are two or more CPUs (Central Processing Units), and each CPU is connected through a shared interconnect bus (i.e., I / O bus), which provides each CPU with equal access to system resources (such as memory and input / output ports) through the shared interconnect bus.

[0042] NUMA (Non-Uniform Memory Access Architecture) allows each CPU to access its own local memory. NUMA is a derivative of SMP (Single-Socket Multiprocessing) design and is found in multi-socket systems. In a NUMA system, system memory is divided into units or nodes associated with specific CPUs, known as NUMA units or nodes. Memory can be requested from other nodes via a shared interconnect bus. However, due to the limited bandwidth of the shared interconnect bus, resource contention can lead to performance degradation for individual NUMA nodes in the NUMA system.

[0043] In OpenStack, SMP CPUs are called cores, and NUMA units or nodes are called sockets. SMT CPUs are called threads. For example, in a quad-socket eight-core system with hyper-threading, there are four sockets, each with eight cores, and each core has two threads, for a total of 64 CPUs.

[0044] NUMA affinity: The program should, as far as possible, ensure that the running CPU and the memory used belong to the same NUMA node.

[0045] NVME (Non-Volatile Memory express): A protocol similar to AHCI (Advanced Host Controller Interface) built on the M.2 interface (M2 interface, an interface specification), and is a protocol specifically designed for flash memory storage.

[0046] Kubernetes (a container orchestration engine, or K8S for short): a portable and scalable open-source platform for managing containerized workloads and services, which facilitates declarative configuration and automation. Essentially, K8S is a resource orchestration tool that includes compute, storage, and network resources.

[0047] kubelet: A proxy component on Kubernetes worker nodes, running on each node.

[0048] Containerd: An industry-standard container runtime responsible for managing the lifecycle of images and containers. It emphasizes simplicity, robustness, and portability. Containerd can manage the complete container lifecycle within the host machine: container image transfer and storage, container execution and management, storage, and networking. In more detail, containerd is primarily responsible for: managing the container lifecycle (from container creation to destruction), pulling / pushing container images, storage management (managing the storage of images and container data), calling runC to run containers (interacting with container runtimes such as runC), and managing container network interfaces and networks. Containerd is designed to be embedded in a larger system, rather than being directly used by developers or end users.

[0049] gRPC: A high-performance, general-purpose open-source RPC framework used in Kubernetes for communication between various components. RPC (remote procedure call) identifies a request for a service from a program on a remote computer over a network, without needing to understand the underlying network protocol. It allows a program running on one computer to call a subroutine on another computer without having to code the interaction.

[0050] CRI (Container Runtime Interface): Defines the interface for services such as containers and images to provide computing resources. Specifically, it defines several gRPC APIs (application interfaces) and corresponding message types for managing workloads, such as operations like "pull image" (ImageService.PullImage), "create pod" (RuntimeService.RunPodSandbox), "create container" (RuntimeService.CreateContainer), "start container" (RuntimeService.StartContainer), and "stop container" (RuntimeService.StopContainer).

[0051] The CRI plugin is a built-in plugin for containerd, comprising two main parts: an image service and a runtime service. When a project or service is deployed within containerd, it first schedules to a specific machine. The Kubernetes service on that machine then queries which images the project or service needs, has the runtime service fetch the relevant images, and then starts the corresponding pod or container. Secondly, it interacts with CRI. It's important to note that during this interaction, a paused container (sandbox container) may be created in advance—this is a placeholder process.

[0052] CNI (Container Network Interface): Defines the service interface for container networks to provide network resources.

[0053] CSI (Container Storage Interface): Defines the service interface for container storage to provide storage resources.

[0054] Pod: The smallest unit of management and scheduling in a Kubernetes cluster. A pod can encapsulate one or more containers. By default, pods are scheduled to run on NUMA nodes. A pod can run independently as the smallest application instance, facilitating deployment, horizontal scaling and shrinking, and convenient scheduling management and resource allocation. Containers within a pod share the same data and network namespace, allowing for unified resource management and allocation across different pods. Containers operate on a single-process model, while pods operate on a multi-process model.

[0055] Cgroups (control groups): A mechanism provided by the Linux kernel (an operating system) that can limit, record, and isolate the physical resources (such as CPU, memory, I / O, etc.) used by process groups.

[0056] IOPS (Input / Output Operations Per Second): A metric used to test the performance of computer storage devices such as hard disk drives (HDDs), solid state drives (SSDs), or storage area networks (SANs).

[0057] The READY event typically occurs when the DOM (Document Object Model) has been loaded and the page (including images) has been fully rendered.

[0058] Cold boot: Starting a program while the power is off requires clearing all system data (such as memory) and reloading the program.

[0059] Warm start: Starts a program without interrupting power, without clearing system data, and runs the program directly.

[0060] A daemon process is a continuously running server program, also known as a daemon process. It typically runs in the system background, has no controlling terminal, and does not interact with the foreground. Daemon processes are generally used as system services, waiting for client programs to communicate with them. Daemon processes are long-running processes, usually starting after system startup and ending when the system shuts down.

[0061] The following section explains the process of kubelet calling the CRI interface to create a Pod, using the terminology mentioned above. It mainly includes three steps:

[0062] 1. Creating a PodSandbox: The corresponding CRI interface is RunPodSandbox. This interface is used to create a PodSandbox, which is the sandbox environment itself for the Kubernetes Pod. A pause container will run within the Pod by default to "hold" the network address space for configuring the Pod's network. The implementation of the pod sandbox varies depending on the container runtime used. For example, when using Kata as the runtime, the pod sandbox is implemented as a virtual machine; while when using Runc as the runtime, the pod sandbox is simply an independent namespace and cgroups.

[0063] 2. Creating a PodContainer: The corresponding CRI interface is CreatePodContainer. This interface allows you to create the PodContainer that you want to run, such as an nginx container. The created PodContainer will be added to the PodSanbox and will share the pod's network address space with other containers in the PodSanbox.

[0064] 3. Start the PodContainer: The corresponding CRI interface is StartPodContainer. Through this interface, you can start the container created in the previous step, namely the PodContainer.

[0065] In the application launch method provided in this application embodiment, the target application can be any application. For example, the target application can be any local application or any cloud application. When the target application is a cloud application, it can specifically include, but is not limited to, cloud gaming, online office software, cloud storage, email, map navigation, cloud music, cloud video, weight loss and fitness, financial management software, e-commerce, and search engines. Virtualization technology is one of the technological foundations of cloud applications. Cloud applications migrate resources that were originally on the client side to the cloud. In order to manage application resources more flexibly and efficiently, the cloud needs to realize the transformation from physical to virtual in the cloud application runtime environment. Container technology is a lightweight virtualization technology that is suitable for cloud applications.

[0066] Cloud technology refers to a managed technology that unifies hardware, software, network, and other resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. The application startup method provided in this application embodiment can be implemented based on cloud computing.

[0067] Cloud computing refers to obtaining the required resources in an on-demand and easily scalable manner through the network. It is the product of the development and integration of traditional computer and network technologies such as grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing.

[0068] Cloud gaming, also known as gaming on demand, is an online gaming technology based on cloud computing. It enables thin clients with relatively limited graphics processing and data processing capabilities to run high-quality games. In cloud gaming, the game logic does not reside on the player's terminal but runs on a cloud server. The cloud server renders the game scene as video and audio streams, which are then transmitted to the player's terminal via the network. The player's terminal does not need powerful graphics processing and data processing capabilities; it only needs basic streaming media playback capabilities and the ability to receive player input commands and send them to the cloud server.

[0069] The target application provided in this application embodiment can be an application applied to any operating system. This application embodiment does not limit the specific system type of the operating system. For example, it can be, but is not limited to, the Android system. For ease of description, the following embodiment description will take the Android system as an example.

[0070] To better understand the application startup method provided in this application embodiment, the process of application startup in related technologies will be explained below in conjunction with a specific application scenario.

[0071] like Figure 1As shown, taking launching a cloud gaming application (hereinafter referred to as cloud gaming) in an Android system as an example, the cloud gaming launch process can include three steps: launching the Android container (when the operating system of the user terminal corresponding to the game player is Android, the corresponding container can be called an Android container), launching the Android system, and launching the game within the container. In this application scenario, the game launch time mainly includes three aspects: container launch time, operating system launch time, and game launch time within the container. During the cloud gaming launch process, the Android container launch process, the Android system launch process, and the game launch process within the container all require disk read and write operations, that is, corresponding launch operations are performed based on the resources in the disk.

[0072] Taking the Ampere model with a dual-CPU dual-NUMA architecture (i.e., two CPUs, two NUMA nodes, with each NUMA node corresponding to one CPU) as an example, such as Figure 2 As shown, in this dual-C dual-NUMA architecture, the NUMA0 node corresponds to one pod (i.e., a collection of multiple pods), CPU0, memory0, and disk0, while the NUMA1 node corresponds to one pod, CPU1, and memory1. Each pod includes one or more containers, and one container corresponds to one cloud gaming session. It is evident that in this architecture, there is only one disk, disk0, located within the NUMA0 node. The NUMA0 node and the NUMA1 node share the storage resources on disk0.

[0073] Managing a container typically involves processes such as create, start, stop, remove, and execute for both the pod containing the container and the container itself. These management processes can be categorized into two types: initial container creation and other operations. When a container is created, a container ID is generated. After creation, operations like start, stop, remove, and execute can be performed using this pod ID or container ID to control the corresponding operations within the container.

[0074] Considering that container management is primarily handled by a daemon process strongly associated with container management, namely containerd (a container process corresponding to a NUMA node; since containerd operates within a NUMA node, operations performed by containerd can be considered as operations performed on its corresponding NUMA node), it manages the downloading, use, deletion, and updating of container image files—activities spanning the entire container lifecycle. Each containerd can manage one or more containers. During the container image download process, it needs to download the container image files to the root directory (hereinafter referred to as the image management directory) in the default configuration file corresponding to containerd (e.g., / etc / containerd / config.toml). It also stores the mapping relationships between each container image file, its corresponding container, and the specific applications implemented based on each container. When it's necessary to use a corresponding container image file within a container, the containerd instance needs to query and retrieve the container image file for the container to be managed from its corresponding image management directory. The retrieved container image file is then loaded into the containerd instance, and the container is controlled to perform the operations corresponding to that container image file. It's important to note that each containerd instance can only be configured with one disk location as its image management directory; different containers cannot use image management directories on different disks.

[0075] The native kubelet can only communicate with one containerd instance. Figure 3 It shows the basis Figure 2 The diagram illustrates an application startup system implemented using a dual-C, dual-NUMA architecture. Figure 3 As shown, gRPC requests such as container creation and deletion can be sent to the corresponding containerd via CRI to control the corresponding containerd to perform the operations corresponding to the gRPC requests.

[0076] exist Figure 3 In the system shown, kubelet calls the CRI plugin through the CRI interface, and controls containerd to operate on the corresponding containers through function calls. When containerd runs as a Kubernetes container, its complete default configuration can be directly queried using the `containerd config default` command.

[0077] like Figure 3 As shown, taking the creation and startup of a cloud game in container A via kubelet as an example, container A corresponds to... Figure 2 The specific working process of the NUMA0 node is as follows:

[0078] 1. The kubelet calls the runtime service API (RuntimeService.RunPodSandbox) in the CRI Plugin through the CRI interface to create pod A (i.e., Figure 2 (A pod in the pods of the NUMA0 node);

[0079] 2. The CRI Plugin uses containerd to create and start a specific pause container (i.e., Figure 3 The sandbox container in pod A is placed in pod A's Cgroups and network address space;

[0080] 3. The CRI Plugin uses ocicni to configure the network address space of pod A via CNI;

[0081] 4. The kubelet uses the CRI interface to call the image service API (ImageService.PullImage) and containerd client API in the CRI Plugin to pull the application container image from disk 0;

[0082] 5. If the application container image is not on this node, the CRI Plugin uses containerd to pull the application container image;

[0083] 6. The kubelet calls the runtime service API (RuntimeService.CreateContainer) and containerd client API in the CRI Plugin through the CRI interface. Inside pod A, it uses the pulled application container image to create container A through the containerd shim and puts it into pod A's Cgroups and namespace.

[0084] 7. The kubelet calls the runtime service API (RuntimeService.StartContainer) in the CRI Plugin and the containerd client API through the CRI interface. Inside pod A, it uses the pulled application container image to start container A and start the cloud game in container A.

[0085] Based on Figure 3 When the system shown starts a cloud game, if disk reads are required, for container processes running on CPU0 corresponding to the NUMA0 node, since the NUMA0 node has disk 0, read and write operations can be performed directly from disk 0. For container processes running on CPU1 corresponding to the NUMA1 node, since the NUMA1 node does not have a corresponding disk, read and write operations need to be performed on disk 0 corresponding to the NUMA0 node. The application container image is pulled from disk 0 to start the corresponding container and application, which requires disk read and write operations across NUMA nodes (hereinafter referred to as cross-NUMA).

[0086] Since disk 0 corresponds to NUMA 0, the disk cache (also known as disk buffer) is directly related to I / O performance. This means that for containers running on CPU1 corresponding to NUMA 1 nodes, there will be situations where I / O reads and writes occur across NUMA nodes. Therefore, in a dual-CPU dual-NUMA architecture system, although CPU NUMA affinity can be guaranteed through cgroups or core binding (i.e., CPU operation does not need to cross NUMA nodes), I / O NUMA affinity cannot be guaranteed.

[0087] Since disk access is typically performed via the NVMe protocol, taking a random 4k mixed read / write disk as an example, the following metrics are used to evaluate the performance of disk access via NVMe across NUMA, with average read IOPS, average read bandwidth, average write IOPS, and average write bandwidth as evaluation indicators:

[0088]

[0089] Based on the experimental data above, compared to NVME accessing disks without crossing NUMA, the performance is significantly reduced when accessing disks via NVME across NUMA, by approximately 45%.

[0090] Considering the disk access time mentioned above, when launching cloud games, especially when loading large amounts of game image data from disk into memory, the READY event latency is significantly longer when cross-NUMA disk access is required compared to NVMe access without crossing NUMA, resulting in slower game launch speeds and impacting user experience. Therefore, the time overhead required for cross-NUMA disk access is a crucial factor affecting container and game launch speeds. Reducing the time overhead of cross-NUMA disk access is essential for improving both container and game launch speeds.

[0091] To address the aforementioned technical problem of reducing the time overhead required for cross-NUMA disk access, the application startup method provided in this application embodiment is illustrated below using a possible example. This method can be implemented based on a multi-CPU, multi-NUMA architecture, specifically, the number of CPUs corresponds one-to-one with the number of NUMA nodes. The following description uses a system based on a dual-CPU, dual-NUMA architecture as an example to illustrate the application startup method provided in this application embodiment.

[0092] Figure 4 A schematic diagram of the dual-C dual-NUMA architecture provided in an embodiment of this application is shown. Figure 4 As shown, in this dual-C dual-NUMA architecture, a NUMA0 node corresponds to one pod, CPU0, memory0, and disk0, while a NUMA1 node corresponds to one pod, CPU1, memory1, and disk1. Each pod includes one or more containers, and each container corresponds to one cloud gaming session. When disk reads are required, the container process running on the CPU corresponding to each NUMA node can directly perform read and write operations on the disk corresponding to that NUMA node.

[0093] Figure 5 A schematic diagram of an application startup system based on a multi-C, multi-NUMA architecture, provided in an embodiment of this application, is shown. Figure 5 As shown, this application startup system is implemented based on the Kubernetes platform. Within this system, the middleware component nesoi can establish a connection between the kubelet and multiple NUMA nodes, i.e., through... Figure 5The system shown can establish a connection between kubelet and multiple containerd instances. Each containerd instance corresponds to a NUMA node, and in its default configuration file / etc / containerd / config.toml, each containerd instance sets its image management directory to the disk (i.e., the disk with NUMA affinity) bound to its corresponding NUMA node.

[0094] The following explanation uses two containerd instances (containerd0 and containerd1) configured in this K8S platform as examples. When accessing disk 0 bound to NUMA0 node, the protocol used is nvme0, and when accessing disk 1 bound to NUMA1 node, the protocol used is nvme1. containerd0 uses / data0 / containerd as its image management directory, and containerd1 uses / data1 / containerd as its image management directory.

[0095] like Figure 5 As shown, nesoi is essentially a middleware component that implements the CRI, CNI, and CSI of kubelet. It is responsible for the routing of gRPC messages (i.e., selecting a containerd according to the demand), forwarding (forwarding the gRPC message stream to the selected containerd), and congestion control (limiting the request rate per unit time).

[0096] nesoi primarily serves two layers of masquerading. For kubelet, nesoi acts as a disguised containerd exposing the ` / var / run / containerd.sock` port. Communication between nesoi and kubelet occurs through the `container-runtime-endpoint` or `image-service-endpoint` socket specified by kubelet. For each containerd, nesoi acts as a disguised kubelet issuing CRI requests. Communication between nesoi and each containerd (containerd in CPU0 nvme0 and containerd in CPU1 nvme1) occurs through the `container-runtime-endpoint` or `image-service-endpoint` specified in the containerd's `config.toml` configuration file—i.e., independent and different sockets. Figure 5 The files generated correspond to / run / containerd / containerd0.sock for containerd0 and / run / containerd / containerd1.sock for containerd1, respectively.

[0097] like Figure 5 As shown, since there are multiple containerd instances in the system, when a container is initially created through this system, a containerd is needed as the acceptor and manager of the newly created container. That is, how to select a containerd that conforms to IO NUMA affinity to accommodate and manage the newly created container.

[0098] Figure 6 A schematic diagram illustrating the initial container creation process provided in an embodiment of this application is shown. For example... Figure 6As shown, the process of kubelet creating containers via nesoi can be divided into three main steps: pod scheduling, pod creation, and container creation. During pod scheduling, resource allocation is achieved by adding an annotation to the pod to record the GPU it uses. During pod creation, the GPU information (e.g., / sys / devices / pci0000\:00 / 0000\:00\:01.0 / numa_node) is extracted from the host machine (i.e., the application management server where the system resides) using the GPU information recorded in the pod's annotation. Based on this GPU information, the NUMA node number (i.e., the NUMA node identifier, such as 0 / numa_node indicating NUMA number 0) of the pod is obtained. Based on this NUMA number, the first digit of both the pod ID (ID, i.e., identification) randomly generated via the CRI interface call to the runtime service API (RuntimeService.RunPodSandbox) and the first digit of the container ID randomly generated via the CRI interface call to the runtime service API are modified to this NUMA number. The modified pod ID is then used as the ID of the created pod, and the modified container ID is used as the ID of the created container. This facilitates the selection of the corresponding pod and container using nesoi based on the pod ID and container ID, enabling the management of pods and containers. Furthermore, when modifying the randomly generated container ID, modifications can also be made based on the identification information in the relevant positions within the pod ID.

[0099] Since randomly generated pod IDs and container IDs are generally 128 bits, when modifying the randomly generated pod ID and container ID based on the NUMA number, only one bit of the 128 bits is modified. The probability of it being the same as other randomly generated pod IDs and container IDs is 1 / 2^128, which is extremely small. It can be considered that the current pod ID will not be the same as other randomly generated pod IDs, and the current container ID will not be the same as other randomly generated container IDs.

[0100] After a container is created, when operations such as start, stop, remove, and exec need to be performed on it, nesoi can receive operation requests from the kubelet based on the socket of the container-runtime-endpoint or image-service-endpoint specified by the kubelet. It then converts the operation request into an operation command based on the first digit of the pod ID or container ID in the request. Finally, it selects the containerd daemon corresponding to the first digit of the pod ID or container ID and routes the operation command to the selected containerd daemon for processing. The specific processing logic is the same as for a single containerd.

[0101] It should be noted that when nesoi receives an image pull request from kubelet based on the socket of the image-service-endpoint specified by kubelet, nesoi will convert the image pull request into an image pull instruction and forward the image pull instruction to each containerd through the gRPC socket of each containerd1. This notifies all containersd to pull the container image file at the same time and save it to the image management directory on the disk configured for each containerd. The image pull instruction is considered to have been executed successfully after all containersd have completed downloading the container image file.

[0102] Taking container startup as an example, Figure 7 A schematic diagram of a container startup process based on an embodiment of this application is shown. Figure 7As shown, when nesoi receives a startup request for container a from kubelet based on the container-runtime-endpoint specified by kubelet, it extracts the first bit of container a's ID, i.e., 1, to determine that container a corresponds to NUMA1, i.e., NUMA node 1. It further determines the gRPC socket of containerd1 bound to NUMA1, i.e., / var / run / containerd1 / containerd.sock. After converting the startup request into a startup command, it forwards the startup request to containerd1 via / var / run / containerd1 / containerd.sock. This controls containerd1 to load the container image file corresponding to container a from its corresponding image management directory and starts container a based on the loaded container image file. This achieves NUMA isolation for I / O, eliminating the need to read container image files from disk across NUMA nodes. Furthermore, when it is necessary to start cloud game A corresponding to container a, after starting container a, the corresponding cloud game A can be started within container a based on the read container image file.

[0103] As can be seen, when starting a container or game using the application startup method provided in this application embodiment, the container image file can be loaded from the corresponding image management directory, thereby achieving NUMA isolation for I / O, avoiding disk read time overhead caused by cross-NUMA, and speeding up the startup speed of multi-way cloud gaming containers and games in each cloud gaming container. Furthermore, experiments show that compared to application startup methods in related technologies, the application startup method provided in this application embodiment can improve application startup time by more than 50%.

[0104] The technical solutions of this application and their effects are described below through several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0105] Figure 8 A flowchart of an application startup method provided in an embodiment of this application is shown. This method is applied to... Figure 9 In the application system shown, the application system 10 includes an application management server 110 and an application server 120. The application server includes at least two non-uniform memory access NUMA nodes 121. Each NUMA node has its own storage resources. At least one container is created in each NUMA node. The method is executed by the application management server.

[0106] Among them, the application management server or application server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server or server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0107] Those skilled in the art should understand that the above-described server is merely an example, and other existing or future servers that are applicable to this application should also be included within the scope of protection of this application, and are hereby incorporated by reference.

[0108] In this embodiment of the application, the mapping relationship between the node identifier of each NUMA node and the resource identifier of the storage resource corresponding to each NUMA node, as well as the mapping relationship between the container identifier of each container and its corresponding application identifier, can be stored in the storage resources of the application server in advance.

[0109] In this application embodiment, the specific forms of node identifiers, resource identifiers, container identifiers, and application identifiers are not limited, and can be configured according to actual application needs. For example, for application identifiers, they can be set based on, but are not limited to, the name of the application corresponding to the application identifier, numeric identifiers, alphabetic identifiers, or a combination of two or more of these.

[0110] It should be noted that the node identifier of the NUMA node to which the container identifier belongs can also be set in the container identifier. For example, when the container identifier is in numerical form, one or more bits of the container identifier can be set as the node identifier of the NUMA node to which the container belongs. Thus, the NUMA node to which the container identifier belongs can be determined based on any container identifier.

[0111] By running the container image file corresponding to each container in each container, the application corresponding to that container can be launched. In this application system, the applications corresponding to each container can be the same or different; this application embodiment does not impose any restrictions on this. Of course, even if any two containers correspond to the same application, their corresponding operating systems or application operators can be different. For example, for the same cloud game, when the corresponding game players are different, the cloud game is launched on the player's game terminal corresponding to each game player through different containers in this application system.

[0112] This application system can provide, in its specific implementation, the following: Figure 5The application startup system shown corresponds to Kubernetes (K8S) and kubelet as the application management server, and each containerd instance corresponds to an application server. When the application server includes two NUMA nodes, the corresponding architecture can be as follows: Figure 4 As shown, each NUMA node corresponds to one Figure 4 The image shows a disk (i.e., storage resource). The operations performed on each NUMA node are primarily executed by the process within that NUMA node that controls container operations (e.g., the containerd process).

[0113] Those skilled in the art should understand that the containerd process in the embodiments of this application is merely an example of a process used to control the execution of container operations in a NUMA node. Other existing or future processes used to control the execution of container operations in a NUMA node that are applicable to this application should also be included within the scope of protection of this application, and are hereby incorporated by reference.

[0114] It should be noted that when the application server includes three or more NUMA nodes, it can be based on... Figure 4 The illustrated construction method for NUMA0 or NUMA1 nodes is used to build new NUMA nodes, forming a NUMA architecture. This can be based on... Figure 5 The construction methods shown for containerd0 or containerd1 construct a new containerd and establish a connection with nesoi to form a new application system. This application embodiment does not impose restrictions on the specific NUMA architecture, the number of NUMA nodes in the new application system, or the number of containersd; these can be selected according to actual circumstances.

[0115] like Figure 8 As shown, the method includes the following steps S810 to S830:

[0116] Step S810: Receive an application launch request for the target application sent by the user terminal. The application launch request includes the application identifier of the target application.

[0117] The user terminal can be the terminal corresponding to the target application. This terminal (also called a user terminal or user equipment (UE)) can be a smartphone, tablet, laptop, desktop computer, intelligent voice interaction device (such as a smart speaker), wearable electronic device (such as a smartwatch), in-vehicle terminal, smart home appliance (such as a smart TV), AR (Augmented Reality) / VR (Virtual Reality) device, etc.

[0118] Those skilled in the art should understand that the above-described terminals are merely examples, and other existing or future terminals that are applicable to this application should also be included within the scope of protection of this application, and are hereby incorporated by reference.

[0119] Step S820: Based on the application identifier, determine the target container corresponding to the application startup request.

[0120] Specifically, based on the mapping relationship between each container identifier and its corresponding application identifier stored in the application server's storage resources, the container corresponding to the application identifier of the target application can be determined, and the determined container can be identified as the target container corresponding to the application startup request.

[0121] Of course, if it is determined that the application server's storage resources do not include the target container corresponding to the application identifier, the target container corresponding to the application identifier can be created based on the above container creation method to further execute the subsequent target application startup process.

[0122] Step S830: Send an application startup request to the target NUMA node to which the target container belongs, so that the target NUMA node controls the target container to start the target application based on the target container image file of the target application. The target container image file is read by the target NUMA node from the storage resources corresponding to the target NUMA node, and the target container image file includes a first image file for running the target application.

[0123] In this example, the target NUMA node to which the target container belongs can be determined by a preset position in the target container's identifier (i.e., one or more digits in the container identifier). Alternatively, the target NUMA node to which the target container belongs can be determined based on the mapping relationship between the node identifier of each NUMA node and the container identifiers of the containers it creates.

[0124] For any container image file, it may include an image file used to start the container corresponding to the container image file, and an image file for running the corresponding application in the container corresponding to the container image file.

[0125] The target container image file can include a first image file for running the target application and a second image file for starting the target container. After the target NUMA node receives the application startup request, if the target container is already running, it directly controls the target container to start the target application based on the first image file. If the target container is not running, it can be controlled to first start the target container based on the second image file, and then start the target application based on the first image file.

[0126] In this application startup method, since each NUMA node has its own storage resources, when it is necessary to read storage resources, the container image file corresponding to the container created by that NUMA node can be read from the storage resources corresponding to that NUMA node. It is not necessary to read the corresponding container image file from other NUMA nodes, which reduces the time overhead of reading data across NUMA nodes and speeds up the data reading speed.

[0127] After receiving an application startup request, the target container corresponding to the application startup request can be determined based on the application identifier of the target application in the application startup request. The application startup request is then sent to the target NUMA node to which the target container belongs. This allows the target NUMA node to directly read the target container image file corresponding to the target container created by the target NUMA node from its corresponding storage resources. Based on the first image file used to run the target application in the target container image file, the target container is controlled to start the target application. This can speed up the startup of the target application and avoid problems such as slow data reading speed caused by reading data across NUMA nodes, which in turn slows down the subsequent application startup speed.

[0128] Furthermore, by setting up at least two NUMA nodes, at least two target applications in the application startup request can be started independently and in parallel by each NUMA node, further accelerating the application startup speed. This avoids the problem of high load on a single NUMA node caused by frequent scheduling to obtain the target container image file when starting at least two target applications in the application startup request sequentially through a single NUMA node, and reduces the waiting time required to start other target applications when a single NUMA node is processing any one of the at least two target applications.

[0129] When the target application is a cloud gaming application, the specific application launch method can be as follows:

[0130] In one possible implementation, the target application is a cloud gaming application, the application system is a cloud gaming scheduling backend, the application server is a cloud gaming server, and the target container is a container of the operating system corresponding to the cloud gaming application.

[0131] The aforementioned receiving of an application launch request for the target application sent by the user terminal includes:

[0132] Receive an application launch request for the cloud gaming application sent by the user terminal. The application launch request includes the application identifier of the cloud gaming application and the target operating system corresponding to the user terminal.

[0133] The above-mentioned sending of an application startup request to the target NUMA node to which the target container belongs, so that the target NUMA node controls the target container to start the target application based on the target application's target container image file, includes:

[0134] Send an application launch request for the cloud gaming application to the target NUMA node to which the target container belongs, so that the target NUMA node can launch the target operating system and the cloud gaming application based on the container image file of the cloud gaming application.

[0135] When the target application is a cloud gaming application, the user terminal can be any terminal held by a player of the cloud gaming application. Of course, the user terminal can also be a terminal held by a tester testing the cloud gaming application. This application embodiment does not impose any restrictions on this.

[0136] By identifying the application identifier of the cloud game, the target container corresponding to the cloud game application is determined, and the operating system corresponding to the user terminal is launched in the target container, thereby launching the cloud game application in the container, which speeds up the startup speed of the cloud game application.

[0137] In one possible implementation, the method may further include, before receiving the application launch request:

[0138] In the application server's storage resources, determine the storage resources corresponding to each NUMA node, and store the binding relationship between each NUMA node and its corresponding storage resources;

[0139] Create at least one container on each NUMA node;

[0140] An image file retrieval instruction is sent to each NUMA node so that each NUMA node retrieves at least one container image file corresponding to an application according to the image file retrieval instruction, and stores the retrieved container image file in the storage resource corresponding to each NUMA node. Here, at least one application corresponding to at least one NUMA node includes the target application, and the container image file corresponding to at least one application includes the target container image file.

[0141] By identifying the storage resources corresponding to each NUMA node in the application server's storage resources and storing the binding relationship between each NUMA node and its corresponding storage resources, the application management server can easily build corresponding pods or containers through each NUMA node when creating each NUMA node, and create the corresponding storage resources and pods to that NUMA node.

[0142] It should be noted that after each NUMA node receives the image file retrieval instruction, it can retrieve the container image files corresponding to all applications, and based on the container creation instruction, select the corresponding container image file according to each container identifier in the container creation instruction, and create the container corresponding to each container identifier. Alternatively, it can retrieve only the container image files corresponding to some applications, and create the corresponding containers based on the retrieved container image files. This application embodiment does not impose any limitations on this.

[0143] After each NUMA node has created its corresponding container, the mapping relationship between the node identifier of each NUMA node, the container identifier of each container created by each NUMA node, and the application identifier corresponding to each container can be fed back to the application management server. This allows the application management server to determine the container corresponding to the application based on the operation request corresponding to the application identifier of any application, and then send the operation request to the NUMA node corresponding to the determined container, so as to control the NUMA node to perform the operation corresponding to the operation request on the corresponding container.

[0144] As described above, the target NUMA node can start the target container either after receiving the application startup request or before receiving it. The following example illustrates the application startup process as starting the target container after receiving the application startup request. Since this target container starts after receiving the application startup request, it can also be called a cold start. The specific application startup process is as follows:

[0145] In one possible implementation, sending an application startup request to the target NUMA node to which the target container belongs, so that the target NUMA node controls the target container to start the target application based on the target application's target container image file, may include:

[0146] Based on the application startup request, a container startup instruction is sent to the target NUMA node, so that the target NUMA node responds to the container startup instruction and reads the target container image file corresponding to the target application from the storage resources corresponding to the target NUMA node. The target container image file may also include a second image file for starting the target container.

[0147] Send the target container image file to the target container so that the target container can start the target container based on the first image file and start the target application based on the second image file.

[0148] Figure 10 A schematic diagram of an application startup process provided in an embodiment of this application is shown. Figure 10As shown, after receiving a message from a game player that they have started a cloud game on an Android system, the cloud game scheduling backend (i.e., the aforementioned application system) can retrieve the container image file corresponding to the cloud game, and then start the Android container, the Android system, and the cloud game in the Android container in sequence according to the container image file.

[0149] After receiving the application startup request, the target NUMA node is controlled to read the target container image file corresponding to the target application from the storage resources corresponding to the target NUMA node, and start the target container based on the first image file in the target container image file, and start the target application based on the second image file, thereby speeding up the startup speed of the target application.

[0150] The following example illustrates the application startup process by starting the target container after receiving the application startup request. Since the target container is started before receiving the application startup request, this can also be referred to as a hot start. The specific application startup process is as follows:

[0151] In one possible implementation, after each NUMA node stores the acquired container image file into its respective storage resource, the method may further include:

[0152] Send a container startup command for the target application to at least one NUMA node, so that the NUMA node receiving the container startup command performs the following operations:

[0153] Read the target container image file from the storage resources corresponding to the NUMA node;

[0154] Send the read target container image file to at least one container on the NUMA node so that the container that receives the target container image file starts the container based on the second image file;

[0155] The above method of determining the target container corresponding to the application startup request based on the application identifier may include:

[0156] Based on the application identifier, the target container is determined from the already started containers corresponding to the target application;

[0157] Specifically, the target NUMA node controls the target container to start the target application based on the target container image file of the target application, which may include: the target NUMA node sending an application start instruction to the target container so that the target container starts the target application based on the first image file.

[0158] After storing the acquired container image file in the corresponding storage resources of each NUMA node, a container startup command for the target application is sent to at least one NUMA node. The NUMA nodes are then controlled to read the target container image file and start the corresponding container. Based on the acquired application startup request, the target container is determined from the already started containers based on the application identifier in the application startup request. This achieves pre-starting of the target container before the application startup request is acquired. This reduces the time to start the target container after the application startup request is acquired, and the target application is started directly in the target container, reducing the waiting time for the user terminal to start the target application and improving the user experience.

[0159] Figure 11 A schematic diagram illustrating another application startup process provided in an embodiment of this application is shown. For example... Figure 11 As shown, the target container image files corresponding to 100 containers can be obtained through the pre-fetch scheduling backend. Any NUMA node can be selected to start 100 containers in batches. After receiving the game player's launch of cloud gaming on the Android system, the target container corresponding to the cloud game can be selected from the 100 launched containers through the cloud gaming scheduling backend. According to the target container image file, the Android system and cloud gaming are launched in the Android container in sequence, reducing the time for game players to wait for cloud gaming to start.

[0160] Alternatively, the target container image file corresponding to 100 containers can be obtained through the pre-fetch scheduling backend. Two NUMA nodes can be selected to start 50 containers in batches. After receiving the game player's launch of cloud gaming on the Android system, the target container corresponding to the cloud game can be selected from the 100 launched containers through the cloud gaming scheduling backend. The NUMA node corresponding to the target container can be controlled to start the Android system and cloud gaming in the Android container in sequence according to the target container image file.

[0161] When the application system is a Kubernetes-based system, considering that kubelet can only establish a communication relationship with one containerd, i.e., one NUMA node, in order to simultaneously control different NUMA nodes to perform corresponding operations on the corresponding containers, this application embodiment also provides the following optional implementation methods, which can be achieved through... Figure 5 The system implementation shown is as follows:

[0162] In one possible implementation, the application management server communicates with at least two NUMA nodes via middleware components. Sending the application startup request to the target NUMA node to which the target container belongs may include:

[0163] Based on the container identifier of the target container, perform the following operations through the middleware component:

[0164] Based on the container identifier of the target container, determine the target NUMA node to which the target container belongs;

[0165] Send an application startup request to the target NUMA node through the communication interface corresponding to the target NUMA node.

[0166] In this implementation, the application management server can be one of the aforementioned kubelets, the middleware component can be one of the aforementioned nesoi, and the communication interface corresponding to the target NUMA node can be the socket specified in the config.toml configuration file of containerd corresponding to the target NUMA node, such as the aforementioned container-runtime-endpoint or image-service-endpoint, specifically including / run / containerd / containerd0.sock and / run / containerd / containerd1.sock.

[0167] By using the middleware component to determine the target NUMA node to which the target container belongs based on the container identifier of the target container, and sending the application startup request to the target NUMA node through the communication interface corresponding to the target NUMA node, it is possible to control multiple NUMA nodes to execute the corresponding startup requests simultaneously through the application system.

[0168] In one possible implementation, the application management server communicates with the at least two NUMA nodes through a proxy component corresponding to each NUMA node. Sending the application startup request to the target NUMA node to which the target container belongs may include:

[0169] Based on the container identifier of the target container, determine the target NUMA node corresponding to the container identifier;

[0170] The application startup request is sent to the target NUMA node through the proxy component corresponding to the target NUMA node.

[0171] In this implementation, the application management server can be one of the aforementioned Kubernetes clusters, and each proxy component can be a kubelet. That is, a native kubelet controls a NUMA node, thereby enabling control operations on the containers corresponding to that NUMA node.

[0172] By using the container identifier of the target container, the application management server determines the target NUMA node corresponding to the container identifier, and sends application startup requests to each NUMA node through the proxy component corresponding to each NUMA node. This enables control operations on each container corresponding to the NUMA node, and thus allows the application system to simultaneously control multiple NUMA nodes to execute the corresponding startup requests.

[0173] Figure 12 This illustration shows a schematic diagram of another application startup system implemented based on a multi-C, multi-NUMA architecture, as provided in an embodiment of this application. For example... Figure 12 As shown, containerd0 can be directly controlled through kubelet0, and containerd1 can be directly controlled through kubelet1.

[0174] In one possible implementation, the method may further include:

[0175] Obtain container operation instructions, which include the container identifier of the container to be processed;

[0176] Based on the container identifier of the container to be processed, a container operation instruction is sent to the NUMA node to which the container to be processed belongs, so that the NUMA node that receives the container operation instruction can perform the operation corresponding to the container operation instruction on the container to be processed.

[0177] A container operation instruction may include operation instructions such as create, start, stop, remove, and execute for the pod containing the container and the container itself. This application embodiment does not limit this, and can be determined according to the actual situation.

[0178] By sending container operation instructions to the NUMA node to which the container to be processed belongs based on the container identifier in the container operation instructions, the NUMA node that receives the container operation instructions can execute the operation corresponding to the container operation instructions for the container to be processed, thereby realizing control over each container.

[0179] Figure 13 A schematic diagram illustrating the process of launching a cloud game, as shown in an embodiment of this application, is illustrated. Figure 13 As shown, the startup process may include the following steps:

[0180] Step 1: Bind a disk to each NUMA node in a kubelet;

[0181] Step 2: Launch containerd on each NUMA node and configure it to use the disk bound to the corresponding NUMA node as the root directory;

[0182] Step 3: Start a nesoi in a kubelet and use the nesoi as the communication routing and forwarding module for all containersd and kubelet.

[0183] Step 4: Pull the image through each containerd and save the downloaded image to its corresponding disk.

[0184] Step 5: Read the corresponding container image file on the disk for each containerd, start the corresponding container based on the read container image file, and start the cloud game in the corresponding container.

[0185] Based on the same principle as the application launch method provided in the embodiments of this application, the embodiments of this application also provide an application launch device. Figure 14 A schematic diagram of an application launch device provided in an embodiment of this application is shown. Figure 14 As shown, the device 400 is applied in an application system, which includes an application management server and an application server. The application server includes at least two non-uniform memory access NUMA nodes, each NUMA node having its own storage resources, and each NUMA node having at least one container. The device is implemented through the application management server and includes:

[0186] The application launch request receiving module 410 is used to receive an application launch request for a target application sent by a user terminal. The application launch request includes the application identifier of the target application.

[0187] The target container determination module 420 is used to determine the target container corresponding to the application startup request based on the application identifier;

[0188] The target application startup module 430 is used to send an application startup request to the target NUMA node to which the target container belongs, so that the target NUMA node controls the target container to start the target application based on the target container image file of the target application. The target container image file is read by the target NUMA node from the storage resources corresponding to the target NUMA node, and the target container image file includes a first image file for running the target application.

[0189] In one possible implementation, the device also includes a resource management module, a container creation module, and an image file acquisition module.

[0190] The resource management module is used to determine the corresponding storage resources for each NUMA node in the application server's storage resources before receiving the application startup request, and to store the binding relationship between each NUMA node and its corresponding storage resources.

[0191] The container creation module is used to create at least one container in each NUMA node;

[0192] The image file acquisition module is used to send an image file acquisition instruction to each NUMA node, so that each NUMA node can acquire at least one container image file corresponding to an application according to the image file acquisition instruction, and store the acquired container image file in the storage resource corresponding to each NUMA node. The at least one application corresponding to at least one NUMA node includes the target application, and the container image file corresponding to at least one application includes the target container image file.

[0193] In one possible implementation, the target application startup module 430 sends an application startup request to the target NUMA node to which the target container belongs, so that the target NUMA node controls the target container to start the target application based on the target container image file of the target application. Specifically, this can be used for:

[0194] Based on the application startup request, a container startup instruction is sent to the target NUMA node, so that the target NUMA node responds to the container startup instruction and reads the target container image file corresponding to the target application from the storage resources corresponding to the target NUMA node. The target container image file may also include a second image file for starting the target container.

[0195] Send the target container image file to the target container so that the target container can start the target container based on the first image file and start the target application based on the second image file.

[0196] In one possible implementation, after each NUMA node stores the acquired container image file into its respective storage resource, the above apparatus further includes a container pre-start module, which is used for:

[0197] Send a container startup command for the target application to at least one NUMA node, so that the NUMA node receiving the container startup command performs the following operations:

[0198] Read the target container image file from the storage resources corresponding to the NUMA node;

[0199] Send the read target container image file to at least one container on the NUMA node so that the container that receives the target container image file starts the container based on the second image file;

[0200] When determining the target container corresponding to the application startup request based on the application identifier, the aforementioned target container determination module 420 can be specifically used for:

[0201] Based on the application identifier, the target container is determined from the already started containers corresponding to the target application;

[0202] Specifically, the target NUMA node controls the target container to start the target application based on the target container image file of the target application, which may include: the target NUMA node sending an application start instruction to the target container so that the target container starts the target application based on the first image file.

[0203] In one possible implementation, the application management server communicates with at least two NUMA nodes via middleware components. When the target application startup module 430 sends an application startup request to the target NUMA node to which the target container belongs, it can specifically be used for:

[0204] Based on the container identifier of the target container, perform the following operations through the middleware component:

[0205] Based on the container identifier of the target container, determine the target NUMA node to which the target container belongs;

[0206] Send an application startup request to the target NUMA node through the communication interface corresponding to the target NUMA node.

[0207] In one possible implementation, the application management server communicates with the at least two NUMA nodes through a proxy component corresponding to each NUMA node. The target application startup module 430 sends an application startup request to the target NUMA node to which the target container belongs. The target application startup module 430 may then:

[0208] Based on the container identifier of the target container, determine the target NUMA node corresponding to the container identifier;

[0209] The application startup request is sent to the target NUMA node through the proxy component corresponding to the target NUMA node.

[0210] In one possible implementation, the above-described apparatus may further include a container management module for:

[0211] Obtain container operation instructions, which include the container identifier of the container to be processed;

[0212] Based on the container identifier of the container to be processed, a container operation instruction is sent to the NUMA node to which the container to be processed belongs, so that the NUMA node that receives the container operation instruction can perform the operation corresponding to the container operation instruction on the container to be processed.

[0213] In one possible implementation, the target application is a cloud gaming application, the application system is a cloud gaming scheduling backend, the application server is a cloud gaming server, and the target container is a container of the operating system corresponding to the cloud gaming application.

[0214] When the application launch request receiving module 410 receives an application launch request for a target application sent by a user terminal, it can be specifically used for:

[0215] Receive an application launch request for the cloud gaming application sent by a user terminal. The application launch request includes the application identifier of the cloud gaming application and the target operating system corresponding to the user terminal.

[0216] When the aforementioned target application startup module 430 sends an application startup request to the target NUMA node to which the target container belongs, so that the target NUMA node controls the target container to start the target application based on the target application's target container image file, it can be specifically used for:

[0217] Send an application launch request for the cloud gaming application to the target NUMA node to which the target container belongs, so that the target NUMA node can start the target operating system and the cloud gaming application based on the container image file of the cloud gaming application.

[0218] The apparatus in this application embodiment can execute the method provided in this application embodiment, and the implementation principle is similar. The actions performed by each module in the apparatus of each embodiment of this application correspond to the steps in the method of each embodiment of this application. For detailed functional descriptions of each module of the apparatus, please refer to the descriptions in the corresponding methods shown above, which will not be repeated here.

[0219] Based on the same principle as the application launch method and apparatus provided in the embodiments of this application, the embodiments of this application also provide an electronic device (such as a server), which may include a memory, a processor and a computer program stored in the memory, the processor executing the computer program to implement the steps of the method provided in any optional embodiment of this application.

[0220] Optionally, Figure 15 A schematic diagram of the structure of an electronic device to which this application embodiment applies is shown, such as... Figure 15 As shown, Figure 15 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this application.

[0221] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0222] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 15 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0223] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0224] The memory 4003 stores computer programs that execute embodiments of this application, and its execution is controlled by the processor 4001. The processor 4001 executes the computer programs stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0225] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the steps and corresponding content of the aforementioned method embodiments.

[0226] This application also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0227] The terms "first," "second," "third," "fourth," "1," "2," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown in the figures or text.

[0228] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.

[0229] The above description is only an optional implementation method for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application without departing from the technical concept of this application also fall within the protection scope of the embodiments of this application.

Claims

1. An application launch method, characterized in that, The method is applied to an application system, which includes an application management server and an application server. The application server includes at least two non-uniform memory access NUMA nodes, each of which has its own storage resources and at least one container is created in each of the NUMA nodes. The storage resources corresponding to the NUMA node include disks; the image management directory corresponding to the NUMA node points to the disk corresponding to the NUMA node. At least one container image file is stored in the image management directory corresponding to the NUMA node; The method is executed by the application management server, and the method includes: Receive an application launch request for a target application sent by a user terminal, wherein the application launch request includes the application identifier of the target application; Based on the application identifier, the target container corresponding to the application startup request is determined; The application startup request is sent to the target NUMA node to which the target container belongs, so that the target NUMA node controls the target container to start the target application based on the target container image file of the target application. The target container image file is read by the target NUMA node from the image management directory in the storage resource corresponding to the target NUMA node, and the target container image file includes a first image file for running the target application.

2. The method according to claim 1, characterized in that, Before receiving the application launch request, the method further includes: In the storage resources of the application server, the storage resources corresponding to each NUMA node are determined, and the binding relationship between each NUMA node and its corresponding storage resources is stored. Create at least one container in each of the NUMA nodes; An image file retrieval instruction is sent to each NUMA node, so that each NUMA node retrieves at least one container image file corresponding to an application according to the image file retrieval instruction, and stores the retrieved container image file in the storage resource corresponding to each NUMA node. The at least one application corresponding to at least one NUMA node includes the target application, and the container image file corresponding to the at least one application includes the target container image file.

3. The method according to claim 2, characterized in that, Sending the application startup request to the target NUMA node to which the target container belongs, so that the target NUMA node controls the target container to start the target application based on the target container image file of the target application, includes: Based on the application startup request, a container startup instruction is sent to the target NUMA node, so that the target NUMA node responds to the container startup instruction and reads the target container image file corresponding to the target application from the storage resources corresponding to the target NUMA node. The target container image file also includes a second image file for starting the target container. The target container image file is sent to the target container so that the target container starts the target container based on the first image file and starts the target application based on the second image file.

4. The method according to claim 3, characterized in that, After each NUMA node stores the acquired container image file into its corresponding storage resource, the method further includes: Send a container startup command for the target application to the at least one NUMA node, so that the NUMA node receiving the container startup command performs the following operations: Read the target container image file from the storage resources corresponding to the NUMA node; The target container image file is read and sent to at least one container on the NUMA node, so that the container that receives the target container image file starts the container based on the second image file; The step of determining the target container corresponding to the application startup request based on the application identifier includes: Based on the application identifier, the target container is determined from the started containers corresponding to the target application; The process of the target NUMA node controlling the target container to start the target application based on the target container image file of the target application includes: the target NUMA node sending an application start instruction to the target container so that the target container starts the target application based on the first image file.

5. The method according to claim 1, characterized in that, The application management server communicates with the at least two NUMA nodes via middleware components. Sending the application startup request to the target NUMA node to which the target container belongs includes: Based on the container identifier of the target container, perform the following operations through the middleware component: Based on the container identifier of the target container, determine the target NUMA node to which the target container belongs; The application startup request is sent to the target NUMA node through the communication interface corresponding to the target NUMA node.

6. The method according to claim 1, characterized in that, The application management server communicates with the at least two NUMA nodes through a proxy component corresponding to each NUMA node. Sending the application startup request to the target NUMA node to which the target container belongs includes: Based on the container identifier of the target container, determine the target NUMA node corresponding to the container identifier; The application startup request is sent to the target NUMA node through the proxy component corresponding to the target NUMA node.

7. The method according to claim 2, characterized in that, The method further includes: Obtain container operation instructions, wherein the container operation instructions include the container identifier of the container to be processed; Based on the container identifier of the container to be processed, the container operation instruction is sent to the NUMA node to which the container to be processed belongs, so that the NUMA node that receives the container operation instruction performs the operation corresponding to the container operation instruction on the container to be processed.

8. The method according to claim 1, characterized in that, The target application is a cloud gaming application, the application system is a cloud gaming scheduling backend, the application server is a cloud gaming server, and the target container is a container of the operating system corresponding to the cloud gaming application. The process of receiving an application launch request for a target application sent by a user terminal includes: The system receives an application launch request for the cloud gaming application sent by the user terminal. The application launch request includes the application identifier of the cloud gaming application and the target operating system corresponding to the user terminal. Sending the application startup request to the target NUMA node to which the target container belongs, so that the target NUMA node controls the target container to start the target application based on the target container image file of the target application, includes: Send an application launch request for the cloud gaming application to the target NUMA node to which the target container belongs, so that the target NUMA node can launch the target operating system and the cloud gaming application based on the container image file of the cloud gaming application.

9. An application startup device, characterized in that, The device is applied to an application system, which includes an application management server and an application server. The application server includes at least two non-uniform memory access NUMA nodes, each of which has its own storage resources and at least one container is created in each of the NUMA nodes. The storage resources corresponding to the NUMA node include disks; the image management directory corresponding to the NUMA node points to the disk corresponding to the NUMA node. At least one container image file is stored in the image management directory corresponding to the NUMA node; The device is implemented through the application management server and includes: An application launch request receiving module is used to receive an application launch request for a target application sent by a user terminal, wherein the application launch request includes the application identifier of the target application; The target container determination module is used to determine the target container corresponding to the application startup request based on the application identifier; The target application startup module is used to send the application startup request to the target NUMA node to which the target container belongs, so that the target NUMA node controls the target container to start the target application based on the target container image file of the target application. The target container image file is read by the target NUMA node from the image management directory in the storage resource corresponding to the target NUMA node, and the target container image file includes a first image file for running the target application.

10. The apparatus according to claim 9, characterized in that, The device also includes a resource management module, a container creation module, and an image file acquisition module. The resource management module is used to determine the storage resources corresponding to each NUMA node in the storage resources of the application server before receiving the application startup request, and to store the binding relationship between each NUMA node and the corresponding storage resources. The container creation module is used to create at least one container in each NUMA node; The image file acquisition module is used to send an image file acquisition instruction to each NUMA node, so that each NUMA node can acquire at least one container image file corresponding to an application according to the image file acquisition instruction, and store the acquired container image file in the storage resource corresponding to each NUMA node. The at least one application corresponding to at least one NUMA node includes the target application, and the container image file corresponding to the at least one application includes the target container image file.

11. The apparatus according to claim 10, characterized in that, When the target application startup module sends the application startup request to the target NUMA node to which the target container belongs, so that the target NUMA node controls the target container to start the target application based on the target container image file of the target application, it is specifically used for: Based on the application startup request, a container startup instruction is sent to the target NUMA node, so that the target NUMA node responds to the container startup instruction and reads the target container image file corresponding to the target application from the storage resources corresponding to the target NUMA node. The target container image file also includes a second image file for starting the target container. The target container image file is sent to the target container so that the target container starts the target container based on the first image file and starts the target application based on the second image file.

12. The apparatus according to claim 11, characterized in that, The device further includes a container pre-start module, the container pre-start module being used for: After each NUMA node stores the acquired container image file into its corresponding storage resource, a container startup command for the target application is sent to the at least one NUMA node, causing the NUMA node receiving the container startup command to perform the following operations: Read the target container image file from the storage resources corresponding to the NUMA node; The target container image file is read and sent to at least one container on the NUMA node, so that the container that receives the target container image file starts the container based on the second image file; When determining the target container corresponding to the application startup request based on the application identifier, the target container determination module can be specifically used for: Based on the application identifier, the target container is determined from the started containers corresponding to the target application; The process of the target NUMA node controlling the target container to start the target application based on the target container image file of the target application includes: the target NUMA node sending an application start instruction to the target container so that the target container starts the target application based on the first image file.

13. The apparatus according to claim 9, characterized in that, The application management server communicates with the at least two NUMA nodes through middleware components. When the target application startup module sends the application startup request to the target NUMA node to which the target container belongs, it can specifically be used to: Based on the container identifier of the target container, perform the following operations through the middleware component: Based on the container identifier of the target container, determine the target NUMA node to which the target container belongs; The application startup request is sent to the target NUMA node through the communication interface corresponding to the target NUMA node.

14. The apparatus according to claim 9, characterized in that, The application management server communicates with the at least two NUMA nodes through a proxy component corresponding to each NUMA node. When the target application startup module sends the application startup request to the target NUMA node to which the target container belongs, it is specifically used for: Based on the container identifier of the target container, determine the target NUMA node corresponding to the container identifier; The application startup request is sent to the target NUMA node through the proxy component corresponding to the target NUMA node.

15. The apparatus according to claim 10, characterized in that, The device may further include a container management module for: Obtain container operation instructions, wherein the container operation instructions include the container identifier of the container to be processed; Based on the container identifier of the container to be processed, the container operation instruction is sent to the NUMA node to which the container to be processed belongs, so that the NUMA node that receives the container operation instruction performs the operation corresponding to the container operation instruction on the container to be processed.

16. The apparatus according to claim 9, characterized in that, The target application is a cloud gaming application, the application system is a cloud gaming scheduling backend, the application server is a cloud gaming server, and the target container is a container of the operating system corresponding to the cloud gaming application. When the application launch request receiving module receives an application launch request for a target application sent by a user terminal, it is specifically used for: The system receives an application launch request for the cloud gaming application sent by the user terminal. The application launch request includes the application identifier of the cloud gaming application and the target operating system corresponding to the user terminal. When the target application startup module sends the application startup request to the target NUMA node to which the target container belongs, so that the target NUMA node controls the target container to start the target application based on the target container image file of the target application, it is specifically used for: Send an application launch request for the cloud gaming application to the target NUMA node to which the target container belongs, so that the target NUMA node can launch the target operating system and the cloud gaming application based on the container image file of the cloud gaming application.

17. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-8.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-8.

19. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-8.

Images