A virtualized computing power providing method and system

Abstract

The application discloses a virtualized computing power providing method and system. The virtualized computing power providing method comprises the following steps: obtaining a cluster management request, wherein the cluster management request comprises a deployment request of a virtual computing power device; obtaining the virtual computing power device from a kernel space of a computing node according to the deployment request, and generating a service program corresponding to the virtual computing power device in a user space of the computing node; mounting the virtual computing power device to a user container; scheduling physical resources for a target application provided by the user container in a resource set through the virtual computing power device and the service program, wherein the physical resources have a mapping relationship with the virtual computing power device; obtaining a running result of the target application through the virtual computing power device and the service program, and sending the running result to the user container. The application solves the problem that the existing technology has poor applicability due to different software and hardware and the lack of unified protocols for software stack specification, and achieves the technical effect of adapting to multiple scenarios.

Description

Technical Field

[0001] This application relates to the field of Internet technology applications, specifically to a virtualized computing power provision method and system. Background Technology

[0002] As an emerging field, artificial intelligence has developed rapidly in recent years, with numerous new algorithms and applications emerging in both academia and industry. The three key elements of artificial intelligence are algorithms, computing power, and data. Among these, computing power is the ultimate carrier for implementing algorithms. Figure 1 This is a diagram illustrating the hierarchical relationship between AI applications and hardware in existing technologies, such as... Figure 1 The diagram shows the hierarchical relationship between AI applications (algorithms) and hardware (computing power). Due to the differences in the underlying hardware, each layer above it has a different implementation. Currently, there is no unified protocol to standardize the software stack from top to bottom. Therefore, when users use and execute algorithms, they need to adapt to various different software and hardware to enable them and achieve the desired data processing results.

[0003] The problem of poor applicability caused by differences in hardware and software and the lack of a unified protocol for software stack specifications in current related technologies has not yet been effectively solved. Summary of the Invention

[0004] This application provides a virtualized computing power provision method and system to at least solve the problem of poor applicability in the prior art due to differences in software and hardware and the lack of a unified protocol for software stack specifications.

[0005] According to one aspect of this application, a method for providing virtualized computing power is provided, comprising: obtaining a cluster management request, wherein the cluster management request includes a deployment request for a virtual computing power device; obtaining the virtual computing power device from the kernel space of a computing node according to the deployment request, and generating a service program corresponding to the virtual computing power device in the user space of the computing node; mounting the virtual computing power device to a user container; scheduling physical resources for a target application provided to the user container in a resource set through the virtual computing power device and the service program, wherein the physical resources and the virtual computing power device form a mapping relationship; obtaining the running result of the target application through the virtual computing power device and the service program, and sending the running result to the user container.

[0006] Optionally, the method may also include: creating a virtual computing device in the kernel space based on the deployment request.

[0007] Optionally, the virtual computing device provides a unified data interface; wherein, scheduling physical resources for the target application provided by the user container in the resource set through the virtual computing device and the service program includes: obtaining the target application provided by the user container through the data interface provided by the virtual computing device, wherein the target application includes user data and computation instructions for the user data; sending the target application from the virtual computing device to the service program through shared memory; and sending the target application to the physical resources corresponding to the virtual computing device through the service program.

[0008] Further, optionally, the physical resources are a set of heterogeneous or homogeneous physical resources. The resource set and the service program are connected through a connection pool, wherein the protocols in the connection pool include: local transport protocols and network transport protocols.

[0009] Optionally, the local transport protocol includes PCIe; the network transport protocol includes at least one of the following: RDMA, TCP.

[0010] Optionally, the virtual computing device provides a unified management and control interface; the method further includes: obtaining management and control requests provided by user containers through the management and control interface of the virtual computing device, wherein the management and control requests include at least one of the following: computing power query request, computing power configuration request, transmission channel attribute configuration request, computing power migration request, virtual computing device status query request, and virtual computing device status configuration request; and processing the management and control requests through a service program.

[0011] Further, optionally, when the control request is a computing power configuration request, processing the computing power configuration request through the service program includes: sending the computing power configuration request to the physical resource management node through the service program, so that the physical resource management node can establish a mapping relationship between the virtual computing power device and the physical resource.

[0012] Optionally, cluster management requests may also include at least one of the following: a virtual computing device's running status query request, or a virtual computing device's deletion request; cluster management requests are obtained from the cluster management center through device plugins.

[0013] Optionally, cluster management requests are managed by the device management center. The method further includes: detecting and storing the status of the service program and / or the device management center, and performing system recovery based on the status of the stored service program and / or the device management center in the event of an abnormal status of the service program and / or the device management center.

[0014] Optionally, the virtual computing device can be simulated as a block device, character device, or network device.

[0015] According to another aspect of this application, a virtualized computing power provision system is provided, comprising: a device management center for obtaining cluster management requests, wherein the cluster management requests include deployment requests for virtual computing power devices; the device management center is further configured to obtain virtual computing power devices from the kernel space of a computing node according to the deployment requests, and generate service programs corresponding to the virtual computing power devices in the user space of the computing node; the device management center is further configured to mount the virtual computing power devices to user containers; the virtual computing power devices and service programs are configured to schedule physical resources for target applications provided to user containers in a resource set, wherein the physical resources and virtual computing power devices form a mapping relationship; the virtual computing power devices and service programs are further configured to obtain the running results of the target applications and send the running results to the user containers.

[0016] Optionally, physical resources are heterogeneous or homogeneous physical resources provided by the resource set. The resource set and the service program are connected through a connection pool. The protocols in the connection pool include local transport protocols and network transport protocols. The local transport protocol includes PCIe. The network transport protocol includes at least one of the following: RDMA and TCP.

[0017] Optionally, the system may also include: a testing center, which is used to test the status of the service procedures and / or the equipment control center.

[0018] Further, optionally, the system also includes: a device management file system for storing the status of the service program and / or the device control center, so as to perform system recovery based on the stored status of the service program and / or the device control center in the event of an anomaly in the status of the service program and / or the device control center.

[0019] In this embodiment, a cluster management request is obtained, including a deployment request for a virtual computing device. Based on the deployment request, a virtual computing device is obtained from the kernel space of the computing node, and a service program corresponding to the virtual computing device is generated in the user space of the computing node. The virtual computing device is then mounted to a user container. Through the virtual computing device and the service program, physical resources are scheduled for the target application provided to the user container from the resource set, wherein the physical resources and the virtual computing device form a mapping relationship. Finally, through the virtual computing device and the service program, the running results of the target application are obtained and sent to the user container. This application solves the problem of poor applicability in the prior art due to differences in hardware and software and the lack of a unified protocol for software stack standardization, achieving a technical effect of adapting to multiple scenarios. Attached Figure Description

[0020] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0021] Figure 1 This is a schematic diagram of a virtualized computing power provision system according to Embodiment 1 of this application;

[0022] Figure 2 This is a schematic diagram illustrating the interaction between underlying resources and virtual computing devices in a virtualized computing power provision system according to Embodiment 1 of this application;

[0023] Figure 3 This is a schematic diagram of the system framework of a virtual computing power device in a virtualized computing power provision system according to Embodiment 1 of this application;

[0024] Figure 4 This is a flowchart illustrating a virtualized computing power provision method according to Embodiment 2 of this application;

[0025] Figure 5 This is a schematic diagram illustrating the creation of virtual devices and the startup of service threads / processes in a virtualized computing power provision method according to Embodiment 2 of this application;

[0026] Figure 6 This is a schematic diagram of the computing power configuration and invocation process based on virtual devices in a virtualized computing power provision method according to Embodiment 2 of this application;

[0027] Figure 7 This is a schematic diagram of the device management and allocation process in a virtualized computing power provision method according to Embodiment 2 of this application. Detailed Implementation

[0028] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0029] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0030] Technical terms used in the embodiments of this application:

[0031] vXPU: Virtual XPU device, where X represents different types of computing chips, such as G for GPU, N for NPU, T for TPU, etc. Here, X is used as a general term.

[0032] Computing power: The unit of measurement for the computing power of a computing chip, usually expressed in OPS. In the AI ​​field, FLOPS is often used to characterize the computing power of a chip.

[0033] CN: Computing Node, typically located at the front end of a large server system;

[0034] HN: Heterogeneous Node, typically located at the backend of a large server system;

[0035] MN: ManagementNode;

[0036] K8S: Kubernetes, also known as K8S (hereinafter referred to as K8S), is a portable, scalable, open-source platform for managing containerized workloads and services, which facilitates declarative configuration and automation.

[0037] Containers: Containers are lightweight application code packages that also contain dependencies, such as specific versions of programming language runtimes and libraries required to run software services. From an implementation perspective, containers are divided into ordinary containers and secure containers. The main difference between secure containers and ordinary containers is that each container (more accurately, a pod) runs in a separate micro-virtual machine with an independent operating system kernel and secure isolation at the virtualization layer. Because cloud container instances use shared multi-group clusters, container security isolation has stricter requirements than when users have independent private Kubernetes clusters. Through secure containers, the kernel, computing resources, storage, and network are isolated between containers of different tenants. This protects user resources and data from being preempted and stolen by other users. This application focuses on ordinary containers;

[0038] POD: The smallest unit of Kubernetes scheduling, consisting of one or more containers;

[0039] IO: Input / Output;

[0040] ENTRY: The entry point of the container, which can be an executable file such as a script or an application;

[0041] Block device: One of the three basic device types in Linux systems;

[0042] RDMA: Remote DMA, short for Remote Direct Memory Access, was developed to address the latency issues in server-side data processing during network transmission. It transfers data directly from one computer's memory to another without the intervention of either operating system, allowing for high-throughput, low-latency network communication, making it particularly suitable for use in large-scale parallel computer clusters.

[0043] Example 1

[0044] According to one aspect of this application, a system using a virtual computing device is provided. Figure 1 This is a schematic diagram of a system using a virtual computing device according to Embodiment 1 of this application, as shown below. Figure 1 As shown, the virtualized computing power provision system provided in this embodiment includes: a management terminal 12, a scheduling terminal 14, a server terminal 16, and a resource set 18, wherein...

[0045] The control terminal 12 is used to receive user requests, convert the request content obtained from parsing the user requests into corresponding resource forms, including virtual resources and backend entity resources; the scheduling terminal 14 is used to provide virtual resources, create virtual devices according to user requests, and mount the virtual devices to the user container, where the user container is the space for the user to run computing activities; the scheduling terminal 14 is used to allocate backend entity resources from the resource set; the resource set is used to provide backend entity resources and data transmission services.

[0046] Specifically, such as Figure 1 As shown in the illustration, the management terminal 12 in the system using virtual computing power devices provided in this application embodiment is illustrated using Kubernetes (hereinafter referred to as K8S) as an example. Figure 1 The central control terminal 12 is labeled K8S, the scheduling terminal 14 is labeled MN, the server terminal 16 is labeled CN, and the resource set 18 is labeled HN. Among these,

[0047] K8S is responsible for receiving user requests, parsing the request content, and converting it into the actual required resource form, including virtual resources and backend entity resources.

[0048] The CN provides virtual resources and mounts them as Linux system devices to the data processing tasks / containers on the CN side; the user container is the space for users to run computing activities, including compilation, inference, training, etc.; wherein, in the embodiments of this application, the types of Linux system devices include at least: character devices, block devices and network devices.

[0049] MN is responsible for allocating backend entity resources from HN;

[0050] HN provides basic services such as physical resources and data transmission. In the embodiments of this application, the resource set can be a hardware resource set, specifically manifested as HN.

[0051] The system using virtual computing power devices provided in this application focuses on the implementation and deployment methods of the virtual device portion on the CN side. The specific implementation and deployment methods of the CN side virtual device portion are as follows:

[0052] The device management center is used to obtain cluster management requests, including deployment requests for virtual computing devices. Based on these requests, the device management center also retrieves virtual computing devices from the kernel space of the compute nodes and generates corresponding service programs in the user space of the compute nodes. Furthermore, the device management center mounts the virtual computing devices to user containers. The virtual computing devices and service programs are used to schedule physical resources for the target applications provided to user containers within the resource set, where physical resources are mapped to virtual computing devices. Finally, the virtual computing devices and service programs are used to obtain the running results of the target applications and send these results to the user containers.

[0053] Optionally, physical resources are heterogeneous or homogeneous physical resources provided by the resource set. The resource set and the service program are connected through a connection pool. The protocols in the connection pool include local transport protocols and network transport protocols. The local transport protocol includes PCIe. The network transport protocol includes at least one of the following: RDMA and TCP.

[0054] Specifically, after the virtual resources and backend physical resources are requested, a mapping relationship is established between them, and control and data interaction are carried out between the virtual and physical devices through communication methods or protocols such as RDMA / TCP / PCIe. Figure 2 This is a schematic diagram illustrating the interaction between underlying resources and virtual computing devices in a virtualized computing power provision system according to Embodiment 1 of this application, as shown below. Figure 2 As shown, the purpose of pooling in this embodiment is to improve resource utilization by uniformly managing underlying resources through networks or other transmission methods, with one or more control centers scheduling and allocating resources. Current network protocols include TCP (Transmission Control Protocol) and RDMA (Remote Direct Memory Access). Local computing devices are mostly connected via PCIe (Peripheral Component Interconnect express) topology. That is, in this embodiment, the connection to the computing terminal is achieved through preset communication services, which include: Internet communication protocols and / or device interface communication protocols. In this embodiment, the Internet communication protocol can be TCP or RDMA, and the device interface communication protocol can be PCIe.

[0055] Optionally, the virtualized computing power provision system provided in this application embodiment further includes: a detection center, which is used to detect the status of the service program and / or the device management center.

[0056] Furthermore, optionally, the virtualized computing power provision system provided in this application embodiment further includes: a device management file system, used to store the status of the service program and / or device control center, so as to restore the system based on the stored status of the service program and / or device control center in the event of an abnormal status of the service program and / or device control center.

[0057] Specifically, Figure 3 This is a schematic diagram of the system framework of a virtual computing power device in a virtualized computing power provision system according to Embodiment 1 of this application. The internal architecture of the server 16 in implementing the virtual computing power device is as follows: Figure 3 As shown, the resource format converted by the management terminal 12 according to the user request is received through the K8S management node service, and forwarded to the virtual device management and service container through the Device Plugin. In this embodiment, the container running environment is the virtual device management and service container. In the management function, the container entry is responsible for starting the device management center and external detection; external detection is responsible for detecting the entire device management center, reporting to the cluster detection center when an anomaly occurs, and handling it according to the anomaly level, such as restarting the device management center; the device management center is used to connect with the Device Plugin, handle plugin requests, such as creating new virtual devices, querying device running status, deleting devices, etc., and is also responsible for starting the internal detection function; internal detection is used to detect the backend service programs (processes or threads) of all devices, reporting to external detection when an anomaly occurs, and handling it according to the anomaly level, such as restarting the device backend service; the device management file is responsible for storing the running status of devices and services. When an anomaly occurs, such as coredump, the program can restore the system according to the device management file.

[0058] It should be noted that the virtual device management and service container in this embodiment is implemented in the form of SPDK (Storage Performance Development Kit).

[0059] like Figure 3 As shown, in the device I / O plane, server 16 also includes: device backend service. In the process of implementing the virtual device function, the virtual device consists of two parts. One part is in the kernel driver, which is responsible for receiving user requests and data and is the producer; the other part is the device backend service, which is in user space and is responsible for processing user requests, forwarding data, etc. and is the consumer.

[0060] The kernel driver is used to create virtual devices in the form of block devices (i.e., computing devices in this embodiment of the application; block devices are used as an example in this embodiment of the application. In addition, virtual computing devices can also be simulated in the form of character devices, or virtual computing devices can be simulated using standard form devices (such as virtio devices)). Through a shared memory area, it realizes the interaction and processing of data and commands.

[0061] Specifically, the device backend services include: a management and control program, which handles requests from the device for computing power queries, configurations, etc.; and a data service program, which is responsible for data transmission between data processing tasks and communication services.

[0062] The kernel driver includes: data and queue management: interacting with the device backend service through memory mapping; and control commands: mainly receiving requests such as computing power configuration queries from the data processing task side and sending the requests to the backend service program through netlink.

[0063] also, Figure 3 It also includes user containers, which include computing power configuration / query functions, inference / training functions, and virtual computing devices. User containers interact with kernel drivers to mount virtual devices created by kernel drivers to the corresponding user containers. By mounting devices, hardware transmission networks can be shielded. User containers only need to focus on computing power configuration and data transmission, without needing to worry about network configuration. User containers do not require privileged mode for access.

[0064] Example 2

[0065] According to one aspect of this application, a virtualized computing power provision method is provided. Figure 4 This is a flowchart illustrating a virtualized computing power provision method according to Embodiment 2 of this application, as shown below. Figure 4 As shown, the virtualized computing power provision method provided in this application embodiment includes:

[0066] Step S402: Obtain cluster management and control requests, wherein the cluster management and control requests include deployment requests for virtual computing devices;

[0067] In step S402 of this application, the virtualized computing power providing system applied in Embodiment 1 receives a cluster management request through K8S on the server side. The cluster management request can be an application instance request that includes at least an AI application (algorithm). After K8S receives the cluster management request, it parses the corresponding virtual resources and backend entity resources. The server executes the service thread / process / program for creating virtual devices based on the cluster management request parsed by K8S, i.e., step S404.

[0068] Step S404: According to the deployment request, obtain the virtual computing power device from the kernel space of the computing node, and generate the service program corresponding to the virtual computing power device in the user space of the computing node;

[0069] Optionally, the virtualized computing power provision method provided in this application embodiment further includes: creating a virtual computing power device in the kernel space according to a deployment request.

[0070] Specifically, as in Example 1 Figure 1 In the system shown, the K8S management node service receives the resource form converted by the management end according to the deployment request, forwards it to the virtual device management and service container through the Device Plugin, and creates a virtual device by parsing the deployment request through the virtual device management and service container. In the memory driver, it takes the form of a computing device and realizes the interaction and processing of data and commands through the shared memory area, thus creating a virtual computing device.

[0071] Figure 5 This is a schematic diagram illustrating the creation of virtual devices and the startup of service threads / processes in a virtualized computing power provision method according to Embodiment 2 of this application. Figure 5 As shown, creating virtual computing devices and user containers in the kernel space based on the deployment request can be:

[0072] S1, Start the Device Management Center program / SPDKAPP;

[0073] S2, is it a user request to create a virtual device?

[0074] S3 sends a request to the kernel space driver of the compute node to create a vring virtual device;

[0075] S4, register user account Admin to manage Poller to SPDKreactor;

[0076] S5, register the IO processing Poller to SPDKreactor;

[0077] S6, update device list.

[0078] Step S406: Mount the virtual computing device to the user container;

[0079] Specifically, as in Example 1 Figure 3 As shown, on the CN side, after the virtual device is created in the memory driver, a virtual computing instance device is mapped and generated in the corresponding user container. Thus, the virtual computing device is mounted to the user container through the interaction between the virtual device and the virtual computing device.

[0080] Step S408: Through virtual computing devices and service programs, physical resources are scheduled for the target application provided by the user container in the resource set, wherein the physical resources and the virtual computing devices form a mapping relationship.

[0081] Specifically, based on the user container in step S406, virtual computing devices and service programs are used to schedule corresponding physical resources for the target application.

[0082] Step S410: Obtain the running results of the target application through virtual computing devices and service programs, and send the running results to the user container.

[0083] Specifically, through the interaction between virtual computing devices and service programs and the user container, the results are returned to the user container after the target application completes.

[0084] Figure 6 This is a schematic diagram of the device management and allocation process in a method for using virtual computing power devices according to Embodiment 2 of this application. Combined with steps S402-S410, the virtualized computing power provision method provided by this embodiment can be:

[0085] S1, K8S requests resources;

[0086] S2, Device Plugin sends a request to the virtual device management process;

[0087] S3, the virtual device management process, creates virtual devices;

[0088] S4, the virtual device management process, creates service threads / processes for virtual devices;

[0089] S5, the virtual device management process returns to Device Plugin / K8S;

[0090] S6 and Kubernetes mount virtual devices to user containers;

[0091] S7 and Kubernetes mount virtual devices to user containers;

[0092] S8, user containers use virtual devices.

[0093] Optionally, the virtual computing device provides a unified data interface; wherein, scheduling physical resources for the target application provided by the user container in the resource set through the virtual computing device and the service program includes: obtaining the target application provided by the user container through the data interface provided by the virtual computing device, wherein the target application includes user data and computation instructions for the user data; sending the target application from the virtual computing device to the service program through shared memory; and sending the target application to the physical resources corresponding to the virtual computing device through the service program.

[0094] The virtualized computing power provision method provided in this application embodiment can be applied to cloud application scenarios. The computing power demand instance can be an instance that realizes various scenarios and various needs. For example, a user sends a data query request through a user terminal. In the cloud, a data query is executed based on the data query request, and the data query result is returned to the user terminal. The entire data query process can be a computing power demand instance.

[0095] Specifically, the above-mentioned method of sending the computing power demand instance provided by the user container to the physical resource corresponding to the virtual computing power device through the virtual computing power device and service program includes: as in Example 1. Figure 3 As shown, the physical resources for scheduling the target application provided to the user container in the resource set include: obtaining the target application provided by the user container through the data interface provided by the virtual computing device, wherein the target application includes user data and computation instructions for the user data; sending the target application from the virtual computing device to the service program through shared memory; and sending the target application to the physical resources corresponding to the virtual computing device through the service program.

[0096] Further, optionally, the physical resources are a set of heterogeneous or homogeneous physical resources. The resource set and the service program are connected through a connection pool, wherein the protocols in the connection pool include: local transport protocols and network transport protocols.

[0097] Optionally, the local transport protocol includes PCIe; the network transport protocol includes at least one of the following: RDMA, TCP.

[0098] Specifically, after the virtual resources and backend physical resources are requested, a mapping relationship is established between them, and control and data interaction are performed between the virtual and physical devices through communication methods or protocols such as RDMA / TCP / PCIe. (See Example 1.) Figure 2As shown, the purpose of pooling in this embodiment is to improve resource utilization by uniformly managing underlying resources through networks or other transmission methods, with one or more control centers scheduling and allocating resources. Current network protocols include TCP (Transmission Control Protocol) and RDMA (Remote Direct Memory Access). Local computing devices are mostly connected via PCIe (Peripheral Component Interconnect express) topology. That is, in this embodiment, the connection to the computing terminal is achieved through preset communication services, which include: Internet communication protocols and / or device interface communication protocols. In this embodiment, the Internet communication protocol can be TCP or RDMA, and the device interface communication protocol can be PCIe.

[0099] Optionally, the virtual computing power device provides a unified management and control interface; the virtualized computing power provision method provided in this application embodiment further includes: obtaining management and control requests provided by user containers through the management and control interface of the virtual computing power device, wherein the management and control requests include at least one of the following: computing power query request, computing power configuration request, transmission channel attribute configuration request, computing power migration request, virtual computing power device status query request, and virtual computing power device status configuration request; and processing the management and control requests through a service program.

[0100] Further, optionally, when the control request is a computing power configuration request, processing the computing power configuration request through the service program includes: sending the computing power configuration request to the physical resource management node through the service program, so that the physical resource management node can establish a mapping relationship between the virtual computing power device and the physical resource.

[0101] Specifically, Figure 7 This is a schematic diagram illustrating the computing power configuration and invocation process based on virtual devices in a virtualized computing power provision method according to Embodiment 2 of this application, as shown below. Figure 7 As shown, the computing power configuration and invocation process based on virtual devices in this application embodiment can be as follows:

[0102] S1, User starts the program;

[0103] S2, the user prepares computing resources, data, etc.;

[0104] S3, the user calls the computing power operation interface;

[0105] S4 is the computing power operation interface call, which is used to configure computing power attributes through the virtual device management interface.

[0106] S5, the computing power operation interface call, and the virtual device management and control interface for configuring transmission channel attributes;

[0107] S6, the computing power operation interface call, and the data path interface of the virtual device for data transmission;

[0108] S7, the computing power operation interface call, obtains the running results from the data path interface of the virtual device.

[0109] Optionally, cluster management requests may also include at least one of the following: a virtual computing device's running status query request, or a virtual computing device's deletion request; cluster management requests are obtained from the cluster management center through device plugins.

[0110] Optionally, cluster management requests are managed by the device management center. The virtualization computing power provision method provided in this application embodiment further includes: detecting and storing the status of the service program and / or the device management center, and performing system recovery based on the status of the stored service program and / or the device management center if the status of the service program and / or the device management center is abnormal.

[0111] Optionally, the virtual computing device can be simulated as a block device, character device, or network device.

[0112] The virtual computing device in this application embodiment is illustrated using a block device as an example. In addition, it can also be simulated in the form of a character device, or a standard device (such as a virtio device) can be used to simulate a virtual computing device. Through a shared memory area, data and command interaction and processing can be realized.

[0113] Furthermore, the virtual computing power device interface is defined as follows in the virtualized computing power provision method provided in this application embodiment:

[0114] Virtual computing power operation interface

[0115] Computing power operation (pseudocode):

[0116] 1. Obtain the number of computing devices

[0117] vxpu_get_xpu_count(fd,&count);

[0118] 2. Obtain computing power attributes

[0119] vxpu_get_feature(fd,vxpu_attr,id);

[0120] 3. Set computing power attributes

[0121] vxpu_set_feature(fd,vxpu_attr,id);

[0122] Control and management operations:

[0123] 1. Initialize the VXPU device

[0124] vxpu_init(vxpu_name)

[0125] 2. Release the VXPU device

[0126] vxpu_deinit(vxpu_name)

[0127] 3. Allocate memory for VXPU

[0128] buf = vxpu_malloc(size)

[0129] 4. Release the memory allocated by vxpu

[0130] vxpu_free(buf)

[0131] Read / write operations (synchronous):

[0132] 1. Open the VXPU device

[0133] fd=vxpu_open(" / dev / vxpu",flags,mode)

[0134] 2. Read data from the VXPU device

[0135] vxpu_read(fd,buf,count,offset,direct)

[0136] 3. Write data to the VXPU device

[0137] vxpu_write(fd,buf,count,offset,direct)

[0138] 4. VXPU Management Operations

[0139] vxpu_admin(fd,opcode,flag,buf,size,direction,opaque)

[0140] 5. Turn off the VXPU device.

[0141] vxpu_close(fd)

[0142] Read and write operations (asynchronous):

[0143] 1. Configure asynchronous context

[0144] vxpu_io_setup(unsignednr_events,aio_context_t*ctx_idp);

[0145] 2. Submit a request

[0146] vxpu_io_submit(aio_context_t ctx_id,long nr,struct iocb**iocbpp);

[0147] 3. Obtain the results

[0148] vxpu_io_getevents(aio_context_t ctx_id,long min_nr,long nr,

[0149] struct io_event*events,struct timespec*timeout);

[0150] 4. Destroy the context

[0151] vxpu_io_destroy(aio_context_t ctx_id);

[0152] 5. Cancel request

[0153] vxpu_io_cancel(aio_context_t ctx_id,struct iocb*iocb,

[0154] struct io_event*result);

[0155] The virtual computing power provision method provided in this application embodiment implements a virtual computing power device that is naturally adapted to cloud-native scenarios. The virtual device service runs in the form of a container and interacts with cluster management centers such as Kubernetes through device plugins. The virtual device can be created and queried at any time and is available on demand. Theoretically, a single node can support thousands of instances. Taking 32MB of shared memory read and write as an example (virtual device size), for a system with 512GB of memory, it can theoretically provide 512GB / 2 / 64MB = 4096 instances. By mounting the device, the hardware transmission network can be shielded, and user containers only need to focus on the configuration of computing power and data transmission, without needing to worry about network configuration. User container access does not require privileged mode. The backend service can connect to local physical computing resources or connect to network transmission services for pooling, adapting to multiple scenarios.

[0156] In this embodiment, a cluster management request is obtained, including a deployment request for a virtual computing device. Based on the deployment request, a virtual computing device is obtained from the kernel space of the computing node, and a service program corresponding to the virtual computing device is generated in the user space of the computing node. The virtual computing device is then mounted to a user container. Through the virtual computing device and the service program, physical resources are scheduled for the target application provided to the user container from the resource set, wherein the physical resources and the virtual computing device form a mapping relationship. Finally, through the virtual computing device and the service program, the running results of the target application are obtained and sent to the user container. This application solves the problem of poor applicability in the prior art due to differences in hardware and software and the lack of a unified protocol for software stack standardization, achieving a technical effect of adapting to multiple scenarios.

[0157] Example 3

[0158] According to another aspect of this application, an electronic device is also provided, including a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps in Embodiment 2 above.

[0159] In this application, an embodiment can provide an electronic device, which can be any one of a group of electronic devices. Optionally, in this embodiment, the aforementioned electronic device can also be replaced by a terminal device such as a mobile terminal.

[0160] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0161] Optionally, the electronic device may include one or more processors and memory.

[0162] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the load balancing method and apparatus in this application embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned load balancing method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to electronic devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0163] The processor can access information and applications stored in memory via the transmission module to perform the following steps: obtaining a cluster management request, which includes a deployment request for virtual computing devices; obtaining virtual computing devices from the kernel space of the compute node according to the deployment request, and generating a service program corresponding to the virtual computing devices in the user space of the compute node; mounting the virtual computing devices to the user container; scheduling physical resources for the target application provided to the user container in the resource set through the virtual computing devices and service programs, wherein the physical resources and virtual computing devices are mapped to each other; obtaining the running results of the target application through the virtual computing devices and service programs, and sending the running results to the user container.

[0164] Optionally, the processor may also execute program code that creates a virtual computing device in the kernel space according to a deployment request.

[0165] Optionally, the processor may also execute program code that performs the following steps: the virtual computing device provides a unified data interface; wherein, scheduling physical resources for the target application provided by the user container in the resource set includes: obtaining the target application provided by the user container through the data interface provided by the virtual computing device, wherein the target application includes user data and computation instructions for the user data; sending the target application from the virtual computing device to the service program through shared memory; and sending the target application to the physical resources corresponding to the virtual computing device through the service program.

[0166] Further, optionally, the processor may also execute program code that includes the following steps: the physical resources are a set of heterogeneous or homogeneous physical resources, the resource set is connected to the service program through a connection pool, wherein the protocols in the connection pool include: local transport protocols and network transport protocols.

[0167] Optionally, the processor may also execute program code that includes the following steps: local transport protocol including PCIe; network transport protocol including at least one of the following: RDMA, TCP.

[0168] Optionally, the processor may also execute program code that performs the following steps: the virtual computing device provides a unified management and control interface; obtains management and control requests provided by user containers through the management and control interface of the virtual computing device, wherein the management and control requests include at least one of the following: computing power query request, computing power configuration request, transmission channel attribute configuration request, computing power migration request, virtual computing device status query request, and virtual computing device status configuration request; and processes the management and control requests through a service program.

[0169] Further, optionally, the processor may also execute program code that performs the following steps: when the control request is a computing power configuration request, the computing power configuration request is processed by the service program, including: sending the computing power configuration request to the physical resource management node through the service program, so that the physical resource management node can establish a mapping relationship between the virtual computing power device and the physical resource.

[0170] Optionally, the processor may also execute program code that includes the following steps: the cluster management request may include at least one of the following: a virtual computing device running status query request, a virtual computing device deletion request; the cluster management request is obtained from the cluster management center through the device plugin.

[0171] Optionally, the processor may also execute program code that performs the following steps: cluster management requests are managed by the device management center, the method further includes: detecting and storing the status of the service program and / or the device management center, and performing system recovery based on the status of the stored service program and / or the device management center in the event of an abnormal status of the service program and / or the device management center.

[0172] Optionally, the processor may also execute program code that simulates the virtual computing device as a block device, character device, or network device.

[0173] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0174] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0175] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0176] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0177] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0178] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0179] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for providing virtualized computing power, comprising: Obtain cluster management and control requests, wherein the cluster management and control requests include deployment requests for virtual computing devices; According to the deployment request, the virtual computing power device is created in the kernel space of the computing node, and a service program corresponding to the virtual computing power device is generated in the user space of the computing node. Attach the virtual computing power device to the user container; Through the virtual computing device and the service program, physical resources are scheduled for the target application provided by the user container in the resource set, wherein the physical resources and the virtual computing device form a mapping relationship. The virtual computing device and the service program are used to obtain the running results of the target application and send the running results to the user container.

2. The virtualized computing power provision method according to claim 1, wherein, The virtual computing power device provides a unified data interface; wherein, through the virtual computing power device and the service program, the physical resources for target application scheduling provided to the user container in the resource set include: The target application provided by the user container is obtained through the data interface provided by the virtual computing device, wherein the target application includes user data and calculation instructions for the user data; The target application is sent from the virtual computing device to the service program via shared memory; The service program sends the target application to the physical resources corresponding to the virtual computing device.

3. The virtualized computing power provision method according to claim 2, wherein, The physical resources are either heterogeneous or homogeneous physical resources. The resource set is connected to the service program through a connection pool, wherein the protocols in the connection pool include local transmission protocols and network transmission protocols.

4. The virtualized computing power provision method according to claim 3, wherein, The local transport protocol includes PCIe; the network transport protocol includes at least one of the following: RDMA, TCP.

5. The virtualized computing power provision method according to claim 1, wherein, The virtual computing power device provides a unified management and control interface; the method further includes: The management and control request provided by the user container is obtained through the management and control interface of the virtual computing power device, wherein the management and control request includes at least one of the following: computing power query request, computing power configuration request, transmission channel attribute configuration request, computing power migration request, virtual computing power device status query request, and virtual computing power device status configuration request. The control request is processed by the service program.

6. The virtualized computing power provision method according to claim 5, wherein, When the control request is a computing power configuration request, processing the computing power configuration request through the service program includes: The service program sends the computing power configuration request to the physical resource management node, so that the physical resource management node can establish a mapping relationship between the virtual computing power device and the physical resources.

7. The virtualized computing power provision method according to claim 1, wherein, The cluster management and control request also includes at least one of the following: a virtual computing power device operation status query request, a virtual computing power device deletion request; the cluster management and control request is obtained from the cluster management and control center through the device plugin.

8. The virtualized computing power provision method according to any one of claims 1 to 7, wherein, The cluster management request is managed by the device management center, and the method further includes: The system detects and stores the status of the service program and / or the device management center, and performs system recovery based on the stored status of the service program and / or the device management center in the event of an abnormal status of the service program and / or the device management center.

9. The virtualization computing power provision method according to any one of claims 1 to 7, wherein, The virtual computing power device is simulated in the form of a block device, a character device, or a network device.

10. A virtualized computing power provision system, comprising: The device management center is used to obtain cluster management requests, wherein the cluster management requests include deployment requests for virtual computing devices; The device management center is also used to create the virtual computing power device in the kernel space of the computing node according to the deployment request, and generate a service program corresponding to the virtual computing power device in the user space of the computing node; The device management center is also used to mount the virtual computing power device to the user container; The virtual computing device and the service program are used to schedule physical resources for the target application provided by the user container in the resource set, wherein the physical resources and the virtual computing device have a mapping relationship. The virtual computing device and the service program are also used to obtain the running results of the target application and send the running results to the user container.

11. The virtualized computing power provision system according to claim 10, wherein, The physical resources are heterogeneous or homogeneous physical resources provided by the resource set. The resource set and the service program are connected through a connection pool. The protocols in the connection pool include: local transmission protocols and network transmission protocols. The local transmission protocol includes: PCIe. The network transmission protocol includes at least one of the following: RDMA, TCP.

12. The virtualized computing power provision system according to claim 10, wherein, The system also includes: The testing center is used to detect the status of the service program and / or the device control center.

13. The virtualized computing power provision system according to claim 12, wherein, The system also includes: A device management file system is used to store the status of the service program and / or the device management center, so as to perform system recovery based on the stored status of the service program and / or the device management center in the event of an abnormal status of the service program and / or the device management center.

Images