Method and system for providing computing resource, and electronic device and storage medium

Abstract

Provided in the embodiments of the present description are a method and system for providing a computing resource, and an electronic device and a storage medium. The system for providing a computing resource implements the reception and transmission of a computing request in a virtual computing power device by means of para-virtualization technology, such that the computing request can be transmitted, by means of a virtual front-end unit running in a user mode and on the basis of a shared memory, to a virtual back-end unit on which a virtual computing resource is mounted, and thus it is not necessary to perform data copying between the user mode and a kernel mode for the computing request. In this way, the number of instances of data copying can be greatly reduced, thereby improving the data transmission efficiency, and reducing the performance loss when a computing resource is provided. In addition, by means of the system for providing a computing resource, a virtual back-end unit is separated from a virtual front-end unit, functions such as data receiving and forwarding, and network communication can be integrated in the virtual back-end unit, and data receiving and forwarding, and network communication are hidden by means of providing an SDK for a user, thereby reducing invasion of a user container to the greatest extent.

Description

A method, system, electronic device and storage medium for providing computing resources

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on June 27, 2023, with application number 202310772297.2 and application name “A method, system, electronic device and storage medium for providing computing resources”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments in this specification relate to the field of computer application technology, specifically, to computing power pooling technology under the field of computer application technology, and more specifically, to a method, system, electronic device and storage medium for providing computing resources. Background Art

[0003] With the continuous development of artificial intelligence, from facial recognition and intelligent voice to autonomous driving and industrial intelligence, more and more scenarios are deeply integrating artificial intelligence technology, improving product competitiveness and forming intelligent application products. Currently, most intelligent application products rely on computing resources provided by cloud platforms to perform computing tasks and meet various application needs.

[0004] With the increase in the number of cloud platform users, it is of great significance to reduce the performance loss of cloud platforms when providing computing resources.

[0005] Summary of the Invention

[0006] Multiple embodiments in this specification provide a method, system, electronic device, and storage medium for providing computing resources to achieve the purpose of reducing performance loss when providing computing resources.

[0007] One embodiment of this specification provides a method for providing computing resources, including:

[0008] In response to a computing request, the computing request is sent to a virtual computing power device, the virtual computing power device including a user module, the user module including a virtual front-end unit and a virtual back-end unit both running in user mode; the virtual front-end unit is used to receive the computing request and save the computing request to a shared memory based on a paravirtualization protocol; the virtual back-end unit is used to mount a virtual computing resource, receive the computing request from the shared memory based on the paravirtualization protocol, and send the computing request to the virtual computing resource; the computing request is used to request the virtual computing resource to perform a computing task; the virtual computing resource includes at least one of computing power and video memory;

[0009] Receive the calculation result returned by the virtual computing power device.

[0010] One embodiment of the present specification provides a computing resource provision system, including: a management and control module and a data module; wherein,

[0011] The control module is configured to respond to a computing request, send the computing request to the virtual computing device, and receive a computing result returned by the virtual computing device;

[0012] The data module is used to establish the virtual computing power device; the virtual computing power device includes a user module, and the user module includes a virtual front-end unit and a virtual back-end unit, both of which run in user state; the virtual front-end unit is used to receive the computing request and save the computing request to the shared memory based on the paravirtualization protocol; the virtual back-end unit is used to mount virtual computing resources, receive the computing request from the shared memory based on the paravirtualization protocol, and send the computing request to the virtual computing resource; the computing request is used to request the virtual computing resource to perform a computing task; the virtual computing resources include at least one of computing power and video memory.

[0013] In a third aspect, an embodiment of this specification provides an electronic device, including: a processor and a memory;

[0014] The memory is connected to the processor, and is used to store computer programs;

[0015] The processor is configured to implement the above-mentioned method for providing computing resources by running the computer program stored in the memory.

[0016] In a fourth aspect, an embodiment of this specification provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for providing computing resources as described above is implemented.

[0017] In a fifth aspect, an embodiment of this specification provides a computer program product or a computer program, wherein the computer program product includes a computer program, and the computer program is stored in a computer-readable storage medium; the processor of the computer device reads the computer program from the computer-readable storage medium, and when the processor executes the computer program, the steps of the above-mentioned method for providing computing resources are implemented.

[0018] Multiple implementations provided in this specification utilize paravirtualization technology to implement the reception and transmission of computing requests within virtual computing devices. This allows computing requests to be transmitted via shared memory from a virtual front-end unit running in user mode to a virtual back-end unit hosting virtual computing resources. This eliminates the need for data copying between user mode and kernel mode. This significantly reduces the number of data copies, improves data transmission efficiency, and minimizes performance loss when providing computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] FIG1 is a schematic diagram of an application scenario of a method for providing computing resources according to an embodiment of the present disclosure;

[0020] FIG2 is a schematic diagram of a system architecture of a cloud platform provided in one embodiment of this specification;

[0021] FIG3 is a schematic diagram of the structure of a computing resource provision system provided in one embodiment of this specification;

[0022] FIG4 is a schematic diagram of a heterogeneous architecture of a computing resource provision system provided in one embodiment of this specification;

[0023] FIG5 is a schematic diagram of a multi-threaded architecture of a virtual computing device provided by one embodiment of this specification;

[0024] FIG6 is a schematic diagram of a specific implementation architecture of a virtual computing device provided in one embodiment of this specification;

[0025] FIG7 is a flow chart of a method for providing computing resources according to an embodiment of the present disclosure;

[0026] FIG8 is a schematic structural diagram of an electronic device provided in accordance with an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] Unless otherwise defined, technical or scientific terms used in the embodiments of this specification should have the same ordinary meaning as those understood by persons of ordinary skill in the art to which this specification pertains. The terms "first," "second," and similar terms used in the embodiments of this specification do not denote any order, quantity, or importance, but are provided solely to avoid confusion between constituent elements.

[0028] Unless the context requires otherwise, throughout this specification, the term "plurality" means "at least two," and "including" is to be interpreted as open and inclusive, meaning "including, but not limited to." Throughout this specification, the terms "one embodiment," "some embodiments," "exemplary embodiments," "example," "specific example," or "some examples" are intended to indicate that a particular feature, structure, material, or characteristic associated with the embodiment or example is included in at least one embodiment or example of this specification. The schematic representations of these terms do not necessarily refer to the same embodiment or example.

[0029] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the embodiments described are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this specification.

[0030] Overview

[0031] As described in the background technology, in order to support the demand for computing resources of various applications, various cloud platform manufacturers have achieved the purpose of virtualizing physical computing chips into virtual computing resources and providing them to users through virtualization technology and pooling technology. Among them, based on virtualization technology, multiple applications can share hardware. For example, taking the GPU as an example, by dividing the physical GPU into multiple virtual GPUs, each virtual GPU can be assigned to a virtual machine or container. Each virtual GPU can have its own virtual driver, which is responsible for virtualizing the GPU hardware and sending the virtualized commands to the physical GPU. The purpose of multiple applications sharing hardware can be achieved through the virtualization of computing power.

[0032] Based on pooling technology, all computing resources can be aggregated into a single set, managing and allocating the computing and storage resources of the entire set. Computing power pooling effectively reduces the waste of computing resources, automatically manages and schedules resources, quickly and dynamically adjusts the scale and allocation of resources based on the required computing volume, and shares computing resources across the entire set. Computing power pooling can be used in areas such as massive data processing, distributed computing, and highly concurrent information systems, providing users with low-cost, high-efficiency, and highly scalable computing power. Pooling technology allows for remote access to hardware, removing the limitations of a single machine. Furthermore, pooling abstracts hardware capabilities into computing power metrics through computing power devices, while providing a unified interface to shield different computing chip hardware ecosystems, thereby reducing the difficulty of adaptation and improving ease of use.

[0033] Currently, when cloud platforms provide virtual computing resources to users through virtual computing devices and pooling technology, computing requests and results usually require data copying operations between kernel state and user state, which causes performance loss to the cloud platform.

[0034] To address this issue, the inventors discovered through research that paravirtualization technology can be used to implement the reception and transmission of computing requests within virtual computing devices. This allows computing requests to be transmitted via shared memory from a virtual front-end unit running in user mode to a virtual back-end unit hosting virtual computing resources. This eliminates the need for data copying between user mode and kernel mode. This significantly reduces the number of data copies required, improves data transmission efficiency, and minimizes performance losses when providing computing resources.

[0035] Based on the above concept, an embodiment of this specification provides a method for providing computing resources. The following will exemplarily describe the method for providing computing resources provided in the embodiment of this specification in conjunction with the accompanying drawings.

[0036] Scenario Example

[0037] This specification provides an example of an application scenario of a method for providing computing resources. As shown in Figure 1, a user submits a computing request to the cloud platform. The cloud platform forwards the computing request to the mounted virtual computing resource through the virtual computing power device corresponding to the user. After the virtual computing resource performs actual calculation processing on the computing task carried by the computing request, the calculation result is returned to the virtual computing power device, and finally the virtual computing power device returns the calculation result to the user.

[0038] The feasible hardware architecture of the cloud platform can refer to Figure 2. The cloud platform 10 may include multiple nodes. According to the differences in node functions, hardware, etc., the nodes may be divided into management and control nodes 111, computing nodes 121, and heterogeneous resource nodes 122. Among them, the management and control node 111 may be in the control plane 11, and the computing nodes 121 and heterogeneous resource nodes 122 may be in the data plane 12. In the control plane 11, the number of management and control nodes 111 may be multiple, and data consistency may be maintained between multiple management and control nodes 111 through a consistency protocol, thereby improving the reliability of the system. The computing node 121 may refer to a node including a CPU (Central Processing Unit, general-purpose processor), and the heterogeneous resource node 122 may refer to a node including computing chips such as a GPU (Graphical Processing Unit, graphics processor), a TPU (Tensor Processing Unit, tensor processor), and a DPU (Deep Learning Process Unit, deep learning processor). In some cases, the computing node 121 may also include computing chips such as GPU, TPU, and DPU. It is not difficult to understand that since computing node 121 includes hardware resources such as CPU and memory, computing node 121 can be suitable for running user applications. And since heterogeneous resource node 122 includes computing chips such as GPU, TPU and DPU suitable for AI (Artificial Intelligence) computing, heterogeneous resource node 122 can be suitable for performing specific computing tasks. When computing node 121 also includes computing chips such as GPU, TPU and DPU, computing node 121 can also perform computing tasks.

[0039] Users can send specific computing requests to the cloud platform through a client. The client can be a device with certain computing and communication capabilities, such as a desktop computer, tablet computer, laptop computer, smartphone, digital assistant, smart wearable device, shopping guide terminal, television, smart speaker, microphone, etc. Smart wearable devices include but are not limited to smart bracelets, smart watches, smart glasses, smart helmets, smart necklaces, etc. Alternatively, the client can be software that can run on these electronic devices.

[0040] Nodes in a cloud platform may include servers, which can be electronic devices with certain computing and processing capabilities. They may include network communication modules, processors, and memory. The servers may also be distributed servers, comprising multiple processors, memories, network communication modules, and other components operating in coordination. Alternatively, the servers may be clusters of multiple servers. Alternatively, with the advancement of science and technology, servers may also be new technological means capable of implementing the functions described in the embodiments of this specification. For example, they may be new forms of "servers" based on quantum computing.

[0041] Before introducing the specific implementation methods, the technical terms that may be involved in the implementation methods of this specification are explained:

[0042] Computing power refers to the computing power of a computing chip, generally measured in OPS (Operations Per Second). In the field of artificial intelligence, FLOPS (Floating Point Operations Per Second) is often used, that is, the number of floating-point operations per second to characterize the computing power of a chip.

[0043] AI computing, a mathematically intensive process of calculating machine learning algorithms, often using accelerated systems and software, can extract new insights from massive data sets and learn new capabilities in the process.

[0044] Video memory, also known as frame buffer, is used to store rendering data that has been processed or is about to be retrieved by the graphics chip. Like computer memory, video memory is a component used to store graphics information to be processed.

[0045] Computing chips may refer to chips used for AI computing, such as GPUs, DPUs, and TPUs.

[0046] A computing node (CN) may refer to a server including a CPU. In some cases, a computing node may also include a computing chip.

[0047] A heterogeneous resource node (HN) may refer to a server including a computing chip.

[0048] Computing resources can refer to the computing power and video memory provided by computing chips.

[0049] Virtual computing resources may refer to computing resources that have been virtualized and pooled.

[0050] K8s: Kubernetes (Container Orchestration Platform), also known as K8s, is an open source system for automating the deployment, scaling, and management of containerized applications.

[0051] Container: A container is a package of software that contains all the elements needed to run software, including code, executable files, and dependencies, such as specific versions of programming language runtimes and libraries required to run software services. Containers can run in any environment, whether it's a private data center, a public cloud, or a personal laptop.

[0052] Pod (container group): The smallest deployable computing unit created and managed in Kubernetes, consisting of one or more containers.

[0053] YAML (YAML Ain't Markup Language): A human-readable, easy-to-understand markup language with a specific format. In Kubernetes, user requests are described and submitted to Kubernetes through YAML configuration files.

[0054] IO: Input / Ouput, input and output.

[0055] RDMA (Remote Direct Memory Access) was developed to address server-side data processing latency during network transmission. It transfers data directly from the memory of one computer to another, without requiring intervention from either computer's operating system. This enables high-throughput, low-latency network communication, making it particularly suitable for use in large-scale parallel computer clusters.

[0056] Virtio (virtualized I / O): A paravirtualization technology that defines a set of general I / O device abstractions and a communication protocol, virtio spec (virtio device specification), which can achieve higher performance than full virtualization solutions. Generally speaking, virtio consists of the following three parts: the guest virtio driver (also known as the virtio frontend), which refers to the virtio driver in the virtual machine and is mainly used to issue I / O commands and receive I / O results in the virtual machine; the host virtio driver (also known as the virtio backend), which refers to the virtio driver on the host machine and is mainly used to receive I / O commands from the virtual machine and return processed I / O results; virtio spec refers to the data exchange protocol between the frontend and backend virtio drivers.

[0057] epoll: epoll stands for eventpoll (event polling mechanism), which is an implementation of IO multiplexing in the Linux kernel. IO multiplexing means monitoring multiple input and output sources simultaneously in one operation, returning when one or more of the input and output sources are available, and then performing read and write operations on them.

[0058] RPC (Remote Procedure Call) is a communication protocol used for communication between different processes in a distributed system. It allows one process to call the procedures or functions of another process over the network, just like calling local functions, making communication between different nodes in a distributed system simpler and more convenient. RPC uses an underlying data transmission protocol (such as TCP (Transmission Control Protocol) or UDP (User Datagram Protocol)) to transmit data and is typically implemented using specific programming languages ​​and frameworks.

[0059] ARPC is a high-performance RPC framework based on protobuf (Protocol Buffers) and anet. Protobuf is a data serialization format that is efficient, scalable, and cross-language. anet is a network communication library that provides TCP / UDP socket encapsulation and supports asynchronous operations.

[0060] SDK (Software Development Kit): A software development kit is usually provided by the manufacturer of a hardware platform, operating system, or other third-party tools, in the form of an interface provided by users to developers.

[0061] SPDK (Storage Performance Development Kit) uses user-space drivers to eliminate the CPU processing overhead caused by traditional kernel drivers and delivers high throughput and low latency by effectively utilizing modern multi-core CPUs. SPDK also provides common libraries and drivers for rapid prototyping and performance optimization across high-speed storage technologies and novel storage protocols.

[0062] fd (File Descriptor): It is the operating system's unique identifier for files and other I / O resources. Users can use fd to identify a file opened by a process.

[0063] Remote mode: In this mode, the submission and execution of computing requests are not on the same node.

[0064] Example system

[0065] One embodiment of this specification provides a computing resource provision system, as shown in FIG3 , including: a management and control module 310 and a data module 320; wherein,

[0066] The management and control module 310 is configured to respond to a computing request, send the computing request to the virtual computing device, and receive a computing result returned by the virtual computing device;

[0067] The data module 320 is used to establish the virtual computing power device; the virtual computing power device includes a user module, and the user module includes a virtual front-end unit 321 and a virtual back-end unit 322, both of which run in user state; the virtual front-end unit 321 is used to receive the computing request and save the computing request to the shared memory based on the paravirtualization protocol; the virtual back-end unit 322 is used to mount virtual computing resources, receive the computing request from the shared memory based on the paravirtualization protocol, and send the computing request to the virtual computing resource; the computing request is used to request the virtual computing resource to perform a computing task; the virtual computing resources include at least one of computing power and video memory.

[0068] The computing resource provisioning system can be implemented on a cloud platform, which can include K8s, a distributed system consisting of multiple nodes that manages and schedules containerized applications through collaboration and communication between the nodes. In the K8s architecture, each node has its own container runtime environment and K8s agent, which communicate and collaborate over a cluster network to manage and schedule containers.

[0069] A virtual computing device may refer to a device that mounts virtual computing resources for users, thereby providing users with virtual computing power and assisting users in completing computing tasks. The "virtual" in a virtual computing device means that the application does not directly access the physical hardware (physical devices, such as computing chips), but instead accesses a fake, virtual device. In some embodiments of this specification, a virtual computing device may refer to a set of user-mode APIs (Application Programming Interfaces) that have interface forms such as opening and closing devices, operating devices, etc.

[0070] User state can refer to the part of a computer system that runs user applications, also known as user space.

[0071] Para-virtualization, also known as paravirtualization technology or quasi-virtualization, builds upon full virtualization by adding a specialized API to optimize user instructions. This protocol eliminates the need for the hypervisor to expend resources for translation, reducing the workload of the virtual machine and improving overall performance. By adding a specialized API to optimize instructions and sharing system hardware drivers, the para-virtualization protocol can replace hardware functionality with software interfaces.

[0072] Mounting may refer to the operation of establishing a mapping between devices. In some feasible implementations, the mapping between the device to be mounted and the target device may be established by connecting the directory of the device to be mounted to a directory of the target device, and access to the device to be mounted may be achieved by accessing the directory where the device to be mounted is located in the target device. In some feasible implementations, the process of mounting a virtual computing resource to a virtual backend unit may include: establishing a mapping between the virtual computing resource and the virtual backend unit, so that the virtual backend unit can access the virtual computing resource (such as computing power or video memory) based on the established mapping.

[0073] In a virtual computing device, the virtual front-end unit 321 may refer to a user container based on a paravirtualization protocol (e.g., Virtio), and the virtual back-end unit 322 may refer to a sidecar container based on a paravirtualization protocol (e.g., Virtio). In some embodiments, the virtual back-end unit 322 may also be referred to as a vXPU Service (virtual computing service) container. In some embodiments, the virtual front-end unit 321 and the virtual back-end unit 322 together constitute a user pod. In some embodiments, the virtual front-end unit 321 and the virtual back-end unit 322 may run on the same node. The virtual back-end unit 322 may access the local computing chip (e.g., submit computing requests or obtain computing results) by communicating with the AI ​​runtime running locally. The AI ​​runtime may include a framework or software system (such as an API (Application Programming Interface) and a library) that interacts with the hardware device's computing chip. It may receive requests from the virtual back-end unit 322, pass the requests to the underlying hardware for execution, and ultimately return the computing results to the application. In some embodiments, in order to utilize the virtual computing resources provided by the computing chips in other nodes, the virtual computing power device may further include a network receiving unit 323, which runs in a different node from the user pod. The node where the network receiving unit 323 runs may be a heterogeneous resource node. In the K8s framework, the network receiving unit 323 may be included in the server pod and is the main body for accessing the computing chip in remote mode. It is responsible for receiving user data sent by the computing node running the virtual backend unit 322 over the network, submitting computing tasks to the lower-level AI runtime, receiving computing results, and returning them to the virtual backend unit 322. The network receiving unit 323 may also be called a vXPU server (virtual computing power server) container.

[0074] The control module 310 may include a node in the control plane of the cloud platform. In some embodiments, the control module 310 may run on a node in the K8s architecture. The control module 310 may include a scheduler 311 and a controller 312. The scheduler 311 may be an extended K8s component that, after receiving a user pod creation request, determines the node where the user pod is located based on the scheduling policy and the resource status in the cluster. The controller 312 may be responsible for monitoring the user pod and managing the life cycle of the user pod. The data module 320 may include a node in the data plane of the cloud platform.

[0075] As previously mentioned, in addition to computing requests being transmitted to virtual computing resources via the virtual computing power device, computing results can also be returned to the requester of the computing request via the virtual computing power device. That is, in some embodiments, the virtual backend unit 322 is further configured to receive computing results returned by the target computing resource after executing the computing task and save the computing results to the shared memory; the virtual frontend unit 321 is further configured to read the computing results from the shared memory.

[0076] In this embodiment, the computing resource provisioning system uses paravirtualization technology to receive and transmit computing requests within virtual computing devices. This allows computing requests to be transmitted via shared memory from a virtual front-end unit 321 running in user mode to a virtual back-end unit 322 that mounts virtual computing resources. This eliminates the need for data copying between user mode and kernel mode. This significantly reduces the number of data copies required, improves data transmission efficiency, and minimizes performance loss when providing computing resources.

[0077] In addition, the computing resource provision system separates the virtual backend unit 322 from the virtual front-end unit 321, and can integrate functions such as data reception, forwarding, and network communication into the virtual backend unit 322. By providing SDK to users, data reception, forwarding, and network communication are hidden, thereby minimizing intrusion into the user container (virtual front-end unit 321).

[0078] In some embodiments, the management and control module 310 can achieve communication with the virtual computing device and mounting of virtual computing resources through the device plug-in 324. In this embodiment, the device plug-in 324 may include a virtual device plug-in and a physical device plug-in. Among them, the virtual device plug-in can be responsible for interacting with the virtual computing device and injecting necessary external information (such as computing requests, configuration information of the virtual computing device, etc.) into the virtual computing device. The physical device plug-in loads the virtual computing resources provided by the computing device and mounts the virtual computing device, so that the virtual computing device can access the virtual computing resources normally.

[0079] In order to improve the throughput of the system, in some embodiments, as shown in FIG4 , the calculation request includes a callback function; the callback function is used to return the calculation result to the requester of the calculation request after the calculation task is completed;

[0080] The control module 310 responds to the computing request and sends the computing request to the virtual computing device for:

[0081] In response to the computation request, return a response parameter to the requester of the computation request; the response parameter is used to describe whether the response to the computation request is normal;

[0082] The calculation request carrying the callback function is sent to the virtual computing power device, so that after the calculation task is completed, the virtual computing power device returns the calculation result to the requester of the calculation request by calling the callback function.

[0083] In this embodiment, to reduce the response time of the virtual computing device's data interface, upon receiving a computation request, a response parameter is returned to the requester of the computation request, without blocking the user thread. The computation result is returned via a callback function, allowing the user thread (i.e., the thread that sent the computation request) to perform other tasks, thereby improving system throughput. Furthermore, the time required for a computing chip to execute a computation task is uncertain. Using the asynchronous mechanism provided by this embodiment can avoid blocking the entire communication link of the virtual computing device, thereby improving the throughput of the virtual computing device itself.

[0084] As shown in Figure 4, the interface between the virtual computing device and the user is asynchronous. The user thread returns directly after initiating a request. Before the calculation result is returned, the user's CPU can be used to process other tasks. The interface between the virtual computing device and the AI ​​runtime is asynchronous. After the virtual computing device submits a computing task to the computing chip through the AI ​​runtime, it will not block the thread and can continue to process other computing requests. When the computing task is completed, the virtual computing device is notified through the callback function. In addition, in some embodiments, the interface between the containers within the virtual computing device can also be designed to be asynchronous. That is, after the virtual front-end unit 321 (i.e., the user container) sends the computing request and callback function to the virtual back-end unit 322, the virtual back-end unit 322 sends a response parameter (response) to the virtual front-end unit 321 to release the thread of the virtual front-end unit 321. In an embodiment with a network receiving unit 323, after the virtual back-end unit 322 sends the computing request and callback function to the network receiving unit 323, the network receiving unit 323 sends a response parameter (response) to the virtual back-end unit 322 to release the thread of the virtual back-end unit 322. After the virtual computing resources run the computing task and obtain the computing result (data), the computing result (data) is returned to the virtual front-end unit 321 through the callback function of the network receiving unit 323 or the virtual back-end unit 322, and the computing result (data) is returned to the user (i.e., the requester of the computing request) through the callback function running in the virtual front-end unit 321.

[0085] In order to improve the throughput of the virtual computing device, in one embodiment of the present specification, referring to FIG5 , the virtual front-end unit 321 includes a first balancing module, a plurality of user threads, and a plurality of first shared memory threads;

[0086] The virtual backend unit 322 includes a second balancing module, multiple second shared memory threads and multiple network forwarding threads; wherein,

[0087] The user thread is used to receive the computing request;

[0088] The first shared memory thread is used to send data to the shared memory and receive data from the shared memory;

[0089] The second shared memory thread is used to send data to the shared memory and receive data from the shared memory;

[0090] The network forwarding thread is used to send data to the mounted virtual computing resource and receive data from the virtual computing resource;

[0091] The first balancing module is configured to select a target user thread and a target first shared memory thread from the plurality of user threads and the plurality of first shared memory threads, and establish a mapping between the target user thread and the target first shared memory thread;

[0092] The second balancing module is configured to select a target network forwarding thread and a target second shared memory thread from the plurality of network forwarding threads and the plurality of second shared memory threads, and establish a mapping between the target network forwarding thread and the target second shared memory thread.

[0093] In this embodiment, in the virtual computing power device, mapping between threads is achieved through a first balancing module and a second balancing module. The first balancing module in the virtual front-end unit 321 is used for mapping between user threads and the first shared memory thread, and the second balancing module in the virtual back-end unit 322 is used for mapping between the second shared memory thread and the network forwarding thread. In the virtual computing power device, the mapping established between different threads through the balancing module (first balancing module) can be called a channel of the virtual computing power device. In some embodiments, the selection of the channel can be achieved by inputting a channel identifier (context) into the balancing module. Specifically, when the balancing module is configured for minimum response time mapping through the channel identifier, the balancing module will select the channel with the least current pressure to establish the mapping. When the hash mapping method is selected through the channel identifier, the channels corresponding to different channel identifiers are fixed channels.

[0094] In this embodiment, by designing a multi-threaded model in the virtual computing power device, the virtual computing power device can fully utilize the multi-core server, process more computing requests at the same time, and improve the performance of the virtual computing power device.

[0095] In some embodiments, the data module 320 includes a plurality of nodes;

[0096] The control module 310 is further configured to determine a target node from the plurality of nodes in response to a device creation request carrying configuration information, wherein the configuration information includes an operating mode of the virtual computing device;

[0097] Create the virtual computing device at the target node according to the configuration information.

[0098] In this embodiment, users are supported to create virtual computing devices. After receiving the device creation request, the management and control module 310 can determine the target node among multiple nodes based on load balancing and other algorithms, and thus create a virtual computing device in the target node to fully utilize the cluster performance.

[0099] In addition, the configuration information may include the operating mode of the virtual computing device, so that users can use the configuration information to flexibly configure the created virtual computing device.

[0100] Optionally, the operating mode includes a local mode or a remote mode; when the operating mode includes the remote mode, the virtual computing power device further includes a network receiving unit 323;

[0101] The control module 310, in response to a device creation request carrying configuration information, determines a target node specifically for:

[0102] If the operation mode includes a local mode, determining a computing node including a computing chip as the target node; the computing chip is used to provide the virtual computing resource;

[0103] If the operation mode includes a remote mode, determining a computing node and a heterogeneous resource node as the target node, the computing node and the heterogeneous resource node are different nodes; the heterogeneous resource node includes the computing chip;

[0104] The creating the virtual computing device at the target node according to the configuration information includes:

[0105] If the operation mode includes the local mode, the virtual front-end unit 321 and the virtual back-end unit 322 are created on the computing node according to the configuration information, and the virtual back-end unit 322 is used to mount the virtual computing resources on the computing node;

[0106] If the operating mode includes a remote mode, the virtual front-end unit 321 and the virtual back-end unit 322 are created on the computing node according to the configuration information, and a network receiving unit 323 is created on the heterogeneous resource node. The virtual back-end unit 322 mounts the virtual computing resources on the heterogeneous resource node through the network receiving unit 323.

[0107] In this embodiment, the operation mode may include a local mode and a remote mode, thereby achieving reasonable utilization of virtual computing resources on different nodes.

[0108] Optionally, in some embodiments, the operating mode includes at least one of an adaptive computing power mode and a fixed computing power mode, the configuration information further includes demand information, and the demand information includes at least one of a required computing power value and a required video memory value. The management and control module 310 creates the virtual computing power device at the target node according to the configuration information, specifically for:

[0109] According to the demand information, the virtual computing device is created, and the virtual computing resources corresponding to the demand information are mounted on the virtual backend unit 322.

[0110] In fixed computing power mode, users can configure the computing request by specifying the required computing power and video memory values ​​in the YAML configuration file, without specifying the specific type and quantity of computing chips, which helps to simplify the user's configuration difficulty.

[0111] In the adaptive computing power mode, users do not need to specify the required computing power and video memory values ​​in the YAML configuration file. After the user pod is created, the controller 312 will immediately create a virtual backend unit 322, but will not create a user container (virtual front-end unit 321). Instead, after evaluating the specific computing tasks carried in the user's computing request, it will dynamically allocate virtual computing resources based on the evaluation results. When the user applies for computing power, the controller 312 and the scheduler 311 cooperate to complete the selection of heterogeneous resource nodes, the creation of the network receiving unit 323, and the segmentation and mounting of virtual computing resources.

[0112] In a specific embodiment, a feasible implementation framework of a virtual computing device is provided. As shown in FIG6 , the virtual front-end unit 321 and the virtual back-end unit 322 can run on a computing node, and the network receiving unit 323 can run on a heterogeneous resource node.

[0113] The virtual front-end unit 321 can expose an interface to the outside world in the form of a dynamic library, providing a user interface that allows the user to configure and generate computing requests through the interface. The dynamic library may include two modules: a user adapter module and a virtio front-end module. The user adapter module is responsible for converting interfaces such as the control path interface and the data path interface into block device read and write interfaces in the virtio front-end module. The control path interface provides the ability to configure the virtual computing power device and obtain device attributes, and the data path interface provides the ability to receive the results of the submitted computing tasks. The virio front-end module is responsible for write and read operations (write / read) in the shared memory. The virtio bdev module in this module may include an epoll loop, which is responsible for saving the metadata of user commands (including control commands and data commands) to the shared memory and informing the virtual computing power device to process and forward data.

[0114] The virtual backend module can include a virtio backend module and a network forwarding module. The virtio backend module also includes an epoll loop, which is responsible for receiving user commands (such as computing requests) from the shared memory area, and also includes a custom virtual block device layer, which can parse the commands sent by the virtio frontend and convert them back into management commands and data commands. The network forwarding module is an RPC client that is compatible with both TCP and RDMA network protocols. It is responsible for sending user data to the remote end. When the virtual computing power device works in local mode, user data goes directly to the hardware without passing through the network stack. The user information of the virtual host (vhost user message) can be transmitted through the unix domain socket (Unix domain socket, UDS) of the virtual backend unit 322 and the virtio frontend of the virtual frontend unit 321.

[0115] The network receiving unit 323 and the virtual backend unit 322 are connected to the network (Network) through the network interface (INC) for data transmission. The network receiving unit 323 may include a network receiving module, which may be an RPC server. Depending on the type of computing request, the AI ​​runtime interface is called to obtain virtual computing resource information or submit computing tasks.

[0116] The following describes a feasible initialization process for virtual computing devices, which may include:

[0117] S11. The control module 310 determines whether it is in local mode or computing power adaptive mode based on the configuration information;

[0118] S12. If none of them is true, the network receiving unit 323 is created, the RPC server is initialized, and the listening port is started; otherwise, this stage is skipped and the process proceeds directly to step S13.

[0119] S13. The control module 310 creates a user Pod, where the virtual backend unit 322 is first started to create a virtio backend process;

[0120] S14. The virtio backend creates a socket file, a virtual block device layer, and a controller 312;

[0121] S15. The virtio backend process obtains the operating mode and computing power and memory information of the virtual computing power device, and determines whether it is in local mode or computing power adaptive mode. If neither is true, the network receiving unit 323 is connected; otherwise, this stage is skipped and the process goes directly to step S16;

[0122] S16. The virtio front-end process starts, applies for huge page memory as the shared memory area, completes a handshake with the virtio back-end through the socket, negotiates the shared memory address range, queue depth and number, and establishes a data channel.

[0123] In one embodiment, an exemplary data flow of a virtual computing device is also provided, which may specifically include:

[0124] S21. The user applies for huge page memory and writes the input data into the huge page memory.

[0125] S22. The virtio front-end selects a data channel (e.g., an SPDK channel) based on the channel identifier specified by the user.

[0126] S23. The virtio front-end writes fd to notify the virtio back-end that the command has been written;

[0127] S24.virtio backend monitors fd and obtains the command sent by the frontend from the shared memory;

[0128] S25. The virtio backend determines the current operating mode. If it is local mode, it submits computing tasks to the virtual computing resources through the AI ​​runtime and obtains the computing results.

[0129] If it is remote mode, the virtio backend selects a network channel based on the context and sends the command.

[0130] S27. The network receiving unit 323 receives the command and submits the computing task to the hardware through the AI ​​runtime;

[0131] S28. The network receiving unit 323 returns the calculation result via the network;

[0132] S29. The virtio backend process puts the result into shared memory and writes fd to notify the frontend that the result has been returned;

[0133] S210.virtio front-end monitors fd and retrieves the calculation results.

[0134] In one embodiment, a feasible process for using a virtual computing device is also provided, which may specifically include:

[0135] S31. Start;

[0136] S32. The user calls the open virtual computing device interface and obtains a handle representing the virtual computing device;

[0137] S33. The user calls the management interface of the virtual computing device to obtain the attribute information of the virtual computing device, which includes the current operating mode of the virtual computing device (computing adaptive mode or fixed computing mode), as well as computing power and video memory information;

[0138] S34. The user determines whether the current computing power is in adaptive mode.

[0139] If S34.1 is in computing power adaptive mode, determine whether the current computing power and video memory meet user requirements;

[0140] S34.1.1 If the computing power and video memory do not meet the user's needs;

[0141] S34.1.1.1 Call the management and control interface of the virtual computing power device to release the currently allocated computing power;

[0142] S34.1.1.2 Call the computing power device's management and control interface, specify the minimum computing power and video memory required by the user, and apply for computing power;

[0143] S34.1.2 If the computing power and video memory meet the user's needs, skip the process of releasing computing power and applying for computing power;

[0144] If S34.2 is in fixed computing power mode, skip the computing power information determination process;

[0145] S35. The user calls the management interface of the virtual computing device and creates a channel identifier for the device to send a computing request;

[0146] S36. The user calls the management interface of the virtual computing device to apply for memory on the computing node side. The memory space size includes the size of the data to be transmitted and the data to be received, and writes the data to be transmitted to the specified memory;

[0147] S37. The user calls the data interface of the virtual computing device, and the device forwards the data to the virtual computing resource. After the calculation is completed, the result is returned to the computing node side;

[0148] S38. After the user obtains the calculation results of the computing task from the specified memory, he calls the management interface of the virtual computing device to release the unused memory;

[0149] S39. The user calls the management interface of the virtual computing device to destroy the channel identifier of the device;

[0150] S310. The user determines the operating status of the device;

[0151] If S310.1 is in computing power adaptive mode, call the management and control interface of the virtual computing power device to release computing power;

[0152] If S310.2 is in fixed computing power mode, skip the process of releasing computing power;

[0153] S311. The user calls to close the virtual computing device interface;

[0154] S312. End.

[0155] The above steps are the complete process from opening to closing the virtual computing power device. The order of the above steps can be adjusted according to the actual needs of the user. For example, when the user needs to submit computing tasks repeatedly, steps S35 to S38 can be repeated.

[0156] Exemplary Methods

[0157] Accordingly, the embodiments of this specification also provide a method for providing computing resources, as shown in FIG7 , including:

[0158] S701: In response to a computing request, the computing request is sent to a virtual computing device, the virtual computing device including a user module, the user module including a virtual front-end unit and a virtual back-end unit both running in user mode; the virtual front-end unit is used to receive the computing request and save the computing request to a shared memory based on a paravirtualization protocol; the virtual back-end unit is used to mount a virtual computing resource, receive the computing request from the shared memory based on the paravirtualization protocol, and send the computing request to the virtual computing resource; the computing request is used to request the virtual computing resource to perform a computing task; the virtual computing resource includes at least one of computing power and video memory;

[0159] S702: Receive the calculation result returned by the virtual computing power device.

[0160] In an optional embodiment, the calculation request includes a callback function; the callback function is used to return the calculation result to the requester of the calculation request after the calculation task is completed;

[0161] The sending, in response to the computing request, the computing request to the virtual computing power device comprises:

[0162] In response to the computation request, return a response parameter to the requester of the computation request; the response parameter is used to describe whether the response to the computation request is normal;

[0163] The calculation request carrying the callback function is sent to the virtual computing power device, so that after the calculation task is completed, the virtual computing power device returns the calculation result to the requester of the calculation request by calling the callback function.

[0164] In an optional embodiment, the virtual backend unit is further used to receive the calculation result returned by the target computing resource when executing the computing task, and save the calculation result to the shared memory; the virtual front-end unit is further used to read the calculation result from the shared memory;

[0165] The receiving of the calculation result returned by the virtual computing power device includes:

[0166] The calculation result read by the virtual front end unit from the shared memory is received.

[0167] In an optional embodiment, the method for providing computing resources further includes:

[0168] In response to a device creation request carrying configuration information, determining a target node; the configuration information includes an operating mode of the virtual computing power device;

[0169] Create the virtual computing device at the target node according to the configuration information.

[0170] In an optional embodiment, the operating mode includes a local mode or a remote mode; when the operating mode includes the remote mode, the virtual computing power device further includes a network receiving unit;

[0171] The determining of the target node in response to the device creation request carrying the configuration information includes:

[0172] If the operation mode includes a local mode, determining a computing node including a computing chip as the target node; the computing chip is used to provide the virtual computing resource;

[0173] If the operation mode includes a remote mode, determining a computing node and a heterogeneous resource node as the target node, the computing node and the heterogeneous resource node are different nodes; the heterogeneous resource node includes the computing chip;

[0174] The creating the virtual computing device at the target node according to the configuration information includes:

[0175] If the operation mode includes the local mode, creating the virtual front-end unit and the virtual back-end unit on the computing node according to the configuration information, the virtual back-end unit being used to mount the virtual computing resources on the computing node;

[0176] If the operating mode includes a remote mode, the virtual front-end unit and the virtual back-end unit are created on the computing node according to the configuration information, and a network receiving unit is created on the heterogeneous resource node. The virtual back-end unit mounts the virtual computing resources on the heterogeneous resource node through the network receiving unit.

[0177] In an optional embodiment, if the operating mode includes a fixed computing power mode, the configuration information further includes demand information, the demand information includes at least one of a required computing power value and a required video memory value, and creating the virtual computing power device at the target node according to the configuration information includes:

[0178] According to the demand information, the virtual computing device is created, and the virtual computing resources corresponding to the demand information are mounted on the virtual backend unit.

[0179] The method for providing computing resources provided in this embodiment is based on the same concept as the computing resource provision system provided in the above-mentioned embodiments of this application, and has the functional steps and beneficial effects of the corresponding functional modules of the computing resource provision system. For technical details not fully described in this embodiment, please refer to the specific processing content of the computing resource provision system provided in the above-mentioned embodiments of this application, and will not be repeated here.

[0180] Example electronic devices, storage media, and software

[0181] Another embodiment of the present application further proposes an electronic device, as shown in Figure 8. An exemplary embodiment of this specification also provides an electronic device, including: a memory and a processor, the memory storing a computer program, and the processor executing the computer program when executing the computer program performs the steps of the method for providing computing resources according to various embodiments of this specification described in the above embodiments of this specification.

[0182] The internal structure of the electronic device can be shown in Figure 8. The electronic device includes a processor, a memory, a network interface and an input device connected via a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the central control device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it follows the steps in the method for providing computing resources according to various embodiments of this specification as described in the above embodiments of this specification.

[0183] The processor may include a main processor, and may also include a baseband chip, a modem, etc.

[0184] The memory stores a computer program that executes the technical solution of the present invention, and may also store an operating system and other key programs. Specifically, the computer program may include program code, which includes computer operating instructions. More specifically, the memory may include read-only memory (ROM), other types of static storage devices that can store static information and instructions, random access memory (RAM), other types of dynamic storage devices that can store information and instructions, disk storage, flash memory, etc.

[0185] The processor may be a general-purpose processor, such as a general-purpose processor (CPU), a microprocessor, or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware components.

[0186] Input devices may include devices that receive data and information input by a user, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0187] Output devices may include means that allow information to be output to a user, such as display screens, printers, speakers, and the like.

[0188] The communication interface may include any device such as a transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0189] The processor executes the computer program stored in the memory and calls other devices, which can be used to implement the various steps of any method for providing computing resources provided in the above embodiments of the present application.

[0190] The electronic device may further include a display component and a voice component. The display component may be a liquid crystal display or an electronic ink display. The input device of the electronic device may be a touch layer covering the display component, or a button, trackball or touchpad provided on the housing of the electronic device, or an external keyboard, touchpad or mouse.

[0191] Those skilled in the art will understand that the structure shown in FIG8 is merely a block diagram of a portion of the structure related to the scheme of this specification, and does not constitute a limitation on the electronic device to which the scheme of this specification is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0192] Exemplary computer program products and computer-readable storage media

[0193] In addition to the above-mentioned methods and devices, the method for providing computing resources provided in the embodiments of this specification may also be a computer program product, which includes a computer program, which, when executed by a processor, enables the processor to execute the steps of the method for providing computing resources according to various embodiments of this specification described in the above-mentioned "Exemplary Method" section of this specification.

[0194] The computer program product may be written in any combination of one or more programming languages ​​to implement the operations of the embodiments of this specification, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0195] In addition, an embodiment of this specification also provides a computer-readable storage medium on which a computer program is stored, and the computer program is used by a processor to execute the steps of the method for providing computing resources according to various embodiments of this specification described in the above "Exemplary Method" section of this specification.

[0196] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0197] It should be understood that the specific examples in this document are only intended to help those skilled in the art better understand the implementation methods of this specification, and are not intended to limit the scope of this specification.

[0198] It can be understood that in the various implementations of this specification, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the implementation methods of this specification.

[0199] It can be understood that the various embodiments described in this specification can be implemented individually or in combination, and the embodiments in this specification are not limited to this.

[0200] Unless otherwise indicated, all technical and scientific terms used in the embodiments of this specification have the same meaning as those commonly understood by those skilled in the art in the technical field of this specification. The terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the scope of this specification. The term "and / or" used in this specification includes any and all combinations of one or more related listed items. The singular forms "a", "above", and "the" used in the embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise.

[0201] It is understood that the processor in the embodiments of this specification can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiment can be completed by hardware integrated logic circuits in the processor or software instructions. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this specification can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this specification can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0202] It will be understood that the memory in the embodiments of this specification may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (Programmable ROM, PROM), an erasable programmable read-only memory (Erasable PROM, EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory may be a random access memory (RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0203] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this specification.

[0204] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, devices and units can refer to the corresponding processes in the aforementioned method implementation methods and will not be repeated here.

[0205] In the several embodiments provided in this specification, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0206] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of this embodiment.

[0207] In addition, each functional unit in each embodiment of this specification may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0208] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this specification, or the part that contributes to the prior art, or the part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of this specification. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0209] The above description is merely a specific embodiment of this specification, but the scope of protection of this specification is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this specification should be included in the scope of protection of this specification. Therefore, the scope of protection of this specification should be based on the scope of protection of the claims.

Claims

1. A method for providing computing resources, characterized in that: include: In response to a computing request, the computing request is sent to a virtual computing device, the virtual computing device includes a user module, the user module includes a virtual front-end unit and a virtual back-end unit both running in user mode; the virtual front-end unit is used to receive the computing request and save the computing request to a shared memory based on a paravirtualization protocol; The virtual backend unit is used to mount the virtual computing resource, receive the computing request from the shared memory based on the paravirtualization protocol, and send the computing request to the virtual computing resource; The computing request is used to request the virtual computing resource to perform a computing task; The virtual computing resources include at least one of computing power and video memory; Receive the calculation result returned by the virtual computing power device.

2. The method according to claim 1, characterized in that The calculation request includes a callback function; the callback function is used to return the calculation result to the requester of the calculation request after the calculation task is completed; The sending, in response to the computing request, the computing request to the virtual computing power device comprises: In response to the computing request, returning a response parameter to the requester of the computing request; the response parameter is used to describe whether the response to the computing request is normal; The computing request carrying the callback function is sent to the virtual computing power device, so that after the computing task is completed, the virtual computing power device returns the computing result to the requester of the computing request by calling the callback function.

3. The method according to claim 1, characterized in that The virtual backend unit is further used to receive the calculation result returned by the target computing resource when executing the computing task, and save the calculation result to the shared memory; The virtual front end unit is further used to read the calculation result from the shared memory; The receiving the calculation result returned by the virtual computing power device includes: The calculation result read by the virtual front end unit from the shared memory is received.

4. The method according to claim 1, characterized in that: Also includes: In response to a device creation request carrying configuration information, determining a target node; the configuration information includes an operation mode of the virtual computing power device; The virtual computing device is created at the target node according to the configuration information.

5. The method according to claim 4, characterized in that The operation mode includes a local mode or a remote mode; when the operation mode includes the remote mode, the virtual computing power device further includes a network receiving unit; The determining of the target node in response to the device creation request carrying the configuration information comprises: If the operation mode includes a local mode, determining a computing node including a computing chip as the target node; the computing chip is used to provide the virtual computing resource; If the operation mode includes a remote mode, determining a computing node and a heterogeneous resource node as the target node, the computing node and the heterogeneous resource node are different nodes; the heterogeneous resource node includes the computing chip; The creating the virtual computing device at the target node according to the configuration information includes: If the operation mode includes a local mode, creating the virtual front-end unit and the virtual back-end unit on the computing node according to the configuration information, the virtual back-end unit being used to mount the virtual computing resources on the computing node; If the operating mode includes a remote mode, the virtual front-end unit and the virtual back-end unit are created on the computing node according to the configuration information, and a network receiving unit is created on the heterogeneous resource node. The virtual back-end unit mounts the virtual computing resources on the heterogeneous resource node through the network receiving unit.

6. The method according to claim 4, characterized in that If the operation mode includes a fixed computing power mode, the configuration information further includes demand information, the demand information includes at least one of a required computing power value and a required video memory value, and creating the virtual computing power device at the target node according to the configuration information includes: According to the demand information, the virtual computing device is created, and the virtual computing resources corresponding to the demand information are mounted on the virtual backend unit.

7. A computing resource provision system, characterized in that: include: Control module and data module; among them, The control module is used to respond to a computing request, send the computing request to the virtual computing device, and receive the computing result returned by the virtual computing device; The data module is used to establish the virtual computing power device; the virtual computing power device includes a user module, and the user module includes a virtual front-end unit and a virtual back-end unit both running in user state; the virtual front-end unit is used to receive the computing request and save the computing request to the shared memory based on the paravirtualization protocol; the virtual back-end unit is used to mount the virtual computing resources, receive the computing request from the shared memory based on the paravirtualization protocol, and send the computing request to the virtual computing resource; the computing request is used to request the virtual computing resource to perform a computing task; the virtual computing resources include at least one of computing power and video memory.

8. The system according to claim 7, characterized in that The virtual back-end unit is further used to receive the calculation result returned by the target computing resource when executing the computing task, and save the calculation result to the shared memory; the virtual front-end unit is further used to read the calculation result from the shared memory.

9. The system according to claim 7, characterized in that The virtual front end unit includes a first balancing module, a plurality of user threads and a plurality of first shared memory threads; The virtual backend unit includes a second balancing module, a plurality of second shared memory threads and a plurality of network forwarding threads; wherein, The user thread is used to receive the computing request; The first shared memory thread is used to send data to the shared memory and receive data from the shared memory; The second shared memory thread is used to send data to the shared memory and receive data from the shared memory; The network forwarding thread is used to send data to the mounted virtual computing resource and receive data from the virtual computing resource; The first balancing module is used to select a target user thread and a target first shared memory thread from the plurality of user threads and the plurality of the first shared memory threads, and establish a mapping between the target user thread and the target first shared memory thread; The second balancing module is used to select a target network forwarding thread and a target second shared memory thread from the plurality of network forwarding threads and the plurality of the second shared memory threads, and establish a mapping between the target network forwarding thread and the target second shared memory thread.

10. The system according to claim 7, characterized in that The calculation request includes a callback function; the callback function is used to return the calculation result to the requester of the calculation request after the calculation task is completed; The control module responds to the computing request and sends the computing request to the virtual computing device for: In response to the computing request, returning a response parameter to the requester of the computing request; the response parameter is used to describe whether the response to the computing request is normal; The computing request carrying the callback function is sent to the virtual computing power device, so that after the computing task is completed, the virtual computing power device returns the computing result to the requester of the computing request by calling the callback function.

11. The system according to any one of claims 7 to 10, characterized in that: The data module includes a plurality of nodes; The control module is further configured to determine a target node from the plurality of nodes in response to a device creation request carrying configuration information; the configuration information includes an operation mode of the virtual computing power device; The virtual computing device is created at the target node according to the configuration information.

12. The system according to claim 11, characterized in that The operation mode includes a local mode or a remote mode; when the operation mode includes the remote mode, the virtual computing power device further includes a network receiving unit; The control module responds to the device creation request carrying the configuration information and determines that the target node is specifically used for: If the operation mode includes a local mode, determining a computing node including a computing chip as the target node; the computing chip is used to provide the virtual computing resource; If the operation mode includes a remote mode, determining a computing node and a heterogeneous resource node as the target node, the computing node and the heterogeneous resource node being different nodes; The heterogeneous resource node includes the computing chip; The creating the virtual computing device at the target node according to the configuration information includes: If the operation mode includes a local mode, creating the virtual front-end unit and the virtual back-end unit on the computing node according to the configuration information, the virtual back-end unit being used to mount the virtual computing resources on the computing node; If the operating mode includes a remote mode, the virtual front-end unit and the virtual back-end unit are created on the computing node according to the configuration information, and a network receiving unit is created on the heterogeneous resource node. The virtual back-end unit mounts the virtual computing resources on the heterogeneous resource node through the network receiving unit.

13. An electronic device, characterized in that: include: Processor and memory; Wherein, the memory is connected to the processor, and the memory is used to store a computer program; The processor is used to implement the method for providing computing resources as described in any one of claims 1 to 6 by running the computer program stored in the memory.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for providing computing resources according to any one of claims 1 to 6 is implemented.

Images