Heterogeneous computing power providing system and method based on cloud computing technology

By implementing heterogeneous computing power provision system on the cloud computing platform and dynamically scheduling heterogeneous hardware resources, the problem of inconsistent floating-point computing accuracy requirements in the integration of high-performance computing and artificial intelligence in cloud technology is solved, and computing performance and efficiency are improved.

CN118963981APending Publication Date: 2024-11-15HUAWEI CLOUD COMPUTING TECHNOLOGIES CO LTD
View PDF 0 Cites 10 Cited by

Patent Information

Application Number
CN202310626543.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-05-05
Filing Date
2023-05-30
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

In the field of cloud technology, the integration of high-performance computing and artificial intelligence needs to solve the problem of inconsistent requirements for floating-point computing accuracy, and it is difficult for the operating system or virtualization layer to enable coordinated management of heterogeneous computing power, resulting in challenges in the evolution of computing architecture.

Method used

Provide a heterogeneous computing power supply system based on cloud computing. The node records the computing power of different heterogeneous hardware through the computing power runtime nodes, and dynamically schedules according to the application's requests, providing heterogeneous hardware resources that meet preset requirements, supporting The integration of diverse computing power.

Benefits of technology

It realizes flexible deployment and expansion of diverse computing power equipment, uses appropriate computing power resources as needed, improves computing performance and efficiency, and solves the problem of coordinated management of heterogeneous hardware resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118963981A_ABST
    Figure CN118963981A_ABST
Patent Text Reader

Abstract

The invention discloses a heterogeneous computing power providing system and method based on a cloud computing technology, and relates to the field of cloud technologies, the system comprises an infrastructure which comprises a plurality of servers and a plurality of heterogeneous hardware, and the types of the heterogeneous hardware are different from the types of CPUs of the servers; the computing power runtime node is used for recording heterogeneous computing power which can be provided by heterogeneous hardware of different heterogeneous hardware types; the application program is arranged in one of a plurality of servers of the infrastructure, and sends a heterogeneous computing power request to the computing power runtime node under the condition that the heterogeneous computing power of a preset heterogeneous hardware type needs to be used; and the computing power runtime node is also used for providing target heterogeneous hardware which conforms to the preset heterogeneous hardware type and can provide heterogeneous computing power to the application program for use according to the heterogeneous computing power request.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application claims the priority of the Chinese patent application filed with the Intellectual Property Office of the People's Republic of China on May 5, 2023, with application number 202310497813.5 and application name "A system and method for providing heterogeneous computing power based on cloud computing technology", all contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of cloud technology, and in particular to a system and method for providing heterogeneous computing power based on cloud computing technology. Background Art

[0003] In the past, the cloud technology field often carried a certain type of computing through a software and hardware vertical domain, such as general computing, probabilistic computing, artificial intelligence (AI) computing, physical computing, and graphic computing. However, from the development trend of cloud technology, the computing architecture is evolving from "general computing + special computing" to "diversified computing power integration", but there are also many challenges. Taking the integration of high performance computing (HPC) and AI (expressed as "HPC+AI") as an example, it is necessary to solve the problem that the accuracy requirements of HPC and AI are different. The former has very high requirements for floating-point operation accuracy, while the latter has relatively high requirements for floating-point operation accuracy, and sometimes Int8 and Int16 are used for approximate calculations. This "HPC+AI" requires the operating system to move from vertical domain division to heterogeneous integration, and effectively coordinate scalar, vector and other computing power.

[0004] At the same time, the chip architecture has also evolved from the original central processing unit (CPU)-centric to the current CPU+XPU heterogeneous processor collaboration, and gradually formed a unified memory architecture (UMA) to reduce data migration and effectively alleviate the memory wall and storage wall problems. In addition, storage and computing integration and near-data computing can also improve energy efficiency, that is, computing is done where the data is, such as on smart network cards and smart SSD cards equipped with powerful computing units.

[0005] The evolution of the above applications and hardware requires the operating system or virtualization layer to enable them to support the diversified computing in the era of the Internet of Everything. However, how to achieve the disabling of the operating system or virtualization layer is still an important issue that needs to be solved urgently. Summary of the invention

[0006] The embodiments of the present application provide a heterogeneous computing power provision system and method based on cloud computing technology, which is used to flexibly deploy and expand various computing power devices, and use appropriate computing power resources as needed to achieve computing power acceleration for related services to be implemented by the application.

[0007] In the first aspect, an embodiment of the present application provides a heterogeneous computing power providing system based on cloud computing technology, including: an infrastructure, including multiple servers and multiple heterogeneous hardware, wherein the types of the multiple heterogeneous hardware are different from the types of CPUs of the multiple servers; a computing power runtime node, used to record the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types; an application, which is set in one of the multiple servers of the infrastructure, and sends a heterogeneous computing power request to the computing power runtime node when the heterogeneous computing capabilities of a preset heterogeneous hardware type are needed; the computing power runtime node is also used to provide the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application for use according to the heterogeneous computing power request.

[0008] In one possible design, before the computing power runtime node records the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types, it is also used to: load the dynamic link library files associated with the multiple heterogeneous hardware by calling the registration interface, and record the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types in a preset data structure storage medium.

[0009] In one possible design, the computing power runtime node is also used to obtain the dynamic link library file input or selected by the developers of the multiple heterogeneous hardware on the cloud management platform.

[0010] In one possible design, before the application sends a heterogeneous computing power request to the computing power runtime node, the computing power runtime node is also used to: provide the application with an API call interface of the algorithms supported by the multiple heterogeneous hardware, so that the application sends a heterogeneous computing power request to the computing power runtime node through an API interface call.

[0011] In one possible design, the computing power runtime node is also used to: obtain at least one data access strategy sent by the framework developer of the computing power operation node, the at least one data access strategy includes multiple fields and parameters entered by the developer for each field, the multiple fields are associated with the application programming interface API of the at least one data access strategy, the API is provided by the cloud management platform, and the multiple fields are used to characterize different attributes of the at least one data access strategy.

[0012] In one possible design, the multiple heterogeneous hardware belong to at least one resource queue, and the computing power runtime node provides the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application for use according to the heterogeneous computing power request, including: determining the target resource queue according to the heterogeneous computing power request; and providing the target heterogeneous hardware in the target resource queue that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application for use.

[0013] In one possible design, the computing power runtime node uses any of the following scheduling algorithms to provide the target heterogeneous hardware in the target resource queue that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application for use: first-in-first-out scheduling algorithm; polling scheduling algorithm or lowest load first scheduling algorithm.

[0014] In one possible design, one of the multiple servers of the infrastructure includes any of the following types: a bare metal server, a virtual machine, a container, a lightweight container, or a serverless server.

[0015] In one possible design, the architecture of the heterogeneous computing power providing system includes an offload card, the computing power runtime node is deployed in the offload card, and some or all of the multiple heterogeneous hardware belong to the offload card.

[0016] In one possible design, the heterogeneous computing power system is deployed on a rack server in the same data center, wherein multiple computing nodes in the data center are the computing power runtime nodes, and different computing power runtime nodes communicate and interact with each other through switches.

[0017] In one possible design, the computing runtime nodes and infrastructure cross-rack servers of the heterogeneous computing system are deployed on rack servers in different data centers, and different rack servers communicate and interact with each other through switches.

[0018] In the second aspect, an embodiment of the present application provides a method for providing heterogeneous computing power based on cloud computing technology, which is applied to a computing power runtime node. The computing power runtime node belongs to a heterogeneous computing power providing system based on cloud computing technology. The heterogeneous computing power providing system also includes an infrastructure, which includes multiple servers and multiple heterogeneous hardware. The types of the multiple heterogeneous hardware are different from the types of CPUs of the multiple servers. The computing power runtime node records the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types. The method includes: receiving a heterogeneous computing power request from an application, the heterogeneous computing power request is associated with the heterogeneous computing capabilities of a preset heterogeneous hardware type that the application needs to use; according to the heterogeneous computing power request, the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities is provided to the application for use.

[0019] In a possible design, the method may also include: loading the dynamic link library files associated with the multiple heterogeneous hardware by calling a registration interface, and recording the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types in a preset data structure storage medium.

[0020] In one possible design, the method may further include: obtaining the dynamic link library file input or selected by the developers of the multiple heterogeneous hardware on the cloud management platform.

[0021] In one possible design, the method may also include: providing the application with an API calling interface of the algorithms supported by the multiple heterogeneous hardware, so that the application sends a heterogeneous computing power request to the computing power runtime node through an API interface call.

[0022] In one possible design, the method may also include: obtaining at least one data access strategy sent by a framework developer of the computing power running node, the at least one data access strategy comprising multiple fields and parameters entered by the developer for each field, the multiple fields being associated with an application programming interface API of the at least one data access strategy, the API being provided by a cloud management platform, and the multiple fields being used to characterize different attributes of the at least one data access strategy.

[0023] In one possible design, the multiple heterogeneous hardware belong to at least one resource queue, and the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities is provided to the application for use according to the heterogeneous computing power request, including: determining a target resource queue according to the heterogeneous computing power request; and providing the target heterogeneous hardware in the target resource queue that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application for use.

[0024] In one possible design, the target heterogeneous hardware in the target resource queue that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities can be provided to the application for use by using but not limited to any of the following scheduling algorithms: first-in-first-out scheduling algorithm; polling scheduling algorithm or lowest load first scheduling algorithm.

[0025] In one possible design, one of the multiple servers of the infrastructure includes any of the following types: a bare metal server, a virtual machine, a container, a lightweight container, or a serverless server.

[0026] In one possible design, the architecture of the heterogeneous computing power providing system includes an offload card, the computing power runtime node is deployed in the offload card, and some or all of the multiple heterogeneous hardware belong to the offload card.

[0027] In one possible design, the heterogeneous computing power system is deployed on a rack server in the same data center, wherein multiple computing nodes in the data center are the computing power runtime nodes, and different computing power runtime nodes communicate and interact with each other through switches.

[0028] In one possible design, the computing runtime nodes and infrastructure cross-rack servers of the heterogeneous computing system are deployed on rack servers in different data centers, and different rack servers communicate and interact with each other through switches.

[0029] In the third aspect, an embodiment of the present application provides a heterogeneous computing power providing device based on cloud computing technology, which is applied to a computing power runtime node. The computing power runtime node belongs to a heterogeneous computing power providing system based on cloud computing technology. The heterogeneous computing power providing system also includes an infrastructure, which includes multiple servers and multiple heterogeneous hardware. The types of the multiple heterogeneous hardware are different from the types of CPUs of the multiple servers. The computing power runtime node records the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types. The device includes: a communication interface for receiving a heterogeneous computing power request from an application, wherein the heterogeneous computing power request is associated with the heterogeneous computing capabilities of a preset heterogeneous hardware type that the application needs to use; a processing unit for providing the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application for use according to the heterogeneous computing power request.

[0030] In one possible design, the processing unit is further used to: load the dynamic link library files associated with the multiple heterogeneous hardware by calling a registration interface, and record the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types in a preset data structure storage medium.

[0031] In one possible design, the communication interface is also used to obtain the dynamic link library file input or selected by the developers of the multiple heterogeneous hardware on the cloud management platform.

[0032] In one possible design, the communication interface is also used to provide the application with an API calling interface of the algorithms supported by the multiple heterogeneous hardware, so that the application sends a heterogeneous computing power request to the computing power runtime node through an API interface call.

[0033] In one possible design, the communication interface is also used to: obtain at least one data access strategy sent by the framework developer of the computing power running node, the at least one data access strategy includes multiple fields and parameters entered by the developer for each field, the multiple fields are associated with the application programming interface API of the at least one data access strategy, the API is provided by the cloud management platform, and the multiple fields are used to characterize different attributes of the at least one data access strategy.

[0034] In one possible design, the multiple heterogeneous hardware belong to at least one resource queue, and the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities is provided to the application for use according to the heterogeneous computing power request. The processing unit is used to: determine the target resource queue according to the heterogeneous computing power request; and provide the target heterogeneous hardware in the target resource queue that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application for use.

[0035] In one possible design, the processing unit may adopt but is not limited to any of the following scheduling algorithms to provide the target heterogeneous hardware in the target resource queue that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application for use: first-in-first-out scheduling algorithm; polling scheduling algorithm or lowest load first scheduling algorithm.

[0036] In one possible design, one of the multiple servers of the infrastructure includes any of the following types: a bare metal server, a virtual machine, a container, a lightweight container, or a serverless server.

[0037] In one possible design, the architecture of the heterogeneous computing power providing system includes an offload card, the computing power runtime node is deployed in the offload card, and some or all of the multiple heterogeneous hardware belong to the offload card.

[0038] In one possible design, the heterogeneous computing power system is deployed on a rack server in the same data center, wherein multiple computing nodes in the data center are the computing power runtime nodes, and different computing power runtime nodes communicate and interact with each other through switches.

[0039] In one possible design, the computing runtime nodes and infrastructure cross-rack servers of the heterogeneous computing system are deployed on rack servers in different data centers, and different rack servers communicate and interact with each other through switches.

[0040] In a fourth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed by a computing device cluster, enables the computing device cluster to execute the method described in the first aspect and any possible design of the first aspect.

[0041] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method described in the first aspect and any possible design of the first aspect.

[0042] In a sixth aspect, an embodiment of the present application provides a computing device cluster, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method described in the first aspect above and any possible design of the first aspect.

[0043] Based on the implementations provided in the above aspects, the embodiments of the present application can be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic diagram of a cloud service system according to an embodiment of the present application is shown;

[0045] Figure 2 A schematic diagram of a heterogeneous computing power providing system according to an embodiment of the present application is shown;

[0046] Figure 3 A schematic diagram of a heterogeneous computing power providing system according to an embodiment of the present application is shown;

[0047] Figure 4 A flowchart of a heterogeneous hardware registration method according to an embodiment of the present application is shown;

[0048] Figure 5 A schematic diagram of a heterogeneous computing resource pool according to an embodiment of the present application is shown;

[0049] Figure 6 A schematic diagram showing a process flow of a method for providing heterogeneous computing power according to an embodiment of the present application is shown;

[0050] Figure 7 A schematic diagram showing the principle of a method for providing heterogeneous computing power according to an embodiment of the present application is shown;

[0051] Figure 8 A schematic diagram of the architecture of a heterogeneous computing power provision system deployed on a single machine according to an embodiment of the present application is shown;

[0052] Fig. 9 A schematic diagram showing the principle of a single-machine deployment solution of an embodiment of the present application is shown;

[0053] Fig.10 A schematic diagram of the architecture of a heterogeneous computing power provision system deployed by virtual machines according to an embodiment of the present application is shown;

[0054] Fig.11 A schematic diagram showing the principle of a virtual machine deployment solution according to an embodiment of the present application is shown;

[0055] Fig.12 A schematic diagram showing a cloud system architecture based on an offload card is shown;

[0056] Fig.13 A schematic diagram of the architecture of a heterogeneous computing power provision system based on offload card deployment according to an embodiment of the present application is shown;

[0057] Fig.14 A schematic diagram of the architecture of a heterogeneous computing power provision system deployed in rack servers of a single data center according to an embodiment of the present application is shown;

[0058] Fig.15 A schematic diagram of the architecture of a heterogeneous computing power provision system deployed in rack servers across data centers according to an embodiment of the present application is shown;

[0059] Fig.16 A schematic diagram showing a process flow of a method for providing heterogeneous computing power according to an embodiment of the present application is shown;

[0060] Fig.17 A schematic diagram of a heterogeneous computing power providing device according to an embodiment of the present application is shown;

[0061] Fig.18 A schematic diagram showing a computing device according to an embodiment of the present application is shown;

[0062] Fig.19 A schematic diagram showing a computing device cluster according to an embodiment of the present application is shown;

[0063] Fig. 20 A schematic diagram of a computing device cluster according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0064] In order to better explain the embodiments of the present application, some of the terms or technologies mentioned in the embodiments of the present application are introduced below.

[0065] 1. Cloud service: It is an Internet-based service provided in the form of representational state transfer application programming interface (REST API) to meet the needs of enterprise Internet technology (IT) such as elastic computing, virtual network, data storage, and database.

[0066] In the embodiments of the present application, the provider of cloud services may be referred to as a cloud vendor.

[0067] 2. Cloud resources: The abbreviation for cloud computing resources provided by cloud vendors to tenants. It refers to the integration of cloud computing-related resources, including cloud services and cloud instances.

[0068] Among them, cloud services may include, but are not limited to, services provided by virtual private cloud (VPC), gateway services, firewall services, NAT services, cloud disks, elastic public IP addresses (elastic IP addresses, EIPs), cloud monitoring services, and other cloud services provided by various cloud vendors. Cloud instances may include, but not precede, virtual machines, containers, or bare metal servers, among which virtual machines, containers, or bare metal servers are virtual instances provided by cloud vendors to tenants in their data centers.

[0069] 3. Tenant: The owner of cloud resources is the subject with management rights over cloud resources. Among them, tenants can order or rent cloud resources from cloud vendors in accordance with business needs through payment methods in order to use cloud services implemented based on the cloud resources. The management rights of tenants over purchased cloud resources may include but are not limited to security management rights, access control rights, etc. In some embodiments, the tenant's electronic device may be referred to as a tenant node.

[0070] 4. User: A person who operates cloud resources and can only operate cloud resources after being authorized by the owner of the cloud resources. In the embodiments of the present application, the user's operation on cloud resources can be implemented through the user's electronic device (including physical devices and / or virtual devices). In some embodiments, the user's electronic device can be called a user node.

[0071] It should be noted that in the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0072] Furthermore, unless otherwise specified, the ordinal numbers such as "first" and "second" mentioned in the embodiments of the present application are used to distinguish multiple objects, and are not used to limit the priority or importance of multiple objects. For example, the first device and the second device are only used to distinguish different electronic devices, rather than indicating the difference in priority or importance of the two devices. For example, in some embodiments, the method steps performed by the first device and the second device can be interchangeable.

[0073] The present application is described in detail below with reference to the accompanying drawings and embodiments.

[0074] Figure 1 A schematic diagram of the architecture of a cloud service system according to an embodiment of the present application is shown.

[0075] like Figure 1 As shown, the cloud service system 100 may include a cloud management platform 110 and a backend system 120 provided by a cloud vendor for tenants.

[0076] The cloud management platform 110 can be connected to the user equipment (UE) operated by the tenant through the Internet. The tenant can order cloud services from the cloud vendor through the cloud management platform 110 and configure resources for the purchased cloud services. For example, a tenant can register an account on the cloud management platform 110 through a user device, and the account is eligible to purchase (including different payment forms such as ordering or renting) cloud resources. After successfully purchasing cloud resources, the cloud management platform 110 can notify the corresponding back-end system 120 to create cloud resources for the tenant, and provide appropriate access methods to the tenant and the user authorized by the tenant, so that the tenant can remotely manage cloud resources, and control the tenant / user's remote access or use of cloud resources, to provide at least one cloud service for the tenant and the user.

[0077] For example, a cloud resource is a virtual machine. The tenant can select the specifications of the virtual machine (such as memory, processor, and disk, etc.) on the cloud management platform 110. After the tenant successfully pays, the cloud management platform 110 notifies the corresponding back-end system 120 to create a virtual machine with the specifications. The tenant can configure the access rights of the user authorized by the tenant to the virtual machine (such as logging in to the virtual machine, shutting down the virtual machine, deleting the virtual machine, etc.). After the configuration is successful, the user can perform the corresponding authorized operations on the virtual machine. The tenant can also grant the user the right to purchase or create a virtual machine. It should be understood that in the embodiment of the present application, cloud resources can also be, for example, various cloud services such as containers, bare metal servers, and elastic public IPs (Elastic IP Address, EIP), and the embodiment of the present application does not limit the type of cloud services.

[0078] The cloud management platform 110 can be connected to the corresponding back-end system 120 through the internal network within the cloud server system 110. The back-end system 120 can provide various data processing functions such as access, processing, aggregation, and storage for requests or data from user devices or third-party devices, as well as provide tenants / users with functions such as query and use of stored data, to provide tenants / users with corresponding cloud services.

[0079] In one example, the backend system 120 can be implemented as follows Figure 2 A heterogeneous computing power providing system 200 based on cloud computing technology (hereinafter referred to as the heterogeneous computing power providing system 200) is shown. The heterogeneous computing power providing system 200 may include a computing power runtime node 210 and an infrastructure for providing cloud services.

[0080] Among them, the infrastructure may include a resource pool, such as servers, storage devices, network devices, security devices, virtualization software, storage software and other related hardware and software devices required for computing resources, storage resources, network resources and security resources. The computing power runtime node 210 can realize unified management and scheduling of computing resources, storage resources, network resources and security resources, and provide cloud services for tenants / users. At the same time, the computing power runtime node 210 can also support the security of cloud service operation and maintenance through relatively complete cloud security defense, monitoring, analysis and response, so as to achieve rapid discovery, rapid isolation and rapid recovery of cloud security risks, threats and attacks.

[0081] For example, Figure 2 As shown, the infrastructure may include, for example, a plurality of servers 220 and a plurality of heterogeneous hardware 230 .

[0082] The multiple servers 220 are also referred to as a server cluster, for example, they may include server 220_1, server 220_2, ..., server 220_n, etc., where n is an integer representing the number of servers. The multiple heterogeneous hardware 230 may also be referred to as a heterogeneous hardware cluster or a heterogeneous hardware resource pool, for example, they may include heterogeneous hardware 230_1, heterogeneous hardware 230_2, ..., heterogeneous hardware 230_m, where m is an integer representing the number of heterogeneous hardware.

[0083] The processor of one of the multiple servers 220 may be a general-purpose processor, such as a central processing unit (CPU) or a microprocessor. At least one application (APP) may be provided in one of the multiple servers 220, and the processor and the application may provide cloud services in the form of software calls.

[0084] The processor type of the multiple heterogeneous hardware 230 may be different from the type of the CPU of the multiple servers, and may form a "CPU+XPU" heterogeneous computing architecture with one of the multiple servers 220, and support the deployment of software development kits (SDKs) for some models or algorithms, so that any server can use the multiple heterogeneous hardware and the models or algorithms they support to achieve diverse computing power fusion and computing acceleration to improve the performance of the entire system.

[0085] In specific implementation, the computing power runtime node 210 is located between multiple servers 220 and multiple heterogeneous hardware 230, and can be used to record the heterogeneous computing capabilities that heterogeneous hardware of different heterogeneous hardware types can provide. When an application program set on the server needs to use the heterogeneous computing capabilities of a preset heterogeneous hardware type, it can send a heterogeneous computing power request to the computing power runtime node 210. After receiving the heterogeneous computing power request from the application program, the computing power runtime node 210 can provide the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the corresponding application program according to the heterogeneous computing power request.

[0086] In order to improve the architectural consistency, scalability, versatility, portability and on-demand enabling functions of the heterogeneous computing power providing system 200, one implementation method is to design a computing power runtime framework, which enables the computing power runtime node to face different customer platforms, support cloud resources for virtual machines, bare metal, containers and serverless scenarios, and allow different heterogeneous computing power resources to be registered through an abstract registration interface. When the tenant successfully purchases or rents cloud resources, the computing power runtime framework can create cloud resources for the tenant and build the resource pool required by the tenant according to the tenant's demand information or configuration information. Later, in the application stage, the computing power runtime framework can perform unified scheduling of the received heterogeneous computing power requests based on the resource pool, so as to provide the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the corresponding application for use, so as to realize the relevant business of the application.

[0087] In one example, if Figure 2 As shown, the computing power runtime node 210 can be logically divided into an API layer 211, a Lib library 212, a management service module 213 and a computing power backend gasket 214.

[0088] Among them, the API layer 211 can be used to provide a unified interface specification, support abstraction into a general atomic API interface based on industry standard models, algorithms, etc., so that tenants can quickly build applications and services based on the API layer 211 and the Lib library 212. At the same time, it supports deployment in multiple forms (such as bare metal servers (or bare metal), virtual machines, containers, lightweight containers, serverless, etc.) without the need for tenants / users to pay attention to the underlying hardware. The Lib library 212 can be used in conjunction with the upper API layer 211 for integrated calls to applications set on the server. The computing power backend shim 214 can include multiple types of computing power shims, which can enable the computing power runtime node to support dynamic registration of SDKs of different heterogeneous hardware and different models and algorithms, and can also support virtualized computing power devices.

[0089] The management service module 213 can be used to implement registration management (including registration management in the later computing power expansion stage) and configuration management for different heterogeneous hardware, and establish tasks and scheduling tasks for heterogeneous computing power requests from applications, so as to schedule the heterogeneous computing power requests to the appropriate target heterogeneous hardware. At the same time, the management service module 213 can also provide management such as computing power service quality, traffic statistics, and security in the process of the target heterogeneous hardware processing the heterogeneous computing power request, so as to assist the management service module 213 in realizing load balancing scheduling, accurate billing, security protection and other services.

[0090] For example, Figure 3As shown, the application program set in one of the multiple servers 220 can be used to provide but not limited to at least one of the following services: rendering service, video encoding and decoding service, machine learning service, security service, artificial intelligence (Artificial Intelligence, AI) service, high performance computing (High performance computing, HPC) service, storage service or network service.

[0091] The types of the multiple heterogeneous hardware may include, but are not limited to: CPU, co-processor, graphics processing unit (GPU), field programmable gate array (FPGA), various domain specific architectures (DSA), application-specific integrated circuit (ASIC), virtual computing device (Virtual Computing) or other types of processor engines (expressed as xPU, where x represents unknown).

[0092] The Lib library 212 may include, but is not limited to, library files for implementing operations such as inference, training, encryption and decryption, data plane development tool set (Data Plane Development Kit, DPDK), storage performance development tool set (Storage Performance Development Kit, SPDK), data processor (Data Processing Unit, DPU), and codec.

[0093] The models or algorithms supported by the multiple heterogeneous hardware may include but are not limited to: OpenGL, VAAPI, OpenCL, CUDA, DirectX, TensorFlow, etc. The computing power backend shim may include but are not limited to: OpenGL shim, OpenCL shim, TensorFlow shim, PyTorch shim, virtualized computing power device shim (expressed as VM-shim) and shims of other types of heterogeneous hardware or algorithms.

[0094] The management service module 213 can be logically divided into task modules (tasks), scheduling modules (scheduler), quality of service (QoS) management modules, statistical modules (acoounting), security modules (security), registration modules (register) and configuration modules (configration) based on its functions.

[0095] Among them, the registration module can be used to dynamically register different heterogeneous computing resources (such as heterogeneous hardware) into the computing runtime framework. The registration process may include identification, verification, initialization and configuration of heterogeneous hardware devices to ensure that heterogeneous hardware devices are correctly identified and configured in the heterogeneous computing power provision system so that heterogeneous hardware devices can be used correctly in the heterogeneous computing power provision system.

[0096] The configuration module can provide the parameter setting and control management process of the framework system, software, etc. of the entire computing power runtime node, including but not limited to the configuration of reliability, maintainability, scalability, locatability and fault diagnosis.

[0097] The task module can be managed according to the tenant's business needs, by different algorithm types, task difficulty, etc., so as to assign heterogeneous computing power requests as tasks to appropriate heterogeneous hardware for execution.

[0098] In order to maximize resource utilization and task response time, the scheduling module can design a task scheduling algorithm, which can intelligently schedule heterogeneous computing power request tasks based on factors such as task type, difficulty, and heterogeneous hardware resource pool load. When a task is scheduled to a resource pool or resource queue, the scheduling module also needs to send the task to the corresponding heterogeneous hardware for execution, monitor the task execution status to ensure that the task can be completed on time, and feedback the task execution results to the application.

[0099] The QoS management module can be used to perform reasonable QoS management on heterogeneous computing power request tasks, and can be used in conjunction with the scheduling module to ensure the service experience of tenants / users, and to ensure the performance and reliability of the system.

[0100] The statistics module can perform statistical management based on the heterogeneous computing power request tasks and heterogeneous hardware resource usage received by the computing power runtime node to ensure the normal operation and maintenance of computing power resources. For example, this type of statistical information can be used by cloud vendors to provide detailed bills and other content so that tenants can be charged corresponding fees. Or, for example, this type of statistical information can be used in conjunction with the configuration module to perform debugging actions on heterogeneous hardware or related configuration information in operation and maintenance scenarios.

[0101] The security module can be used to take appropriate security measures, such as data encryption, access control, firewalls, etc., to ensure the security of user data and privacy in the process of computing resource service.

[0102] It should be understood that the above combination Figure 3 The heterogeneous computing power provision system introduced is only an example and not a limitation. In specific implementation scenarios, it is also necessary to combine specific business requirements and technical implementation details to build the heterogeneous computing power provision system required by tenants, so as to achieve the best resource utilization and user experience as much as possible. In some embodiments, cloud vendors can provide different heterogeneous hardware required for general scenarios, and build the framework of the heterogeneous computing power provision system for tenants according to the tenants' ordering requirements. In other implementations, cloud vendors can also provide customized services to tenants, and build personalized heterogeneous computing power provision systems for tenants according to the tenants' business needs. This application embodiment does not limit this.

[0103] Below Figure 3 Taking the heterogeneous computing power providing system shown as an example, the functions of the functional modules and sub-functional units included in the heterogeneous computing power providing system of an embodiment of the present application are introduced.

[0104] 1. Standardization of API interface:

[0105] In an embodiment of the present application, the API layer of the computing power runtime node can provide a unified interface specification, including but not limited to standardizing API interface parameters, requests, responses, etc., to ensure that a unified interface is used on different platforms (such as different tenant platforms) to implement the same function.

[0106] In one implementation, the API layer may implement standardization of different API interfaces through code.

[0107] For example, the API layer can obtain at least one data access strategy sent by the framework developer of the computing power running node. The at least one data access strategy may include multiple fields and parameters entered by the developer for each field. The multiple fields are associated with the application programming interface API of the at least one data access strategy, and the API is provided by the cloud management platform. The multiple fields are used to characterize the different attributes of the at least one data access strategy. Thus, the computing power runtime node can provide the application with API call interfaces for multiple algorithms supported by heterogeneous hardware, so that the application can send heterogeneous computing power requests to the computing power runtime node through API interface calls when it needs to implement computing power acceleration tasks.

[0108] Example (1), taking the encryption and decryption scenario as an example, in the specific implementation, for example, a set of symmetric encryption and decryption interfaces can be defined using C language, including input parameters and output parameters, to support the expansion of various symmetric encryption algorithms, keys, vectors, encryption modes, etc.

[0109] The code example of the symmetric encryption and decryption interface can be as follows:

[0110]

[0111]

[0112] It should be understood that the above is only an example of a code that simply defines a symmetric encryption and decryption interface. In specific implementation, different encryption and decryption algorithms, key lengths, vector lengths, encryption and decryption modes, etc. can be expanded according to actual needs. The specific implementation of computingRuntimeEncrypt and computingRuntimeDecrypt can be based on the selected encryption and decryption library and algorithm, and the management service module can select different heterogeneous hardware for computing power acceleration.

[0113] Example (2), taking the HPC scenario as an example, in the specific implementation, we strive to use C language to define the HPC matrix multiplication interface, including the requirements of parameters such as input matrix, output matrix, matrix dimension, data type and storage format, so as to achieve the standardization of the HPC matrix multiplication interface.

[0114] A code example for this HPC matrix multiplication interface can be shown as follows:

[0115]

[0116] In the above example (2), the computingRuntimeMatrixMultiply() function is a matrix multiplication interface function. Based on this interface function, by passing in pointers to input matrix A, input matrix B, and output matrix C, as well as parameters such as matrix dimension, data type, and storage format, it is possible to support HPC matrix multiplication interfaces of multiple data types and different storage formats, thereby achieving interface standardization. Within the function, corresponding matrix multiplication calculations can be performed according to specific requirements.

[0117] It should be understood that the matrix multiplication implementation code in the above example is only a simplified example. In actual applications, performance optimization and error handling can be performed as needed. The management service module can provide auxiliary processing support, which will not be repeated here.

[0118] Example (3), taking the HPC scenario as an example, in the specific implementation, for example, the HPC reduction operation interface can be defined using C language to meet the requirements of parameters including input array or vector, output result, and operation type, and support multiple data types to achieve normalization.

[0119] The code example of the HPC reduction operation interface is as follows:

[0120]

[0121] In the above example (3), the computingRuntimeReduction() function is a reduction operation interface function. Based on this function, by passing in the pointer of the input array or vector, the pointer of the output result, as well as the size of the array or vector, the data type and the reduction operation type and other parameters, it can support multiple data types and realize interface standardization. Within the function, corresponding reduction operations can be performed according to specific needs, such as summing, finding the maximum value, finding the minimum value, etc.

[0122] The reduction operation implementation code in the above example is only a simplified example. In actual applications, performance optimization and error handling can be performed as needed. The management service module can provide auxiliary processing support, which will not be repeated here.

[0123] In other examples, cloud vendors can write code for standardizing API interfaces in a manner similar to the above examples (1) to (3), supporting the standardization of some standard APIs (such as TensorFlow, PyTorch, DirectX, OpenGL, CUDA, OpenCL, VDPAU, VAAPI, Crypto, etc.) so that tenants or customer platforms can quickly build applications and services, while also speeding up the launch of new hardware by cloud vendors.

[0124] 2. Lib library:

[0125] In the embodiment of the present application, Lib is the abbreviation of Library, and Lib is a library file directory for storing all library files useful to the system. A library file is a file required for the correct execution of an application, command, or process. The role of Lib is similar to that of a DLL file in Windows, and almost all applications need to use the shared library files in the Lib directory. In the embodiment of the present application, the Lib library includes but is not limited to library files for implementing various libraries such as reasoning, training, encryption and decryption, SDPK, DPDK, DPU, encoding and decoding, storage, and network.

[0126] 3. Management service module:

[0127] like Figure 3As shown, the management service module can be logically divided into a task module, a scheduling module, a quality of service (QoS) management module, a statistics module, a security module, a registration module and a configuration module.

[0128] For ease of understanding, the following introduces the implementation details of the functions of each module of the management service module in stages based on the usage sequence of heterogeneous hardware.

[0129] (1) Registration phase for heterogeneous hardware:

[0130] In the embodiment of the present application, the computing power runtime node can be designed with the goal of dynamically registering different heterogeneous computing power resources. Based on this, when designing the computing power runtime framework, for example, different heterogeneous computing power resources (such as heterogeneous hardware) can be used as computing power objects, and the registration interfaces of different computing power objects can be abstracted and compiled into dynamic link library files. The computing power runtime framework dynamically loads the dynamic link library files and uses the execution-related registration code to complete the registration of the corresponding heterogeneous hardware. For example: the computing power runtime framework loads the dynamic link library files associated with the multiple heterogeneous hardware by calling the registration interface, and records the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types in a preset data structure storage medium.

[0131] In specific implementation, the computing power runtime node can obtain dynamic link library files entered or selected by developers of multiple heterogeneous hardware on the cloud management platform. When registering heterogeneous hardware, the computing power runtime framework can actively discover and complete the registration of different heterogeneous computing power resources, or the computing power runtime framework can complete the registration of different heterogeneous computing power resources after receiving online requests from different heterogeneous computing power resources. This embodiment of the application does not limit this.

[0132] In one implementation, the registration interface may be as follows:

[0133]

[0134]

[0135]

[0136]

[0137] In the above registration interface example, heterogeneous hardware is represented by defining the structure `hw_t`, and different `type` attributes and dynamic link library file names are used to distinguish different hardware types. When registering heterogeneous hardware, the `dlopen` and `dlsym` functions are used to load and obtain the hardware operation functions in the dynamic link library, and save them in the corresponding attributes of the `hw_t` structure. In this way, different hardware types can be selected dynamically. In the specific implementation process, different hardware operation functions can be continuously added according to different hardware types, such as encryption and decryption operation functions, reduction operation functions, matrix multiplication operation functions, etc., which are not limited in the embodiments of the present application.

[0138] Based on the registration interface introduced in the above example, take the computing power runtime framework (specifically, the registration module) actively discovering and completing the registration of different heterogeneous computing power resources as an example. Figure 4 As shown, the discovery and registration process may include the following steps:

[0139] S401 (optional step): The framework developer provides a framework registration interface and a computing power usage interface.

[0140] S402: The heterogeneous hardware developer abstracts the registration interfaces of multiple heterogeneous hardware as required, so that the heterogeneous hardware implements the computing power runtime interface, wherein the computing power runtime interface may include both the registration interface and the operation interface implemented by the heterogeneous hardware.

[0141] S403: Compile the computing resources of the heterogeneous hardware into a dynamic link library file (different hardware will be compiled into different dynamic link library files) through a heterogeneous hardware developer or an automated program.

[0142] S404: The computing power runtime framework is configured and initialized, and a preset data structure storage medium (such as a linked list) is used to record the heterogeneous hardware type and the computing power acceleration type.

[0143] S405: The computing power runtime framework calls the registration interface and loads the dynamic link library file in a polling manner to complete the registration of the heterogeneous hardware.

[0144] For example, the computing power runtime framework scans all dynamic link library files under the specified path in the computing power runtime system in a polling manner, adds the heterogeneous hardware and the computing power acceleration type implemented by the hardware to the preset data structure storage medium, completes the registration of all heterogeneous hardware, and establishes a heterogeneous hardware resource pool. Among them, for the convenience of management and control of various types of heterogeneous hardware, different types of resource pools can be established, or different types of resource queues can be established in the same resource pool, that is, one type of heterogeneous hardware can correspond to one resource queue or one resource pool.

[0145] When a new heterogeneous hardware needs to be added, just compile the computing power resources of the heterogeneous hardware into a dynamic link library file according to the above method, and the computing power runtime framework will call the registration interface and load the corresponding dynamic link library file in a polling manner.

[0146] The above-mentioned linked list is only an example of a data structure storage medium and is not a limitation. In other embodiments, other data structure storage media may be used according to business scenarios or requirements, and the embodiments of the present application do not limit this. Figure 5 A schematic diagram of a heterogeneous computing resource pool registered by four heterogeneous hardware is shown.

[0147] like Figure 5 As shown, for heterogeneous hardware A, heterogeneous hardware B, heterogeneous hardware C, and heterogeneous hardware D, the developers of heterogeneous hardware can abstract the registration interfaces of different heterogeneous hardware and compile their computing resources into dynamic link library files. The computing runtime framework can dynamically load the dynamic link library files to load the corresponding heterogeneous hardware into the resource pool.

[0148] The computing resources in the heterogeneous computing resource pool can be divided into different resource queues according to the type of algorithm. When the framework loads the dynamic link library file, the computing resources of the heterogeneous hardware can be loaded into the corresponding resource queue according to the type.

[0149] For example, heterogeneous hardware A supports encryption and decryption operations. The computing power resources of heterogeneous hardware A can be compiled into libcomputing-a.so and added to the encryption and decryption operation resource queues when registered in the computing power runtime framework. Heterogeneous hardware B supports encryption and decryption operations. The computing power resources of heterogeneous hardware B can be compiled into libcomputing-b.so and added to the encryption and decryption operation resource queues when registered in the computing power runtime framework. Heterogeneous hardware C supports matrix multiplication and reduction operations. The computing power resources of heterogeneous hardware C can be compiled into libcomputing-c.so and added to the matrix multiplication and reduction operation resource queues when registered in the computing power runtime framework. Heterogeneous hardware D supports matrix multiplication operations. The computing power resources of heterogeneous hardware C can be compiled into libcomputing-d.so and added to the matrix multiplication operation resource queue when registered in the computing power runtime framework. Among them, different filled graphics schematically represent different operations supported by the same heterogeneous hardware.

[0150] (2) Operation phase of computing power runtime framework:

[0151] In an embodiment of the present application, the computing power runtime framework may be designed to support a scheduling algorithm, which may be used to schedule heterogeneous computing power requests from applications to appropriate heterogeneous hardware for processing.

[0152] Exemplarily, the scheduling algorithms supported by the computing power runtime framework may include but are not limited to the following examples:

[0153] 1) First Input First Output (FIFO) scheduling algorithm: The scheduling module can schedule different heterogeneous hardware in the order in which they are registered to the resource pool, in a first-in-first-out manner. That is, for heterogeneous hardware of the same type, the heterogeneous computing power request is first scheduled to the heterogeneous hardware that registered first.

[0154] 2) Round Robin Scheduling Algorithm: The scheduling module selects heterogeneous hardware for scheduling in a round-robin manner, and evenly distributes heterogeneous computing power requests to available heterogeneous hardware. Each heterogeneous hardware processes the heterogeneous computing power requests assigned to it in sequence, and after completion, distributes them to the next heterogeneous hardware for processing. Among them, in some optional implementations, after the request processing of a heterogeneous hardware is completed, the processing result can be fed back to the scheduling module, and the scheduling module continues to allocate the next heterogeneous hardware.

[0155] 3) Lowest load first scheduling algorithm: The scheduling module schedules different heterogeneous hardware in the same resource queue according to the load weights, and selects the heterogeneous hardware with the lightest load and the least number of tasks from the resource queue for execution.

[0156] The above scheduling algorithms are only examples and not limitations. In specific implementations, cloud vendors can provide tenants with at least one scheduling algorithm, and tenants can configure the scheduling algorithm according to their own business needs. In other embodiments, more suitable scheduling algorithms can be expanded according to actual needs and business characteristics to achieve scheduling and management of different heterogeneous hardware in the resource pool, which will not be repeated here.

[0157] The scheduling module can schedule heterogeneous computing power requests from applications based on its own scheduling algorithm. For example, the computing power acceleration task is triggered by the encryption operation. Figure 6 As shown, the scheduling process may include the following steps:

[0158] S601 (optional step): The framework developer standardizes various API interfaces of the API layer.

[0159] S602: When the application provides cloud services to tenants / users, it triggers a computing power acceleration task when it detects heterogeneous computing capabilities using preset heterogeneous hardware types. For example, an encryption operation triggers a computing power acceleration task.

[0160] S603: The application uses the user-mode API to call the computing power runtime framework through the standardized API interface provided by the computing power runtime framework to send a heterogeneous computing power request to the computing power runtime node.

[0161] S604: The computing power runtime framework schedules the heterogeneous computing power request according to the task type to which the received heterogeneous computing power request belongs using a set scheduling algorithm, and provides the target heterogeneous hardware that meets the preset heterogeneous hardware type of the heterogeneous computing power request and can provide heterogeneous computing capabilities to the application for use.

[0162] S605: The computing power runtime framework sends the heterogeneous computing power request to a resource queue of the corresponding type that is registered and available in the resource pool.

[0163] S606: The resource queue of the corresponding type in the resource pool receives the heterogeneous computing power request, and uses the target heterogeneous hardware to perform task processing on the received heterogeneous computing power request.

[0164] S607: After the task processing is completed, the target heterogeneous hardware returns the processing result to the computing power runtime framework.

[0165] S608: The computing power runtime framework returns the processing result to the application.

[0166] If there are multiple different applications through Figure 6 The method shown initiates heterogeneous computing power requests, and the computing power runtime framework can use the same method to schedule heterogeneous computing power requests from different applications to appropriate heterogeneous hardware for processing.

[0167] For example, Figure 7 As shown, APP1, APP2, APP3, and APP4 represent four different applications. When providing cloud services to tenants / users, APP1, APP2, APP3, and APP4 can all follow the following steps: Figure 6 The method shown, after triggering the computing power acceleration task, sends a heterogeneous computing power request to the computing power runtime framework by calling the interface of the computing power runtime API provided by the computing power runtime framework. The computing power runtime API can provide the heterogeneous computing power request to the scheduling module of the computing power runtime framework. The scheduling module can perform task scheduling based on the set scheduling algorithm, the computing power demand operation type, the resource pool load and other factors, and provide the target heterogeneous hardware that meets the requirements to the corresponding APP for use. For example, the heterogeneous computing power request from APP1 is scheduled to the resource queue corresponding to the encryption operation and the resource queue corresponding to the decryption operation, the heterogeneous computing power request from APP2 is scheduled to the resource queue corresponding to the encryption operation and the resource queue corresponding to the decryption operation, the heterogeneous computing power request from APP3 is scheduled to the resource queue corresponding to the matrix multiplication operation, and the heterogeneous computing power request from APP4 is scheduled to the resource queue corresponding to the matrix multiplication operation and the resource queue corresponding to the reduction operation.

[0168] 4. Computing power backend shim layer:

[0169] In an embodiment of the present application, the computing power backend shim layer of the computing power runtime node may include shims (Shim) of SDKs that support different types of heterogeneous hardware, different models and algorithms, support dynamic registration of heterogeneous hardware, and support virtualized computing power devices.

[0170] Exemplarily, for different cloud resources and different types of heterogeneous hardware, in different deployment scenarios, the heterogeneous computing power providing system of the embodiment of the present application can be deployed as different architectures. The system architecture diagram and its deployment method in the optional implementation mode are introduced as examples below.

[0171] (1) Single machine (single server) deployment architecture:

[0172] In the embodiment of the present application, the computing power runtime node can simultaneously support pure bare metal deployment, container deployment on bare metal, or serverless deployment on bare metal, and can also support virtual machine deployment, lightweight virtual machine or secure container deployment, such as Figure 8 shown.

[0173] When deploying this architecture, Fig. 9 As shown, in step ①, you can first deploy the system, configure and register heterogeneous hardware computing resources. The registration process can be found in the previous article. Figure 4-Figure 5 The relevant description is not repeated here. Then, in step ②, the application deployed in the client space (Guest Space) and / or host space (Host Space) calls the interface provided by the computing power runtime node, creates and manages tasks through the task module, and schedules tasks through the scheduling module to select appropriate heterogeneous hardware computing power resources for different applications. This process can be referred to in the previous article combined with Figure 6-Figure 7 The relevant description will not be repeated here.

[0174] (2) Virtual machine deployment architecture:

[0175] The virtual machine deployment scenarios can include two situations:

[0176] 1) Directly pass PCI hardware computing devices (including SR-IOV) to the virtual machine. This method is similar to the bare metal deployment method. For more information, see Figure 8-Figure 9 The introduction of , I will not go into details here.

[0177] 2) Present virtual hardware computing devices to virtual machines. The specific deployment forms can be as follows: Fig.10 The virtual computing power front end is the virtual machine hardware device, and the specific form of the device can be a PCI device, a shared memory, or a SOCKET device.

[0178] Fig.10The deployment process of the architecture shown is as follows Fig.11 As shown, the following steps may be included:

[0179] ①: Deploy the computing runtime node on the host side, configure and register heterogeneous hardware computing resources.

[0180] ②: Configure and simulate virtual computing devices for the virtual machine and start the virtual machine.

[0181] ③: Deploy the computing power runtime node on the virtual machine side (i.e. the client side), configure and register the virtual hardware computing power device.

[0182] ④: The application calls the interface of the computing power runtime node, uses the task management module to create and manage tasks, and the scheduling management module to schedule tasks and select appropriate computing power resources for the application (here, the virtual computing power front-end device will be selected). At the same time, the statistics module performs performance statistics. The data flow is forwarded to the virtual machine simulator on the host side through the virtual computing power front-end device.

[0183] ⑤: The virtual machine simulator enters the computing power runtime framework on the host side through the data forwarding module. Among them, the data forwarding module supports deployment in the virtual machine simulator and can also support independent user-mode process deployment, which is not limited in the embodiment of the present application.

[0184] ⑥: Based on the computing power runtime framework on the host side, complete the selection and use of heterogeneous computing power resources.

[0185] (3) Uninstall card deployment architecture:

[0186] like Fig.12 As shown in the figure, the back-end offload card can be used as an external device for the front-end physical server. Since each server can be equipped with the offload card, the offload card has general CPU computing resources and can contain other heterogeneous resources such as NPU, GPU, etc. The cloud management software can be offloaded to the card and the offload card can be connected to the cloud network and cloud storage. At this time, the management plane operation requests issued by the tenant / user through the cloud management platform are only sent to the offload card.

[0187] The heterogeneous provision system of the embodiment of the present application can be applied to the architecture using the offload card to support the external Serverless service provision of the offload card computing resources and cloud service resources, can be used for offloading acceleration of customer business, and can facilitate the sale of the offload card as an independent device.

[0188] In this scenario, the computing power runtime nodes and heterogeneous hardware can be deployed inside the offload card, such as Fig.13As shown, the virtual machine manager deployed on the server communicates with the computing power runtime node on the offload card through a high-speed interconnection channel (such as PCle, etc.), and uses the corresponding heterogeneous hardware to accelerate the computing power to provide corresponding cloud services for tenants / users.

[0189] The offload card deployment solution can support serverless storage services and network services. At the same time, the deployment solution supports providing the computing resources on the offload card and other capabilities to the front-end server and virtual machine / secure container in a service-oriented manner for offload acceleration or sales as an independent product. Similarly, in this architecture, the connection form between the front and back ends can be virtual devices, shared memory, vsock, etc. In some embodiments, the offload card deployment solution can also support distributed pooling expansion.

[0190] (4) Pooled deployment architecture within a data center (e.g., rack server):

[0191] The heterogeneous provision system of the embodiment of the present application can be applied to the architecture deployment of a data center to support the pooled deployment of heterogeneous computing power and improve the overall resource utilization.

[0192] like Fig.14 As shown, the data center may include multiple nodes and multiple resource pools. When the heterogeneous computing power system of the present application is established in the data center, each node can be used as a computing power runtime node. The multiple resource pools may include, for example, a GPU resource pool, a FGPA resource pool, a DSA resource pool or other types of xPU resource pools. Each node may interact with different resource pools through a switch to distribute the heterogeneous computing power requests triggered by the implementation of the business to the appropriate heterogeneous hardware computing power resource pool for processing. Among them, the switch may support but is not limited to high-speed interconnection protocols such as UB / CXL.

[0193] This deployment architecture supports fast interconnection within the Rack (extremely low latency), and can expand the heterogeneous hardware devices of the computing power runtime system from single host resources to remote heterogeneous computing power, that is, it supports the pooled deployment of heterogeneous computing power. At the same time, using high-speed interconnection buses (such as Unified Bus, CXL, etc.), computing resources of different types and architectures (such as GPU, FPGA, DSA, xPU, etc.) can be integrated into a unified computing power pool, and through resource pooling and scheduling management technology, unified management and efficient use of these heterogeneous computing resources can be achieved.

[0194] Similarly, the deployment process of this architecture may include the following processes:

[0195] 1) Resource pooling: Register computing resources of different types and architectures into a unified resource pool. The registration process involves identifying, registering, and managing different types of hardware resources (including heterogeneous hardware resources), including the collection and storage of hardware characteristics, performance indicators, status, and other information. For implementation, see the previous article combined with Figure 4-Figure 5 The relevant description will not be repeated here.

[0196] 2) Resource management: Manage resources in the resource pool, including resource registration, deregistration, status monitoring, performance statistics, fault handling, etc. This can be achieved through interaction with hardware resources and management interfaces, such as encapsulating and managing hardware resource drivers, APIs, SDKs, etc. The implementation method can be found in the previous description of the API interface standardization process, the registration process of heterogeneous hardware, and the computing power backend shim layer, which will not be repeated here.

[0197] 3) Resource Scheduling: Design and implement appropriate scheduling algorithms so that when computing power is running, nodes can reasonably allocate and schedule resources according to the requirements of tasks, the characteristics of resources, and the resource status in the resource pool, including task scheduling strategies, resource allocation strategies, resource status monitoring and dynamic adjustment, etc. The implementation of this process is similar to the introduction of the task module, scheduling module, management service module, and statistics module in the previous article. Please refer to the relevant description in the previous article and will not be repeated here.

[0198] 4) Heterogeneous computing power orchestration: Heterogeneous computing power is orchestrated and configured according to the characteristics of the tasks and the capabilities of the resources. For example, computing-intensive tasks can be assigned to high-performance GPU resources, and data processing tasks can be assigned to FPGA resources suitable for processing large-scale data.

[0199] 5) Security management: Ensure the security of resources and tasks in the resource pool during deployment, scheduling and execution. This may involve measures such as permission management, identity authentication, data encryption, and security isolation to protect the security of resources and tasks.

[0200] 6) Performance optimization: According to the characteristics of different hardware resources and the requirements of tasks, we optimize the performance, including the optimization of resource utilization, response time, throughput, etc., to improve the overall computing performance. The implementation of this part is similar to the function of the statistics module introduced above. Please refer to the relevant description in the previous article and will not be repeated here.

[0201] (5) Pooled deployment architecture across data centers (e.g. Rack):

[0202] The heterogeneous provision system of the embodiment of the present application is applicable to scenarios in which multiple data centers are used, such as Fig.15As shown in the figure, in this scenario, communication can be carried out within the data center through switches and high-speed interconnection links, and communication can be carried out in different data centers, that is, across racks, through switches and Ethernet / Remote Direct Memory Access (RDMA) links. This architecture allows tenants to deploy and use on demand, which helps to improve the resource utilization of the overall architecture.

[0203] The above heterogeneous computing power system and architecture examples introduce the scenarios and architectures applicable to the heterogeneous provision system of the embodiment of the present application. These architectures can support bare metal and virtual machine forms deployed on cloud vendors and customers at the same time (similar to virtual computing power devices), and can support containers and Serverless scenarios (supporting micro VM and independent process methods at the same time), which helps to achieve architectural consistency. At the same time, the heterogeneous computing power provision system can enable different types of heterogeneous hardware to be registered in the framework in a service-oriented, modular, or plug-in manner, and can support the expansion of different computing powers, such as adding GPUs, xPUs, FPGAs, ASICs, etc., and support distributed deployment and pooled deployment and expansion of computing power. At the same time, the heterogeneous computing power provision system can support industry standard APIs (TensorFlow, PyTorch, DirectX, OpenGL, CUDA, OpenCL, VDPAU, VAAPI, Crypto, etc.). Based on the computing power runtime architecture, cloud tenants / cloud users can quickly build applications and services, and cloud vendors can also speed up the launch of new hardware. In this, the application is independent of the hardware, which facilitates seamless porting of applications. Cloud vendors can also enable computing power acceleration on demand in a service-oriented manner based on different cloud services and different types of heterogeneous hardware, accurately control computing power QoS, and appropriately measure and charge fees.

[0204] Fig.16 The schematic diagram of the process of providing heterogeneous computing power in the embodiment of the present application is shown. The method can be deployed in the above Figure 1 , Figure 2 , Figure 3 , Figure 8 , Fig.10 , Figure 13-Figure 15 The computing power runtime node in the system architecture shown is executed. The heterogeneous computing power providing system also includes an infrastructure, the infrastructure includes multiple servers and multiple heterogeneous hardware, the types of the multiple heterogeneous hardware are different from the types of the CPUs of the multiple servers, and the computing power runtime node records the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types.

[0205] like Fig.16 As shown, the method may include the following steps:

[0206] S1601: Receive a heterogeneous computing power request from an application, where the heterogeneous computing power request is associated with heterogeneous computing capabilities of a preset heterogeneous hardware type that the application needs to use;

[0207] S1602: According to the heterogeneous computing power request, target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities is provided to the application for use.

[0208] The specific implementation details of the above heterogeneous computing power provision method can be found in the previous article. Figure 1-Figure 13 The relevant introduction will not be repeated here.

[0209] In combination with the above method embodiments, the embodiments of the present application also provide a heterogeneous computing power providing device based on cloud computing technology, which can be used to execute the method executed by the node when the computing power is running in the above method embodiments.

[0210] Among them, the computing power runtime node can belong to the heterogeneous computing power providing system based on cloud computing technology introduced above, and the heterogeneous computing power providing system also includes an infrastructure, and the infrastructure includes multiple servers and multiple heterogeneous hardware. The types of the multiple heterogeneous hardware are different from the types of CPUs of the multiple servers. The computing power runtime node records the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types.

[0211] like Fig.17 As shown, the device 1700 may include: a communication interface 1701, used to receive a heterogeneous computing power request from an application, the heterogeneous computing power request being associated with the heterogeneous computing capabilities of a preset heterogeneous hardware type that the application needs to use; a processing unit 1702, used to provide target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application for use according to the heterogeneous computing power request.

[0212] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. Each functional unit in the embodiments of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0213] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0214] The present application also provides a computing device 1800. Fig.18 As shown, computing device 1800 includes: bus 1802, processor 1804, memory 1806 and communication interface 1808. Processor 1804, memory 1806 and communication interface 1808 communicate through bus 1802. Computing device 1800 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in computing device 1800.

[0215] The bus 1802 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.18 The bus 1802 may include a path for transmitting information between various components of the computing device 1800 (eg, the memory 1806, the processor 1804, and the communication interface 1808).

[0216] The processor 1804 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0217] The memory 1806 may include a volatile memory, such as a random access memory (RAM). The processor 1804 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0218] The memory 1806 stores executable program codes, and the processor 1804 executes the executable program codes to respectively implement the functions of the aforementioned cloud management platform and computing power runtime node, thereby implementing the method of the embodiment of the present application. That is, the memory 1806 stores instructions for executing the method of the embodiment of the present application.

[0219] The communication interface 1808 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1800 and other devices or communication networks.

[0220] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0221] like Fig.19 As shown, the computing device cluster includes at least one computing device 1800. The memory 1806 in one or more computing devices 1800 in the computing device cluster may store the same instructions for executing the method of the embodiment of the present application.

[0222] In some possible implementations, the memory 1806 of one or more computing devices 1800 in the computing device cluster may also store partial instructions for executing the method of the embodiment of the present application. In other words, the combination of one or more computing devices 1800 can jointly execute the instructions for executing the method of the embodiment of the present application.

[0223] It should be noted that the memory 1806 in different computing devices 1800 in the computing device cluster can store different instructions, which are respectively used to perform part of the functions of the cloud service system. That is, the instructions stored in the memory 1806 in different computing devices 1800 can implement the functions of one or more modules of the cloud management platform and the computing power runtime node.

[0224] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig. 20 A possible implementation is shown. Fig. 20 As shown, two computing devices 1800A and 1800B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 1806 in the computing device 1800A stores instructions for executing the functions of the cloud management platform. At the same time, the memory 1806 in the computing device 1800B stores instructions for executing the functions of the computing power runtime node.

[0225] Fig. 20 The connection method between the computing device clusters shown can be considered to be that the heterogeneous computing power provision method provided in this application requires users to customize functional configuration (such as storing large amounts of data) and implement computing functions, so it is considered to hand over the functions implemented by the computing power runtime node to the computing device 1800B for execution.

[0226] It should be understood that Fig. 9 The functionality of the computing device 1800A shown in FIG. 1800A may also be implemented by multiple computing devices 1800. Similarly, the functionality of the computing device 1800B may also be implemented by multiple computing devices 1800.

[0227] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Fig.19 and Fig. 20 The connection mode of the computing device cluster is different in that the memory 1806 in one or more computing devices 1800 in the computing device cluster may store the same instructions for executing the heterogeneous computing power providing method.

[0228] In some possible implementations, the memory 1806 of one or more computing devices 1800 in the computing device cluster may also store some instructions for executing the heterogeneous computing power providing method. In other words, the combination of one or more computing devices 1800 can jointly execute the instructions for executing the heterogeneous computing power providing method.

[0229] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be a software or program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes a heterogeneous computing power provision method.

[0230] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by the computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to perform a heterogeneous computing power provision method.

[0231] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

[0232] An embodiment of the present application also relates to a processor, which is used to call a computer program or computer instruction stored in a memory so that the processor executes the above method embodiment.

[0233] The processor mentioned in any of the above places can be a general-purpose central processing unit, a microprocessor, an application-specific integrated circuit (ASIC), or one or more processors for controlling the above Fig. 9 The integrated circuit for executing the program of the method in the illustrated embodiment. The memory mentioned in any of the above places can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0234] It should be understood that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0235] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0236] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0237] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the scope of the embodiments of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A heterogeneous computing power provision system based on cloud computing technology, characterized in that: include: An infrastructure comprising a plurality of servers and a plurality of heterogeneous hardware, wherein a type of the plurality of heterogeneous hardware is different from a type of CPUs of the plurality of servers; Computing runtime nodes are used to record the heterogeneous computing capabilities that heterogeneous hardware of different types can provide; An application, disposed in one of the plurality of servers of the infrastructure, sends a heterogeneous computing power request to the computing power runtime node when heterogeneous computing power of a preset heterogeneous hardware type is required; The computing power runtime node is also used to provide the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application program for use according to the heterogeneous computing power request.

2. The system according to claim 1, characterized in that Before the computing power runtime node records the heterogeneous computing capabilities that heterogeneous hardware of different heterogeneous hardware types can provide, it is also used to: By calling the registration interface, the dynamic link library files associated with the plurality of heterogeneous hardware are loaded, and the heterogeneous computing capabilities that can be provided by the heterogeneous hardware of different heterogeneous hardware types are recorded in a preset data structure storage medium.

3. The system according to claim 2, characterized in that The computing power runtime node is also used for: The dynamic link library file input or selected by the developers of the plurality of heterogeneous hardware on the cloud management platform is obtained.

4. The system according to any one of claims 1 to 3, characterized in that: Before the application sends a heterogeneous computing power request to the computing power runtime node, the computing power runtime node is further used to: An API calling interface of the algorithms supported by the multiple heterogeneous hardware is provided to the application program, so that the application program sends a heterogeneous computing power request to the computing power runtime node by calling the API interface.

5. The system according to claim 4, characterized in that The computing power runtime node is also used for: Obtain at least one data access strategy sent by a framework developer of the computing power running node, wherein the at least one data access strategy includes multiple fields and parameters input by the developer for each field, the multiple fields are associated with an application programming interface API of the at least one data access strategy, and the API is provided by a cloud management platform, and the multiple fields are used to characterize different attributes of the at least one data access strategy.

6. The system according to any one of claims 1 to 5, characterized in that: The plurality of heterogeneous hardware belong to at least one resource queue, and the computing power runtime node provides the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application program for use according to the heterogeneous computing power request, including: Determine a target resource queue according to the heterogeneous computing power request; The target heterogeneous hardware in the target resource queue that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities is provided to the application for use.

7. The system according to claim 6, characterized in that The computing power runtime node uses any one of the following scheduling algorithms to provide the target heterogeneous hardware in the target resource queue that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application: First-in-first-out scheduling algorithm; polling scheduling algorithm or lowest load first scheduling algorithm.

8. The system according to any one of claims 1 to 7, characterized in that: One of the multiple servers of the infrastructure includes any of the following types: a bare metal server, a virtual machine, a container, a lightweight container, or a serverless server.

9. The system according to any one of claims 1 to 8, characterized in that: The architecture of the heterogeneous computing power providing system includes an offload card, the computing power runtime node is deployed in the offload card, and part or all of the multiple heterogeneous hardware belong to the offload card.

10. The system according to any one of claims 1 to 8, characterized in that: The heterogeneous computing power system is deployed on a rack server in the same data center, wherein the multiple computing nodes in the data center are the computing power runtime nodes, and different computing power runtime nodes communicate and interact with each other through switches.

11. The system according to any one of claims 1 to 8, characterized in that: The computing power runtime nodes and infrastructure cross-rack servers of the heterogeneous computing power system are deployed on rack servers in different data centers, and different rack servers communicate and interact with each other through switches.

12. A method for providing heterogeneous computing power based on cloud computing technology, characterized in that: Applied to a computing power runtime node, the computing power runtime node belongs to a heterogeneous computing power providing system based on cloud computing technology, the heterogeneous computing power providing system also includes an infrastructure, the infrastructure includes multiple servers and multiple heterogeneous hardware, the types of the multiple heterogeneous hardware are different from the types of CPUs of the multiple servers, the computing power runtime node records the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types, and the method includes: Receiving a heterogeneous computing power request from an application, the heterogeneous computing power request being associated with heterogeneous computing capabilities of a preset heterogeneous hardware type that the application needs to use; According to the heterogeneous computing power request, target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities is provided to the application for use.

13. The method according to claim 12, characterized in that Also includes: By calling the registration interface, the dynamic link library files associated with the plurality of heterogeneous hardware are loaded, and the heterogeneous computing capabilities that can be provided by the heterogeneous hardware of different heterogeneous hardware types are recorded in a preset data structure storage medium.

14. The method according to claim 13, characterized in that Also includes: The dynamic link library file input or selected by the developers of the plurality of heterogeneous hardware on the cloud management platform is obtained.

15. The method according to any one of claims 12 to 14, characterized in that: Also includes: An API calling interface of the algorithms supported by the multiple heterogeneous hardware is provided to the application program, so that the application program sends a heterogeneous computing power request to the computing power runtime node by calling the API interface.

16. The method according to claim 15, characterized in that Also includes: Obtain at least one data access strategy sent by a framework developer of the computing power running node, wherein the at least one data access strategy includes multiple fields and parameters input by the developer for each field, the multiple fields are associated with an application programming interface API of the at least one data access strategy, and the API is provided by a cloud management platform, and the multiple fields are used to characterize different attributes of the at least one data access strategy.

17. The method according to any one of claims 12 to 16, characterized in that: The plurality of heterogeneous hardware belong to at least one resource queue, and providing the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application for use according to the heterogeneous computing power request, including: Determine a target resource queue according to the heterogeneous computing power request; The target heterogeneous hardware in the target resource queue that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities is provided to the application for use.

18. The method according to claim 17, characterized in that The target heterogeneous hardware in the target resource queue that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities is provided to the application program using any of the following scheduling algorithms: First-in-first-out scheduling algorithm; polling scheduling algorithm or lowest load first scheduling algorithm.

19. The method according to any one of claims 12 to 18, characterized in that: One of the multiple servers of the infrastructure includes any of the following types: a bare metal server, a virtual machine, a container, a lightweight container, or a serverless server.

20. The method according to any one of claims 12 to 19, characterized in that The architecture of the heterogeneous computing power providing system includes an offload card, the computing power runtime node is deployed in the offload card, and part or all of the multiple heterogeneous hardware belong to the offload card.

21. The method according to any one of claims 12 to 19, characterized in that The heterogeneous computing power system is deployed on a rack server in the same data center, wherein the multiple computing nodes in the data center are the computing power runtime nodes, and different computing power runtime nodes communicate and interact with each other through switches.

22. The method according to any one of claims 12 to 19, characterized in that The computing power runtime nodes and infrastructure cross-rack servers of the heterogeneous computing power system are deployed on rack servers in different data centers, and different rack servers communicate and interact with each other through switches.

23. A heterogeneous computing power providing device based on cloud computing technology, characterized in that: Applied to a computing power runtime node, the computing power runtime node belongs to a heterogeneous computing power providing system based on cloud computing technology, the heterogeneous computing power providing system also includes an infrastructure, the infrastructure includes multiple servers and multiple heterogeneous hardware, the types of the multiple heterogeneous hardware are different from the types of CPUs of the multiple servers, the computing power runtime node records the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types, and the device includes: A communication interface for receiving a heterogeneous computing power request from an application, wherein the heterogeneous computing power request is associated with a heterogeneous computing capability of a preset heterogeneous hardware type that the application needs to use; The processing unit is used to provide the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application program for use according to the heterogeneous computing power request.

24. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 12 to 22.

25. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method as claimed in any one of claims 12 to 22.

26. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 12 to 22.

Citation Information

Cited By

  • Heterogeneous computing power fused cloud computing platform, resource scheduling method and device, equipment and medium

    CN119583650A

  • Request allocation method, system and equipment based on heterogeneous computing system and medium

    CN119690687A

  • Hierarchical abstraction method and system for computing resources

    CN120371317A

  • Network element instance deployment method and device based on heterogeneous computing power resources and electronic equipment

    CN120499273A

  • Heterogeneous computing resource-based network element instance deployment method and device and electronic equipment

    CN120499273B