Cloud computing technology-based heterogeneous computing power providing system and method

By providing heterogeneous computing power systems in cloud computing technology, and using nodes to record and schedule computing power during computing power operation, the problem of inconsistent floating-point computing accuracy requirements for the integration of high-performance computing and artificial intelligence in cloud technology is solved. The integration of diverse computing power and efficient utilization of resources have been achieved.

WO2024230603A1PCT designated stage expired Publication Date: 2024-11-14HUAWEI CLOUD COMPUTING TECHNOLOGIES CO LTD

Patent Information

Application Number
PCT/CN2024/090935
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-30
Filing Date
2024-04-30
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

In the field of cloud technology, the integration of high-performance computing and artificial intelligence needs to solve the problem of inconsistent floating-point computing accuracy requirements, and the chip architecture shifts from CPU-centered to heterogeneous processors, resulting in the operating system or virtualization layer that requires diversified computing to be enabled. , but how to achieve this enablement is still an important issue that needs to be solved urgently.

Method used

Provide a heterogeneous computing power supply system based on cloud computing technology. The node records the computing power of different heterogeneous hardware during computing power operation. The application can send heterogeneous computing power requests. The node provides a corresponding preset hardware type according to the request. Heterogeneous hardware uses scheduling algorithms such as first-in-first out, polling or minimum load-first scheduling, and supports deployment scenarios such as bare metal, virtual machines, containers, etc.

Benefits of technology

It realizes flexible deployment and expansion of diverse computing power equipment, uses appropriate computing power resources as needed, improves the computing performance and resource utilization of applications, supports the integration of diverse computing power, and alleviates the memory and storage wall problems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024090935_14112024_PF_FP_ABST
    Figure CN2024090935_14112024_PF_FP_ABST
Patent Text Reader

Abstract

Disclosed are a cloud computing technology-based heterogeneous computing power providing system and method, relating to the field of cloud technologies. The system comprises: an infrastructure comprising a plurality of servers and a plurality of heterogeneous hardware, wherein the type of the plurality of heterogeneous hardware is different from the type of CPUs of the plurality of servers; a computing power run-time node configured for recording heterogeneous computing power that can be provided by heterogeneous hardware of different types; and an application program arranged in one of the plurality of servers of the infrastructure, which sends a heterogeneous computing power request to the computing power run-time node when the heterogeneous computing power of a preset heterogeneous hardware type is required to be used. The computing power run-time node is further configured for providing target heterogeneous hardware, which conforms to the preset heterogeneous hardware type and can provide the heterogeneous computing power, to the application program for use according to the heterogeneous computing power request.
Need to check novelty before this filing date? Find Prior Art

Description

A heterogeneous computing power provision system and method based on cloud computing technology

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on May 5, 2023, with application number 202310497813.5 and application name “A system and method for providing heterogeneous computing power based on cloud computing technology”, the entire contents of which are incorporated by reference into this application; this application claims priority to the Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on May 30, 2023, with application number 202310626543.3 and application name “A system and method for providing heterogeneous computing power based on cloud computing technology”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of cloud technology, and in particular to a system and method for providing heterogeneous computing power based on cloud computing technology. Background Art

[0004] In the past, cloud technology often used a single vertical domain of software and hardware to carry a specific type of computing, such as general-purpose computing, probabilistic computing, artificial intelligence (AI), physical computing, and graphics computing. However, the development trend of cloud technology shows that computing architecture is evolving from "general-purpose computing + specialized computing" to "the integration of diverse computing power," but this also presents numerous challenges. For example, the integration of high-performance computing (HPC) and AI (represented as "HPC+AI") requires addressing the differing precision requirements of HPC and AI. HPC requires very high floating-point precision, while AI requires relatively high floating-point precision, sometimes using Int8 or Int16 for approximate calculations. This "HPC+AI" approach requires operating systems to move from vertical domain separation to heterogeneous integration, effectively coordinating scalar and vector computing power.

[0005] At the same time, chip architectures have evolved from being centered around the central processing unit (CPU) to the current heterogeneous CPU + XPU collaboration, and are gradually forming a unified memory architecture (UMA). This reduces data migration and effectively alleviates the memory and storage wall issues. Furthermore, integrated storage and computing, as well as near-data computing, can improve energy efficiency. This means computing is performed where the data resides, such as on smart network cards and smart SSDs equipped with powerful computing units.

[0006] The evolution of these applications and hardware requires an operating system or virtualization layer to support the diverse computing landscape of the IoT era. However, how to achieve this remains a critical issue that needs to be addressed.

[0007] Summary of the Invention

[0008] The embodiments of the present application provide a heterogeneous computing power provision system and method based on cloud computing technology, which is used to flexibly deploy and expand various computing power devices, and use appropriate computing power resources on demand to achieve computing power acceleration for related services to be implemented by the application.

[0009] In the first aspect, an embodiment of the present application provides a heterogeneous computing power provision system based on cloud computing technology, including: an infrastructure, including multiple servers and multiple heterogeneous hardware, wherein the types of the multiple heterogeneous hardware are different from the types of the CPUs of the multiple servers; a computing power runtime node, used to record the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types; an application, set in one of the multiple servers of the infrastructure, sending a heterogeneous computing power request to the computing power runtime node when the heterogeneous computing capabilities of a preset heterogeneous hardware type are required; the computing power runtime node is also used to provide the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application for use according to the heterogeneous computing power request.

[0010] In one possible design, before the computing power runtime node records the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types, it is also used to: load the dynamic link library files associated with the multiple heterogeneous hardware by calling the registration interface, and record the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types in a preset data structure storage medium.

[0011] In one possible design, the computing power runtime node is also used to obtain the dynamic link library file input or selected by the developers of the multiple heterogeneous hardware on the cloud management platform.

[0012] In one possible design, before the application sends a heterogeneous computing power request to the computing power runtime node, the computing power runtime node is also used to: provide the application with an API call interface of the algorithms supported by the multiple heterogeneous hardware, so that the application sends a heterogeneous computing power request to the computing power runtime node through an API interface call.

[0013] In one possible design, the computing power runtime node is also used to: obtain at least one data access strategy sent by the framework developer of the computing power runtime node, the at least one data access strategy includes multiple fields and parameters entered by the developer for each field, the multiple fields are associated with the application programming interface API of the at least one data access strategy, the API is provided by the cloud management platform, and the multiple fields are used to characterize different attributes of the at least one data access strategy.

[0014] In one possible design, the multiple heterogeneous hardware belong to at least one resource queue, and the computing power runtime node provides the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application for use according to the heterogeneous computing power request, including: determining the target resource queue according to the heterogeneous computing power request; and providing the target heterogeneous hardware in the target resource queue that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application for use.

[0015] In one possible design, the computing power runtime node uses any of the following scheduling algorithms to provide the target heterogeneous hardware in the target resource queue that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application for use: first-in-first-out scheduling algorithm; polling scheduling algorithm or lowest load first scheduling algorithm.

[0016] In one possible design, one of the multiple servers of the infrastructure includes any of the following types: a bare metal server, a virtual machine, a container, a lightweight container, or a serverless server.

[0017] In one possible design, the architecture of the heterogeneous computing power providing system includes an offload card, the computing power runtime node is deployed in the offload card, and some or all of the multiple heterogeneous hardware belong to the offload card.

[0018] In one possible design, the heterogeneous computing power system is deployed on a rack server in the same data center, wherein the multiple computing nodes in the data center are the computing power runtime nodes, and different computing power runtime nodes communicate and interact with each other through switches.

[0019] In one possible design, the computing runtime nodes and infrastructure of the heterogeneous computing system are deployed across rack servers in rack servers in different data centers, and different rack servers communicate and interact with each other through switches.

[0020] In the second aspect, an embodiment of the present application provides a heterogeneous computing power provision method based on cloud computing technology, which is applied to a computing power runtime node. The computing power runtime node belongs to a heterogeneous computing power provision system based on cloud computing technology. The heterogeneous computing power provision system also includes an infrastructure, which includes multiple servers and multiple heterogeneous hardware. The types of the multiple heterogeneous hardware are different from the types of the CPUs of the multiple servers. The computing power runtime node records the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types. The method includes: receiving a heterogeneous computing power request from an application, the heterogeneous computing power request is associated with the heterogeneous computing capabilities of a preset heterogeneous hardware type that the application needs to use; according to the heterogeneous computing power request, the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities is provided to the application for use.

[0021] In one possible design, the method may further include: loading the dynamic link library files associated with the multiple heterogeneous hardware by calling a registration interface, and recording the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types in a preset data structure storage medium.

[0022] In one possible design, the method may further include: obtaining the dynamic link library file input or selected by the developers of the multiple heterogeneous hardware on the cloud management platform.

[0023] In one possible design, the method may also include: providing the application with an API calling interface of the algorithms supported by the multiple heterogeneous hardware, so that the application sends a heterogeneous computing power request to the computing power runtime node through the API interface call.

[0024] In one possible design, the method may also include: obtaining at least one data access strategy sent by the framework developer of the computing power running node, the at least one data access strategy includes multiple fields and parameters entered by the developer for each field, the multiple fields are associated with the application programming interface API of the at least one data access strategy, the API is provided by the cloud management platform, and the multiple fields are used to characterize different attributes of the at least one data access strategy.

[0025] In one possible design, the multiple heterogeneous hardware belong to at least one resource queue, and the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities is provided to the application for use according to the heterogeneous computing power request, including: determining the target resource queue according to the heterogeneous computing power request; and providing the target heterogeneous hardware in the target resource queue that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application for use.

[0026] In one possible design, the target heterogeneous hardware in the target resource queue that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities can be provided to the application for use by using but not limited to any of the following scheduling algorithms: first-in-first-out scheduling algorithm; polling scheduling algorithm or lowest load first scheduling algorithm.

[0027] In one possible design, one of the multiple servers of the infrastructure includes any of the following types: a bare metal server, a virtual machine, a container, a lightweight container, or a serverless server.

[0028] In one possible design, the architecture of the heterogeneous computing power providing system includes an offload card, the computing power runtime node is deployed in the offload card, and some or all of the multiple heterogeneous hardware belong to the offload card.

[0029] In one possible design, the heterogeneous computing power system is deployed on a rack server in the same data center, wherein the multiple computing nodes in the data center are the computing power runtime nodes, and different computing power runtime nodes communicate and interact with each other through switches.

[0030] In one possible design, the computing runtime nodes and infrastructure of the heterogeneous computing system are deployed across rack servers in rack servers in different data centers, and different rack servers communicate and interact with each other through switches.

[0031] In the third aspect, an embodiment of the present application provides a heterogeneous computing power providing device based on cloud computing technology, which is applied to a computing power runtime node. The computing power runtime node belongs to a heterogeneous computing power providing system based on cloud computing technology. The heterogeneous computing power providing system also includes an infrastructure, which includes multiple servers and multiple heterogeneous hardware. The types of the multiple heterogeneous hardware are different from the types of the CPUs of the multiple servers. The computing power runtime node records the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types. The device includes: a communication interface for receiving a heterogeneous computing power request from an application, wherein the heterogeneous computing power request is associated with the heterogeneous computing capabilities of a preset heterogeneous hardware type that the application needs to use; a processing unit for providing the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application for use according to the heterogeneous computing power request.

[0032] In one possible design, the processing unit is further used to: load the dynamic link library files associated with the multiple heterogeneous hardware by calling the registration interface, and record the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types in a preset data structure storage medium.

[0033] In one possible design, the communication interface is further used to obtain the dynamic link library file input or selected by the developers of the multiple heterogeneous hardware on the cloud management platform.

[0034] In one possible design, the communication interface is also used to provide the application with an API calling interface of the algorithms supported by the multiple heterogeneous hardware, so that the application can send heterogeneous computing power requests to the computing power runtime node through the API interface call.

[0035] In one possible design, the communication interface is also used to: obtain at least one data access strategy sent by the framework developer of the computing power running node, the at least one data access strategy includes multiple fields and parameters entered by the developer for each field, the multiple fields are associated with the application programming interface API of the at least one data access strategy, the API is provided by the cloud management platform, and the multiple fields are used to characterize different attributes of the at least one data access strategy.

[0036] In one possible design, the multiple heterogeneous hardware belong to at least one resource queue, and the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities is provided to the application for use according to the heterogeneous computing power request. The processing unit is used to: determine the target resource queue according to the heterogeneous computing power request; and provide the target heterogeneous hardware in the target resource queue that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application for use.

[0037] In one possible design, the processing unit may adopt but is not limited to any of the following scheduling algorithms to provide the target heterogeneous hardware in the target resource queue that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application for use: first-in-first-out scheduling algorithm; polling scheduling algorithm or lowest load first scheduling algorithm.

[0038] In one possible design, one of the multiple servers of the infrastructure includes any of the following types: a bare metal server, a virtual machine, a container, a lightweight container, or a serverless server.

[0039] In one possible design, the architecture of the heterogeneous computing power providing system includes an offload card, the computing power runtime node is deployed in the offload card, and some or all of the multiple heterogeneous hardware belong to the offload card.

[0040] In one possible design, the heterogeneous computing power system is deployed on a rack server in the same data center, wherein the multiple computing nodes in the data center are the computing power runtime nodes, and different computing power runtime nodes communicate and interact with each other through switches.

[0041] In one possible design, the computing runtime nodes and infrastructure of the heterogeneous computing system are deployed across rack servers in rack servers in different data centers, and different rack servers communicate and interact with each other through switches.

[0042] In a fourth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed by a computing device cluster, enables the computing device cluster to execute the method described in the first aspect and any possible design of the first aspect.

[0043] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method described in the first aspect and any possible design of the first aspect.

[0044] In a sixth aspect, an embodiment of the present application provides a computing device cluster comprising at least one computing device, each computing device comprising a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method described in the first aspect above and any possible design of the first aspect.

[0045] Based on the implementations provided in the above aspects, the embodiments of the present application can be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] FIG1 shows a schematic diagram of a cloud service system according to an embodiment of the present application;

[0047] FIG2 shows a schematic diagram of a heterogeneous computing power provision system according to an embodiment of the present application;

[0048] FIG3 shows a schematic diagram of a heterogeneous computing power provision system according to an embodiment of the present application;

[0049] FIG4 shows a flowchart of a heterogeneous hardware registration method according to an embodiment of the present application;

[0050] FIG5 shows a schematic diagram of a heterogeneous computing resource pool according to an embodiment of the present application;

[0051] FIG6 is a schematic diagram showing a flow chart of a method for providing heterogeneous computing power according to an embodiment of the present application;

[0052] FIG7 is a schematic diagram showing the principle of a method for providing heterogeneous computing power according to an embodiment of the present application;

[0053] FIG8 shows a schematic diagram of the architecture of a heterogeneous computing power provision system deployed on a single machine according to an embodiment of the present application;

[0054] FIG9 shows a schematic diagram showing the principle of a single-machine deployment solution according to an embodiment of the present application;

[0055] FIG10 shows a schematic diagram of the architecture of a heterogeneous computing power provision system deployed in virtual machines according to an embodiment of the present application;

[0056] FIG11 is a schematic diagram showing the principle of a virtual machine deployment solution according to an embodiment of the present application;

[0057] FIG12 shows a schematic diagram of a cloud system architecture based on an offload card;

[0058] FIG13 shows a schematic diagram of the architecture of a heterogeneous computing power provision system based on offload card deployment according to an embodiment of the present application;

[0059] FIG14 shows a schematic diagram of the architecture of a heterogeneous computing power provision system deployed in rack servers in a single data center according to an embodiment of the present application;

[0060] FIG15 shows a schematic diagram of the architecture of a heterogeneous computing power provision system deployed in rack servers across data centers according to an embodiment of the present application;

[0061] FIG16 is a schematic diagram showing a flow chart of a method for providing heterogeneous computing power according to an embodiment of the present application;

[0062] FIG17 is a schematic diagram of a heterogeneous computing power providing device according to an embodiment of the present application;

[0063] FIG18 shows a schematic diagram of a computing device according to an embodiment of the present application;

[0064] FIG19 shows a schematic diagram of a computing device cluster according to an embodiment of the present application;

[0065] FIG20 shows a schematic diagram of a computing device cluster according to an embodiment of the present application. DETAILED DESCRIPTION

[0066] In order to better explain the embodiments of the present application, some of the terms or technologies mentioned in the embodiments of the present application are introduced below.

[0067] 1. Cloud service: An Internet-based service that provides elastic computing, virtual networking, data storage, database, and other services in the form of a representational state transfer application programming interface (REST API) to meet the needs of enterprise internet technology (IT).

[0068] In the embodiments of the present application, the cloud service provider may be referred to as a cloud vendor.

[0069] 2. Cloud resources: The abbreviation for cloud computing resources provided by cloud vendors to tenants, which refers to the integration of cloud computing-related resources, including cloud services and cloud instances.

[0070] Cloud services may include, but are not limited to, virtual private cloud (VPC) services, gateway services, firewall services, NAT services, cloud disks, elastic public IP addresses (EIPs), cloud monitoring services, and other cloud services provided by various cloud vendors. Cloud instances may include, but not precede, virtual machines, containers, or bare metal servers, all of which are virtual instances provided by cloud vendors to tenants in their data centers.

[0071] 3. Tenant: The owner of cloud resources and the entity with management rights over them. Tenants can subscribe to or lease cloud resources from cloud vendors based on their business needs and pay to use the cloud services enabled by these resources. Tenants' management rights over purchased cloud resources may include, but are not limited to, security management rights and access control rights. In some embodiments, a tenant's electronic device may be referred to as a tenant node.

[0072] 4. User: A person who operates cloud resources and must be authorized by the cloud resource owner to operate cloud resources. In embodiments of the present application, user operations on cloud resources can be implemented through the user's electronic device (including physical devices and / or virtual devices). In some embodiments, the user's electronic device can be referred to as a user node.

[0073] It should be noted that, in the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0074] Furthermore, unless otherwise specified, ordinal numbers such as "first" and "second" in the embodiments of this application are used to distinguish between multiple objects and are not used to define the priority or importance of multiple objects. For example, the first device and the second device are only used to distinguish different electronic devices, and do not indicate a difference in priority or importance between the two devices. For example, in some embodiments, the method steps performed by the first device and the second device can be interchangeable.

[0075] The present application is described in detail below with reference to the accompanying drawings and embodiments.

[0076] FIG1 shows a schematic diagram of the architecture of a cloud service system according to an embodiment of the present application.

[0077] As shown in FIG1 , the cloud service system 100 may include a cloud management platform 110 and a backend system 120 provided by a cloud vendor for tenants.

[0078] The cloud management platform 110 can be connected to the user equipment (UE) operated by the tenant through the Internet. The tenant can order cloud services from the cloud vendor through the cloud management platform 110 and configure resources for the purchased cloud services. For example, a tenant can register an account on the cloud management platform 110 through a user device, and the account is eligible to purchase (including different payment forms such as subscription or leasing) cloud resources. After successfully purchasing cloud resources, the cloud management platform 110 can notify the corresponding back-end system 120 to create cloud resources for the tenant, and provide appropriate access methods to the tenant and the user authorized by the tenant, so that the tenant can remotely manage cloud resources and control the tenant / user's remote access or use of cloud resources to provide at least one cloud service for the tenant and the user.

[0079] For example, a cloud resource is a virtual machine. The tenant can select the specifications of the virtual machine (such as memory, processor, and disk, etc.) on the cloud management platform 110. After the tenant successfully pays, the cloud management platform 110 notifies the corresponding back-end system 120 to create a virtual machine with such specifications. The tenant can configure the access rights of the user authorized by the tenant to the virtual machine (such as logging in to the virtual machine, shutting down the virtual machine, deleting the virtual machine, etc.). After the configuration is successful, the user can perform the corresponding authorized operations on the virtual machine. The tenant can also grant the user the right to purchase or create a virtual machine. It should be understood that in the embodiment of the present application, cloud resources can also be, for example, various cloud services such as containers, bare metal servers, elastic public IPs (Elastic IP Address, EIP), and the embodiment of the present application does not limit the type of cloud services.

[0080] The cloud management platform 110 can be connected to the corresponding back-end system 120 through the internal network within the cloud service system 100. The back-end system 120 can provide various data processing functions such as access, processing, aggregation, and storage for requests or data from user devices or third-party devices, as well as provide tenants / users with functions such as query and use of stored data to provide corresponding cloud services to tenants / users.

[0081] In one example, the back-end system 120 can be implemented as a heterogeneous computing power providing system 200 based on cloud computing technology as shown in Figure 2 (hereinafter referred to as the heterogeneous computing power providing system 200). The heterogeneous computing power providing system 200 may include a computing power runtime node 210 and an infrastructure for providing cloud services.

[0082] The infrastructure may include a resource pool, such as servers, storage devices, network devices, security devices, virtualization software, storage software, and other related hardware and software devices required for computing resources, storage resources, network resources, and security resources. The computing power runtime node 210 can achieve unified management and scheduling of computing resources, storage resources, network resources, and security resources, and provide cloud services to tenants / users. At the same time, the computing power runtime node 210 can also support the security of cloud service operation and maintenance through relatively comprehensive cloud security defense, monitoring, analysis, and response methods, thereby achieving rapid discovery, isolation, and recovery of cloud security risks, threats, and attacks.

[0083] Exemplarily, as shown in FIG. 2 , the infrastructure may include, for example, multiple servers 220 and multiple heterogeneous hardware 230 .

[0084] The multiple servers 220 may also be referred to as a server cluster, and may include, for example, server 220_1, server 220_2, ..., server 220_n, etc., where n is an integer representing the number of servers. The multiple heterogeneous hardware 230 may also be referred to as a heterogeneous hardware cluster or a heterogeneous hardware resource pool, and may include, for example, heterogeneous hardware 230_1, heterogeneous hardware 230_2, ..., heterogeneous hardware 230_m, where m is an integer representing the number of heterogeneous hardware.

[0085] The processor of one of the multiple servers 220 may be a general-purpose processor, such as a central processing unit (CPU) or a microprocessor. One of the multiple servers 220 may be provided with at least one application (APP), and the processor and application may provide cloud services in the form of software calls.

[0086] The processor type of the multiple heterogeneous hardware 230 can be different from the type of the CPU of the multiple servers, and can form a "CPU+XPU" heterogeneous computing architecture with one of the multiple servers 220, and support the deployment of software development kits (SDKs) for some models or algorithms, so that any server can use the multiple heterogeneous hardware and the models or algorithms they support to achieve diverse computing power fusion and computing acceleration to improve the performance of the entire system.

[0087] In a specific implementation, the computing power runtime node 210 is located between multiple servers 220 and multiple heterogeneous hardware 230, and can be used to record the heterogeneous computing capabilities that heterogeneous hardware of different heterogeneous hardware types can provide. When an application installed on the server needs to use the heterogeneous computing capabilities of a preset heterogeneous hardware type, it can send a heterogeneous computing power request to the computing power runtime node 210. After receiving the heterogeneous computing power request from the application, the computing power runtime node 210 can provide the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the corresponding application based on the heterogeneous computing power request.

[0088] In order to improve the architectural consistency, scalability, versatility, portability and on-demand enabling capabilities of the heterogeneous computing power providing system 200, one implementation method is to design a computing power runtime framework, which enables the computing power runtime node to be oriented to different customer platforms, supports cloud resources for virtual machines, bare metal, containers and serverless scenarios, and allows different heterogeneous computing power resources to be registered through an abstract registration interface. When a tenant successfully purchases or rents cloud resources, the computing power runtime framework can create cloud resources for the tenant and build the resource pool required by the tenant according to the tenant's demand information or configuration information. Later, in the application stage, the computing power runtime framework can uniformly schedule the received heterogeneous computing power requests based on the resource pool to provide the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the corresponding application for use, so as to realize the relevant business of the application.

[0089] In one example, as shown in FIG2 , the computing power runtime node 210 may be logically divided into an API layer 211 , a Lib library 212 , a management service module 213 , and a computing power backend shim 214 .

[0090] Among them, the API layer 211 can be used to provide a unified interface specification, and support abstraction into a universal atomic API interface based on industry standard models, algorithms, etc., so that tenants can quickly build applications and services based on the API layer 211 and the Lib library 212. At the same time, it supports deployment in multiple forms (such as bare metal servers (or bare metal), virtual machines, containers, lightweight containers, serverless, etc.) without the need for tenants / users to pay attention to the underlying hardware. The Lib library 212 can be used in conjunction with the upper API layer 211 for application integration calls set up on the server. The computing power backend shim 214 can include multiple types of computing power shims, which can enable the computing power runtime node to support dynamic registration of SDKs of different heterogeneous hardware and different models and algorithms, and can also support virtualized computing power devices.

[0091] The management service module 213 can be used to implement registration management (including registration management during the later computing power expansion phase) and configuration management for different heterogeneous hardware, and establish and schedule tasks for heterogeneous computing power requests from applications, so as to dispatch the heterogeneous computing power requests to the appropriate target heterogeneous hardware. At the same time, the management service module 213 can also provide management of aspects such as computing power service quality, traffic statistics, and security during the process of the target heterogeneous hardware processing the heterogeneous computing power requests, to assist the management service module 213 in implementing services such as load balancing scheduling, accurate billing, and security protection.

[0092] Exemplarily, as shown in FIG3 , an application program set in one of the multiple servers 220 may be used to provide, but not limited to, at least one of the following services: rendering service, video encoding and decoding service, machine learning service, security service, artificial intelligence (AI) service, high performance computing (HPC) service, storage service, or network service.

[0093] The types of the multiple heterogeneous hardware may include, but are not limited to: CPU, co-processor, graphics processing unit (GPU), field programmable gate array (FPGA), various domain specific architectures (DSA), application-specific integrated circuit (ASIC), virtual computing device (Virtual Computing) or other types of processor engines (represented as xPU, where x represents unknown).

[0094] The Lib library 212 may include but is not limited to library files for implementing operations such as inference, training, encryption and decryption, data plane development tool set (Data Plane Development Kit, DPDK), storage performance development tool set (Storage Performance Development Kit, SPDK), data processor (Data Processing Unit, DPU), and codec.

[0095] The models or algorithms supported by the multiple heterogeneous hardware may include, but are not limited to, OpenGL, VAAPI, OpenCL, CUDA, DirectX, TensorFlow, etc. The computing power backend shims may include, but are not limited to, OpenGL shims, OpenCL shims, TensorFlow shims, PyTorch shims, virtualized computing power device shims (expressed as VM-shims), and shims for other types of heterogeneous hardware or algorithms.

[0096] The management service module 213 can be logically divided into task modules (tasks), scheduling modules (scheduler), quality of service (QoS) management modules, statistics modules (acoounting), security modules (security), registration modules (register) and configuration modules (configration) based on its functions.

[0097] Among them, the registration module can be used to dynamically register different heterogeneous computing resources (such as heterogeneous hardware) into the computing runtime framework. The registration process may include identification, verification, initialization and configuration of heterogeneous hardware devices, ensuring that heterogeneous hardware devices are correctly identified and configured in the heterogeneous computing power provision system so that heterogeneous hardware devices can be used correctly in the heterogeneous computing power provision system.

[0098] The configuration module can provide the parameter setting and control management process of the framework system, software, etc. of the entire computing power runtime node, including but not limited to the configuration of matters such as reliability, maintainability, scalability, locatability and fault diagnosis.

[0099] The task module can be managed according to the tenant's business needs, based on different algorithm types, task difficulty, etc., so as to assign heterogeneous computing power requests as tasks to appropriate heterogeneous hardware for execution.

[0100] To maximize resource utilization and task response time, the scheduling module can design a task scheduling algorithm. This scheduling algorithm can intelligently schedule heterogeneous computing power request tasks based on factors such as task type, difficulty, and the load of heterogeneous hardware resource pools. Once a task is scheduled to a resource pool or resource queue, the scheduling module also needs to send the task to the corresponding heterogeneous hardware for execution, monitor task execution to ensure timely completion, and feedback the task execution results to the application.

[0101] The QoS management module can be used to perform reasonable QoS management on heterogeneous computing power request tasks. It can be used in conjunction with the scheduling module to ensure the service experience of tenants / users and the performance and reliability of the system.

[0102] The statistics module manages statistics based on the heterogeneous computing requests and hardware resource usage received by nodes during computing runtime, ensuring the normal operation and maintenance of computing resources. For example, this statistical information can be used by cloud providers to provide detailed billing information and collect appropriate fees from tenants. Alternatively, this statistical information can be used in conjunction with the configuration module to debug heterogeneous hardware or related configuration information in maintenance scenarios.

[0103] The security module can be used to take appropriate security measures, such as data encryption, access control, firewalls, etc., to ensure the security of user data and privacy during the process of computing resource service.

[0104] It should be understood that the heterogeneous computing power provision system introduced above in conjunction with Figure 3 is only an example and not a limitation. In specific implementation scenarios, it is also necessary to combine specific business needs and technical implementation details to build the heterogeneous computing power provision system required by tenants, so as to achieve the best resource utilization and user experience as much as possible. In some embodiments, cloud vendors can provide different heterogeneous hardware required for general scenarios, and build the framework of the heterogeneous computing power provision system for tenants based on the tenants' subscription requirements. In other implementations, cloud vendors can also provide customized services to tenants, and build personalized heterogeneous computing power provision systems for tenants based on the tenants' business needs. This embodiment of the present application does not limit this.

[0105] The following takes the heterogeneous computing power providing system shown in Figure 3 as an example to introduce the functions of the functional modules and sub-functional units included in the heterogeneous computing power providing system of an embodiment of the present application.

[0106] 1. Standardization of API interface:

[0107] In an embodiment of the present application, the API layer of the computing power runtime node can provide a unified interface specification, including but not limited to standardizing API interface parameters, requests, responses, etc., to ensure that a unified interface is used on different platforms (such as different tenant platforms) to achieve the same function.

[0108] In one embodiment, the API layer can implement standardization of different API interfaces through code.

[0109] For example, the API layer can obtain at least one data access strategy sent by the framework developer of the computing power running node. The at least one data access strategy may include multiple fields and parameters entered by the developer for each field. The multiple fields are associated with the application programming interface API of the at least one data access strategy, which is provided by the cloud management platform. The multiple fields are used to characterize the different attributes of the at least one data access strategy. Thus, the computing power runtime node can provide the application with an API call interface for multiple algorithms supported by heterogeneous hardware, so that the application can send heterogeneous computing power requests to the computing power runtime node through API interface calls when it needs to implement computing power acceleration tasks.

[0110] Example (1) takes the encryption and decryption scenario as an example. In the specific implementation, for example, a set of symmetric encryption and decryption interfaces can be defined using C language, including input parameters and output parameters, to support the expansion of various symmetric encryption algorithms, keys, vectors, encryption modes, etc.

[0111] The code example of the symmetric encryption and decryption interface can be as follows:

[0112] It should be understood that the above is merely an example of code defining a symmetric encryption and decryption interface. In actual implementation, different encryption and decryption algorithms, key lengths, vector lengths, and encryption and decryption modes can be expanded based on actual needs. The specific implementation of computingRuntimeEncrypt and computingRuntimeDecrypt can be based on the selected encryption and decryption library and algorithm, with the management service module selecting different heterogeneous hardware for computing power acceleration.

[0113] Example (2) takes the HPC scenario as an example. In the specific implementation, we strive to use C language to define the HPC matrix multiplication interface, including the requirements of parameters such as input matrix, output matrix, matrix dimension, data type and storage format, so as to achieve the standardization of the HPC matrix multiplication interface.

[0114] The code example of the HPC matrix multiplication interface can be shown as follows:

[0115] In the above example (2), the computingRuntimeMatrixMultiply() function is a matrix multiplication interface function. Based on this interface function, by passing in pointers to input matrix A, input matrix B, and output matrix C, as well as parameters such as matrix dimensions, data type, and storage format, it can support HPC matrix multiplication interfaces with multiple data types and different storage formats, thus achieving interface standardization. Within the function, corresponding matrix multiplication calculations can be performed according to specific needs.

[0116] It should be understood that the matrix multiplication implementation code in the above example is only a simplified example. In actual applications, performance optimization and error handling can be performed as needed. The management service module can provide auxiliary processing support, which will not be repeated here.

[0117] Example (3) takes the HPC scenario as an example. In specific implementation, for example, the HPC reduction operation interface can be defined using C language to meet the requirements of parameters including input arrays or vectors, output results, and operation types, and support multiple data types to achieve normalization.

[0118] The code example of the HPC reduction operation interface is as follows:

[0119] In example (3), the computingRuntimeReduction() function is a reduction interface function. By passing in pointers to input arrays or vectors, pointers to output results, and parameters such as the array or vector size, data type, and reduction operation type, this function supports multiple data types and implements a standardized interface. Within the function, reduction operations can be performed based on specific needs, such as summing, finding the maximum value, and finding the minimum value.

[0120] The reduction operation implementation code in the above example is only a simplified example. In actual applications, performance optimization and error handling can be performed as needed. The management service module can provide auxiliary processing support, which will not be repeated here.

[0121] In other examples, cloud vendors can use methods similar to those in Examples (1) to (3) above to write code for standardizing API interfaces, supporting the standardization of some standard APIs (such as TensorFlow, PyTorch, DirectX, OpenGL, CUDA, OpenCL, VDPAU, VAAPI, Crypto, etc.) so that tenants or customer platforms can quickly build applications and services, while also accelerating the launch of new hardware by cloud vendors.

[0122] 2. Lib library:

[0123] In the embodiment of the present application, Lib is the abbreviation of Library. Lib is a library file directory for storing all library files useful to the system. A library file is a file required for the correct execution of an application, command, or process. The role of Lib is similar to that of a DLL file in Windows. Almost all applications require the use of shared library files in the Lib directory. In the embodiment of the present application, the Lib library includes but is not limited to library files for implementing various libraries such as reasoning, training, encryption and decryption, SDPK, DPDK, DPU, encoding and decoding, storage, and networking.

[0124] 3. Management service module:

[0125] As shown in FIG3 , the management service module can be logically divided into a task module, a scheduling module, a quality of service (QoS) management module, a statistics module, a security module, a registration module, and a configuration module.

[0126] To facilitate understanding, the following introduces the implementation details of the functions of each module of the management service module in stages based on the usage sequence of heterogeneous hardware.

[0127] (1) Registration phase for heterogeneous hardware:

[0128] In an embodiment of the present application, a computing power runtime node can be designed with the goal of dynamically registering different heterogeneous computing power resources. Based on this, when designing a computing power runtime framework, for example, different heterogeneous computing power resources (such as heterogeneous hardware) can be used as computing power objects, and the registration interfaces of different computing power objects can be abstracted and compiled into dynamic link library files. The computing power runtime framework dynamically loads the dynamic link library files and uses the execution-related registration code to complete the registration of the corresponding heterogeneous hardware. For example, the computing power runtime framework loads the dynamic link library files associated with the multiple heterogeneous hardware by calling the registration interface, and records the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types in a preset data structure storage medium.

[0129] In specific implementations, the computing runtime node can retrieve dynamic link library files entered or selected by developers of multiple heterogeneous hardware on the cloud management platform. When registering heterogeneous hardware, the computing runtime framework can proactively discover and complete the registration of different heterogeneous computing resources, or it can complete the registration of different heterogeneous computing resources after receiving online requests from different heterogeneous computing resources. This embodiment of the application does not limit this.

[0130] In one embodiment, the registration interface may be as follows:

[0131] In the above example of the registration interface, heterogeneous hardware is represented by defining the structure `hw_t`, and different `type` attributes and dynamic link library file names are used to distinguish different hardware types. When registering heterogeneous hardware, the `dlopen` and `dlsym` functions are used to load and obtain the hardware operation functions in the dynamic link library, and save them in the corresponding attributes of the `hw_t` structure. In this way, different hardware types can be selected dynamically. In the specific implementation process, different hardware operation functions can be continuously added according to different hardware types, such as encryption and decryption operation functions, reduction operation functions, matrix multiplication operation functions, etc., which are not limited in the embodiments of the present application.

[0132] Based on the registration interface described in the above example, taking the example of the computing power runtime framework (specifically, the registration module) actively discovering and completing the registration of different heterogeneous computing power resources, as shown in Figure 4, the discovery and registration process can include the following steps:

[0133] S401 (optional step): The framework developer provides a framework registration interface and a computing power usage interface.

[0134] S402: The heterogeneous hardware developer abstracts the registration interfaces of multiple heterogeneous hardware as required, so that the heterogeneous hardware implements a computing power runtime interface, wherein the computing power runtime interface may include both a registration interface and an operation interface implemented by the heterogeneous hardware.

[0135] S403: Compile the computing power resources of the heterogeneous hardware into a dynamic link library file through a heterogeneous hardware developer or an automated program (different hardware will be compiled into different dynamic link library files).

[0136] S404: The computing power runtime framework is configured and initialized, and a preset data structure storage medium (such as a linked list) is used to record the heterogeneous hardware type and computing power acceleration type.

[0137] S405: The computing power runtime framework calls the registration interface and loads the dynamic link library file in a polling manner to complete the registration of the heterogeneous hardware.

[0138] For example, the computing power runtime framework scans all dynamic link library files in the specified path of the computing power runtime system in a polling manner, adds the heterogeneous hardware and the computing power acceleration type implemented by the hardware to the preset data structure storage medium, completes the registration of all heterogeneous hardware, and establishes a heterogeneous hardware resource pool. Among them, for the convenience of management and control of multiple different types of heterogeneous hardware, different types of resource pools can be established, or different types of resource queues can be established in the same resource pool. In other words, one type of heterogeneous hardware can correspond to one resource queue or one resource pool.

[0139] When a new heterogeneous hardware needs to be added, just compile the computing power resources of the heterogeneous hardware into a dynamic link library file according to the above method. The computing power runtime framework will call the registration interface and load the corresponding dynamic link library file in a polling manner.

[0140] The above linked list is only an example of a data structure storage medium and is not intended to be limiting. In other embodiments, other data structure storage media may be used according to business scenarios or requirements, and this application embodiment does not limit this. Figure 5 shows a schematic diagram of a heterogeneous computing resource pool registered by four heterogeneous hardware.

[0141] As shown in Figure 5, for heterogeneous hardware A, heterogeneous hardware B, heterogeneous hardware C, and heterogeneous hardware D, the developers of heterogeneous hardware can abstract the registration interfaces of different heterogeneous hardware and compile their computing power resources into dynamic link library files. The computing power runtime framework can dynamically load the dynamic link library files to load the corresponding heterogeneous hardware into the resource pool.

[0142] The computing resources in the heterogeneous computing resource pool can be divided into different resource queues according to the type of algorithm. When the framework loads the dynamic link library file, the computing resources of the heterogeneous hardware can be loaded into the corresponding resource queue according to the type.

[0143] For example, if heterogeneous hardware A supports encryption and decryption operations, the computing resources of heterogeneous hardware A can be compiled into libcomputing-a.so and added to the encryption and decryption operation resource queues when registered with the computing runtime framework. Heterogeneous hardware B supports encryption and decryption operations, and the computing resources of heterogeneous hardware B can be compiled into libcomputing-b.so and added to the encryption and decryption operation resource queues when registered with the computing runtime framework. Heterogeneous hardware C supports matrix multiplication and reduction operations, and the computing resources of heterogeneous hardware C can be compiled into libcomputing-c.so and added to the matrix multiplication and reduction operation resource queues when registered with the computing runtime framework. Heterogeneous hardware D supports matrix multiplication operations, and the computing resources of heterogeneous hardware C can be compiled into libcomputing-d.so and added to the matrix multiplication operation resource queue when registered with the computing runtime framework. Different filled graphics schematically represent different operations supported by the same heterogeneous hardware.

[0144] (2) Operational phase of computing power runtime framework:

[0145] In an embodiment of the present application, the computing power runtime framework can be designed to support a scheduling algorithm, which can be used to schedule heterogeneous computing power requests from applications to appropriate heterogeneous hardware for processing.

[0146] For example, the scheduling algorithms supported by the computing power runtime framework may include but are not limited to the following examples:

[0147] 1) First-in-first-out (FIFO) scheduling algorithm: The scheduling module can schedule heterogeneous hardware in the order they are registered in the resource pool, using a first-in-first-out (FIFO) basis. That is, for heterogeneous hardware of the same type, heterogeneous computing power requests are dispatched to the heterogeneous hardware that registered first.

[0148] 2) Round Robin Scheduling Algorithm: The scheduling module selects heterogeneous hardware for scheduling in a round-robin manner, evenly distributing heterogeneous computing requests to available heterogeneous hardware. Each heterogeneous hardware processes the heterogeneous computing requests assigned to it in sequence and, upon completion, allocates them to the next heterogeneous hardware for processing. In some optional implementations, after a heterogeneous hardware completes processing a request, the processing result can be fed back to the scheduling module, which then allocates the next heterogeneous hardware.

[0149] 3) Lowest load first scheduling algorithm: The scheduling module schedules different heterogeneous hardware in the same resource queue according to the load weight, and selects the heterogeneous hardware with the lightest load and the least number of tasks from the resource queue for execution.

[0150] The above scheduling algorithms are examples only and are not intended to be limiting. In practice, cloud vendors can provide tenants with at least one scheduling algorithm, which tenants can configure based on their business needs. In other embodiments, more appropriate scheduling algorithms can be developed based on actual needs and business characteristics to implement scheduling and management of heterogeneous hardware in the resource pool. This description is omitted here.

[0151] The scheduling module can schedule heterogeneous computing power requests from applications based on its own scheduling algorithm. Taking the computing power acceleration task triggered by encryption operation as an example, as shown in Figure 6, the scheduling process can include the following steps:

[0152] S601 (optional step): The framework developer standardizes the various API interfaces of the API layer.

[0153] S602: When providing cloud services to tenants / users, the application triggers a computing power acceleration task when detecting heterogeneous computing capabilities using preset heterogeneous hardware types, such as encryption operations triggering a computing power acceleration task.

[0154] S603: The application uses the user-mode API to call the computing power runtime framework through the standardized API interface provided by the computing power runtime framework to send a heterogeneous computing power request to the computing power runtime node.

[0155] S604: The computing power runtime framework schedules the heterogeneous computing power request according to the task type to which the received heterogeneous computing power request belongs using the set scheduling algorithm, and provides the target heterogeneous hardware that meets the preset heterogeneous hardware type of the heterogeneous computing power request and can provide heterogeneous computing capabilities to the application for use.

[0156] S605: The computing power runtime framework sends the heterogeneous computing power request to a resource queue of the corresponding type that is registered and available in the resource pool.

[0157] S606: The resource queue of the corresponding type in the resource pool receives the heterogeneous computing power request and uses the target heterogeneous hardware to perform task processing on the received heterogeneous computing power request.

[0158] S607: After the task processing is completed, the target heterogeneous hardware returns the processing result to the computing power runtime framework.

[0159] S608: The computing power runtime framework returns the processing result to the application.

[0160] If there are multiple different applications that initiate heterogeneous computing power requests using the method shown in Figure 6, the computing power runtime framework can use the same method to schedule the heterogeneous computing power requests from different applications to appropriate heterogeneous hardware for processing.

[0161] For example, as shown in Figure 7, APP1, APP2, APP3, and APP4 represent four different applications. When providing cloud services to tenants / users, APP1, APP2, APP3, and APP4 can all follow the method shown in Figure 6. After triggering a computing power acceleration task, they can send heterogeneous computing power requests to the computing power runtime framework by calling the computing power runtime API provided by the computing power runtime framework. The computing power runtime API can then submit these heterogeneous computing power requests to the computing power runtime framework's scheduling module. This scheduling module can then schedule tasks based on a pre-defined scheduling algorithm, the type of computing power operation required, resource pool load, and other factors, and provide the corresponding APPs with target heterogeneous hardware that meets the requirements. For example, a heterogeneous computing power request from APP1 can be scheduled to the resource queues corresponding to encryption and decryption operations; a heterogeneous computing power request from APP2 can be scheduled to the resource queues corresponding to encryption and decryption operations; a heterogeneous computing power request from APP3 can be scheduled to the resource queue corresponding to matrix multiplication operations; and a heterogeneous computing power request from APP4 can be scheduled to the resource queues corresponding to matrix multiplication and reduction operations.

[0162] 4. Hash power backend shim layer:

[0163] In an embodiment of the present application, the computing power backend shim layer of the computing power runtime node may include shims (Shims) that support different types of heterogeneous hardware, SDKs of different models and algorithms, support dynamic registration of heterogeneous hardware, and support virtualized computing power devices.

[0164] For example, for different cloud resources and different types of heterogeneous hardware, in different deployment scenarios, the heterogeneous computing power providing system of the embodiment of the present application can be deployed as different architectures. The following is an example introduction to the system architecture diagram and its deployment method in the optional implementation method.

[0165] (1) Single-machine (single server) deployment architecture:

[0166] In an embodiment of the present application, the computing power runtime node can simultaneously support pure bare metal deployment, container deployment on bare metal, or serverless deployment on bare metal, and can also support virtual machine deployment, lightweight virtual machine or secure container deployment, as shown in Figure 8.

[0167] When deploying this architecture, as shown in Figure 9, you can first deploy the system in step ①, configure and register heterogeneous hardware computing resources. The registration process can be found in the previous text in conjunction with the relevant description of Figures 4-5, which will not be repeated here. Then, in step ②, the application deployed in the client space (Guest Space) and / or host space (Host Space) creates and manages tasks through the task module by calling the interface provided by the computing power runtime node, and schedules tasks through the scheduling module to select appropriate heterogeneous hardware computing resources for different applications. This process can be found in the previous text in conjunction with the relevant description of Figures 6-7, which will not be repeated here.

[0168] (2) Virtual machine deployment architecture:

[0169] The virtual machine deployment scenarios can include two situations:

[0170] 1) Directly pass PCI hardware computing power devices (including SR-IOV) to the virtual machine. This method is similar to the bare metal deployment method. Please refer to the introduction of Figures 8 and 9, and will not be repeated here.

[0171] 2) Presenting virtual hardware computing devices to virtual machines. The specific deployment form can be shown in Figure 10. The virtual computing power front end is the virtual machine hardware device, which can be in the form of a PCI device, shared memory, or socket device.

[0172] The deployment process of the architecture shown in Figure 10 is shown in Figure 11 and may include the following steps:

[0173] ①: Deploy the computing runtime node on the host side, configure and register heterogeneous hardware computing resources.

[0174] ②: Configure and simulate virtual computing devices for the virtual machine and start the virtual machine.

[0175] ③: Deploy the computing power runtime node on the virtual machine side (i.e. the client side), configure and register the virtual hardware computing power device.

[0176] ④: The application calls the computing runtime node's interface and uses the task management module to create and manage tasks. The scheduling management module schedules tasks and selects appropriate computing resources for the application (here, the virtual computing front-end device is selected). The statistics module also generates performance statistics. Data flows are forwarded by the virtual computing front-end device to the host's virtual machine simulator.

[0177] ⑤: The virtual machine simulator enters the computing power runtime framework on the host side through the data forwarding module. Among them, the data forwarding module supports deployment in the virtual machine simulator and can also support independent user-mode process deployment, which is not limited in this embodiment of the application.

[0178] ⑥: Based on the host-side computing power runtime framework, complete the selection and use of heterogeneous computing power resources.

[0179] (3) Uninstall card deployment architecture:

[0180] As shown in Figure 12, the backend offload card can serve as an external device for the frontend physical server. Since each server can be equipped with this offload card, it provides general-purpose CPU computing resources and can also include other heterogeneous resources such as NPUs and GPUs. Cloud management software can be offloaded to the card and connected to the cloud network and cloud storage. In this case, management plane operation requests issued by tenants / users through the cloud management platform are only sent to the offload card.

[0181] The heterogeneous provision system of the embodiment of the present application can be applied to the architecture using the offload card to support the external Serverless service provision of the offload card computing resources and cloud service resources, can be used for offload acceleration of customer business, and can facilitate the sale of the offload card as an independent device.

[0182] In this scenario, the computing power runtime node and heterogeneous hardware can be deployed inside the offload card, as shown in Figure 13, so that the virtual machine manager deployed on the server can communicate with the computing power runtime node on the offload card through a high-speed interconnection channel (such as PCle, etc.), and use the corresponding heterogeneous hardware to accelerate computing power to provide corresponding cloud services for tenants / users.

[0183] This offload card deployment solution can support serverless storage services and network services. Furthermore, this deployment solution allows the computing resources and other capabilities on the offload card to be provided as services to front-end servers and virtual machines / secure containers, for use in offload acceleration or sold as standalone products. Similarly, in this architecture, the front-end and back-end connections can be virtual devices, shared memory, vsock, and other forms. In some embodiments, this offload card deployment solution can also support distributed pooling expansion.

[0184] (4) Pooled deployment architecture within a data center (e.g., rack server):

[0185] The heterogeneous provision system of the embodiment of the present application can be applied to the architecture deployment of a data center to support the pooled deployment of heterogeneous computing power and improve the overall resource utilization.

[0186] As shown in FIG14 , a data center may include multiple nodes and multiple resource pools. When the heterogeneous computing power system of the present application is established in the data center, each node can serve as a computing power runtime node. The multiple resource pools may include, for example, a GPU resource pool, an FGPA resource pool, a DSA resource pool, or other types of xPU resource pools. Each node may interact with different resource pools through a switch to distribute heterogeneous computing power requests triggered by the implementation business to the appropriate heterogeneous hardware computing power resource pool for processing. The switch may support, but is not limited to, high-speed interconnection protocols such as UB / CXL.

[0187] This deployment architecture supports fast interconnection within the Rack (extremely low latency), enabling the expansion of heterogeneous hardware devices in the computing runtime system from single-host resources to remote heterogeneous computing power, thus supporting pooled deployment of heterogeneous computing power. Furthermore, using high-speed interconnect buses (such as Unified Bus and CXL), computing resources of different types and architectures (such as GPUs, FPGAs, DSAs, and xPUs) can be integrated into a unified computing power pool. Through resource pooling and scheduling management technologies, these heterogeneous computing resources can be uniformly managed and efficiently utilized.

[0188] Similarly, the deployment process of this architecture can include the following processes:

[0189] 1) Resource Pooling: Computing resources of different types and architectures are registered into a unified resource pool. This registration process involves identifying, registering, and managing different types of hardware resources (including heterogeneous hardware resources), including the collection and storage of information such as hardware characteristics, performance indicators, and status. The implementation method can be found in the previous description in conjunction with Figures 4 and 5, and will not be repeated here.

[0190] 2) Resource Management: Manage resources in the resource pool, including registration and deregistration, status monitoring, performance statistics, and fault handling. This can be achieved through interaction with hardware resources and management interfaces, such as encapsulating and managing hardware resource drivers, APIs, and SDKs. For implementation details, please refer to the previous descriptions of the API interface standardization process, heterogeneous hardware registration process, and the computing power backend shim layer, and will not be repeated here.

[0191] 3) Resource Scheduling: Design and implement appropriate scheduling algorithms to ensure that computing runtime nodes allocate and schedule resources appropriately based on task requirements, resource characteristics, and the resource status in the resource pool. This includes task scheduling strategies, resource allocation strategies, resource status monitoring, and dynamic resource adjustments. The implementation of this process is similar to the previous description of the task module, scheduling module, management service module, and statistics module. Please refer to the relevant description above and will not be repeated here.

[0192] 4) Heterogeneous computing power orchestration: Heterogeneous computing power is orchestrated and configured based on task characteristics and resource capabilities. For example, compute-intensive tasks can be assigned to high-performance GPU resources, while data processing tasks can be assigned to FPGA resources suitable for processing large amounts of data.

[0193] 5) Security Management: Ensure the security of resources and tasks in the resource pool during deployment, scheduling, and execution. This may involve measures such as permission management, identity authentication, data encryption, and security isolation to protect the security of resources and tasks.

[0194] 6) Performance Optimization: Based on the characteristics of different hardware resources and task requirements, we optimize performance, including resource utilization, response time, and throughput, to improve overall computing performance. The implementation of this part is similar to the functionality of the statistics module introduced above, so please refer to the relevant description above and will not be repeated here.

[0195] (5) Pooled deployment architecture across data centers (e.g., Rack):

[0196] The heterogeneous provisioning system of the embodiment of the present application is applicable to scenarios involving multiple data centers. As shown in FIG15 , in this scenario, communication and interaction can be carried out within the data center through switches and high-speed interconnect links. Communication and interaction can be carried out between different data centers, i.e., across racks, through switches and Ethernet / Remote Direct Memory Access (RDMA) links. This architecture enables tenants to deploy and use services on demand, helping to improve resource utilization of the overall architecture.

[0197] The above heterogeneous computing power system and architecture examples introduce the scenarios and architectures applicable to the heterogeneous provision system of the embodiment of the present application. These architectures can support simultaneous deployment on bare metal and virtual machine forms of cloud vendors and customers (similar to virtual computing power devices), and can support containers and Serverless scenarios (supporting both micro VM and independent process modes), which helps to achieve architectural consistency. At the same time, the heterogeneous computing power provision system can enable different types of heterogeneous hardware to be registered in the framework in a service-oriented, modular, or plug-in manner, and can support the expansion of different computing powers, such as the addition of GPUs, xPUs, FPGAs, ASICs, etc., and support distributed deployment and pooled deployment and expansion of computing power. At the same time, the heterogeneous computing power provision system can support industry standard APIs (TensorFlow, PyTorch, DirectX, OpenGL, CUDA, OpenCL, VDPAU, VAAPI, Crypto, etc.). Based on the computing power runtime architecture, cloud tenants / cloud users can quickly build applications and services, and cloud vendors can also speed up the launch of new hardware. In this, the application is independent of the hardware, which facilitates seamless porting of applications. Cloud vendors can also enable computing power acceleration on demand in a service-oriented manner based on different cloud services and different types of heterogeneous hardware, accurately control computing power QoS, and appropriately measure and charge fees.

[0198] Figure 16 shows a flow chart of the heterogeneous computing power provision method of an embodiment of the present application. This method can be executed by a computing power runtime node deployed in the system architecture shown in Figures 1, 2, 3, 8, 10, and 13-15 above. The heterogeneous computing power provision system also includes an infrastructure, wherein the infrastructure includes multiple servers and multiple heterogeneous hardware, wherein the types of the multiple heterogeneous hardware are different from the types of the CPUs of the multiple servers, and the computing power runtime node records the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types.

[0199] As shown in FIG16 , the method may include the following steps:

[0200] S1601: Receive a heterogeneous computing power request from an application, where the heterogeneous computing power request is associated with heterogeneous computing capabilities of a preset heterogeneous hardware type that the application needs to use;

[0201] S1602: According to the heterogeneous computing power request, target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities is provided to the application for use.

[0202] The specific implementation details of the above-mentioned heterogeneous computing power provision method can be found in the relevant introduction in conjunction with Figures 1 to 13 in the previous text, and will not be repeated here.

[0203] In combination with the above method embodiments, the embodiments of the present application also provide a heterogeneous computing power providing device based on cloud computing technology, which can be used to execute the method executed by the node during computing power runtime in the above method embodiments.

[0204] Among them, the computing power runtime node can belong to the heterogeneous computing power providing system based on cloud computing technology introduced above. The heterogeneous computing power providing system also includes infrastructure, and the infrastructure includes multiple servers and multiple heterogeneous hardware. The types of the multiple heterogeneous hardware are different from the types of the CPUs of the multiple servers. The computing power runtime node records the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types.

[0205] As shown in Figure 17, the device 1700 may include: a communication interface 1701, used to receive a heterogeneous computing power request from an application, wherein the heterogeneous computing power request is associated with the heterogeneous computing capabilities of a preset heterogeneous hardware type that the application needs to use; a processing unit 1702, used to provide the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application for use according to the heterogeneous computing power request.

[0206] It should be noted that the division of units in the embodiments of the present application is schematic and is merely a logical functional division. In actual implementation, other division methods may be used. The functional units in the embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0207] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0208] This application also provides a computing device 1800. As shown in Figure 18, computing device 1800 includes a bus 1802, a processor 1804, a memory 1806, and a communication interface 1808. Processor 1804, memory 1806, and communication interface 1808 communicate with each other via bus 1802. Computing device 1800 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 1800.

[0209] Bus 1802 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG18 illustrates a single bus line, but this does not imply a single bus or type of bus. Bus 1802 may include a path for transmitting information between various components of computing device 1800 (e.g., memory 1806, processor 1804, and communication interface 1808).

[0210] The processor 1804 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0211] The memory 1806 may include volatile memory, such as random access memory (RAM). The processor 1804 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0212] Memory 1806 stores executable program code, and processor 1804 executes the executable program code to implement the functions of the aforementioned cloud management platform and computing power runtime node, thereby implementing the method of the embodiment of the present application. In other words, memory 1806 stores instructions for executing the method of the embodiment of the present application.

[0213] The communication interface 1808 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1800 and other devices or a communication network.

[0214] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0215] As shown in Figure 19, the computing device cluster includes at least one computing device 1800. The memory 1806 in one or more computing devices 1800 in the computing device cluster may store the same instructions for executing the method of the embodiment of the present application.

[0216] In some possible implementations, the memory 1806 of one or more computing devices 1800 in the computing device cluster may also store partial instructions for executing the method of the embodiment of the present application. In other words, the combination of one or more computing devices 1800 can jointly execute the instructions for executing the method of the embodiment of the present application.

[0217] It should be noted that the memory 1806 in different computing devices 1800 in the computing device cluster can store different instructions, each used to perform a portion of the functions of the cloud service system. In other words, the instructions stored in the memory 1806 in different computing devices 1800 can implement the functions of one or more modules of the cloud management platform and the computing power runtime node.

[0218] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network. The network may be a wide area network, a local area network, or the like. FIG20 illustrates a possible implementation. As shown in FIG20 , two computing devices 1800A and 1800B are connected via a network. Specifically, the connection to the network is made via a communication interface in each computing device. In this type of possible implementation, the memory 1806 in the computing device 1800A stores instructions for executing the functions of the cloud management platform. At the same time, the memory 1806 in the computing device 1800B stores instructions for executing the functions of a computing power runtime node.

[0219] The connection method between the computing device clusters shown in Figure 20 can be based on the consideration that the heterogeneous computing power provision method provided in this application requires users to customize functional configuration (such as storing large amounts of data) and implement computing functions, so it is considered to entrust the functions implemented by the computing power runtime node to the computing device 1800B for execution.

[0220] It should be understood that the functionality of the computing device 1800A shown in FIG9 may also be implemented by multiple computing devices 1800. Similarly, the functionality of the computing device 1800B may also be implemented by multiple computing devices 1800.

[0221] The present application also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to the connection method of the computing device cluster described in Figures 19 and 20. The difference is that the memory 1806 of one or more computing devices 1800 in the computing device cluster can store the same instructions for executing the heterogeneous computing power provision method.

[0222] In some possible implementations, the memory 1806 of one or more computing devices 1800 in the computing device cluster may also store some instructions for executing the method for providing heterogeneous computing power. In other words, the combination of one or more computing devices 1800 can jointly execute the instructions for executing the method for providing heterogeneous computing power.

[0223] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored on any available medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to execute the heterogeneous computing power provision method.

[0224] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the heterogeneous computing power provision method.

[0225] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the various embodiments of the present invention.

[0226] An embodiment of the present application also relates to a processor, which is used to call a computer program or computer instruction stored in a memory so that the processor executes the above method embodiment.

[0227] The processor mentioned in any of the above may be a general-purpose central processing unit, a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the method in the embodiment shown in FIG9 . The memory mentioned in any of the above may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM), etc.

[0228] It should be understood that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0229] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0230] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0231] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the scope of the embodiments of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the present application also intends to include such modifications and variations.

Claims

1. A heterogeneous computing power provision system based on cloud computing technology, It is characterized in that include: An infrastructure comprising a plurality of servers and a plurality of heterogeneous hardware, wherein a type of the plurality of heterogeneous hardware is different from a type of CPUs of the plurality of servers; Computing runtime nodes are used to record the heterogeneous computing capabilities that heterogeneous hardware of different types can provide; An application, disposed in one of the plurality of servers of the infrastructure, sends a heterogeneous computing power request to the computing power runtime node when heterogeneous computing power of a preset heterogeneous hardware type is required; The computing power runtime node is also used to provide the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application program for use according to the heterogeneous computing power request.

2. The system according to claim 1, It is characterized in that Before the computing power runtime node records the heterogeneous computing capabilities that heterogeneous hardware of different heterogeneous hardware types can provide, it is also used to: By calling the registration interface, the dynamic link library files associated with the plurality of heterogeneous hardware are loaded, and the heterogeneous computing capabilities that can be provided by the heterogeneous hardware of different heterogeneous hardware types are recorded in a preset data structure storage medium.

3. The system according to claim 2, It is characterized in that The computing power runtime node is also used for: The dynamic link library file input or selected by the developers of the plurality of heterogeneous hardware on the cloud management platform is obtained.

4. The system according to any one of claims 1 to 3, It is characterized in that Before the application sends a heterogeneous computing power request to the computing power runtime node, the computing power runtime node is further used to: An API calling interface of the algorithms supported by the multiple heterogeneous hardware is provided to the application program, so that the application program sends a heterogeneous computing power request to the computing power runtime node by calling the API interface.

5. The system according to claim 4, It is characterized in that The computing power runtime node is also used for: Obtain at least one data access strategy sent by a framework developer of the computing power running node, wherein the at least one data access strategy includes multiple fields and parameters input by the developer for each field, the multiple fields are associated with an application programming interface API of the at least one data access strategy, and the API is provided by a cloud management platform, and the multiple fields are used to characterize different attributes of the at least one data access strategy.

6. The system according to any one of claims 1 to 5, It is characterized in that The plurality of heterogeneous hardware belong to at least one resource queue, and the computing power runtime node provides the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application program for use according to the heterogeneous computing power request, including: Determine a target resource queue according to the heterogeneous computing power request; The target heterogeneous hardware in the target resource queue that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities is provided to the application for use.

7. The system according to claim 6, It is characterized in that The computing power runtime node uses any one of the following scheduling algorithms to provide the target heterogeneous hardware in the target resource queue that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application: First-in-first-out scheduling algorithm; polling scheduling algorithm or lowest load first scheduling algorithm.

8. The system according to any one of claims 1 to 7, It is characterized in that One of the multiple servers of the infrastructure includes any of the following types: a bare metal server, a virtual machine, a container, a lightweight container, or a serverless server.

9. The system according to any one of claims 1 to 8, It is characterized in that The architecture of the heterogeneous computing power providing system includes an offload card, the computing power runtime node is deployed in the offload card, and part or all of the multiple heterogeneous hardware belong to the offload card.

10. The system according to any one of claims 1 to 8, It is characterized in that The heterogeneous computing power system is deployed on a rack server in the same data center, wherein the multiple computing nodes in the data center are the computing power runtime nodes, and different computing power runtime nodes communicate and interact with each other through switches.

11. The system according to any one of claims 1 to 8, It is characterized in that The computing power runtime nodes and infrastructure cross-rack servers of the heterogeneous computing power system are deployed on rack servers in different data centers, and different rack servers communicate and interact with each other through switches.

12. A method for providing heterogeneous computing power based on cloud computing technology. It is characterized in that Applied to a computing power runtime node, the computing power runtime node belongs to a heterogeneous computing power providing system based on cloud computing technology, the heterogeneous computing power providing system also includes an infrastructure, the infrastructure includes multiple servers and multiple heterogeneous hardware, the types of the multiple heterogeneous hardware are different from the types of CPUs of the multiple servers, the computing power runtime node records the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types, and the method includes: Receiving a heterogeneous computing power request from an application, the heterogeneous computing power request being associated with heterogeneous computing capabilities of a preset heterogeneous hardware type that the application needs to use; According to the heterogeneous computing power request, target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities is provided to the application for use.

13. The method according to claim 12, It is characterized in that Also includes: By calling the registration interface, the dynamic link library files associated with the plurality of heterogeneous hardware are loaded, and the heterogeneous computing capabilities that can be provided by the heterogeneous hardware of different heterogeneous hardware types are recorded in a preset data structure storage medium.

14. The method according to claim 13, It is characterized in that Also includes: The dynamic link library file input or selected by the developers of the plurality of heterogeneous hardware on the cloud management platform is obtained.

15. The method according to any one of claims 12 to 14, It is characterized in that Also includes: An API calling interface of the algorithms supported by the multiple heterogeneous hardware is provided to the application program, so that the application program sends a heterogeneous computing power request to the computing power runtime node by calling the API interface.

16. The method according to claim 15, It is characterized in that Also includes: Obtain at least one data access strategy sent by a framework developer of the computing power running node, wherein the at least one data access strategy includes multiple fields and parameters input by the developer for each field, the multiple fields are associated with an application programming interface API of the at least one data access strategy, and the API is provided by a cloud management platform, and the multiple fields are used to characterize different attributes of the at least one data access strategy.

17. The method according to any one of claims 12 to 16, It is characterized in that The plurality of heterogeneous hardware belong to at least one resource queue, and providing the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application for use according to the heterogeneous computing power request, including: Determine a target resource queue according to the heterogeneous computing power request; The target heterogeneous hardware in the target resource queue that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities is provided to the application for use.

18. The method according to claim 17, It is characterized in that The target heterogeneous hardware in the target resource queue that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities is provided to the application program using any of the following scheduling algorithms: First-in-first-out scheduling algorithm; polling scheduling algorithm or lowest load first scheduling algorithm.

19. The method according to any one of claims 12 to 18, It is characterized in that One of the multiple servers of the infrastructure includes any of the following types: a bare metal server, a virtual machine, a container, a lightweight container, or a serverless server.

20. The method according to any one of claims 12 to 19, It is characterized in that The architecture of the heterogeneous computing power providing system includes an offload card, the computing power runtime node is deployed in the offload card, and part or all of the multiple heterogeneous hardware belong to the offload card.

21. The method according to any one of claims 12 to 19, It is characterized in that The heterogeneous computing power system is deployed on a rack server in the same data center, wherein the multiple computing nodes in the data center are the computing power runtime nodes, and different computing power runtime nodes communicate and interact with each other through switches.

22. The method according to any one of claims 12 to 19, It is characterized in that The computing power runtime nodes and infrastructure cross-rack servers of the heterogeneous computing power system are deployed on rack servers in different data centers, and different rack servers communicate and interact with each other through switches.

23. A heterogeneous computing power providing device based on cloud computing technology, It is characterized in that Applied to a computing power runtime node, the computing power runtime node belongs to a heterogeneous computing power providing system based on cloud computing technology, the heterogeneous computing power providing system also includes an infrastructure, the infrastructure includes multiple servers and multiple heterogeneous hardware, the types of the multiple heterogeneous hardware are different from the types of CPUs of the multiple servers, the computing power runtime node records the heterogeneous computing capabilities that can be provided by heterogeneous hardware of different heterogeneous hardware types, and the device includes: A communication interface for receiving a heterogeneous computing power request from an application, wherein the heterogeneous computing power request is associated with a heterogeneous computing capability of a preset heterogeneous hardware type that the application needs to use; The processing unit is used to provide the target heterogeneous hardware that meets the preset heterogeneous hardware type and can provide heterogeneous computing capabilities to the application program for use according to the heterogeneous computing power request.

24. A computer program product comprising instructions, It is characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 12 to 22.

25. A computer-readable storage medium, It is characterized in that The method comprises computer program instructions, and when the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method as claimed in any one of claims 12 to 22.

26. A computing device cluster, It is characterized in that comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 12 to 22.

Citation Information

Patent Citations

  • Computing power processing network system, service processing method and equipment

    CN114095579A

  • Construction method of universal heterogeneous acceleration framework

    CN114282641A

  • Embedded heterogeneous computing scheduling system and method

    CN114924856A

  • Computing power unloading method and device, electronic equipment and storage medium

    CN115509644A

  • Resource Provider SDK

    US20170168813A1

Cited By

  • Distributed task memory allocation method and device in heterogeneous system, medium and product

    CN119621355A

  • Multi-agent computing power scheduling method and device for intelligent computing center cloud platform

    CN120256148A

  • Hierarchical abstraction method and system for computing resources

    CN120371317A

  • Hardware virtualization server system and intelligent scheduling method

    CN120407126A

  • Tidal computing power scheduling system and scheduling method oriented to heterogeneous computing power resource pool

    CN120950209A