A computing power overselling method and system compatible with multi-architecture GPUs and a storage medium

Abstract

This invention relates to a method, system, and storage medium for overselling computing power compatible with multi-architecture GPUs. The method includes: acquiring GPU computing power resources of different architectures from different suppliers; classifying each GPU computing power resource according to its manufacturer to form different resource pools; generating a logical computing power inventory available for listing and trading based on resource parameters, historical load data, and real-time pressure monitoring values; obtaining the willingness to trade between the demander and the supplier; calculating whether the logical computing power inventory of the supplier's GPU computing power resources matches the demander's task requirements based on the willingness to trade; if they match, matching the transaction; generating a time-sensitive identity authentication token and sending it to the demander, which is used to jump to and access the corresponding GPU computing power resources for computing operations. Compared with the prior art, this invention has advantages such as improved resource utilization, flexibility in computing power allocation, operational cost efficiency, and system stability.

Description

Technical Field

[0001] This invention relates to the field of computing power supply technology, and in particular to a method, system and storage medium for overselling computing power compatible with multi-architecture GPUs. Background Technology

[0002] As the scale and complexity of artificial intelligence (AI) models increase, the demand for underlying computing resources is also expanding dramatically. For example, invention CN120315884A discloses a multi-cluster heterogeneous computing power scheduling method for AI training and inference tasks. This method uses a scheduling mechanism centered on dispatched AI tasks, monitors the resource pool status in real time, dynamically allocates resources, ensures efficient execution of AI tasks, improves the overall resource pool utilization efficiency, monitors the load status of AI application instances in real time, and automatically scales up and down AI application instances through elastic scaling to ensure stable operation of AI tasks. It integrates the resources of multiple clusters into a unified resource pool, achieving centralized management and collaborative scheduling of resources. It also monitors the load of each cluster in real time, automatically performs load balancing between clusters, and rationally distributes AI tasks to different clusters. This solution enables the allocation of AI tasks to different clusters for processing.

[0003] However, current computing power supply platforms typically contain various graphics processing unit (GPU) resources with different architectures, such as NVIDIA's CUDA architecture, AMD's ROCm architecture, and dedicated architectures of domestically produced chips (such as Huawei Ascend and Hygon DCU). Problems and shortcomings of existing technologies: 1. Architectural compatibility barriers: Existing computing power scheduling is typically "architecture-bound." Tenant code tasks often depend on specific drivers (such as CUDA), making smooth migration between heterogeneous GPUs impossible. This forces computing platforms to divide resources into independent "NVIDIA pools" and "AMD pools," preventing the formation of a unified resource pool and reducing overall scheduling flexibility.

[0004] 2. Low resource utilization and lack of over-selling mechanism: Traditional scheduling adopts a "pass-through mode," where a physical graphics card can only be exclusively used by one tenant at a time. That is, the tenant's business load is extremely low, and the graphics card resources are completely locked, making it impossible to over-sell them like CPUs.

[0005] 3. Heterogeneous overselling is technically difficult to implement: GPUs of different architectures have completely different implementation mechanisms for context switching and memory isolation. Existing technologies lack a unified intermediate layer to hide these underlying differences, making it difficult to achieve unified overselling management across architectures.

[0006] 4. Disconnect between transactions and payments: Most existing platforms are information matching platforms, lacking a closed-loop technical link from "demand release - bidding and transaction - jump to use", which makes it difficult to schedule resources in real time after overselling. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art, such as architectural compatibility barriers and low resource utilization, and to provide a method, system and storage medium for overselling computing power compatible with multi-architecture GPUs.

[0008] The objective of this invention can be achieved through the following technical solutions: A method for overselling computing power compatible with multi-architecture GPUs includes the following steps: Acquire GPU computing power resources with different architectures from different suppliers, classify each GPU computing power resource according to the manufacturer to form different resource pools, and generate a logical computing power inventory that can be listed for trading based on the parameters, historical load data and real-time pressure monitoring values ​​of each GPU computing power resource. The system obtains the willingness of the demand side and the supplier to reach a transaction. Based on the demand side's task requirements, the system determines the required vendors and selects GPU computing power resources from the corresponding resource pool. Based on the willingness of the demand side, the system determines whether the supplier's logical computing power inventory of GPU computing power resources matches the demand side's task requirements. If they match, the system matches the transaction. After a transaction is matched, a time-sensitive identity authentication token is generated and sent to the requesting party. The requesting party uses the identity authentication token to access the GPU computing resources in the corresponding resource pool to perform computing tasks.

[0009] Furthermore, the process of generating the logical computing power inventory available for listing and trading includes: The parameters of each GPU computing resource are converted into unified standard computing parameters to form a standardized service capability profile; Based on the historical load data and real-time pressure monitoring values ​​of each GPU computing power resource, a dynamic overselling factor is added to the service capability profile to generate the logical computing power inventory available for listing and trading.

[0010] Furthermore, the dynamic overselling factor is set based on the historical average utilization rate of GPU computing resources.

[0011] Furthermore, the calculation process for the available logical computing power inventory of the GPU computing power resources includes: The maximum value of the historical average utilization rate of GPU computing resources is used as the benchmark. If the value is within the preset oversold threshold range, the difference between the oversold threshold and the benchmark value is used as the oversold factor. The oversold factor is multiplied by the total computing power of the GPU computing resources to obtain the oversold reserve, which is then added to the actual physical reserve of the GPU computing resources to obtain the logical computing power inventory. Otherwise, the actual physical reserve of the GPU computing resources is used as the logical computing power inventory.

[0012] Furthermore, the oversold factor is dynamically updated at fixed time intervals.

[0013] Furthermore, the method employs a listing and delisting model to generate the transaction completion intention; The listing and bidding process includes: the demand side publishes a task requirement profile containing specific computational characteristics and lists it in a pre-set public demand pool; the supplier proposes a bidding action based on the task requirement profile, generating the transaction completion intention.

[0014] Furthermore, the specific computing characteristics include video memory, computing power precision, and environment mirroring.

[0015] Furthermore, the method also includes: when the requester uses GPU computing resources to perform computing jobs, monitoring the physical computing load of the corresponding GPU computing resources; if the physical computing load exceeds a preset load threshold, suspending or migrating part of the computing job to other GPU computing resources in the resource pool where the GPU computing resources are located.

[0016] This invention also provides a computing power overselling system that implements the above-described method for overselling computing power compatible with multi-architecture GPUs, comprising: The management center is used to acquire GPU computing power resources with different architectures from different vendors, classify each GPU computing power resource according to the vendor to form different resource pools, and generate a logical computing power inventory that can be listed for trading based on the parameters, historical load data and real-time pressure monitoring values ​​of each GPU computing power resource; after the transaction is matched, a time-sensitive identity authentication token is generated and sent to the demand party. The demand side is used to submit task requirements. Based on the demand side's task requirements, the required vendors are determined to select GPU computing resources from the corresponding resource pool, generate a transaction agreement, and jump to and access the corresponding GPU computing resources in the resource pool based on the identity authentication token for computing operations. Suppliers are responsible for listing their GPU computing power resources and generating transaction intentions. The review panel is used to calculate whether the supplier's logical computing power inventory of GPU computing resources matches the task requirements of the demander based on the willingness of the demander and the supplier to reach a transaction. If they match, the transaction is matched.

[0017] The present invention also provides a computer-readable storage medium on which a computer program is stored, the computer program being executed by a processor using the method described above.

[0018] Compared with the prior art, the present invention has the following advantages: (1) The present invention converts the parameters, historical load data and real-time pressure monitoring values ​​of each GPU computing power resource into a unified standard logical computing power inventory to realize the state synchronization of GPU computing power resources of different architectures, and classifies them according to the manufacturers to form different resource pools for scheduling layer isolation, thereby realizing operation layer isolation, which greatly improves resource utilization, computing power allocation flexibility, operating cost efficiency, system stability and multi-tenant isolation. During the transaction process, the willingness to complete the transaction is reviewed based on whether the supplier's logical computing power inventory matches the demander's task requirements. When the transaction is completed, a time-sensitive identity authentication token is generated and sent to the demander to realize the call for GPU computing power resources. The roles of auditors and administrators have been added to verify fake computing power resources and ensure the security of overselling.

[0019] (2) In the process of calculating the logical computing power inventory available for listing and trading, the maximum value of the historical average utilization rate of GPU computing power resources is used as the benchmark value. The benchmark is used to determine whether overselling can be carried out. If the benchmark value is within the preset overselling threshold range, the difference between the overselling threshold and the benchmark value is used as the overselling factor to calculate the logical margin and add it to the actual physical margin of GPU computing power resources to obtain the logical computing power inventory. This approach employs a conservative and safe strategy. On one hand, it selects the maximum value from historical GPU average utilization rather than the average value as the benchmark. This represents the load level during the busiest period and can prevent performance crashes caused by over-scaling during peak load periods. On the other hand, it uses the product of the difference between the over-scaling threshold and the benchmark value as a dynamically updated over-scaling factor, which can achieve an optimal balance between resource utilization and system stability, enabling refined management and maximizing benefits.

[0020] (3) The present invention performs real-time monitoring during the use of GPU computing resources. If the physical computing load exceeds the threshold, some oversold tasks can be suspended or migrated to ensure the execution of high-priority tasks and achieve dynamic load balancing. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating a method for overselling computing power compatible with multi-architecture GPUs provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of a computing power overselling system compatible with multi-architecture GPUs provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0023] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0024] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0025] Example 1 like Figure 1 As shown, this embodiment provides a method for overselling computing power compatible with multi-architecture GPUs, including the following steps: S1: Obtain GPU computing power resources with different architectures from different suppliers, classify each GPU computing power resource according to the manufacturer to form different resource pools, and generate logical computing power inventory available for listing and trading based on the parameters, historical load data and real-time pressure monitoring values ​​of each GPU computing power resource. S2: Obtain the willingness to reach a transaction between the demand side and the supplier. The willingness to reach a transaction determines the required vendor based on the demand side's task requirements, so as to select the GPU computing power resources in the corresponding resource pool. Based on the willingness to reach a transaction, determine whether the supplier's logical computing power inventory of GPU computing power resources matches the demand side's task requirements. If they match, match the transaction. S3: After the transaction is matched, an identity authentication token with a time limit is generated and sent to the requesting party. The requesting party can use the identity authentication token to jump to and access the GPU computing power resources in the corresponding resource pool to perform computing tasks.

[0026] In step S1, the process of generating the logical computing power inventory available for listing and trading includes: The parameters of each GPU computing resource are converted into unified standard computing parameters to form a standardized service capability profile; Based on the historical load data and real-time stress monitoring values ​​of each GPU computing power resource, a dynamic overselling factor is added to the service capability profile to generate a logical computing power inventory that can be listed for trading.

[0027] In this embodiment, the parameters such as video memory and bandwidth of GPUs with different architectures are converted into unified standard computing power parameters, and the platform can be synchronized with the third-party platform in real time.

[0028] Based on the supplier's historical load data, real-time stress monitoring, and GPU architecture characteristics, different strategies are adopted for different architectures (such as NVIDIA's sharding, Ascend's virtualization, etc.). Executing GPU scheduling requires virtualizing the GPU and converting GPU parameters into standard computing power units.

[0029] Preferably, the dynamic overselling factor is set based on the historical average utilization rate of GPU computing resources.

[0030] The calculation process for the logical computing power inventory of GPU computing power resources available for listing and trading includes: The maximum value of the historical average utilization rate of GPU computing resources is used as the benchmark. If the value is within the preset oversold threshold range, the difference between the oversold threshold and the benchmark value is used as the oversold factor. The oversold factor is multiplied by the total computing power of the GPU computing resources to obtain the oversold reserve, which is then added to the actual physical reserve of the GPU computing resources to obtain the logical computing power inventory. Otherwise, the actual physical reserve of the GPU computing resources is used as the logical computing power inventory.

[0031] The oversold factor is dynamically updated at fixed time intervals.

[0032] In addition, the GPU virtualization interfaces of different manufacturers are inconsistent and incompatible, making it impossible to allocate a task to GPUs of different manufacturers at the same time. By adopting the pooling technology of GPUs from different manufacturers, GPU resources are pooled according to the manufacturer and isolated at the scheduling layer, thereby achieving isolation at the runtime layer. This greatly improves resource utilization, computing power allocation flexibility, operating cost efficiency, system stability, and multi-tenant isolation.

[0033] The transaction process in step S2 adopts a listing and delisting mode, which includes: the demander publishes a task demand profile containing specific computing characteristics and lists it in a preset public demand pool; the supplier proposes a delisting action based on the task demand profile, generating a transaction agreement.

[0034] Specific computing characteristics include video memory, computing power precision, and environment image.

[0035] In the transaction process of this embodiment, the task requirement profile published by the demand party, which includes specific computing characteristics (such as video memory, computing power precision, environment image, etc.), is identified and listed in the public demand pool; in response to the supplier's action of delisting the task requirement profile, the matching engine verifies the alignment between the supplier's logical computing power margin and the task requirement; a clear transaction decision is generated, and an identity authentication token with time expiration is automatically generated.

[0036] In step S3, this solution uses identity authentication and API forwarding mechanisms to allow the requesting party to directly access the corresponding computing resources on a third-party computing power platform after completing the transaction on the platform through unified authentication.

[0037] Specifically, this embodiment utilizes identity authentication tokens to establish a cross-platform automated delivery link, enabling users to directly jump from the transaction platform and transparently access the specific GPU computing environment of a selected supplier to perform computationally intensive tasks such as AI model training or inference.

[0038] As a preferred implementation, the method further includes: when the demander uses GPU computing resources to perform computing tasks, monitoring the physical computing load of the corresponding GPU computing resources; if the physical computing load exceeds a preset load threshold, suspending or migrating part of the computing task to other GPU computing resources in the resource pool where the GPU computing resources are located.

[0039] The following is a specific implementation example of the above solution: Resource import and review: Suppliers register their computing resources (such as A100, V100, S80, etc.) in the background, and the reviewers review the registered resources through the platform.

[0040] Operational Analysis: The system calculates the corresponding safe oversold ratio based on the volatility of the supplier's past resources and actual transaction conditions.

[0041] Market listing and delisting / direct purchase: Demanders can list the computing power resources they need, and suppliers can complete transactions with them through delisting. Auditors review the transaction's validity and facilitate the deal. Alternatively, demanders can directly purchase computing power resources from the supplier marketplace.

[0042] Transaction Processing and Environment Redirection: After a transaction is completed, the platform issues instructions to the node manager on the supplier's side through the management center. When the requester clicks the "Use Computing Power" button, the platform automatically generates an SSO certificate with an expiration date, and the user's browser can directly redirect to the third-party computing power platform to use computing power resources.

[0043] Dynamic load balancing: If the physical computing power load exceeds the threshold, the system auditor can suspend or migrate some oversold tasks to ensure the service quality of high-priority tasks.

[0044] Example 2 like Figure 2 As shown, this embodiment provides a computing power overselling system that implements a multi-architecture GPU compatible computing power overselling method as described in Embodiment 1, comprising: The management center is used to acquire GPU computing power resources with different architectures from different vendors, classify each GPU computing power resource according to the vendor to form different resource pools, and generate a logical computing power inventory that can be listed for trading based on the parameters, historical load data and real-time pressure monitoring values ​​of each GPU computing power resource; after the transaction is matched, a time-sensitive identity authentication token is generated and sent to the demand party. The demand side is used to submit task requirements. Based on the demand side's task requirements, the required vendors are determined to select GPU computing resources from the corresponding resource pool, generate a transaction agreement, and jump to and access the corresponding GPU computing resources in the resource pool based on the identity authentication token for computing operations. Suppliers are responsible for listing their GPU computing power resources and generating transaction intentions. The review panel is used to calculate whether the supplier's logical computing power inventory of GPU computing resources matches the task requirements of the demander based on the willingness of the demander and the supplier to reach a transaction. If they match, the transaction is matched.

[0045] The specific usage process of the above system is as follows: The management center transforms physical GPU clusters with different underlying hardware architectures (such as NVIDIA, Huawei Ascend, AMD, etc.) provided by suppliers into standardized service capability profiles; based on real-time telemetry data and historical load characteristics, it injects dynamic overselling factors into the service capability profiles, thereby generating logical computing power inventory available for listing and trading.

[0046] The transaction process includes: identifying the task requirement profile published by the demander that includes specific computing characteristics (such as video memory, computing power precision, environment image, etc.) and listing it in the public demand pool; responding to the supplier's action of removing the task requirement profile, verifying the alignment between the supplier's logical computing power margin and the task requirement through the matching engine on the review end; generating a clear transaction decision and automatically generating an identity authentication token with a time limit.

[0047] Further leverage tokens to establish a cross-platform automated delivery chain, enabling users to jump directly from the trading platform and transparently access the specific GPU computing environment of selected vendors to perform computationally intensive tasks such as AI model training or inference.

[0048] It should be noted that the specific details and beneficial effects of the device in this application can be found in the above-described method embodiments, and will not be repeated here.

[0049] This embodiment also provides a computer-readable storage medium storing a computer program, which is executed by a processor as the computing power overselling method compatible with multi-architecture GPUs in Embodiment 1.

[0050] The computer program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This computer program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the computer program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0051] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be a machine-readable signal medium or a machine-readable storage medium. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0052] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for overselling computing power compatible with multi-architecture GPUs, characterized in that, Includes the following steps: Acquire GPU computing power resources with different architectures from different suppliers, classify each GPU computing power resource according to the manufacturer to form different resource pools, and generate a logical computing power inventory that can be listed for trading based on the parameters, historical load data and real-time pressure monitoring values ​​of each GPU computing power resource. The system obtains the willingness of the demand side and the supplier to reach a transaction agreement. Based on the demand side's task requirements, the system determines the required vendors and selects GPU computing power resources from the corresponding resource pool. Based on the willingness of the demand side, the system determines whether the supplier's logical computing power inventory of GPU computing power resources matches the demand side's task requirements. If they match, the system matches the transaction. After a transaction is matched, a time-sensitive identity authentication token is generated and sent to the requesting party. The requesting party uses the identity authentication token to access the GPU computing resources in the corresponding resource pool to perform computing tasks.

2. The method for overselling computing power compatible with multi-architecture GPUs according to claim 1, characterized in that, The process of generating the logical computing power inventory available for listing and trading includes: The parameters of each GPU computing resource are converted into unified standard computing parameters to form a standardized service capability profile; Based on the historical load data and real-time pressure monitoring values ​​of each GPU computing power resource, a dynamic overselling factor is added to the service capability profile to generate the logical computing power inventory available for listing and trading.

3. The method for overselling computing power compatible with multi-architecture GPUs according to claim 2, characterized in that, The dynamic overselling factor is set based on the historical average utilization rate of GPU computing resources.

4. The method for overselling computing power compatible with multi-architecture GPUs according to claim 3, characterized in that, The calculation process for the available logical computing power inventory of the GPU computing power resources includes: The maximum value of the historical average utilization rate of GPU computing resources is used as the benchmark. If the value is within the preset oversold threshold range, the difference between the oversold threshold and the benchmark value is used as the oversold factor. The oversold factor is multiplied by the total computing power of the GPU computing resources to obtain the oversold reserve, which is then added to the actual physical reserve of the GPU computing resources to obtain the logical computing power inventory. Otherwise, the actual physical reserve of the GPU computing resources is used as the logical computing power inventory.

5. The method for overselling computing power compatible with multi-architecture GPUs according to claim 4, characterized in that, The oversold factor is dynamically updated at fixed time intervals.

6. The method for overselling computing power compatible with multi-architecture GPUs according to claim 1, characterized in that, The method employs a listing and delisting model to generate the transaction completion intention; The listing and bidding process includes: the demand side publishes a task requirement profile containing specific computational characteristics and lists it in a pre-set public demand pool; the supplier proposes a bidding action based on the task requirement profile, generating the transaction completion intention.

7. The method for overselling computing power compatible with multi-architecture GPUs according to claim 6, characterized in that, The specific computing characteristics include video memory, computing power precision, and environment image.

8. The method for overselling computing power compatible with multi-architecture GPUs according to claim 1, characterized in that, The method further includes: when the demander uses GPU computing resources to perform computing jobs, monitoring the physical computing load of the corresponding GPU computing resources; if the physical computing load exceeds a preset load threshold, suspending or migrating part of the computing job to other GPU computing resources in the resource pool where the GPU computing resources are located.

9. A computing power overselling system that implements the computing power overselling method compatible with multi-architecture GPUs as described in any one of claims 1-8, characterized in that, include: The management center is used to acquire GPU computing power resources with different architectures from different vendors, classify each GPU computing power resource according to the vendor to form different resource pools, and generate a logical computing power inventory that can be listed for trading based on the parameters, historical load data and real-time pressure monitoring values ​​of each GPU computing power resource; after the transaction is matched, a time-sensitive identity authentication token is generated and sent to the demand party. The demand side is used to propose task requirements, generate the willingness to complete the transaction, and jump to and access the corresponding GPU computing resources in the resource pool based on the identity authentication token to perform computing tasks. Suppliers are responsible for listing their GPU computing power resources and generating transaction intentions. The review panel is used to calculate whether the supplier's logical computing power inventory of GPU computing resources matches the task requirements of the demander based on the willingness of the demander and the supplier to reach a transaction. If they match, the transaction is matched.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is executed by a processor according to any one of claims 1 to 8.

Images