A GPU computing power scheduling method based on a cloud-multi-core heterogeneous computing power platform

Abstract

This invention proposes a GPU computing power scheduling method based on a cloud-based multi-core heterogeneous computing platform. The method includes the following steps: Step S1: Heterogeneous resource registration and modeling. A resource registration module is used to access hardware devices containing multiple types of GPUs, collecting device model, memory capacity, and performance index data to construct a resource feature database containing topological relationships, supporting mixed chip access; Step S2: Virtualization resource reconstruction. Hardware abstraction layer technology is used to pool physical GPUs into virtual GPU resources. Containerization technology is used to achieve memory isolation and computing unit partitioning. Each virtual GPU instance is equipped with an independent driver stack and security sandbox; Step S3: Multimodal task submission. CUDA / OpenCL computing tasks submitted by users are received, and task requirement parameters, including the number of computing cores, memory usage, and data throughput threshold, are parsed to generate a task descriptor containing priority tags.

Description

Technical Field

[0001] This invention belongs to the field of computing power scheduling, and specifically relates to a GPU computing power scheduling method based on a cloud-multi-core heterogeneous computing power platform. Background Technology

[0002] Currently, with the accelerating pace of enterprise digital transformation and application innovation, the concept of "one cloud, multiple chips" is constantly expanding. It not only refers to the management of resources from different GPU architectures, but also includes the management of computing power from CPUs, FPGAs, DPUs, and other components. "One cloud, multiple chips" uses a single cloud platform to manage computing resources from different chip architectures, achieving unified management and scheduling of heterogeneous resources. By shielding the underlying architecture differences through the cloud platform, it provides users with a consistent cloud computing service experience.

[0003] Hardware resources, including different types of processors (such as CPUs and GPUs) and their corresponding memory and storage devices, collectively constitute a heterogeneous computing environment. Hardware virtualization pooling technology abstracts physical resources into virtual resources, enabling flexible scheduling and management. Virtualizing hardware resources such as GPUs allows for unified management and scheduling by the cloud platform. Tasks are rationally allocated to GPUs for execution based on task characteristics and GPU resource availability. The system includes scheduling algorithms and an engine responsible for task allocation and scheduling based on task requirements and resource status. Users define their required computing resources through a declarative interface, and the scheduling system transparently allocates and manages these resources. Users only need to focus on application-layer requirements without needing to concern themselves with the specific implementation of the underlying hardware resources.

[0004] The "one cloud, multiple chips" approach to GPU computing power scheduling has profound implications. It not only meets users' diverse computing power needs but also optimizes resource allocation and utilization, reduces supply chain risks, promotes technological innovation and industrial upgrading, and fosters the development of the digital economy. Therefore, under this premise, a "one cloud, multiple chips" heterogeneous computing power platform needs to shield the hardware differences between different GPU architectures at the lower level and encapsulate a service-oriented computing power model at the upper level, providing computing power-related storage, network, operating system, monitoring, and maintenance in a service-oriented manner. Thus, supporting GPU architectures across different domestic IT innovation industries and solving the "one cloud, multiple chips" problem is the most fundamental capability requirement for a "one cloud, multiple chips" heterogeneous computing power platform.

[0005] With the explosive growth of business innovation across various industries, the computing power requirements of different business scenarios are becoming increasingly differentiated. Artificial intelligence has relatively lower requirements for computational precision but higher demands for parallel computing performance, and GPUs are a key component to meet these high-performance computing needs. By implementing GPU computing power scheduling through a "one cloud, multiple chips" approach, the diverse computing power needs of users in different application scenarios can be met, improving computing efficiency and performance.

[0006] In the cloud computing industry, the performance of server chips varies greatly, leading to inconsistent user experiences and vastly different application results. By implementing GPU computing power scheduling through a "one cloud, multiple chips" approach, the risks associated with technology selection can be minimized, as users can flexibly switch between different chip architectures without relying on a specific vendor or architecture. This helps improve business stability and the flexibility of business transformation.

[0007] In cloud computing environments, the utilization of various computing resources (including GPUs) is often uneven. By implementing GPU computing power scheduling through a "one cloud, multiple chips" approach, these heterogeneous resources can be integrated into a unified cloud platform for management, and dynamically allocated and scheduled according to actual needs. This not only improves resource utilization but also optimizes resource allocation and reduces operating costs.

[0008] "One cloud, multiple chips" enables GPU computing power scheduling, a technological innovation in the field of cloud computing that drives cloud computing platforms towards greater flexibility, efficiency, and intelligence. Simultaneously, it provides broader opportunities for the application of GPUs and other accelerator processors in cloud computing, contributing to the upgrading and development of related industries.

[0009] Computing power has become a core support and driving force for the development of the digital economy. By implementing GPU computing power scheduling through a "one cloud, multiple chips" approach, more computing resources can be released, providing strong support for the innovative development of the digital economy. This helps accelerate the pace of digital transformation and promotes high-quality economic and social development. Summary of the Invention

[0010] This invention proposes a GPU computing power scheduling method based on a cloud-multi-core heterogeneous computing power platform. This method solves the problems of low resource utilization, unbalanced task scheduling, poor hardware topology adaptability, lag in dynamic load response, and insufficient system reliability in heterogeneous GPU computing power platforms, and realizes unified management, efficient scheduling, and real-time optimization of multi-type GPU resources.

[0011] The technical solution of this invention is implemented as follows: a GPU computing power scheduling method based on a cloud-multi-core heterogeneous computing power platform, the method comprising the following steps:

[0012] Step S1: Heterogeneous resource registration and modeling. The resource registration module connects to hardware devices containing multiple types of GPUs, collects device model, memory capacity and performance index data, and builds a resource feature database containing topological relationships to support mixed access of chips.

[0013] Step S2: Virtualization resource reconstruction. The physical GPU is pooled into virtual GPU resources using hardware abstraction layer technology. Containerization technology is used to achieve memory isolation and computing unit partitioning. Each virtual GPU instance is equipped with an independent driver stack and security sandbox.

[0014] Step S3: Multimodal task submission. Receive CUDA / OpenCL computing tasks submitted by users, parse task requirement parameters, including the number of computing cores, memory usage, and data throughput threshold, and generate a task descriptor containing priority tags.

[0015] Step S4: Topology-aware resource matching. Based on the real-time updated resource status graph, an improved Hungarian algorithm is used for multi-dimensional resource matching. When selecting computing nodes, PCIe bandwidth, NVLink connection status and memory fragmentation are evaluated simultaneously.

[0016] Step S5: Dynamic load scheduling. Deploy a distributed scheduling engine to collect real-time data on node GPU temperature, utilization, and task queue depth. When a local hotspot is detected, trigger a load balancing strategy, using a combination of task migration and computing power reallocation mechanism.

[0017] Step S6: Adaptive task execution, inject runtime monitoring agent into GPU instance, dynamically adjust CUDA stream priority and memory page locking strategy, and use MPS multi-process service technology to achieve exclusive resource guarantee for critical tasks.

[0018] Step S7: Full-link monitoring and diagnosis, building a three-dimensional monitoring system: monitoring the SM unit utilization and ECC error rate of GPU hardware through the resource layer; tracking the computation progress and memory leakage risk through the task layer; implementing health scoring and triggering automatic isolation mechanisms through the node layer.

[0019] As a preferred implementation, steps S8 and S9 are performed after the three-dimensional monitoring system is constructed.

[0020] Step S8: Parameterized computing power optimization, using an improved Adam optimizer to dynamically adjust scheduling strategy parameters; calculate the first and second moment estimates of the cluster load gradient; correct deviations based on historical load fluctuations; update task allocation weight coefficients according to the corrected momentum values; introduce Nesterov acceleration gradient to predict future load trends;

[0021] Step S9: Closed-loop feedback update. Collect actual resource consumption data during task execution, compare and analyze it with preset requirement parameters, iteratively optimize the resource matching algorithm through reinforcement learning model, and update the virtual GPU specification template library.

[0022] Existing technologies typically design scheduling strategies for single-type GPUs, making it difficult to adapt to the mixed access requirements of GPUs from multiple vendors and architectures (such as NVIDIA, AMD, and domestic chips). This solution, through heterogeneous resource registration and modeling in step S1, constructs a resource feature database containing topological relationships, supporting mixed chip access and solving problems such as device model differences, memory capacity mismatch, and performance fragmentation. The technical challenge lies in how to unify and abstract the hardware characteristics of different GPU architectures (such as the number of CUDA cores and Tensor Core configuration) and establish a cross-vendor topological relationship model (such as NVLink and PCIe hybrid connection scenarios). Traditional GPU virtualization technologies (such as GPU passthrough) cannot achieve fine-grained resource partitioning and have risks of memory conflicts and driver compatibility. This solution, through the hardware abstraction layer and containerization technology in step S2, pools physical GPUs into independent virtual instances, equipped with isolated driver stacks and security sandboxes, solving problems of memory leakage, computing unit preemption, and security isolation between multiple tasks. The technical challenge lies in how to achieve dynamic reconstruction of virtual GPU instances through memory page locking strategies and driver stack hot-plugging mechanisms, while avoiding the performance loss caused by containerization.

[0023] Existing scheduling algorithms (such as greedy algorithms) struggle to handle complex task requirements with multi-dimensional resource constraints (such as GPU memory, bandwidth, and computing cores). This solution addresses the problem of accurately matching task requirements with hardware topology by generating task descriptors in step S3 and performing topology-aware matching in step S4. It employs an improved Hungarian algorithm combined with PCIe bandwidth, NVLink connection status, and GPU memory fragmentation assessment. The key challenge lies in designing a multi-dimensional weighted scoring function in a dynamic resource environment while reducing algorithm complexity to meet real-time scheduling requirements. Traditional scheduling systems rely on periodic resource acquisition, making it difficult to respond promptly to local hotspots (such as sudden GPU temperature spikes or task queue backlogs). This solution addresses these issues by using a distributed scheduling engine and load balancing strategy in step S5 to collect temperature, utilization, and queue depth data in real time. It also employs a combination of task migration and computing power reallocation mechanisms to resolve local resource overload and low heat dissipation efficiency. The key challenge is designing a low-latency task migration protocol to avoid performance fluctuations caused by data transfer.

[0024] Existing systems lack the ability to guarantee exclusive resources for critical tasks (such as AI inference), and runtime parameter adjustments rely on manual intervention. This solution, through the runtime monitoring agent and MPS multi-process service technology in step S6, dynamically adjusts CUDA stream priorities and memory page locking strategies, resolving latency fluctuations caused by resource contention in critical tasks. The technical challenge lies in how to dynamically switch memory allocation strategies without interrupting task execution and balance resource contention conflicts between multiple processes. Traditional monitoring systems only focus on hardware-level indicators, lacking end-to-end health assessments at the task and node levels. This solution, through the three-dimensional monitoring system in step S7, simultaneously tracks SM unit utilization, memory leak risk, and node health scores, resolving issues of ECC error accumulation, high concealment of memory leaks, and fault propagation. The technical challenge lies in how to design a cross-layer correlation analysis model to achieve causal reasoning between hardware anomalies (such as SM unit overheating) and task anomalies (such as memory leaks).

[0025] As a preferred implementation, the dynamic load scheduling in step S5 includes a scheduling policy optimization engine based on deep reinforcement learning. Specifically, this includes: constructing a state feature space, collecting node GPU utilization, memory usage, PCIe link bandwidth utilization, and task queue waiting time to form a 16-dimensional state vector; designing a dual-channel Dueling DQN network architecture, where the value stream network evaluates global scheduling benefits and the advantage stream network calculates the advantage value of local scheduling actions; defining a reward function R = α * (1 - load variance) + β task completion rate - γ migration cost, where α, β, and γ are adjustable weight coefficients; deploying an experience replay pool to store historical scheduling decision data, and using a priority sampling mechanism to filter high-value training samples; implementing an online policy update mechanism, triggering immediate fine-tuning of network parameters when a cluster topology change or a sudden change in load pattern is detected; and introducing action masking technology to constrain invalid scheduling operations, ensuring that task migration only occurs between nodes with hardware compatibility.

[0026] As a preferred implementation, the virtualization resource reconstruction in step S2 further includes: creating securely isolated vGPU instances; using hardware-assisted SR-IOV technology to divide the physical GPU into multiple VFs, with each VF bound to an independent secure enclave; implementing dynamic memory quota management, completing elastic expansion of vGPU memory from 1GB to 24GB within 0.1 seconds according to task requirements; deploying a driver-level protection module to intercept abnormal CUDA API requests through system call hijacking technology, blocking unauthorized memory access operations; designing a hot migration channel, migrating the vGPU instance along with running tasks to a backup node without loss when physical GPU overheating is detected; and building an energy-aware scheduler to dynamically adjust the core frequency according to the SM unit utilization rate of the vGPU instance, achieving a 30% reduction in power consumption with a performance loss of <5%.

[0027] As a preferred implementation, the resource matching in step S4 further includes: constructing a multi-dimensional topology feature graph, integrating NVLink connection counts, PCIe Switch levels, and RDMA network latency data between nodes; developing a graph neural network matching model to abstract task requirements into feature subgraphs and search for connected subgraphs that satisfy the constraints in the topology graph; implementing transmission cost estimation, calculating the bandwidth-delay product of data transport paths for candidate node combinations, and selecting the topology path with the lowest cost; deploying a prefetch optimization module to synchronously schedule relevant datasets to the HBM memory of the target node during task allocation; and implementing NUMA-aware memory binding to lock the task process to the CPU memory domain directly connected to the target GPU.

[0028] As a preferred implementation, the end-to-end monitoring in step S7 further includes: deploying an LSTM fault prediction model to analyze historical SM unit error rates, temperature gradient changes, and ECC correction counts to predict hardware faults 30 minutes in advance; establishing a tiered alarm mechanism, defining yellow, orange, and red alerts; implementing an active self-healing strategy, initiating driver reloading and memory remapping for yellow alert nodes, triggering service migration for orange alerts, and isolating the entire fault domain during red alerts; designing a fault-tolerant training mode that automatically switches to redundant computing nodes to continue execution and rebuilds the computing context when an unrepairable error is detected; generating a fault tracing graph, correlating and analyzing hardware logs, task characteristics, and environmental variables to locate the root cause and update the health model.

[0029] The beneficial effects of this invention after adopting the above technical solutions are as follows: This GPU computing power scheduling method significantly improves the resource utilization, task execution efficiency, and system reliability of heterogeneous computing power platforms through the collaborative design of heterogeneous resource registration and modeling, virtualization resource reconstruction, topology-aware matching, dynamic load scheduling, and full-link monitoring. The heterogeneous resource registration module supports the mixed access of multiple types of GPUs and achieves unified management of cross-vendor devices through topology relationship modeling, solving the hardware fragmentation problem; the virtualization resource reconstruction technology ensures the security and stability of multi-task parallel execution through containerized memory isolation and independent deployment of driver stacks. The improved Hungarian algorithm, combined with a multi-dimensional resource scoring mechanism, achieves accurate matching between task requirements and hardware topology, reducing scheduling latency caused by PCIe bandwidth bottlenecks or memory fragmentation; the dynamic load balancing strategy effectively avoids local resource overload and improves the overall throughput of the cluster through real-time hotspot detection and task migration mechanisms. The runtime monitoring agent and MPS technology provide exclusive resource guarantees for critical tasks and ensure low-latency execution of high-priority tasks by dynamically adjusting CUDA stream priorities and memory page locking strategies. The three-dimensional monitoring system covers the resource layer, task layer, and node layer. Through SM unit utilization analysis, memory leak tracking, and health scoring, it achieves early warning and automatic isolation of faults, reducing the risk of system downtime. The end-to-end diagnostic function, through cross-layer correlation analysis, quickly locates the causal relationship between hardware anomalies and task anomalies, improving operational efficiency. This method provides a highly elastic, highly reliable, and low-latency scheduling solution for heterogeneous GPU computing platforms, suitable for various scenarios such as AI training, scientific computing, and edge inference.

[0030] Multi-chip support enhances resource utilization. The multi-chip heterogeneous computing platform can adapt to and manage resources from various chip architectures (such as GPUs, CPUs, and FPGAs), effectively supporting diverse computing power. This allows the platform to flexibly schedule different types of computing resources according to different business needs, thereby improving resource utilization and efficiency. Heterogeneous computing power collaborates through a unified cloud management platform, enabling collaborative operation of heterogeneous computing resources. Different types of chip architectures can work together in the same cloud computing environment, providing users with more powerful computing capabilities. Users do not need to concern themselves with the differences in underlying hardware architectures; they can apply for, allocate, and use computing power simply through a unified interface or platform.

[0031] Computing power pooling: The multi-core heterogeneous computing power platform integrates different types of computing resources into a GPU computing power pool, achieving standardized and pooled management of computing power. This allows computing resources to be utilized more efficiently, avoiding resource idleness and waste. Intelligent scheduling algorithm: The platform adopts an advanced intelligent scheduling algorithm that can dynamically adjust computing power allocation based on business needs and resource status. This enables computing resources to be intelligently allocated according to factors such as the priority and computing requirements of different tasks, improving resource utilization and computing efficiency.

[0032] By reducing hardware investment costs, the multi-chip heterogeneous computing platform can be compatible with computing resources of different architectures, avoiding the waste of hardware investment caused by the limitations of a single chip architecture. Users can choose the most suitable chip architecture according to their actual needs, thereby reducing hardware investment costs. Optimizing computing power costs, the platform uses intelligent scheduling algorithms to dynamically adjust computing resources according to business needs, avoiding resource idleness and waste. This allows users to utilize computing resources more efficiently and reduce computing power costs.

[0033] To improve computing efficiency, the multi-core heterogeneous computing platform leverages optimized computing power scheduling strategies to fully utilize different types of computing capabilities. When processing large-scale image data, GPUs can complete processing tasks faster than CPUs, thus improving computing efficiency. Furthermore, the platform uses intelligent scheduling and load balancing technologies to allocate user tasks to the most suitable computing resources, thereby reducing computation time. This allows for faster results and improves work efficiency.

[0034] Flexible scalability: The multi-chip heterogeneous computing platform supports flexible expansion of computing resources as business grows and demands change. Users can dynamically increase or decrease computing resources according to actual needs, thereby meeting ever-changing business requirements. Multi-architecture compatibility: The platform can adapt to various chip architectures and operating systems, providing users with a wider range of choices. This allows users to more flexibly select the computing resources and software environment that best suit their needs.

[0035] Intelligent monitoring and management: The multi-core heterogeneous computing platform provides intelligent monitoring and management functions, enabling real-time monitoring of the status and usage of computing resources. This allows users to promptly identify and resolve potential problems, ensuring the stable operation of computing resources. Simplified operation and maintenance processes: The platform simplifies operation and maintenance processes through a unified cloud management platform. Users can manage, monitor, and troubleshoot resources through a unified portal, reducing the difficulty and cost of operation and maintenance. Furthermore, the platform provides a wealth of operation and maintenance tools and documentation to facilitate daily operation and maintenance and troubleshooting. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a system structure diagram in an embodiment of the present invention;

[0038] Figure 2 This is a flowchart illustrating the GPU computing power scheduling implementation of the present invention;

[0039] Figure 3 This is a diagram showing the connection relationships between the various components of the system in this embodiment of the invention;

[0040] Figure 4 This is a system diagram of the new generation cloud platform for energy sales internet application services in Embodiment 2 of the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] Example:

[0043] like Figures 1-3 As shown, the working process of a GPU computing power scheduling method based on a multi-core heterogeneous computing power platform is a complex and sophisticated systems engineering project. It involves multiple stages, including hardware resource registration and access, task submission and allocation, dynamic management of GPU resources, and task execution and result feedback. Through reasonable hardware abstraction and integration, task submission and allocation, dynamic management of GPU resources, and task execution and result feedback, the cloud platform can fully utilize the performance advantages of heterogeneous computing resources such as GPUs, improve computing efficiency, reduce power consumption, and accelerate task execution.

[0044] 1. Resource Registration: The resource registration module registers various types of processors (GPUs), storage devices, memory, and other resources, supporting the access of multiple domestically produced chips. These hardware resources collectively form the foundation of the heterogeneous computing environment. This module allows the platform to identify available resource information and obtain real-time computing topology and status information, ensuring the effective utilization of GPU resources.

[0045] 2. Resource Modeling: Modeling the registered GPU resources on the cloud platform, including key information such as GPU model, performance, and memory capacity. This information will serve as the basis for scheduling decisions.

[0046] 3. Resource Reconfiguration: Through virtualization and pooling technologies, cloud platforms can dynamically reconfigure GPU resources, abstracting physical hardware resources into virtual resources to meet the needs of different tasks, achieving resource pooling and flexible scheduling. At this layer, hardware resources such as GPUs are virtualized into virtual GPUs (vGPUs), dividing GPU resources into multiple independent virtual environments for sharing and scheduling among multiple virtual machines or containers, enabling unified management and scheduling on the cloud platform. When a task requires more GPU resources, the cloud platform can dynamically allocate more GPU resources from the resource pool to that task.

[0047] 4. Resource isolation: When multiple tasks share GPU resources, containerization technology is used to achieve resource isolation, which can prevent interference and conflicts between different tasks, thereby ensuring the stability and security of the tasks.

[0048] 5. Security Policy: Employing container-level sandbox security technology, the platform formulates and enforces strict security policies to ensure that GPU resources are not accessed or misused without authorization. The platform prevents unauthorized access and malicious attacks, protecting user data and privacy.

[0049] 6. Task Submission: Users can create new tasks and submit the tasks to be executed (such as computing tasks, data processing tasks, etc.) to the cloud platform through the interfaces or tools provided by the cloud platform. Multiple task types and programming models are supported, such as CUDA and OpenCL, so that users can flexibly utilize GPU resources for accelerated computing.

[0050] 7. Task Analysis: Analyze the GPU resource requirements of the task to be executed, including computational load, memory requirements, and data transfer speed.

[0051] 8. Task Analysis: The cloud platform analyzes the submitted tasks to understand their specific requirements, including the necessary computing and storage resources.

[0052] 9. Resource Assessment: The cloud platform assesses the currently available GPU resources, including the number of GPUs, performance, utilization, etc., to determine which GPU resources can meet the needs of the current task.

[0053] 10. Resource Matching: Based on task requirements, select suitable GPU resources from the resource pool. During the matching process, topology-aware scheduling techniques can be considered to optimize data transfer speed and computational efficiency.

[0054] 11. Task Allocation: Based on resource matching results, the cloud platform rationally allocates tasks to available GPUs for execution. The scheduling algorithm considers multiple factors, such as task priority, GPU resource availability, and task execution time, to optimize overall computing performance.

[0055] 12. Task Execution: Once a task is assigned to the GPU for execution, the GPU begins executing the task and stores the results on the designated storage device. The cloud platform monitors the task's execution progress and status in real time.

[0056] 13. Computing Power Scheduling: Based on the real-time demands of task execution and the availability of GPU resources, intelligent scheduling through load balancing strategies dynamically adjusts resource allocation. The scheduling engine can perform scheduling based on different strategies (such as fairness, priority, performance, etc.) to meet the needs of different applications. It can automatically adjust resource allocation according to task requirements and system status, rationally distributing tasks across different cloud hosts to achieve efficient utilization and load balancing.

[0057] Its load balancing strategy monitors the GPU load of each node in the cluster in real time. The scheduler can identify which nodes are currently idle or under low load, and which are under high load or overload. Based on this information, the scheduler can make decisions to prioritize assigning new tasks or jobs to nodes with lower loads, thereby reducing task waiting time and improving overall system performance.

[0058] Node selection: Based on the processed load data, the scheduler evaluates the load of each node. It prioritizes nodes with lower loads and more abundant resources for assigning new tasks.

[0059] Resource allocation: Based on the task requirements and the node's resource availability, the scheduler determines the specific GPU resources to allocate to each task. This may include specifying the number of GPUs, the size of video memory, etc.

[0060] Load balancing: Load scheduling strategies also consider how to achieve load balancing across different nodes. By dynamically adjusting task allocation, it ensures that the load on each node remains at a relatively balanced level, thereby improving the overall performance and stability of the system.

[0061] 14. Resource Sharing: When multiple tasks need to access GPU resources simultaneously, GPU shared scheduling technology can be used. This allows multiple containers or tasks to share the same GPU resource, thereby improving resource utilization and reducing computing costs.

[0062] 15. Task Feedback: Once the task is completed, the cloud platform will return the task execution result to the user. Simultaneously, the cloud platform will update the current GPU resource status information to provide a reference for subsequent task scheduling.

[0063] 16. Task Monitoring: The cloud platform supports full-process observability, providing observation capabilities at the resource and task levels. It can monitor the execution status and progress of tasks throughout the entire process, making it convenient for users to monitor tasks and manage resources.

[0064] 17. Resource Monitoring: Real-time monitoring via cloud platform continuously tracks GPU usage, including temperature, utilization rate, and fault information. Potential problems are identified promptly, and appropriate adjustments are made.

[0065] 18. Node Diagnosis: The cloud platform provides a one-click diagnosis function, which can diagnose the node's hardware and software configuration, and quickly locate and resolve faults.

[0066] 19. Computing Power Optimization: The platform can automatically adjust GPU configuration and parameters to adapt to different application requirements and load conditions. It employs adaptive optimization algorithms to continuously optimize scheduling algorithms and resource management mechanisms, performing real-time analysis and optimization of GPU computing power to improve computational efficiency and performance, thereby enhancing the efficiency and accuracy of GPU computing power scheduling. The implementation is as follows:

[0067] State initialization: The algorithm also needs to initialize some state variables, such as the average gradient of the parameters and the average value of the momentum. These state variables will be used for subsequent parameter updates and adaptive adjustments.

[0068] Loss function calculation: The algorithm needs to calculate the loss function value under the current parameters, which is usually done by forward propagation network and comparing the predicted results with the actual labels.

[0069] Gradient Calculation: Next, the algorithm uses backpropagation to calculate the gradient of the loss function with respect to the parameters. These gradients will be used to guide the direction of parameter updates.

[0070] Adaptive learning rate adjustment: This is the core step of the adaptive optimization algorithm. The algorithm dynamically adjusts the learning rate based on the current gradient, historical gradient information (such as the average gradient, cumulative sum, etc.), and preset parameters (such as the learning rate decay factor, momentum factor, etc.).

[0071] The Adam algorithm used combines the idea of ​​dynamically adjusting the learning rate by calculating the average gradient and momentum optimization, and considers both the average gradient and momentum to dynamically adjust the learning rate.

[0072] The Adam algorithm's mathematical principle is based on the first and second moment estimates of the gradient, adaptively adjusting the learning rate for each parameter. The algorithm's iterative formula is as follows:

[0073] (Calculate the first moment estimate)

[0074] (Calculate the second moment estimate)

[0075] mt^=mt1-β1t (Correction to the first-order moment estimate)

[0076] vt^=vt1-β2t (Correction of bias in the second moment estimate)

[0077] θt+1=θt-α√vt (update parameters)

[0078] Where mt and vt represent the first and second moment estimates of the gradient in the t-th iteration, respectively, mt is the estimated value of mt and vt after bias correction, α is the learning rate, β1 and β2 are momentum decay factors, and ∈ is a small constant added to avoid the denominator being zero.

[0079] Parameter update: The parameters are updated using the adjusted learning rate and the calculated gradient. This is typically done via simple gradient descent or its variants (such as momentum optimization, Nesterov accelerated gradients, etc.).

[0080] Performance evaluation: After each parameter update, the algorithm needs to evaluate its performance (such as loss function value, accuracy, etc.) under the current parameters. This can be done by evaluating it on the validation set.

[0081] Iteration and Termination: If the performance meets the preset stopping conditions (such as reaching the maximum number of iterations, the loss function value being lower than a certain threshold, etc.), the algorithm terminates; otherwise, the algorithm will continue to iterate and repeat the above steps.

[0082] A multi-core heterogeneous computing power platform is typically built on cloud computing technology and adopts a layered architecture, including a resource layer, a security layer, a task management layer, a monitoring layer, and an application layer.

[0083] Main structural modules:

[0084] The resource layer is responsible for resource registration: it provides registration information for various types of processors, storage devices, memory, and other resources, which together form the foundation of the heterogeneous computing environment. Resource modeling: it provides modeling of key information about the GPU resources already registered on the cloud platform. Resource reconfiguration: through virtualization and pooling technologies, the cloud platform can dynamically reconfigure GPU resources.

[0085] Resource Assessment: Determines which GPU resources can meet the requirements of the current task by evaluating the currently available GPU resources. Resource Matching: Filters out suitable GPU resources from the resource pool based on task requirements.

[0086] The security layer is responsible for security policies: it employs container-level sandbox security technology to formulate and enforce strict security policies to ensure that GPU resources are not accessed or misused without authorization. Resource isolation: it provides containerization technology to achieve resource isolation when multiple tasks share GPU resources.

[0087] The task layer is responsible for task submission: providing users with interfaces or tools provided by the cloud platform to create new tasks and submit them to the cloud platform. Task analysis: analyzing the GPU resource requirements of the tasks to be executed. Task parsing: the cloud platform parses the submitted tasks to understand their specific requirements, including the required computing and storage resources. Task allocation: providing resource matching results, the cloud platform rationally allocates tasks to available GPUs for execution. Task execution: providing GPUs to execute tasks and storing the results on designated storage devices. Task feedback: providing the cloud platform to return the task execution results to the user after the task is completed.

[0088] The monitoring layer is responsible for resource monitoring: providing real-time monitoring through the cloud platform to continuously track GPU usage. Task monitoring: providing full-process monitoring of task execution status and progress. Node diagnostics: the cloud platform provides one-click diagnostic functionality to diagnose node hardware and software configurations. Computing power optimization: the platform can automatically adjust GPU configurations and parameters to adapt to different application needs and load conditions.

[0089] The application layer is responsible for computing power scheduling: it provides intelligent scheduling through smart scheduling algorithms to dynamically adjust the allocation of computing power resources. Resource sharing: it enables multiple containers or tasks to share the same GPU resources.

[0090] like Figure 3 The components and their connections are shown below:

[0091] (1) Resource Layer and Task Layer: Task-driven resource allocation. In a cloud computing environment, when a user submits a computationally intensive task, the system allocates corresponding computing resources (such as GPUs, memory, etc.) based on the task's computational requirements. In the task layer, once a specific task is defined, the system allocates corresponding resources based on the task's requirements. These resources can be computing resources, storage resources, network resources, etc., depending on the task type and objectives. Resources support task execution. The resources provided by the resource layer are the foundation for the task layer to execute tasks. Without resource support, tasks in the task layer cannot be effectively executed. In information systems, processors (GPUs), storage devices, memory, and other resources are crucial for completing work tasks. The performance and availability of these resources directly affect the efficiency and quality of task execution. Dynamic adjustment and coordination. In actual operation, the task layer may dynamically adjust the allocation and use of resources based on the task's execution status and changing requirements. At the same time, the resource layer also needs to coordinate different resources based on the resource status and usage to meet the task's needs. This dynamic adjustment and coordination is key to ensuring effective cooperation between the task layer and the resource layer.

[0092] (2) Task Layer and Application Layer: In implementing dynamic allocation of computing resources, in a cloud computing environment, the computing power scheduling system can dynamically adjust the number and configuration of virtual machines according to the load of the task layer to meet the performance requirements of the task layer. Computing power scheduling can dynamically allocate shared computing resources according to the actual needs of the task layer, avoiding idle and wasted computing resources and improving the utilization rate of resource sharing. Improved task execution efficiency: In big data processing scenarios, the computing power scheduling system can distribute tasks to multiple computing nodes for parallel execution, thereby significantly improving data processing speed. Through computing power scheduling, the task layer can obtain the required computing resources more quickly, thereby shortening task execution time and improving execution efficiency. Enhanced task reliability: In a distributed computing environment, the computing power scheduling system can replicate tasks to multiple nodes for execution, ensuring that tasks can continue to execute and complete when a single node fails. The computing power scheduling system can also provide fault recovery and fault tolerance mechanisms to ensure that the task layer can continue to execute when computing resources fail.

[0093] (3) Resource Layer and Security Layer: The resource layer is the foundation of resource isolation security. The servers, computing power, storage, and network resources provided by the resource layer are the basis for implementing resource isolation security. Without the support of these resources, resource isolation security cannot be achieved. The resource layer uses virtualization, containerization, and other technologies to divide physical resources into multiple independent logical resources, thus providing a foundation for resource isolation security. Drivers adapt to hardware resources of different architectures, and virtualization technology virtualizes hardware resources to form virtual resource pools. Resource isolation security protects the security of the resource layer. By implementing access control, authentication, and other measures, resource isolation security ensures that only authorized users or applications can access the resources of the resource layer. This isolation measure prevents unauthorized access or abuse of resources, thereby protecting the security of the resource layer. The resource layer and resource isolation security work together to ensure the overall security of the system. The resource layer provides basic resources, while resource isolation security is responsible for ensuring the secure use of these resources. The two cooperate to jointly address various security threats and ensure the overall security of the system. Resource isolation enhances the utilization of the resource layer. By isolating resources, resources can be divided into multiple independent units, each of which can be allocated, managed and used independently. This division can improve the utilization of resources and enable resources to better meet the needs of different users or applications.

[0094] (4) Resource Layer and Monitoring Layer: The resource layer is the data source for the monitoring layer. The monitoring layer needs to collect various data from the resource layer in real time, such as GPU utilization, memory usage, disk I / O, etc., to evaluate the system's operating status and performance. This data is the basis for the monitoring layer's analysis and decision-making, and also an important basis for system maintenance personnel to troubleshoot and optimize performance. The monitoring layer ensures the stability and security of the resource layer. By monitoring the status and performance indicators of the resource layer in real time, the monitoring layer can promptly discover potential problems and anomalies, such as resource overload and performance bottlenecks. The monitoring layer can also provide alarm and notification mechanisms. When an anomaly occurs in the resource layer, it can promptly notify maintenance personnel to handle it, thereby ensuring the stability and security of the resource layer. The two work together to improve the overall system performance. The resource layer provides basic resources, while the monitoring layer is responsible for monitoring and optimizing the use of these resources to ensure that the system can run efficiently and stably. The monitoring layer provides optimization for the resource layer. By analyzing the data of the resource layer, the monitoring layer can discover the patterns and trends of resource usage and provide suggestions for the optimization of the resource layer. For example, the monitoring layer can suggest increasing or decreasing resource capacity, adjusting resource allocation strategies, etc., to optimize resource utilization efficiency and performance.

[0095] (5) Task Layer and Monitoring Layer: Functional Collaboration. The task layer is responsible for the specific execution and management of tasks, while the monitoring layer is responsible for real-time monitoring and early warning of the task layer's operational status. They collaborate functionally to ensure the smooth progress of the project. The monitoring layer provides data support for the task layer's decision-making by collecting operational data from the task layer in real time. When the monitoring layer detects an anomaly in task execution, it can promptly issue an early warning to the task layer so that the task layer can take appropriate corrective measures. Mutual Influence: The operational status of the task layer directly affects the output of the monitoring layer. When problems occur in the task layer, the monitoring layer will issue timely warnings, thereby reminding project managers to take appropriate measures to resolve the issues. The accuracy and timeliness of the monitoring layer also have a significant impact on the execution efficiency of the task layer. If there are errors or delays in the data collection and processing of the monitoring layer, it may lead to incorrect decisions by the task layer, thereby affecting the overall progress and quality of the project. Information Flow: A sound information flow mechanism needs to be established between the task layer and the monitoring layer. On the one hand, the task layer needs to provide the monitoring layer with real-time task execution data so that the monitoring layer can process and analyze the data; on the other hand, the monitoring layer also needs to promptly feed back the processed data and analysis results to the task layer so that the task layer can make corresponding decisions and adjustments based on this information.

[0096] (6) Application Layer and Monitoring Layer: The monitoring layer is responsible for data collection and analysis. It collects real-time operational data of computing resources, such as the usage of GPUs, memory, and network bandwidth, as well as key performance indicators such as response time and system load during task execution. This data is an important input for computing power scheduling decisions, helping the scheduling system understand the current resource distribution and task load. In the intelligent decision-making of computing power scheduling at the application layer, the computing power scheduling system performs intelligent resource allocation and task scheduling based on the data provided by the monitoring layer. It dynamically adjusts the allocation of computing resources according to factors such as task priority, resource requirements, and real-time load to ensure efficient task operation and maximum resource utilization. The accuracy of the monitoring layer affects computing power scheduling. If the data collection of the monitoring layer is inaccurate or delayed, the computing power scheduling system may make decisions based on incorrect information, leading to unreasonable resource allocation or low task execution efficiency. Therefore, the accuracy and real-time performance of the monitoring layer have a significant impact on the effectiveness of computing power scheduling. The optimization of computing power scheduling at the application layer increases the demand for data from the monitoring layer. With the continuous advancement of computing power scheduling technology, the data demand of the monitoring layer is also constantly increasing. To manage resources more precisely, computing power scheduling systems may need to monitor more diverse resource metrics or finer-grained data, prompting the monitoring layer to continuously upgrade and optimize its data acquisition and processing capabilities. The monitoring layer provides real-time data to the computing power scheduling system by collecting and transmitting operational data of computing resources in real time, offering accurate information support. This data forms the basis for the intelligent decision-making of the computing power scheduling system. The application-layer computing power scheduling system feeds back scheduling results to the monitoring layer. After completing resource allocation and task scheduling, the system can provide feedback to the monitoring layer. Based on this feedback data, the monitoring layer can further monitor task execution and resource usage, providing a reference for subsequent scheduling decisions.

[0097] Example 2

[0098] like Figure 4 As shown, specifically in the next-generation cloud platform system for energy sales internet application services, this invention relates to a next-generation cloud platform system for energy sales internet application services, aiming to support heterogeneous management capabilities and the unified management, monitoring, and allocation of GPU computing resources with different architectures. A large energy company needs to process a large number of image recognition tasks, which require high-performance GPU computing power. Simultaneously, the company also needs to process some general computing tasks that can be completed by CPUs.

[0099] This platform shields the hardware differences between different GPU architectures, decoupling applications' dependencies on underlying hardware by masking the variations in underlying chips and devices. This forms a unified "cloud resource pool," providing a stable operating environment and rich ecosystem services. Administrators can manage all resources using a single system and allocate heterogeneous resources to different tenants. Users only need to log in to one platform to apply for and use shared resources from different architectures within the environment. It provides service-oriented computing power model encapsulation, supporting the application of the cloud platform in various domestic IT innovation industries. The system provides general cloud computing services through unified task management and scheduling of resources from different chip architectures, maximizing the utilization of resources across different chip architectures and providing a stable operating environment and rich ecosystem services during user business expansion and application innovation. This embodiment demonstrates the optimal configuration of the system and its working principle.

[0100] The platform architecture of the entire system is as follows:

[0101] ①Resource layer

[0102] It includes capabilities for resource registration, resource modeling, resource reconfiguration, resource evaluation, and resource matching, providing access to various types of processors, storage devices, memory, and other resources. These resources are encapsulated into standard computing resource pools through virtualization technology, allowing for unified management and scheduling by the cloud platform layer. Together, these hardware resources form the foundation of a heterogeneous computing environment.

[0103] ② Security layer

[0104] It includes security policies and resource isolation capabilities, providing containerization technology to achieve resource isolation when multiple tasks share GPU resources. It employs container-level sandbox security technology, formulating and enforcing strict security policies to ensure that GPU resources are not accessed or misused without authorization.

[0105] ③Task layer

[0106] This module includes capabilities for task submission, task analysis, task parsing, task allocation, task execution, and task feedback, and is responsible for the creation, submission, execution, and monitoring of GPU tasks. Through this module, users can submit tasks requiring GPU acceleration to the platform and monitor the task's execution status and progress. The cloud platform parses the submitted tasks, provides resource-matching results, and allocates the tasks appropriately to available GPUs for execution. After the task is completed, the cloud platform returns the execution result to the user.

[0107] ④ Monitoring layer

[0108] It includes resource monitoring, task monitoring, node diagnostics, and computing power optimization capabilities, providing real-time monitoring via a cloud platform to continuously track GPU usage. It also monitors the execution status and progress of tasks throughout the entire process. A one-click diagnostic function is provided to diagnose node hardware and software configurations. The platform can automatically adjust GPU configurations and parameters to adapt to different application requirements and load conditions.

[0109] ⑤ Application Layer

[0110] It includes computing power scheduling and resource sharing capabilities, as well as interfaces for user interaction with the platform. Users can submit computing power requests through web interfaces, API interfaces, and other methods. GPU computing power is dynamically scheduled based on application requirements and GPU resource status. The scheduling engine can perform scheduling based on different scheduling strategies (such as fairness, priority, performance, etc.) to meet the needs of different applications.

[0111] The platform's functions include: Unified management of heterogeneous resources: The platform enables unified management of computing resources across different chip architectures, including resource allocation, monitoring, and scheduling. Dynamic scheduling of computing resources: Based on the needs of upper-layer applications, the platform can dynamically adjust the allocation of computing resources to ensure maximum resource utilization. Cross-architecture application deployment: The platform supports cross-architecture application deployment, allowing users to run the same applications on different chip architectures without additional adaptation work. Security isolation and protection: The platform employs advanced security isolation technology to ensure data and applications are isolated between different tenants, while providing comprehensive security protection measures to safeguard user data security.

[0112] In practical implementation, the steps are as follows: resource preparation: at the cloud platform layer, prepare a pool of computing resources including both CPU and GPU chip architectures. Ensure that these resources have been encapsulated and standardized using virtualization technology.

[0113] Application Deployment: Image recognition tasks and general computing tasks are deployed on their respective chip architectures. For image recognition tasks, high-performance GPUs are selected for computation; for general computing tasks, CPUs are selected for computation.

[0114] Resource scheduling: The resource scheduling layer performs intelligent scheduling based on task requirements and the status of underlying resources. When an image recognition task requires more GPU computing power, the resource scheduling layer will automatically allocate more resources from the GPU resource pool to these tasks; similarly, when a general computing task requires more CPU computing power, it will automatically allocate resources from the CPU resource pool.

[0115] Monitoring and Optimization: Utilize application-layer monitoring functions to view task execution status and results in real time. Based on monitoring data, perform necessary optimizations and adjustments to the platform to ensure maximum resource utilization and successful task completion.

[0116] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A GPU computing power scheduling method based on a cloud-multi-core heterogeneous computing power platform, characterized in that, The method includes the following steps: Step S1: Heterogeneous resource registration and modeling. The resource registration module connects to hardware devices containing multiple types of GPUs, collects device model, memory capacity and performance index data, and builds a resource feature database containing topological relationships to support mixed access of chips. Step S2: Virtualization resource reconstruction. The physical GPU is pooled into virtual GPU resources using hardware abstraction layer technology. Containerization technology is used to achieve memory isolation and computing unit partitioning. Each virtual GPU instance is equipped with an independent driver stack and security sandbox. Step S3: Multimodal task submission. Receive CUDA / OpenCL computing tasks submitted by users, parse task requirement parameters, including the number of computing cores, memory usage, and data throughput threshold, and generate a task descriptor containing priority tags. Step S4: Topology-aware resource matching. Based on the real-time updated resource status graph, an improved Hungarian algorithm is used for multi-dimensional resource matching. When selecting computing nodes, PCIe bandwidth, NVLink connection status and memory fragmentation are evaluated simultaneously. Step S5: Dynamic load scheduling. Deploy a distributed scheduling engine to collect real-time data on node GPU temperature, utilization, and task queue depth. When a local hotspot is detected, trigger a load balancing strategy, using a combination of task migration and computing power reallocation mechanism. The dynamic load scheduling in step S5 includes a scheduling strategy optimization engine based on deep reinforcement learning, specifically implemented as follows: Construct a state feature space and collect node GPU utilization, video memory usage, PCIe link bandwidth utilization, and task queue waiting time to form a 16-dimensional state vector. Design a dual-channel Dueling DQN network architecture, in which the value stream network evaluates the global scheduling benefit and the advantage stream network calculates the advantage value of local scheduling actions. Define the reward function R = α * (1 - load variance) + β task completion rate - γ migration cost, where α, β, and γ are adjustable weight coefficients; A deployment experience replay pool stores historical scheduling decision data, and a priority sampling mechanism is used to select high-value training samples. Implement an online policy update mechanism to trigger real-time fine-tuning of network parameters when a cluster topology change or a sudden change in load pattern is detected; Introducing action masking technology to constrain invalid scheduling operations ensures that task migration only occurs between nodes with hardware compatibility; Step S6: Adaptive task execution, inject runtime monitoring agent into GPU instance, dynamically adjust CUDA stream priority and memory page locking strategy, and use MPS multi-process service technology to achieve exclusive resource guarantee for critical tasks. Step S7: Full-link monitoring and diagnosis, building a three-dimensional monitoring system: monitoring the SM unit utilization and ECC error rate of GPU hardware through the resource layer; tracking the computation progress and memory leakage risk through the task layer; implementing health scoring and triggering automatic isolation mechanisms through the node layer.

2. The GPU computing power scheduling method based on a multi-core heterogeneous computing power platform in one cloud as described in claim 1, characterized in that: After completing the construction of the three-dimensional monitoring system, proceed to steps S8 and S9; Step S8: Parameterized computing power optimization, using an improved Adam optimizer to dynamically adjust scheduling strategy parameters; calculate the first and second moment estimates of the cluster load gradient; correct deviations based on historical load fluctuations; update task allocation weight coefficients according to the corrected momentum values; introduce Nesterov acceleration gradient to predict future load trends; Step S9: Closed-loop feedback update. Collect actual resource consumption data during task execution, compare and analyze it with preset requirement parameters, iteratively optimize the resource matching algorithm through reinforcement learning model, and update the virtual GPU specification template library.

3. The GPU computing power scheduling method based on a multi-core heterogeneous computing power platform in one cloud as described in claim 1, characterized in that: The virtualization resource reconstruction in step S2 also includes: Create securely isolated vGPU instances and use hardware-assisted SR-IOV technology to divide the physical GPU into multiple VFs, with each VF bound to an independent secure enclave; It enables dynamic quota management of video memory, and can elastically expand vGPU video memory from 1GB to 24GB within 0.1 seconds according to task requirements; Deploy a driver-level protection module to intercept abnormal CUDA API requests and block unauthorized memory access operations through system call hijacking technology; Design a hot migration channel to seamlessly migrate the vGPU instance along with its running tasks to a standby node when the physical GPU is detected to be overheating. Build an energy-aware scheduler to dynamically adjust the core frequency based on the SM unit utilization of vGPU instances, achieving a 30% reduction in power consumption with a performance loss of less than 5%.

4. The GPU computing power scheduling method based on a multi-core heterogeneous computing power platform in one cloud as described in claim 1, characterized in that: The resource matching in step S4 further includes: Construct a multi-dimensional topology feature map and integrate NVLink connection counts between nodes, PCIe Switch levels, and RDMA network latency data; Develop a graph neural network matching model that abstracts task requirements into feature subgraphs and searches for connected subgraphs that satisfy the constraints in the topological graph. Transmission cost prediction is performed, and the bandwidth-delay product of the data transport path is calculated for candidate node combinations to select the topology path with the lowest cost. Deploy a prefetch optimization module to synchronously schedule relevant datasets to the HBM memory of the target node during task allocation; Implement NUMA-aware memory binding to lock the task process to the CPU memory domain directly connected to the target GPU.

5. The GPU computing power scheduling method based on a multi-core heterogeneous computing power platform in one cloud as described in claim 1, characterized in that: The end-to-end monitoring in step S7 also includes: Deploy an LSTM fault prediction model to analyze historical SM cell error rates, temperature gradient changes, and ECC correction times to predict hardware faults 30 minutes in advance. Establish a tiered alert mechanism, defining yellow, orange, and red alerts; Implement a proactive self-healing strategy: initiate driver reload and memory remapping for yellow-alert nodes, trigger service migration for orange-alert nodes, and isolate the entire fault domain when a red-alert occurs. The design incorporates a fault-tolerant training mode that automatically switches to a redundant computing node to continue execution and rebuilds the computing context when an unrecoverable error is detected. Generate a fault tracing map, correlate and analyze hardware logs, task characteristics and environmental variables, locate the root cause and update the health model.

Images