Distributed GPU resource scheduling method and device, equipment and storage medium
By employing a distributed GPU resource scheduling method, and leveraging the collaborative work of task management agents and resource management agents, the problems of single point of failure and low resource utilization in traditional centralized scheduling schemes are solved, thereby achieving efficient and unified scheduling of heterogeneous GPU resources and improving system scalability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-30
- Publication Date
- 2026-04-03
AI Technical Summary
Traditional centralized GPU resource scheduling schemes suffer from single points of failure, poor scalability, and low resource utilization, making them difficult to adapt to large-scale, dynamic, and heterogeneous AI workloads. Furthermore, they lack the ability to provide a unified abstraction of heterogeneous hardware and multi-objective collaborative optimization capabilities.
By employing a distributed GPU resource scheduling method, and through the collaborative work of task management agents and resource management agents, resource information broadcasting, demand standardization, cost calculation, and target selection are achieved, a decentralized agent society is constructed to make local decisions to achieve global optimization.
It improves system resource utilization and reliability, supports unified scheduling and management of heterogeneous GPU resources, reduces fragmentation and migration overhead, and improves system scalability and economy.
Smart Images
Figure CN121785792A_ABST
Abstract
Description
Technical Field
[0001] The embodiments in this specification relate to the field of artificial intelligence, and in particular, to a distributed GPU resource scheduling method, apparatus, device, and storage medium. Background Technology
[0002] With the rapid development of artificial intelligence and deep learning technologies, GPU clusters have become the core infrastructure supporting large-scale model training and inference tasks.
[0003] Traditional centralized static scheduling schemes exhibit a series of fundamental limitations when dealing with large-scale, dynamic, and heterogeneous AI workloads: First, they rely on a single central node for global decision-making, which not only poses a single point of failure risk but also creates performance bottlenecks as the cluster size increases, severely restricting scalability; second, static resource allocation strategies (such as fixed resource reservation) cannot adapt to the dynamically changing resource requirements of AI tasks, easily leading to resource fragmentation and low utilization; finally, traditional schedulers lack a unified abstraction of heterogeneous hardware (such as different models of GPUs) and the ability to collaboratively optimize multiple objectives (such as latency, energy efficiency, and cost), making it difficult to achieve efficient cross-platform scheduling and resource guarantees for critical tasks.
[0004] This invention aims to solve the problems of single point of failure, poor scalability and low resource utilization in traditional centralized GPU resource scheduling schemes, improve system resource utilization and system reliability, and support unified scheduling and management of heterogeneous GPU resources, providing a high-performance and easy-to-maintain solution for complex environments such as cloud computing centers and AI training platforms. Summary of the Invention
[0005] The purpose of the embodiments in this specification is to provide a distributed GPU resource scheduling method, apparatus, device, and storage medium to improve system resource utilization and system reliability.
[0006] To achieve the above objectives, on the one hand, embodiments of this specification provide a distributed GPU resource scheduling method, including: a task management agent and at least one resource management agent; After the resource management agent starts, it broadcasts its own GPU resource information at a certain frequency; The task management agent accepts the task, parses the task to generate standardized GPU resource requirements, and publishes the GPU resource requirements to each of the resource management agents. Each of the resource management agents receives the GPU resource requirements, analyzes its own GPU resource information based on the resource requirements, and decides whether the task is acceptable. When the task is acceptable, each of the resource management agents calculates the cost of executing the task and sends the cost to the task management agent. The task management agent receives the costs from each of the resource management agents and selects a target resource management agent to execute the task based on the costs.
[0007] Preferably, the GPU resource information includes: resource utilization, hardware status, task context information, topology links, and fault warning indicators.
[0008] Preferably, the process of each resource management agent receiving the GPU resource request, analyzing its own GPU resource information based on the resource request, and deciding whether the task is acceptable further includes: Each of the resource management agents receives the GPU resource requirements and determines whether its own free video memory meets the video memory requirements in the GPU resource requirements. If the conditions are met, the task can be accepted; If these conditions are not met, the task is unacceptable.
[0009] Preferably, the process of each resource management agent calculating the cost of executing the task and sending the cost to the task management agent further includes: Each resource management agent calculates the time cost of executing the task based on the computing power requirement in the GPU resource requirements and its own GPU utilization. Each resource management agent calculates the operating cost of executing the task based on the time requirement in the GPU resource requirements and its own unit operating cost. The cost of the task is obtained by combining the time cost and the operating cost.
[0010] Preferably, the step of selecting the target resource management agent based on the cost to perform the task further includes: Obtain the historical task completion status of each resource management agent; The credibility of each resource management agent is obtained based on the historical task completion status. Based on the cost of each resource management agent and its corresponding credibility, a target resource management agent is selected.
[0011] Preferred options also include: When the target resource management agent executes the task, it monitors the GPU resource requirements of the task in real time and sends them to the task management agent. When the change in GPU resource requirements exceeds the resource requirement threshold, the task management agent selects another resource management agent to execute the task.
[0012] Preferred options also include: When the target resource management agent executes the task, it monitors the GPU resource requirements of the task in real time and sends them to the task management agent. If the task management agent does not receive a GPU resource request after a preset time threshold is exceeded, the task management agent will select another resource management agent to execute the task.
[0013] On the other hand, embodiments of this specification provide a distributed GPU resource scheduling device, including: a task management agent and at least one resource management agent; The broadcast module is used to broadcast the GPU resource information of the resource management agent at a certain frequency after it starts up. The parsing module is used for the task management agent to receive tasks, parse the tasks to generate standardized GPU resource requirements, and publish the GPU resource requirements to each of the resource management agents. The decision module is used by each of the resource management agents to receive the GPU resource requirements, analyze its own GPU resource information based on the resource requirements, and decide whether the task is acceptable. The calculation module is configured to, when the task is acceptable, each of the resource management agents calculates the cost of executing the task and sends the cost to the task management agent; The selection module is used by the task management agent to receive the costs of each resource management agent and select a target resource management agent to execute the task based on the costs.
[0014] In another aspect, embodiments of this specification also provide a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the computer program, when executed by the processor, implements the steps of any of the methods described above.
[0015] In another aspect, embodiments of this specification also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor of a computer device, implements the steps of any of the methods described above.
[0016] As can be seen from the technical solutions provided in the embodiments of this specification above, the embodiments of this specification achieve global real-time visibility by periodically broadcasting GPU resources through resource management agents; task management agents standardize and uniformly publish task requirements, making heterogeneous resources comparable on the same scale. Each resource agent performs local parsing and decision-making, avoiding central bottlenecks and communication back-and-forth, and quickly selecting executable nodes; cost-based bidding and optimal allocation internalize factors such as computing power, bandwidth, data location, queuing latency, and energy consumption into a unified metric, improving throughput and utilization while ensuring response latency, reducing fragmentation and migration overhead, and adaptively balancing load according to dynamic resource fluctuations. Multi-agent redundancy and decentralized collaboration also improve fault tolerance and scalability, making large-scale distributed GPU scheduling more efficient, stable, and economical.
[0017] To make the above and other objects, features and advantages of this specification more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 A flowchart illustrating a distributed GPU resource scheduling method provided in an embodiment of this specification is shown. Figure 2 A flowchart illustrating the decision-making process for determining the acceptability of a task, as provided in an embodiment of this specification, is shown. Figure 3 A schematic diagram illustrating the process of a resource management agent performing tasks based on the cost selection target provided in an embodiment of this specification is shown. Figure 4 A flowchart illustrating another distributed GPU resource scheduling method provided in an embodiment of this specification is shown. Figure 5 A flowchart illustrating another distributed GPU resource scheduling method provided in an embodiment of this specification is shown. Figure 6 This specification shows a schematic diagram of the module structure of a distributed GPU resource scheduling device provided in an embodiment. Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of this specification is shown.
[0020] Explanation of symbols in the attached drawings: 100. Broadcast module; 200. Parsing module; 300. Decision-making module; 400. Calculation module; 500. Select module; 702. Computer equipment; 704, Processor; 706. Memory; 708. Drive mechanism; 710. Input / Output Module; 712. Input devices; 714. Output devices; 716. Presentation equipment; 718. Graphical User Interface; 720. Network interface; 722. Communication link; 724. Communication bus. Detailed Implementation
[0021] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the embodiments of this specification.
[0022] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation access points for users to choose to authorize or refuse.
[0023] To address the aforementioned issues, embodiments of this specification provide a distributed GPU resource scheduling method, apparatus, device, and storage medium. Figure 1This is a flowchart illustrating a distributed GPU resource scheduling method provided in an embodiment of this specification. This specification provides the operational steps of the method as described in the embodiments or flowchart, but based on conventional or non-inventive methods, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system or device products, the methods shown in the embodiments or accompanying drawings can be executed sequentially or in parallel.
[0024] It should be noted that the terms "first," "second," etc., in the description, claims, and accompanying drawings of the embodiments in this specification are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, apparatus, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.
[0025] Reference Figure 1 This specification provides a distributed GPU resource scheduling method, comprising: a task management agent and at least one resource management agent; S101: After the resource management agent starts, it broadcasts its own GPU resource information at a certain frequency; S102: The task management agent accepts the task, parses the task to generate standardized GPU resource requirements, and publishes the GPU resource requirements to each of the resource management agents; S103: Each of the resource management agents receives the GPU resource requirement, analyzes its own GPU resource information based on the resource requirement, and decides whether the task is acceptable. S104: When the task is acceptable, each of the resource management agents calculates the cost of executing the task and sends the cost to the task management agent; S105: The task management agent receives the cost of each of the resource management agents, and selects a target resource management agent to execute the task based on the cost.
[0026] The scheduling system comprises both CPU and GPU. The Resource Management Agent (RMA) runs on the CPU, reading GPU status and communicating with the message bus or control plane via NVML / DCGM, without consuming GPU computing power. The Task Management Agent (TMA) also runs on the CPU, handling tasks and issuing requirements without requiring GPU resources. Both RMA and TMA are control plane processes, deployed on the CPU by default. They can co-operate with GPU tasks but do not consume GPU resources; GPU resources are only requested and used when actually participating in a task.
[0027] A task is a workload description with code, configuration or data references and resource requirements, and has a complete lifecycle (commit → deployment → execution → monitoring → completion / failure → recycling).
[0028] This specification's embodiments abandon the scheduling mode of a single central node, abstracting tasks into Task Management Agents (TMAs) and Resource Management Agents (RMAs) with autonomous decision-making capabilities, thereby constructing a decentralized "agent society." An agent generally refers to a system or entity that can perceive the environment, make decisions, and take actions to achieve a goal. Unlike a node, which is a pure resource carrier and not equivalent to an agent, an agent is a higher-level control logic that can run on it and make decisions on behalf of tasks or resources.
[0029] The task management agent and the resource management agent can interact, enabling the system to emerge with global optimization capabilities through a large number of concurrent local decisions, fundamentally improving the system's scalability and reliability. The embodiments in this specification elevate the MCP protocol to a "general semantic layer" between agents, providing a standardized communication foundation for collaborative scheduling in heterogeneous environments. Combined with a high-performance message middleware (such as Kafka) to ensure communication efficiency, and using a RESTful API as an auxiliary management interface, a high-performance, easy-to-maintain system integration framework is formed.
[0030] This design enables the system to be compatible with different types of hardware, achieve unified pooling and scheduling of heterogeneous GPU resources, and provide plug-and-play high-efficiency resource solutions for scenarios such as cloud computing centers and AI training platforms, significantly reducing the operational burden in complex heterogeneous environments.
[0031] When the resource management agent starts up, it broadcasts its complete GPU resource information to the entire system via a high-performance message middleware (such as Kafka) to complete the registration process. The broadcast uses a standard resource information description framework, including GPU hardware specifications (model, architecture generation), computing power (FP32 / FP16 / TF32 TFLOPS), memory configuration (total video memory, bandwidth), node network topology information (such as the switch the node belongs to, bandwidth / latency with other nodes), GPU hardware topology information (such as NVLink bandwidth for multi-GPU interconnection), and fault history (such as the number of faults in the past 30 days, recovery time). A structured JSON format is used to ensure system parsability and scalability.
[0032] Crucially, during the registration process, RMAs not only provide static resource information but also initialize a dynamic resource tracker to continuously monitor the actual available resources of the GPU. This "socialized" registration allows newly added RMAs to quickly integrate into the resource pool and participate in global resource allocation. To quantify whether a new RMA is "complete and reliable," a completeness score is introduced:
[0033] in For resource feature completeness, Resource feature weights (configurable, default 0.8). For topological information completeness, Weights for topology information (configurable, default 0.2).
[0034] Resource feature completeness includes GPU hardware specifications, computing power, memory configuration, and fault history. If all four are included, the resource feature completeness is 1; if any one is missing, the resource feature completeness is reduced by 0.25. Topology information completeness includes node network topology information and GPU hardware topology information. If both are included, the topology information completeness is 1; if either is missing, the topology information completeness is reduced by 0.5. The closer this score is to 1, the more comprehensive and usable the RMA information is.
[0035] Furthermore, RMAs broadcast their own GPU resource information at a certain frequency. All active RMAs will send heartbeat signals through the message middleware at a certain frequency (typically 1Hz), broadcasting their real-time GPU resource information, including: Resource utilization: video memory utilization, computing power utilization, PCIe bandwidth utilization.
[0036] Hardware status: GPU temperature, power consumption, error counter.
[0037] Task context information: current task list, task priority, and estimated remaining execution time of the current task.
[0038] Topology links: such as bandwidth utilization between nodes.
[0039] Fault warning indicator: GPU error counter growth rate The heartbeat mechanism not only serves resource information tracking but also forms the basis for system fault detection. Through continuous heartbeat monitoring, the system can quickly identify faults and trigger recovery processes.
[0040] To quantify the health status of RMA, we introduce a health score:
[0041] Where t is the current time, Δt is the length of the review time window, τ is the integration variable (within the time window), λ is the decay factor that determines the weight of recent observations, and ω1, ω2, ω3, and ω4 are weights (non-negative, summing to 1). Let τ be the GPU temperature at time τ. The preset upper temperature limit, This is the temperature change stability coefficient. Let θ be the first derivative of temperature with respect to time, and let θ be the temperature change sensitivity coefficient. Let be the normalized value of the error counter growth rate at time τ. As an indicator of resource stability, This represents the average computing power utilization rate. The standard deviation of computing power utilization. λ is the normalized value of the historical fault frequency, and λ is the attenuation factor.
[0042] A specific method for normalization is to map each original value to [0, 1], where 1 represents "worst / least healthy" and 0 represents "best / healthiest", and then synthesize them according to weights. Commonly used methods include piecewise linear transformation + pruning + smoothing, with logistic functions used for extreme values.
[0043] Furthermore, users submit tasks through the gateway, and the task management agent accepts the tasks, parses them to generate standardized GPU resource requirements, and extracts key parameters including memory requirements (representing the memory capacity required by the task), computing power requirements (representing the throughput and parallel resources required by the task), priority flags (representing the importance of the task), and heterogeneous computing support capabilities (representing whether the task is compatible with different hardware). For situations where users cannot accurately determine resource requirements, the system provides a lightweight sandbox environment that allows for the execution and monitoring of task segments.
[0044] After a user submits a task through the gateway, the system instantiates a dedicated Task Management Agent (TMA), which will operate throughout the entire task lifecycle. The TMA uses a state machine model to manage task state transitions, including initialization, task deployment, execution, monitoring, and completion. For tasks with unclear resource requirements, the TMA initiates a resource exploration phase, running representative code snippets of the task in a sandbox to monitor and generate precise GPU resource requirements.
[0045] TMA generates standardized GPU resource requirements. In particular, for deep learning training tasks, TMA will refer to the resource consumption patterns of similar models to predict and verify the requirements, and publish the GPU resource requirements to each resource management agent in a standardized format based on the MCP protocol.
[0046] TMA publishes GPU resource requests to a dedicated topic via a message middleware, which is subscribed to by all active RMAs across the network. This publish-subscribe model ensures efficient broadcasting of information to all potential resource providers.
[0047] The above-mentioned system includes a timeout mechanism, where the TMA waits for a preset time window (usually 5-10 seconds) to collect the RMA's response. To improve efficiency, the system implements a tiered strategy: first, a rapid local domain deployment is performed; if there are insufficient responses, it is expanded to a global deployment. This strategy strikes a balance between deployment speed and resource matching quality.
[0048] In the embodiments described in this specification, reference is made to Figure 2 The resource management agents receiving the GPU resource requirements, analyzing their own GPU resource information based on the requirements, and deciding whether the task is acceptable further include: S201: Each of the resource management agents receives the GPU resource requirements and determines whether its own free video memory meets the video memory requirements in the GPU resource requirements. S202: If satisfied, the task is acceptable; S203: If not satisfied, the task is unacceptable.
[0049] In the embodiments of this specification, "acceptable task" means "able to receive tasks," and "unacceptable task" means "unable to receive tasks."
[0050] When the task is acceptable, each of the resource management agents calculates the cost of executing the task and sends the cost to the task management agent, which further includes: S2021: Each of the resource management agents obtains the time cost of executing the task based on the computing power requirement in the GPU resource requirements and its own GPU utilization rate; S2022: Each of the resource management agents obtains the operating cost of executing the task based on the time requirement in the GPU resource requirements and its own unit operating cost; S2023: The cost of the task is obtained by combining the time cost and the operating cost.
[0051] The method for calculating the time cost of performing the task is as follows: Time cost = Total computational load / Effective computing power; The total computational load refers to the throughput required by the task under ideal conditions. Effective computing power = computing power requirement × implementation efficiency × idle ratio; Computing power requirement refers to the peak throughput of a task processed by a graphics card under ideal conditions; Efficiency = Actual Bandwidth / Theoretical Memory Bandwidth; Idle percentage = 1 - GPU utilization.
[0052] Operating cost = Time requirement × Unit operating cost.
[0053] The cost of a task consists of time cost and operating cost, and the resource management agent sends the cost to the task management agent.
[0054] The resource management agent generates a message based on a template containing the following key information and sends it to the task management agent: rma_id: A unique identifier for an RMA.
[0055] expected_time_sec: Time cost, used by TMA to evaluate efficiency.
[0056] cost: The cost of the task, used by TMA to assess its economic viability.
[0057] free_vram_after_allocation: The estimated remaining free video memory after the task is executed. It is obtained by subtracting the video memory requirement from the free video memory and the video memory requirement in the GPU resource requirements. This helps RMA to make subsequent resource planning and decisions.
[0058] Ultimately, this message will be sent back to TMA by RMA through the message middleware.
[0059] In the embodiments described in this specification, reference is made to Figure 3 The step of selecting the target resource management agent based on the cost to perform the task further includes: S301: Obtain the historical task completion status of each resource management agent; S302: Based on the historical task completion status, obtain the credibility of each resource management agent; S303: Based on the cost of each resource management agent and its corresponding credibility, select the target resource management agent.
[0060] After collecting all messages, TMA evaluates and selects the target resource management agent based on Multi-Attribute Utility Theory (MAUT). MAUT is a systematic decision-making method that compares and ranks multiple alternatives by quantifying the utility of multiple attributes and synthesizing them into a single score.
[0061] The TMA waits for a preset time (e.g., 5 seconds) to collect messages sent from each RMA. After the timeout, the TMA will no longer accept new messages to ensure timely decision-making. This guarantees that the system can respond within the specified time.
[0062] TMA can obtain the historical task completion status of each resource management agent stored in the storage unit, including the total number of historical tasks and the number of completed historical tasks. It calculates the ratio of completed historical tasks to the total number of historical tasks to obtain the historical task completion rate. The historical task completion rate can characterize credibility, and there is a positive correlation between historical task completion rate and credibility. Furthermore, it iterates through the messages sent by each resource management agent and calculates their utility score based on the historical task completion rate, time cost, and operating cost. This process follows the core idea of MAUT, which is to define a utility function for each key piece of information and then weight and aggregate these utilities into a total utility value.
[0063] When calculating the total utility value, the historical task completion rate, time cost, and operating cost need to be normalized to [0,1] respectively. Specifically, the normalization can be done in the following ways: Obtain the historical task completion rate of each resource management agent, and find the maximum value x and the minimum value y. Calculate the historical task completion rate a of any resource management agent as follows: (ay) / (xy) to obtain the normalized historical task completion rate.
[0064] Similarly, obtain the time cost of each resource management agent, find the maximum value x and the minimum value y, and calculate the time cost a of any resource management agent as follows: (ay) / (xy) to obtain the normalized time cost.
[0065] Similarly, obtain the operating costs of each resource management agent, find the maximum value x and the minimum value y, and calculate the operating cost a of any resource management agent as follows: (ay) / (xy) to obtain the normalized operating cost.
[0066] Based on the normalized historical task completion rate α, the normalized time cost β, and the normalized operating cost γ, the total utility value is calculated as follows: Total utility value = α· +β· +γ· ; in, + + =1, , , These are all weighting coefficients, and specific values can be set according to actual circumstances.
[0067] Select the resource management agent with the highest total utility value as the target resource management agent.
[0068] Reference Figure 4 The embodiments in this specification also include: S401: When the target resource management agent executes the task, it monitors the GPU resource requirements of the task in real time and sends them to the task management agent; S402: When the change in GPU resource requirements exceeds the resource requirement threshold, the task management agent selects another resource management agent to execute the task.
[0069] Essentially, the GPU resource requirements of the task change, and the target resource management agent may not be able to meet the new task requirements, so it is necessary to select another resource management agent.
[0070] Reference Figure 5 The embodiments in this specification also include: S501: When the target resource management agent executes the task, it monitors the GPU resource requirements of the task in real time and sends them to the task management agent; S502: When the preset time threshold is exceeded and the task management agent does not receive a GPU resource request, the task management agent selects another resource management agent to execute the task.
[0071] The essence of the problem is that the target resource management agent is unstable and may be malfunctioning, failing to maintain communication with the task management agent. In this case, it is necessary to select another resource management agent.
[0072] Fault recovery employs a checkpoint restart strategy, where the TMA reschedules tasks to healthy RMAs. The system maintains a fault history to avoid repeatedly scheduling tasks to unreliable RMAs; this self-healing capability ensures continuous service capability even in the event of component failure.
[0073] Specifically, it can be: S1: The task management agent parses the task to generate standardized GPU resource requirements and publishes the GPU resource requirements to each of the resource management agents; S2: Each of the resource management agents receives the GPU resource requirements, analyzes its own GPU resource information based on the resource requirements, and decides whether the task is acceptable. S3: When the task is acceptable, each resource management agent calculates the cost of executing the task and sends the cost to the task management agent; S4: The task management agent receives the cost from each of the resource management agents and selects other resource management agents to execute the task based on the cost.
[0074] In addition, the system can also implement a fine-grained monitoring system. The RMA collects GPU resource information at a high frequency (up to 100Hz), and this information is transmitted to the TMA in real time through a stream processing pipeline to update task progress and estimated completion time.
[0075] The monitoring data is aggregated and visualized on the TMA platform, providing users with a comprehensive view of the tasks through a dashboard. Simultaneously, the monitoring data is also used for performance analysis to identify potential bottlenecks and optimization opportunities. Specifically, the system detects resource contention and performs micro-scheduling optimizations when multiple tasks share the same GPU.
[0076] When the task is successfully completed, the RMA sends a task completion signal to the TMA, containing a complete execution summary: actual resource consumption, performance metrics, and result metadata. After verifying the task output, the TMA triggers resource release, ensuring that GPU resources are completely cleared and returned to the resource pool.
[0077] The resource reclamation process employs a gradual release strategy, releasing computing resources first, then storage resources, and finally network resources. This orderly release avoids resource fragmentation and ensures the resource pool is in an optimal state for receiving new tasks. Once TMA confirms that all resources have been released, it will self-deregister, ending its lifecycle.
[0078] The embodiments in this specification achieve global real-time visibility by periodically broadcasting GPU resources through a resource management agent; a task management agent standardizes and uniformly publishes task requirements, making heterogeneous resources comparable on the same scale. Each resource agent performs local parsing and decision-making, avoiding central bottlenecks and communication back-and-forth, and quickly selecting executable nodes; cost-based bidding and optimal allocation internalize factors such as computing power, bandwidth, data location, queuing latency, and energy consumption into a unified metric, improving throughput and utilization while ensuring response latency, reducing fragmentation and migration overhead, and adaptively balancing load according to dynamic resource fluctuations. Multi-agent redundancy and decentralized collaboration also improve fault tolerance and scalability, making large-scale distributed GPU scheduling more efficient, stable, and economical.
[0079] This application provides users with access to relevant big data analysis (such as personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.), allowing users to choose to agree to or reject automated decision results; if the user chooses to reject, the process will proceed to the expert decision-making process.
[0080] Based on the distributed GPU resource scheduling method described above, this specification also provides a distributed GPU resource scheduling device. The device may include a system (including a distributed system), software (application), module, component, server, client, etc., using the method described in this specification, combined with necessary hardware implementation. Based on the same innovative concept, the devices in one or more embodiments provided in this specification are as described in the following embodiments. Since the implementation schemes and methods for solving the problem are similar, the implementation of specific devices in this specification can refer to the implementation of the aforementioned method, and repeated details will not be repeated. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.
[0081] Specifically, Figure 6 This is a schematic diagram of the module structure of one embodiment of a distributed GPU resource scheduling device provided in this specification. (Refer to...) Figure 6 As shown in the embodiments of this specification, a distributed GPU resource scheduling device includes: a task management agent and at least one resource management agent; The broadcast module 100 is used to broadcast its own GPU resource information at a certain frequency after the resource management agent starts up. The parsing module 200 is used for the task management agent to accept tasks, parse the tasks to generate standardized GPU resource requirements, and publish the GPU resource requirements to each of the resource management agents. Decision module 300 is used for each of the resource management intelligent agents to receive the GPU resource requirements, analyze its own GPU resource information based on the resource requirements, and decide whether the task is acceptable. The calculation module 400 is configured to, when the task is acceptable, each of the resource management agents calculates the cost of executing the task and sends the cost to the task management agent; Selection module 500 is used for the task management agent to receive the costs of each of the resource management agents, and to select a target resource management agent to execute the task based on the costs.
[0082] Reference Figure 7 As shown, based on the distributed GPU resource scheduling method described above, one embodiment of this specification also provides a computer device 702, wherein the above method runs on the computer device 702. The computer device 702 may include one or more processors 704, such as one or more central processing units (CPUs) or graphics processing units (GPUs), each processing unit may implement one or more hardware threads. The computer device 702 may also include any memory 706 for storing any kind of information such as code, settings, data, etc. In one specific embodiment, a computer program is stored on the memory 706 and can run on the processor 704. When the computer program is run by the processor 704, it can execute instructions according to the above method. Non-limitingly, for example, the memory 706 may include any type of RAM, any type of ROM, flash memory device, hard disk, optical disk, etc. More generally, any memory can use any technology to store information. Further, any memory can provide volatile or non-volatile retention of information. Further, any memory can represent a fixed or removable component of the computer device 702. In one scenario, when processor 704 executes associated instructions stored in any memory or combination of memories, computer device 702 can perform any operation of the associated instructions. Computer device 702 also includes one or more drive mechanisms 708 for interacting with any memory, such as hard disk drive mechanisms, optical disk drive mechanisms, etc.
[0083] Computer device 702 may also include an input / output module 710 (I / O) for receiving various inputs (via input device 712) and providing various outputs (via output device 714). A specific output mechanism may include a presentation device 716 and an associated graphical user interface 718 (GUI). In other embodiments, the input / output module 710 (I / O), input device 712, and output device 714 may be omitted, and the device may function solely as a computer device within a network. Computer device 702 may also include one or more network interfaces 720 for exchanging data with other devices via one or more communication links 722. One or more communication buses 724 couple the components described above together.
[0084] Communication link 722 can be implemented in any way, such as via a local area network, a wide area network (e.g., the Internet), a point-to-point connection, or any combination thereof. Communication link 722 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.
[0085] Corresponding to Figures 1-5 In addition to the methods described above, embodiments of this specification also provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the methods described above.
[0086] This specification also provides computer-readable instructions, wherein when a processor executes the instructions, the program therein causes the processor to perform the following... Figures 1 to 5 The method shown.
[0087] This specification also provides a computer program product, which, when executed by the processor of a computer device, performs the following... Figures 1 to 5 The method shown.
[0088] The computer program product described in this specification is a software product that mainly implements the methods described in this specification through a computer program.
[0089] It should be understood that in the various embodiments of this specification, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this specification.
[0090] It should also be understood that, in the embodiments of this specification, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the embodiments of this specification, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.
[0091] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this specification can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the embodiments in this specification.
[0092] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0093] In the several embodiments provided in this specification, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, devices, or units, or they may be electrical, mechanical, or other forms of connection.
[0094] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments described in this specification, depending on actual needs.
[0095] Furthermore, the functional units in the various embodiments of this specification can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0096] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this specification, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this specification. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0097] This specification uses specific embodiments to illustrate the principles and implementation methods of the embodiments. The above description of the embodiments is only for the purpose of helping to understand the methods and core ideas of the embodiments in this specification. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the embodiments in this specification. Therefore, the content of this specification should not be construed as a limitation on the embodiments in this specification.
Claims
1. A distributed GPU resource scheduling method, characterized in that, include: One task management agent and at least one resource management agent; After the resource management agent starts, it broadcasts its own GPU resource information at a certain frequency; The task management agent accepts the task, parses the task to generate standardized GPU resource requirements, and publishes the GPU resource requirements to each of the resource management agents. Each of the resource management agents receives the GPU resource requirements, analyzes its own GPU resource information based on the resource requirements, and decides whether the task is acceptable. When the task is acceptable, each of the resource management agents calculates the cost of executing the task and sends the cost to the task management agent. The task management agent receives the costs from each of the resource management agents and selects a target resource management agent to execute the task based on the costs.
2. The distributed GPU resource scheduling method based on multi-agent systems according to claim 1, characterized in that, The GPU resource information includes: resource utilization, hardware status, task context information, topology links, and fault warning indicators.
3. The distributed GPU resource scheduling method based on multi-agent systems according to claim 1, characterized in that, The resource management agent receiving the GPU resource request, analyzing its own GPU resource information based on the resource request, and deciding whether the task is acceptable further includes: Each of the resource management agents receives the GPU resource requirements and determines whether its own free video memory meets the video memory requirements in the GPU resource requirements. If the conditions are met, the task can be accepted; If these conditions are not met, the task is unacceptable.
4. The distributed GPU resource scheduling method based on multi-agent systems according to claim 1, characterized in that, Each of the resource management agents calculates the cost of executing the task, and sending the cost to the task management agent further includes: Each resource management agent calculates the time cost of executing the task based on the computing power requirement in the GPU resource requirements and its own GPU utilization. Each resource management agent calculates the operating cost of executing the task based on the time requirement in the GPU resource requirements and its own unit operating cost. The cost of the task is obtained by combining the time cost and the operating cost.
5. The distributed GPU resource scheduling method based on multi-agent systems according to claim 1, characterized in that, The step of the resource management agent selecting the target resource based on the cost to perform the task further includes: Obtain the historical task completion status of each resource management agent; The credibility of each resource management agent is obtained based on the historical task completion status. Based on the cost of each resource management agent and its corresponding credibility, a target resource management agent is selected.
6. The distributed GPU resource scheduling method based on multi-agent systems according to claim 1, characterized in that, Also includes: When the target resource management agent executes the task, it monitors the GPU resource requirements of the task in real time and sends them to the task management agent. When the change in GPU resource requirements exceeds the resource requirement threshold, the task management agent selects another resource management agent to execute the task.
7. The distributed GPU resource scheduling method based on multi-agent technology according to claim 1, characterized in that, Also includes: When the target resource management agent executes the task, it monitors the GPU resource requirements of the task in real time and sends them to the task management agent. If the task management agent does not receive a GPU resource request after a preset time threshold is exceeded, the task management agent will select another resource management agent to execute the task.
8. A distributed GPU resource scheduling device, characterized in that, include: One task management agent and at least one resource management agent; The broadcast module is used to broadcast the GPU resource information of the resource management agent at a certain frequency after it starts up. The parsing module is used for the task management agent to receive tasks, parse the tasks to generate standardized GPU resource requirements, and publish the GPU resource requirements to each of the resource management agents. The decision module is used by each of the resource management agents to receive the GPU resource requirements, analyze its own GPU resource information based on the resource requirements, and decide whether the task is acceptable. The calculation module is configured to, when the task is acceptable, each of the resource management agents calculates the cost of executing the task and sends the cost to the task management agent; The selection module is used by the task management agent to receive the costs of each resource management agent and select a target resource management agent to execute the task based on the costs.
9. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, When the computer program is run by the processor, it implements the steps of the method according to any one of claims 1-7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor of the computer device, it implements the steps of the method according to any one of claims 1-7.