Heterogeneous computing power model scheduling method and device, electronic equipment and storage medium

Abstract

The invention provides a heterogeneous computing power model scheduling method and device, electronic equipment and a storage medium, and relates to the technical field of heterogeneous computing power resource dynamic scheduling. The heterogeneous computing resource demand parameter can be automatically analyzed, the real-time heterogeneous resource state of each computing node in the cluster is obtained, and the target computing node meeting the demand is accurately determined through the preset multi-dimensional resource matching strategy. Then, a containerized task instance is automatically started on the target node, heterogeneous computing resources are distributed according to requirements, the corresponding model mirror image is loaded, and the execution instruction is executed. The problems that in the prior art, due to the fact that a model mirror image management system and a scheduling system are separated, a scheduling strategy cannot sense the heterogeneous computing power requirement of an AI model, task definition lacks a reusable template and is tedious in operation, nodes are appointed manually, efficiency is low, errors are prone to occurring, and the resource utilization rate is low can be solved.

Description

Technical Field

[0001] This disclosure relates to the field of dynamic scheduling technology for heterogeneous computing resources, and in particular to a method and apparatus for scheduling heterogeneous computing power models, electronic equipment and storage medium. Background Technology

[0002] In cloud-native architectures, AI model deployment and resource scheduling are core technologies for enterprise-level AI platforms. These rely on components like Kubernetes and Harbor image repositories, covering the entire process from image building and containerization to resource scheduling. However, as the scale and complexity of AI models increase, existing technologies employ a loosely coupled architecture of a general-purpose scheduler and an independent image repository, resulting in several systemic flaws: First, the general-purpose scheduler treats GPUs as homogeneous resources, lacking fine-grained awareness of heterogeneous computing power characteristics such as different models and memory specifications, making it impossible to accurately match model requirements; second, the model image repository and scheduling system are disconnected, failing to form a closed-loop management system, making image invocation inconvenient; third, task deployment requires writing complex YAML configuration files, lacking standardized templates for reuse, resulting in a high operational threshold; and fourth, weak operation and maintenance monitoring capabilities, lacking a unified view, and making problem localization difficult. These flaws lead to severe resource fragmentation and mismatched allocation, necessitating manual node scheduling, which not only consumes significant operational manpower but also easily leads to scheduling errors and persistently high resource idle rates, severely restricting the large-scale deployment of AI platforms in hybrid GPU cluster scenarios. Summary of the Invention

[0003] This disclosure provides a method, apparatus, electronic device, and storage medium for scheduling heterogeneous computing power models. Its main purpose is to at least partially solve one of the technical problems in related technologies.

[0004] According to a first aspect of this disclosure, a heterogeneous computing power model scheduling method is provided, comprising: Create reusable task templates that include model identifiers, execution instructions, and heterogeneous computing resource requirements. In response to the task start command, the heterogeneous computing resource requirement parameters in the task template are parsed, and the real-time heterogeneous resource status of each computing node in the cluster is obtained. Based on the resource requirement parameters and the real-time heterogeneous resource status, a target computing node that meets the requirements is determined through a pre-set multi-dimensional resource matching strategy. Launch a containerized task instance on the target computing node, allocate heterogeneous computing resources according to the resource requirement parameters, load the corresponding model image, and execute the execution instructions.

[0005] Optionally, the creation of a reusable task template containing model identifiers, execution instructions, and heterogeneous computing resource requirement parameters includes: The system receives user input of model image reference information, container startup commands, and resource requirement configurations through a graphical interface. The resource requirements configuration includes one or more of the following: graphics processor model, number of graphics processors, video memory capacity, and driver version requirements.

[0006] Optionally, obtaining the real-time heterogeneous resource status of each computing node in the cluster includes: Periodically query the cluster management system for available resource information for each node; The available resource information includes the currently available graphics processor models, remaining video memory, and driver compatibility status.

[0007] Optionally, determining the target computing node that meets the requirements through a pre-defined multi-dimensional resource matching strategy includes: Calculate the matching score between each node and the resource requirement parameters, the matching score being based on at least one dimension of graphics processor model consistency and video memory sufficiency; Nodes are selected as target calculation nodes based on their matching scores, from highest to lowest.

[0008] Optionally, launching a containerized task instance on the target computing node includes: Send a resource request to the cluster management system, explicitly stating the required graphics processor model and quantity; After successful resource allocation, the image file corresponding to the model identifier is pulled from the integrated image repository, and the container startup process is executed.

[0009] Optional, also includes: Collect and visualize the running status, resource usage data, and log information of the containerized task instance; The operational status includes scheduling node information, resource allocation results, and model execution phase logs.

[0010] According to a second aspect of this disclosure, a heterogeneous computing power model scheduling device is provided, comprising: The creation unit is used to create reusable task templates that include model identifiers, execution instructions, and heterogeneous computing resource requirements. The parsing unit is used to respond to the task start command, parse the heterogeneous computing resource requirement parameters in the task template, and obtain the real-time heterogeneous resource status of each computing node in the cluster. The determining unit is used to determine the target computing node that meets the requirements based on the resource requirement parameters and the real-time heterogeneous resource status, through a preset multi-dimensional resource matching strategy. The loading unit is used to launch a containerized task instance on the target computing node, allocate heterogeneous computing resources according to the resource requirement parameters, load the corresponding model image, and execute the execution instructions.

[0011] Optionally, creating a cell can also be used for: The system receives user input of model image reference information, container startup commands, and resource requirement configurations through a graphical interface. The resource requirements configuration includes one or more of the following: graphics processor model, number of graphics processors, video memory capacity, and driver version requirements.

[0012] Optionally, the parsing unit is also used for: Periodically query the cluster management system for available resource information for each node; The available resource information includes the currently available graphics processor models, remaining video memory, and driver compatibility status.

[0013] Optionally, the determining unit is also used for: Calculate the matching score between each node and the resource requirement parameters, the matching score being based on at least one dimension of graphics processor model consistency and video memory sufficiency; Nodes are selected as target calculation nodes based on their matching scores, from highest to lowest.

[0014] Optionally, the loading unit is also used for: Send a resource request to the cluster management system, explicitly stating the required graphics processor model and quantity; After successful resource allocation, the image file corresponding to the model identifier is pulled from the integrated image repository, and the container startup process is executed.

[0015] Optional, also includes: The collection unit is used to collect and visualize the running status, resource usage data, and log information of the containerized task instance; The operational status includes scheduling node information, resource allocation results, and model execution phase logs.

[0016] According to a third aspect of this disclosure, an electronic device is provided, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect above.

[0017] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method described in the first aspect above.

[0018] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described in the first aspect above.

[0019] The heterogeneous computing power model scheduling method, apparatus, electronic device, and storage medium disclosed herein, by creating a reusable task template containing model identifiers, execution instructions, and heterogeneous computing resource requirement parameters, can automatically parse the heterogeneous computing resource requirement parameters, obtain the real-time heterogeneous resource status of each computing node in the cluster, and accurately determine the target computing node that meets the requirements through a pre-set multi-dimensional resource matching strategy. Then, on the target node, a containerized task instance is automatically started, heterogeneous computing resources are allocated according to requirements, the corresponding model image is loaded, and the execution instructions are executed. Therefore, it can solve the problems in the prior art where the model image management and scheduling system are disconnected, the scheduling strategy cannot perceive the heterogeneous computing power requirements of the AI ​​model, and the task definition lacks reusable templates and is cumbersome to operate, resulting in reliance on manual node specification, low efficiency, easy errors, and low resource utilization. It achieves the technical effects of standardizing and reusing task definitions, accurately and automatically matching heterogeneous computing power resources, and creating a collaborative closed loop of model scheduling and image use, thereby improving task deployment efficiency and heterogeneous computing resource utilization, reducing human operation errors, and lowering the user threshold.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0021] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 A flowchart illustrating a heterogeneous computing power model scheduling method provided in an embodiment of this disclosure; Figure 2 This is a schematic diagram of the structure of a heterogeneous computing power model scheduling device provided in an embodiment of the present disclosure; Figure 3 A schematic block diagram of an example electronic device provided for embodiments of this disclosure. Detailed Implementation

[0022] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0023] The following description, with reference to the accompanying drawings, outlines a heterogeneous computing power model scheduling method, apparatus, electronic device, and storage medium according to embodiments of the present disclosure.

[0024] Figure 1 This is a flowchart illustrating a heterogeneous computing power model scheduling method provided in an embodiment of this disclosure.

[0025] like Figure 1 As shown, the method includes the following steps: Step 101: Create a reusable task template that includes model identifiers, execution instructions, and heterogeneous computing resource requirements.

[0026] In the embodiments of this disclosure, a reusable task template is created. This template standardizes and encapsulates model identifiers, execution instructions, and heterogeneous computing resource requirement parameters. The model identifier uniquely associates with the target algorithm model, the execution instructions provide the startup control information required for model operation, and the heterogeneous computing resource requirement parameters (i.e., specific requirements for different types of computing resources, such as computing unit type, quantity, and storage capacity) specify the heterogeneous computing power configuration required for model operation. This task template unifies and solidifies the core configurations related to model deployment, avoiding redundant definitions and enabling cross-scenario reuse. As one implementation method, users can select an integrated model image through the platform's interactive interface to determine the model identifier, configure container startup-related instructions as execution instructions, and specify heterogeneous computing resource requirement parameters such as GPU type, number of GPUs, number of CPU cores, and memory size to complete the creation of the task template.

[0027] This technology effectively solves the problems of cumbersome task definition, non-reusable configuration, and high learning cost in existing technologies. It standardizes model deployment configuration through templated encapsulation, reduces the user's operational threshold, and provides a unified and clear input basis for subsequent accurate resource matching and scheduling execution, thereby improving the efficiency and accuracy of task deployment.

[0028] Step 102: In response to the task start command, parse the heterogeneous computing resource requirement parameters in the task template and obtain the real-time heterogeneous resource status of each computing node in the cluster.

[0029] In the embodiments of this disclosure, when a task start command is received, the system first parses the heterogeneous computing resource requirement parameters (i.e., the various heterogeneous computing power configuration requirements required for model operation, such as specific types of computing units, quantities, storage resources, etc.) encapsulated in the aforementioned reusable task template to clarify the specific requirements of the model operation for heterogeneous computing power. Simultaneously, the system interacts with the cluster management architecture to collect the current heterogeneous resource status of all computing nodes in the cluster in real time, including key information such as the type, quantity, and availability of heterogeneous computing resources occupied and idle by each node, forming a complete node resource status dataset to provide accurate and real-time data support for subsequent resource matching. As one implementation method, the system can parse specific requirement parameters such as GPU type, number of GPUs, number of CPU cores, and memory size in the template, and query the real-time status information of each node, including GPU model and idle quantity, and available CPU and memory resources, through the interaction interface with the Kubernetes cluster.

[0030] Existing technologies suffer from the problem that scheduling systems cannot perceive the heterogeneous computing power requirements of models and the real-time resource status of nodes. By accurately analyzing the requirements and acquiring node resource data in real time, a reliable data foundation is provided for subsequent resource matching, avoiding scheduling errors caused by information asymmetry. This lays the foundation for achieving accurate matching of "model-resources" and improves the scientific nature and timeliness of scheduling decisions.

[0031] Step 103: Based on the resource requirement parameters and the real-time heterogeneous resource status, determine the target computing node that meets the requirements through a preset multi-dimensional resource matching strategy.

[0032] In the embodiments of this disclosure, the system performs intelligent matching analysis based on the parsed heterogeneous computing resource requirement parameters and the acquired real-time heterogeneous resource status of each computing node in the cluster, using a pre-set multi-dimensional resource matching strategy. This multi-dimensional resource matching strategy can cover core matching dimensions such as resource type adaptability, resource quantity sufficiency, and node load balancing. By verifying the multi-dimensional correspondence between requirements and status, the system selects target computing nodes from the cluster nodes that can fully meet the heterogeneous computing power requirements and have optimal operating conditions. As one implementation method, the multi-dimensional resource matching strategy can prioritize the accurate matching of heterogeneous computing resource types, and then combine the number of idle resources on the node and the current load situation for comprehensive judgment. For example, for a specific GPU type and quantity requirement specified in the template, nodes with corresponding idle GPU resources and a reasonable load range are matched from the cluster nodes as target nodes.

[0033] Existing general-purpose schedulers cannot perceive heterogeneous computing power requirements and are difficult to achieve accurate matching. By using a multi-dimensional matching strategy, intelligent adaptation of resource requirements and node status is achieved. The optimal target node can be determined without manual intervention, which not only improves the accuracy and efficiency of scheduling decisions, but also effectively avoids resource waste and improves the overall utilization rate of heterogeneous computing resources in the cluster.

[0034] Step 104: Start a containerized task instance on the target computing node, allocate heterogeneous computing resources according to the resource requirement parameters, load the corresponding model image and execute the execution instructions.

[0035] In the embodiments of this disclosure, after determining the target computing node that meets the heterogeneous computing resource requirements, the system starts a containerized task instance (i.e., a task execution unit encapsulated based on container technology) on that node. It then strictly applies to and allocates corresponding heterogeneous computing resources from the cluster management architecture according to the previously analyzed heterogeneous computing resource requirement parameters (i.e., the various heterogeneous computing power configurations required by the model, such as specific computing unit types and quantities, storage capacity, etc.), ensuring a precise match between resource supply and task requirements. Simultaneously, based on the model identifier in the task template, the system associates the corresponding model image, loads the model image through collaborative interaction with the image storage management system, and executes the preset execution instructions in the template, driving the model to run orderly in the allocated resource environment. As one implementation method, the system can create a Pod as a containerized task instance on the target node of the Kubernetes cluster, allocate resources according to the GPU type and quantity, CPU core count, and memory size specified in the template, pull the corresponding model image from the integrated image repository, and execute preset container startup instructions to complete model initialization and operation.

[0036] Existing technologies suffer from problems such as fragmented model image management and scheduling systems, mismatch between resource allocation and task requirements, and cumbersome task execution processes. The embodiments disclosed herein can achieve on-demand and precise allocation of heterogeneous computing resources and efficient collaborative invocation of model images, ensuring stable execution of tasks in an appropriate resource environment. At the same time, a complete closed loop is formed from resource allocation and image loading to task execution, improving the efficiency and reliability of model deployment and execution.

[0037] The heterogeneous computing power model scheduling method disclosed herein creates a reusable task template containing model identifiers, execution instructions, and heterogeneous computing resource requirement parameters. This template automatically parses the heterogeneous computing resource requirement parameters, obtains the real-time heterogeneous resource status of each computing node in the cluster, and accurately determines the target computing node that meets the requirements through a pre-set multi-dimensional resource matching strategy. Then, it automatically starts a containerized task instance on the target node, allocates heterogeneous computing resources according to requirements, loads the corresponding model image, and executes the execution instructions. Therefore, it solves the problems in existing technologies where model image management and scheduling systems are fragmented, scheduling strategies cannot perceive the heterogeneous computing power requirements of AI models, and task definitions lack reusable templates and are cumbersome to operate, leading to reliance on manual node designation, low efficiency, error-proneness, and low resource utilization. This method achieves the technical effects of standardizing and reusing task definitions, accurately and automatically matching heterogeneous computing resources, and creating a collaborative closed loop between model scheduling and image usage. It improves task deployment efficiency and heterogeneous computing resource utilization, reduces human error, and lowers the user threshold.

[0038] As a specific implementation of this disclosure, based on the basic solution, the creation of a reusable task template containing model identifiers, execution instructions, and heterogeneous computing resource requirements parameters is further defined as follows: receiving model image reference information, container startup commands, and resource requirement configurations input by the user through a graphical interface; wherein, the resource requirement configurations include one or more of the following: graphics processor model, number of graphics processors, video memory capacity, and driver version requirements.

[0039] Specifically, the creation of reusable task templates is implemented through the platform's built-in graphical user interface (i.e., web interface). This interface has a pre-set standardized input module to receive user configuration information: the model image reference information module establishes data linkage with the integrated Harbor private image repository, and the interface displays the names and unique identifiers of all model images stored in the repository in a drop-down list. Users complete the input of reference information by selecting the target image; the container startup command module provides a text input box, allowing users to directly write or paste the container startup commands required for model operation (such as image initialization commands, inference service port configuration commands, etc.), and has a built-in syntax verification function to ensure that the command format is compliant; the resource requirement configuration module is further subdivided into several precise configuration items, including a graphics processor (GPU) model selection box (providing mainstream model options such as A100, V100, T4, etc.), a GPU quantity numeric input device (supporting integer configuration from 1 to 8), a video memory capacity configuration item (in GB, supporting input of specific values ​​according to model requirements), and a driver version drop-down list (listing driver versions compatible with each GPU model). Users can select or fill in one or more of these configurations according to the actual running requirements of the model. After all the user-input information is verified by the interface backend program, it will be automatically encapsulated into a standardized data structure and stored as a reusable task template. The template data will establish a data interaction channel with the platform scheduling core layer to support subsequent scheduling execution.

[0040] The graphical user interface with precise input modules completely eliminates the need for complex YAML configuration file writing in existing technologies, lowering the technical threshold for task definition. At the same time, preset options and verification mechanisms ensure the accuracy of model image references, the validity of container startup commands, and the precision of GPU-related resource requirements configuration, effectively avoiding human configuration errors and further improving the standardization and reusability of task templates.

[0041] As a specific implementation of this disclosure, based on the basic scheme, the method of obtaining the real-time heterogeneous resource status of each computing node in the cluster is further defined as follows: periodically querying the cluster management system for the available resource information of each node; the available resource information includes the currently available graphics processor model, the remaining amount of video memory, and the driver compatibility status.

[0042] Specifically, the process of obtaining the real-time heterogeneous resource status of each compute node in the cluster is achieved through periodic interaction between the scheduling core layer and the cluster management system, which uses a Kubernetes cluster managed by Rancher. The scheduling core layer has a built-in scheduled task module with a preset fixed query period (e.g., 30 seconds / time). This module triggers Kubernetes API calls periodically, initiating resource query commands to the cluster management system. During the query process, all available compute nodes in the cluster are first located using the node tag filtering function. Then, target resource information is collected for each node: the currently available graphics processor (GPU) model is obtained by reading the identifier field of the node's hardware configuration; the total and used video memory capacity are collected by calling the GPU device's status monitoring interface to calculate the remaining video memory; and the GPU driver version information in the node's operating system is queried and compared with a preset driver version database to determine the driver compatibility status (compatible / incompatible). All collected node available resource information is updated in real-time to the scheduling core's local resource status cache, forming a structured node resource dataset to provide the latest data support for subsequent resource matching.

[0043] The periodic query mechanism ensures the real-time status of heterogeneous resources on cluster nodes, avoiding scheduling misjudgments caused by lagging resource information. It accurately collects three core types of information: GPU model, remaining video memory, and driver compatibility status, providing comprehensive and accurate data for subsequent multi-dimensional resource matching. This further improves the accuracy of "model-resource" matching and reduces task execution failures caused by driver incompatibility or insufficient video memory.

[0044] As a specific implementation of this disclosure, based on the basic scheme, the step of determining the target computing node that meets the requirements through a pre-set multi-dimensional resource matching strategy is further defined as follows: calculating the matching degree score between each node and the resource requirement parameters, wherein the matching degree score is based on at least one dimension, namely, graphics processor model consistency and video memory sufficiency; and selecting nodes as the target computing nodes according to the matching degree score from high to low.

[0045] Specifically, the multi-dimensional resource matching strategy is implemented through a scoring algorithm built into the scheduling core layer. This algorithm uses GPU model consistency and memory sufficiency as the core evaluation dimensions, with pre-configured weights for each dimension (e.g., GPU model consistency accounts for 60%, and memory sufficiency accounts for 40%). When calculating the matching score, firstly, for the GPU model consistency dimension: if the node's currently available GPU model is completely consistent with the model required in the task template (e.g., the requirement is for an A100 GPU, and the node's available GPU is an A100), then this dimension receives full marks; if the models are compatible but not completely consistent (e.g., the requirement is for an A100, and the node has an A10), then the score is calculated using a preset compatibility coefficient (e.g., 0.5); if the models are incompatible, this dimension receives 0 marks. Secondly, for the memory sufficiency dimension: the real-time remaining memory capacity of the node's GPU is obtained through the Kubernetes API, the ratio of remaining memory to required memory is calculated (e.g., if the required memory is 24GB and the node has 36GB remaining, then the ratio is 1.5), and then this ratio is multiplied by the weight of the memory sufficiency dimension to obtain the score for this dimension. Subsequently, the scheduling core adds the scores of the two dimensions to obtain the total matching score of each node. At the same time, it filters out nodes with a total score lower than a preset threshold (such as 60 points), sorts the remaining nodes from high to low according to the total matching score, and selects the node with the highest score as the target computing node. If there are multiple nodes with the same total score, the CPU idle rate of the nodes is further compared as a supplementary judgment condition to finally determine the unique target node.

[0046] The quantitative matching score mechanism makes the node selection process more objective and accurate, avoiding resource mismatch caused by subjective judgment. Combining the core dimensions of GPU model consistency and sufficient video memory, it ensures that the target node can fully adapt to the heterogeneous computing power requirements of the model, significantly improving the stability of task execution. At the same time, the matching strategy can be flexibly adjusted through weight configuration, further optimizing the utilization efficiency of heterogeneous resources in the cluster.

[0047] As a specific implementation of this disclosure, based on the basic scheme, the step of launching a containerized task instance on the target computing node is further defined as follows: sending a resource request to the cluster management system, wherein the required graphics processor model and quantity are clearly stated; after the resource allocation is successful, pulling the image file corresponding to the model identifier from the integrated image repository and executing the container startup process.

[0048] Specifically, the process of launching containerized task instances on the target compute node is achieved through the collaborative interaction of the scheduling core layer, the cluster management system, and the integrated image repository. The cluster management system is a Kubernetes cluster managed by Rancher, and the integrated image repository is the Harbor private image repository. The scheduling core layer first generates a standardized resource request based on the GPU model and quantity requirements obtained from the task template. This request explicitly states the specific model of the target GPU (e.g., A100, V100), the required quantity, and the associated task identifier information. This request is sent to the cluster management system via the Kubernetes API. Upon receiving the request, the cluster management system locks and allocates the GPU resources on the target compute node. After completing the resource reservation, it returns a successful resource allocation response to the scheduling core layer. Upon receiving the response, the scheduling core layer immediately initiates an image pull request to the Harbor private image repository through a pre-configured communication link, based on the model identifier in the task template. This request carries a unique identifier for the image corresponding to the model identifier, ensuring that the pulled image file accurately matches the task requirements. After the image is pulled, the scheduling core layer sends a container start command to the target node. The driver node creates a Pod instance that conforms to the Kubernetes specification, loads the pulled image file into the Pod, and executes the container start command preset in the task template to complete the model initialization and inference service ready process.

[0049] By specifying the GPU model and quantity in the resource application, the accurate allocation of heterogeneous computing resources is ensured, avoiding the problems of resource mismatch or redundant allocation. Relying on direct interaction with the integrated image repository, efficient retrieval and accurate invocation of model images are realized, forming a collaborative closed loop of resource allocation and image loading, which significantly improves the startup efficiency and running stability of containerized task instances.

[0050] As a specific implementation of this disclosure, in addition to the basic solution, it further includes: collecting and visualizing the running status, resource usage data and log information of the containerized task instance; wherein, the running status includes scheduling node information, resource allocation results and model execution phase logs.

[0051] Specifically, the monitoring and visualization of containerized task instances are achieved through the platform's built-in operation and maintenance monitoring component. This component establishes a real-time data interaction channel with the cluster management system (i.e., the Kubernetes cluster managed by Rancher) and the containerized task instances (i.e., Pods). The operation and maintenance monitoring component continuously collects Pod runtime status data through the Kubernetes API. This includes scheduling node information such as the unique identifier, IP address, and node type of the target node; resource allocation results such as the actual allocated GPU model and quantity, CPU core usage, memory usage, and GPU memory consumption; and model execution phase logs captured by the container log collection module, covering complete output content such as model initialization status, inference task execution progress, and error messages. Simultaneously, the component collects dynamic resource usage data at preset intervals (e.g., every 5 seconds) to form a time-series dataset. After all collected data is standardized, it is synchronized to the monitoring and display module of the platform's web interface. This module presents the data in a multi-dimensional visualization format: scheduling node information and resource allocation results are displayed intuitively through dashboards and list components; resource usage data is presented in the form of line graphs to show real-time trends; and model execution phase logs are displayed in a scrollable text panel, supporting filtering and querying by timestamp.

[0052] The end-to-end data visualization of containerized task instances, from scheduling to execution, allows operations and maintenance personnel to intuitively grasp the task running status and resource usage. This completely solves the problems of weak operation and maintenance monitoring capabilities and difficulty in problem localization in existing technologies, significantly improves the efficiency of fault diagnosis and the transparency of platform operation and maintenance, and provides a strong guarantee for the reliability of task execution.

[0053] It should be noted that the embodiments of this disclosure may include multiple steps. For ease of description, these steps are numbered, but these numbers are not a limitation on the execution time slots or execution order between the steps; these steps can be implemented in any order, and the embodiments of this disclosure do not limit this.

[0054] Corresponding to the heterogeneous computing power model scheduling method described above, this disclosure also proposes a heterogeneous computing power model scheduling device. Since the device embodiments of this disclosure correspond to the method embodiments described above, details not disclosed in the device embodiments can be referred to the method embodiments described above, and will not be repeated here.

[0055] Figure 2 This is a schematic diagram of the structure of a heterogeneous computing power model scheduling device provided in an embodiment of this disclosure, as shown below. Figure 2 As shown, it includes: Create unit 21, which is used to create a reusable task template containing model identifiers, execution instructions and heterogeneous computing resource requirements parameters; The parsing unit 22 is used to parse the heterogeneous computing resource requirement parameters in the task template in response to the task start command, and obtain the real-time heterogeneous resource status of each computing node in the cluster. Determining unit 23 is used to determine the target computing node that meets the requirements based on the resource requirement parameters and the real-time heterogeneous resource status, through a preset multi-dimensional resource matching strategy; The loading unit 24 is used to start a containerized task instance on the target computing node, allocate heterogeneous computing resources according to the resource requirement parameters, load the corresponding model image and execute the execution instructions.

[0056] The heterogeneous computing power model scheduling device disclosed herein creates a reusable task template containing model identifiers, execution instructions, and heterogeneous computing resource requirement parameters. It can automatically parse these parameters, obtain the real-time heterogeneous resource status of each computing node in the cluster, and accurately determine the target computing node that meets the requirements through a pre-set multi-dimensional resource matching strategy. Then, it automatically starts a containerized task instance on the target node, allocates heterogeneous computing resources according to requirements, loads the corresponding model image, and executes the execution instructions. Therefore, it solves the problems in existing technologies where model image management and scheduling systems are fragmented, scheduling strategies cannot perceive the heterogeneous computing power requirements of AI models, and task definitions lack reusable templates and are cumbersome to operate, leading to reliance on manual node designation, low efficiency, error-proneness, and low resource utilization. This achieves the technical effects of standardizing and reusing task definitions, accurately and automatically matching heterogeneous computing resources, and creating a collaborative closed loop between model scheduling and image usage. It improves task deployment efficiency and heterogeneous computing resource utilization, reduces human error, and lowers the user threshold.

[0057] Furthermore, in one possible implementation of this embodiment, the creation unit 21 is also used for: The system receives user input of model image reference information, container startup commands, and resource requirement configurations through a graphical interface. The resource requirements configuration includes one or more of the following: graphics processor model, number of graphics processors, video memory capacity, and driver version requirements.

[0058] Furthermore, in one possible implementation of this embodiment, the parsing unit 22 is also used for: Periodically query the cluster management system for available resource information for each node; The available resource information includes the currently available graphics processor models, remaining video memory, and driver compatibility status.

[0059] Furthermore, in one possible implementation of this embodiment, the determining unit 23 is further configured to: Calculate the matching score between each node and the resource requirement parameters, the matching score being based on at least one dimension of graphics processor model consistency and video memory sufficiency; Nodes are selected as target calculation nodes based on their matching scores, from highest to lowest.

[0060] Furthermore, in one possible implementation of this embodiment, the loading unit 24 is also used for: Send a resource request to the cluster management system, explicitly stating the required graphics processor model and quantity; After successful resource allocation, the image file corresponding to the model identifier is pulled from the integrated image repository, and the container startup process is executed.

[0061] Furthermore, in one possible implementation of this embodiment, such as Figure 2 As shown, it also includes: The collection unit 25 is used to collect and visualize the running status, resource usage data and log information of the containerized task instance; The operational status includes scheduling node information, resource allocation results, and model execution phase logs.

[0062] It should be noted that the foregoing explanation of the method embodiments also applies to the apparatus of this embodiment, and the principle is the same, so it is not limited in this embodiment.

[0063] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0064] Figure 3 A schematic block diagram of an example electronic device 300 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0065] like Figure 3As shown, the electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes based on a computer program stored in ROM (Read-Only Memory) 302 or a computer program loaded from storage unit 308 into RAM (Random Access Memory) 303. The RAM 303 may also store various programs and data required for the operation of the electronic device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An I / O (Input / Output) interface 305 is also connected to the bus 304.

[0066] Multiple components in electronic device 300 are connected to I / O interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of displays, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows electronic device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0067] The computing unit 301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, CPUs (Central Processing Units), GPUs (Graphics Processing Units), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSPs (Digital Signal Processors), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as the heterogeneous computing power model scheduling method. For example, in some embodiments, the heterogeneous computing power model scheduling method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, computing unit 301 may be configured to perform the aforementioned heterogeneous computing power model scheduling method by any other suitable means (e.g., by means of firmware).

[0068] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System-on-Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0069] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0070] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0071] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0072] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include LANs (Local Area Networks), WANs (Wide Area Networks), the Internet, and blockchain networks.

[0073] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0074] It's important to note that artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0075] The various numerical designations such as "first," "second," etc., used in this disclosure are merely for ease of description and are not intended to limit the scope of the embodiments of this disclosure, nor do they indicate a sequential order.

[0076] At least one of the features described in this disclosure can also be described as one or more, and multiple features can be two, three, four or more, and this disclosure does not impose any limitations. In the embodiments of this disclosure, for a technical feature, the technical features in that technical feature are distinguished by "first", "second", "third", "A", "B", "C" and "D", etc., and there is no sequential order or size order among the technical features described by "first", "second", "third", "A", "B", "C" and "D".

[0077] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0078] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A scheduling method for a heterogeneous computing power model, characterized in that, include: Create reusable task templates that include model identifiers, execution instructions, and heterogeneous computing resource requirements. In response to the task start command, the heterogeneous computing resource requirement parameters in the task template are parsed, and the real-time heterogeneous resource status of each computing node in the cluster is obtained. Based on the resource requirement parameters and the real-time heterogeneous resource status, a target computing node that meets the requirements is determined through a pre-set multi-dimensional resource matching strategy. Launch a containerized task instance on the target computing node, allocate heterogeneous computing resources according to the resource requirement parameters, load the corresponding model image, and execute the execution instructions.

2. The method according to claim 1, characterized in that, The creation of a reusable task template, which includes model identifiers, execution instructions, and heterogeneous computing resource requirements, includes: The system receives user input of model image reference information, container startup commands, and resource requirement configurations through a graphical interface. The resource requirements configuration includes one or more of the following: graphics processor model, number of graphics processors, video memory capacity, and driver version requirements.

3. The method according to claim 1, characterized in that, The process of obtaining the real-time heterogeneous resource status of each computing node in the cluster includes: Periodically query the cluster management system for available resource information for each node; The available resource information includes the currently available graphics processor models, remaining video memory, and driver compatibility status.

4. The method according to claim 1, characterized in that, The process of determining the target computing node that meets the requirements through a pre-defined multi-dimensional resource matching strategy includes: Calculate the matching score between each node and the resource requirement parameters, the matching score being based on at least one dimension of graphics processor model consistency and video memory sufficiency; Nodes are selected as target calculation nodes based on their matching scores, from highest to lowest.

5. The method according to claim 1, characterized in that, Starting a containerized task instance on the target computing node includes: Send a resource request to the cluster management system, explicitly stating the required graphics processor model and quantity; After successful resource allocation, the image file corresponding to the model identifier is pulled from the integrated image repository, and the container startup process is executed.

6. The method according to claim 1, characterized in that, Also includes: Collect and visualize the running status, resource usage data, and log information of the containerized task instance; The operational status includes scheduling node information, resource allocation results, and model execution phase logs.

7. A heterogeneous computing power model scheduling device, characterized in that, include: The creation unit is used to create reusable task templates that include model identifiers, execution instructions, and heterogeneous computing resource requirements. The parsing unit is used to respond to the task start command, parse the heterogeneous computing resource requirement parameters in the task template, and obtain the real-time heterogeneous resource status of each computing node in the cluster. The determining unit is used to determine the target computing node that meets the requirements based on the resource requirement parameters and the real-time heterogeneous resource status, through a preset multi-dimensional resource matching strategy. The loading unit is used to launch a containerized task instance on the target computing node, allocate heterogeneous computing resources according to the resource requirement parameters, load the corresponding model image, and execute the execution instructions.

8. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1-6.

Images