Big model-based heterogeneous computing power intelligent scheduling and task migration method

Abstract

The application provides a heterogeneous computing power intelligent scheduling and task migration method based on a large model. The method mainly senses the computing power demand of a computing task and the real-time state of a cluster through intelligence, automatically decides the optimal execution device and node, and completes the automatic adaptation and deployment of a running environment and code, so as to realize intelligent scheduling of heterogeneous computing power and seamless migration of tasks, and improve the overall resource utilization.

Description

Technical Field

[0001] This application belongs to the field of Artificial Intelligence (AI) technology, and in particular relates to a method for intelligent scheduling and task migration of heterogeneous computing power based on a large model. Background Technology

[0002] With the development of artificial intelligence and high-performance computing, the demand for computing power is increasing day by day, and single-type computing devices can no longer meet the needs of diverse scenarios. At present, data centers generally deploy heterogeneous computing devices, including graphics processing units (GPUs), neural processing units (NPUs), tensor processing units (TPUs), and various central processing unit (CPU) architectures, in order to provide more flexible and powerful integrated computing power.

[0003] In related technologies, mainstream task scheduling and management schemes mainly focus on the general resource status of computing nodes, such as the number of CPU cores and memory capacity, and allocate resources and place tasks based on this. However, in actual operation, such schemes are difficult to coordinate efficiently between different types of computing resources, resulting in limited overall resource utilization. Summary of the Invention

[0004] This application provides a method for intelligent scheduling and task migration of heterogeneous computing power based on a large model, which can realize intelligent scheduling of heterogeneous computing power and seamless task migration, thereby improving the overall resource utilization rate.

[0005] The first aspect of this application provides a method for intelligent scheduling and task migration of heterogeneous computing power based on a large model, comprising: acquiring a computing task containing a resource configuration file and source code; parsing the resource configuration file to obtain resource requirement information and analyzing the source code to obtain computing characteristic information; inputting the resource requirement information, computing characteristic information, and real-time collected heterogeneous computing power resource status information into a preset scheduling decision model, wherein the scheduling decision model outputs the target execution device type and the target computing node; generating a container image compatible with the target computing node based on the target execution device type and the target computing node; inputting the source code into a preset code adaptation large model, wherein the code adaptation large model outputs adapted code adapted to the target execution device type; and deploying and executing the computing task on the target computing node based on the container image and the adapted code.

[0006] In the technical solution of this application, the computing power requirements of the computing task and the real-time status of the cluster are intelligently perceived, and the optimal execution device and node are automatically determined. The operating environment and code are automatically adapted and deployed, thereby realizing intelligent scheduling of heterogeneous computing power and seamless migration of tasks, and improving the overall resource utilization rate.

[0007] Optionally, in one possible implementation of the first aspect, the analysis of the source code to obtain computational feature information includes: identifying the type of machine learning framework used in the source code; extracting key computational operators from the source code; determining the performance mapping relationship of key computational operators on various heterogeneous computing devices based on a historical performance database; and identifying the machine learning framework type, key computational operators, and performance mapping relationship as computational feature information. Thus, by identifying the machine learning framework type of the source code, extracting key computational operators, and determining their cross-hardware performance mapping relationship in conjunction with historical performance data, a refined and quantifiable basis for intelligent scheduling decisions is provided, enabling scheduling decisions to more accurately match task characteristics with hardware capabilities.

[0008] Optionally, in another possible implementation of the first aspect, the scheduling decision model makes computing power scheduling decisions based on a cost function. The cost function evaluates at least two of the following factors: task execution time predicted on different types of computing power devices based on computational characteristic information and historical performance databases; power consumption data of each candidate computing power device obtained in real-time from the cluster monitoring system; and the financial cost of using each candidate computing power device calculated according to a pre-designed pricing strategy. The scheduling decision model outputs the target execution device type and target computing node, including: the scheduling decision model outputs the computing power device type and computing node with the optimal cost function evaluation result as the target execution device type and target computing node. Thus, by enabling the scheduling decision model to make decisions based on a cost function that comprehensively considers multiple factors such as execution time, power consumption, and financial cost, and selecting the computing power device and node with the optimal evaluation result, multi-objective optimization of overall execution efficiency, energy consumption, and economic cost is achieved while meeting task requirements.

[0009] Optionally, in another possible implementation of the first aspect, generating a container image compatible with the target computing node based on the target execution device type and the target computing node includes: parsing the original container definition file associated with the computing task; determining the appropriate base image based on the target execution device type and the system architecture of the target computing node; replacing the base image reference in the original container definition file with the appropriate base image; modifying the container definition file after image replacement based on the target execution device type to inject the corresponding hardware driver dependency library and computing framework library into the container definition file after image replacement; and performing a build operation based on the modified container definition file to generate the container image. Thus, by parsing the original container definition file, replacing it with a base image compatible with the target architecture and device type, and automatically injecting the corresponding hardware driver and computing library dependencies, a container image compatible with the target computing node is automatically built, avoiding the tedious work of manually preparing and debugging the runtime environment for different hardware platforms.

[0010] Optionally, in another possible implementation of the first aspect, the above-mentioned input of source code into a preset code adaptation model, and the output of adapted code adapted to the target execution device type by the code adaptation model, includes: identifying hardware binding code segments related to the source execution device type from the source code; constructing prompt information to guide code conversion based on the target execution device type, wherein the prompt information includes the target execution device type, the hardware binding code segment to be converted, and the code conversion rules; inputting the prompt information into the code adaptation model to obtain the initial adaptation code generated by the code adaptation model; verifying the initial adaptation code, and determining the initial adaptation code as the adapted code after successful verification. Thus, by identifying hardware-related code segments from the source code, constructing prompt information containing the target device type and conversion rules to drive the large model to generate adaptation code, and verifying the initial code, automated and intelligent conversion and preliminary quality assurance from source code to target hardware code are achieved.

[0011] Optionally, in another possible implementation of the first aspect, the initial adaptation code is verified. If the verification passes, the initial adaptation code is determined as the adapted code. This includes: running simplified test cases of the initial adaptation code in a preset target architecture sandbox environment; if the simplified test cases fail, reconstructing the erroneous code segment and corresponding error message into a correction prompt, and re-inputting it into the code adaptation model for iterative correction; if, after a preset number of iterative corrections, the initial adaptation code still fails verification, determining that the current task migration has failed; triggering a task rollback process, resubmitting the computation task to the scheduling decision model, and excluding the target execution device type from the resource requirement information. Thus, by reconstructing the error message into a correction prompt and feeding it back to the model for iterative correction when verification fails, and triggering a task rollback and rescheduling process after multiple correction failures, a self-correcting and safety-tolerant mechanism is provided for the code adaptation process, enhancing the robustness of the entire migration system and the final success rate of the task.

[0012] Optionally, in another possible implementation of the first aspect, after deploying and executing the computing task on the target computing node based on the container image and adapted code, the method further includes: monitoring the performance indicators of the computing task during its execution, wherein the performance indicators include iteration speed and device power consumption; associating the performance indicators with task computing characteristic information and storing them in a historical performance database to update the historical performance database; and optimizing the performance prediction module in the scheduling decision model based on the updated historical performance database. Thus, by monitoring performance indicators during task execution and associating them with task characteristics to update the historical database for optimizing the prediction module of the scheduling model, a continuous self-learning and optimization closed loop is formed, enabling the system's scheduling decisions to continuously improve with the accumulation of operational experience, thereby continuously improving resource utilization efficiency.

[0013] Optionally, in another possible implementation of the first aspect, the aforementioned historical performance database stores performance metrics recorded from benchmark tests of various standard models and operators running on different hardware types. These performance metrics include floating-point operations per second, throughput, and latency.

[0014] Optionally, in another possible implementation of the first aspect, the aforementioned heterogeneous computing resource status information includes at least one of the following: the utilization rate of the central processing unit of each computing node, the memory utilization rate, the number of various heterogeneous acceleration cards occupied, the real-time power consumption of the node, and the node health status.

[0015] Optionally, in another possible implementation of the first aspect, after the above-mentioned computation task based on the container image and adapted code is deployed and executed on the target computing node, the method further includes: after the computation task is completed, generating a task execution report, which includes the scheduling decision basis, code adaptation change summary, resource consumption data and performance indicators. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a method for intelligent scheduling and task migration of heterogeneous computing power based on a large model, provided in an embodiment of this application.

[0018] Figure 2 This is a flowchart illustrating a method for intelligent scheduling and task migration of heterogeneous computing power based on a large model, as provided in an embodiment of this application. Detailed Implementation

[0019] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0020] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0021] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0022] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0023] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0024] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0025] It should be understood that the sequence number of each step in this embodiment does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0026] In related technologies, mainstream task scheduling and management schemes mainly focus on the general resource status of computing nodes, such as the number of CPU cores and memory capacity, and allocate resources and place tasks based on this. However, in actual operation, such schemes are difficult to coordinate efficiently between different types of computing resources, resulting in limited overall resource utilization.

[0027] In view of this, embodiments of this application provide a method for intelligent scheduling and task migration of heterogeneous computing power based on a large model. This method first acquires a computing task containing resource configuration files and source code; then, it parses the resource configuration files to obtain resource requirement information and analyzes the source code to obtain computing characteristic information; next, it inputs the resource requirement information, computing characteristic information, and real-time collected heterogeneous computing power resource status information into a preset scheduling decision model, which outputs the target execution device type and target computing node; further, based on the target execution device type and target computing node, it generates a container image compatible with the target computing node; then, it inputs the source code into a preset code adaptation large model, which outputs adapted code adapted to the target execution device type; finally, based on the container image and adapted code, it deploys and executes the computing task on the target computing node. Thus, by intelligently sensing the computing power requirements of the computing task and the real-time status of the cluster, it automatically decides the optimal execution device and node, and completes the automated adaptation and deployment of the runtime environment and code, thereby achieving intelligent scheduling and seamless task migration of heterogeneous computing power and improving overall resource utilization.

[0028] The following example illustrates the application scenario of the intelligent scheduling and task migration method based on large models for heterogeneous computing power provided in this application. Assume an AI R&D team is training a model in a cluster with GPUs and NPUs. In traditional solutions, when GPU resources are fully loaded, even if NPU nodes are idle, the team cannot automatically migrate training tasks to the NPU, resulting in resource idleness and task queuing. Using this application's embodiment, upon receiving a training task, its resource requirements and computational characteristics can be automatically analyzed. Based on the cluster's real-time status scheduling decision, an idle NPU is selected as the target. Subsequently, a container image suitable for the NPU architecture is automatically built, and GPU code is adapted to NPU code using a large model. Finally, training is seamlessly started on the NPU node. This allows previously idle NPU computing power to be utilized, shortens task waiting time, and eliminates the tedious work of developers manually configuring the environment and rewriting code, achieving intelligent scheduling and automated migration across heterogeneous computing power.

[0029] To illustrate the technical solution of this application, specific embodiments are described below.

[0030] Reference Figure 1 This diagram illustrates a flowchart of a heterogeneous computing power intelligent scheduling and task migration method based on a large model, provided in an embodiment of this application. This method can be executed by a task management system, or by a task migration device or electronic device as described in the following embodiments.

[0031] like Figure 1 As shown, this method for intelligent scheduling and task migration of heterogeneous computing power based on a large model may include the following steps: Step 101: Obtain the computing task containing resource configuration files and source code.

[0032] A computational task refers to a specific job that a user wishes to perform within a computing cluster, such as training a machine learning model, inference on large-scale data, or a scientific computing simulation. This computational task is submitted by the user through the system's Application Programming Interface (API) or web-based management interface. The process of retrieving computational tasks is typically handled by a task receiving service deployed on the cluster management node, which continuously listens for submission requests from users.

[0033] In this embodiment, the computing task submitted by the user mainly comprises two core components: a resource configuration file and source code. The resource configuration file is a structured text file used to declare the computing task's requirements for underlying hardware resources. The resource configuration file can be written using an easy-to-read and write markup language such as YAML (YAML Ain't Markup Language). In the resource configuration file, the user explicitly specifies the various resources required for the task to run and their quantities, such as the number of CPU cores, memory capacity, and the number of specific types of heterogeneous accelerator cards.

[0034] Specifically, for GPU resources, a request might be declared in the form of xxx.com / gpu:2; for NPU resources, a request might be declared in the form of a custom resource field such as xxx.com / npu:4. In addition to these explicit resource quantity declarations, resource configuration files can also contain tags or annotations describing task characteristics, such as indicating the task's preference for low latency or high throughput, task priority, or the project group it belongs to. This information can serve as auxiliary decision-making factors for subsequent intelligent scheduling.

[0035] In this embodiment, source code refers to a collection of program code files that implement the core logic of the computational task. Source code is typically hosted in a version control system (such as Git) repository. When submitting a task, the user needs to provide the corresponding repository address and a specific version identifier (e.g., Git Commit ID or branch name). The main form of source code can be Python scripts, C++ files, or files in other programming languages, and its core content is the program logic that implements a specific algorithm or model. The system obtains source code by cloning or pulling the user-specified repository version to the local staging area to ensure that the objects of subsequent analysis and processing are definite and consistent.

[0036] For example, a typical computational task might be training a natural language processing model based on the Transformer architecture. Its resource configuration file declares the need for four GPUs and 64GB of memory, while its source code repository contains model definition, data loading, and training loop scripts written using the PyTorch framework. After successfully obtaining the resource configuration file and source code, the task receiving service generates a unique task identifier for the computational task and passes its metadata and storage path to subsequent processing flows, thus completing this step.

[0037] Step 102: Parse the resource configuration file to obtain resource requirement information, and analyze the source code to obtain computational characteristic information.

[0038] The parsing of resource configuration files aims to transform user-declared resource requirements into quantifiable and comparable resource requirement information within the system. This process can be executed by the resource parsing engine. The resource parsing engine reads the resource configuration file submitted by the user and extracts the quantity requests for various types of computing devices according to predefined resource key rules. For example, for the key xxx.com / gpu, if the parsed quantity is 4, a resource requirement entry of "4 GPUs required" is generated.

[0039] In addition, the resource resolution engine can handle custom, non-standard resource declarations, such as the unique resource keys commonly used by computing cards. By querying an extensible resource key registry, the engine maps these custom declarations to a unified device type identifier within the system, thus achieving compatibility with the diverse and evolving hardware ecosystem. Simultaneously, CPU architecture requirements (such as x86_64, aarch64) and memory capacity requirements are also accurately extracted. Finally, all parsed resource requirement entries are integrated into structured resource requirement information for subsequent scheduling decisions.

[0040] In one embodiment, the steps of analyzing source code to obtain computational feature information may specifically include: identifying the type of machine learning framework used in the source code; extracting key computational operators from the source code; determining the performance mapping relationship of key computational operators on various heterogeneous computing devices based on a historical performance database; and identifying the machine learning framework type, key computational operators, and performance mapping relationship as computational feature information. Thus, by identifying the machine learning framework type of the source code, extracting key computational operators, and determining their cross-hardware performance mapping relationship in conjunction with historical performance data, a refined and quantifiable basis for computational feature analysis is provided for intelligent scheduling decisions, enabling scheduling decisions to more accurately match task characteristics with hardware capabilities.

[0041] It should be noted that the analysis of the source code aims to deeply understand the inherent characteristics and potential hardware dependencies of the computing tasks, thereby extracting computational feature information crucial for intelligent scheduling. This process can be completed by a feature analysis engine, which comprehensively utilizes techniques such as static code analysis, dependency resolution, and performance profiling. The feature analysis engine can use static code analysis tools (such as abstract syntax tree parsers) for preliminary analysis and query the operator-hardware performance mapping table maintained internally by the system to obtain performance relationships.

[0042] Specifically, the feature analysis engine first identifies the type of machine learning framework used in the source code. This is achieved by scanning key import statements in the code files; for example, identifying `import torch` indicates a PyTorch framework, and identifying `import tensorflow` indicates a TensorFlow framework. The framework type is fundamental to subsequent environment setup and code adaptation. The feature analysis engine not only identifies the main framework but also the specific domain libraries or high-level APIs used, such as transformers (Hugging Face library) or mmdetection (object detection library). This helps to more accurately predict task dependencies and behavioral patterns.

[0043] Secondly, the feature analysis engine extracts key computational operators from the source code. This is achieved by performing Abstract Syntax Tree (AST) analysis on the code and combining it with domain knowledge. It identifies frequently occurring or computationally intensive core operations in the model definition and computation graph, such as convolution, matrix multiplication, softmax and linear transformations in the attention mechanism, as well as user-defined CUDA or custom operator kernels. For example, for a Vision Transformer model, the feature analysis engine identifies its core operators as MultiHeadAttention and fully connected layers in the FeedForward Network.

[0044] Next, based on the historical performance database, the performance mapping relationship of key computational operators across various heterogeneous computing devices is determined. This historical performance database is a continuously accumulating database that stores performance metrics (such as floating-point operations per second, actual throughput, and latency) recorded when running various standard models or operator benchmark tests on different hardware types. The feature analysis engine matches and correlates the key operators extracted from the current task with the performance records in the database. For example, for a task identified as containing a large number of convolution operators, the system queries the database to obtain the relative computational efficiency ratio of this type of convolution on GPU A and NPU B. This performance mapping relationship is the core basis for quantitatively evaluating the expected execution efficiency of the task on different hardware.

[0045] Finally, the machine learning framework type, key computational operators, and performance mapping relationships are integrated and defined as structured computational feature information. Furthermore, the feature analysis engine precisely lists all Python dependency libraries and their versions by parsing the project's dependency declaration files (such as requirements.txt or pyproject.toml), especially libraries closely related to hardware acceleration, such as specific versions of torch (which may be associated with CUDA versions), tensorflow-gpu, cudatoolkit, or torch_npu for NPUs. This dependency list is used as an important component of the computational feature information to ensure the accuracy of subsequent environment construction. The resulting computational feature information allows the system to understand the task beyond simple resource quantity requirements, delving into its computational essence and cross-platform performance potential.

[0046] Step 103: Input the resource demand information, computing characteristic information and real-time collected heterogeneous computing power resource status information into the preset scheduling decision model. The scheduling decision model outputs the target execution device type and target computing node.

[0047] In this embodiment, the scheduling decision model is a software-based intelligent decision-making module. Its core function is to comprehensively analyze task requirements and real-time cluster status to select the optimal execution location for computing tasks. The input of this model mainly includes three parts of structured data: the first part is the resource requirement information parsed in step 102, which clarifies the quantity and type requirements of various hardware resources for the task; the second part is the computing characteristic information analyzed in step 102, which reveals the computing characteristics of the task and its performance potential on different hardware; the third part is the real-time heterogeneous computing power resource status information, which is continuously collected and reported to the central monitoring system by monitoring agents deployed on each node of the cluster. It reflects the dynamic status of all available computing nodes in the cluster at the current moment, including but not limited to the CPU utilization, memory utilization, the number of occupied and idle heterogeneous acceleration cards (such as GPU, NPU, TPU), real-time power consumption, network bandwidth, and node health status of each computing node.

[0048] The scheduling decision model can be triggered based on a preset queue length or waiting time threshold when initiating the decision-making process. When interpreting time predictions, in addition to matching historical data, lightweight performance profiling of the task (such as rapid profiling through mini-benchmarks) can be added to obtain more accurate predictions that better reflect the current task characteristics.

[0049] In one embodiment, the scheduling decision model makes computing power scheduling decisions based on a preset cost function. This cost function is a multi-objective optimization function used to quantitatively evaluate the comprehensive cost of scheduling the current task to a candidate computing power device and its corresponding computing node for execution.

[0050] Specifically, the cost function is designed to comprehensively evaluate at least two of the following key factors: First, the task execution time is predicted on different types of computing devices based on computational feature information and historical performance databases. The scheduling decision model uses the computational features extracted in the aforementioned embodiments (such as model structure and key operators) to match with performance benchmarks in the historical performance database to estimate the expected completion time of the task on various candidate hardware. For example, for a computational task identified as having a Transformer attention mechanism at its core, the model queries historical data on the training time records of similar models on GPUs and NPUs, and combines this with parameters such as the current task's batch size to generate a relatively accurate execution time prediction.

[0051] Secondly, real-time factors are comprehensively considered, including power consumption data of the nodes where each candidate computing power device is located, obtained in real time from the cluster monitoring system, and the financial cost of using each candidate computing power device calculated according to the pre-designed pricing strategy. Power consumption data is directly related to energy efficiency, and financial costs may be determined based on the unit-time rental fee of different hardware resources or internal cost accounting models. The cost function C can be expressed as C=f(estimated time, power consumption, financial cost). It can also be formally expressed as the following weighted combination: C=w1 T_predicted+w2 Power_current+w3 Cost_financial; Where T_predicted is the prediction time, Power_current is the real-time power consumption, Cost_financial is the financial cost, and w1, w2, and w3 are weight coefficients that can be dynamically adjusted according to the scheduling strategy. During evaluation, the scheduling decision model calculates the cost function value for each candidate device and node combination that meets basic resource requirements (such as a sufficient number of cards).

[0052] Furthermore, the process by which the scheduling decision model outputs the target execution device type and target computing node can specifically include: comparing the cost function evaluation results of all candidate device and node combinations, and outputting the computing device type and computing node with the optimal cost function evaluation result (usually the lowest overall cost value) as the target execution device type and target computing node. Thus, by enabling the scheduling decision model to make decisions based on a cost function that comprehensively considers multiple factors such as execution time, power consumption, and financial costs, and selecting the computing device and node with the optimal evaluation result, multi-objective optimization of overall execution efficiency, energy consumption, and economic costs can be achieved while meeting task requirements.

[0053] For example, in a scenario where GPU resources are scarce but NPU nodes are idle, even if the original task request is made by the GPU, the scheduling decision model may find after calculating the cost function that migrating the task to the NPU for execution, although the single iteration time is slightly longer (the prediction time T_predicted is higher), is actually less costly than waiting for GPU resources because there is no need to wait in line and the NPU has lower power consumption and financial cost per unit time. Therefore, the decision is to output a certain NPU as the target execution device type and specify a specific NPU node as the target computing node.

[0054] Through the above embodiments, the scheduling decision model has achieved a leap from simple resource surplus matching to intelligent decision-making that optimizes both overall efficiency and cost.

[0055] Step 104: Generate a container image compatible with the target compute node based on the target execution device type and the target compute node.

[0056] The container image generation process can be executed by a container build engine. Its core goal is to automatically create an isolated, consistent, and executable software environment containing all necessary dependencies for the computing tasks that will run on the target compute nodes. This process begins by parsing the raw container definition file associated with the computing task. The raw container definition file can be a Dockerfile that the user submits with the task or that is stored in a code repository. It defines a series of steps, including the base image for building the application runtime environment, system dependency installation instructions, application code copying, and startup commands.

[0057] In one embodiment, the container image generation process may specifically include: parsing the original container definition file associated with the computing task; determining the appropriate base image based on the target execution device type and the system architecture of the target computing node; replacing the base image reference in the original container definition file with the appropriate base image; modifying the container definition file after image replacement according to the target execution device type to inject the corresponding hardware driver dependency library and computing framework library into the replaced container definition file; and performing a build operation based on the modified container definition file to generate the container image. Thus, by parsing the original container definition file, replacing it with a base image adapted to the target architecture and device type, and automatically injecting the corresponding hardware driver and computing library dependencies, a container image compatible with the target computing node is automatically built, avoiding the tedious work of manually preparing and debugging the runtime environment for different hardware platforms.

[0058] Specifically, system architecture mainly refers to the processor's instruction set architecture, such as x86_64 (64-bit x86 architecture), aarch64 (64-bit ARM architecture), or riscv64 (64-bit RISC-V architecture). The container build engine queries a pre-defined base image mapping table, which maps to the official or certified optimal base image based on the target device type (e.g., "general-purpose GPU," "dedicated AI accelerator card A") and system architecture. For example, if the base image used in the original Dockerfile is a dedicated CUDA base image designed for a certain brand of GPU environment with x86 architecture, while the target device output by the scheduling decision model is a certain brand of dedicated AI accelerator with aarch64 architecture, then the container build engine will replace the base image with a suitable, clean operating system image, such as the officially provided python:3.9-slim ARM64 architecture version.

[0059] Next, the base image reference in the original container definition file is replaced with the adapted base image. The container build engine directly modifies the FROM instruction in the Dockerfile to complete the switch of the basic runtime environment.

[0060] Then, based on the target device type, the container definition file is modified to inject the corresponding hardware driver dependencies and computing framework libraries. This involves inserting or modifying a series of run commands (RUN) and copy instructions (COPY) in the Dockerfile. Specifically, the container build engine can obtain the necessary software stack installation steps from a predefined configuration template based on the target device type. For example, for a target device type of "dedicated AI accelerator card A", the build engine will inject commands into the Dockerfile to install the computing toolkit specific to that accelerator card, the corresponding version of the deep learning framework adaptation library, and related dependencies. This process also includes intelligent dependency installation optimization. For Python packages that need to be compiled from source code, the build engine can identify the target architecture and automatically set the corresponding environment variables or prioritize installation from a pre-compiled Wheel package repository for that architecture to avoid compatibility failures or performance losses caused by native compilation.

[0061] Finally, based on the modified container definition file, a build operation is performed to generate a container image. The container build engine calls multi-architecture build tools such as docker buildx or BuildKit, utilizing efficient layer caching strategies to accelerate the build process. To improve efficiency, common base layers (such as the operating system layer and the Python runtime layer) are cached in a private image repository. The successfully built final image is pushed to the specified image repository with a unique tag (e.g., combining the task identifier, code version hash, and target device type, such as training-job:abcd1234-acceleratorA) to ensure accurate traceability and retrieval during deployment.

[0062] In this embodiment of the application, through the above automated process, a standardized runtime container image that is deeply compatible with the target hardware environment and requires no manual intervention is dynamically generated for the user task, laying the environmental foundation for the smooth execution of subsequent tasks.

[0063] Step 105: Input the source code into the preset code adaptation model, and the code adaptation model outputs the adapted code that is compatible with the target execution device type.

[0064] In this embodiment, the code adaptation large model is a specially trained Large Language Model (LLM). Its core function is to understand the source code logic of the computing task and automatically generate functionally equivalent but hardware-dependent code implementations based on the characteristics and programming interfaces of the target hardware platform. The input received by this model is the complete source code obtained in the aforementioned embodiments and the target execution device type.

[0065] In one embodiment, step 105 may include: identifying hardware-binding code segments related to the source execution device type from the source code; constructing prompt information to guide code conversion based on the target execution device type, wherein the prompt information includes the target execution device type, the hardware-binding code segment to be converted, and the code conversion rules; inputting the prompt information into the code adaptation model to obtain the initial adaptation code generated by the code adaptation model; verifying the initial adaptation code, and determining the initial adaptation code as the adapted code after successful verification. Thus, by identifying hardware-related code segments from the source code, constructing prompt information containing the target device type and conversion rules to drive the large model to generate adaptation code, and verifying the initial code, automated and intelligent conversion and preliminary quality assurance from source code to target hardware code are achieved.

[0066] It's important to note that, firstly, the code adaptation process identifies hardware-bound code segments from the source code that are related to the source execution device type. The source execution device type typically refers to the hardware platform that was defaulted to or explicitly specified when the user's original code was written. For example, the code might heavily utilize CUDA extended kernels or API calls for general-purpose GPUs. The code adaptation model or static analysis tools that work with it scan the source code to identify device-specific calls such as `import torch.cuda`, `model.cuda()`, and `torch.cuda.is_available()`, as well as custom kernel functions defined by keywords like `__global__`. These code segments are deeply tied to the specific hardware ecosystem and are key objects for migration and adaptation.

[0067] Secondly, based on the target execution device type, prompts are constructed to guide the code conversion. These prompts are structured text instructions designed to guide large models in generating code that meets specific requirements. The prompts include several core components: a clearly defined target execution device type (e.g., "dedicated AI accelerator card A"), the hardware-bound code segment to be converted (i.e., the code snippet identified in the previous step), and the code conversion rules. These rules are typically embedded within the context of the prompts or provided as examples, such as requiring the conversion of CUDA kernel function calls to the computation API corresponding to the target accelerator card, or changing the device move operation from `.cuda()` to `.to('acceleratorA')`. Constructing the prompts is a crucial step, as its quality directly impacts the accuracy of the generated code. The system maintains a prompt template library, pre-configuring optimized prompt templates for different source-target device combinations and frameworks (e.g., PyTorch to target device, TensorFlow to target device). These templates clearly define the conversion syntax rules, the import method for the target framework, and common adaptation patterns.

[0068] Next, the constructed prompts are input into the code adaptation model to obtain the initial adaptation code generated by the model. Based on its deep understanding of programming languages ​​and hardware computing models, the model generates one or more pieces of transformed code according to the instructions and context in the prompts. For example, for a simple code snippet that moves the model and data to the GPU, `model.cuda(); data=data.cuda()`, if the target device is "dedicated AI accelerator card A" and the PyTorch adaptation library `torch_acceleratorA` is used, the model might generate `model.to('acceleratorA'); data=data.to('acceleratorA')` as the initial adaptation code.

[0069] Then, the initial adaptation code is verified, and only after successful verification is it determined as the final adapted code. Verification is a crucial step in ensuring the correctness and usability of the generated code.

[0070] In one embodiment, the verification of the initial adaptation code, after successful verification, determines the initial adaptation code as the adapted code. This specifically includes the following steps: running simplified test cases of the initial adaptation code in a preset target architecture sandbox environment; if the simplified test cases fail, reconstructing the erroneous code segment and corresponding error information into a correction prompt, and re-inputting it into the code adaptation model for iterative correction; if, after a preset number of iterative corrections, the initial adaptation code still fails verification, determining that the current task migration has failed; triggering a task rollback process, resubmitting the computation task to the scheduling decision model, and excluding the target execution device type from the resource requirement information. Therefore, by reconstructing error information into correction prompts and feeding them back to the model for iterative correction when verification fails, and triggering task rollback and rescheduling processes after multiple correction failures, a self-correcting and safety fault-tolerance mechanism is provided for the code adaptation process, enhancing the robustness of the entire migration system and the final success rate of the task.

[0071] It's important to note that the verification process begins by running simplified test cases of the initial adapted code within a pre-defined target architecture sandbox environment. This sandbox environment is a lightweight, isolated container environment pre-installed with a basic software stack compatible with the target device (such as Python for the target architecture and deep learning framework adaptation libraries) for securely executing code snippet tests. The simplified test cases aim to quickly verify the basic syntactic correctness and core logic of the generated code. For example, performing a forward propagation on an adapted model checks whether it produces the expected output tensor shape, allowing for minimal numerical precision errors.

[0072] In cases where simplified test cases fail to run, the system initiates an iterative correction loop. Specifically, the system reconstructs the erroneous code segment and its corresponding error message (such as the exception trace stack thrown by Python) into a new, more detailed correction prompt (e.g., "The following code encountered an error when running on the target device: [Error Code], Error Message: [Specific Error]. Please analyze the cause and correct it."), and then re-enters it into the code adaptation model for iterative correction. This "generate-verify-feedback-regenerate" loop can be performed a preset number of times (e.g., 3 times).

[0073] Furthermore, if the initial adaptation code fails verification after a predetermined number of iterations, the system will determine that the current task migration has failed. At this point, the system will trigger a task rollback process, resubmitting the computation task to the scheduling decision model and excluding the target execution device type from the task's resource requirement information. This prevents the scheduler from reassigning the task to an incompatible device of the same type, ensuring that the task can be intelligently rescheduled to other feasible computing resources for continued execution. Only the verified initial adaptation code will be ultimately determined as reliable adapted code and proceed to the next step.

[0074] In this embodiment, after the adapted code is determined, the dependency declaration files related to the computing task (such as requirements.txt or pyproject.toml) can be automatically updated to specify the version of the deep learning framework as compatible with the target execution device type, and add necessary device adaptation library dependencies (e.g., adding torch_npu). Furthermore, the system performs dependency conflict resolution and security scanning to ensure the newly built software environment is stable and secure.

[0075] In this embodiment of the application, through the above series of automated and fault-tolerant steps, intelligent and highly reliable migration and adaptation from source code to target hardware code is achieved.

[0076] Step 106: Based on the container image and the adapted code, deploy and execute the computing task on the target compute node.

[0077] The deployment and execution phase is the final step in this embodiment of the application to put all the aforementioned intelligent decision-making and automated preparation results into practice, aiming to successfully start and run the user's computing tasks on the target computing node. This process can be coordinated by the deployment and execution engine.

[0078] In this embodiment, the deployment execution engine can first generate a final task deployment definition file based on the target execution device type, target compute node information, container image identifier generated through the aforementioned embodiments, and verified adapted code. Taking the widely used container orchestration platform Kubernetes as an example, this file is a Pod definition file (a declarative configuration file in YAML format). This file precisely specifies several key constraints and configurations: the nodeSelector field restricts this Pod to be scheduled only on target compute nodes that have the target execution device type (such as the hardware tag accelerator-vendor.com / type:acceleratorA) and a matching system architecture (such as kubernetes.io / arch:arm64); the containers.image field specifies the container image built and pushed to the repository using the aforementioned embodiments; and the containers.command or containers.args field specifies the startup command to execute the entry script of the adapted code.

[0079] The deployment execution engine can submit this deployment definition file to the cluster's orchestration system (such as the Kubernetes API Server). The cluster scheduler will strictly follow the node selector constraints in the file to schedule Pod instances to the specified target compute nodes. The container runtime (such as containerd) on the target node will pull the specified container image and create the container. Finally, it will start and execute the adapted code inside the container, thereby officially running the user's computing tasks (e.g., starting the training iteration of the model).

[0080] In one embodiment, after the computing task is deployed and executed on the target computing node, resource utilization efficiency can be continuously improved by: monitoring the performance indicators of the computing task during its operation, including iteration speed and device power consumption; associating the performance indicators with task computing characteristic information and storing them in a historical performance database to update the historical performance database; and optimizing the performance prediction module in the scheduling decision model based on the updated historical performance database. Thus, by monitoring performance indicators during task execution and associating them with task characteristics to update the historical database for optimizing the prediction module of the scheduling model, a continuous self-learning and optimization loop is formed, enabling the system's scheduling decisions to continuously improve with the accumulation of operational experience, thereby continuously improving resource utilization efficiency.

[0081] It's important to note that a parallel monitoring and feedback process is initiated to create a closed-loop optimization. Specifically, this involves monitoring a series of performance metrics during the execution of the computing task. These metrics include at least core iteration speeds (such as the number of samples processed per second or the time required to complete one training iteration) and real-time power consumption data of relevant devices on the target computing node. These metrics are collected and reported in real-time by monitoring agents deployed on the nodes.

[0082] Next, the collected performance metrics are associated with the task's computational characteristics and stored to update the historical performance database. This association means establishing a mapping between the actual performance data of this task run (such as actual execution time and average power consumption) and the computational characteristics of the task extracted in step 102 (such as model structure, key operators, and framework type), and storing this as a new experience record in the database. This fresh and realistic runtime data greatly enriches the content of the historical performance database, making its performance predictions for future tasks more accurate.

[0083] Finally, the system optimizes the performance prediction module in the scheduling decision model based on the updated historical performance database. The performance prediction module is a sub-module within the scheduling decision model used to estimate the execution time of tasks on different hardware. By retraining or adjusting the algorithm parameters of this prediction module using newly accumulated runtime data (e.g., updating the prediction model based on operator performance mapping), its future prediction results can better reflect the evolution of the actual cluster environment and hardware performance, thereby continuously improving the accuracy and overall efficiency of the scheduling decision in step 103.

[0084] Furthermore, after the computation task is completed, a detailed task execution report can be automatically generated, including the decision basis for this intelligent scheduling, a summary of changes in the code automatic adaptation, and the actual resources consumed and performance indicators, providing users with a complete insight into task execution.

[0085] In this embodiment of the application, through the complete “deployment-execution-monitoring-feedback-optimization” closed loop, not only is the automated migration and execution of a single task realized, but the entire system also has the ability to continuously learn from historical experience and improve itself, thereby improving the overall utilization efficiency and intelligence level of the heterogeneous computing power cluster in the long term.

[0086] The method for intelligent scheduling and task migration of heterogeneous computing power based on a large model disclosed in the above embodiments of this application first obtains a computing task containing resource configuration files and source code; then, it parses the resource configuration files to obtain resource requirement information and analyzes the source code to obtain computing characteristic information; next, it inputs the resource requirement information, computing characteristic information, and real-time collected heterogeneous computing power resource status information into a preset scheduling decision model, which outputs the target execution device type and target computing node; further, it generates a container image compatible with the target computing node based on the target execution device type and target computing node; then, it inputs the source code into a preset code adaptation large model, which outputs adapted code adapted to the target execution device type; finally, it deploys and executes the computing task on the target computing node based on the container image and adapted code. Thus, by intelligently sensing the computing power requirements of the computing task and the real-time status of the cluster, it automatically decides the optimal execution device and node, and completes the automated adaptation and deployment of the operating environment and code, thereby achieving intelligent scheduling and seamless task migration of heterogeneous computing power and improving overall resource utilization.

[0087] To facilitate understanding of the above embodiments, the following is in conjunction with... Figure 2 The flowchart shown here is an example of a heterogeneous computing power intelligent scheduling and task migration method based on a large model, illustrating a specific task migration process. For example... Figure 2 As shown, the process begins with task submission and first enters the task parsing phase. Simultaneously, the resource monitoring and perception module continuously reads the historical performance database and collects real-time status information from each node in the heterogeneous computing resource pool. The results of the task parsing, along with the real-time resource status information, are input into the intelligent scheduling decision module, which then integrates this information to make a scheduling decision.

[0088] The scheduling decision will trigger the multi-architecture container production phase, automatically generating a container image compatible with the target computing architecture. Subsequently, the process enters the code adaptation and migration phase based on the large model; if the task does not require migration or the migration fails, it will directly jump to the task deployment phase; if the migration is successful, the migrated code will be used to continue.

[0089] Finally, the process enters the task deployment and execution phase, where tasks are deployed to a heterogeneous computing resource pool that includes various computing cards (such as GPUs, NPUs, and TPUs) and various main control processor architectures (such as RISC-V, x86, and ARM). Performance and status data generated during task execution are transmitted back to the resource monitoring and awareness module in real time, thus forming a continuously optimized and self-improving closed-loop system.

[0090] The embodiments described above are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for intelligent scheduling and task migration of heterogeneous computing power based on a large model, characterized in that, include: Obtain computational tasks that include resource configuration files and source code; The resource configuration file is parsed to obtain resource requirement information, and the source code is analyzed to obtain computational characteristic information; The resource demand information, the computing characteristic information, and the real-time collected heterogeneous computing power resource status information are input into a preset scheduling decision model, and the scheduling decision model outputs the target execution device type and the target computing node. Based on the target execution device type and the target computing node, generate a container image compatible with the target computing node; The source code is input into a preset code adaptation model, and the code adaptation model outputs adapted code that is compatible with the target execution device type. Based on the container image and the adapted code, the computing task is deployed and executed on the target computing node.

2. The method according to claim 1, characterized in that, The analysis of the source code to obtain computational feature information includes: Identify the type of machine learning framework used in the source code; Extract the key computational operators from the source code; Based on the historical performance database, the performance mapping relationship of the key computing operators on various heterogeneous computing devices is determined; The machine learning framework type, the key computational operators, and the performance mapping relationship are determined as the computational feature information.

3. The method according to claim 1, characterized in that, The scheduling decision model makes computing power scheduling decisions based on a cost function, which is used to evaluate at least two of the following factors: the task execution time on different types of computing power devices predicted based on the computing characteristic information and historical performance database, the power consumption data of each candidate computing power device obtained in real time from the cluster monitoring system, and the financial cost of using each candidate computing power device calculated according to the pre-designed pricing strategy. The scheduling decision model outputs the target execution device type and the target computing node, including: the scheduling decision model outputs the computing device type and the computing node with the best cost function evaluation result as the target execution device type and the target computing node.

4. The method according to claim 1, characterized in that, The step of generating a container image compatible with the target computing node based on the target execution device type and the target computing node includes: Parse the original container definition file associated with the computation task; Based on the target execution device type and the system architecture of the target computing node, determine the appropriate base image; Replace the base image reference in the original container definition file with the adapted base image; Based on the target execution device type, modify the container definition file after image replacement to inject the corresponding hardware driver dependency library and computing framework library into the container definition file after image replacement; Based on the modified container definition file, a build operation is performed to generate the container image.

5. The method according to claim 1, characterized in that, The step of inputting the source code into a preset code adaptation model, and the code adaptation model outputting adapted code to the target execution device type, includes: From the source code, hardware binding code segments related to the source execution device type were identified; Based on the target execution device type, a prompt message is constructed to guide the code conversion, wherein the prompt message includes the target execution device type, the hardware-bound code segment to be converted, and the code conversion rules; Input the prompt information into the code adaptation model and obtain the initial adaptation code generated by the code adaptation model; The initial adaptation code is verified, and if the verification is successful, the initial adaptation code is determined as the adapted code.

6. The method according to claim 5, characterized in that, The step of verifying the initial adaptation code, and determining the initial adaptation code as the adapted code after successful verification, includes: Simplified test cases of the initial adaptation code are run in a pre-defined target architecture sandbox environment; If the simplified test case fails to run, the error code segment and the corresponding error message are reconstructed into a correction prompt and re-entered into the code adaptation model for iterative correction. If, after a preset number of iterations and corrections, the initial adaptation code still fails to pass verification, the current task migration is deemed to have failed. Trigger the task rollback process, resubmit the computing task to the scheduling decision model, and exclude the target execution device type from the resource requirement information.

7. The method according to claim 1, characterized in that, After deploying and executing the computing task on the target compute node based on the container image and the adapted code, the method further includes: Monitor the performance metrics of the computing task during its execution, wherein the performance metrics include iteration speed and device power consumption; The performance metrics are associated with the task computation feature information and stored in the historical performance database to update the historical performance database; Based on the updated historical performance database, the performance prediction module in the scheduling decision model is optimized.

8. The method according to claim 2, 3 or 7, characterized in that, The historical performance database stores performance metrics recorded from benchmark tests of various standard models and operators running on different hardware types. These performance metrics include floating-point operations per second, throughput, and latency.

9. The method according to claim 1, characterized in that, The heterogeneous computing power resource status information includes at least one of the following: the utilization rate of the central processing unit of each computing node, the memory utilization rate, the number of various heterogeneous accelerator cards occupied, the real-time power consumption of the node, and the health status of the node.

10. The method according to claim 1, characterized in that, After deploying and executing the computing task on the target compute node based on the container image and the adapted code, the method further includes: After the computation task is completed, a task execution report is generated. The task execution report includes the scheduling decision basis, code adaptation change summary, resource consumption data and performance indicators.

Images