A method for heterogeneous scheduling of computing power under a large model training inference framework

Abstract

The application discloses a method for power heterogenous scheduling under a large model training and reasoning framework, and relates to the technical field of power scheduling. The application solves the technical problem that the task migration, splitting and recovery mechanism is imperfect, and task interruption or resource waste is easily caused when facing insufficient computing power or device abnormity. The application collects the load rate, residual computing power, temperature and other parameters of CPU, GPU and FPGA dynamically, establishes a quantitative model in combination with the calculation amount, time length and delay demand of the task, improves the resource utilization rate, establishes a temperature prediction model in stages, sets a dynamic threshold in combination with the device type, environmental temperature and health degree, realizes temperature warning and migration in the task execution process, reduces the risk of frequency reduction caused by device overheating, and ensures that the task interruption loss is reduced by more than 50% through the task splitting, intelligent migration and suspension recovery mechanism, thereby improving the stability of the system in peak traffic or hardware failure.

Description

Technical Field

[0001] This invention relates to the field of computing power scheduling technology, specifically a method for heterogeneous computing power scheduling under a large model training and inference framework. Background Technology

[0002] With the rapid development of artificial intelligence technology, the demand for computing resources for training and inference of large models has exploded, and computing devices are gradually evolving towards heterogeneity, including hardware with different architectures such as CPU, GPU, and FPGA.

[0003] According to patent application CN118897739B, a multi-cloud heterogeneous computing power scheduling system and method are disclosed. The system includes a computing power capacity analysis module, a cloud computing power platform object evaluation module, and a computing power scheduling execution module. It efficiently analyzes and processes the target cloud computing power platform type by standard storage of cloud platform objects corresponding to different computing power capacities, combined with an intelligent search algorithm and the computing power capacity parameters of the task to be processed. This enables accurate searching of target cloud computing power platform objects based on multi-cloud heterogeneity. Furthermore, it digitally analyzes the service prices of multiple target cloud computing power platform objects, scientifically identifying the cloud computing power platform with the lowest service price. This allows for precise selection of the best cloud computing power platform object to meet the needs of the task to be processed, improving the accuracy and reliability of the computing power scheduling system's scheduling operations and enhancing the quality of the scheduling operations.

[0004] However, existing computing power scheduling methods mostly rely on static resource allocation or single-dimensional load balancing, which has the following limitations: The lack of dynamic perception of the performance parameters of heterogeneous devices (such as load rate, remaining computing power, and temperature) makes it difficult to accurately match task requirements. Task classification and priority determination are mechanical and not dynamically adjusted in combination with business scenarios (such as peak traffic, core model iteration); The lack of correlation analysis between equipment temperature and task processing time can easily lead to equipment overheating and frequency reduction or hardware aging due to prolonged high load. When faced with insufficient computing power or equipment malfunctions, the imperfect task migration, splitting, and recovery mechanisms can easily lead to task interruptions or resource waste. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method for heterogeneous scheduling of computing power within a large model training and inference framework. This method solves the problem that in the face of insufficient computing power or equipment malfunctions, the task migration, splitting, and recovery mechanisms are imperfect, which can easily lead to task interruption or resource waste.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for heterogeneous scheduling of computing power within a large model training and inference framework, the method specifically comprising the following steps: S1. Obtain the performance parameters of the computing power equipment and the task requirements of the large model training and inference task, classify the tasks, and determine whether the computing power of the computing power equipment meets the requirements of the classified tasks, and generate a computing power satisfaction signal or a computing power non-satisfaction signal. S2. If the generated computing power satisfies the signal, prioritize the tasks to be processed, analyze the relationship between task processing time and equipment temperature, establish a temperature prediction model, predict the equipment temperature during task processing based on the model, and compare it with the dynamic temperature threshold. If the predicted temperature exceeds the threshold, generate a task migration signal. S3. If a task migration signal is generated, analyze the signal to obtain the tasks to be processed, filter the tasks based on the safety load threshold of the computing power device to obtain the remaining tasks to be processed, filter the pre-selected computing power devices according to the computing power requirements of the remaining tasks, and then allocate the remaining tasks based on the idle computing power of the pre-selected devices and generate computing power scheduling information. S4. If a signal indicating insufficient computing power is generated, analyze the signal to obtain the devices that do not meet the requirements and their corresponding real-time tasks. Determine whether the real-time tasks can be split. If they can be split, split them to obtain real-time subtasks. If they cannot be split, pause them and mark their recovery priority. Match target devices for the real-time subtasks based on the task requirements. Perform secondary classification on the real-time subtasks and schedule them separately to generate computing power scheduling information.

[0007] As a further aspect of the present invention, the computing power device includes a CPU, a GPU, and an FPGA, the performance parameters include load rate, remaining computing power, and operating temperature, and the task classification includes large model training tasks, inference preprocessing tasks, and low-latency inference tasks. The performance parameters of the computing power device include the CPU load rate as the proportion of the core computing power occupied by the current process to the total core computing power, the remaining computing power as calculated based on the single core computing power and the number of idle cores, and the operating temperature as the CPU packaging temperature. The GPU load rate is the utilization rate of the streaming multiprocessor (SM), the remaining computing power is the total computing power minus the used computing power, and the operating temperature is the core temperature. The FPGA load rate is the logic unit utilization rate, the remaining computing power is calculated based on the configurable computing power resources, and the operating temperature is the chip junction temperature.

[0008] As a further aspect of the present invention, the core features of the task classification include: Large model training tasks are computationally intensive, long-term, and memory intensive. The inference preprocessing task is I / O intensive, multithreaded friendly, and computationally inefficient. Low-latency inference tasks are characterized by high real-time requirements, moderate computational intensity, and sensitivity to stability.

[0009] As a further aspect of the present invention, the method for determining whether the computing power meets the requirements is as follows: The available computing power of the computing device is calculated as follows: available computing power = baseline available computing power × temperature adjustment coefficient × task duration, where baseline available computing power = remaining computing power × (1 - current load rate); the total computing power of the task is compared with the available computing power. If the total computing power is greater than the available computing power, a computing power insufficiency signal is generated; otherwise, a computing power satisfaction signal is generated.

[0010] As a further aspect of the present invention, the relationship between the analysis processing time and temperature includes: Outliers in the temperature data were removed, and the equipment operation was divided into cold start period, stable operation period and long-term high load period according to the time axis. A linear regression model of processing time and temperature was established for each period. The model expression is temperature = a × processing time + b × load rate + c × ambient temperature + d, where a, b, and c are influence intensity coefficients, and d is the intercept term.

[0011] As a further aspect of the present invention, the selection criteria for the pre-selected computing power equipment include: Task and device type compatibility: FPGA / GPU is preferred for low-latency inference tasks, GPU of the same model is preferred for large model training tasks, and CPU / FPGA is preferred for batch preprocessing tasks; network topology and status stability of devices: devices with low latency, low load and no hardware alarms within the same node are preferred.

[0012] As a further aspect of the present invention, the criteria for determining the divisible task are: the task has data independence, can be split into data batches and there is no strong dependency between subtasks, and the intermediate state of the task can be serialized and stored; the secondary classification of the real-time subtasks includes core subtasks and non-core subtasks, core subtasks adopt dynamic resource reservation and enter an independent high-priority queue, and non-core subtasks adopt a batch processing mechanism and enter a delay queue.

[0013] As a further aspect of the present invention, the calculation method of the safety load threshold is as follows: if migration is triggered due to excessive temperature, safety load = maximum load × (1 - temperature exceeding the standard ratio); if migration is triggered due to hardware abnormality, safety load = maximum load × 50%; the filtering rule of the remaining tasks to be processed is: traverse according to task priority, if the computing power required by the task accounts for a proportion of the safety load exceeding the threshold or has a high impact on the equipment, it is marked as a remaining task to be processed.

[0014] This invention provides a method for heterogeneous scheduling of computing power within a large model training and inference framework. Compared with existing technologies, it has the following advantages: This invention dynamically collects parameters such as CPU, GPU, and FPGA load rate, remaining computing power, and temperature, and combines them with the task's computational load, duration, and latency requirements to establish a quantitative model, thereby improving resource utilization. By establishing a phased temperature prediction model and setting dynamic thresholds based on device type, ambient temperature, and health status, it enables temperature warning and migration during task execution, reducing the risk of frequency reduction caused by device overheating. For insufficient computing power or device abnormalities, it ensures that task interruption losses are reduced by more than 50% through task splitting, intelligent migration, and pause recovery mechanisms, thereby improving system stability during peak traffic or hardware failures. Attached Figure Description

[0015] Figure 1 This is a diagram illustrating the steps and methods of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Please see Figure 1 This application provides a method for heterogeneous scheduling of computing power under a large model training and inference framework, which specifically includes the following steps: S1. Obtain the performance parameters of the computing power equipment, which includes CPU, GPU, and FPGA. The performance parameters include load rate, remaining computing power, and operating temperature. For example, for CPU: focus on collecting the load rate (the proportion of core computing power currently used by the process to the total core computing power; for example, if 6 out of 8 cores of an 8-core CPU are used, the load rate is 75%), remaining computing power (calculated based on single-core computing power and the number of idle cores; for example, if a CPU with 200 GFLOPS per core has 2 idle cores, the remaining computing power is 400 GFLOPS), and operating temperature (CPU packaging temperature; exceeding 90℃ may trigger frequency reduction). GPU: Focus on load rate (utilization of streaming multiprocessors (SM), such as the A100's SM utilization reaching 80%), remaining computing power (total computing power minus used computing power, such as the H100's FP16 peak of 4 PFlops, currently using 1.5 PFlops, then 2.5 PFlops remain), and operating temperature (core temperature, exceeding 85℃ will limit computing power output). FPGA: Pay attention to load rate (logic unit utilization, such as 60% of the logic units of an FPGA are occupied), remaining computing power (calculation based on configurable computing resources, such as an FPGA that supports 100 TOPS INT8 computing power, if 30 TOPS is currently used, then 70 TOPS is remaining), and operating temperature (chip junction temperature, exceeding 95℃ will affect stability). Simultaneously, we obtain the large-scale model training and inference tasks and their corresponding requirements, and classify the tasks into large-scale model training tasks, inference preprocessing tasks, and low-latency inference tasks. Specifically, the core characteristic of large-scale model training tasks is that they are computationally intensive (training a model with hundreds of billions of parameters requires 10... 18 The core characteristics of pre-inference processing tasks are: FLOPS-level computational complexity, long cycle (lasting several days to weeks), and memory-intensive (requiring storage of model weights and intermediate activation values). These are O-intensive (e.g., text tokenization, image decoding), multithreaded friendly (capable of parallel processing of numerous small tasks), and low computational intensity. Conversely, the core characteristics of low-latency inference tasks are: high real-time requirements (e.g., dialogue systems require ≤100ms response time), and moderate computational intensity (10 FLOPS-level computational complexity per inference). 9 The algorithm is sensitive to FLOPS and stability (latency fluctuation ≤10ms). Then, it obtains the total computational load and total computation time corresponding to the classified tasks, and compares the total computational load with the available computational load. The available computational load is represented as the available load of the device. If the total computational load is greater than the available computational load, it means that the current device's computing power resources cannot meet all tasks, and a computing power insufficiency signal is generated. Conversely, if the total computational load is less than the available computational load, it means that the current device's computing power resources can meet all tasks, and a computing power satisfaction signal is generated. Specifically, this refers to the computing power resources corresponding to a single computing power device.

[0018] Taking the computing power scheduling of a certain AI laboratory as an example: Equipment resources: 1 GPU (H100, nominal computing power 400 TFLOPS FP16, current remaining computing power 200 TFLOPS, load rate 40%, temperature 75℃), 1 CPU (32 cores, single core computing power 0.2 TFLOPS, 16 cores idle, load rate 30%, temperature 60℃), 1 FPGA (nominal computing power 100 TOPS INT8, remaining computing power 60 TOPS, load rate 20%, temperature 70℃).

[0019] Large model training task: total computation 1500 TFLOPS·h, requires 10 hours; inference preprocessing task: total computation 50 TFLOPS·h, requires 2 hours; low-latency inference task: total computation 80 TFLOPS·h (approximately 800 TOPS·h converted to INT8), requires 8 hours, latency requirement ≤50ms.

[0020] GPU: Baseline available computing power = 200 TFLOPS × (1-40%) = 120 TFLOPS; Temperature 75℃ (within safe range), adjustment factor 1.0; Available computing power for 10 hours = 120 × 10 = 1200 TFLOPS·h.

[0021] CPU: Single core remaining computing power 0.2 TFLOPS, 16 cores idle, total remaining computing power = 0.2 × 16 = 3.2 TFLOPS; load rate 30%, baseline available computing power = 3.2 × (1 - 30%) = 2.24 TFLOPS; temperature 60℃, adjustment factor 1.0; available computing power for 2 hours = 2.24 × 2 = 4.48 TFLOPS·h.

[0022] FPGA: Remaining computing power 60 TOPS, load rate 20%, baseline available computing power = 60 × (1 - 20%) = 48 TOPS; temperature 70℃, adjustment factor 1.0; available computing power in 8 hours = 48 × 8 = 384 TOPS·h (converted to TFLOPS·h approximately 384 × 0.1 = 38.4 TFLOPS·h, since 1 TOPS INT8 ≈ 0.1 TFLOPS FP16).

[0023] Large model training task (total computation 1500 TFLOPS·h): GPU available computation 1200 TFLOPS·h < 1500, and there are no other compatible devices (insufficient CPU / FPGA computing power), generating computational power does not meet the signal.

[0024] Inference preprocessing task (total computational load 50 TFLOPS·h): The available computational load of the CPU is 4.48 TFLOPS·h < 50. However, if combined with the FPGA (although the FPGA is more suitable for inference, it can temporarily undertake part of the preprocessing), the total available computational load of the two is 4.48 + 38.4 ≈ 42.88 TFLOPS·h, which is still < 50, generating a signal that the computing power is insufficient.

[0025] Low-latency inference task (total computation 800 TOPS·h): FPGA available computation 384 TOPS·h < 800, generating a signal that the computational power is insufficient.

[0026] S2. Analyze the obtained computing power satisfaction signal, obtain all tasks to be processed and label them as i, where i = 1, 2, ..., j, and j represents the number of tasks to be processed. Specifically, the tasks to be processed here include all classified tasks. At the same time, the priority of the task to be processed i is judged, and the task priority is determined according to the real-time performance and computing power requirements of the corresponding task. The tasks to be processed i are sorted from highest to lowest priority. Specifically, according to the basic rule: low latency inference (high real-time performance) > large model training (long cycle but fault-tolerant) > inference preprocessing (high throughput but low real-time performance). Then, obtain the total processing time corresponding to all tasks to be processed i and analyze the relationship between processing time and temperature. Obtain the temperature value corresponding to the processing time and remove outliers. Then, align the processing time and temperature according to the time axis and establish a linear regression model for processing time and temperature. The specific expression is: Temperature = a × processing time + b × load rate + c × ambient temperature + d. a, b, and c quantify the influence of running time, load rate, and ambient temperature on temperature, respectively. Here, a represents the contribution of temperature per unit time, and d is the intercept term. The calculated total processing time is substituted into the above formula to obtain the corresponding predicted temperature value. At the same time, the predicted temperature is compared with the preset temperature value, which is set by the operator. If the predicted temperature is greater than the preset temperature value, it means that the computing power is used for processing with the total processing time, and there is an abnormal equipment temperature. A task migration signal is generated. Conversely, if the predicted temperature is less than the preset temperature value, it means that the computing power is used for processing with the total processing time, the equipment is working normally, and no processing is performed.

[0027] To illustrate with a practical example, Task queue (sorted by priority): T1 (low-latency inference, requires 1 hour, to be executed during peak hours of 8-9 pm) → T2 (training task, requires 3 hours, to be executed during 9 pm-12 am) → T3 (inference preprocessing, requires 2 hours, to be executed during 6 pm-8 pm).

[0028] Device status: 1 A100 GPU (current temperature 60℃, load rate 40%, ambient temperature 26℃).

[0029] Dynamic threshold: T1 (inference, summer 26℃): GPU threshold = 85℃ - (26-22) × 0.5 = 83℃; T2 (training, summer 26℃): GPU threshold = 78℃ - (26-22) × 0.5 = 76℃; T3 (preprocessing, summer 26℃): GPU threshold = 80℃ - (26-22) × 0.5 = 78℃.

[0030] Temperature prediction and decision: T3 (pretreatment, 2 hours, in cold start period): Substitute into the model → Temperature = 0.8 × 120 minutes + 0.3 × 60% + 0.5 × 26 + 30 ≈ 65℃ (< 78℃) → Normal execution; T1 (Inference, 1 hour, in stable period): Cumulative duration = 2 + 1 = 3 hours → Substitute into stable period model → Temperature = 0.1 × 60 + 0.3 × 80% + 0.5 × 26 + 60 ≈ 70℃ (< 83℃) → Normal execution; T2 (training, 3 hours, in a long-term high-load period): Cumulative duration = 3 + 3 = 6 hours → Substitute into the long-term model → Temperature = 0.02 × 180 + 0.3 × 90% + 0.5 × 26 + 70 ≈ 79℃ (> 76℃) → Trigger task migration signal.

[0031] S3. Analyze the generated task migration signal to obtain the tasks to be processed corresponding to the computing power devices. At the same time, filter the tasks to be processed based on the maximum load of the computing power devices and obtain the remaining tasks to be processed. Specifically, calculate the number of tasks to be processed that the maximum load can meet, and calculate them in descending order of priority. Then, filter the remaining computing power devices based on the computing power requirements corresponding to the remaining tasks to be processed and obtain the pre-selected computing power devices. The pre-selected computing power devices are represented by the computing power resource requirements that can meet the remaining tasks to be processed, specifically in terms of computing power. At the same time, obtain the available idle computing power corresponding to the pre-selected computing power devices and allocate the remaining tasks to be processed based on the idle computing power, and generate computing power scheduling information.

[0032] If the computing device's "safe load threshold" is triggered to migrate due to excessive temperature: Safe load = maximum load × (1 - temperature exceedance percentage) (e.g., if the GPU temperature is 88℃, exceeding the limit by 3℃, safe load = 100% × 90% = 90%). If migration is triggered due to hardware malfunction: Safe load = maximum load × 50% (only necessary tasks are retained to reduce equipment pressure).

[0033] Traverse by task priority (core tasks → important tasks → ordinary tasks) and calculate the "proportion of computing power required by the task relative to the safe load": If the proportion is less than or equal to the safe load and the task has a low impact on the equipment (e.g., temperature contribution < 1℃ / hour), then retain it; If the proportion is greater than the safety load or the impact is high, then mark it as "remaining tasks that need to be migrated".

[0034] Different tasks have different dependencies on device characteristics, so the appropriate adaptation type should be matched first: For low-latency inference tasks (such as user dialogue): prioritize FPGA / GPU (single task latency <50ms), exclude CPU (high scheduling latency); For large model training tasks (such as model iteration): prioritize GPUs of the same model (such as A100→A100 to avoid cross-architecture adaptation costs) and support model parallelism; For batch preprocessing tasks (such as data cleaning): CPU / FPGA is preferred (low cost, suitable for high concurrency).

[0035] S4. Analyze the obtained computing power non-satisfaction signals, obtain the corresponding computing power devices and record them as non-satisfaction devices. At the same time, obtain the real-time tasks corresponding to the non-satisfaction devices and determine whether the real-time tasks can be split. If they can be split, split the real-time tasks to obtain real-time subtasks. Otherwise, if they cannot be split, obtain the corresponding pending tasks and pause them. At the same time, prioritize the processing of real-time tasks. Specifically, before pausing a task, automatically save the current state (e.g., save model weights, optimizer parameters, and current batch data index for training tasks). The saving time is ≤10 seconds (to avoid excessive resource consumption). Add a "recovery priority" to the paused task (e.g., mark core training tasks as "P0" and ordinary tasks as "P1") to ensure priority recovery when resources are sufficient. When the computing power of the original device recovers (e.g., the load rate drops from 95% to 60%), or a new device is added to the cluster (e.g., expand by 1 GPU), wake up the task according to the "recovery priority". For the obtained real-time subtasks, corresponding low-load devices of the same type are obtained according to their task requirements. Low load is determined by the device's "remaining resources × safety factor" > task requirements (e.g., a GPU with 100 TFLOPS of remaining computing power and a safety factor of 0.8 can handle tasks of ≤80 TFLOPS). The real-time subtasks are then further classified into core subtasks and non-core subtasks. The specific classification criteria are based on a comprehensive analysis of the real-time performance and uninterruptibility of the subtasks. Specifically, core tasks directly affect user experience or business continuity (e.g., "real-time product recommendation inference" and "payment link risk identification" during e-commerce promotions), while non-core tasks have low real-time requirements but high throughput (e.g., offline analysis of inference logs and auxiliary data cleaning for training tasks). For the classified core subtasks, fixed computing power is reserved for them, and the allocated device resources are locked. They are also placed in an independent queue. For non-core subtasks, the idle time slices of core tasks are automatically filled, and they are placed in a delay queue. After accumulating a certain number, they are processed in batches, and computing power scheduling information is generated.

[0036] For core tasks, reserve 30% of the target device's peak computing power (e.g., 60 TFLOPS for a GPU peak of 200 TFLOPS) for core subtasks. If the core task latency exceeds a threshold (e.g., recommended inference latency > 100ms), temporarily increase the reservation to 50% (preempting resources from non-core tasks). If the latency is consistently < 50ms, reduce it to 20% (releasing resources to non-core tasks), and enter an independent high-priority queue. The scheduler scans the queue every 10ms to ensure zero queuing for subtasks (direct allocation of reserved resources). For non-core tasks, when the length of the non-core subtask queue is greater than 1000 (e.g., 1000 logs to be analyzed), batch processing is triggered. At the same time, if the target device load rate is less than 50% (e.g., CPU load rate is 30%), the queue tasks are actively pulled and executed in batches, and then placed into a delayed queue. The tasks are allocated according to the first-in-first-out (FIFO) + resource affinity principle (e.g., log analysis tasks of the same type are centrally allocated to a certain CPU to reduce data migration costs).

[0037] The data in the above formulas are all calculated using numerical values, without substituting the units of the parameters. In addition, the contents not described in detail in this specification are all prior art known to those skilled in the art.

[0038] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A method for heterogeneous scheduling of computing power within a large model training and inference framework, characterized in that, The method specifically includes the following steps: S1. Obtain the performance parameters of computing devices and the task requirements of large model training and inference tasks, classify the tasks, and determine whether the computing power of the computing devices meets the requirements of the classified tasks, generating a computing power satisfaction signal or a computing power insufficiency signal. The computing devices include CPUs, GPUs, and FPGAs, and the performance parameters include load rate, remaining computing power, and operating temperature. The task classifications include large model training tasks, inference preprocessing tasks, and low-latency inference tasks. Large model training tasks are computationally intensive, long-cycle, and memory-intensive; inference preprocessing tasks are I / O intensive, multi-threaded friendly, and have low computational intensity; low-latency inference tasks have high real-time requirements, medium computational intensity, and are stability sensitive. The performance parameters of the computing power equipment include the CPU load rate as the proportion of the core computing power currently used by the process to the total core computing power, the remaining computing power as calculated based on the single core computing power and the number of idle cores, and the operating temperature as the CPU packaging temperature. The GPU load rate is the utilization rate of the streaming multiprocessor (SM), the remaining computing power is the total computing power minus the used computing power, and the operating temperature is the core temperature. The FPGA load rate is the logic unit utilization rate, the remaining computing power is calculated based on the configurable computing power resources, and the operating temperature is the chip junction temperature. S2. If the generated computing power satisfies the signal, prioritize the tasks to be processed, analyze the relationship between task processing time and equipment temperature, establish a temperature prediction model, predict the equipment temperature during task processing based on the model, and compare it with the dynamic temperature threshold. If the predicted temperature exceeds the threshold, generate a task migration signal. S3. If a task migration signal is generated, analyze the signal to obtain the tasks to be processed, filter the tasks based on the safety load threshold of the computing power device to obtain the remaining tasks to be processed, filter the pre-selected computing power devices according to the computing power requirements of the remaining tasks, and then allocate the remaining tasks based on the idle computing power of the pre-selected devices and generate computing power scheduling information. S4. If a signal indicating insufficient computing power is generated, analyze the signal to obtain the devices that do not meet the requirements and their corresponding real-time tasks. Determine whether the real-time tasks can be split. If they can be split, split them to obtain real-time subtasks. If they cannot be split, pause them and mark their recovery priority. Match target devices for the real-time subtasks based on the task requirements. Perform secondary classification on the real-time subtasks and schedule them separately to generate computing power scheduling information.

2. The method for heterogeneous scheduling of computing power under a large model training and inference framework according to claim 1, characterized in that, The method for determining whether the computing power meets the requirements is as follows: The available computing power of the computing equipment is calculated as follows: available computing power = baseline available computing power × temperature adjustment coefficient × task duration, where baseline available computing power = remaining computing power × (1 - current load rate). The total computational load of the task is compared with the available computational load. If the total computational load is greater than the available computational load, a signal indicating that the computational power is insufficient is generated; otherwise, a signal indicating that the computational power is sufficient is generated.

3. The method for heterogeneous scheduling of computing power under a large model training and inference framework according to claim 1, characterized in that, The relationship between the analysis processing time and temperature includes: Outliers in the temperature data were removed, and the equipment operation was divided into cold start period, stable operation period and long-term high load period according to the time axis. A linear regression model of processing time and temperature was established for each period. The model expression is temperature = a × processing time + b × load rate + c × ambient temperature + d, where a, b, and c are influence intensity coefficients, and d is the intercept term.

4. The method for heterogeneous scheduling of computing power under a large model training and inference framework according to claim 1, characterized in that, The selection criteria for the pre-selected computing power equipment include: Task and device type compatibility: FPGA / GPU is preferred for low-latency inference tasks, GPU of the same model is preferred for large model training tasks, and CPU / FPGA is preferred for batch preprocessing tasks; network topology and status stability of devices: devices with low latency, low load and no hardware alarms within the same node are preferred.

5. The method for heterogeneous scheduling of computing power under a large model training and inference framework according to claim 1, characterized in that, The criteria for determining a divisible task are: the task has data independence, can be split into data batches and has no strong dependency between subtasks, and the intermediate state of the task can be serialized and stored; the secondary classification of real-time subtasks includes core subtasks and non-core subtasks, core subtasks adopt dynamic resource reservation and enter an independent high-priority queue, and non-core subtasks adopt a batch processing mechanism and enter a delay queue.

6. The method for heterogeneous scheduling of computing power under a large model training and inference framework according to claim 1, characterized in that, The calculation method for the safety load threshold is as follows: if migration is triggered due to excessive temperature, safety load = maximum load × (1 - temperature exceedance ratio); if migration is triggered due to hardware abnormality, safety load = maximum load × 50%; the filtering rule for the remaining tasks to be processed is: traverse according to task priority, if the computing power required by the task accounts for a proportion of the safety load exceeding the threshold or has a high impact on the equipment, then it is marked as a remaining task to be processed.

Images