A method for collaborative scheduling and optimization of heterogeneous resources in an algorithm training platform

Abstract

This invention relates to the field of heterogeneous computing resource scheduling technology, and discloses a method for collaborative scheduling and optimization of heterogeneous resources in an algorithm training platform. The method includes: real-time acquisition of telemetry parameters within an intelligent chip, and calculation of a scheduling entropy inversely proportional to the matching degree between the task and the chip architecture; when the scheduling entropy continuously exceeds a preset threshold, the computing task is migrated to heterogeneous chips of different types. This invention achieves spontaneous coupling between the task and the hardware architecture through scheduling entropy, avoiding architectural mismatch caused by resource abstraction in existing technologies. Its scheduling decisions no longer rely on complex performance predictions. Simultaneously, combined with multi-level verification and control mechanisms, it effectively improves the robustness and operational reliability of the entire scheduling system.

Description

Technical Field

[0001] This invention relates to a method for collaborative scheduling and optimization of heterogeneous resources in an algorithm training platform, belonging to the field of heterogeneous computing resource scheduling technology. Background Technology

[0002] Currently, mainstream scheduling systems generally adopt a general abstract model based on resource requests and constraints, which unifies intelligent chips with different hardware architectures into standardized computing power units for allocation. This approach simplifies the complexity of resource management to some extent.

[0003] However, this general abstraction, undertaken for the sake of management convenience, reveals its inherent limitations in practical applications, especially when dealing with complex machine learning tasks involving multiple computational stages. For example, a typical large model fine-tuning task naturally includes a data loading stage primarily using CPU processing, a model training stage primarily using high-precision floating-point operations, and a model quantization and verification stage primarily using low-precision integer operations. Under existing scheduling methods, the system often binds the entire task cycle to a single type of resource based on the single resource type request submitted at the time of task submission. As a result, in computational stages where the specialized intelligent chip resources are not suited, they either operate inefficiently or remain idle, while other heterogeneous chips more suitable for the current stage cannot be utilized. This leads to a pervasive and chronic architectural mismatch and resource waste.

[0004] The root cause of this problem lies not in insufficient monitoring methods, but in the lack of a dimension in scheduling itself. Existing technologies focus excessively on the quantitative satisfaction of resource sufficiency, failing to effectively perceive the quality matching of resources in real time. Specifically, existing technologies suffer from the following shortcomings: 1. Lack of direct means to measure the real-time matching degree between the actual operating state of the intelligent chip and the task computing mode; 2. Static resource allocation strategies are difficult to adapt to the dynamically changing computing characteristics of computing tasks at different lifecycle stages, resulting in structural resource idleness and mismatch; 3. Lack of a low-overhead, intrinsic adaptive mechanism that allows tasks to spontaneously find and migrate to the optimal hardware architecture. Therefore, the technical problem this invention aims to solve is how to establish a mechanism that does not rely on complex predictive models but is based on direct feedback from the operating state of the intelligent chip, quantifies the real-time matching degree between tasks and hardware architecture, and drives the adaptive migration of computing tasks between different types of heterogeneous resources, thereby effectively avoiding architectural mismatch. Summary of the Invention

[0005] This invention provides a method for collaborative scheduling and optimization of heterogeneous resources in an algorithm training platform. Its main purpose is to solve the problems of architectural mismatch between computing tasks and heterogeneous intelligent chips caused by the use of general resource abstraction in the prior art, as well as the resulting resource waste and low scheduling efficiency.

[0006] To achieve the above objectives, this invention provides a method for heterogeneous resource collaborative scheduling and optimization of an algorithm training platform, comprising the following steps:

[0007] Step a: Real-time acquisition of at least one telemetry parameter running inside the intelligent chip carrying the computing task. The telemetry parameter reflects the operating status of the internal architecture of the intelligent chip. The intelligent chip is one of a graphics processor, a neural network processor, and a tensor processor.

[0008] Step b: Calculate a scheduling entropy based on at least one real-time telemetry parameter obtained; the calculation rule for the scheduling entropy is preset so that its value is inversely proportional to the degree of matching between the computing mode of the computing task and the hardware architecture of the smart chip.

[0009] Step c: Determine whether the real-time calculated scheduling entropy is continuously higher than a preset entropy threshold.

[0010] In step d, if the judgment result is yes, it is determined that there is an architectural mismatch between the computing task and the current intelligent chip. After the computing task completes a computing batch or a training cycle, the computing task is migrated from the current intelligent chip to a currently available heterogeneous intelligent chip with a different chip type than the current intelligent chip.

[0011] Preferably, at least one telemetry parameter includes at least one of instruction pipeline stall rate, memory access latency variance, and average utilization of computing units.

[0012] Preferably, the calculation of the scheduling entropy in step b is a dimensionless value obtained by normalizing at least one real-time telemetry parameter and then performing a weighted summation. The calculation method is as follows: Among them, S 调度 The scheduling entropy is n; the number of types of real-time telemetry parameters acquired is n; P is the scheduling entropy. i,norm w is the normalized value of the i-th real-time telemetry parameter; i These are the preset weighting coefficients corresponding to the i-th real-time telemetry parameter, and the sum of all weighting coefficients is 1.

[0013] Preferably, the method further includes: when the scheduling entropy is continuously higher than a first preset threshold and lower than a second preset threshold which is the entropy threshold, the migration operation in step d is not performed, but in response to the scheduling entropy being in the threshold range, the core operating frequency of the current smart chip or its power consumption limit is adjusted.

[0014] Preferably, the heterogeneous smart chip to which the migration takes place in step d is a usable heterogeneous smart chip with the highest historical affinity score, determined based on the statistical data of the running performance of historical tasks of the same type as the current computing task recorded in the historical task log on different types of heterogeneous smart chips.

[0015] Preferably, the execution of step d is also based on an additional judgment condition, namely, that an application layer performance indicator of the computing task is lower than a preset performance threshold; the application layer performance indicator is one of the number of samples processed per second and the query response rate.

[0016] Preferably, before executing step d, a decision arbitration step is also included, which includes: checking whether the migration operation conforms to a preset system-level security and management policy rule set, and only executing the migration if the migration operation does not violate the rule set; the rule set includes rules for determining whether the target chip is in maintenance state and rules for determining whether task data is allowed to move across nodes.

[0017] Preferably, the method further includes: while executing step b, monitoring the application-layer performance residual of the computing task in real time, where the performance residual is the rate of change of the loss function value; when it is determined that the performance residual fluctuates continuously above the statistical baseline, performing a preset micro-benchmark test on the current intelligent chip during the computing interval of the computing task, and generating a compensation coefficient that quantifies the physical drift of the current hardware computing characteristics based on the comparison between the actual output and the theoretical output of the test; and in subsequent executions of step b, using the generated compensation coefficient to adjust one or more parameters used to calculate the scheduling entropy in real time.

[0018] Preferably, the method further includes: injecting a standardized probing computation instruction into the smart chip during a preset idle period; capturing and recording the response characteristics of the probing computation instruction during multiple executions, the response characteristics including the jitter value of execution latency and the fluctuation value of peak power consumption; calculating a health index characterizing the hardware health status of the smart chip based on the time series data of the response characteristics; and excluding smart chips with a health index lower than a safety threshold from the list of available resources in subsequent scheduling decisions.

[0019] Preferably, the timing of migrating the computing task from the current smart chip in step d is limited to after the computing task has completed a complete batch of computing processing or after the computing task has completed a complete training cycle.

[0020] Compared with the prior art, the beneficial effects of the present invention are:

[0021] 1. By introducing scheduling entropy, a metric that directly reflects the internal operating state of intelligent chips, the basis for scheduling decisions shifts from satisfying the quantity of external resource requests to pursuing the real-time matching quality between computing tasks and chip hardware architecture. It no longer relies on complex performance prediction models, but instead uses a negative feedback mechanism based on high entropy migration to allow computing tasks to adaptively migrate between heterogeneous resources. This approach avoids the common architectural mismatch and waste caused by resource abstraction in existing technologies, enabling different types of intelligent chips to proactively serve their most proficient computing modes. This transforms the entire heterogeneous cluster from a homogeneous computing power pool into a system that facilitates efficient matching between tasks and hardware.

[0022] 2. A multi-layered entropy control and verification mechanism was established, which improved the smoothness of system operation while ensuring scheduling robustness. This invention does not adopt a single migration strategy, but constructs a response system that takes into account both efficiency and cost. When the architectural mismatch is not serious, the in-situ optimization mechanism is initiated first, and the entropy increase is suppressed by fine-tuning the chip's operating parameters. This resolves most of the mismatch caused by instantaneous load fluctuations without interrupting tasks, and significantly reduces the frequency of cross-node task migration. At the same time, by introducing application-layer performance indicators independent of the chip's internal telemetry for cross-verification, and establishing a final system-level security policy arbitration, a complete verification and security process from signal input to decision output is formed, which effectively avoids erroneous decisions caused by sensor noise or software failures and ensures the reliability of scheduling behavior in complex production environments.

[0023] 3. This invention provides the scheduling system with a deep understanding of the hardware lifecycle status, extending the optimization scope from functional correctness to the dimensions of result reliability and failure prediction. Addressing the issue of physical characteristic drift that occurs during long-term hardware operation, this invention uses application-layer performance residuals as an indirect representation of hardware status. Through dynamic benchmark testing and compensation coefficient correction, the calculation of scheduling entropy can adapt to hardware performance degradation, ensuring the long-term reliability of calculation results in high-precision tasks such as medical AI. Furthermore, by utilizing scheduling gaps to proactively detect chip health, this invention can quantify and predict the sub-health state of hardware based on minute changes in response characteristics, thereby isolating hardware units on the verge of failure in advance, realizing a shift from passive fault response to proactive predictive maintenance. Attached Figure Description

[0024] Figure 1 This is a flowchart of a heterogeneous resource collaborative scheduling and optimization method for an algorithm training platform according to the present invention;

[0025] Figure 2 This is a schematic diagram illustrating the change of health index of different types of smart chips in this invention over running time;

[0026] Figure 3 This is a timing diagram of the generation and application of a smart chip health index according to the present invention.

[0027] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in further detail below. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0029] The heterogeneous resource collaborative scheduling and optimization method for algorithm training platforms disclosed in this invention can be applied to task scenarios involving multiple stages with vastly different computational characteristics, such as large model fine-tuning. This method constructs a closed-loop control system with scheduling entropy as the core feedback signal. Its systematic operation includes the following core stages: First, real-time telemetry sensing is performed to collect telemetry parameters from the hardware layer of the intelligent chip carrying the computational task; then, architecture matching quantification is performed, calculating the telemetry parameters and converting them into scheduling entropy characterizing the matching quality; next, multi-level control decisions are made based on the scheduling entropy; finally, task migration with pre-arbitration and health status awareness is executed. The entire process forms an adaptive closed loop that continuously seeks optimal coupling between the task and the hardware architecture. In a specific running example, when a computational task is deployed on a system containing a graphics processor, a neural network processor, and a tensor processor... In heterogeneous computing clusters, existing technologies often struggle to perceive the real-time matching quality between tasks at different computational stages and the chip microarchitecture, potentially leading to architectural mismatches and resource waste. To address this issue, this solution uses agent programs deployed on each intelligent chip node to acquire at least one telemetry parameter reflecting the chip's internal architecture's operational status in real time. Specifically, telemetry parameters may include at least one of instruction pipeline stall rate, memory access latency variance, and average utilization of computing units. Acquisition of telemetry parameters is achieved through periodic queries of the monitoring interface provided by the chip driver layer, thus establishing an information path for transmitting the underlying hardware operational status to the scheduling decision layer. After obtaining the real-time telemetry parameters, to transform the multi-dimensional parameters into a unified matching index, the system calculates a dimensionless value called scheduling entropy according to preset rules. The calculation method is as follows: Among them, S 调度 Where is the scheduling entropy; n is the number of types of telemetry parameters; P i,norm w is the normalized value of the i-th telemetry parameter; i The corresponding preset weight coefficients, and the sum of all weight coefficients is 1. Parameter normalization can be achieved using the min-max normalization method. The required maximum and minimum values ​​are determined in advance during the offline benchmark testing phase by running various typical tasks on the current chip. The weight coefficient w iThe determination of w also follows an offline calibration process: run a set of benchmark tasks covering different computing modes, record their performance and telemetry data, and then use statistical methods such as regression analysis to solve for the influence weights of each parameter on performance, and use this to set w. i The scheduling entropy calculation rule is set so that its value is inversely proportional to the degree of matching between the computing task and the chip hardware architecture. For example, negative indicators such as instruction pipeline stall rate have a positive weight, while positive indicators such as computing unit utilization have a negative weight or are reversed during normalization. In this way, a high scheduling entropy value represents a significant architectural mismatch.

[0030] To ensure that each core parameter in this method has a reproducible deterministic basis, its calibration procedure is defined as follows: First, the weight coefficient w in the scheduling entropy formula... i This is generated through an offline regression analysis procedure that utilizes the normalized values ​​P of various telemetry parameters collected from benchmark tests. i,norm A multiple linear regression model is constructed using the matrix of independent variables and the dependent variable, which is the reciprocal of the task execution time. Where, β i The regression coefficients, obtained after solving the model, characterize the impact of the i-th telemetry parameter on performance, while the weighting coefficients w i This is the result after normalizing the absolute values ​​of each regression coefficient, and its calculation formula is: Secondly, the first preset threshold S th1 The scheduling entropy S collected was determined to be the operating scenario where all computing tasks and chip types are matched. 调度 The 90th percentile of the statistical distribution of the sample data; and again, the second preset threshold S as the entropy threshold. th2 The determination of this is based on a quantitative trade-off between the performance loss from architectural mismatch and the overhead of task migration, where the task migration overhead T... mig Defined as the sum of the historical statistical averages of the time required for task state saving, inter-network node transmission, and loading and recovery at the target node, i.e. The performance loss is due to a time dilation factor R. delay (S) is quantified. This factor is derived from regression analysis of mismatch scenario data, representing the functional relationship between scheduling entropy S and task execution time extension rate. This is applied within the preset evaluation time window T. win Within, the expected additional execution time T due to mismatch. win ·(R delay (S)―1) First time exceeding migration cost T mig At this moment, the corresponding scheduling entropy S value is determined as the second preset threshold S. th2 .

[0031] Furthermore, to ensure the integrity and reliability of the scheduling decision chain, the target chip selection and filtering mechanism also follows a clear quantitative procedure, among which the historical affinity score used for selection is used. aff The calculation integrates two dimensions: the operational performance of historical tasks and the degree of architectural fit. Its formula is: Here, T and C refer to the task type and chip type, respectively. and These are the dimensionless values ​​of the historical average performance index and the historical average scheduling entropy after min-max normalization, respectively, under this combination, while w perf With w entropy The system-level weights, summing to 1, are used to regulate the scheduling strategy's tendency to balance pursuing ultimate performance with maintaining architectural stability. Meanwhile, the generation of the health index H, used to filter hardware resources, is based on a quantitative comparison of the chip's response characteristics to probe instructions executed during idle periods with its initial baseline characteristics; its calculation formula is... J here curr With P curr These refer to the standard deviations of the execution latency jitter and peak power consumption fluctuation of the current probe command, respectively, while J... base With P base The baseline parameters of the same name were calibrated when the chip was first connected to the platform, w j With w p These are the weighting coefficients that characterize the importance of the two dimensions of latency and power consumption. Ultimately, when making decisions, the scheduling system will only include chips with a health index H higher than the preset safety threshold into the available resource pool.

[0032] To avoid unnecessary and costly task migrations due to momentary fluctuations, this solution employs a multi-layered entropy control mechanism. This mechanism sets a first preset threshold and a second preset threshold as the entropy threshold. When the real-time calculated scheduling entropy is consistently higher than the first preset threshold but lower than the second preset threshold, the system considers the mismatch not yet severe and does not perform a migration operation. Instead, it prioritizes in-situ optimization mechanisms, such as adjusting the core operating frequency or power consumption limits of the current smart chip to suppress entropy increase. Only when the scheduling entropy consistently exceeds the second preset threshold does the system recognize a severe architectural mismatch that requires migration to resolve. To further ensure the necessity of the migration decision, the migration execution can also be based on an additional judgment condition: an application-layer performance indicator is lower than a preset performance threshold. The application-layer performance indicator can be the number of samples processed per second or the query response rate. Only when both conditions of high scheduling entropy and low application-layer performance are met simultaneously is the migration decision finally confirmed. This cross-validation mechanism binds hardware-level inefficiency with application-level performance loss.

[0033] After confirming the migration, to select the optimal target chip, the system makes a decision based on historical task logs. Specifically, the system queries the logs for performance statistics of historical tasks of the same type as the current task on different types of heterogeneous chips, and selects an available heterogeneous smart chip with the highest historical affinity score as the migration target. Before issuing the migration command, a decision arbitration step is also performed. This step checks whether the migration operation conforms to a preset set of system-level security and management policy rules, such as determining whether the target chip is under maintenance or whether the task data movement violates security policies. Migration is only performed if the rule set is not violated. The timing of task migration is limited to after the computing task completes a full computing batch or a full training cycle to ensure... To ensure the consistency of task states and address potential physical characteristic drift during long-term operation of intelligent chips, this method also introduces a hardware state adaptive adjustment mechanism. This mechanism triggers a micro-benchmark test on the current chip by monitoring abnormal fluctuations in the rate of change of application layer performance residuals, such as the loss function value. Based on the comparison between the actual output and the theoretical output of the test, a compensation coefficient is generated for real-time adjustment of one or more telemetry parameters in subsequent scheduling entropy calculations. In addition, to achieve predictive maintenance, the system can inject standardized probing calculation instructions into the chip during preset idle periods. By capturing and analyzing its response characteristics, such as the jitter value of execution latency, a health index characterizing the hardware health status is calculated, and chips with an index below a safety threshold are excluded from the list of available resources.

[0034] Example 1: In a large-scale model fine-tuning business scenario for financial risk control, a computational task flow is submitted to a heterogeneous computing platform containing GPUs, NPUs, and CPUs. The initial stage of this task flow involves cleaning and preprocessing large-scale text data, characterized by high-intensity I / O throughput. The subsequent model training stage focuses on high-precision floating-point operations, while the final model quantization stage primarily involves dimensionality reduction and low-precision integer calculations. When this task is initially assigned to the GPU node, its computational mode during the data preprocessing stage... Due to hardware architecture mismatch, a large number of dedicated computing units within the GPU chip are idle, and the instruction pipeline frequently stalls due to waiting for data or processing incompatible instructions. This directly results in the inability to fully utilize the performance of this specialized hardware and prolongs the overall execution cycle of the task. In the application of the present invention, when the data preprocessing stage of the fine-tuning task is executed on the GPU node, the agent program deployed on that node obtains real-time data showing that its average utilization rate of computing units remains consistently low, and the memory access latency variance deviates significantly from the baseline. These two telemetry parameters are combined and calculated into a scheduling entropy S that is consistently higher than the second preset threshold. 调度Meanwhile, the system detected that the application layer performance metric of the task, namely the number of samples processed per second, was also lower than its preset performance threshold. The simultaneous satisfaction of the two conditions of high scheduling entropy and low application performance constituted a collaborative verification from hardware status to application performance, and triggered a migration decision accordingly. Subsequently, the scheduler queried the historical task logs and, based on the historical task performance data recorded therein that were of the same type as the current data preprocessing task, determined that the CPU node had the highest historical affinity score. Therefore, after the computing task completed a batch of computing, the system migrated it from the GPU to an available CPU node.

[0035] After a task is migrated to a CPU node, its scheduling entropy S 调度 Once the data quality drops below the threshold, the performance metric of processing samples per second recovers. After data preprocessing is complete, the task enters the model training phase, which focuses on high-precision floating-point operations. During CPU operation, the combination of telemetry parameters again causes the scheduling entropy S to rise. 调度 Once the threshold is exceeded, the system restarts the migration process, migrating the task from the CPU node back to the GPU node best suited for that stage based on historical affinity scores. Similarly, when training completes and enters the model quantization stage, the scheduling entropy mechanism is triggered again, migrating the task to the NPU node with the highest efficiency for low-precision integer calculations. This process demonstrates the resolution of the inherent contradiction between the specialization of dedicated hardware and the variability of complex task flows. This solution, through a unified scheduling entropy metric, enables tasks to dynamically find the dedicated hardware best suited to their current computing mode at different stages, thus allowing the entire heterogeneous cluster to function as a whole, combining the high efficiency of dedicated hardware with the smoothness of complex task flows. The scheduling method enhances the operational flexibility of the platform. It transforms the traditional static allocation problem of seeking a single optimal hardware for the entire lifecycle of a task into a dynamic process of bidirectional selection between tasks and hardware resources based on real-time operational status. Tasks no longer passively accept allocations but actively break away from incompatible hardware architectures through the intrinsic feedback signal of scheduling entropy. The role of the scheduling system changes from a resource allocator to a regulatory mechanism that maintains efficient coupling of the system. Ultimately, the end-to-end execution time of the fine-tuning task in this financial risk control model is shortened, and the average utilization of the three heterogeneous chips (CPU, GPU, and NPU) in the cluster is improved throughout the entire task execution cycle.

[0036] Example 2: To objectively verify the effectiveness of the method of the present invention in improving the resource utilization and task execution efficiency of heterogeneous computing platforms, this example provides a set of comparative test data. The purpose of the test is to quantitatively compare the differences in task completion time and resource matching degree between the cooperative scheduling method of the present invention and the traditional static resource allocation scheduler by running a mixed computing load with multiple stages of different computing characteristics in a controlled environment. The test platform consists of a server configured with a graphics processing unit (GPU), a server configured with a neural network processing unit (NPU), and a server configured with a multi-core general-purpose processor (CPU). The three servers are interconnected through a high-speed network to form a heterogeneous computing cluster.

[0037] The experiment employed a standardized task flow named Hybrid Computational Load (MCW-1), which sequentially comprises three stages: data preprocessing, model training, and model quantization. The experiment included a control group and an experimental group. The control group used a static resource allocation scheduler to assign the entire MCW-1 task flow to the GPU server for execution at once. The experimental group employed the cooperative scheduling method based on scheduling entropy as described in this invention. During the experiment, the sampling period for telemetry parameters was set to 5 seconds. This value was chosen to balance monitoring real-time performance with system overhead. Given that the shortest computation stage in the MCW-1 task flow lasts on the order of minutes, a 5-second sampling period can capture chip state changes caused by task stage switching while keeping the monitoring's own resource consumption at a low level. After the experiment started, the control group's task was executed entirely on the GPU server. In the experimental group, when the initial data preprocessing stage of the task was executed on the GPU node, its scheduling entropy S... 调度 The value rapidly exceeds the second preset threshold required for migration decisions due to architectural mismatch, and is thus migrated to the CPU node with the highest historical affinity score; after entering the model training phase, the scheduling entropy S... 调度 The task was rapidly migrated back to the GPU node after a sharp increase in CPU node performance. Finally, during the model quantization phase, the task was migrated to the NPU node. Table 1 records the key performance data of the two sets of experiments at each stage.

[0038] Table 1: Comparison of performance data of MCW-1 at each stage under two scheduling methods.

[0039] Scheduling methods Task Phase Bearer node Stage duration (seconds) Scheduling entropy (mean) Static allocation Data preprocessing GPU 652 0.85 Static allocation Model training GPU 1830 0.12 Static allocation Model Quantization GPU 415 0.76 Method of the present invention Data preprocessing CPU 288 0.18 Method of the present invention Model training GPU 1845 0.11 Method of the present invention Model Quantization NPU 155 0.15

[0040] Referring to Table 1, the data shows that the static allocation method suffers from architectural mismatch in the data preprocessing and model quantization stages, resulting in average scheduling entropy values ​​as high as 0.85 and 0.76, respectively, leading to longer stage execution times. In contrast, the experimental group using the method of this invention runs its tasks on nodes with high matching degree in each stage, and the average scheduling entropy value of each stage remains low, below 0.2. Consequently, the time consumption of the data preprocessing and model quantization stages is significantly shortened. Although the method of this invention slightly increases the time consumption of the model training stage due to task migration, its end-to-end total time reduction is still significant. The task completion time is significant, a result consistent with the core mechanism of scheduling entropy as a measure of architectural mismatch and driving task migration to matching nodes. Experimental results confirm that, compared to static resource allocation, the cooperative scheduling method based on scheduling entropy proposed in this invention enables multi-stage complex computing tasks to adaptively seek hardware resources that match their current computing mode in heterogeneous environments, thereby shortening the end-to-end completion time of tasks and improving the effective utilization of different types of chips in the entire computing platform. This provides a feasible technical path for solving the resource allocation problem in heterogeneous computing environments.

[0041] Example 3: This example combines Figures 1 to 3 This document describes a method for collaborative scheduling and optimization of heterogeneous resources in an algorithm training platform, such as... Figure 1As shown, the process begins with a computational task running on a specific intelligent chip such as a GPU, NPU, or TPU. The system continuously acquires parameters reflecting the chip's internal architecture's operational status through real-time telemetry sensing, such as instruction pipeline stall rate, memory access latency variance, and computational unit utilization. Subsequently, in the architecture matching quantification stage, the system calculates these multi-dimensional telemetry parameters into a single dimensionless value inversely proportional to the task and chip architecture matching degree—the scheduling entropy Sscheduling—using a preset weighted summation rule. Next, the process enters a multi-level control decision stage. The system compares the real-time value of this scheduling entropy with two preset thresholds to determine the severity of architectural mismatch: if the entropy value is normal, the task continues to run and monitoring continues; if the entropy value is higher than the first threshold but lower than the second threshold, indicating a slight mismatch, the system initiates an in-situ optimization mechanism, adjusting... The current chip's core frequency or power consumption is limited to suppress entropy increase and avoid unnecessary task migration. If the entropy value continues to be higher than the second threshold, it indicates a serious mismatch. The system then enters the migration decision arbitration stage. This stage first checks whether the migration operation to be executed complies with the system-level security and management policies, such as equipment maintenance status or data location policies. If it does not comply, migration is prohibited and monitoring continues. If it complies with the policies, migration is allowed, and the system enters the stage of selecting the optimal migration target. Based on historical task logs, the system queries available heterogeneous chips with the highest historical affinity score with the current task type as migration targets. Finally, after the computing task completes a full computing batch or training cycle, the task migration is executed, and the task is deployed to the newly selected chip to start running. This leads to a new round of telemetry sensing and scheduling entropy evaluation, forming an adaptive control closed loop that continuously seeks the optimal architecture match.

[0042] like Figure 2 As shown in the figure, this graph, in the form of a curve, illustrates the trend of the health index of three different types of intelligent chips, namely GPU-01, NPU-01, and CPU-01, changing over the course of months. The health index in the graph is a quantitative indicator between 0 and 1, with a higher value indicating that the hardware is closer to its ideal initial state. As can be seen from the graph, the health index of all chips decays to varying degrees over time, simulating the natural aging or wear and tear process of hardware during long-term operation. The graph also sets a safety threshold at a health index of 0.75. This threshold is the baseline used by the scheduling system to determine whether the hardware is still suitable for carrying out critical tasks. For example, the health index of the CPU-01 chip shown in the graph begins to fall below this safety threshold after 10 months of operation. Based on this judgment, the scheduling system can exclude the chip from the list of available resources for high-priority tasks or trigger a predictive maintenance alert, thereby effectively avoiding the risk of computational errors or task failures that may be caused by the potential sub-health state of the hardware, and ensuring the operational reliability of the entire computing platform.

[0043] like Figure 3 As shown, the process is collaboratively completed by five logical units: a scheduling system, a health monitoring module, an intelligent chip, an analysis module, and an available resource pool. The process begins when the scheduling system detects that the intelligent chip is in a preset idle period and sends an instruction to the health monitoring module to trigger a health check. The health monitoring module then injects one or more standardized probing computation instructions into the target intelligent chip and executes the test multiple times in a cyclical manner to capture and record the response characteristics of the instructions during multiple executions, such as the returned execution latency and peak power consumption. This time-series data, containing minute variations, is transmitted to the analysis module, which is responsible for calculating its statistical characteristics, such as... The analysis module collects latency jitter and power consumption fluctuation values, and generates a quantified health index based on these characteristics. Then, the analysis module compares this index with a preset safety threshold and makes a decision: if the health index is lower than the safety threshold, the analysis module marks the smart chip as unavailable, notifies the available resource pool to remove it from the pool, and triggers a predictive maintenance request to the scheduling system; otherwise, if the health index is normal, its health status is updated only in the available resource pool, and its available status is maintained. Finally, the analysis module returns the detection results to the scheduling system, thus completing a proactive closed-loop health status assessment and management of a single hardware unit.

[0044] Example 4: In scenarios where the scheduling system of this invention is deployed for the first time on an algorithm training platform or a new type of heterogeneous intelligent chip is introduced into the platform, in order to ensure the effectiveness of scheduling decisions, its core parameters need to be calibrated. This example discloses a specific procedure for standardizing and offline calibrating its core parameters before the system is officially launched. The parameters include those used to calculate the scheduling entropy S. 调度 weighting coefficient w i The procedure first defines a standardized set of benchmark tasks, including a compute-intensive task, a memory access-intensive task, and an I / O-intensive task, to cover representative computing patterns in the subsequent production environment. The next step is to run the entire benchmark task set independently on each type of smart chip in the heterogeneous computing cluster to be calibrated. During each run, the system records the time series of telemetry parameters within the chip, including instruction pipeline stall rate, memory access latency variance, and average utilization of computing units. Simultaneously, it records the end-to-end execution time of each benchmark task as a benchmark performance indicator for that run. After data acquisition, the procedure enters the weighting coefficient w... iThe determination phase aims to establish a quantitative relationship between telemetry parameters and operational performance. The specific steps are as follows: using all collected telemetry parameter-task execution time data pairs as input, a multiple linear regression model is constructed, where the normalized value P of each telemetry parameter is... i,norm P is the independent variable in the model, while the reciprocal of the task execution time is the dependent variable; by solving this regression model, each telemetry parameter P can be obtained. i,norm The quantitative impact coefficients on task execution efficiency, after being normalized, are used as the formula for calculating scheduling entropy. The weighting coefficient w in i .

[0045] Next, the procedure utilizes the determined weighting coefficients w i The collected data then enters a calibration phase with two key thresholds. First, the procedure filters out all data points that match the computing tasks with the chip type. This matching relationship is predefined based on the computing mode of the task and the hardware characteristics of the chip, and the scheduling entropy S under these matching scenarios is calculated. 调度 The distribution of values ​​is then analyzed; subsequently, the 90th percentile of this distribution is determined as the first preset threshold; next, the procedure further analyzes the data under all mismatch scenarios and establishes the scheduling entropy S. 调度 A model relating the value to the task execution time extension rate is established, and combined with a pre-quantified average time cost for performing cross-node task migration, the scheduling entropy critical point is calculated when the performance loss caused by mismatch is equal to the cost of performing a migration operation. This critical point is then determined as the second preset threshold. After calibration, all determined weight coefficients and threshold parameters are written into the scheduling system's configuration file. Subsequently, when the platform is officially running, every decision based on scheduling entropy is based on empirical quantification of its own hardware environment and load characteristics.

[0046] Example 5: In a cloud computing platform deployed in a multi-tenant environment that must comply with data security and service level agreements, to ensure that the migration decision of this invention conforms to the upper-level management strategy, the scheduling system executes a set of policy rule set loading and compilation procedures during the initialization phase. In this procedure, the platform administrator, based on security and management requirements, enters specific policy rules into a preset rule base in the form of structured data. Each rule defines clear triggering conditions and corresponding adjudication actions. The conditions cover task attributes such as data security level, business priority, and target chip status such as maintenance mode and physical location. The adjudication actions are limited to allowing or prohibiting migration. When the scheduling system starts, it reads this rule base and compiles all rules into an efficient, real-time queryable in-memory lookup table. Each subsequent migration arbitration step obtains the final adjudication result by matching the migration operation to be executed with the conditions in this lookup table.

[0047] When adding a batch of intelligent chips to this heterogeneous computing platform, to ensure the accuracy of subsequent hardware health status monitoring, each new chip must undergo a hardware baseline characteristic calibration process before being officially added to the available resource list. This process places the chip under test in a standard, no-load initial state and continuously injects a standardized probing computation instruction into it a preset number of times. During this process, the system captures and records the response characteristics of each instruction execution, namely the jitter value of execution latency and the fluctuation value of peak power consumption. After all the injected instructions have been executed, the system calculates the statistical distribution parameters of these response characteristic time series data, including the mean and standard deviation, and stores these parameters as the chip's unique, initial hardware health status baseline in the system's device file. The chip's health index is then obtained by quantitatively comparing its current response characteristics with this initial baseline.

[0048] Example 6: In a heterogeneous computing platform where the scheduling system of this invention has been deployed and is continuously running, to further improve its long-term decision-making adaptability and stability, this example discloses a set of online model optimization and operational fault-tolerant procedures. After the system has accumulated sufficient historical task logs, the scheduling system will periodically perform parameter optimization for the historical affinity score calculation model. Specifically, the system extracts logs of all completed tasks, which include task type, actual chip type carried at each stage, and corresponding operational performance indicators. The system uses a cross-validation method to iteratively adjust the weight coefficients in the historical affinity score calculation formula to find a set of weight coefficients that maximize the match between the optimal chip predicted based on affinity score in historical data and the chip with the highest actual operational performance. This set of coefficients is then updated in the online scheduling model. Regarding compensation for physical drift in hardware computing characteristics, when the system obtains the theoretical output O of the hardware through micro-benchmark testing... theory With actual output O actual Then, its compensation coefficient C comp It was determined to be C comp =O theory / O actual In subsequent calculations of scheduling entropy, telemetry parameters related to computing power will be multiplied by this compensation coefficient to quantify and correct the physical degradation of hardware performance.

[0049] The security threshold corresponding to the hardware health index is set in relation to the business importance level of the task to be scheduled. For routine offline analysis tasks, the security threshold can be set to the 5th percentile of the health index distribution of all chips of the same type in the current cluster to balance resource utilization and stability. For computing tasks in security-critical domains, the security threshold is set at a more stringent level, such as the 25th percentile, to exclude any hardware units with potential instability risks from the list of available resources in advance. To cope with potential failures during task migration, the migration operation is designed as a transactional process with a defined state. Before entering the transmission state, the scheduler will perform a final availability and health index check on the target chip. If the check fails, the migration operation is canceled and re-enters the scheduling queue to wait for a new target chip to be selected. If the migration fails during transmission or submission due to a sudden failure of the target chip or network interruption, the system will automatically trigger a rollback mechanism to restore the computing task to the most recent consistent state checkpoint saved on the source chip and mark the target chip as abnormal, thus forming a fault-tolerant mechanism to deal with migration failures.

[0050] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0051] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for heterogeneous resource collaborative scheduling and optimization of an algorithm training platform, characterized in that, Includes the following steps: Step a: Real-time acquisition of at least one telemetry parameter running inside the intelligent chip carrying the computing task. The telemetry parameter reflects the operating status of the internal architecture of the intelligent chip. The intelligent chip is one of a graphics processor, a neural network processor, and a tensor processor. Step b: Calculate a scheduling entropy based on at least one real-time telemetry parameter obtained; the calculation rule for the scheduling entropy is preset so that its value is inversely proportional to the degree of matching between the computing mode of the computing task and the hardware architecture of the smart chip. Step c: Determine whether the real-time calculated scheduling entropy is continuously higher than a preset entropy threshold. In step d, if the judgment result is yes, it is determined that there is an architectural mismatch between the computing task and the current intelligent chip. After the computing task completes a computing batch or a training cycle, the computing task is migrated from the current intelligent chip to a currently available heterogeneous intelligent chip with a different chip type than the current intelligent chip.

2. The heterogeneous resource collaborative scheduling and optimization method for an algorithm training platform according to claim 1, characterized in that, At least one telemetry parameter, including at least one of instruction pipeline stall rate, memory access latency variance, and average utilization of computing units.

3. The heterogeneous resource collaborative scheduling and optimization method for an algorithm training platform according to claim 1, characterized in that, The scheduling entropy in step b is calculated by normalizing at least one real-time telemetry parameter and then performing a weighted summation to obtain a dimensionless value. The calculation method is as follows: Among them, S 调度 The scheduling entropy is n; the number of types of real-time telemetry parameters acquired is n; P is the scheduling entropy. i,norm w is the normalized value of the i-th real-time telemetry parameter; i These are the preset weighting coefficients corresponding to the i-th real-time telemetry parameter, and the sum of all weighting coefficients is 1.

4. The heterogeneous resource collaborative scheduling and optimization method for an algorithm training platform according to claim 1, characterized in that, The method also includes: when the scheduling entropy is continuously higher than a first preset threshold and lower than a second preset threshold which is the entropy threshold, the migration operation in step d is not performed, but in response to the scheduling entropy being in the threshold range, the core operating frequency of the current smart chip or its power consumption limit is adjusted.

5. The method for heterogeneous resource collaborative scheduling and optimization of an algorithm training platform according to claim 1, characterized in that, The heterogeneous smart chip migrated to in step d is determined based on the performance statistics of historical tasks of the same type as the current computing task recorded in the historical task log on different types of heterogeneous smart chips. It is a usable heterogeneous smart chip with the highest historical affinity score.

6. The method for heterogeneous resource collaborative scheduling and optimization of an algorithm training platform according to claim 1, characterized in that, The execution of step d is also based on an additional judgment condition, namely, that one of the application layer performance indicators of the computing task is lower than a preset performance threshold; the application layer performance indicator is one of the number of samples processed per second and the query response rate.

7. The method for heterogeneous resource collaborative scheduling and optimization of an algorithm training platform according to claim 1, characterized in that, Before step d, there is a decision arbitration step, which includes checking whether the migration operation conforms to a preset system-level security and management policy rule set. The migration is only performed if the migration operation does not violate the rule set. The rule set includes rules for determining whether the target chip is in maintenance mode and rules for determining whether task data is allowed to move across nodes.

8. The method for heterogeneous resource collaborative scheduling and optimization of an algorithm training platform according to claim 1, characterized in that, The method further includes: while executing step b, monitoring the application-layer performance residual of the computing task in real time, where the performance residual is the rate of change of the loss function value; when it is determined that the performance residual fluctuates continuously above the statistical baseline, a preset micro-benchmark test is performed on the current intelligent chip during the computing interval of the computing task, and a compensation coefficient is generated based on the comparison between the actual output and the theoretical output of the test to quantify the physical drift of the current hardware computing characteristics; when executing step b in a subsequent step, the generated compensation coefficient is used to adjust one or more parameters used to calculate the scheduling entropy in real time.

9. The method for heterogeneous resource collaborative scheduling and optimization of an algorithm training platform according to claim 1, characterized in that, The method also includes: injecting a standardized probing computation instruction into the smart chip during a preset idle period; capturing and recording the response characteristics of the probing computation instruction during multiple executions, including the jitter value of execution latency and the fluctuation value of peak power consumption; calculating a health index characterizing the hardware health status of the smart chip based on the time series data of the response characteristics; and excluding smart chips with a health index lower than a safety threshold from the list of available resources in subsequent scheduling decisions.

10. The method for heterogeneous resource collaborative scheduling and optimization of an algorithm training platform according to claim 1, characterized in that, The timing of migrating the computing task from the current smart chip in step d is limited to after the computing task has completed a full batch of computing processing or after the computing task has completed a full training cycle.

Images