Large-scale heterogeneous computing power cluster training and pushing acceleration method

Abstract

The invention discloses a large-scale heterogeneous computing power cluster training and pushing acceleration method, which comprises the following steps: S1, collecting operation characteristics of each node of a heterogeneous cluster, and constructing an equipment state parameter set; s2, loading a training and reasoning model, constructing an intermediate representation graph structure, and generating graph structure information; s3, inputting the graph structure information and the equipment state parameter set into a scheduling strategy model constructed based on a neural network ordinary differential equation, and generating an optimization decision result; s4, reconstructing a model graph according to an optimization decision result, and dividing the model graph into a training sub-graph and a reasoning sub-graph; s5, mapping the sub-graphs to heterogeneous nodes, performing resource matching and task distribution according to the equipment state parameter set, and generating a scheduling result; s6, collecting performance feedback information; and S7, inputting feedback information into the scheduling model to update parameters, and adjusting evolution function strategy parameters. According to the method, the scheduling acceleration of the training and pushing task of the deep model in the heterogeneous cluster is realized.

Description

Technical Field

[0001] The present invention relates to the field of computing resource scheduling and heterogeneous computing acceleration technology, and in particular to a large-scale heterogeneous computing cluster training and acceleration method. Background Art

[0002] With the continued growth of artificial intelligence models and the continuous expansion of application scenarios, the demand for computing resources for deep neural network training and inference tasks is increasing. In the process of deploying large-scale models, traditional single computing platforms can no longer meet the requirements of diverse performance indicators. Therefore, more and more system architectures are adopting heterogeneous computing clusters, integrating GPUs, CPUs, TPUs, FPGAs, and other specialized accelerators to achieve efficient scheduling and resource sharing for training and inference tasks. In such heterogeneous environments, the training model structure is complex, the task graph is large, and the node dependencies are close. Various operators have significant differences in execution efficiency, communication latency, memory bandwidth, and power consumption characteristics, making the unified scheduling of training and inference tasks a major challenge.

[0003] Existing heterogeneous computing resource scheduling methods mostly use static scheduling strategies or rule-based allocation mechanisms. Such methods usually rely on pre-set hardware capability classifications and task type classifications, and use heuristic algorithms to complete operator partitioning and node mapping before resource scheduling. However, such methods lack the ability to perceive the operating status of the equipment in real time, and are difficult to adapt to performance jitter caused by load changes, communication bottlenecks, or equipment fluctuations during the scheduling process. This can easily lead to low resource utilization, high task execution latency, and insufficient scheduling accuracy. At the same time, due to the lack of dynamic feedback mechanisms and policy optimization paths, existing scheduling systems find it difficult to adaptively evolve strategies based on actual operating results, which restricts the application effect of scheduling systems in large-scale and complex scenarios.

[0004] Furthermore, while some existing neural network acceleration scheduling solutions have introduced graph neural networks to model task graph structures, most use standard graph convolution or attention mechanisms for graph feature aggregation. These methods have limited expressive power and are unable to capture the continuous dynamic changes in scheduling strategies over time. This discrete state mapping approach cannot reflect the differential evolution of inter-node transfer behavior, making it difficult to form a highly accurate and highly generalizable scheduling strategy model. Existing models, in particular, have little impact on improving scheduling accuracy and stability in scenarios with multiple concurrent tasks and frequent resource conflicts.

[0005] When it comes to scheduling strategy optimization, traditional methods typically use a fixed cost function to evaluate scheduling results for a single objective. This fails to fully integrate task execution latency, resource utilization, energy efficiency, and scheduling adaptability, and lacks a multi-objective trade-off mechanism. Furthermore, some strategy evolution mechanisms employ fixed threshold adjustments or periodic parameter updates, with update frequencies disconnected from the actual execution of training tasks and unable to achieve closed-loop optimization based on performance feedback.

[0006] Therefore, how to provide a large-scale heterogeneous computing power cluster training and push acceleration method is a problem that technical personnel in this field urgently need to solve. Summary of the Invention

[0007] One objective of the present invention is to propose a method for accelerating training and inference on large-scale heterogeneous computing power clusters. Combining a neural network ordinary differential equation modeling mechanism with a multi-objective scheduling optimization strategy, the present invention describes in detail how to dynamically construct a scheduling strategy based on device state parameters and performance feedback information within a complex task graph structure, and achieve adaptive updates of the scheduling function through evolutionary optimization. This method offers the advantages of high task mapping accuracy, excellent resource utilization efficiency, and an evolvable scheduling strategy, making it suitable for accelerating deep learning training and inference tasks on heterogeneous computing power platforms.

[0008] A large-scale heterogeneous computing power cluster training and inference acceleration method according to an embodiment of the present invention includes the following steps: S1. Collect the operating characteristics of each computing node in the heterogeneous computing power cluster and build a set of device status parameters; S2. Load the deep learning model to be trained and inferred, build a unified intermediate representation graph structure, and generate model graph structure information; S3. Inputting the model graph structure information and the device state parameter set into a scheduling strategy model constructed based on a neural network ordinary differential equation, wherein the scheduling strategy model includes a state representation layer, a differential evolution function, an ordinary differential equation solver, and a decision decoding module to generate a graph optimization decision result; S4. Perform a structural reconstruction operation on the model graph structure according to the graph optimization decision result to generate an optimized graph structure, and divide the optimized graph structure into a training subgraph and an inference subgraph; S5. Map the training subgraph and the inference subgraph to multiple computing nodes in the heterogeneous computing power cluster, perform resource matching and task distribution based on the characteristics of each node in the device state parameter set, and form a training and inference collaborative scheduling execution result; S6. Continuously collect performance feedback information during the training and propulsion collaborative scheduling process; S7. Input the performance feedback information into the scheduling strategy model to update the parameters and adjust the scheduling strategy model parameters in the differential evolution function.

[0009] Optionally, the device status parameter set includes a computing capability indicator, a memory bandwidth indicator, a communication delay indicator, a supported precision type identifier, and a current power consumption level; The computing capability indicator includes the number of floating-point operations that can be performed per second; the memory bandwidth is expressed as the maximum data transmission rate per unit time; the communication delay is the average single round-trip time between the node and the central scheduler or other nodes; the supported precision type identifier is used to identify the architecture category to which the computing node belongs; the power consumption level is expressed as the power consumption per unit time at the current moment.

[0010] Optionally, the S2 specifically includes: S21. Load the model to be trained and the model to be inferred, extract the network structure information of the training model and the network structure information of the inference model respectively, and construct a training model structure diagram and an inference model structure diagram, wherein the training model structure diagram and the inference model structure diagram both include an operator node set, an operator connection relationship, and an operator parameter attribute; S22. Perform a structural-level merger on the training model structure graph and the inference model structure graph to generate a joint model graph structure. The joint model graph structure uses operator nodes as nodes in the graph and directed dependencies between operators as edges in the graph to form unified model graph structure information. S23. For each operator node in the unified model graph structure information , extract the structural attribute vector , the structural attribute vector includes the parameter size , memory usage , Number of input channels , Number of output channels and operator type identifier ; S24. Arrange the structural attribute vectors of all operator nodes in the order of node numbers to form a node feature matrix , where n represents the total number of operator nodes in the unified model graph structure information; S25, extract the node connection relationship in the unified model graph structure information, generate the connection relationship matrix A, and combine the node feature matrix G with the connection relationship matrix A to form the graph structure information .

[0011] Optionally, the S3 specifically includes: S31. Receive graph structure information and the device state parameter set, where A is the connection relationship matrix A and G is the node feature matrix; S32, input the node feature matrix G into the state representation layer to generate the initial graph structure state representation ; S33, mapping the device state parameter set R to the control variable function c(t), expressing the dynamic evolution characteristics of the device state in a time series manner; S34. Constructing structural state evolution function ,in is the structural state variable, c(t) is the control variable function, is the neural network parameter set; S35, input the differential evolution function into the ordinary differential equation solver, use the numerical integration method to solve the state evolution process, and obtain the structural state in the time interval The evolutionary results ; S36. Input the structural state evolution result into the decoding module to generate a graph optimization decision result, which includes path adjustment instructions, node fusion and replacement instructions, and calculation accuracy switching strategy.

[0012] Optionally, the S34 specifically includes: S341, set the structure state variable , where n is the number of nodes in the unified model graph structure, and d is the state representation dimension of each node; S342, set control variable function , where N is the number of device nodes and k is the dimension of each device state vector; S343, constructing a differential evolution function, wherein the differential evolution function is composed of a set of neural network parameters control: ; Among them, h(t) is the structural state variable, t is the time variable, is the weight matrix of the first layer, is the weight matrix of the second layer, is the activation function, is the output layer activation function, is the control variable function, is the bias vector of the first layer of neural network, is the bias vector of the second layer neural network, Represents a tensor concatenation operation; S344. Combining the differential evolution function with the structural state variables and the control variable functions to form a structural state evolution expression for characterizing the continuous dynamic evolution process of the unified model graph structure driven by the device state.

[0013] Optionally, the S4 specifically includes: S41, receiving a graph optimization decision result, and generating a structure reconstruction control instruction set, wherein the structure reconstruction control instruction set includes a path adjustment instruction set, a node fusion and replacement instruction set, and a calculation accuracy switching strategy set; S42: Perform a topological update on the connection relationship matrix A in the unified model graph structure according to the path adjustment instruction set to generate a connection relationship matrix after path adjustment. ; S43. According to the node fusion and replacement instruction set, identify the node subset to be fused in the node feature matrix G, generate the fused node and the corresponding structure attribute vector, delete the original node attribute vector and add the fused structure attribute vector, and generate the node feature matrix after the node fusion. ; S44, according to the calculation accuracy switching strategy set, the node feature matrix after node fusion Each structural attribute vector in Perform precision replacement operations based on computing node resource status indicators Update the precision label and generate the node feature matrix after precision switching ; S45. Connection relationship matrix based on path adjustment Node feature matrix after switching with precision , construct the graph structure information after structural reconstruction; S46. Based on the task type division rule set, perform a functional graph division operation on the graph structure information after the structure reconstruction to generate a training subgraph and a reasoning subgraph.

[0014] Optionally, the S5 specifically includes: S51, respectively receive the training subgraph, the reasoning subgraph and the device state parameter set, and for each node in the training subgraph Extract structural attribute vector , the node demand vector and the device state parameter vector are input into the resource matching function respectively; S52, perform maximum matching scheduling on all training nodes, and select The largest device node i generates a training node scheduling mapping set; S53, repeating steps S51 and S52, performing the same resource matching calculation and allocation on the nodes in the reasoning subgraph, and generating a reasoning node scheduling mapping set; S54: Deploy the training subgraph and the inference subgraph to the corresponding computing nodes according to the training node scheduling mapping set and the inference node scheduling mapping set, initialize the task execution environment, and complete the training and inference collaborative scheduling execution operation; S55. Output the training-pushing collaborative scheduling execution result, which includes the task deployment status of each computing node, the data input binding relationship, and the execution start flag, for subsequent performance feedback collection and strategy update.

[0015] Optionally, the resource matching function: ; in, is the scheduling fitness score, The k-th operator node in the training subgraph or inference subgraph, The device state vector of the i-th computing node, To match the weights for computing power, is the computing capability indicator, is the memory bandwidth indicator, is the communication delay indicator, To support precision type identification, is the current power consumption level, is the memory bandwidth matching weight, is the communication delay penalty weight, is the power consumption penalty weight, Match weights for precision types, For nodes The computational requirements, For nodes Memory requirements, is the type matching indicator function, For nodes The precision type identifier.

[0016] Optionally, the performance feedback information includes execution delay, resource utilization, energy efficiency ratio and scheduling accuracy score; the execution delay represents the time interval from the start to the completion of the task node, which is used to measure the task execution efficiency; the resource utilization represents the proportion of computing resources occupied on the corresponding computing node during task execution, including CPU occupancy and memory occupancy; the energy efficiency ratio represents the task completion efficiency under unit power consumption, which is used to evaluate the energy utilization efficiency of the task on a specific computing node; the scheduling accuracy score represents the degree of match between the structural attributes of the task node and the state parameters of the deployed computing node, which is used to reflect the accuracy of the current scheduling strategy.

[0017] Optionally, the S7 specifically includes: S71. Collect performance feedback information set, task graph structure information, device state parameter set, and current scheduling function weight parameter set to construct a strategy evolution input sample set; S72. Based on the evolutionary input sample set, define the scheduling optimization objective function: ; Among them, J is the scheduling optimization objective function, K is the total number of task nodes in the constructed task graph structure, k is the task node index in the task graph structure, is the weight coefficient of the inverse ratio of delay, is the actual execution delay of the kth task node, is the weight coefficient of the resource utilization term, is the resource utilization of the kth task node, is the weight coefficient of the energy efficiency ratio term, is the energy efficiency ratio of the kth task node, is the weight coefficient of the scheduling accuracy scoring item, Score the scheduling accuracy of the kth task node; S73, based on the evolutionary input sample set and the scheduling optimization objective function J, perform evolutionary optimization operations, use genetic algorithms or differential evolution algorithms to optimize the current scheduling function weight parameter set Perform multiple rounds of population updates and fitness evaluations to obtain the updated scheduling function weight parameter set, which is recorded as the evolutionary update weight set; S74. Applying the evolution update weight set to the scheduling score function In the process, the resource mapping relationship of the task nodes is updated with the scheduling strategy to complete the evolution and iteration of the scheduling function.

[0018] The beneficial effects of the present invention are: The present invention realizes the dynamic modeling and optimization of the relationship between task graph structure, device state parameters and scheduling strategy by constructing a large-scale heterogeneous computing power cluster training and acceleration method. In terms of scheduling strategy modeling, a scheduling function model based on neural network ordinary differential equations is introduced, which can capture the evolution relationship between task characteristics and resource status in the continuous time domain. Compared with the traditional static graph model, it has stronger state expression ability and scheduling timing adaptability. In the task and resource matching mechanism, by clearly defining multi-dimensional state parameters such as device computing power, memory bandwidth, communication delay, and power consumption level, and inputting them into a unified scheduling score function, the accuracy of resource selection is effectively improved.

[0019] This paper also introduces a performance feedback-driven strategy evolution mechanism into the scheduling strategy optimization process. This mechanism comprehensively considers task execution latency, resource utilization, energy efficiency, and scheduling accuracy scores, constructs a multi-objective optimization function, and uses an evolutionary algorithm to iteratively update weight parameters, thereby achieving closed-loop adaptive optimization of the scheduling strategy based on actual execution results. This mechanism enhances the system's robustness to complex task structures and dynamic resource changes, improving the overall performance of task scheduling.

[0020] Through this approach, the present invention achieves precise matching and dynamic mapping between task nodes and computing resources for large-scale, complex, and resource-heterogeneous training and inference tasks, improving the intelligent level of scheduling response and global resource utilization efficiency. Compared with existing methods, this invention achieves functional expansion and improved results in terms of scheduling strategy modeling depth, performance feedback utilization efficiency, and policy update mechanisms. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 This is a flowchart of the large-scale heterogeneous computing power cluster training and push acceleration method proposed by the present invention; Figure 2 This is a schematic diagram of the scheduling function model structure based on the neural network ordinary differential equation in the large-scale heterogeneous computing power cluster training and push acceleration method proposed by the present invention; Figure 3 This is a schematic diagram of the joint modeling of the task graph structure and heterogeneous device status information in the large-scale heterogeneous computing power cluster training and push acceleration method proposed in this invention. DETAILED DESCRIPTION

[0022] The present invention will now be described in further detail with reference to the accompanying drawings, which are simplified schematic diagrams that illustrate the basic structure of the present invention in a schematic manner.

[0023] refer to Figure 1-3 The training and induction acceleration method for a large-scale heterogeneous computing cluster includes the following steps: S1. Collect the operating characteristics of each computing node in the heterogeneous computing power cluster and build a set of device status parameters; S2. Load the deep learning model to be trained and inferred, build a unified intermediate representation graph structure, and generate model graph structure information; S3. Inputting the model graph structure information and the device state parameter set into a scheduling strategy model constructed based on a neural network ordinary differential equation, wherein the scheduling strategy model includes a state representation layer, a differential evolution function, an ordinary differential equation solver, and a decision decoding module to generate a graph optimization decision result; S4. Perform a structural reconstruction operation on the model graph structure according to the graph optimization decision result to generate an optimized graph structure, and divide the optimized graph structure into a training subgraph and an inference subgraph; S5. Map the training subgraph and the inference subgraph to multiple computing nodes in the heterogeneous computing power cluster, perform resource matching and task distribution based on the characteristics of each node in the device state parameter set, and form a training and inference collaborative scheduling execution result; S6. Continuously collect performance feedback information during the training and propulsion collaborative scheduling process; S7. Input the performance feedback information into the scheduling strategy model to update the parameters and adjust the scheduling strategy model parameters in the differential evolution function.

[0024] The large-scale heterogeneous computing power cluster training and inference acceleration method proposed in the present invention can effectively realize the coordinated scheduling of deep learning model training and inference in a complex cluster environment. By constructing a set of device state parameters and model graph structure information, integrating neural network ordinary differential equations to build a scheduling strategy model, it can dynamically generate optimized structural division and resource allocation schemes. This method introduces a performance feedback mechanism in the process of task division, resource mapping and scheduling strategy update, so that the scheduling model has continuous optimization capabilities. Practical application results show that this method can significantly reduce average execution delay, improve resource utilization, and reduce overall power consumption while maintaining model accuracy. It has good stability, generalization ability and scheduling intelligence, and is suitable for intelligent training and inference task management in large-scale heterogeneous environments.

[0025] In this embodiment, the device status parameter set includes a computing capability indicator, a memory bandwidth indicator, a communication delay indicator, a supported precision type identifier, and a current power consumption level; The computing capability indicator includes the number of floating-point operations that can be performed per second; the memory bandwidth is expressed as the maximum data transmission rate per unit time; the communication delay is the average single round-trip time between the node and the central scheduler or other nodes; the supported precision type identifier is used to identify the architecture category to which the computing node belongs; the power consumption level is expressed as the power consumption per unit time at the current moment.

[0026] This paper constructs a set of device state parameters to comprehensively describe the performance characteristics of heterogeneous computing nodes, including computing power, memory bandwidth, communication latency, supported precision types, and power consumption levels. This allows scheduling strategies to accurately match model task requirements with hardware resource capabilities. During dynamic scheduling, resource allocation is performed based on multi-dimensional indicators, effectively improving computing resource utilization, reducing execution latency for inference and training tasks, and optimizing overall power consumption without sacrificing performance.

[0027] In this embodiment, S2 specifically includes: S21. Load the model to be trained and the model to be inferred, extract the network structure information of the training model and the network structure information of the inference model respectively, and construct a training model structure diagram and an inference model structure diagram, wherein the training model structure diagram and the inference model structure diagram both include an operator node set, an operator connection relationship, and an operator parameter attribute; S22. Perform a structural-level merger on the training model structure graph and the inference model structure graph to generate a joint model graph structure. The joint model graph structure uses operator nodes as nodes in the graph and directed dependencies between operators as edges in the graph to form unified model graph structure information. S23. For each operator node in the unified model graph structure information , extract the structural attribute vector , the structural attribute vector includes the parameter size , memory usage , Number of input channels , Number of output channels and operator type identifier ; S24. Arrange the structural attribute vectors of all operator nodes in the order of node numbers to form a node feature matrix , where n represents the total number of operator nodes in the unified model graph structure information; S25, extract the node connection relationship in the unified model graph structure information, generate the connection relationship matrix A, and combine the node feature matrix G with the connection relationship matrix A to form the graph structure information .

[0028] The present invention constructs a unified graph structure representation by extracting network structure information from the training model and the inference model, thereby improving the expression consistency between the model structure and the computing task. During the graph composition process, the parameter scale, memory usage, number of input and output channels, and type identification of each operator node are extracted as structural attribute vectors, and the node feature matrix is composed in sequence; at the same time, a connection relationship matrix is generated based on the connection relationship between operators. Finally, the graph structure information is combined, and the graph structure is represented as a structure composed of a node feature matrix and a connection relationship matrix, which can provide a clear and standardized model structure input for subsequent scheduling strategies. The above method helps to unify the model representation, reduce structural conversion loss, and improve the efficiency and compatibility of scheduling modeling.

[0029] In this embodiment, S3 specifically includes: S31. Receive graph structure information and the device state parameter set, where A is the connection relationship matrix A and G is the node feature matrix; S32, input the node feature matrix G into the state representation layer to generate the initial graph structure state representation ; S33, mapping the device state parameter set R to the control variable function c(t), expressing the dynamic evolution characteristics of the device state in a time series manner; S34. Constructing structural state evolution function ,in is the structural state variable, c(t) is the control variable function, is the neural network parameter set; S35, input the differential evolution function into the ordinary differential equation solver, use the numerical integration method to solve the state evolution process, and obtain the structural state in the time interval The evolutionary results ; S36. Input the structural state evolution result into the decoding module to generate a graph optimization decision result, which includes path adjustment instructions, node fusion and replacement instructions, and calculation accuracy switching strategy.

[0030] The present invention models the state evolution process of the graph structure by introducing a neural network ordinary differential equation mechanism, thereby realizing the expansion of the scheduling strategy from static rules to continuous-time dynamics. Specifically, the graph structure node feature matrix is input into the state representation layer to generate the initial state representation, and the device state parameter set is mapped into a time control variable function to form a differential evolution formula, that is, the derivative of the structural state with respect to time is equal to the joint mapping function of the structural state and the control variable. This formula is used to describe the continuous evolution path of the structural state driven by the resource state, and then solved by a numerical solver within a specified time interval to obtain the final state structural representation. Finally, the evolution result is input into the decision module to generate the scheduling optimization result. This mechanism can improve the accuracy of structural modeling, enable the scheduling strategy to have continuity, differentiability and feedback tuning capabilities, and enhance the adaptability in complex heterogeneous scenarios.

[0031] In this embodiment, the S34 specifically includes: S341, set the structure state variable , where n is the number of nodes in the unified model graph structure, and d is the state representation dimension of each node; S342, set control variable function , where N is the number of device nodes and k is the dimension of each device state vector; S343, constructing a differential evolution function, wherein the differential evolution function is composed of a set of neural network parameters control: ; Among them, h(t) is the structural state variable, t is the time variable, is the weight matrix of the first layer, is the weight matrix of the second layer, is the activation function, is the output layer activation function, is the control variable function, is the bias vector of the first layer of neural network, is the bias vector of the second layer neural network, Represents a tensor concatenation operation; S344. Combining the differential evolution function with the structural state variables and the control variable functions to form a structural state evolution expression for characterizing the continuous dynamic evolution process of the unified model graph structure driven by the device state.

[0032] The present invention constructs a differential evolution function based on neural network parameter control, which is used to model the continuous change process of the graph structure state in the time domain. By representing the structural state variables as multidimensional vectors and the device state as the time control variable, a system of ordinary differential equations is formed, in which the derivative expression is controlled by a two-layer neural network. The specific form is a weighted combination of the current state and the control input, and the evolution direction is output after the activation function transformation. This mechanism can realize the continuous modeling of state representation and has differentiability and learnability. Compared with the traditional static scheduling model, this method can still stably output the structural evolution trajectory under the dynamic fluctuation of resources, significantly improving the adaptability and reasoning robustness of the task scheduling strategy to changes in heterogeneous environments.

[0033] In this embodiment, the S4 specifically includes: S41, receiving a graph optimization decision result, and generating a structure reconstruction control instruction set, wherein the structure reconstruction control instruction set includes a path adjustment instruction set, a node fusion and replacement instruction set, and a calculation accuracy switching strategy set; S42: Perform a topological update on the connection relationship matrix A in the unified model graph structure according to the path adjustment instruction set to generate a connection relationship matrix after path adjustment. ; S43. According to the node fusion and replacement instruction set, identify the node subset to be fused in the node feature matrix G, generate the fused node and the corresponding structure attribute vector, delete the original node attribute vector and add the fused structure attribute vector, and generate the node feature matrix after the node fusion. ; S44, according to the calculation accuracy switching strategy set, the node feature matrix after node fusion Each structural attribute vector in Perform precision replacement operations based on computing node resource status indicators Update the precision label and generate the node feature matrix after precision switching ; S45. Connection relationship matrix based on path adjustment Node feature matrix after switching with precision , construct the graph structure information after structural reconstruction; S46. Based on the task type division rule set, perform a functional graph division operation on the graph structure information after the structure reconstruction to generate a training subgraph and a reasoning subgraph.

[0034] The present invention realizes the dynamic optimization and adjustment of the unified model graph structure by introducing a set of structural reconstruction control instructions. The instruction set includes three types of operations: path adjustment, node fusion and replacement, and calculation accuracy switching, which correspond to data flow topology optimization, structural redundancy compression and computing resource adaptation respectively. By updating the connection relationship matrix, the execution path between task nodes can be optimized, and the parallelism and execution efficiency can be improved; by node fusion and attribute update, redundant operation nodes can be reduced, and the graph scale and resource occupancy can be reduced; by replacing the precision label, combined with the computing node resource status, the appropriate operation accuracy can be automatically matched to effectively control power consumption and delay. Finally, the reconstructed graph structure is functionally partitioned in combination with the graph partitioning rules to generate training subgraphs and reasoning subgraphs, providing a refined structural foundation for efficient task scheduling and collaborative execution in heterogeneous cluster environments. This process improves the flexibility and adaptability of the scheduling strategy and enhances the system's ability to cope with model complexity and resource differences.

[0035] In this embodiment, the S5 specifically includes: S51, respectively receive the training subgraph, the reasoning subgraph and the device state parameter set, and for each node in the training subgraph Extract structural attribute vector , the node demand vector and the device state parameter vector are input into the resource matching function respectively; S52, perform maximum matching scheduling on all training nodes, and select The largest device node i generates a training node scheduling mapping set; S53, repeating steps S51 and S52, performing the same resource matching calculation and allocation on the nodes in the reasoning subgraph, and generating a reasoning node scheduling mapping set; S54: Deploy the training subgraph and the inference subgraph to the corresponding computing nodes according to the training node scheduling mapping set and the inference node scheduling mapping set, initialize the task execution environment, and complete the training and inference collaborative scheduling execution operation; S55. Output the training-pushing collaborative scheduling execution result, which includes the task deployment status of each computing node, the data input binding relationship, and the execution start flag, for subsequent performance feedback collection and strategy update.

[0036] The present invention jointly models the node structure attributes and device status parameters in the training subgraph and the reasoning subgraph by constructing a resource matching function, thereby achieving the optimal mapping between task nodes and heterogeneous computing nodes. Maximum matching scheduling is performed separately in the training and reasoning stages, effectively improving the task resource adaptation accuracy. By generating a scheduling mapping set, each subgraph task is finely allocated to the computing node, ensuring a reasonable match between computing power, memory bandwidth and power consumption level, and improving overall execution efficiency. The final output collaborative scheduling execution result includes deployment status and execution flags, providing an accurate basis for performance feedback collection and strategy evolution, and enhancing the system's scheduling closed-loop capability and operational stability.

[0037] In this implementation, the resource matching function: ; in, is the scheduling fitness score, The k-th operator node in the training subgraph or inference subgraph, The device state vector of the i-th computing node, To match the weights for computing power, is the computing capability indicator, is the memory bandwidth indicator, is the communication delay indicator, To support precision type identification, is the current power consumption level, is the memory bandwidth matching weight, is the communication delay penalty weight, is the power consumption penalty weight, Match weights for precision types, For nodes The computational requirements, For nodes Memory requirements, is the type matching indicator function, For nodes The precision type identifier.

[0038] The present invention constructs a resource matching function to quantify and score the multi-dimensional state characteristics of the task node's computing and memory requirements with the device's computing power, bandwidth, latency, power consumption, and supported precision types. The function takes the form of a weighted combination of multiple indicators, including bonus points for computing power and memory bandwidth, and penalty terms for communication latency and power consumption. It also introduces a precision type matching indicator function to determine whether the task's required precision is consistent with the device's supported type. This matching function enables a dynamic trade-off between scheduling accuracy and efficiency, effectively improving resource utilization and scheduling adaptability.

[0039] In this embodiment, the performance feedback information includes execution delay, resource utilization, energy efficiency ratio and scheduling accuracy score; the execution delay represents the time interval from the start to the completion of the task node, which is used to measure the task execution efficiency; the resource utilization represents the proportion of computing resources occupied on the corresponding computing node during task execution, including CPU occupancy and memory occupancy; the energy efficiency ratio represents the task completion efficiency under unit power consumption, which is used to evaluate the energy utilization efficiency of the task on a specific computing node; the scheduling accuracy score represents the degree of match between the structural attributes of the task node and the state parameters of the deployed computing node, which is used to reflect the accuracy of the current scheduling strategy.

[0040] The present invention realizes multi-dimensional evaluation and optimization of scheduling strategies by collecting execution delay, resource utilization, energy efficiency ratio and scheduling accuracy scores, thereby improving the accuracy of resource allocation and the overall efficiency of system operation.

[0041] In this embodiment, the S7 specifically includes: S71. Collect performance feedback information set, task graph structure information, device state parameter set, and current scheduling function weight parameter set to construct a strategy evolution input sample set; S72. Based on the evolutionary input sample set, define the scheduling optimization objective function: ; Among them, J is the scheduling optimization objective function, K is the total number of task nodes in the constructed task graph structure, k is the task node index in the task graph structure, is the weight coefficient of the inverse ratio of delay, is the actual execution delay of the kth task node, is the weight coefficient of the resource utilization term, is the resource utilization of the kth task node, is the weight coefficient of the energy efficiency ratio term, is the energy efficiency ratio of the kth task node, is the weight coefficient of the scheduling accuracy scoring item, Score the scheduling accuracy of the kth task node; S73, based on the evolutionary input sample set and the scheduling optimization objective function J, perform evolutionary optimization operations, use genetic algorithms or differential evolution algorithms to optimize the current scheduling function weight parameter set Perform multiple rounds of population updates and fitness evaluations to obtain the updated scheduling function weight parameter set, which is recorded as the evolutionary update weight set; S74. Applying the evolution update weight set to the scheduling score function In the process, the resource mapping relationship of the task nodes is updated with the scheduling strategy to complete the evolution and iteration of the scheduling function.

[0042] The present invention constructs a multi-index scheduling optimization objective function, transforming task execution delay, resource utilization, energy efficiency ratio, and scheduling accuracy score into a unified evaluation model. The objective function is modeled in a weighted cumulative form, which can quantitatively measure the performance of the current scheduling strategy in different performance dimensions. The weight parameters of each indicator are adaptively optimized through an evolutionary algorithm to generate new weight combinations, which are dynamically applied to the resource matching function to achieve continuous iteration and improvement of the scheduling strategy. This mechanism improves the scheduling system's responsiveness to actual operating results and enhances the performance orientation and long-term adaptability of the scheduling strategy.

[0043] Example 1: To verify the feasibility of this invention, we applied it to a real-world deep learning model deployment platform. We constructed a multi-node computing cluster comprised of heterogeneous GPU, CPU, and NPU resources, and conducted end-to-end training and inference task scheduling tests for a typical large-scale natural language processing model. The platform deployment structure included 24 computing nodes, each with a different hardware configuration, including high-performance A100 GPU nodes, general-purpose x86 CPU nodes, and customized NPU inference acceleration nodes, simulating the computational variability of heterogeneous clusters in actual production environments.

[0044] During the experiment, the test model was a text generation model with 500 million parameters. Its forward reasoning phase included matrix multiplication, attention mechanism, multi-layer normalization and activation operations, and the training phase included gradient backpropagation and parameter updates. The traditional scheduling strategy uses a static weight matching mechanism. The task graph structure is generated by ONNX and then statically allocated to the device, ignoring runtime resource fluctuations. The present invention, on the other hand, constructs a scheduling strategy model by constructing a unified task graph structure and device state parameter input, using neural network ordinary differential equations. After the model graph structure is reconstructed, the training subgraph and reasoning subgraph are divided, and device mapping is performed based on the scheduling score function. In actual operation, performance feedback indicators are collected for scheduling strategy evolution and update.

[0045] Under the same computing power scale and model scale, the experimental results of comparing the traditional static scheduling scheme with the dynamic graph scheduling method proposed in this invention show that the present invention significantly improves task execution efficiency and resource utilization. In the inference task, the average latency of the present invention is reduced by 21.3%, and the throughput is increased by 17.8%; in the training task, the average time per iteration is reduced by 14.5%, and the peak power consumption is reduced by 12.2%. In addition, due to the adaptive evolution mechanism of the scheduling strategy that continuously optimizes the scheduling score function, after 5 consecutive updates, the scheduling accuracy score increased from 0.746 to 0.873, further reducing the resource waste caused by device mismatch.

[0046] By monitoring the dynamic scheduling results of 24 nodes at different time periods, we found that the proposed method can maintain high resource scheduling stability when node loads fluctuate frequently. The standard deviation of resource utilization is reduced by 23.5%, indicating that the system has good robustness and scheduling accuracy. The experimental data is shown in Table 1.

[0047] Table 1 Performance comparison data of the method of the present invention and the traditional scheduling strategy ; As can be seen from the above table, the present invention has a significant improvement in the scheduling performance of training and inference tasks in large-scale heterogeneous computing power clusters compared to traditional static scheduling methods. First, in terms of the execution efficiency of inference tasks, the method of the present invention reduces the average delay from 82.7 milliseconds to 65.1 milliseconds, a reduction of 21.3%. At the same time, the inference throughput is increased from 145.3 samples / s to 171.1 samples / s, an increase of 17.8%, indicating that the scheduling strategy is significantly optimized in terms of resource allocation rationality and device matching. Secondly, in terms of the acceleration effect of training tasks, the present invention compresses the average time consumption of a single round of training iteration from 6.41 seconds to 5.48 seconds, shortening the training time by 14.5%, effectively improving the training efficiency.

[0048] In terms of energy consumption control, the scheduling mechanism of the present invention reduces the peak power consumption of cluster operation from 289.2 watts to 253.9 watts through precise scheduling and node task load balancing, achieving a 12.2% energy saving. Furthermore, by dynamically sensing node load and task attributes, the present invention significantly reduces the fluctuation of resource utilization, with the standard deviation dropping from 0.108 to 0.082, a decrease of 23.5%. This demonstrates that the scheduling strategy has greater stability and robustness in high-concurrency heterogeneous cluster environments.

[0049] It is noteworthy that with the continuous evolution and optimization of the scheduling strategy, the scheduling accuracy score increased from an initial 0.746 to 0.873 after the fifth round of evolution, significantly enhancing scheduling accuracy. This indicates that the scheduling function update process driven by the performance feedback mechanism can effectively enhance the accuracy of resource-task matching. In summary, this invention not only achieves systematic optimization of multi-objective scheduling strategies, but also demonstrates significant advantages in delay control, energy consumption management, resource utilization, and scheduling intelligence, and has broad practical application value.

[0050] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A large-scale heterogeneous computing power cluster training and inference acceleration method, characterized by: The steps include: S1. Collect the operating characteristics of each computing node in the heterogeneous computing power cluster and build a set of device status parameters; S2. Load the deep learning model to be trained and inferred, build a unified intermediate representation graph structure, and generate model graph structure information; S3. Inputting the model graph structure information and the device state parameter set into a scheduling strategy model constructed based on a neural network ordinary differential equation, wherein the scheduling strategy model includes a state representation layer, a differential evolution function, an ordinary differential equation solver, and a decision decoding module to generate a graph optimization decision result; S4. Perform a structural reconstruction operation on the model graph structure according to the graph optimization decision result to generate an optimized graph structure, and divide the optimized graph structure into a training subgraph and an inference subgraph; S5. Map the training subgraph and the inference subgraph to multiple computing nodes in the heterogeneous computing power cluster, perform resource matching and task distribution based on the characteristics of each node in the device state parameter set, and form a training and inference collaborative scheduling execution result; S6. Continuously collect performance feedback information during the training and propulsion collaborative scheduling process; S7. Input the performance feedback information into the scheduling strategy model to update the parameters and adjust the scheduling strategy model parameters in the differential evolution function.

2. The large-scale heterogeneous computing power cluster training and push acceleration method according to claim 1 is characterized in that: The device status parameter set includes a computing capability indicator, a memory bandwidth indicator, a communication delay indicator, a supported precision type identifier, and a current power consumption level; The computing power indicator includes the number of floating-point operations that can be performed per second; the memory bandwidth is expressed as the maximum data transmission rate per unit time; the communication delay is the average single round-trip time between the node and the central scheduler or other nodes; the power consumption level is expressed as the power consumption per unit time at the current moment.

3. The large-scale heterogeneous computing power cluster training and inference acceleration method according to claim 1 is characterized in that: The S2 specifically includes: S21. Load the model to be trained and the model to be inferred, extract the network structure information of the training model and the network structure information of the inference model respectively, and construct a training model structure diagram and an inference model structure diagram, wherein the training model structure diagram and the inference model structure diagram both include an operator node set, an operator connection relationship, and an operator parameter attribute; S22. Perform a structural-level merger on the training model structure graph and the inference model structure graph to generate a joint model graph structure. The joint model graph structure uses operator nodes as nodes in the graph and directed dependencies between operators as edges in the graph to form unified model graph structure information. S23. For each operator node in the unified model graph structure information , extract the structural attribute vector , the structural attribute vector includes the parameter size , memory usage , Number of input channels , Number of output channels and operator type identifier ; S24. Arrange the structural attribute vectors of all operator nodes in the order of node numbers to form a node feature matrix , where n represents the total number of operator nodes in the unified model graph structure information; S25, extract the node connection relationship in the unified model graph structure information, generate the connection relationship matrix A, and combine the node feature matrix G with the connection relationship matrix A to form the graph structure information .

4. The large-scale heterogeneous computing power cluster training and inference acceleration method according to claim 1 is characterized in that: The S3 specifically includes: S31. Receive graph structure information and the device state parameter set, where A is the connection relationship matrix A and G is the node feature matrix; S32, input the node feature matrix G into the state representation layer to generate the initial graph structure state representation ; S33, mapping the device state parameter set R to the control variable function c(t), expressing the dynamic evolution characteristics of the device state in a time series manner; S34. Constructing structural state evolution function ,in is the structural state variable, c(t) is the control variable function, is the neural network parameter set; S35, input the differential evolution function into the ordinary differential equation solver, use the numerical integration method to solve the state evolution process, and obtain the structural state in the time interval The evolutionary results ; S36. Input the structural state evolution result into the decoding module to generate a graph optimization decision result, which includes path adjustment instructions, node fusion and replacement instructions, and calculation accuracy switching strategy.

5. The large-scale heterogeneous computing power cluster training and inference acceleration method according to claim 4 is characterized in that: The S34 specifically includes: S341, set the structure state variable , where n is the number of nodes in the unified model graph structure, and d is the state representation dimension of each node; S342, set control variable function , where N is the number of device nodes and k is the dimension of each device state vector; S343, constructing a differential evolution function, wherein the differential evolution function is composed of a set of neural network parameters control: ； Among them, h(t) is the structural state variable, t is the time variable, is the weight matrix of the first layer, is the weight matrix of the second layer, is the activation function, is the output layer activation function, is the control variable function, is the bias vector of the first layer of neural network, is the bias vector of the second layer neural network, Represents a tensor concatenation operation; S344. Combine the differential evolution function with the structural state variables and control variable functions to form a structural state evolution expression.

6. The large-scale heterogeneous computing power cluster training and inference acceleration method according to claim 1 is characterized in that: The S4 specifically includes: S41, receiving a graph optimization decision result, and generating a structure reconstruction control instruction set, wherein the structure reconstruction control instruction set includes a path adjustment instruction set, a node fusion and replacement instruction set, and a calculation accuracy switching strategy set; S42: Perform a topological update on the connection relationship matrix A in the unified model graph structure according to the path adjustment instruction set to generate a connection relationship matrix after path adjustment. ; S43. According to the node fusion and replacement instruction set, identify the node subset to be fused in the node feature matrix G, generate the fused node and the corresponding structure attribute vector, delete the original node attribute vector and add the fused structure attribute vector, and generate the node feature matrix after the node fusion. ; S44, according to the calculation accuracy switching strategy set, the node feature matrix after node fusion Each structural attribute vector in Perform precision replacement operations based on computing node resource status indicators Update the precision label and generate the node feature matrix after precision switching ; S45. Connection relationship matrix based on path adjustment Node feature matrix after switching with precision , construct the graph structure information after structural reconstruction; S46. Based on the task type division rule set, perform a functional graph division operation on the graph structure information after structural reconstruction to generate a training subgraph and a reasoning subgraph.

7. The large-scale heterogeneous computing power cluster training and inference acceleration method according to claim 1 is characterized in that: The S5 specifically includes: S51, respectively receive the training subgraph, the reasoning subgraph and the device state parameter set, and for each node in the training subgraph Extract structural attribute vector , the node demand vector and the device state parameter vector are input into the resource matching function respectively; S52, perform maximum matching scheduling on all training nodes, and select The largest device node i generates a training node scheduling mapping set; S53, repeating steps S51 and S52, performing the same resource matching calculation and allocation on the nodes in the reasoning subgraph, and generating a reasoning node scheduling mapping set; S54: Deploy the training subgraph and the inference subgraph to the corresponding computing nodes according to the training node scheduling mapping set and the inference node scheduling mapping set, initialize the task execution environment, and complete the training and inference collaborative scheduling execution operation; S55 , outputting the training-pushing collaborative scheduling execution result, wherein the training-pushing collaborative scheduling execution result includes the task deployment status of each computing node, the data input binding relationship, and the execution start flag.

8. The large-scale heterogeneous computing power cluster training and inference acceleration method according to claim 7 is characterized in that: The resource matching function: ； in, is the scheduling fitness score, The k-th operator node in the training subgraph or inference subgraph, The device state vector of the i-th computing node, To match the weights for computing power, is the computing capability indicator, is the memory bandwidth indicator, is the communication delay indicator, To support precision type identification, is the current power consumption level, is the memory bandwidth matching weight, is the communication delay penalty weight, is the power consumption penalty weight, Match weights for precision types, For nodes The computational requirements, For nodes Memory requirements, is the type matching indicator function, For nodes The precision type identifier.

9. The large-scale heterogeneous computing power cluster training and push acceleration method according to claim 1, characterized in that: The performance feedback information includes execution latency, resource utilization, energy efficiency ratio and scheduling accuracy score; the execution latency represents the time interval from the start to the completion of the task node; the resource utilization represents the proportion of computing resources occupied on the corresponding computing node during task execution, including CPU occupancy and memory occupancy; the energy efficiency ratio represents the task completion efficiency per unit power consumption; The scheduling accuracy score represents the degree of matching between the structural attributes of the task node and the state parameters of the deployed computing nodes.

10. The large-scale heterogeneous computing power cluster training and push acceleration method according to claim 1, characterized in that: The S7 specifically includes: S71. Collect performance feedback information set, task graph structure information, device state parameter set, and current scheduling function weight parameter set to construct a strategy evolution input sample set; S72. Based on the evolutionary input sample set, define the scheduling optimization objective function: ； Among them, J is the scheduling optimization objective function, K is the total number of task nodes in the constructed task graph structure, k is the task node index in the task graph structure, is the weight coefficient of the inverse ratio of time delay, is the actual execution delay of the kth task node, is the weight coefficient of the resource utilization term, is the resource utilization of the kth task node, is the weight coefficient of the energy efficiency ratio term, is the energy efficiency ratio of the kth task node, is the weight coefficient of the scheduling accuracy scoring item, Score the scheduling accuracy of the kth task node; S73, based on the evolutionary input sample set and the scheduling optimization objective function J, perform evolutionary optimization operation, use genetic algorithm or differential evolution algorithm to optimize the current scheduling function weight parameter set Perform multiple rounds of population updates and fitness evaluations to obtain the updated scheduling function weight parameter set, which is recorded as the evolutionary update weight set; S74. Applying the evolution update weight set to the scheduling score function In the process, the resource mapping relationship of the task nodes is updated with the scheduling strategy to complete the evolution and iteration of the scheduling function.

Images