Large model calculation network scheduling method based on task combination automation

Abstract

The invention discloses a large model calculation network scheduling method based on task combination automation, and relates to the technical field of data processing, and the method comprises the steps: constructing a task factor flow graph; simulating resource linkage between nodes by using a cascade pulse propagation mechanism, generating a pulse propagation topological graph and a pulse intensity matrix, and constructing a computing force field situation awareness network in combination with a pre-trained pulse graph neural network; predicting the resource matching degree of the task factor and the computing power node by using a preset space-time convolution predictor, and generating an affinity tensor; and constructing a joint strategy space, searching and generating a fusion strategy in the joint strategy space by using a multi-target equalization algorithm, and embedding the fusion strategy into the deep reinforcement learning framework to generate an optimal scheduling strategy. By constructing the computing power field situation awareness network, continuous tracking and awareness of the load evolution process, the resource coupling relation and the performance bottleneck dynamic migration of each computing power node are realized, and the self-adaptive decision-making capability and the resource matching efficiency of a scheduling system in a multi-source heterogeneous environment are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a large model algorithm network scheduling method based on task combination automation. BACKGROUND

[0002] With the wide deployment of large-scale artificial intelligence models (such as large language models, generative visual models, etc.), the demand for computing power resources is showing an exponential growth trend. In particular, in complex application scenarios such as multi-task parallel execution, heterogeneous model combination deployment, and dynamic inference and micro-service orchestration, traditional scheduling methods based on static rules or fixed resource allocation have been difficult to meet the comprehensive performance requirements of the system in terms of high throughput, low response time, balanced resource utilization, and architecture scalability.

[0003] Currently, large model computing power scheduling technology generally relies on static or semi-dynamic resource allocation mechanisms, which use fixed rules or preset parameters to uniformly schedule and manage computing tasks. Such methods have certain optimization capabilities in terms of resource utilization, but when faced with scenarios of high-speed workload fluctuations and complex task structure evolution, their flexibility and response efficiency are obviously insufficient. On the one hand, most existing scheduling schemes still rely on predefined rules, static resource tags, or empirical parameters, and consider the "model as a whole" as the scheduling unit for resource allocation. This coarse-grained strategy lacks real-time perception and response capability to changes in runtime state, making it difficult to cope with actual system characteristics such as task load fluctuations, resource bottleneck migration, and dynamic reconstruction of chain-dependent structures. On the other hand, although some research attempts to introduce intelligent optimization methods such as deep reinforcement learning and self-attention mechanisms to enhance the flexibility and adaptability of scheduling strategies, such methods usually treat the scheduling process as a black-box optimization problem, ignoring the explicit modeling of task internal semantic structure and logical constraints, resulting in significant limitations in handling task dependency relationships, multi-objective scheduling conflicts, and the explainability of strategy results.

[0004] To address the problems in the related art, no effective solutions have been proposed so far. SUMMARY

[0005] To address the problems in the related art, the present application proposes a large model algorithm network scheduling method based on task combination automation to overcome the above technical problems existing in the prior art.

[0006] To this end, the specific technical solutions adopted by the present application are as follows:

[0007] The large model algorithm network scheduling method based on task combination automation comprises:

[0008] S1, decompose the task into a plurality of task factors by using a task factor diagram decomposition technology, analyze data flow and logical dependency relationship between the task factors, and construct a task factor flow diagram;

[0009] S2, collect resource data of each computing power node, simulate resource linkage between the nodes by using a cascading pulse propagation mechanism, generate a pulse propagation topology graph and a pulse intensity matrix, and combine a pre-trained pulse graph neural network to construct a computing power field situation awareness network;

[0010] S3, combining the computing power field situation awareness network and the task factor flow diagram, predicting the resource matching degree of the task factor and the computing power node by using a pre-set space-time convolution predictor, and generating an affinity tensor;

[0011] S4, constructing a joint strategy space according to the affinity tensor, searching and generating a fusion strategy in the joint strategy space by using a multi-objective equilibrium algorithm, and embedding the fusion strategy in a deep reinforcement learning framework to generate an optimal scheduling strategy.

[0012] Further, the resource data of each computing power node is collected, the resource linkage between the nodes is simulated by using a cascading pulse propagation mechanism, a pulse propagation topology graph and a pulse intensity matrix are generated, and a pre-trained pulse graph neural network is combined to construct a computing power field situation awareness network, which includes:

[0013] S21, collecting resource data of each computing power node in the computing network, and combining logical dependency relationship in the task factor flow diagram to generate demand pulses of the task factors and supply pulses of the computing power nodes;

[0014] S22, simulating the propagation process of the demand pulses and the supply pulses in the logical dependency relationship between the task factors and the physical connection topology of the computing network by using a decay superposition technology, obtaining a pulse propagation topology graph and a pulse intensity matrix;

[0015] S23, based on the obtained pulse propagation topology graph and pulse intensity matrix, fusing and encoding the logical demand information of the task factors and the resource data of the computing power nodes to form a pulsating feature tensor;

[0016] S24, combining the pulsating feature tensor and the pre-trained pulse graph neural network to establish a pulse interaction matrix, and combining the pulse propagation topology graph to generate a computing power field situation awareness network, the computing power field situation awareness network includes a situation matrix and a stress conduction thermodynamic map.

[0017] Further, the decay superposition technology is used to simulate the propagation process of the demand pulses and the supply pulses in the logical dependency relationship between the task factors and the physical connection topology of the computing network, and the pulse propagation topology graph and the pulse intensity matrix are obtained, which includes:

[0018] S221, combine the logical dependency relationship between task factors and the physical connection topology of the computing network to construct a polymorphic topology graph, and set the propagation impedance as the propagation constraint condition;

[0019] S222, based on the set propagation impedance, drive the demand pulse and the supply pulse to propagate in the polymorphic topology graph in the form of exponential decay, and accumulate and superimpose the pulse signals of each node in the time sequence through the sliding window mechanism to generate the pulse response strength;

[0020] S223, according to the generated pulse response strength, establish a local resource competition field reflecting the current computing load and resource contention state of the node, and adjust the pulse propagation process using the local resource competition field;

[0021] S224, use the simulated annealing algorithm to optimize the adjusted pulse propagation result, generate a pulse propagation topology graph, and count the pulse response strength of each node to construct a pulse intensity matrix.

[0022] Further, combined with the pulsating feature tensor and the pre-trained pulse graph neural network, a pulse interaction matrix is established, and combined with the pulse propagation topology graph, a computing power field situation awareness network is generated, including a situation matrix and a stress conduction thermodynamic map, including:

[0023] S241, input the pulsating feature tensor into the pre-trained pulse graph neural network, slice the resource data in the time dimension through the pulse coding layer, and map the slicing result to the pulse firing frequency to establish a pulse interaction matrix;

[0024] S242, based on the pulse interaction matrix and the pulse propagation topology graph, obtain a node potential set, and use the entanglement evolution algorithm to establish a node evolution set, and generate a potential gradient map by projecting and measuring the node evolution set;

[0025] S243, according to the potential distribution in the potential gradient map, calculate the resource balance entropy, and based on the resource balance entropy, construct a situation matrix of balanced computing power;

[0026] S244, combined with the potential gradient map and the pulse intensity matrix, calculate the blocking risk value of each edge in the task factor flow graph, and convert the blocking risk value to a stress conduction thermodynamic map of the task dependency path through color mapping technology.

[0027] Further, based on the pulse interaction matrix and the pulse propagation topology graph, obtain a node potential set, and use the entanglement evolution algorithm to establish a node evolution set, and generate a potential gradient map by projecting and measuring the node evolution set, including:

[0028] S2421、According to the pulse interaction matrix and the pulse propagation topology map, the pulse response intensity of the computing power node is converted into an amplitude coefficient, and the initial potential energy of each node in the computing power space is constructed to form a node potential energy set;

[0029] S2422、Select a target node pair in the node potential energy set, and use an entanglement operator with pulse response intensity as a weight parameter to perform entanglement evolution on the node potential energy to generate an entangled network;

[0030] S2423、Based on the generated entangled network, the potential energy space is discretized using the finite difference algorithm, and the tunneling transition probability between nodes is calculated to simulate the transition behavior of node potential energy in the computing power space, and output a node evolution set containing node potential energy change probability and direction;

[0031] S2424、Project the node evolution set, collapse to obtain the potential energy value matrix in the computational basis, and extract the correlation matrix that has not collapsed using the entropy weight algorithm;

[0032] S2425、Weighted sum of potential energy value matrix and correlation matrix to generate potential energy gradient map.

[0033] Further, the calculation formula of the tunneling transition probability is:

[0034]

[0035] In the formula, P tunnel (u-v) represents the tunneling transition probability from node u to node v; h represents the reduced Planck constant; path represents the integral path; m eff represents the effective mass; ΔV uv (s) represents the potential energy difference function between nodes; E k represents the kinetic energy term; s represents the path integral variable.

[0036] Further, construct a joint strategy space according to the affinity tensor, use a multi-objective equilibrium algorithm to search for a fusion strategy in the joint strategy space, and embed the fusion strategy into a deep reinforcement learning framework to generate an optimal scheduling strategy including:

[0037] S41、Based on the generated affinity tensor, define a multi-objective optimization function, and construct a joint strategy space according to the multi-objective optimization function;

[0038] S42、According to the constructed joint strategy space, use a non-dominated sorting genetic algorithm for multi-objective optimization to generate a Pareto frontier solution set, and combine a Nash equilibrium verification mechanism to screen the Pareto frontier solution set to obtain a scheduling solution set that satisfies the equilibrium condition;

[0039] S43, filtering the scheduling solution set by using a preset linear programming model according to the hardware constraints and the software constraints, extracting a scheduling decision set, and establishing a scheduling decision space;

[0040] S44, constructing a hypergraph structure according to the scheduling decision space and a multi-objective optimization function, and generating a fusion strategy by combining a preset neural network;

[0041] S45, embedding the fusion strategy as an initial scheduling strategy into a deep reinforcement learning framework, simulating a Markov dynamic load scenario, and adjusting the initial scheduling strategy by using a proximal policy optimization algorithm to generate an optimal scheduling strategy.

[0042] Further, according to the scheduling decision space and the multi-objective optimization function, a hypergraph structure is constructed, and a fusion strategy is generated by combining a preset neural network, which includes:

[0043] S441, mapping each scheduling decision in the scheduling decision space to a vertex of the hypergraph, defining hyperedge connections based on a Pearson correlation coefficient matrix between the multi-objective optimization functions, and constructing a hypergraph structure reflecting the multi-objective coupling relationship;

[0044] S442, based on the constructed hypergraph structure, designing a hypergraph convolutional neural network, and generating a sub-strategy set by fusing the multi-objective constraints in the hyperedge and the structural similarity between the vertices;

[0045] S443, extracting symbolic logic rules from the sub-strategy set by using a rule induction learning algorithm, and generating a fusion vector by combining a preset neural network to construct a neural-symbolic hybrid strategy network;

[0046] S444, using the neural-symbolic hybrid strategy network as a teacher network to guide the learning of a student network, and generating a fusion strategy by combining a KL divergence loss function and a cosine similarity penalty term.

[0047] Further, the neural-symbolic hybrid strategy network is constructed by extracting symbolic logic rules from the sub-strategy set by using a rule induction learning algorithm and combining a preset neural network, which includes:

[0048] S4431, analyzing the symbolic logic patterns in the sub-strategy set by using a rule induction learning algorithm, and generating a symbolic logic distribution map by aggregating the analysis results by using a bagging algorithm;

[0049] S4432, extracting the symbolic logic rules of the target frequency from the symbolic logic distribution map, constructing a symbolic rule template, and optimizing the symbolic rule template by using a policy gradient algorithm with the scheduling success rate as a reward signal;

[0050] S4433, according to the optimized symbol rule template and the preset neural network, a gating fusion mechanism is constructed, neural features and symbol features are extracted and spliced, and the splicing result is input into a logical fusion function to generate a fusion vector;

[0051] S4434, based on the generated fusion vector, the neural network parameters, the symbol rule template weight and the gating fusion parameter are optimized synchronously, and a neural-symbol hybrid strategy network is constructed.

[0052] Further, the expression of the logical fusion function is:

[0053] f = sigma (W g ·concat (h n ,h sym ) +b g ) o a n +(1-sigma (W g ·concat (h n ,h sym ) +b g ) o a sym ;

[0054] In the formula, f represents the fusion vector;Sigma represents the Sigmoid function;W g represents the gating weight matrix;Concat represents the splicing operation;H n represents the neural feature vector;H sym represents the symbol feature vector;B g represents the gating bias vector;A n represents the neural network action output;A sym represents the symbol rule action output;O represents element-wise multiplication.

[0055] The beneficial effects of the present application are:

[0056] 1、The present application decomposes complex tasks into fine-grained task factors through task factor graph decomposition technology, and constructs a task factor flow graph based on the data flow and logical dependency relationship, realizes the explicit modeling of the internal structure of the task, enhances the parsing ability of the scheduling system to the task semantics, and improves the flexibility, accuracy and process controllability of resource scheduling.

[0057] 2、The present application realizes the continuous tracking and perception of the load evolution process, resource coupling relationship and performance bottleneck dynamic migration of each computing power node by constructing a computing power field situation awareness network;At the same time, the spatio-temporal correlation characteristics of task demand and resource supply are fused to generate a multi-dimensional affinity tensor, which improves the adaptive decision-making ability and resource matching efficiency of the scheduling system in a multi-source heterogeneous environment.

[0058] 3、The application constructs a joint strategy space, fuses multiple objectives and constraints, generates a comprehensive scheduling strategy by using a balanced algorithm, and embeds a reinforcement learning framework to realize continuous exploration and self-optimization of the strategy in a dynamic load environment, thereby ensuring the rationality and balance of the scheduling decision in the multiple objective trade-off and ensuring that the scheduling result has good explainability and system transparency. BRIEF DESCRIPTION OF DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0060] Figure 1 is a flowchart of the large model network scheduling method based on task combination automation according to an embodiment of the present application. DETAILED DESCRIPTION

[0061] To further illustrate the embodiments, the present application provides drawings, which are part of the disclosure of the present application, mainly used to illustrate the embodiments, and can explain the operating principle of the embodiments in conjunction with the related description of the specification. With reference to these contents, those skilled in the art should understand other possible embodiments and advantages of the present application.

[0062] According to an embodiment of the present application, a large model network scheduling method based on task combination automation is provided.

[0063] The present application will be further described in conjunction with the drawings and specific embodiments, as shown in the drawings, the large model network scheduling method based on task combination automation according to an embodiment of the present application, the method comprises: Figure 1

[0064] S1, using task factor graph decomposition technology to decompose the task into several task factors, and analyzing the data flow and logical dependency relationship between the task factors to construct a task factor flow graph.

[0065] It should be noted that using task factor graph decomposition technology to decompose the task into several task factors, and analyzing the data flow and logical dependency relationship between the task factors to construct a task factor flow graph specifically comprises:

[0066] ​Firstly, the core idea of the task factor graph decomposition technology is to decompose a complex task into finer substructures, i.e., "task factors", according to its functional components, data requirements, and execution flow. Assuming that the original task is a multi-step deep learning training process, including data preprocessing, feature extraction, model training, and evaluation stages, these stages are further subdivided, for example: data cleaning, missing value filling, feature standardization, convolution operation, weight update, accuracy statistics, etc. Each operation is mapped to an independent task factor node.

[0067] Next, after the task decomposition is completed, the data flow and control logic between the task factors need to be analyzed, which can be achieved through static code analysis, dependency scanning, or call graph tracking. For example, if task factor A (data cleaning) generates structured data, and task factor B (feature extraction) depends on this structured data as input, then a "data dependency edge" can be established between factors A and B. At the same time, if task factor C (model training) must be executed after task factor B is completed, then a "sequential dependency edge" is established at the logical level.

[0068] Then, a task factor flow graph is constructed, which is a directed graph where each node represents a task factor, and the edge represents the data or control dependency between them. The graph structure not only depicts the internal organization of the task, but also reveals the parallel potential and computational bottlenecks. For example, if there are multiple branches developing from the starting node in the graph, they can be marked as parallel execution paths. The confluence point of all paths is a potential synchronization blocking point.

[0069] Finally, to ensure the practicality and scheduling value of the graph, after the graph is constructed, it needs to be normalized and optimized in structure, including merging redundant nodes, standardizing dependency types, and adding attribute labels to each node (such as estimated execution time, resource demand level, etc.). This can enhance the interpretability and computational expressiveness of the task factor flow graph, and provide a solid data foundation for subsequent tasks such as power matching, path heat analysis, scheduling priority evaluation, etc.

[0070] S2, collect resource data of each computing power node, simulate resource linkage between nodes using cascading pulse propagation mechanism, generate pulse propagation topology graph and pulse intensity matrix, and construct computing power field situation awareness network combined with pre-trained pulse graph neural network.

[0071] In this optional embodiment, collecting resource data of each computing power node, simulating resource linkage between nodes using cascading pulse propagation mechanism, generating pulse propagation topology graph and pulse intensity matrix, and constructing computing power field situation awareness network combined with pre-trained pulse graph neural network include:

[0072] S21, collect resource data of each computing power node in the computing network, and generate demand pulses of the task factor and supply pulses of the computing power node in combination with logical dependency relationships in the task factor flow graph.

[0073] It should be noted that collecting resource data of each computing power node in the computing network, and generating demand pulses of the task factor and supply pulses of the computing power node in combination with logical dependency relationships in the task factor flow graph specifically comprises:

[0074] First, a resource collection module needs to be deployed in the computing network, which periodically collects multi-dimensional resource state data from each computing power node, such as CPU utilization, GPU memory occupation, memory remaining amount, bandwidth load, current task queuing situation, etc., and performs standardization processing thereon; in order to express these continuous values in the pulse neural system, a firing frequency coding method needs to be used to convert the resource state into a supply pulse sequence, that is, the more abundant the resource, the higher the firing frequency of the corresponding dimension; for example, a GPU utilization of 30% can be mapped to f gpu =(1-0.3)×f max f gpu =(1-0.3)×f max , where f max is the maximum pulse frequency allowed by the system (such as 100 Hz), and finally a multi-channel supply pulse sequence of each computing power node is constructed.

[0075] At the same time, the demand of each task factor in the task factor flow graph needs to be modeled, and each task factor is essentially a local space-time demand expression of a certain computing power resource, such as a task factor requiring a GPU to perform a convolution operation for 1.5s, while occupying 2GB of video memory and 20% of the main memory; by analyzing the resource configuration requirements, duration and priority level of the task factor, a demand vector is defined for each task factor; similarly, a firing frequency mechanism is used to convert the demand of each dimension into a “demand pulse”, and the frequency is proportional to the intensity of resource consumption (the more urgent and high-consumption, the higher the pulse frequency), and finally a multi-channel demand pulse sequence of the task factor is constructed.

[0076] Then, the task factors are linked in time and control order through logical dependency relationships; logically, if a task factor T a is a predecessor of T b (that is, T b depends on the output data of T a ), the start activation window of the pulse sequence needs to be adjusted to ensure that it only fires pulses after the output completion time of T a , thereby preserving the control dependency between task factors, which is completed by introducing a time gating mechanism into the task factor flow graph, that is, defining a pulse valid window.

[0077] Finally, two types of pulse expressions are obtained: one is the "supply pulse" facing the computing power node, and the other is the "demand pulse" facing the task factor. The relationship between them on the time axis will be used for subsequent matching and scheduling modeling.

[0078] S22, using the attenuation superposition technology, simulating the propagation process of the logical dependency relationship between the demand pulse and the supply pulse among the task factors and the physical connection topology of the computing network, obtaining the pulse propagation topology graph and the pulse intensity matrix.

[0079] In this optional embodiment, using the attenuation superposition technology, simulating the propagation process of the logical dependency relationship between the demand pulse and the supply pulse among the task factors and the physical connection topology of the computing network, obtaining the pulse propagation topology graph and the pulse intensity matrix includes:

[0080] S221, combining the logical dependency relationship among the task factors and the physical connection topology of the computing network, constructing a polymorphic topology graph, and setting the propagation impedance as the propagation constraint condition;

[0081] S222, based on the set propagation impedance, driving the demand pulse and the supply pulse to propagate in the form of exponential decay in the polymorphic topology graph, and through the sliding window mechanism, accumulating and superimposing the pulse signals of each node on the time sequence to generate the pulse response intensity;

[0082] S223, according to the generated pulse response intensity, establishing a local resource competition field reflecting the current computing load and resource contention state of the node, and adjusting the pulse propagation process by using the local resource competition field;

[0083] S224, using the simulated annealing algorithm to optimize the adjusted pulse propagation result, generating the pulse propagation topology graph, and counting the pulse response intensity of each node to construct the pulse intensity matrix.

[0084] S23, based on the obtained pulse propagation topology graph and pulse intensity matrix, fusing and encoding the logical demand information of the task factor and the resource data of the computing power node to form a pulsation feature tensor.

[0085] S24, combining the pulsation feature tensor and the pre-trained pulse graph neural network, establishing a pulse interaction matrix, and combining the pulse propagation topology graph to generate a computing power field situation awareness network, the computing power field situation awareness network includes a situation matrix and a stress conduction thermodynamic map.

[0086] In this optional embodiment, combining the pulsation feature tensor and the pre-trained pulse graph neural network, establishing a pulse interaction matrix, and combining the pulse propagation topology graph to generate a computing power field situation awareness network, the computing power field situation awareness network includes a situation matrix and a stress conduction thermodynamic map includes:

[0087] S241, input the pulsating feature tensor into the pre-trained pulse graph neural network, slice the resource data in the time dimension through the pulse coding layer, and map the slice processing result to the pulse firing frequency to establish a pulse interaction matrix.

[0088] S242, based on the pulse interaction matrix and the pulse propagation topology graph, obtain a node potential set, and use an entanglement evolution algorithm to establish a node evolution set, and generate a potential gradient graph by projecting and measuring the node evolution set.

[0089] In this optional embodiment, based on the pulse interaction matrix and the pulse propagation topology graph, a node potential set is obtained, and an entanglement evolution algorithm is used to establish a node evolution set, and a potential gradient graph is generated by projecting and measuring the node evolution set, comprising:

[0090] S2421, according to the pulse interaction matrix and the pulse propagation topology graph, the pulse response intensity of the computing power node is converted into an amplitude coefficient, and the initial potential of each node in the computing power space is constructed to form a node potential set.

[0091] S2422, select a target node pair in the node potential set, use an entanglement operator with pulse response intensity as a weight parameter to perform entanglement evolution on the node potential, and generate an entanglement network.

[0092] S2423, based on the generated entanglement network, discretize the potential space using a finite difference algorithm, and simulate the transition behavior of the node potential in the computing power space by calculating the tunneling transition probability between nodes, and output a node evolution set containing the node potential change probability and direction.

[0093] In this optional embodiment, the calculation formula of the tunneling transition probability is:

[0094]

[0095] In the formula, P tunnel (u-v) represents the tunneling transition probability from node u to node v; h represents the reduced Planck constant; path represents the integral path; m eff represents the effective mass; ΔV uv (s) represents the potential difference function between nodes; E k represents the kinetic energy term; s represents the path integral variable.

[0096] S2424, project and measure the node evolution set, collapse to obtain a potential value matrix under the calculation basis, and extract an uncollapsed correlation matrix using an entropy weight algorithm.

[0097] S2425, weight and sum the potential value matrix and the correlation matrix to generate a potential gradient graph.

[0098] S243、According to the potential energy distribution in the potential energy gradient map, the resource balance entropy is calculated, and the computing power balance situation matrix is constructed based on the resource balance entropy.

[0099] S244、In combination with the potential energy gradient map and the pulse intensity matrix, the blocking risk value of each edge in the task factor flow graph is calculated, and the blocking risk value is converted into a stress conduction heat map of the task dependency path through color mapping technology.

[0100] It needs to be pointed out that in combination with the potential energy gradient map and the pulse intensity matrix, the blocking risk value of each edge in the task factor flow graph is calculated, and the blocking risk value is converted into a stress conduction heat map of the task dependency path through color mapping technology, which specifically includes:

[0101] Firstly, the potential energy gradient map is read, which reflects the potential energy change gradient of computing power resources in the topological space, which can be understood as the resource accessibility and scheduling tendency of each region in the system at different time periods. The higher the gradient value of each computing power node in the graph, the greater the resource load or the more concentrated the scheduling pressure; At this time, the pulse intensity matrix needs to be introduced, the value of which represents the pulse emission intensity between nodes i and j, that is, the propagation ability of the resource state influence between nodes. This matrix can be regarded as a quantitative expression of the time sequence interconnection structure in the dynamic network, and stronger pulse represents stronger upstream and downstream dependency interference.

[0102] Next, for any two task factors T u and T v , if they are mapped to computing power nodes i and j, the blocking risk value R uv of the task dependency edge can be estimated by the following formula: R uv =α·∣g i -g j ∣+β·(1-P ij )+γ·δ uv ; Wherein, α, β, γ are adjustment parameters, the first term ∣g i -g j ∣ represents the resource tilt risk caused by the potential energy difference, the second term 1-P ij represents the possible upstream and downstream information blockage caused by weak pulse transmission, and the third term δ uv represents the data demand mutation degree of the task factor itself (such as data volume or priority jump), all risk values are normalized to the interval [0, 1] to form the blocking risk atlas of each edge in the task factor flow graph.

[0103] Finally, a standard color mapping function (such as Jet, Viridis or custom RGB gradient) is used to map the risk value R uv of each edge e uv to color C uv, construct a stress conduction heat map of the task dependency path, usually the high-risk area corresponds to deep red or purple, and the low-risk area is cold such as blue-green; if the risk value of the edge is higher than a certain threshold (such as R uv >0.75), an additional warning label or highlighted edge can be added.

[0104] S3, combine the computing power field situation awareness network and the task factor flow graph, use a preset spatio-temporal convolution predictor to predict the resource matching degree of the task factor and the computing power node, and generate an affinity tensor.

[0105] It should be noted that combining the computing power field situation awareness network and the task factor flow graph, using a preset spatio-temporal convolution predictor to predict the resource matching degree of the task factor and the computing power node, and generating an affinity tensor specifically includes:

[0106] First, based on the computing power field situation awareness network and the task factor flow graph, an input tensor structure is constructed to adapt to the spatio-temporal convolution predictor. Usually, each task factor in the task factor flow graph is encoded as a feature vector. At the same time, the historical resource state of each computing power node in the computing power field situation awareness network is spliced into a time series matrix, such as (T = 10, F = 4) representing the CPU / GPU / MEM / BW snapshot of the past 10 time points.

[0107] Next, the task factor and the computing power node are paired to form a combination structure of |V t | x |V c | (V t representing the task factor, V c representing the computing power node) is sent to the preset spatio-temporal convolution predictor. The internal structure of the spatio-temporal convolution predictor usually includes two modules: one is a time convolution module for extracting the time evolution trend of the resource state of the computing power node (such as a convolution kernel size of k = 3 can capture resource burst changes), and the other is a structure convolution module for learning the influence of the dependency relationship in the task factor graph on the resource preference. The adjacency matrix convolution aggregates the upstream task factor features to strengthen the context expression.

[0108] Finally, the task factor embedding and the computing power node state embedding are connected through a feature fusion layer, and then a scalar value is output through a nonlinear mapping, representing the resource matching degree score of the task factor and the computing power node. All scores are combined into a three-dimensional affinity tensor. Not only does it provide a structured similarity measure between tasks and resources, but it also captures the adaptability changes under time-varying system load, improving the flexibility and accuracy of the scheduling strategy in complex environments.

[0109] S4, construct a joint strategy space according to the affinity tensor, use a multi-objective equilibrium algorithm to search and generate a fusion strategy in the joint strategy space, and embed the fusion strategy into a deep reinforcement learning framework to generate an optimal scheduling strategy.

[0110] In this optional embodiment, a joint strategy space is constructed according to the affinity tensor, a fusion strategy is generated in the joint strategy space by using a multi-objective equilibrium algorithm, and the fusion strategy is embedded in a deep reinforcement learning framework to generate an optimal scheduling strategy, which includes:

[0111] S41, based on the generated affinity tensor, a multi-objective optimization function is defined, and a joint strategy space is constructed according to the multi-objective optimization function.

[0112] S42, according to the constructed joint strategy space, a non-dominated sorting genetic algorithm is used for multi-objective optimization to generate a Pareto frontier solution set, and a Nash equilibrium verification mechanism is used to screen the Pareto frontier solution set to obtain a scheduling solution set that meets the equilibrium condition.

[0113] S43, according to the hardware constraint and software constraint condition, a preset linear programming model is used to filter the scheduling solution set, extract the scheduling decision set, and establish a scheduling decision space.

[0114] It needs to be noted that according to the hardware constraint and software constraint condition, the preset linear programming model is used to filter the scheduling solution set, extract the scheduling decision set, and establish a scheduling decision space, which specifically includes:

[0115] First, the hardware constraint usually covers the physical resource limit of the computing node, such as CPU core number, GPU memory capacity, memory upper limit, network bandwidth, etc., for example, for a node N, CPU usage should not exceed c max , memory usage should not exceed m max , etc.; software constraints include business logic, such as some tasks cannot be concurrent, some factors need to be executed after the completion of the preposition factor, or there is service deployment location information binding; these constraints can be abstracted as a set of linear inequality constraints and logical dependency relationships, and input into the preset linear programming model; the linear programming model usually takes "feasibility" as the core target, and does not seek the optimal solution, but is used to evaluate whether there is any violation of any hardware / software constraints in the scheduling solution.

[0116] Then, by solving each scheduling solution in the model (i.e. judging whether it is feasible), the scheduling solution set can be screened to remove infeasible or resource conflict scheduling schemes, and only scheduling instances that meet all constraints are retained; this process is usually implemented through batch LP solvers such as Gurobi, CPLEX or OR-Tools, and can be accelerated by GPU parallel judgment.

[0117] Finally, all the screened scheduling solutions are collected into the scheduling decision set and organized in a structured way as the scheduling decision space, which not only has resource rationality and logical correctness, but also can be used for reinforcement learning strategy modeling, graph neural fusion, distillation strategy migration and other subsequent optimization tasks.

[0118] S44, according to the scheduling decision space and the multi-objective optimization function, construct a hypergraph structure, and generate a fusion strategy combined with a preset neural network.

[0119] In this optional embodiment, according to the scheduling decision space and the multi-objective optimization function, constructing a hypergraph structure and generating a fusion strategy combined with a preset neural network includes:

[0120] S441, map each scheduling decision in the scheduling decision space to a vertex of the hypergraph, define the hyperedge connection based on the Pearson correlation coefficient matrix between the multi-objective optimization functions, and construct a hypergraph structure reflecting the multi-objective coupling relationship.

[0121] S442, based on the constructed hypergraph structure, design a hypergraph convolutional neural network, and generate a sub-strategy set by fusing the multi-objective constraints in the hyperedge and the structural similarity between the vertices.

[0122] S443, use a rule induction learning algorithm to extract symbolic logic rules from the sub-strategy set, generate a fusion vector combined with a preset neural network, and construct a neural-symbolic hybrid strategy network.

[0123] In this optional embodiment, using a rule induction learning algorithm to extract symbolic logic rules from the sub-strategy set, generating a fusion vector combined with a preset neural network, and constructing a neural-symbolic hybrid strategy network includes:

[0124] S4431, analyze the symbolic logic patterns in the sub-strategy set using a rule induction learning algorithm, and aggregate the analysis results using a bagging algorithm to generate a symbolic logic distribution map.

[0125] S4432, extract the symbolic logic rules of the target frequency from the symbolic logic distribution map, construct a symbolic rule template, and optimize the symbolic rule template using a policy gradient algorithm with the scheduling success rate as the reward signal.

[0126] S4433, according to the optimized symbolic rule template and the preset neural network, construct a gating fusion mechanism, extract neural features and symbolic features for splicing, and input the splicing result into a logical fusion function to generate a fusion vector.

[0127] In this optional embodiment, the expression of the logical fusion function is:

[0128] f = σ(W g ·concat(h n ,hsym )+b g )⊙a n +(1-σ(W g ·concat(h n ,h sym )+b g )⊙a sym ;

[0129] In the formula, f represents a fusion vector; sigma represents a Sigmoid function; W g represents a gating weight matrix; concat represents a concatenation operation; h n represents a neural feature vector; h sym represents a symbolic feature vector; b g represents a gating bias vector; a n represents a neural network action output; a sym represents a symbolic rule action output; and represents an element-wise multiplication.

[0130] S4434, based on the generated fusion vector, synchronously optimizing the neural network parameter, the symbolic rule template weight and the gating fusion parameter, constructing a neural-symbolic hybrid strategy network.

[0131] S444, taking the neural-symbolic hybrid strategy network as a teacher network to guide a student network to learn, and combining a KL divergence loss function and a cosine similarity penalty term to generate a fusion strategy.

[0132] It needs to be supplemented that taking the neural-symbolic hybrid strategy network as a teacher network to guide a student network to learn, and combining a KL divergence loss function and a cosine similarity penalty term to generate a fusion strategy specifically includes:

[0133] The neural-symbol hybrid strategy network is taken as a teacher network, which is usually composed of two parts: one part is a trainable graph neural or convolutional network module for extracting spatio-temporal structure information from the scheduling state graph; the other part is an embedded symbolic logic reasoning module for expressing prior scheduling rules, constraint logic, task dependency and other structural knowledge; after training by historical scheduling samples, the network has strong strategy judgment ability and outputs a strategy probability distribution; the student network with lighter initialization structure is suitable for online reasoning or low-resource deployment scenarios, and its structure is usually a deep shallow, parameter-light Transformer or simplified GNN model, aiming to learn and imitate the strategy behavior output by the teacher network; in the training process, for each state s, the teacher network will give the reference strategy πT(a|s), and the student network will give its own strategy output πS(a|s), the difference between the two is measured by the KL divergence loss function, which pushes the student network to converge to the probability output of the teacher network, compresses the distribution structure of the teacher strategy from the information entropy angle, and maintains the decision consistency; at the same time, a cosine similarity penalty term is introduced as an auxiliary regularization term to ensure that the student strategy output maintains directional consistency in the high-dimensional embedding space; the final fusion strategy is a weighted combination of the two.

[0134] S45, embedding the fusion strategy as an initial scheduling strategy into a deep reinforcement learning framework, simulating a Markov dynamic load scenario, and adjusting the initial scheduling strategy through a proximal policy optimization algorithm to generate an optimal scheduling strategy.

[0135] It should be noted that embedding the fusion strategy as an initial scheduling strategy into a deep reinforcement learning framework, simulating a Markov dynamic load scenario, and adjusting the initial scheduling strategy through a proximal policy optimization algorithm to generate an optimal scheduling strategy specifically includes:

[0136] First, the fusion strategy is embedded into the deep reinforcement learning framework as an initial scheduling strategy, serving as an initial behavior model for the reinforcement learning agent in the scheduling environment; the fusion strategy retains the structural judgment of neural symbols and the generalization ability of lightweight student networks, providing a reliable starting point for subsequent policy optimization and avoiding the low learning efficiency caused by cold start; at this time, the reinforcement learning framework chooses a proximal policy optimization algorithm based on policy gradient, which has good stability and training sample utilization in high-dimensional continuous action space.

[0137] Then, a Markov dynamic load environment supporting scheduling interaction is constructed; the environment is modeled as a partially observable Markov decision process, the state space contains information such as the current system load distribution, the resource state of each computing node, the task factor queue, and the time slice position, the action space corresponds to the scheduling decision of assigning a task factor to a node, and the state transition function simulates the change trend of the system resources over time after scheduling; the immediate reward generated by each scheduling action in the environment can be constructed in combination with resource utilization, task waiting time, and blocking rate.

[0138] Finally, in the training phase, the initial strategy is interacted with the environment to generate state-action-reward trajectories, and the advantage function is constructed to estimate the advantage degree of the current strategy relative to the baseline; the proximal policy optimization algorithm prevents the policy from updating too much by clipping the probability ratio; as the training progresses, the policy gradually deviates from the "empirical template" of the fusion strategy and turns to a more adaptive optimal strategy driven by interaction feedback.

[0139] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A large model algorithm network scheduling method based on task combination automation, characterized in that, The method comprises: S1, using a task factor diagram decomposition technology to decompose a task into a plurality of task factors, analyzing data flow and logical dependency relationships between the task factors, and constructing a task factor flow diagram; S2, collecting resource data of each computing power node, simulating resource linkage between the nodes by using a cascading pulse propagation mechanism, generating a pulse propagation topology graph and a pulse intensity matrix, and combining a pre-trained pulse graph neural network to construct a computing power field situation awareness network; S3, combining the computing power field situation awareness network and the task factor flow diagram, using a pre-set space-time convolution predictor to predict resource matching degrees of the task factors and the computing power nodes, and generating an affinity tensor; S4, constructing a joint strategy space according to the affinity tensor, searching for a fusion strategy in the joint strategy space by using a multi-objective equilibrium algorithm, and embedding the fusion strategy in a deep reinforcement learning framework to generate an optimal scheduling strategy.

2. The large model algorithm network scheduling method based on task combination automation according to claim 1, characterized in that, The collection of resource data of each computing power node, the simulation of resource linkage between the nodes by using the cascading pulse propagation mechanism, the generation of the pulse propagation topology graph and the pulse intensity matrix, and the combination of the pre-trained pulse graph neural network to construct the computing power field situation awareness network comprise: S21, collecting resource data of each computing power node in a computing network, and combining logical dependency relationships in the task factor flow diagram to generate demand pulses of the task factors and supply pulses of the computing power nodes; S22, simulating a propagation process of the demand pulses and the supply pulses in the logical dependency relationships between the task factors and a physical connection topology of the computing network by using an attenuation superposition technology, obtaining a pulse propagation topology graph and a pulse intensity matrix; S23, based on the obtained pulse propagation topology graph and the pulse intensity matrix, fusing and encoding logical demand information of the task factors and resource data of the computing power nodes to form a pulsation feature tensor; S24, combining the pulsation feature tensor and the pre-trained pulse graph neural network to establish a pulse interaction matrix, and combining the pulse propagation topology graph to generate a computing power field situation awareness network, the computing power field situation awareness network comprising a situation matrix and a stress conduction thermodynamic map.

3. The large model algorithm network scheduling method based on task combination automation according to claim 2, characterized in that, The simulation of a propagation process of the demand pulses and the supply pulses in the logical dependency relationships between the task factors and the physical connection topology of the computing network by using the attenuation superposition technology to obtain the pulse propagation topology graph and the pulse intensity matrix comprises: S221, combining the logical dependency relationships between the task factors and the physical connection topology of the computing network to construct a polymorphic topology graph, and setting a propagation impedance as a propagation constraint condition; S222, based on the set propagation impedance, driving the demand pulses and the supply pulses to propagate in an exponential attenuation form in the polymorphic topology graph, and generating pulse response intensities by cumulatively superimposing pulse signals of each node in a time sequence through a sliding window mechanism; S223, according to the generated pulse response intensities, establishing a local resource competition field reflecting current computing loads and resource contention states of the nodes, and adjusting the pulse propagation process by using the local resource competition field; S224, optimizing and adjusting the pulse propagation results by using a simulated annealing algorithm, generating a pulse propagation topology graph, and constructing a pulse intensity matrix by counting pulse response intensities of each node.

4. The large model algorithm network scheduling method based on task combination automation according to claim 3, characterized in that, The combination of pulsating feature tensor and pre-trained pulse graph neural network, the establishment of pulse interaction matrix, and the combination of pulse propagation topology graph generate the computing power field situation awareness network, which includes situation matrix and stress conduction thermodynamic map, including: S241, input the pulsating feature tensor into the pre-trained pulse graph neural network, slice the resource data in the time dimension through the pulse coding layer, and map the slicing result to the pulse firing frequency to establish the pulse interaction matrix; S242, based on the pulse interaction matrix and the pulse propagation topology graph, obtain the node potential set, and use the entanglement evolution algorithm to establish the node evolution set, and generate the potential gradient graph by projecting and measuring the node evolution set; S243, according to the potential distribution in the potential gradient graph, calculate the resource balance entropy, and construct the situation matrix of the computing power balance based on the resource balance entropy; S244, combining the potential gradient graph and the pulse intensity matrix, calculating the blocking risk value of each edge in the task factor flow graph, and converting the blocking risk value into the stress conduction thermodynamic map of the task dependency path through color mapping technology.

5. The large model algorithm network scheduling method based on task combination automation according to claim 4, characterized in that, The combination of pulsating feature tensor and pre-trained pulse graph neural network, the establishment of pulse interaction matrix, and the combination of pulse propagation topology graph generate the computing power field situation awareness network, which includes situation matrix and stress conduction thermodynamic map, including: S2421, according to the pulse interaction matrix and the pulse propagation topology graph, convert the pulse response intensity of the computing power node into the amplitude coefficient, and construct the initial potential of each node in the computing power space to form the node potential set; S2422, select a target node pair in the node potential set, use the entanglement operator with pulse response intensity as the weight parameter to perform entanglement evolution on the node potential, and generate an entanglement network; S2423, based on the generated entanglement network, discretize the potential space using the finite difference algorithm, and simulate the transition behavior of the node potential in the computing power space by calculating the tunneling transition probability between nodes, and output the node evolution set containing the node potential change probability and direction; S2424, project and measure the node evolution set, collapse the potential value matrix under the calculation basis, and extract the correlation matrix that has not collapsed using the entropy weight algorithm; S2425, weight and sum the potential value matrix and the correlation matrix to generate the potential gradient graph.

6. The large model algorithm network scheduling method based on task combination automation according to claim 5, characterized in that, The formula for calculating the tunneling transition probability is: where P tunnel (u-v) represents the tunneling transition probability from node u to node v; h represents the reduced Planck constant; path represents the integration path; m eff represents the effective mass; AV uv (s) represents the inter-node potential energy difference function; E k represents the kinetic energy term; s represents the path integration variable.

7. The large model algorithm network scheduling method based on task combination automation according to claim 1, characterized in that, The combination of pulsating feature tensor and pre-trained pulse graph neural network, the establishment of pulse interaction matrix, and the combination of pulse propagation topology graph generate the computing power field situation awareness network, which includes situation matrix and stress conduction thermodynamic map, including: S41, based on the generated affinity tensor, define a multi-objective optimization function, and construct a joint strategy space according to the multi-objective optimization function; S42, according to the constructed joint strategy space, use the non-dominated sorting genetic algorithm for multi-objective optimization to generate a Pareto frontier solution set, and use the Nash equilibrium verification mechanism to screen the Pareto frontier solution set to obtain a scheduling solution set that meets the balance condition; S43, according to the hardware and software constraint conditions, use the preset linear programming model to filter the scheduling solution set, extract the scheduling decision set, and establish the scheduling decision space; S44, constructing a hypergraph structure according to the scheduling decision space and the multi-objective optimization function, and generating a fusion strategy in combination with a preset neural network; S45, embedding the fusion strategy as an initial scheduling strategy into a deep reinforcement learning framework, simulating a Markov dynamic load scenario, and adjusting the initial scheduling strategy through a proximal policy optimization algorithm to generate an optimal scheduling strategy.

8. The large model algorithm network scheduling method based on task combination automation according to claim 7, characterized in that, The constructing a hypergraph structure according to the scheduling decision space and the multi-objective optimization function, and generating a fusion strategy in combination with a preset neural network comprises: S441, mapping each scheduling decision in the scheduling decision space as a vertex of a hypergraph, defining a hyperedge connection based on a Pearson correlation coefficient matrix between the multi-objective optimization functions, and constructing a hypergraph structure reflecting the multi-objective coupling relationship; S442, designing a hypergraph convolutional neural network based on the constructed hypergraph structure, generating a sub-strategy set by fusing the multi-objective constraints in the hyperedge and the structural similarity between the vertices; S443, extracting symbolic logic rules from the sub-strategy set using a rule induction learning algorithm, and generating a fusion vector in combination with a preset neural network to construct a neural-symbolic hybrid strategy network; S444, using the neural-symbolic hybrid strategy network as a teacher network to guide a student network to learn, and generating a fusion strategy in combination with a KL divergence loss function and a cosine similarity penalty term.

9. The large model algorithm network scheduling method based on task combination automation according to claim 8, characterized in that, The extracting symbolic logic rules from the sub-strategy set using a rule induction learning algorithm, and generating a fusion vector in combination with a preset neural network to construct a neural-symbolic hybrid strategy network comprises: S4431, analyzing symbolic logic patterns in the sub-strategy set using a rule induction learning algorithm, and generating a symbolic logic distribution map by aggregating the analysis results through a bagging algorithm; S4432, extracting symbolic logic rules of a target frequency from the symbolic logic distribution map, constructing a symbolic rule template, and optimizing the symbolic rule template based on a scheduling success rate as a reward signal through a policy gradient algorithm; S4433, constructing a gating fusion mechanism according to the optimized symbolic rule template and a preset neural network, extracting neural features and symbolic features for splicing, and inputting the splicing result into a logic fusion function to generate a fusion vector; S4434, synchronously optimizing neural network parameters, symbolic rule template weights, and gating fusion parameters based on the generated fusion vector to construct a neural-symbolic hybrid strategy network.

10. The large model algorithm network scheduling method based on task combination automation according to claim 9, characterized in that, The expression of the logic fusion function is: f=σ(W g ·concat(h n ,h sym )+b g )⊙a n +(1-σ(W g ·concat(h n ,h sym )+b g )⊙a sym ; where f denotes the fusion vector; σ denotes the Sigmoid function; W g denotes the gating weight matrix; concat denotes the concatenation operation; h n denotes the neural feature vector; h sym denotes the symbolic feature vector; b g denotes the gating bias vector; a n denotes the neural network action output; a sym denotes the symbolic rule action output; denotes the element-wise multiplication.

Images