Cross-cloud task arrangement optimization method based on multi-cloud resource scheduling

Abstract

The invention discloses a cross-cloud task arrangement optimization method based on multi-cloud resource scheduling, and the method comprises the steps: constructing a task topological weighted graph, and recognizing a parallelizable node set; constructing a task node preliminary scheduling feasibility table according to task topological weighted hierarchy information; constructing a granularity feature vector, and updating the task structure matrix; identifying granularity collaborative attenuation abnormal nodes, and performing cross-cloud task path repair and structure update; generating a cross-cloud scheduling time matrix based on each cloud platform resource supply capability index; predicting resource competition conflicts in combination with task execution index historical data, and generating a conflict prediction table; identifying granularity strategy deviation, and determining a repair trigger point; and generating a cross-cloud task granularity adaptive arrangement configuration file based on the repair trigger point information. According to the method, high reliability, high flexibility and high adaptability of task scheduling can be realized in a complex multi-cloud heterogeneous environment, and the stability, the resource utilization rate and the overall cooperation efficiency of cross-cloud task execution are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of cloud computing resource management, and in particular to a cross-cloud task orchestration optimization method based on multi-cloud resource scheduling. Background Technology

[0002] With the rapid development of cloud computing technology, multi-cloud and hybrid cloud architectures have become the mainstream models for enterprise task scheduling and resource management. Different cloud platforms differ significantly in computing power, storage, network bandwidth, and resource billing strategies. Users often need to allocate tasks and schedule resources across multiple cloud platforms to achieve optimal cost, performance, and reliability. However, due to the heterogeneity of cloud resources, the dynamic fluctuations of network links, and the complexity of task dependencies, traditional cross-cloud task scheduling methods have significant shortcomings in stability, adaptability, and self-optimization capabilities. Existing technologies are mainly based on static task allocation models or single-cloud scheduling strategies, treating task granularity, dependencies, and resource constraints as fixed parameters. This makes it difficult to cope with dynamic disturbances caused by changes in resource supply and demand, network congestion, and task execution offsets in a multi-cloud environment. Furthermore, most methods only consider single optimization objectives such as resource utilization or task execution time, ignoring key factors such as cross-cloud data transmission latency, dependency path length, and inter-task synchronization relationships, leading to a decrease in overall scheduling efficiency and even causing task chain interruptions or performance degradation. Especially in complex cross-cloud task topologies, excessive splitting or aggregation of task granularity can lead to imbalances in system coordination efficiency, causing granularity drift and execution bottlenecks in task chains. On the other hand, existing scheduling optimization methods typically rely on single centralized computations and manual rule setting, lacking the ability to continuously monitor and correct task execution behavior, and thus failing to achieve adaptive evolution of scheduling strategies. Current multi-cloud task orchestration frameworks also generally lack comprehensive modeling mechanisms for multi-dimensional information such as granularity level, communication strength, and dependency coordination, making it difficult for the system to promptly identify and repair sudden resource imbalances or chain fluctuations. Therefore, a novel cross-cloud task orchestration method that comprehensively considers task structure characteristics, cross-cloud network performance, resource adaptability, and dynamic granular behavior is urgently needed. Summary of the Invention

[0003] This invention addresses the problems existing in the prior art by providing a cross-cloud task orchestration optimization method based on multi-cloud resource scheduling, which mainly includes: Based on the cross-cloud task submission request, obtain the node structure information and dependency paths in the task description file, construct the task topology weighted graph, and identify the set of nodes that can be parallelized. By collecting channel performance data through the cross-cloud network performance detection module, and combining the task dependency matrix and task topology weighted hierarchy information, a hierarchical weighted confidence score matrix is ​​generated, and a preliminary scheduling feasibility table for task nodes is constructed. Based on the real-time resource usage status indicators of the cloud platform and the feature vector of task node execution requirements, the resource matching degree is calculated. Combined with the synchronization frequency and data transmission intensity, a granular feature vector is constructed, a task granularity adjustment index table is generated, and the task structure matrix is ​​updated. Based on the dynamically adjusted task structure matrix, node scheduling window span, and path hop count, identify abnormal nodes with granular collaborative decay and perform cross-cloud task path repair and structure update operations. Based on the resource supply capacity indicators of each cloud platform and the resource demand vector information in the task structure matrix, combined with granular weight factors and scheduling constraints, a cross-cloud scheduling time matrix with granular labels is generated. Based on the granularity labels in the cross-cloud scheduling time matrix, parallel chains and synchronization points are constructed and task topology dependencies are reconstructed. Combined with historical data of task execution indicators, resource contention conflicts are predicted and a conflict prediction table is generated. Based on the full-link behavior data and task granularity adjustment index table in the task monitoring database, the granularity category of the current task node execution status is judged, deviations from the granularity strategy are identified, and the repair trigger point is determined. Based on the repair trigger point information, a granular drift graph is constructed, and the scheduling time matrix of key task nodes is updated to generate a cross-cloud task granularity adaptive orchestration configuration file.

[0004] Furthermore, the step of obtaining node structure information and dependency paths from the task description file based on the cross-cloud task submission request, constructing a task topology weighted graph, and identifying a set of parallelizable nodes includes: Based on the cross-cloud task submission request, the system obtains the task node structure information, task input / output dependencies, data flow description, resource constraints, time window parameters, and task priority from the task description file. The system then performs field extraction and structuring operations on the task description file using a syntax parsing module to obtain the task logical topology graph. A depth-first search algorithm is used to traverse the task logical topology graph, constructing a task node dependency matrix and recording the predecessor and successor relationships between nodes. A Markov transition model is used to normalize the task dependency matrix into a transition probability matrix, and the task node dependency strength vector is obtained by calculating the node transition entropy and jump frequency. A task topology weighted graph is constructed by mapping the values ​​of corresponding task nodes in the dependency strength vector to the node attributes in the topology graph. Based on the out-degree and in-degree of nodes in the task topology weighted graph, a Kahn algorithm based on graph layering is used to perform topological sorting, determining the level number of each task node, and extracting a subset of nodes in each level that do not have data conflicts or share resources, thus identifying the set of nodes that can be parallelized.

[0005] Furthermore, the process involves collecting channel performance data through a cross-cloud network performance detection module, combining the task dependency matrix and task topology weighted hierarchical information to generate a hierarchical weighted confidence score matrix, and constructing a preliminary scheduling feasibility table for task nodes, including: A cross-cloud network performance detection module is used to detect network paths between various cloud platforms in real time, acquiring network performance data for each cross-cloud channel. This data includes bandwidth, latency, jitter, packet loss rate, upper bound of available bandwidth, and temporal jitter distribution. The network performance data is denoised and normalized using a sliding time window method to obtain a multi-cloud channel performance matrix. A Bayesian estimation model is used to predict the channel availability distribution for future time slices. The channel availability distribution is then fused with the task dependency matrix to obtain an executable confidence matrix from task nodes to channels. Based on the executable confidence matrix from task nodes to channels and the task execution hierarchy number obtained from the task topology weighted graph, a hierarchical weighted scoring method is used to calculate the scheduling adaptability of each task node across different cloud platforms, resulting in a hierarchical weighted confidence score matrix. By presetting three scheduling constraints—cloud platform resource threshold, upper limit of dependency path length, and estimated data transmission value—node-platform pairs that meet the conditions are determined, and a preliminary scheduling feasibility table for task nodes is constructed.

[0006] Furthermore, the process of calculating resource matching degree based on real-time resource usage status indicators of the cloud platform and task node execution requirement feature vectors, constructing granular feature vectors by combining synchronization frequency and data transmission intensity, generating a task granularity adjustment index table, and updating the task structure matrix includes: By calling the cloud platform monitoring interface and resource detection agent, real-time resource usage status indicators of each cloud platform are obtained, and the resource matching degree, dependency constraint tightness, and data transmission intensity of each task node are calculated when executing on different cloud platforms. By normalizing the resource matching degree, dependency constraint tightness, and data transmission intensity and synthesizing them according to weights, the granular feature vector of each task node under different cloud platforms is obtained. The magnitude of the granular feature vector is used as the granular risk value to determine whether it exceeds the preset granular risk threshold. For tasks exceeding the granular risk threshold, the node is split, and for nodes below the granular risk threshold, aggregation is performed. By performing node splitting operations on the task topology weighted graph, a single node task is split into multiple sub-task units, and their data flows are remapped to obtain a task splitting mapping table. For aggregation operations, the input and output interfaces of multiple nodes at the same level are merged to obtain a task aggregation mapping table. A hash index is used to record the splitting and aggregation mapping relationship to generate a task granularity adjustment index table. By verifying the consistency of the task granularity adjustment index table, the task structure matrix after dynamic granularity adjustment is obtained.

[0007] Furthermore, the step of identifying abnormal nodes in granular collaborative decay based on the dynamically adjusted task structure matrix, node scheduling window span, and path hop count information, and performing cross-cloud task path repair and structure update operations, includes: Based on the dynamically adjusted task structure matrix, a granularity collaboration detection module is used to obtain the set of task nodes within the same level, filtering out task nodes that underwent splitting or aggregation operations during the previous round of granularity adjustment. The scheduling start and end times of each node are extracted from the task scheduling logs, and the scheduling window span is calculated. The number of communication sessions between node pairs is extracted from task communication monitoring records, and the path hop count between node pairs is calculated based on the adjacency matrix of the task logical topology graph, using the granularity collaboration attenuation factor formula. Calculate the collaborative decay value for each pair of granularity adjustment task nodes (Ti, Tj). ,in, To adjust the scheduling window span of task node i at a granular level, To adjust the scheduling window span of task node j at a granular level, Standardizing relative differences For the number of task communications, The path hop count between two nodes is used. If the collaborative decay value of any pair of granularity adjustment task nodes exceeds the preset decay value threshold, the task node pair is recorded as an abnormal collaborative pair, triggering a granularity collaborative decay alarm event. Based on the task path associated with the abnormal collaborative pair, the corresponding path node set is located in the task topology graph, and broken chain nodes with no intersection in the scheduling time window are identified, and scheduling time reordering is performed. The communication saturation is calculated based on the number of communications of the abnormal pair and the scheduling span. If it exceeds the preset communication threshold, a new data channel identifier is assigned to the path, and an asynchronous buffer node is introduced to delay synchronization. After the path repair is completed, the task granularity adjustment index table and the task splitting mapping table and task aggregation mapping table in the task structure matrix are updated, and the repair record and path verification hash value are written to the scheduling log.

[0008] Furthermore, the generation of a cross-cloud scheduling time matrix with granular labels, based on the resource supply capacity indicators of each cloud platform and the resource demand vector information in the task structure matrix, combined with granular weighting factors and scheduling constraints, includes: A multi-cloud resource monitoring module is used to acquire resource supply capacity indicators for each cloud platform, including CPU core count, memory capacity, I / O bandwidth, GPU acceleration status, storage throughput, container instance load rate, and virtual machine pool latency. By normalizing the standard deviation of these resource supply capacity indicators, a resource performance standard vector is obtained. A multi-dimensional matching calculation is performed between the task structure matrix and the resource performance standard vector to obtain an adaptation score matrix between task nodes and cloud platforms. A greedy matching algorithm is executed on the adaptation score matrix to determine the target cloud platform set for each task node. Based on historical data regarding the number of split sources, data channels, and dependency paths for each task node, a recurrent neural network is used for model training to construct a granular weight factor prediction model, predicting the granular weight factor for each task node. Finally, a constrained optimization model is used based on the target cloud platform set for each task node. The start and end scheduling times of task nodes at the same level are calculated. A granularity weight factor is used as the urgency parameter of the task scheduling window. A time weight correction term is introduced into the objective function, and constrained optimization modeling is performed. Constraints include: minimum waiting time between adjacent tasks is greater than 0; the maximum number of concurrent tasks on each platform does not exceed a preset resource threshold; and all subtasks of the aggregated task must be completed before its start. and These are the start and end times for task Ti, respectively. Let be the start time of the j-th task node. This represents the average start time for tasks at this level. The granular scheduling balance coefficient is obtained by fitting historical data. The optimization model is solved using a heuristic particle swarm optimization algorithm to obtain the specific scheduling start and end times of each task node in the same layer. It is also determined whether the synchronization points of all aggregate nodes meet the latest dependency completion time. The cross-cloud scheduling time matrix with granular weight labels is determined by adjusting the time window.

[0009] Furthermore, the process of constructing parallel chains and synchronization points and reconstructing task topology dependencies based on granular labels in the cross-cloud scheduling time matrix, predicting resource contention conflicts by combining historical task execution metrics data, and generating a conflict prediction table includes: Based on the cross-cloud scheduling time matrix, task instance execution plans are constructed in each cloud platform. Based on the granularity labels in the cross-cloud scheduling time matrix, the parallel chains of split tasks and the synchronization points of aggregated tasks are identified. Synchronization points include granular split nodes and aggregate nodes. A dynamic topology backfilling algorithm is used to establish data transmission paths between split subtasks, and aggregation channels are allocated to aggregate nodes. Historical data on task execution metrics and resource consumption during task execution are obtained from the task monitoring database. A recurrent neural network is used for model training to construct a task execution load prediction model. Task execution metrics include actual resource consumption, CPU cycle consumption, I / O response time, and memory swapping frequency. The entire task chain is simulated in a sandbox environment using a task simulator to obtain the task execution metrics of each subtask. The task execution load prediction model is used to predict the load of each task under actual deployment. Based on the task execution load prediction results, it is determined whether adjacent tasks on the same cloud platform and within the same time window have resource utilization exceeding the load threshold. A conflict prediction table is generated for all task groups with resource contention conflicts.

[0010] Furthermore, the step of determining the granularity category of the current task node's execution status based on the end-to-end behavioral data and task granularity adjustment index table in the task monitoring database, identifying deviations from the granularity strategy, and determining the repair trigger point includes: By accessing log collection interfaces of various cloud platforms, the system acquires end-to-end behavioral data within the task execution cycle and stores it in the task monitoring database. This data includes the actual start and end times of each task node, the number of subtasks, the number of concurrent threads, the number of data streams, the actual I / O wait time, the actual CPU utilization, and the synchronization wait time. Through the task monitoring database, historical end-to-end behavioral data within the task execution cycle is obtained. A random forest algorithm is used for model training to construct a granularity category evaluation model, which determines the granularity category of the current task node's execution status. The granularity categories include fine-grained, coarse-grained, and original configuration granularity. The system also analyzes the performance of split nodes within each time window. The concurrent execution ratio is calculated by dividing the number of concurrently active subtasks by the preset number of subtasks. The aggregation synchronization error is obtained by subtracting the end time of the latest subtask from the actual start time of the aggregated task. The deviation confidence is calculated by comparing the current behavior granularity label with the original configuration in the task granularity adjustment index table. If the current behavior granularity label is inconsistent with the original preset, or the deviation confidence exceeds the set threshold, it is determined that there is a deviation in the granularity strategy. The structural features of behavior deviation are determined by comparing the current execution state vector of the source node with the preset template in the granularity index table. If the matching difference exceeds the threshold, the node is recorded as a repair trigger point.

[0011] Furthermore, based on the repair trigger point information, a granular drift graph is constructed, and the scheduling time matrix of key task nodes is updated to generate a cross-cloud task granularity adaptive orchestration configuration file, including: Based on the granularity classification evaluation and offset trigger list, a granularity drift map is constructed using the task identifier, cloud platform identifier, execution timestamp, and deviation index of the repair trigger point. A shortest path constraint search algorithm is used to determine the set of drift propagation paths. By calculating the influence score of each node in the path set, key granularity nodes are identified, and their configuration parameters in the current scheduling cycle are obtained, including task splitting factor, aggregation threshold, synchronization delay tolerance, and resource reallocation ratio. A gradient descent-based parameter correction algorithm is used to gradually adjust the splitting factor and aggregation threshold to minimize the drift error. By updating the key node parameter vectors and writing them into the scheduling time matrix, a repaired scheduling matrix is ​​obtained. This repaired scheduling matrix is ​​then input into the execution path generation module to regenerate the task path structure. By merging the new task path structure with the original task structure matrix, the optimal cross-cloud task granularity adaptive orchestration configuration file is obtained.

[0012] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: This invention provides a cross-cloud task orchestration optimization method based on multi-cloud resource scheduling. By parsing task descriptions and using topology weighted modeling, this invention integrates task structure information, resource requirement characteristics, and dependency path information into a unified data structure, providing clear boundaries and a global view for task granularity control and scheduling decisions. During the scheduling process, this invention introduces granular feature vectors and granular weight factors to support adaptive splitting and aggregation of task granularity, thereby improving scheduling flexibility and system collaboration efficiency. Furthermore, by constructing a hierarchical weighted confidence score matrix and a task node feasibility table, it achieves dynamic matching and resource adaptation between task nodes and the cloud platform. During task execution, this invention possesses granular collaboration anomaly detection and path self-repair capabilities, automatically identifying broken links, adjusting scheduling windows, and correcting communication channels to ensure the stability of cross-cloud task execution. Combining full-link execution behavior collection, granular policy offset evaluation, and granular drift graph modeling, this invention has the ability to extract deviation signals from historical behavior, backtrack task granularity configuration errors, and automatically correct policy parameters, thus possessing continuously evolving intelligent characteristics. This invention generates a cross-cloud task orchestration configuration file with markers such as granular weights, scheduling windows, parallel chains, and synchronization points by fusing and repairing the scheduling matrix and task path structure. This provides a highly matched execution template for multi-cloud schedulers, supporting efficient, secure, and autonomous task orchestration execution. Overall, this invention achieves high reliability, high flexibility, and high adaptability in task scheduling within complex multi-cloud heterogeneous environments, significantly improving the stability, resource utilization, and overall collaborative efficiency of cross-cloud task execution. Attached Figure Description

[0013] Figure 1 This is a flowchart of a cross-cloud task orchestration optimization method based on multi-cloud resource scheduling according to the present invention; Figure 2 This is a schematic diagram of a cross-cloud task orchestration optimization method based on multi-cloud resource scheduling according to the present invention; Figure 3 This is another schematic diagram of a cross-cloud task orchestration optimization method based on multi-cloud resource scheduling according to the present invention. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0015] like Figure 1-3 This embodiment of a cross-cloud task orchestration optimization method based on multi-cloud resource scheduling may specifically include: Step S101: Based on the cross-cloud task submission request, obtain the node structure information and dependency paths in the task description file, construct the task topology weighted graph, and identify the set of nodes that can be parallelized.

[0016] Based on the cross-cloud task submission request, the system retrieves the task node structure information, task input / output dependencies, data flow description, resource constraints, time window parameters, and task priority from the task description file. The ANTLR syntax tree parsing framework is used to extract and structure the fields of the task description file, identifying task instruction blocks, logic blocks, and flow instruction blocks, and establishing a mapping table between semantic tags and field types. Through field mapping and semantic template comparison, a structured task description dataset is obtained, and a task node structure table is constructed. Based on the task node structure table, a directed graph model is used to construct the task logical topology graph, with task nodes as vertices and dependencies as directed edges. A depth-first search algorithm is used to traverse the task logical topology graph and record the number of reachable paths between all task nodes, resulting in a task dependency path table and a reachability matrix. By combining the dependency path index and the reachability matrix, the task node dependency matrix is ​​obtained, and the task dependency matrix and predecessor / father relationship table are output. A Markov transition model is used to normalize the task dependency matrix into a transition probability matrix. By calculating the node transition entropy and jump frequency, the task node dependency strength vector is obtained. A task topology weighted graph is constructed by mapping the values ​​of corresponding task nodes in the dependency strength vector to node attributes in the topology graph. Based on the out-degree and in-degree of nodes in the task topology weighted graph, the Kahn algorithm, a graph layering algorithm, is used to perform topological sorting to determine the level number of each task node. By extracting a subset of nodes from each level's node set that do not have data conflicts or share resources, a set of nodes that can be parallelized is identified.

[0017] For example, a cross-cloud task scheduling task submission request is received. The task description file contains four task nodes: T1, T2, T3, and T4. There are explicit dependencies between the tasks: the output of T1 is the input of both T2 and T3, and T4 depends on the processing results of T2 and T3 to begin. The task input and output fields are described using field labels as T1.out1→T2.in1, T1.out2→T3.in1, T2.out→T4.in1, and T3.out→T4.in2. The task also sets resource constraints, such as T1 and T2 requiring at least 2 CPU cores and 4GB of memory, T3 requiring GPU acceleration, and T4 requiring a minimum network bandwidth of 50Mbps. Each task node has an optional start and end time window; for example, T1's start time window is [0,5] minutes, and T4 is required to complete before the 15th minute. The system uses the ANTLR syntax tree parser to analyze the task description file, successfully identifying four task blocks and three flow relationship blocks. Semantic labels are created for each field; for example, instruction block T1 is mapped to a type task node, and T1.out1→T2.in1 is labeled as a type data flow. After structured processing, a task node structure table is constructed, explicitly recording fields such as task name, resource requirements, time window, and input / output dependencies. The system models this task set as a directed graph, where nodes correspond to the four tasks T1 to T4, and the direction of edges indicates the dependency direction. For example, an edge from T1 to T2 indicates that T2 depends on the output of T1. When performing a depth-first search on this directed graph, T1 is identified as the unique initial node, and four reachable paths are recorded: from T1 to T2, T1 to T3, T2 to T4, and T3 to T4. This generates a reachability matrix R, where R[i][j] is 1 to indicate that T1 is the task node. i To T j A path exists; otherwise, it is 0. For example, R[T1][T4] is 1, while R[T4][T1] is 0. Combining the reachability matrix and path index, a task node dependency matrix M is generated. This is further normalized to a probability transition matrix P using a Markov transition model, resulting in jump probabilities of 0.5 from T1 to T2 and T3, and 0.6 and 0.4 from T2 and T3 to T4, respectively. After calculating the transition entropy and jump frequency of each node, the dependency strength vector of the task node is obtained. For example, T4 has the highest dependency strength value of 0.89 because it integrates the results of two branch paths. After writing the dependency strength values ​​as weights into the task topology graph, the system executes the Kahn algorithm for graph layering. The results show that T1 is in the first layer, T2 and T3 are in the second layer, and T4 is in the third layer. A subset of tasks without resource conflicts is extracted from each layer. Since T2 and T3 require CPU and GPU respectively, their resources do not overlap, and they are not mutually dependent, therefore they are judged as parallelizable nodes in the second layer.

[0018] Step S102: Collect channel performance data through the cross-cloud network performance detection module, combine the task dependency matrix and task topology weighted hierarchy information to generate a hierarchy weighted confidence score matrix, and construct a preliminary scheduling feasibility table for task nodes.

[0019] A cross-cloud network performance detection module is used to detect network paths between various cloud platforms in real time, acquiring network performance data for each cross-cloud channel. This data includes bandwidth, latency, jitter, packet loss rate, available bandwidth upper bound, and temporal jitter distribution. The network performance data is denoised and normalized using a sliding time window method to obtain a multi-cloud channel performance matrix. A Bayesian estimation model is used to predict the channel availability distribution for future time slices using this matrix. The channel availability distribution is then fused with the task dependency matrix to obtain the executable confidence matrix from task nodes to channels. Based on the executable confidence matrix from task nodes to channels and the task execution hierarchy number obtained from the task topology weighted graph, a hierarchical weighted scoring model is used to calculate the scheduling adaptability of each task node across different cloud platforms. Finally, a hierarchical weighted confidence score matrix is ​​obtained by multiplying each element in the confidence matrix by the inverse weight value of the corresponding task node hierarchy. By pre-setting three scheduling constraint parameters—cloud platform resource threshold, dependency path length limit, and data transmission estimate—it is determined whether each task node has scheduling candidates that meet the scoring threshold and all constraints under a multi-cloud platform environment. A preliminary scheduling feasibility table for task nodes is constructed by recording all node-platform pairs that meet the conditions, including the task ID, candidate cloud platform ID, score, evaluation timestamp, and expected scheduling window.

[0020] For example, in a cross-cloud task scheduling, task node T5 needs to be assigned to the most suitable cloud platform to ensure execution efficiency. This task relies on the output data of the previous node T2 and is located at layer 3 of the task topology. The current network performance detection module collects performance data from three candidate channels, including channel A with a bandwidth of 80Mbps, latency of 30ms, jitter of 5ms, and packet loss rate of 0.8%; channel B with a bandwidth of 60Mbps, latency of 20ms, jitter of 8ms, and packet loss rate of 1.2%; and channel C with a bandwidth as high as 120Mbps, but jitter of 15ms and packet loss rate of 2.3%. After continuously collecting data from each channel within a 5-minute sliding time window, the raw data is normalized and outliers are removed, constructing a 3×6 multi-cloud channel performance matrix, where each row corresponds to one channel, and each column represents the normalized bandwidth, latency, jitter, packet loss rate, bandwidth upper bound, and time-series jitter distribution, respectively. Based on this matrix, a Bayesian time-series prediction model was invoked to estimate the bandwidth stability and transmission success rate of each channel within the next 10 minutes, outputting the availability probability distribution of each channel. Channel A has a predicted stability of 0.87, channel B 0.79, and channel C, despite its high bandwidth, has a stability of only 0.62. This availability distribution was then fused with the path strength (0.92) from T2 to T5 in the task dependency matrix. Matrix multiplication was used to generate an executable confidence matrix from the task node to the channel, resulting in a confidence score of 0.87 × 0.92 = 0.8004 for T5 to channel A. Since T5 is located at layer 3 of the topology, a hierarchical weighted scoring model was used, setting the layer weights to the reciprocal of the layer number (i.e., layer 3 has a weight of 1 / 3). After multiplying by the confidence value, the weighted score of T5 on channel A is approximately 0.8004 × 1 / 3 ≈ 0.267. This calculation was performed on all candidate cloud platforms to obtain the final weighted confidence score matrix. Combining the task scheduling constraints, we introduce that the current remaining CPU resources of cloud platform P1 are 4 cores, the number of hops in the dependency path does not exceed 2, and the estimated data transmission volume from P1 to channel A is 30MB, which does not exceed the upper limit of 50MB. Under the premise that all constraints are met and the score exceeds the set threshold of 0.25, (T5, P1) is recorded as a scheduling candidate and written into the preliminary scheduling feasibility table of task nodes. The corresponding entries are: Task ID=T5, Candidate Cloud Platform ID=P1, Score=0.267, Evaluation Timestamp=1697562000, and Expected Scheduling Window=[10min, 14min].

[0021] Step S103: Calculate the resource matching degree based on the real-time resource usage status indicators of the cloud platform and the task node execution requirement feature vector. Combine the synchronization frequency and data transmission intensity to construct the granular feature vector, generate the task granularity adjustment index table, and update the task structure matrix.

[0022] By calling the cloud platform monitoring interface and resource detection agent, real-time resource usage status indicators of each cloud platform are obtained. After normalization, resource feature vectors are constructed. Real-time resource usage status indicators include CPU utilization, memory usage, I / O throughput, network bandwidth, container queue depth, and GPU acceleration ratio. Cosine similarity is calculated between the resource feature vectors and task node execution requirement vectors to obtain the resource matching degree. By reading the dependency matrix in the task topology weighted graph, the average call interval and dependency path length between nodes are calculated, where the dependency path length is the number of hops on the task dependency path. Historical task execution data logs are backtracked to extract the actual data interaction frequency between task nodes, and the synchronization frequency of each dependency pair is counted within a set time window. Based on the average call interval between nodes, dependency path length, and synchronization frequency of each dependency pair, the dependency constraint tightness formula is used. Determine the tightness value of the dependency constraint for each pair of nodes, where, Let be the dependency constraint tightness value between node i and node j. Let be the synchronization frequency per unit time from node i to node j. The average call interval between nodes. This depends on the path length. The average round-trip time, bandwidth utilization, and packet loss rate of cross-cloud data links are measured using a real-time network probing module, and a linear weighted formula is applied. Calculate the transmission strength index D for each task data stream, where R is the average round-trip time, K is the bandwidth utilization, and B is the packet loss rate. This is the delay weighting coefficient, representing the sensitivity to RTT in transmission strength calculation. This is a bandwidth utilization weighting coefficient used to measure the impact of bandwidth bottlenecks. The packet loss rate weighting coefficient measures the impact of packet loss on overall transmission performance. By normalizing and weighting resource matching degree, dependency constraint tightness, and data transmission intensity, a granular feature vector for each task node under different cloud platforms is obtained. The magnitude of the granular feature vector is used as the granular risk value to determine whether it exceeds a preset granular risk threshold. For tasks exceeding the granular risk threshold, node splitting is performed; for nodes below the threshold, aggregation is performed. By performing node splitting operations on the task topology weighted graph, a single-node task is split into multiple sub-task units, and their data flows are remapped to obtain a task splitting mapping table. For aggregation operations, the input / output interfaces of multiple nodes at the same level are merged to obtain a task aggregation mapping table. A hash index is used to record the splitting and aggregation mapping relationship, generating a task granularity adjustment index table. Consistency verification of the task granularity adjustment index table yields the task structure matrix after dynamic granularity adjustment.

[0023] For example, in a cross-cloud task orchestration process, data is first captured from the monitoring interfaces of cloud platforms P2 and P3 to obtain the current resource usage status. Cloud platform P2 currently has a CPU utilization of 78%, memory usage of 65%, I / O throughput of 210MB / s, network bandwidth of 850Mbps, container queue depth of 5, and GPU acceleration ratio of 1.4. Cloud platform P3's parameters are 64%, 55%, 180MB / s, 760Mbps, 6, and 1.2, respectively. After Z-score normalization, resource feature vectors for each platform are constructed, and cosine similarity is calculated with the resource requirement vector of task node T1. Node T1's resource requirement vector is 70% CPU, 200MB / s I / O, and GPU acceleration capability not less than 1.3. Finally, the resource matching degree of T1 on cloud platform P2 is 0.91, and on cloud platform P3 it is 0.76. The task topology weighted graph reveals a direct data dependency between task T1 and its downstream node T2. Reading the dependency matrix and scheduling logs shows a 2-hop dependency path from T1 to T2, an average call interval of 40 seconds, and 90 data interactions occurring within the past hour. Therefore, according to the dependency constraint tightness formula... Calculation yields ,in, Let be the dependency constraint tightness value between node i and node j. Let be the synchronization frequency per unit time from node i to node j. The average call interval between nodes. This depends on the path length. The average round-trip latency of the data path from cloud platform P2 to cloud platform P3, measured by the cross-cloud link monitoring module, is 120ms, with a bandwidth utilization rate of 0.65% and a packet loss rate of 1.5%. This is determined using a linear weighted formula. Calculate the transmission strength index D for each task data stream, where R is the average round-trip time, K is the bandwidth utilization, and B is the packet loss rate. This is the delay weighting coefficient, representing the sensitivity to RTT in transmission strength calculation. This is a bandwidth utilization weighting coefficient used to measure the impact of bandwidth bottlenecks. This is a packet loss rate weighting coefficient used to measure the impact of packet loss on overall transmission performance. , , All were obtained by fitting historical data. =0.5, =0.3, =0.2, substituting into the linear weighted formula, we obtain the transmission strength index D=60.405. The resource matching degree (0.91), dependency constraint tightness (0.0125), and transmission strength (60.405) are normalized to the [0,1] interval, and then synthesized into a granular feature vector according to preset weights. The magnitude of the granular feature vector of T1 under cloud platform P2 is 0.88, which is higher than the currently set granular risk threshold of 0.75. Therefore, it is determined that T1 needs to perform a granular splitting operation. In the task topology weighted graph, T1 is decomposed into three subtasks T1a, T1b, and T1c, corresponding to different data sub-streams. Simultaneously, the data mapping relationship for these three subtasks is redefined, a task splitting mapping table is generated, and structural changes are recorded through hash indexes, ultimately completing the update of the task structure matrix.

[0024] Step S104: Based on the dynamically adjusted task structure matrix, node scheduling window span, and path hop count information, identify abnormal nodes with granular collaborative decay, and perform cross-cloud task path repair and structure update operations.

[0025] Based on the dynamically adjusted task structure matrix, a granularity-based collaborative detection module is used to obtain the set of task nodes within the same level, filtering out task nodes that underwent splitting or aggregation operations during the previous round of granularity adjustment. The scheduling start and end times of each node are extracted from the task scheduling logs, and the scheduling window span is calculated. The number of communication sessions between node pairs is extracted from task communication monitoring records, and the path hop count between node pairs is calculated based on the adjacency matrix of the task logical topology graph, using the granularity-based collaborative attenuation factor formula. Calculate the collaborative decay value for each pair of granularity adjustment task nodes (Ti, Tj). ,in, To adjust the scheduling window span of task node i at a granular level, To adjust the scheduling window span of task node j at a granular level, Standardizing relative differences For the number of task communications, This represents the path hop count between two nodes. If the collaborative decay value of any pair of granularity adjustment task nodes exceeds a preset decay threshold, this task node pair is recorded as an abnormal collaborative pair, triggering a granularity collaborative decay alarm event. Based on the task paths associated with the abnormal collaborative pair, the corresponding path node set is located in the task topology graph, and broken-chain nodes with no overlap in scheduling time windows are identified, and scheduling time reordering is performed. The communication saturation is calculated based on the number of communications of the abnormal pair and the scheduling span. If it exceeds a preset communication threshold, a new data channel identifier is assigned to the path, and an asynchronous buffer node is introduced to delay synchronization. After path repair is completed, the task granularity adjustment index table and the task splitting mapping table and task aggregation mapping table in the task structure matrix are updated, and the repair record and path verification hash value are written to the scheduling log.

[0026] For example, in a batch of cross-cloud task scheduling, nodes T6 and T7 are both task nodes with adjusted granularity. T6 is one of two subtasks split from a coarse-grained task, and T7 is an aggregation of two smaller tasks. According to the scheduling log, the scheduling time window for T6 is 120ms, and the scheduling time window for T7 is 80ms. Therefore, the time difference between T6 and T7 is 40ms, with the smaller one being 80ms. The path topology diagram shows a direct dependency path between them, and historical monitoring data shows that the number of communications between T6 and T7 is W. 67 =40 times, within a unit time window, the path hop count is D 67 =3. According to the particle size synergistic decay factor formula... Calculate the collaborative decay value for each pair of granularity adjustment task nodes (Ti, Tj). The resulting co-attenuation value was 1.199, where, To adjust the scheduling window span of task node i at a granular level, To adjust the scheduling window span of task node j at a granular level, Standardizing relative differences For the number of task communications, This represents the path hop count between the two nodes. Because the obtained collaborative attenuation value is higher than the system's preset attenuation threshold of 0.9, this granular adjustment task node pair is determined to be an abnormal collaborative pair. The system then locates the upstream and downstream relationship between T6 and T7 in the task path and finds that the actual scheduling windows between T6 and T7 do not overlap, making them typical broken-chain nodes. According to the path repair strategy, the scheduling system reallocates a time window for them. Meanwhile, because... If the path exceeds the communication saturation threshold of 0.15, the data channel is reallocated, and an asynchronous buffering mechanism is inserted. Finally, the repaired path information is written back to the task structure matrix, and its hash value is recorded in the scheduling log for subsequent audit tracing.

[0027] Step S105: Based on the resource supply capacity indicators of each cloud platform and the resource demand vector information in the task structure matrix, and combined with the granular weight factor and scheduling constraints, generate a cross-cloud scheduling time matrix with granular labels.

[0028] A multi-cloud resource monitoring module is used to acquire resource supply capacity indicators for each cloud platform, including CPU core count, memory capacity, I / O bandwidth, GPU acceleration status, storage throughput, container instance load rate, and virtual machine pool latency. By normalizing the standard deviation of these resource supply capacity indicators, a resource performance standard vector is obtained. Multi-dimensional vector matching is performed between the resource demand vector of each task node in the task structure matrix and the resource performance standard vector. Using the inverse Euclidean distance function, the resource fit score between each task and each cloud platform is calculated, resulting in a fit score matrix for each task node and cloud platform. A greedy matching algorithm based on the maximum score priority is applied to the fit score matrix, traversing each task node and prioritizing allocation to the cloud platform with the highest score that is not overloaded, thus determining the target cloud platform set for each task node. Based on historical data regarding the number of split sources, data channels, and dependency paths for each task node, a recurrent neural network is used for model training to construct a granular weight factor prediction model, predicting the granular weight factor for each task node. Based on the target cloud platform set for each task node, a constrained optimization model is used. The start and end scheduling times of task nodes at the same level are calculated. A granularity weight factor is used as the urgency parameter of the task scheduling window. A time weight correction term is introduced into the objective function, and constrained optimization modeling is performed. Constraints include: minimum waiting time between adjacent tasks is greater than 0; the maximum number of concurrent tasks on each platform does not exceed a preset resource threshold; and all subtasks of the aggregated task must be completed before its start. and These are the start and end times for task Ti, respectively. Let be the start time of the j-th task node. This represents the average start time for tasks at this level. The granular scheduling balance coefficient is obtained by fitting historical data. A heuristic particle swarm optimization algorithm is used to solve the optimization model, obtaining the specific start and end times of scheduling for each task node at the same level. It is then determined whether the synchronization points of all aggregated nodes meet the latest dependency completion time. A cross-cloud scheduling time matrix with granular weights is determined by adjusting the time window.

[0029] For example, during a cross-cloud task orchestration process, three task nodes to be scheduled were detected, namely tasks T8, T9, and T1. 10According to data collected by the multi-cloud resource monitoring module, the current resource supply capabilities of the three available cloud platforms are as follows: Platform P4 has 32 CPU cores, 128GB of memory, and 500MB / s I / O bandwidth; Platforms P5 and P6 have 24 cores / 96GB / 400MB / s and 48 cores / 256GB / 600MB / s, respectively. After standard deviation normalization, resource performance standard vectors for the three platforms were generated. By reading the task structure matrix, it was found that T8 requires 16 CPU cores and 64GB of memory, while T9 requires 20 cores / 80GB of memory. 10 Requires 36 cores / 128GB. After calculating the inverse Euclidean distance between the resource requirement vector of each task and the performance vectors of the three major platforms, an adaptation score matrix is ​​obtained. For example, the scores for T8 on platforms P4, P5, and P6 are 0.82, 0.76, and 0.91 respectively, and for T9 they are 0.79, 0.68, and 0.88. 10 The scores are 0.65, 0.59, and 0.97. Since the platform P6 score is the highest and the current resources are not overloaded, a greedy matching algorithm is used to allocate all three tasks to platform P6. Historical data on the number of split sources, data channels, and dependency paths for each task node are obtained by calling historical operation logs. A recurrent neural network is used to train the model, constructing a granular weight factor prediction model, and extracting T8, T9, and T... in real time. 10 The granular context information, T8 originates from a task unit split into 4 subtasks, with 2 output data channels and 3 layers of dependency path depth, T9 and T 10 Based on the data originating from aggregation nodes and non-split nodes respectively, a granularity weighting factor prediction model was used to predict the granularity weighting factors of the three nodes. The results showed that T8 was 0.9 (biased), T9 was 0.4 (moderate), and T... 10 A value of 0.2 is relatively light. To rationally schedule the timing, the objective function... Based on this, a granularity factor is introduced to correct for time urgency, with λ=0.6. Constraints are set as follows: minimum waiting time between adjacent tasks is greater than 0; the maximum number of concurrent tasks on each platform does not exceed a preset resource threshold; and all subtasks of the aggregated task must be completed before its start. and Task T i The start and end times of the scheduling. Let be the start time of the j-th task node. This represents the average start time for tasks at this level. The granularity scheduling balance coefficient is obtained by fitting historical data. Using a heuristic particle swarm optimization algorithm, T8 starts from the 10th second, T9 starts from the 12th second, and T... 10Starting from the 14th second, with scheduling end times of 20 seconds, 22 seconds, and 25 seconds respectively, all tasks run concurrently on platform P6, complying with all resource and topology constraints. The synchronization detection module verifies that the execution point of aggregate task T2 is after all its dependent tasks have been completed, and writes the scheduling time matrix of this batch of tasks into the system.

[0030] Step S106: Based on the granularity labels in the cross-cloud scheduling time matrix, construct parallel chains and synchronization points and reconstruct task topology dependencies. Combine historical data of task execution indicators to predict resource contention conflicts and generate a conflict prediction table.

[0031] Based on the cross-cloud scheduling time matrix, task instance execution plans are constructed in each cloud platform. Using the granularity labels in the cross-cloud scheduling time matrix, the parallel chains of split tasks and the synchronization points of aggregated tasks are identified. Synchronization points include granular split nodes and aggregate nodes. A dynamic topology backfilling algorithm is used to reconstruct the dependency chains of the task nodes after granular splitting, including the set of subtasks split from each original task node Ti. Delete the original edge T in the task topology graph p To T i With T i To T q T p T q Given the predecessor and successor nodes, and traversing the set of subtasks, establish path edges T sequentially according to the input-output mapping rules of the original task. p To T ij T ij To T qThe process involves: 1) Setting data transmission channel identifiers using path update rules, adding bandwidth and latency attributes to each edge, constructing a new set of dependent edges, and merging it into the topology graph to obtain the updated task execution graph. 2) For granular aggregation nodes, by summarizing the start and end times of upstream tasks corresponding to all input edges, selecting the maximum end time as its executable synchronization point, marking the synchronization dependency set, and assigning a unified aggregation channel identifier for the data aggregation path. 3) Obtaining historical data on task execution metrics and resource consumption during task execution from the task monitoring database, using a recurrent neural network for model training, and constructing a task execution load prediction model. Task execution metrics include actual resource consumption, CPU cycle consumption, I / O response time, and memory swapping frequency. 4) Simulating the entire task chain using a task simulator in a sandbox simulation environment to obtain the task execution metrics of each sub-task, and using the task execution load prediction model to predict the load of each task under actual deployment. 5) Based on the task execution load prediction results, determining whether there are adjacent tasks on the same cloud platform and within the same time window whose resource utilization exceeds the load threshold, generating a conflict prediction table for all task groups with resource contention conflicts. The conflict prediction table includes the task ID, expected overlapping time period, conflict resource type, conflict intensity value, and recommended migration candidates.

[0032] For example, in a cross-cloud task scheduling, the scheduler has confirmed, based on the previously calculated scheduling time matrix, that task T1 has been split into three subtasks T1. 1-1 T 1-2 and T 1-3 The tasks are deployed on cloud platforms P3, P4, and P5, with start and end times of (10s, 20s), (11s, 21s), and (12s, 22s), respectively, forming a parallel chain of split tasks. Task T1's original predecessor is T0, and its successor is T2. During dependency chain reconstruction, the edges from T0 to T1 and from T1 to T2 in the original topology graph are first deleted. Based on the split mapping, three edges from T0 to T1 are reconstructed. 1-1 T0 to T 1-2 T0 to T 1-3 Each edge is assigned an independent data channel identifier and performance attributes, such as channel A having an RTT of 20ms, channel B having an RTT of 35ms, and channel C having an RTT of 18ms. Based on the data transmission path strategy, three edges are established, each starting from T... 1-1 T 1-2 and T 1-3 The path to task T2 is identified and tagged with an aggregation label, indicating that the data needs to be converged at T2. When processing aggregation task T2, the end time of the upstream task is extracted from all its input edges, and it is found that T... 1-1 T 1-2 and T 1-3The completion times for T2 are 20s, 21s, and 22s respectively. Therefore, the maximum completion time of 22s is taken as the executable synchronization point for T2, ensuring that T2 starts uniformly after all data is aggregated. A unified aggregation channel identifier is also assigned to these input paths. To predict resource conflicts during the execution phase, historical execution data is loaded from the task monitoring database, and a recurrent neural network is used for model training to build a task execution load prediction model. This model predicts the resource load of each subtask in the actual deployment environment, thus predicting T2. 1-2 On platform P4, the CPU cycle consumption is 12000 cycles, the I / O response time is 15ms, and the memory swapping frequency is 5 times / s. Based on these values ​​and the currently available resources on platform P4, its load utilization is calculated to be 82%. Meanwhile, another task T4 deployed on platform P4 at the same time is also predicted to have a resource utilization of 76%. Because T... 1-2 Both task T and T4 had execution times between 11 and 21 seconds and were on the same platform. Within this time window, the total CPU utilization reached 158%, exceeding the platform P4's set threshold of 140%. Therefore, a conflict record was generated and entered into the conflict prediction table, labeled as task T. 1-2 A CPU resource conflict occurred with task T4 between 11s and 21s, with a conflict intensity of 18%. It is recommended to postpone task T. 1-2 Migrate to P4, a cloud platform with higher resource availability.

[0033] Step S107: Based on the full-link behavior data and task granularity adjustment index table in the task monitoring database, determine the granularity category of the current task node execution status, identify deviations in granularity strategy, and determine the repair trigger point.

[0034] By accessing the log collection interfaces of various cloud platforms, the system acquires end-to-end behavioral data within the task execution cycle and stores it in the task monitoring database. This data includes the actual start and end times of each task node, the number of subtasks, the number of concurrent threads, the number of data streams, the actual I / O wait time, the actual CPU utilization, and the synchronization wait time. Through the task monitoring database, historical end-to-end behavioral data within the task execution cycle is obtained. A random forest algorithm is used for model training to construct a granularity category evaluation model, which determines the granularity category of the current task node's execution status. The granularity categories include fine-grained, coarse-grained, and original configuration granularity. The concurrent execution ratio is calculated by statistically analyzing the number of concurrently active subtasks of split nodes within each time window and comparing it to the preset number of subtasks. The aggregation synchronization error is obtained by subtracting the end time of the latest subtask from the actual start time of the aggregated task. The deviation confidence level is calculated by comparing the current behavior granularity label with the original configuration in the task granularity adjustment index table. If the current behavior granularity label is inconsistent with the original preset, or the deviation confidence level exceeds a set threshold, a granularity strategy deviation is determined. By comparing the current execution state vector of the source node with the preset template in the granularity index table, it is determined whether there are structural features of behavioral deviation. If the matching difference exceeds the threshold, the node is recorded as the repair trigger point.

[0035] For example, during the execution of a cross-cloud task, the actual execution record of task node T5 is obtained from the cloud platform P6 through the log collection interface. Monitoring data shows that the actual start time of T5 is 100 seconds and the end time is 160 seconds. It is divided into 5 sub-tasks within the execution cycle, with 10 concurrent threads and 4 data streams generated. At the same time, the average I / O wait time is 12 milliseconds, the average CPU utilization reaches 87%, and the synchronization wait time is 6 seconds. The above end-to-end behavioral data is stored in the task monitoring database, and a random forest algorithm is used to train the model to build a granularity category evaluation model. This model judges the granularity category of the current task node execution status. The granularity category output by the model is fine-grained, and the granularity categories include fine-grained, coarse-grained, and original configuration granularity. The scheduler further compares the original granularity configuration of task T5 recorded in the task granularity adjustment index table and finds that its preset granularity should be the original configuration granularity, that is, no splitting operation, only single-threaded processing of the main task logic. Since the current behavior granularity label is inconsistent with the preset label, the analysis begins to determine whether the deviation is significant. First, the system calculates the concurrent execution ratio of task T5 within this time period. The original preset number of subtasks for task T5 is 2, while the actual number of active subtasks is 5. Therefore, the concurrent execution ratio is 5 ÷ 2 = 2.5, far exceeding the set upper limit threshold of 1.5, indicating that the task has been over-split, potentially impacting resource efficiency. Next, the execution behavior of task T6, which is related to the aggregation of T5, is examined. The actual start time of aggregation node T6 is 162 seconds, and the completion time of its latest subtask, a subtask in T5, is 160 seconds. Therefore, the aggregation synchronization error is 162 - 160 = 2 seconds, with an aggregation error tolerance within 5 seconds, which is within the normal range. However, in the deviation confidence calculation, a cosine similarity comparison is performed between the behavior state vector and the template at the original configured granularity in the index table. The result shows a matching score of only 0.68, while the threshold is set at 0.85. This indicates a significant deviation between the current execution state and the original preset template. The behavior state vector includes dimensions such as concurrency, resource consumption, and latency. Ultimately, based on a comprehensive assessment, the behavior of task T5 has significantly deviated from its preset granularity configuration, accompanied by an increase in resource scheduling pressure and an abnormal increase in the number of subtasks. Therefore, it is marked as a repair trigger point and written into the scheduler repair trigger log, in preparation for initiating the granularity correction mechanism.

[0036] Step S108: Based on the repair trigger point information, construct a granular drift graph, update the scheduling time matrix of key task nodes, and generate a cross-cloud task granular adaptive orchestration configuration file.

[0037] Based on the granularity classification evaluation and offset trigger list, a granularity drift map is constructed using the task identifier, cloud platform identifier, execution timestamp, and deviation index of the repair trigger point. A shortest path constraint search algorithm is used to determine the set of drift propagation paths. By calculating the influence score of each node in the path set, key granularity nodes are identified, and their configuration parameters for the current scheduling cycle are obtained, including task splitting factor, aggregation threshold, synchronization delay tolerance, and resource reallocation ratio. A gradient descent-based parameter correction algorithm is used to gradually adjust the splitting factor and aggregation threshold to minimize the drift error. By updating the key node parameter vectors and writing them into the scheduling time matrix, a repaired scheduling matrix is ​​obtained. This repaired scheduling matrix is ​​then input into the execution path generation module to regenerate the task path structure. By merging the new task path structure with the original task structure matrix, the optimal cross-cloud task granularity adaptive orchestration configuration file is obtained.

[0038] For example, the scheduling monitoring module detected that task node T8 running on cloud platform P7 exhibited a significant behavioral deviation after this round of granular dynamic adjustment: the system-recorded deviation confidence level was 0.92, exceeding the system's preset threshold of 0.85, therefore T8 was marked as a repair trigger point. The task identifier T8, the cloud platform P7, the current timestamp 14:02:31, and its deviation indicators, including a scheduling window offset of 18s and a resource utilization anomaly ratio of 1.45, were read from the offset trigger list, thus initiating the construction of the granular drift graph. During the construction of the granular drift graph, starting from T8, the potential impact path was analyzed using the task dependency graph and scheduling records, identifying task nodes T8, T9, and T... 10 T 11 The set of drift propagation paths. Further calculation of the influence score for each node in the path; for example, T8 has an influence score of 0.82 due to direct offset and high-frequency communication with T9, T9 has 0.65, while T... 11Because of their distance from the propagation source and weak dependency chain, the impact is only 0.28. Therefore, the system identifies T8 and T9 as key granularity nodes. Next, the key configuration parameters of T8 and T9 in the current scheduling cycle are extracted. T8's current task splitting factor is 3, indicating it's divided into 3 subtasks; the aggregation threshold is 0.6, meaning it needs to wait for 60% of the subtasks to complete before aggregation; the synchronization delay tolerance is 5 seconds; and the resource redistribution ratio is 0.3. These parameters are fine-tuned using gradient descent, with the goal of minimizing the overall path drift error. After 10 iterations, T8's splitting factor is finally reduced to 2.4, and the aggregation threshold is fine-tuned to 0.7. The updated parameters are written into the scheduling time matrix, and the repaired scheduling scheme is regenerated accordingly and input into the execution path generation module. The generated new task path structure shows that T8's start time is advanced to 13:58, and the end time is compressed to 14:06, mitigating the delay impact on T9's task window. Finally, the newly generated path structure is merged with the original structure matrix to form an updated cross-cloud task granularity adaptive orchestration configuration file, and the hash of this round of update records is written to the scheduling log for auditing.

[0039] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the concept of this application. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for cross-cloud task orchestration optimization based on multi-cloud resource scheduling, characterized in that, The method comprises: According to the cross-cloud task submission request, the node structure information and the dependency path in the task description file are obtained, the task topology weighted graph is constructed, and the parallel node set is identified; Through the cross-cloud network performance detection module, the channel performance data is collected, the level weighted confidence score matrix is generated by combining the task dependency matrix and the task topology weighted level information, and the task node preliminary scheduling feasibility table is constructed; According to the cloud platform real-time resource usage state index and the task node execution demand characteristic vector, the resource matching degree is calculated, the granularity characteristic vector is constructed by combining the synchronous frequency and the data transmission intensity, the task granularity adjustment index table is generated, and the task structure matrix is updated; According to the task structure matrix after granularity dynamic adjustment, the node scheduling window span and the path hop information, the granularity cooperative attenuation abnormal node is identified, and the cross-cloud task path repair and structure update operation is performed; Based on the resource supply capacity index of each cloud platform and the resource demand vector information in the task structure matrix, the cross-cloud scheduling time matrix with granularity labels is generated by combining the granularity weight factor and the scheduling constraint condition; According to the granularity labels in the cross-cloud scheduling time matrix, the parallel chain and the synchronization point are constructed, and the task topology dependency relationship is reconstructed, the resource competition conflict is predicted by combining the task execution index historical data, and the conflict prediction table is generated; According to the full-link behavior data in the task monitoring database and the task granularity adjustment index table, the granularity category of the current task node execution state is judged, the granularity strategy deviation is identified, and the repair trigger point is determined; Based on the repair trigger point information, the granularity drift graph is constructed, the scheduling time matrix of the key task node is updated, and the cross-cloud task granularity self-adaptive arrangement configuration file is generated.

2. The method of claim 1, wherein, According to the cross-cloud task submission request, the node structure information and the dependency path in the task description file are obtained, the task topology weighted graph is constructed, and the parallel node set is identified, comprising: According to the cross-cloud task submission request, the task node structure information, the task input and output dependency, the data flow description, the resource constraint item, the time window parameter and the task priority in the task description file are obtained; the task logic topology graph is obtained by field extraction and structuring operation of the task description file through the syntax analysis module; the task node dependency matrix is constructed by traversing the task logic topology graph using the depth-first search algorithm, and the predecessor and successor relationship between nodes is recorded; the task dependency matrix is normalized into a transition probability matrix by using the Markov transition model, and the task node dependency strength vector is obtained by calculating the node transition entropy and the jump frequency; the task topology weighted graph is constructed by mapping the values of the corresponding task nodes in the dependency strength vector to the node attributes of the topology graph; the Kahn algorithm based on graph layering algorithm is used to perform topology sorting through the out-degree and in-degree of the nodes in the task topology weighted graph, the level number of each task node is determined, and the node subset without data conflict and resource sharing in each layer node set is extracted to identify the parallel node set.

3. The method of claim 1, wherein, The channel performance data is collected by the cross-cloud network performance detection module, the hierarchical weighted confidence score matrix is generated by combining the task dependency matrix and the task topology weighted level information, and the task node preliminary scheduling feasibility table is constructed, including: The cross-cloud network performance detection module is used to perform real-time detection on the network paths between the cloud platforms, and the network performance data of each cross-cloud channel is obtained, including bandwidth, delay, jitter, packet loss rate, available bandwidth upper limit and timing jitter distribution; The network performance data is denoised and normalized by using a sliding time window method to obtain a multi-cloud channel performance matrix; The Bayesian estimation model is used to perform timing prediction on the performance matrix to obtain the channel availability distribution of the future time slice; The executable confidence matrix of the task node to the channel is obtained by performing matrix fusion operation on the channel availability distribution and the task dependency matrix; the scheduling adaptability of each task node in different cloud platforms is calculated by using the hierarchical weight scoring method according to the executable confidence matrix of the task node to the channel and the task execution level number obtained from the task topology weighted graph, and the hierarchical weighted confidence score matrix is obtained; the node platform pair that meets the conditions is determined by using three scheduling constraint parameters, i.e., the preset cloud platform resource threshold, the upper limit of the dependent path length and the data transmission estimation value, and the task node preliminary scheduling feasibility table is constructed.

4. The method of claim 1, wherein, According to the real-time resource usage state index of the cloud platform and the task node execution demand feature vector, the resource matching degree is calculated, the granularity feature vector is constructed by combining the synchronous frequency and the data transmission intensity, the task granularity adjustment index table is generated, and the task structure matrix is updated, including: The real-time resource usage state index of each cloud platform is obtained by calling the cloud platform monitoring interface and the resource detection agent, and the resource matching degree, the dependent constraint tightness and the data transmission intensity of each task node when executed in different cloud platforms are calculated; the granularity feature vector of each task node in different cloud platforms is obtained by normalizing the resource matching degree, the dependent constraint tightness and the data transmission intensity and combining them according to the weight; the granularity risk value is taken as the module length of the granularity feature vector to determine whether it exceeds the preset granularity risk threshold, and the task execution node that exceeds the granularity risk threshold is split, and the node that is below the granularity risk threshold is aggregated; the single-node task is split into multiple sub-task units by performing the node splitting operation on the task topology weighted graph, and the data flow is remapped to obtain the task splitting mapping table, and for the aggregation operation, the input and output interfaces of multiple nodes in the same layer are combined to obtain the task aggregation mapping table; The hash index is used to record the splitting and aggregation mapping relationship to generate the task granularity adjustment index table, and the task structure matrix after the granularity dynamic adjustment is obtained by performing consistency verification on the task granularity adjustment index table.

5. The method of claim 1, wherein, According to the task structure matrix after the granularity dynamic adjustment, the node scheduling window span and the path hop information, the granularity collaborative attenuation abnormal node is identified, and the cross-cloud task path repair and structure update operation is performed, including: According to the task structure matrix dynamically adjusted in granularity, a granularity coordination detection module is used to obtain a task node set in the same level, and task nodes subjected to splitting or aggregation operation in a previous round of granularity adjustment are screened out; a scheduling start time and an end time of each node are extracted from a task scheduling log, and a scheduling window span is calculated; a communication frequency between node pairs is extracted from a task communication monitoring record, and a path hop count between the node pairs is calculated according to an adjacency matrix of a task logic topology graph, and a granularity coordination decay factor formula is used to calculate a coordination decay value of each pair of granularity adjustment task nodes (Ti, Tj) , wherein, is a granularity adjustment task node i scheduling window span, is a granularity adjustment task node j scheduling window span, is a relative difference standardization, is a task communication frequency, is a path hop count between two nodes; if the coordination decay value of any pair of granularity adjustment task nodes is greater than a preset decay value threshold, the pair of task nodes is recorded as an abnormal coordination pair, and a granularity coordination decay alarm event is triggered; according to a task path associated with the abnormal coordination pair, a corresponding path node set is located in a task topology graph, a broken link node with no intersection of scheduling time windows is identified, and a scheduling time rearrangement operation is performed; a communication saturation degree is calculated according to the communication frequency of the abnormal pair and the scheduling span, and if the communication saturation degree exceeds a preset communication threshold, a new data channel identifier is assigned to the path, and an asynchronous buffer node is introduced to delay synchronization; after the path is repaired, a task granularity adjustment index table, a task splitting mapping table and a task aggregation mapping table in a task structure matrix are updated, and a repair record and a path check hash value are written in a scheduling log.

6. The method of claim 1, wherein, The cross-cloud scheduling time matrix with granularity labels is generated based on the resource supply capability indicators of each cloud platform, the resource demand vector information in the task structure matrix, the granularity weight factor, and the scheduling constraint conditions, and includes: The multi-cloud resource monitoring module is adopted to obtain the resource supply capability indicators of each cloud platform, including the number of CPU cores, memory capacity, I / O bandwidth, GPU acceleration status, storage throughput, container instance load rate, and virtual machine pool delay; the resource performance standard vector is obtained by performing standard deviation normalization processing on the resource supply capability indicators; the multi-dimensional matching calculation is performed on the task structure matrix and the resource performance standard vector to obtain the adaptation score matrix of the task nodes and the cloud platforms; the greedy matching algorithm is executed on the adaptation score matrix to determine the target cloud platform set of each task node; the recurrent neural network is used for model training based on the historical data of the number of split sources, the number of data channels, and the number of dependent paths of each task node to construct a granularity weight factor prediction model, and the granularity weight factor of each task node is predicted; According to the task node target cloud platform set, a constraint optimization model is adopted The start and end scheduling times of the same layer task nodes are calculated, the granularity weight factor is taken as the task scheduling window urgency parameter, a time weight correction term is introduced into the objective function, constraint optimization modeling is performed, and the constraint conditions include that the minimum waiting time of adjacent tasks is greater than 0, the maximum number of concurrent tasks of each platform does not exceed the preset resource threshold, and all subtasks of the aggregated task must be completed before the start of the aggregated task, wherein, and the scheduling start and end times of the task Ti are respectively, the start time of the jth task node is, the average start time of the level task is, the granularity scheduling balance coefficient is obtained by fitting historical data; a heuristic particle swarm algorithm is used to solve the optimization model, the specific scheduling start and end times of each same layer task node are obtained, whether the synchronization points of all aggregated nodes satisfy the latest dependent completion time is judged, and the cross-cloud scheduling time matrix with the granularity weight mark is determined by adjusting the time window.

7. The method of claim 1, wherein, The parallel chain and the synchronization point are constructed according to the granularity labels in the cross-cloud scheduling time matrix, the task topology dependency relationship is reconstructed, the resource competition conflict is predicted based on the historical data of the task execution indicators, and the conflict prediction table is generated, and includes: The task instance running plan is constructed in each cloud platform according to the cross-cloud scheduling time matrix, and the parallel chain of the split task and the synchronization point of the aggregated task are identified based on the granularity labels in the cross-cloud scheduling time matrix, and the synchronization point includes the granularity split node and the aggregation node; the data transmission path between the split sub-tasks is established by using the dynamic topology backfilling algorithm, and the aggregation channel is allocated for the aggregation node; The historical data of the task execution indicators and the resource occupation during task execution are obtained from the task monitoring database, the recurrent neural network is used for model training to construct a task execution load prediction model, and the task execution indicators include the actual resource occupation, CPU cycle consumption, I / O response time, and memory exchange frequency; the entire task chain is simulated and executed in a sandbox simulation environment by using a task simulator to obtain the task execution indicators of each sub-task, the task execution load prediction model is used to predict the load of each task under actual deployment; whether there is a resource utilization rate exceeding the load threshold in the same cloud platform and the same time window of adjacent tasks is determined based on the task execution load prediction result, and a conflict prediction table is generated by grouping all tasks with resource competition conflicts.

8. The method of claim 1, wherein, The granularity category of the current task node execution state is judged based on the full-link behavior data in the task monitoring database and the task granularity adjustment index table, the granularity strategy deviation is identified, and the repair trigger point is determined, and includes: The full-link behavior data in a task execution period is obtained by accessing the cloud platform log collection interface, and stored in a task monitoring database, including the actual start and end time of each task node, the number of sub-tasks, the number of concurrent threads, the number of data streams, the actual I / O waiting time, the actual CPU utilization, the synchronization waiting time, and the like; the historical full-link behavior data in the task execution period is obtained through the task monitoring database, a model training is performed using a random forest algorithm, a granularity category evaluation model is constructed, a granularity category judgment is performed on the current task node execution state, and the granularity category includes fine granularity, coarse granularity and original configuration granularity; a concurrent execution ratio is obtained by calculating the ratio of the number of concurrent active sub-tasks in each time window to the preset number of sub-tasks; a synchronization error is obtained by aggregating the actual start time of the task and subtracting the end time of the latest sub-task; a deviation confidence is calculated by comparing the current behavior granularity label with the original configuration in the task granularity adjustment index table, and if the current behavior granularity label is inconsistent with the original preset or the deviation confidence exceeds a set threshold, it is judged that there is a granularity strategy deviation; By comparing the current execution state vector of the source node with the preset template in the granularity index table, it is determined whether there is a structural feature of behavior deviation, and if the matching difference exceeds a threshold, the node is recorded as a repair trigger point.

9. The method of claim 1, wherein, Based on the repair trigger point information, a granularity drift graph is constructed, and a scheduling time matrix of key task nodes is updated to generate a cross-cloud task granularity self-adaptive arrangement configuration file, including: According to the granularity classification evaluation and the offset trigger list, a granularity drift graph is constructed based on the task identifier, cloud platform identifier, execution timestamp and deviation index of the repair trigger point; a shortest path constraint search algorithm is used to determine a drift propagation path set, the influence degree score of each node in the path set is calculated to judge the key granularity node, and the configuration parameters of the key granularity node in the current scheduling period are obtained, including the task splitting factor, the aggregation threshold, the synchronization delay tolerance and the resource reallocation ratio; a parameter correction algorithm based on gradient descent is used to gradually adjust the splitting factor and the aggregation threshold to make the drift error tend to be minimum; the scheduling matrix after repair is obtained by updating the key node parameter vector and writing into the scheduling time matrix, the scheduling matrix after repair is input into the execution path generation module to regenerate the task path structure; The optimal cross-cloud task granularity self-adaptive arrangement configuration file is obtained by fusing the new task path structure with the original task structure matrix.

Images