A method and system for seamless migration of distributed computing tasks under fiber optic network interruption

Abstract

This invention discloses a method and system for seamless migration of distributed computing tasks under fiber optic network interruption, comprising: constructing a distributed task migration system and outputting the system architecture; collecting network status indicators, constructing a network topology graph, and analyzing fiber optic link status for fault prediction; obtaining the running status of computing tasks, establishing sub-task mapping relationships, constructing a directed dependency graph, and generating a state snapshot; analyzing resource requirements, reserving backup resources, and deploying a distributed caching system; analyzing the scope of fault impact, extracting the affected computing subgraph, and selecting the migration target node; preloading the environment on the target node, reconstructing the execution context, and redirecting the communication path; setting a data change capture mechanism, synchronizing incremental data, and performing consistency checks. This invention achieves seamless migration of computing tasks, ensures task continuity and data consistency, and improves the reliability of distributed computing systems under fiber optic network failure scenarios.

Description

Technical Field

[0001] This invention relates to the fields of distributed computing and fiber optic networks, specifically to a method and system for seamless migration of distributed computing tasks under fiber optic network interruption. Background Technology

[0002] With the widespread application of distributed computing technology, large-scale distributed computing systems have become a crucial infrastructure supporting fields such as machine learning, big data analytics, and scientific computing. These systems typically consist of multiple computing nodes interconnected via high-speed fiber optic networks or dedicated networks such as InfiniBand to collaboratively complete complex computational tasks. Network reliability is particularly critical during the execution of computational tasks. With the development of computing network infrastructure and the practical need for "raw data not leaving the domain," the demand for cross-domain large-scale distributed computing systems is increasing, making the coordination of computing, storage, and networks equally important. Data interaction and task collaboration between nodes or data centers depend on the reliability and stability of network connections, especially during long-running computational tasks, where network reliability is paramount.

[0003] Common fault-tolerance mechanisms in current distributed computing systems mainly include redundant data storage and computational task checkpointing mechanisms. For example, the Hadoop Distributed File System (HDFS) ensures data availability through a multi-replica strategy of data blocks; computing frameworks such as Spark employ Resilient Distributed Datasets (RDDs) and periodic checkpointing techniques to cope with node failures. These techniques can provide a certain degree of protection when computing nodes fail, but their ability to cope with network outages is limited.

[0004] Protection switching technology in fiber optic networks can restore network connectivity through pre-planned backup paths when a physical link fails. Typical optical network protection mechanisms include 1+1 protection, 1:1 protection, and shared mesh protection. These mechanisms can rebuild the optical path within 20-50 milliseconds. However, existing distributed computing frameworks lack effective coordination with optical network protection mechanisms. When a fiber optic link failure triggers protection switching, distributed computing tasks are often forced to restart due to timeouts or session interruptions, failing to take advantage of the rapid recovery characteristics of optical networks.

[0005] The main drawbacks of existing technologies are: the fault handling mechanisms of distributed computing systems and the underlying network layer are independent of each other, lacking cross-layer collaborative optimization; the recovery of computing tasks after network interruption usually relies on global checkpoint rollback, resulting in a large waste of computing resources; and when the network topology changes, there is a lack of dynamic task migration mechanisms that can quickly adapt to the new network state, making it difficult to guarantee the continuity and efficiency of complex distributed computing tasks. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for seamless migration of distributed computing tasks under fiber optic network interruption, overcoming the shortcomings of existing distributed computing systems that lack coordination with optical network protection mechanisms, and realizing seamless migration of computing tasks under fiber optic network interruption.

[0007] To achieve the above objectives, the present invention provides a method for seamless migration of distributed computing tasks under fiber optic network interruption, comprising:

[0008] Build a distributed task migration system, and output the system architecture by deploying a fault detection agent, task monitor, migration scheduler, resource manager, and cache manager;

[0009] Based on the fault detection agent, network status indicators are collected, a network topology map is constructed, the status of optical fiber links is analyzed and fault prediction is performed, and a fault event report is output.

[0010] Based on the task monitor, the running status of distributed computing tasks is obtained, the mapping relationship between subtasks and computing nodes is established, the data dependency relationship is analyzed to construct a directed dependency graph, task status snapshots are generated, and a set of task status snapshots with timestamps is output.

[0011] Based on the directed dependency graph and the network topology graph, the resource manager analyzes resource requirements, reserves spare resources, identifies key intermediate results to formulate caching strategies, deploys a distributed caching system through the cache manager, and outputs a resource pre-allocation scheme and a cache deployment plan.

[0012] Based on the fault event report and the directed dependency graph, the migration scheduler analyzes the scope of the fault impact, extracts the affected computation subgraph, selects the migration target node based on the resource pre-allocation scheme, and outputs the migration execution plan.

[0013] Based on the migration execution plan, the computing environment is preloaded on the target node, the set of timestamped task status snapshots is obtained and the execution context is reconstructed, the data communication path is redirected, the status consistency verification is started, and the task recovery confirmation information is output.

[0014] For the task recovery confirmation information, a data change capture mechanism is set up to record incremental data, a state merging strategy is formulated, the incremental data is synchronized to the target node and consistency verification is performed, and a migration completion confirmation is output.

[0015] Preferably, the construction of the distributed task migration system, by deploying a fault detection agent, a task monitor, a migration scheduler, a resource manager, and a cache manager, outputs a system architecture, including:

[0016] Deploy the fault detection agent, the task monitor, the migration scheduler, the resource manager, and the cache manager to establish the basic component architecture;

[0017] Based on the aforementioned basic component architecture, a communication mechanism between components is constructed using a message bus and publish-subscribe pattern to enable collaborative work among components and output component communication protocols.

[0018] Based on the aforementioned component communication protocol, it interfaces with the SDN controller and optical network management system to achieve optical network control plane integration and output network control interface;

[0019] Based on the network control interface, a distributed computing framework adaptation module is designed to obtain the system architecture.

[0020] Preferably, the step of collecting network status indicators, constructing a network topology map, analyzing fiber optic link status and performing fault prediction based on the fault detection agent, and outputting a fault event report includes:

[0021] Based on the fault detection agent, network status indicators including optical power, bit error rate, latency, and jitter are collected in real time, and raw monitoring data is output.

[0022] Based on the original monitoring data, a real-time network topology map is dynamically constructed using the link layer discovery protocol, and the network topology map is output.

[0023] For the network topology, the health status of each optical fiber link is assessed using a multi-index threshold and health scoring algorithm, and the link health status assessment results are output.

[0024] Based on the link health status assessment results, combined with historical fault data and machine learning models, fault prediction analysis is performed, and fault warning information is output.

[0025] Based on the fault warning information and real-time monitoring data, when an optical fiber link interruption event is detected, key information including fault type, location, time, and switching path is recorded to obtain the fault event report.

[0026] Preferably, the step of acquiring the running status of distributed computing tasks based on the task monitor, establishing a mapping relationship between subtasks and computing nodes, analyzing data dependencies to construct a dependency graph, generating task status snapshots, and outputting a set of timestamped task status snapshots includes:

[0027] Based on the task monitor, the running status of distributed computing tasks, including execution progress, resource consumption, and data processing volume, is acquired in real time, and task running status data is output.

[0028] Based on the task running status data, analyze the internal structure of the task, divide the overall task into multiple sub-task units, establish the mapping relationship between sub-tasks and physical computing nodes, and output a sub-task distribution mapping table.

[0029] Based on the subtask distribution mapping table, by tracking the data flow and computational logic, the data dependencies between each subtask are analyzed, and a directed dependency graph is constructed.

[0030] For the directed dependency graph, a critical path analysis algorithm is applied to identify critical execution paths and performance bottlenecks, and the critical path analysis results are output.

[0031] Based on the critical path analysis results and the task running status data, a task status snapshot containing the computation context, intermediate results and memory status is generated, resulting in the set of timestamped task status snapshots.

[0032] Preferably, the step of analyzing resource requirements through the resource manager, reserving spare resources, identifying key intermediate results to formulate caching strategies, deploying a distributed caching system through the cache manager, and outputting a resource pre-allocation scheme and a cache deployment plan includes:

[0033] Based on the directed dependency graph and the network topology graph, analyze the bandwidth requirements and key data transmission paths between computing nodes, and output the network resource requirement analysis results.

[0034] Based on the network resource demand analysis results, the resource manager reserves spare optical paths and wavelength resources in the physical network and outputs a wavelength resource pre-allocation scheme.

[0035] Based on the directed dependency graph and the network topology graph, select suitable backup computing nodes and reserve computing resources, and output a computing resource reservation scheme;

[0036] Based on the directed dependency graph, identify key intermediate results and state data in the computation process, formulate a multi-level caching strategy, and output the intermediate result caching strategy.

[0037] Based on the wavelength resource pre-allocation scheme, the computing resource reservation scheme, and the intermediate result caching strategy, a distributed caching system is deployed on selected nodes through the cache manager to establish data version control and consistency protocols, thereby obtaining the resource pre-allocation scheme and the cache deployment plan.

[0038] Preferably, the step of analyzing the scope of the fault impact through the migration scheduler, extracting the affected computational subgraph, selecting the migration target node based on the resource pre-allocation scheme, and outputting the migration execution plan includes:

[0039] Based on the fault event report, combined with the network topology diagram and the subtask distribution mapping table, the migration scheduler quickly analyzes the computing nodes and subtasks affected by the faulty link and outputs the fault impact analysis results.

[0040] Based on the failure impact analysis results, the affected computational subgraphs are extracted from the directed dependency graph, including directly affected subtasks and indirectly dependent upstream and downstream tasks, and the computational subgraph model to be migrated is output.

[0041] Based on the computing subgraph model to be migrated and the computing resource reservation scheme, the adaptability, current load and network connection status of each backup node are evaluated, the optimal combination of migration target nodes is selected, and the migration target node allocation scheme is output.

[0042] Based on the criticality and dependency of each subtask in the computational subgraph model to be migrated, the migration operations are prioritized and a list of subtask migration priorities is output.

[0043] Based on the migration target node allocation scheme and the subtask migration priority list, combined with the optical network protection switching time characteristics and the calculation task fault tolerance time threshold, the available migration time window is evaluated, a time-sensitive migration schedule is formulated, and the migration execution plan is obtained.

[0044] Preferably, migration decision optimization is performed based on a matroid-constrained optimization algorithm, including:

[0045] Based on the failure impact analysis results and the directed dependency graph, the distributed computing migration problem is formalized as a matte-constrained submodulus maximization problem. A matte M=(N,I) is constructed, where N is the set of all possible subtask-target node pairs and I is the set of feasible migration schemes that satisfy resource constraints and dependencies. The matte-constrained problem model is then output.

[0046] Based on the aforementioned matroid constraint problem model, a fixed-parameter algorithm is designed to handle the offline computation of the migration planning stage and the dynamic response of the real-time migration execution stage, and outputs a fixed-parameter algorithm scheme.

[0047] Based on the fixed parameter algorithm scheme, a dynamic adjustment mechanism for the migration scheme is established. When the network status or computing node status is updated, the migration decision is adjusted in real time, and a dynamically optimized migration scheme is output.

[0048] Based on the aforementioned dynamic optimization migration scheme, a unified migration optimization algorithm is designed to handle different scenarios of independent and competitive tasks, and output a unified migration optimization strategy.

[0049] Based on the unified migration optimization strategy, the theoretical guarantee and performance evaluation of the migration scheme are realized, and the optimized migration execution plan is obtained.

[0050] Preferably, the fault-tolerant path optimization mechanism for fiber optic networks based on the Gomory-tree includes:

[0051] Based on the physical topology and wavelength resource allocation of the optical fiber network, a weighted undirected graph G=(V,E) is constructed, where vertex V represents a network node, edge E represents an optical fiber link, and the weight of the edge represents the bandwidth capacity or reliability index of the link. The network graph model is then output.

[0052] For the network graph model, calculate its Gomory-tree T, retain the minimum cut information between all node pairs in the graph, design an incremental update mechanism, and output the network capacity analysis model.

[0053] According to the network capacity analysis model, when it is necessary to find the maximum traffic path between any two computing nodes, the minimum weight edge on the path between nodes in the Gomory-tree is directly queried, the maximum available bandwidth between key node pairs in the distributed computing task is calculated, and the maximum traffic information between nodes is output.

[0054] Based on the maximum traffic information between the nodes and the bandwidth requirements of the computing task, an optimal backup path resource allocation strategy is formulated. For each pair of critical nodes, a set of wavelength resources that meets the corresponding minimum bandwidth requirements and minimizes total resource consumption is allocated, and an optimized backup path resource allocation scheme is output.

[0055] Based on the optimized backup path resource allocation scheme, for the multi-point communication mode in distributed computing, the network graph model is extended into a hypergraph, and the Gomory-Hu tree extension algorithm on the hypergraph is applied to optimize the path planning and resource allocation of group communication, thus obtaining the wavelength resource pre-allocation scheme.

[0056] Preferably, the adaptive migration optimization of distributed computing tasks based on random reuse includes:

[0057] Based on the characteristics of distributed computing tasks and network conditions, the migration optimization problem is decomposed into sub-problems of node selection, path planning, resource allocation and scheduling optimization. For each type of sub-problem, a corresponding stochastic optimization algorithm framework is designed, and a set of stochastic algorithm models is output.

[0058] Based on the set of random algorithm models, the concept of pseudo-independent algorithms is introduced to construct a randomness seed management system. This system ensures that randomness can be appropriately shared at different migration decision stages while maintaining independence. It includes randomness pool design, randomness mapping mechanism and pseudo-independence evaluation, and outputs a randomness management strategy.

[0059] Based on the randomness management strategy and the network state of the distributed computing task, an adaptive switching mechanism is designed to switch between full batch query and sample query. In normal state, sample query is used first, critical decision is switched to full batch query, and a hybrid strategy is used in emergency state. The adaptive query strategy is output.

[0060] Based on the adaptive query strategy, a stochastic reuse strategy across subproblems is implemented, including gradient reuse, sampling reuse and Markov model reuse. An iterative convergence acceleration mechanism is designed, and an optimized transfer decision scheme is output.

[0061] Based on the optimized migration decision scheme, a dynamic trade-off mechanism between accuracy and efficiency is realized. According to the severity of network failure and the available migration time window, the degree of random reuse, the full batch sampling ratio and the iteration termination condition are automatically adjusted to output an optimized migration execution plan.

[0062] Preferably, the steps of preloading the computing environment on the target node, obtaining the set of timestamped task state snapshots and reconstructing the execution context, redirecting the data communication path, initiating state consistency verification, and outputting task recovery confirmation information include:

[0063] Based on the migration execution plan and the migration target node allocation scheme, the computing environment, including libraries, configuration files and runtime dependencies, is preloaded on the selected target node, and the environment is output as ready.

[0064] Based on the environment ready state and the cache deployment plan, the most recent snapshot and key intermediate results in the timestamped task status snapshot set are obtained from the distributed cache system, loaded into the target node memory, and the data loading completed status is output.

[0065] Based on the data loading completion status, reconstruct the execution context of the computing task, including the memory data structure, iteration status, and computing progress information, and output the context reconstruction completion status.

[0066] Based on the context reconstruction completion status, modify the task scheduling information and data flow direction, redirect the subsequent calculation and data communication paths to the new node and network path, and output the process redirection completion status;

[0067] Based on the completed process redirection status, a verification program is initiated to check the consistency and correctness of the calculation status, confirm the integrity of task recovery, and obtain the task recovery confirmation information.

[0068] Preferably, the step of setting up a data change capture mechanism to record incremental data, formulating a state merging strategy, synchronizing the incremental data to the target node and performing consistency checks, and outputting a migration completion confirmation for the task recovery confirmation information includes:

[0069] For the task recovery confirmation information, a data change capture mechanism is set on the source node to record new data and status changes generated during the migration process and output an incremental data record set;

[0070] Based on the incremental data record set and the current computing state of the target node, analyze the data differences between the two, formulate a state merging strategy to resolve potential conflicts, and output a state merging scheme.

[0071] According to the state merging scheme, the incremental data captured by the source node is synchronized to the target node, the change operation is applied to update the calculation state, and the incremental synchronization completion status is output.

[0072] Based on the incremental synchronization completion status, perform data consistency verification to verify the data consistency between the source node and the target node, and output a consistency verification report.

[0073] Based on the consistency verification report, it is confirmed that the task has been successfully restored and reached a consistent state on the target node. Resource reclamation and cleanup of the source node are arranged, and the migration completion is confirmed.

[0074] This invention also provides a system for seamless migration of distributed computing tasks under fiber optic network interruption, comprising:

[0075] The system building module is used to build a distributed task migration system. It outputs the system architecture by deploying a fault detection agent, a task monitor, a migration scheduler, a resource manager, and a cache manager.

[0076] The fault detection module is used to collect network status indicators, construct a network topology map, analyze the status of optical fiber links and perform fault prediction based on the fault detection agent, and output a fault event report.

[0077] The task monitoring module is used to obtain the running status of distributed computing tasks based on the task monitor, establish the mapping relationship between subtasks and computing nodes, analyze data dependencies to construct a directed dependency graph, generate task status snapshots, and output a set of task status snapshots with timestamps.

[0078] The resource pre-allocation management module is used to analyze resource requirements based on the directed dependency graph and the network topology graph, reserve spare resources, identify key intermediate results to formulate caching strategies, deploy a distributed caching system through the caching manager, and output a resource pre-allocation scheme and a caching deployment plan.

[0079] The migration scheduling module is used to analyze the scope of the fault impact based on the fault event report and the directed dependency graph, extract the affected computation subgraph, select the migration target node based on the resource pre-allocation scheme, and output the migration execution plan.

[0080] The migration and reconstruction module is used to preload the computing environment on the target node based on the migration execution plan, obtain the set of timestamped task status snapshots and reconstruct the execution context, redirect the data communication path, start the status consistency verification, and output task recovery confirmation information.

[0081] The data synchronization module is used to set up a data change capture mechanism to record incremental data for the task recovery confirmation information, formulate a state merging strategy, synchronize the incremental data to the target node and perform consistency verification, and output a migration completion confirmation.

[0082] The beneficial effects of this invention are:

[0083] 1. By constructing a distributed task migration system architecture, deep collaboration between the distributed computing system and the optical network protection mechanism was achieved, solving the problem of independent fault handling at the system and network levels in existing technologies;

[0084] 2. Through fault detection and prediction mechanisms, potential faults in fiber optic links can be detected in advance, providing sufficient preparation time for task migration and improving the system's proactive defense capabilities;

[0085] 3. By monitoring task status and analyzing dependencies, the computational subgraphs affected by network failures were accurately identified, minimizing the migration scope and reducing system overhead;

[0086] 4. By pre-allocating resources and managing caches, necessary computing resources and intermediate results are prepared in advance on standby nodes, providing a basic guarantee for rapid migration;

[0087] 5. Through fault-triggered task migration decisions, migration plans can be formulated within millisecond-level time windows, adapting to the time characteristics of optical network protection switching;

[0088] 6. By restoring the computational state and seamlessly switching between states, continuous execution of distributed computing tasks is achieved, avoiding task restarts and waste of computing resources;

[0089] 7. Incremental synchronization and consistency guarantees ensure the integrity and consistency of data during the migration process, thereby improving the reliability of the calculation results. Attached Figure Description

[0090] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0091] Figure 1 This is a flowchart illustrating the method for seamless migration of distributed computing tasks under fiber optic network interruption in an embodiment of the present invention.

[0092] Figure 2 This is a schematic diagram of the distributed computing task seamless migration system under fiber optic network interruption in an embodiment of the present invention. Detailed Implementation

[0093] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0094] like Figure 1 As shown, this embodiment of the invention provides a method for seamless migration of distributed computing tasks under fiber optic network interruption, including the following steps:

[0095] S1: Build a distributed task migration system and output the system architecture by deploying a fault detection agent, task monitor, migration scheduler, resource manager, and cache manager.

[0096] This step first requires deploying five core functional components to build the infrastructure of the distributed task migration system. The Distributed Fault Detection Agent (DFDA) is a software component distributed across network nodes, responsible for real-time monitoring of fiber optic link status, collecting network performance metrics, and providing fault warnings. The Task State Monitor (TSM) is responsible for tracking the execution status of distributed computing tasks, obtaining task progress, resource usage, and data processing status. The Task MigrationScheduler (TMS) is the core decision-making unit of the system, responsible for analyzing the impact of faults and formulating migration plans. The Resource Pre-allocation Manager (RPM) is responsible for reserving computing and network resources in the network to prepare for potential task migrations. The State Cache Manager (SCM) is responsible for maintaining state snapshots and intermediate results of computing tasks, ensuring data recoverability.

[0097] After deploying these components, a communication mechanism between them is built using a message bus and a publish-subscribe pattern. A message bus is a distributed communication architecture that allows different components to communicate asynchronously by sending and receiving messages without direct coupling. The publish-subscribe pattern is a messaging paradigm where message senders (publishers) do not directly send messages to specific receivers (subscribers). Instead, they categorize messages into different types, and subscribers express interest in one or more types. This communication mechanism ensures that components can work together efficiently while maintaining a loosely coupled architecture, improving the system's scalability and fault tolerance.

[0098] Next, the system interfaces with the SDN controller and optical network management system to achieve optical network control plane integration. Software-defined networking (SDN) is a network architecture approach that separates network control functions from data forwarding functions, making network control directly programmable. By interfacing with the SDN controller, the system gains control over network topology, routing policies, and resource allocation. The optical network management system provides monitoring and management functions for optical network resources, including wavelength allocation, link status monitoring, and protection switching control. Through this integration, the system can obtain real-time network status information and adjust network resource allocation as needed.

[0099] Finally, a distributed computing framework adaptation module was designed to ensure compatibility with mainstream distributed computing frameworks (such as Spark, TensorFlow, and GraphX). This module provides standardized interfaces and plugins for acquiring task status information and controlling task execution across different computing frameworks. Through this adaptation, the system can support multiple computing frameworks, enhancing its versatility and practical value. The entire system architecture adopts a hierarchical design, including a data acquisition layer, an analysis and decision-making layer, and an execution control layer, ensuring clear responsibilities and efficient collaboration among all system components.

[0100] S2: Based on the fault detection agent, collect network status indicators, construct network topology maps, analyze fiber optic link status and perform fault prediction, and output fault event reports.

[0101] In this step, fault detection agents deployed at key network nodes collect multi-dimensional network status metrics in real time. These metrics include optical power, bit error rate (BER), latency, and jitter. Optical power is a measurement of optical signal strength, typically measured in dBm; abnormal changes in BER may indicate fiber optic connection problems or laser malfunctions. BER refers to the proportion of erroneous bits transmitted out of the total transmitted bits; a high BER usually indicates degraded signal quality. Latency is the transmission time of a data packet from source to destination, while jitter is a statistical measure of latency variation; both are important indicators of network performance. These metrics are collected through an Optical Performance Monitor (OPM), network traffic analyzer, and link quality detector, forming a raw monitoring data stream.

[0102] Based on the collected raw monitoring data, a real-time network topology map is constructed using the Link Layer Discovery Protocol (LLDP). LLDP is a vendor-neutral Layer 2 protocol that allows network devices to advertise their identity, capabilities, and neighbor information to other devices in the network. By analyzing LLDP packets, the connectivity relationships between network devices can be dynamically discovered, including physical and logical connections, thereby constructing a complete network topology map. This dynamic construction method enables the system to track network topology changes in real time, including the addition of new devices, removal of devices, or changes in link status.

[0103] After constructing the network topology map, the health status of each fiber optic link is assessed using a multi-index thresholding and health scoring algorithm. The multi-index thresholding method sets normal ranges and warning thresholds for each network status index, triggering an alarm when an index exceeds a threshold. The health scoring algorithm comprehensively considers multiple indicators to calculate a comprehensive health score for each link, typically using a weighted average or machine learning model. This assessment method not only considers anomalies in single indicators but also identifies potential problems reflected by combinations of multiple indicators, improving the accuracy and comprehensiveness of fault detection.

[0104] Based on the link health status assessment results, fault prediction analysis is performed by combining historical fault data and machine learning models. The machine learning models used here may include decision trees, random forests, or neural networks, which predict potential future faults by learning historical fault patterns. The predictive analysis considers not only the current link status but also historical fault patterns, seasonal factors, and the impact of related links, thereby identifying potential problems in advance and generating fault warning information.

[0105] When a fiber optic link outage is detected, detailed fault information is immediately recorded, including the fault type (e.g., fiber breakage, signal attenuation, wavelength conflict), location (affected network segment), time (exact timestamp of the fault), and failover path (alternate path activated by the optical network protection mechanism). This information is integrated into a fault event report, serving as crucial input for subsequent task migration decisions. The timeliness and accuracy of fault event reports are essential for the system's rapid response to network failures.

[0106] S3: Based on the task monitor, obtain the running status of distributed computing tasks, establish the mapping relationship between subtasks and computing nodes, analyze data dependencies to construct a directed dependency graph, generate task status snapshots, and output a set of task status snapshots with timestamps.

[0107] In this step, the task monitor acquires real-time runtime status information of distributed computing tasks. Distributed computing tasks refer to large-scale computing jobs executed in parallel on multiple computing nodes, such as data analysis, machine learning training, or scientific simulations. Task runtime status information includes metrics such as execution progress (percentage of completion or stage indicator), resource usage (CPU, memory, and storage utilization), and data processing volume (amount of data processed, throughput). This information is acquired through status collection agents deployed on each computing node and centrally integrated and analyzed by the task monitor. The status collection agent is a lightweight software component that can monitor local computing tasks with low overhead and transmit information to the monitoring center through a secure channel.

[0108] After obtaining the task's running status, its internal structure is analyzed, and the overall task is broken down into multiple subtask units. A subtask is the basic execution unit of a distributed computing task, typically representing a processing stage or data processing operation in the computation flow. Task decomposition is based on the characteristics of the computing framework (such as the map and reduce stages of MapReduce, or RDD transformation operations in Spark) or the execution stages defined by the application. After decomposition, a mapping relationship is established between subtasks and physical computing nodes, recording which node each subtask executes on, forming a subtask distribution mapping table. This mapping relationship is crucial for understanding task distribution and the scope of impact from network failures.

[0109] Based on a subtask distribution mapping table, the system analyzes the data dependencies between subtasks by tracking data flow and computational logic. Data dependencies describe the input-output relationships between subtasks, indicating that the execution of one subtask depends on the output of another. By analyzing these dependencies, the system constructs a directed dependency graph, where nodes represent subtasks and directed edges represent data flow or execution dependencies. A directed dependency graph is a data structure mathematically represented as G = (V, E), where V is the set of subtasks and E is the set of dependencies. Each edge (u, v) ∈ E indicates that subtask v depends on the output of subtask u.

[0110] For the constructed directed dependency graph, critical path analysis (CPath Analysis) algorithms are applied to identify critical execution paths and performance bottlenecks in the computational task. The critical path is the longest path from the start node to the end node in the dependency graph, representing the shortest time required to complete the entire task. CPath Analysis typically uses dynamic programming algorithms to calculate the earliest start time and latest finish time of each subtask, thereby determining which subtasks are located on the critical path. Subtasks located on the critical path directly impact the overall task completion time and therefore have higher priority in migration decisions.

[0111] Based on critical path analysis results and task execution status data, a task state snapshot is generated, containing computation context, intermediate results, and memory state. The computation context includes execution environment information such as the program counter, call stack, and local variables; intermediate results refer to temporary data generated during computation; and memory state includes heap memory and the current values ​​of global variables. This information is serialized into a task state snapshot and timestamped to indicate the exact time the snapshot was generated. Snapshots are generated periodically, forming a set of timestamped task state snapshots, providing a foundation for subsequent task recovery. Snapshot generation uses an incremental mechanism, storing only the changes relative to the previous snapshot to reduce storage overhead and generation time.

[0112] S4: Based on the directed dependency graph and network topology graph, analyze resource requirements through the resource manager, reserve spare resources, identify key intermediate results to formulate caching strategies, deploy a distributed caching system through the cache manager, and output resource pre-allocation scheme and cache deployment plan.

[0113] In this step, the bandwidth requirements and critical data transmission paths between computing nodes are analyzed based on the directed dependency graph and network topology. Bandwidth requirement analysis is based on the data exchange volume and frequency between subtasks, combined with the execution mode of the computing tasks (e.g., batch processing, stream processing, or interactive computing). Critical data transmission paths refer to network paths with high data traffic or significant impact on task completion time during task execution. Through this analysis, the dependence of computing tasks on network resources and critical transmission requirements can be identified, outputting the network resource requirement analysis results.

[0114] Based on network resource demand analysis, the resource manager reserves spare optical paths and wavelength resources in the physical network. An optical path is an end-to-end optical connection in an optical network, typically containing one or more physical links and allocated specific wavelengths. Wavelength resources refer to the available wavelength channels in optical fiber communication; in dense wavelength division multiplexing (DWDM) systems, a single fiber can support tens or even hundreds of wavelength channels. The resource manager employs the Constrained-Based Routing and Wavelength Assignment (CRWA) algorithm to reserve spare optical paths and wavelength resources for computational tasks, considering both physical layer constraints (such as transmission distance and signal quality) and network layer constraints (such as link capacity and allocated wavelengths). The reservation strategy considers various failure scenarios, ensuring immediate switching to pre-configured spare resources in the event of a primary path failure, and outputs a wavelength resource pre-allocation scheme.

[0115] Simultaneously, based on the directed dependency graph and network topology, suitable backup computing nodes are selected and computing resources are reserved. The selection of backup computing nodes considers multiple factors: computing capacity matching (similar CPU, memory, and storage specifications to the original node), network connectivity (network distance and bandwidth to other relevant nodes), current load (avoiding nodes already under high load), and fault domain isolation (selecting nodes located in different fault domains to improve fault tolerance). Resource reservation employs resource pooling and virtualization technologies, allowing for flexible allocation and adjustment of resources across multiple physical nodes, outputting a computing resource reservation scheme.

[0116] Based on the directed dependency graph, key intermediate results and state data in the computation process are identified, and a multi-level caching strategy is formulated. Key intermediate results refer to important temporary data generated during the computation process; their loss would lead to extensive rework. State data includes the current state of the iterative algorithm, the execution status of distributed transactions, and other key execution contexts. The multi-level caching strategy categorizes data into different priorities, employing different caching frequencies, storage locations, and replica numbers for each priority level. High-priority data may utilize real-time caching and multi-replica strategies, while low-priority data employs periodic caching and single-replica strategies. This differentiated caching strategy optimizes system resource utilization while ensuring data security, and it also includes an output intermediate result caching strategy.

[0117] Based on wavelength resource pre-allocation, computing resource reservation, and intermediate result caching strategies, the cache manager deploys a distributed caching system on selected nodes. A distributed caching system is a multi-node in-memory data storage system that provides high-speed data access and cross-node data sharing. A version control mechanism for cached data is established to ensure data consistency in a multi-node environment. Version control typically employs vector clocks or version vector techniques to track causal relationships and concurrent operations in the distributed environment. Simultaneously, the system implements consistency protocols, such as eventual consistency or linear consistency, selecting an appropriate consistency level based on application requirements to balance performance and correctness. Through these mechanisms, the system forms a complete resource pre-allocation scheme and cache deployment plan, providing a fundamental guarantee for task migration.

[0118] S5: Based on the fault event report and the directed dependency graph, analyze the scope of the fault impact through the migration scheduler, extract the affected computation subgraph, select the migration target node based on the resource pre-allocation scheme, and output the migration execution plan.

[0119] In this step, upon receiving a fault event report, the migration scheduler first combines the network topology map and subtask distribution mapping table to quickly analyze the compute nodes and subtasks affected by the faulty link. The fault impact analysis employs a graph traversal algorithm, starting from the faulty link and identifying all compute nodes that rely on that link for communication. For each affected compute node, the subtask distribution mapping table is queried to determine all subtasks executed on that node. Simultaneously, the severity of the fault and the estimated recovery time are assessed, categorizing faults into temporary outages (optical network protection switching can recover in milliseconds) and persistent outages (requiring physical repair and having a long recovery time). Different migration strategies are adopted for different types of faults. This analysis process outputs detailed fault impact analysis results, including a list of affected nodes, a list of affected subtasks, and a description of the fault characteristics.

[0120] Based on the failure impact analysis results, affected computational subgraphs are extracted from the directed dependency graph. The computational subgraph consists of two parts: directly affected subtasks (subtasks that execute on the failed node or rely on the failed link for communication) and indirectly dependent upstream and downstream tasks (tasks that have data dependencies on the directly affected subtasks). Subgraph extraction employs a dependency propagation algorithm. First, all directly affected subtasks are marked. Then, the impact is recursively propagated upstream (tasks providing input data) and downstream (tasks using output data) until the impact boundary is reached. The impact boundary may be a natural boundary (such as the start or end of a computation phase) or a manually set cutoff point (based on the degree of impact or resource constraints). The extracted computational subgraph constitutes the model of the computational subgraph to be migrated and is the core object of migration decisions.

[0121] Based on the computational subgraph model to be migrated and the reserved computational resources, the system evaluates the suitability, current load, and network connectivity of each standby node to select the optimal combination of target nodes for migration. Suitability evaluation considers the compatibility between the hardware specifications (CPU architecture, memory capacity, storage type) of the standby nodes and the original nodes, as well as the compatibility of the software environment (operating system version, library dependencies, runtime environment). Current load evaluation monitors the resource utilization (CPU utilization, memory usage, I / O latency, etc.) of the standby nodes in real time to avoid selecting nodes that are already under high load. Network connectivity evaluation considers the network quality between the standby nodes and other relevant nodes (especially nodes with data dependencies), including bandwidth, latency, and reliability. By comprehensively considering these factors, the system employs multi-objective optimization algorithms, such as weighted sum models or analytic hierarchy process (AHP), to select the optimal combination of target nodes for migration and output a target node allocation scheme.

[0122] Based on the criticality and dependencies of each subtask in the computational subgraph model to be migrated, the system prioritizes migration operations. Criticality assessment is based on several factors: the subtask's position on the critical path (tasks on the critical path have higher priority), the subtask's computational complexity (tasks with higher complexity may require longer migration times and are therefore prioritized), the subtask's data dependency (tasks with many dependencies require coordination of multiple related tasks; prioritizing them simplifies subsequent migrations), and the subtask's restart cost (some tasks have high restart costs, such as long-running tasks that have completed most of their computation). A topology sorting algorithm is used to ensure that dependent tasks are migrated before the tasks they depend on, while also considering the above priority factors, outputting a subtask migration priority list.

[0123] Finally, based on the target node allocation scheme and the subtask migration priority list, the system, combining the optical network protection switching time characteristics and the fault tolerance time threshold of the computation task, assesses the available migration time window and formulates a time-sensitive migration scheduling plan. Optical network protection switching can typically be completed within 20-50 milliseconds, while the fault tolerance time threshold of the computation task (i.e., the communication interruption time the task can tolerate) varies depending on the task type, ranging from hundreds of milliseconds to several seconds. Based on these two time parameters, the system determines the time window available for migration decision-making and execution, and employs real-time scheduling algorithms, such as earliest deadline priority or rate monotonic scheduling, to formulate a detailed migration execution schedule. This schedule considers the migration priority, dependencies, and execution time estimates of each subtask, ensuring that the migration of critical tasks is completed within the limited time window, and outputting a complete migration execution plan.

[0124] S6: Based on the migration execution plan, preload the computing environment on the target node, obtain a set of timestamped task state snapshots and reconstruct the execution context, redirect the data communication path, start state consistency verification, and output task recovery confirmation information.

[0125] In this step, the computing environment is preloaded on the selected target nodes according to the migration execution plan and target node allocation scheme. The computing environment refers to the set of software and configurations required for task execution, including libraries, configuration files, and runtime dependencies. Libraries are collections of external code that task execution depends on, such as mathematical computing libraries, machine learning frameworks, or data processing toolkits. Configuration files contain parameter settings and environment variables for task execution, such as data source addresses, algorithm parameters, or resource limits. Runtime dependencies are components required by the execution environment, such as specific versions of the Java Virtual Machine, Python interpreter, or container runtime. The preloading process uses environment replication techniques, such as container image replication, configuration synchronization, or package manager installation, to ensure that the target environment is consistent with the source environment. The system performs basic verification of the preloaded environment, confirming that key components are installed correctly and can start normally, and outputs an environment ready status.

[0126] Based on the environment readiness status and cache deployment plan, the system retrieves the most recent snapshot and key intermediate results from the timestamped task state snapshot set in the distributed caching system. Task state snapshot selection is based on the principle of closest time, choosing the valid snapshot closest to the current time to minimize state rollback. For incremental snapshots, the system needs to load the base snapshot and all subsequent incremental data to reconstruct the complete state. The retrieval of key intermediate results is based on data dependencies in the directed dependency graph, ensuring that all necessary input data is loaded correctly. The data loading process employs parallelization, retrieving data from multiple cache nodes simultaneously and utilizing the principle of data locality, prioritizing data retrieval from the node closest in the network topology to improve loading efficiency. After loading is complete, the system verifies data integrity and consistency, detects data corruption or loss, and outputs a data loading completion status.

[0127] Based on the data loading completion status, the execution context of the computation task is reconstructed. The execution context is a complete representation of the task's execution state, including in-memory data structures, iteration states, and computation progress information. In-memory data structures refer to the data organization format maintained by the task in memory, such as hash tables, graph structures, or matrices. Iteration states are used for iterative algorithms, recording the current iteration round, convergence parameters, and intermediate variable values. Computation progress information tracks metrics such as the percentage of task completion, the amount of data processed, and the estimated remaining time. The reconstruction process first loads the serialized data from the state snapshot, then executes the state recovery process, including object deserialization, memory layout reconstruction, and state variable initialization. For specific computation frameworks, it may be necessary to call the framework's provided recovery interface or implement custom recovery logic. After reconstruction, the validity of the context is verified, ensuring that key state variables meet the expected value range, and the context reconstruction completion status is output.

[0128] Based on the context reconstruction completion status, the task scheduling information and data flow are modified, redirecting subsequent computation and data communication paths to new nodes and network paths. Task scheduling information modification includes updating the distributed scheduler's task allocation table, adjusting task dependencies, and updating resource allocation strategies. Data flow redirection involves modifying data source addresses, updating communication endpoint configurations, and adjusting network routing strategies. For message-based systems, this may require updating publish-subscribe relationships or message queue configurations; for RPC-based systems, it requires updating service discovery mechanisms and load balancing configurations. The redirection process must consider atomicity and consistency, ensuring all relevant components update their configurations synchronously to avoid inconsistent states caused by partial updates. After redirection, the system verifies the connectivity and performance of the new path to ensure normal data flow and outputs the process redirection completion status.

[0129] Based on the completion status of the process redirection, a verification program is initiated to check the consistency and correctness of the computational state. The verification program is a set of tools and algorithms specifically designed to detect state anomalies, capable of identifying potential data inconsistencies or execution errors. Verification includes data integrity checks (such as checksum verification and referential integrity checks), state consistency checks (such as transaction consistency and distributed consistency verification), and computational logic verification (such as intermediate result sampling checks and computational process reproduction). Verification methods may include techniques such as deterministic replay, state comparison, or invariant checks. If an anomaly is detected during verification, a repair process is triggered, which may include reloading data, adjusting state variables, or, in extreme cases, rolling back to the previous known correct state. Upon successful verification, the system confirms the completeness of the task recovery, generates a detailed report containing a summary of the recovered state, verification results, and performance metrics, and outputs task recovery confirmation information.

[0130] S7: For task recovery confirmation information, set up a data change capture mechanism to record incremental data, formulate a state merging strategy, synchronize incremental data to the target node and perform consistency verification, and output a migration completion confirmation.

[0131] In this step, the first step is to set up a data change capture mechanism on the source node based on the task recovery confirmation information, recording new data and state changes generated during the migration process. Change Data Capture (CDC) is a technique for identifying and tracking data changes, widely used in data synchronization and replication scenarios. In this system, the CDC mechanism may be implemented through log mining, triggers, or application-layer change tracking. Log mining directly extracts change information from system logs or transaction logs, offering low invasiveness and high efficiency. Triggers automatically trigger recording operations during data operations, capturing precise change information, but may increase system overhead. Application-layer change tracking explicitly records changes in the application code, providing maximum flexibility, but requiring modifications to application logic. Regardless of the method used, complete change information will be recorded, including change type (insert, update, or delete), change time, changed data, and its context, forming an incremental data record set.

[0132] Based on the incremental data record set and the current computational state of the target node, the data differences between the two are analyzed, and a state merging strategy is formulated to resolve potential conflicts. Data difference analysis employs a semantic comparison method, considering not only the surface values ​​of the data but also its business meaning and contextual relationships. Conflict detection focuses on concurrent modification scenarios, i.e., situations where the source node and target node simultaneously modify the same data item. State merging strategies may include various conflict resolution mechanisms: timestamp priority (selecting the latest change), source node priority (source node changes overwrite target node changes), target node priority (retaining target node changes), or a custom merging function (merging conflicting changes according to specific rules). A three-way merging technique may also be implemented, resolving conflicts more accurately by comparing the source state, target state, and common ancestor state. The selection of the merging strategy considers the characteristics of the computational task, data consistency requirements, and business rules, outputting a detailed state merging scheme.

[0133] According to the state merging scheme, incremental data captured by the source node is synchronized to the target node, and the computational state is updated by applying change operations. Data synchronization adopts either batch processing or stream processing, selecting the appropriate transmission mode based on data volume and real-time requirements. Batch processing is suitable for efficient transmission of large amounts of data, while stream processing provides lower latency and real-time performance. The change application process follows transactional principles, ensuring atomicity, consistency, isolation, and durability (ACID) to prevent inconsistent states caused by partial applications. For complex change sets, a topology sorting algorithm may be used to determine the application order, ensuring that dependencies are satisfied. During the application change process, the execution status is continuously monitored, potential errors are captured, and rollback mechanisms are provided to ensure data security. After synchronization is complete, the system updates metadata records, including synchronization timestamps, change summaries, and status indicators, and outputs the incremental synchronization completion status.

[0134] Based on the completion status of incremental synchronization, data consistency verification is performed to verify the data consistency between the source and target nodes. Consistency verification employs a multi-layered verification method: the basic layer verification focuses on the existence and integrity of data, such as matching the number of records and consistent values ​​of key fields; the structural layer verification checks the consistency of the data structure, including index structure, referencing relationships, and constraints; and the semantic layer verification focuses on business rules and computational logic, such as the correctness of aggregate values, compliance with business rules, and consistency of computational results. Verification techniques may include hash comparison (calculating and comparing the hash values ​​of key data), sampling inspection (randomly selecting data points for deep comparison), or full scan (performing a complete comparison on the key dataset). If inconsistencies are found, detailed difference information is recorded, and a decision on whether resynchronization or manual intervention is needed is made based on the nature of the differences. The verification results are compiled into a consistency verification report, including the verification scope, methods, and a summary of results.

[0135] Based on the consistency verification report, it is confirmed that the task has been successfully restored and reached a consistent state on the target node. Resource reclamation and cleanup of the source node are then scheduled. The confirmation process includes final state verification, performance indicator comparison, and stability monitoring. Final state verification checks whether the computation task has reached the expected execution stage and data state; performance indicator comparison analyzes changes in execution efficiency before and after the migration, such as response time, throughput, or resource utilization; stability monitoring tracks task execution over a period of time to ensure no latency issues occur. After confirming successful task recovery, the system schedules source node resource reclamation, including releasing computing resources (CPU, memory, storage), cleaning up temporary data, and resetting the execution environment. Resource reclamation adopts a gradual strategy, initially retaining critical states for a period (as a safety net), and then completely cleaning up only after confirming stable operation of the target node. Detailed logs of the entire migration process are recorded, including key events, decision points, and performance data, for subsequent analysis and improvement reference. Finally, a migration completion confirmation is generated, marking the successful completion of the entire seamless migration process.

[0136] Through the above steps, this invention achieves seamless migration of distributed computing tasks under fiber optic network interruption, ensuring continuous execution and data consistency of computing tasks in the event of network failure, and significantly improving system reliability and service continuity. This method fully utilizes the rapid recovery characteristics of optical network protection switching, combined with distributed computing task state maintenance and migration techniques, to achieve collaborative optimization between the network layer and application layer, representing a significant advancement in distributed system fault tolerance technology.

[0137] In one embodiment, the construction of the distributed task migration system, by deploying a fault detection agent, a task monitor, a migration scheduler, a resource manager, and a cache manager, and outputting a system architecture, includes: deploying the fault detection agent, the task monitor, the migration scheduler, the resource manager, and the cache manager to establish a basic component architecture; based on the basic component architecture, constructing an inter-component communication mechanism using a message bus and publish-subscribe pattern to achieve collaborative work among the components and outputting a component communication protocol; based on the component communication protocol, interfacing with an SDN controller and an optical network management system to achieve optical network control plane integration and outputting a network control interface; and based on the network control interface, designing a distributed computing framework adaptation module to obtain the system architecture.

[0138] In this embodiment, the basic component architecture is first established by deploying a fault detection agent, task monitor, migration scheduler, resource manager, and cache manager. The Distributed Fault Detection Agent (DFDA) is a software component deployed at key nodes in the network, responsible for real-time monitoring of fiber optic link status and network performance parameters. Each DFDA contains three core modules: a data acquisition module (collecting raw data such as optical power and bit error rate), a local analysis module (performing preliminary data processing and anomaly detection), and a communication module (reporting the processing results to the central system). The DFDA employs a lightweight design, featuring low resource consumption and high reliability, enabling long-term stable operation on network devices. The Task State Monitor (TSM) tracks the execution status of distributed computing tasks, including task topology, execution progress, and resource usage. It interfaces with various distributed computing frameworks through standardized interfaces to obtain task execution data and construct task state models. The TSM supports multi-level monitoring granularity, from the overall task to specific subtasks, achieving comprehensive and fine-grained state tracking. The Task Migration Scheduler (TMS) is the core decision-making component of the system, responsible for formulating the optimal migration strategy in the event of a failure. It implements various scheduling algorithms, including heuristics, reinforcement learning models, and constraint solving methods, adaptable to migration scenarios of varying complexity and urgency. The Resource Pre-allocation Manager (RPM) manages and reserves system resources, including computing resources (CPU, memory, storage) and network resources (bandwidth, links, wavelength). It maintains a global resource view, optimizes resource allocation, and arbitrates conflicts, ensuring sufficient spare resources are available in failure scenarios. The State Cache Manager (SCM) is responsible for the persistence and recovery of task states, enabling the deployment and management of the distributed caching system. It supports various caching strategies, such as time-driven caching, event-driven caching, and hybrid strategies, and implements version control and consistency guarantees for cached data.

[0139] Secondly, based on the basic component architecture, a message bus and publish-subscribe pattern are used to build an inter-component communication mechanism, enabling collaborative work among components and outputting component communication protocols. The message bus is a distributed communication architecture that provides asynchronous message delivery and event notification services, allowing system components to communicate without direct coupling. In this system, the message bus achieves high throughput and low latency, supporting message prioritization, flow control, and reliable delivery guarantees. The core services of the message bus include message routing (accurately delivering messages to target components), message transformation (handling conversions between different formats), and message persistence (ensuring critical messages are not lost). The publish-subscribe pattern is a messaging paradigm that decouples message senders (publishers) from receivers (subscribers). Publishers send messages to specific topics, while subscribers subscribe to topics of interest to receive messages. This pattern supports one-to-many communication (one publisher and multiple subscribers) and content filtering (subscribers only receive messages that meet specific conditions), improving the system's flexibility and scalability. The component communication protocol defines message formats, interaction modes, and interface specifications, including control messages (such as configuration updates and status queries), data messages (such as monitoring data and task status), and event messages (such as fault alarms and threshold breaches). The protocol adopts a layered design: the bottom layer provides basic communication services, the middle layer implements message routing and transformation, and the top layer defines business interfaces and semantics. To improve interoperability, the protocol is implemented based on open standards such as AMQP (Advanced Message Queuing Protocol) or MQTT (Message Queuing Telemetry Transport) and extends it with domain-specific message types and interaction modes.

[0140] Third, based on the component communication protocol, it interfaces with the SDN controller and optical network management system to achieve optical network control plane integration and output network control interfaces. Software-defined networking is a network architecture approach that separates network control functions from data forwarding functions, achieving centralized network control and programmability. The SDN controller is the core component of the SDN architecture, responsible for maintaining the global network view, calculating network paths, and issuing control commands. It interfaces with the SDN controller through a standardized northbound interface, which is implemented based on REST API or gRPC, supporting network topology queries, path calculation requests, and resource reservation operations. The optical network management system is an integrated platform specifically for managing optical transmission networks, providing configuration, monitoring, and optimization functions for optical network resources. The interface with the optical network management system uses TL1 (Transaction Language 1) commands, SNMP (Simple Network Management Protocol), or proprietary APIs to achieve optical path status queries, wavelength resource management, and protection switching control. The network control plane integration implements three key functions: network awareness (real-time acquisition of network topology, link status, and resource information), resource reservation (pre-allocation of wavelength and bandwidth resources on backup paths), and dynamic adjustment (adjusting routing strategies and resource allocation according to changes in network status). The network control interface abstracts the underlying network details and provides a unified service model, including connectivity services (establishing end-to-end optical paths), bandwidth services (allocating and adjusting bandwidth resources), and protection services (configuring and triggering protection switching). The interface design follows the Service-Oriented Architecture (SOA) principle, supporting service discovery, service composition, and quality of service assurance.

[0141] Finally, based on the network control interface, a distributed computing framework adapter module was designed, resulting in a complete system architecture. Distributed computing frameworks are software platforms that support large-scale parallel computing, such as Apache Hadoop, Spark, and TensorFlow. Each framework has different programming models, scheduling mechanisms, and state management methods. The adapter module achieves seamless integration with multiple mainstream distributed computing frameworks, providing dedicated connectors for each framework. Connectors include a state collector (acquiring the execution status of computing tasks), a task controller (managing the pausing, resumption, and migration of tasks), and a resource coordinator (coordinating resource allocation between the framework and the system). The adapter module adopts a plug-in architecture, allowing dynamic loading and updating of framework adapters, supporting the expansion and upgrading of system functions. Key technologies include state abstraction (converting specific state representations of different frameworks into a unified model), event mapping (mapping framework events to system events), and instruction translation (converting system instructions into framework-specific operations). The adapter module also implements lifecycle management of computing tasks, including task registration (registering computing tasks requiring protection with the system), state tracking (monitoring key state points during task execution), and recovery control (coordinating the task recovery process after a failure). Through these functions, the adaptation module enables the system to support multiple computing frameworks, improving the system's versatility and practical value. The final system architecture is a hierarchical, modular, and scalable design, including an infrastructure layer (physical resources and virtualization platform), a control layer (core components and coordination mechanisms), an adaptation layer (interfaces with external systems), and an application layer (user interface and management tools). Each layer interacts through clearly defined interfaces, ensuring the system's flexibility and maintainability.

[0142] Through the four detailed steps outlined above, a complete distributed task migration architecture was constructed, implementing core functions such as fault detection, task monitoring, resource management, migration scheduling, and state caching. The system architecture employs a loosely coupled design, with components communicating through standardized interfaces and messaging mechanisms, ensuring system scalability and flexibility. Furthermore, deep integration with the network control plane and distributed computing framework enables seamless migration of computing tasks even in the event of fiber optic network failures, guaranteeing business continuity and data integrity. This architectural design is not only suitable for current scenarios but also possesses strong evolutionary capabilities, adapting to future developments in network and computing technologies.

[0143] In one embodiment, the step of collecting network status indicators, constructing a network topology map, analyzing fiber optic link status and performing fault prediction based on the fault detection agent, and outputting a fault event report includes: collecting network status indicators including optical power, bit error rate, latency, and jitter in real time based on the fault detection agent, and outputting raw monitoring data; dynamically constructing a real-time network topology map using a link layer discovery protocol based on the raw monitoring data, and outputting the network topology map; assessing the health status of each fiber optic link using a multi-indicator threshold and health scoring algorithm for the network topology map, and outputting the link health status assessment result; performing fault prediction analysis based on the link health status assessment result, combined with historical fault data and a machine learning model, and outputting fault warning information; and recording key information including fault type, location, time, and switching path when a fiber optic link interruption event is detected based on the fault warning information and real-time monitoring data, and obtaining the fault event report.

[0144] In this embodiment, firstly, based on a fault detection agent, network status indicators including optical power, bit error rate, latency, and jitter are collected in real time, and raw monitoring data is output. The Distributed Fault Detection Agent (DFDA) is a distributed monitoring component deployed on optical network nodes, responsible for collecting, processing, and reporting network performance data. Each agent contains multiple sensing modules for collecting different types of network indicators. Optical power is a key indicator for measuring optical signal strength, typically measured in dBm (decibels per milliwatt), and is collected by an optical power monitor. Optical power monitoring is based on the photoelectric detection principle, converting optical signals into electrical signals for measurement. Its typical monitoring range is -30dBm to +10dBm, with an accuracy of ±0.5dB. Abnormal changes in optical power (such as sudden drops or fluctuations) may indicate fiber optic connection problems, laser failures, or optical path interruptions. Bit error rate (BER) is an indicator for measuring the quality of digital communication, defined as the ratio of incorrectly received bits to the total number of transmitted bits, and is collected by a BER tester. Bit error rate (BER) testing is based on known test sequences or forward error correction (FEC) coding statistics. In normal fiber optic links, the BER typically remains around 10%. -12The following indicates a significant increase in bit error rate, signifying a deterioration in signal quality. Latency is the transmission time of data from source to destination, typically measured in milliseconds (ms), and is collected by a latency measurement probe. Latency measurement employs active probing methods, such as sending ICMP Echo requests or dedicated test frames, to calculate the round-trip time. In fiber optic networks, latency primarily consists of propagation delay (the time required for light to travel through the medium), processing delay (the time required for devices to process data), and queuing delay (the time required for data to wait in the buffer). Jitter is a statistical measure of latency variation, representing the degree of latency difference between consecutive data packets, and is collected by a jitter analyzer. Jitter calculation is based on the variance or standard deviation of consecutive latency samples; high jitter values ​​indicate network instability or congestion. This raw data collection utilizes a combination of technologies, including built-in network device monitoring interfaces (such as SNMP MIB), optical layer monitoring equipment (such as OPM), and dedicated testing equipment, forming a comprehensive network status monitoring system. The data acquisition process considers sampling frequency optimization. Key indicators (such as optical power) are sampled at high frequency (e.g., multiple times per second), while stable indicators (such as link capacity) are sampled at low frequency (e.g., once per minute), balancing monitoring accuracy and system load.

[0145] Secondly, based on the raw monitoring data, a real-time network topology map is dynamically constructed using the Link Layer Discovery Protocol (LLDP), and the network topology map is output. LLDP is a vendor-neutral Layer 2 protocol that enables network devices to broadcast their identity, capabilities, and configuration information to neighboring devices and receive similar information from neighboring devices. LLDP operates based on the periodic broadcasting of LLDP Data Units (LLDPDUs), each containing multiple Type-Length-Value (TLV) fields that transmit information such as device ID, port ID, and system capabilities. Devices collect and maintain LLDP neighbor information through local LLDP agents, constructing adjacency tables. In addition to standard LLDP, the system also integrates optical network-specific extension protocols, such as the OTN (Optical Transport Network) discovery protocol and the ROADM (Reconfigurable Optical Add-Drop Multiplexer) configuration information exchange protocol, to obtain optical layer information such as wavelength allocation, optical amplifier parameters, and optical switching status. The network topology construction process comprises four stages: adjacency collection (collecting LLDP neighbor information from various network devices), topology aggregation (integrating scattered adjacency relationships into a global view), physical / logical association (establishing a mapping relationship between physical connections and logical channels), and dynamic updates (processing topology change events in real time). The constructed network topology graph contains multiple layers of information: physical layer topology (fiber optic connections, optical amplifier locations, optical cross-connections), transport layer topology (OTN channels, wavelength channels, sub-wavelength channels), and logical layer topology (IP links, virtual private networks). The topology graph is represented using a graph data structure, where nodes represent network devices (such as optical switches, routers, or wavelength division multiplexers), edges represent connections (such as fiber optic links or logical channels), and attributes are attached to nodes and edges to record detailed characteristics of the devices and links. To support efficient querying and analysis, the system implements a topology indexing mechanism, such as a neighborhood index (for quickly obtaining node adjacency relationships), a path index (for quickly obtaining paths between nodes), and an attribute index (for quickly filtering elements based on specific attributes).

[0146] Third, based on the network topology map, the health status of each fiber optic link is assessed using a multi-index threshold and health scoring algorithm, outputting the link health status assessment results. The multi-index threshold is an anomaly detection method based on predefined thresholds, setting normal ranges and multi-level warning thresholds for each key network indicator. Threshold settings consider both industry standards and equipment specifications (such as ITU-T G.977's recommended values ​​for optical power) and historical performance in specific network environments. The system implements multi-level thresholds (such as normal, warning, severe, and critical), and sets differentiated thresholds for different link types and service importance. The threshold detection algorithm considers not only absolute values ​​(such as triggering a warning when optical power is below -25dBm), but also the rate of change (such as triggering a warning when optical power drops by 3dB within 10 minutes) and duration (such as a bit error rate exceeding 10 dBm for 5 minutes). -9 (Warning triggered). The health scoring algorithm is a method that integrates multiple network metrics into a single health score, providing a holistic measure of link health status. The scoring algorithm is based on a weighted scoring model, averaging the health status of each metric according to its importance, while also considering the correlation and interaction between metrics. Weight allocation is based on the degree of influence of each metric on link performance; for example, optical power and bit error rate typically have higher weights, while environmental factors such as temperature have lower weights. The scoring is standardized using a percentage scale (0-100, higher for healthier) or letter grades (A, B, C, D, F) for easy intuitive understanding. The system also implements context-aware health scoring capabilities, dynamically adjusting the scoring criteria based on link type (e.g., long-distance backbone links or data center intralinks), service type (e.g., critical or non-critical services), and time factors (e.g., peak or off-peak periods). The link health status assessment results include the current health score, detailed metric status, historical trend analysis, and expected evolution direction, providing a basis for subsequent fault prediction and migration decisions. The system uses visualization technology to intuitively display the health status of the links, such as color coding (green indicates health, yellow indicates warning, and red indicates danger) or dashboard display, to help operators quickly identify problematic links.

[0147] Fourth, based on the link health status assessment results, combined with historical fault data and machine learning models, fault prediction analysis is performed to output fault warning information. Historical fault data consists of records of past network faults collected and stored by the system, including information such as fault type, occurrence time, duration, impact scope, and solutions. A structured fault database has been established, converting unstructured fault reports into standardized fault records for easier statistical analysis and pattern recognition. Historical data undergoes preprocessing, including data cleaning (handling missing and outliers), feature extraction (extracting key features from raw fault records), and labeling (adding standardized category labels to fault events). The machine learning model is a series of data-based learning algorithms used to discover patterns from historical data and predict future events. The system employs multiple model combination strategies, including time series analysis models, classification models, and anomaly detection models. Time series analysis models, such as ARIMA (AutoRegressive Integrated Moving Average) and LSTM (Long Short-Term Memory) networks, are used to predict future trends in network metrics; classification models, such as random forests and support vector machines, are used to categorize the current network state into different risk levels; anomaly detection models, such as isolated forests and autoencoders, are used to identify anomalous patterns in network metrics. The model training process employs cross-validation and hyperparameter optimization to ensure the model's generalization ability and predictive accuracy. Fault prediction analysis integrates multiple information sources and analytical techniques: rule-based analysis (applying expert knowledge and industry best practices), statistical analysis (utilizing the statistical characteristics of historical data), and machine learning prediction (using trained models for prediction). Prediction results include fault probability (the likelihood of a fault occurring within a specific time window), expected fault type (such as hardware failure, configuration error, or external interference), and potential impact range (potentially affected services and systems). The system assigns a confidence score to the prediction results, indicating the reliability of the prediction, and generates different levels of early warning information based on the prediction results and confidence scores.

[0148] Finally, based on fault warning information and real-time monitoring data, when an optical fiber link interruption event is detected, key information including fault type, location, time, and switching path is recorded to obtain a fault event report. An optical fiber link interruption event refers to a physical or logical interruption of the optical fiber communication path, causing a temporary or permanent halt to data transmission. Interruption event detection is based on a combination of various signals, including lost signals, lost frames, and sudden drops in optical power. The system implements a rapid detection mechanism, capable of identifying interruption events at the millisecond level and distinguishing between different types of interruptions: complete interruption (complete signal loss), partial interruption (signal quality severely degraded but transmission still occurs), and intermittent interruption (repeated signal loss and recovery). Fault type classification is based on root cause analysis, including physical layer faults (such as fiber breakage, connector damage, or laser failure), transmission layer faults (such as optical amplifier failure, dispersion compensation problems, or signal interference), and control layer faults (such as configuration errors, software defects, or protocol failures). The system determines the most likely fault type through feature matching and expert system reasoning and records the fault's characteristic signature. Fault location is based on a combination of technologies, including Optical Time Domain Reflectometer (OTDR) analysis, bidirectional delay measurement, and triangulation from node status reports. It can accurately locate fiber optic breakpoints or problematic devices and visually identify the fault location on the network topology map. Fault time recording includes the initial detection time, confirmation time, and reporting time, synchronized using high-precision timestamps (millisecond resolution) and Network Time Protocol (NTP) to ensure consistency of time records across nodes. A failover path is the backup path to which traffic is transferred from the faulty path when the optical network protection mechanism is activated. The system records the failover trigger conditions, failover mechanism type (e.g., 1+1 protection, 1:1 protection, or shared mesh protection), failover completion time, and post-failure path performance data. The fault event report is a structured document containing all the above key information, as well as an impact assessment (affected services and users), priority classification (based on business importance), and recommended actions (e.g., manual intervention required). The report is distributed to relevant systems and personnel through standard interfaces, including network management systems, service assurance systems, and network operations and maintenance teams, and is also stored in the event database for subsequent analysis and auditing.

[0149] Through the above steps, comprehensive monitoring, accurate analysis, and predictive early warning of fiber optic network status are achieved. From the collection of initial network metrics to network topology construction, and then to link health assessment, fault prediction, and event reporting, a complete fault detection and early warning process is formed. This process not only promptly detects existing network faults but, more importantly, predicts potential faults, providing ample preparation time for task migration decisions and significantly improving the system's ability to cope with network faults. In particular, the application of machine learning models enables the system to learn fault patterns from historical data, continuously improving prediction accuracy and achieving a shift from passive response to proactive prevention. Simultaneously, detailed fault event reports provide accurate decision-making basis for subsequent task migrations, ensuring that migration operations can formulate optimal strategies for specific fault situations.

[0150] In one embodiment, the step of acquiring the running status of a distributed computing task based on the task monitor, establishing a mapping relationship between subtasks and computing nodes, analyzing data dependencies to construct a dependency graph, generating task status snapshots, and outputting a set of timestamped task status snapshots includes: acquiring the running status of a distributed computing task in real time, including execution progress, resource consumption, and data processing volume, based on the task monitor, and outputting task running status data; analyzing the internal structure of the task based on the task running status data, dividing the overall task into multiple subtask units, establishing a mapping relationship between subtasks and physical computing nodes, and outputting a subtask distribution mapping table; analyzing the data dependencies between subtasks based on the subtask distribution mapping table by tracking data flow and computation logic, and constructing a directed dependency graph; applying a critical path analysis algorithm to the directed dependency graph to identify critical execution paths and performance bottlenecks, and outputting critical path analysis results; and generating task status snapshots containing computation context, intermediate results, and memory status based on the critical path analysis results and the task running status data, thus obtaining the set of timestamped task status snapshots.

[0151] In this embodiment, firstly, based on the task monitor, the running status of distributed computing tasks, including execution progress, resource consumption, and data processing volume, is acquired in real time, and task running status data is output. The Task State Monitor (TSM) is a system component specifically designed to track and record the execution status of distributed computing tasks. It adopts a lightweight agent architecture, deploying a monitoring agent on each computing node and setting up a central coordinator to integrate data from each node. Each monitoring agent collects task status in three ways: direct API calls (integrated with the management interface of the computing framework), log analysis (parsing application and framework logs), and system metric collection (obtaining operating system-level resource usage). Execution progress refers to a quantitative representation of the completion status of the computing task, including the proportion of completed work to the total workload (e.g., percentage), the current execution stage (e.g., data loading, computation processing, or result output), and the estimated remaining time. Execution progress tracking employs different strategies for different types of computing tasks: for batch processing tasks, it mainly tracks the proportion of processed data to the total data volume; for iterative algorithms, it tracks the current iteration count and convergence status; for stream processing tasks, it focuses on changes in processing latency and throughput. Resource usage refers to the extent to which computing tasks utilize system resources, including CPU utilization (the percentage of processor time allocated to each core), memory usage (the ratio of allocated memory to total available memory), storage I / O (read / write operation rates and latency), and network bandwidth (data transfer rates and connection status). Resource usage monitoring employs a multi-level sampling strategy: high-frequency sampling captures short-term fluctuations (e.g., second-level CPU bursts), while low-frequency sampling reflects long-term trends (e.g., hourly memory growth). Data processing volume is an indicator of the scale and rate at which computing tasks process data, including the number of processed records (e.g., number of rows or objects), processing throughput (the amount of data processed per unit time), and data distribution (the proportion of data processed by each node). This status data is collected using a combination of event-driven and periodic methods, triggering immediate reports when critical status changes occur, while maintaining fixed-interval periodic sampling to balance real-time performance and system overhead. The collected status data undergoes standardization, converting it into a unified format, including metric identifiers, timestamps, numerical values, units, and contextual information, forming a structured task runtime status data stream.

[0152] Secondly, based on the task's runtime status data, the internal structure of the task is analyzed, and the overall task is broken down into multiple sub-task units. A mapping relationship between sub-tasks and physical computing nodes is established, and a sub-task distribution mapping table is output. Distributed computing tasks refer to large-scale computing jobs executed in parallel on multiple computing nodes, such as data analysis, machine learning training, or scientific simulations. The internal structure analysis of the task is based on the characteristics of the computing framework and the application logic, identifying the components and execution flow of the task. For different computing frameworks, the system adopts corresponding analysis strategies: for MapReduce-type tasks, the task is decomposed into Map and Reduce stages; for Spark applications, the RDD (Resilient Distributed Dataset) transformation and operation sequences are identified; for deep learning frameworks such as TensorFlow, the computation graph structure and operation nodes are analyzed. Task decomposition is the process of breaking down the overall computing task into the smallest independently schedulable execution units, called sub-tasks. The granularity of sub-tasks depends on the design philosophy of the computing framework and the complexity of the task, such as MapReduce's Mapper / Reducer, Spark's Stages and Tasks, or TensorFlow's Operations. By analyzing the execution plan and resource allocation strategy of the computing framework, natural boundary points in the computing graph are identified, enabling logical task decomposition. For complex tasks not explicitly delineated by the framework, the system also implements an automatic decomposition algorithm based on dependency analysis to identify independent execution units and synchronization points in the computing process. The mapping relationship between subtasks and physical computing nodes refers to the record of the actual execution location of each subtask, reflecting the distribution of computing load on the physical infrastructure. The mapping relationship is collected through two methods: obtaining formal allocation information from the computing framework's scheduler (such as allocation records from YARN ResourceManager or Mesos Master), and directly observing the actual execution status on each node through a monitoring agent. The mapping information includes subtask identifiers, physical node identifiers, allocated resources, and execution time windows. The subtask distribution mapping table is a structured dataset that fully records the execution location and resource allocation of all subtasks, typically stored in the form of a key-value pair set or a relational table. The mapping table supports multi-dimensional queries and can be filtered and statistically analyzed by dimensions such as subtasks, physical nodes, or time windows, providing a foundation for subsequent dependency analysis and migration decisions. The system uses visualization technology to convert the subtask distribution mapping table into an intuitive heatmap or Gantt chart, which helps to understand the distribution of computational load and execution timing.

[0153] Third, based on the subtask distribution mapping table, the system analyzes the data dependencies between subtasks by tracking data flow and computational logic, and constructs a directed dependency graph. Data dependency refers to the situation where the input of one operation or task depends on the output of another operation or task during computation; it is the most fundamental constraint relationship in distributed computing tasks. Dependency analysis obtains information through multiple methods: static analysis (analyzing application source code or intermediate representations to identify data flow patterns), framework API tracing (intercepting data access and transmission operations at the framework layer), and runtime monitoring (observing actual data read / write and transmission behavior). The system pays particular attention to three types of key dependencies: data production-consumption dependency (data generated by one subtask is consumed by another subtask), state read / write dependency (multiple subtasks access the same shared state), and control flow dependency (the completion of one subtask triggers the start of another). Data flow tracing is a technique for monitoring the movement path of data in the computing system, used to discover implicit data dependencies. The system implements multi-level data flow tracing: file system level (monitoring read / write operations of the distributed file system), memory level (tracing access patterns of shared memory regions), and network level (recording data transmission between nodes). For computing frameworks that support data lineage (such as Spark), the system directly utilizes the lineage information provided by the framework to construct initial dependencies. Computational logic analysis is the process of understanding the intrinsic connections between algorithms and processing steps, supplementing dependencies that pure data flow analysis might overlook. The system identifies logical preconditions and postconditions by parsing the execution plan of the computing framework, analyzing algorithm characteristics, and checking synchronization points. A directed dependency graph is a data structure representing dependencies between subtasks, where nodes represent subtasks and directed edges represent dependencies (from the dependent subtask to the dependent). The dependency graph uses a Directed Acyclic Graph (DAG) model to ensure that the computation process can terminate normally. Graph construction adopts an incremental approach: first, the basic structure is established based on explicit dependencies; then, implicit dependency edges are added through transitivity analysis and constraint reasoning; finally, graph optimization algorithms are applied to eliminate redundant edges and simplify the graph structure. The constructed dependency graph contains rich metadata: each node is appended with subtask characteristics (such as computational complexity and resource requirements) and execution statistics (such as average runtime and failure rate); each edge is appended with attributes such as dependency type, data flow, and criticality. A dynamic update mechanism for the dependency graph is implemented, which continuously adjusts the graph structure as the task progresses to reflect the actual changes in dependencies during the computation process.

[0154] Fourth, for directed dependency graphs, the critical path analysis algorithm is applied to identify critical execution paths and performance bottlenecks, outputting the critical path analysis results. Critical path analysis is a project management and planning evaluation technique used to identify task sequences in a project network that directly impact the overall completion time. In the context of distributed computing, the critical path is the longest path from the start node to the end node in the dependency graph, determining the shortest completion time for the entire computation task. The critical path analysis algorithm is based on the principle of dynamic programming, calculating the Earliest Start Time (EST) and Latest Finish Time (LFT) for each subtask. For each node v in the task graph, the algorithm first calculates EST(v) through forward propagation, i.e., the earliest possible start time after all preceding tasks are completed; then it calculates LFT(v) through backpropagation, i.e., the latest possible completion time without delaying the overall task completion. The time slack of a node is defined as LFT(v) - EST(v) - Duration(v), and nodes with zero slack are located on the critical path. The algorithm implementation considers the specific characteristics of distributed environments, such as network transmission latency, resource contention, and execution time variability, enhancing prediction accuracy by introducing probabilistic models and statistical analysis. Critical execution paths are sequences of subtasks in the dependency graph that decisively impact the total execution time; any delay in these subtasks directly leads to a delay in the overall task. It not only identifies static critical paths (based on average execution time) but also simulates various execution scenarios, calculating the probability of each subtask becoming part of the critical path, generating a criticality heatmap, and providing a more comprehensive assessment of criticality. Performance bottlenecks refer to components or resources that limit the overall performance of the system. In distributed computing, these may manifest as computationally intensive subtasks, nodes with high data dependencies, or points of resource contention. Bottlenecks are identified through multiple methods: structural analysis (finding nodes with high fan-in / fan-out degrees in the dependency graph), execution statistical analysis (identifying abnormally time-consuming subtasks), and resource contention analysis (discovering situations where multiple subtasks contend for the same resource). The critical path analysis results include multiple dimensions: path details (the sequence of subtasks and dependencies on the critical path), time estimation (total execution time and the contribution of each node), bottleneck identification (major limiting factors and optimization suggestions), and sensitivity analysis (the degree to which changes in the execution time of each subtask affect the total time). The analysis results are presented intuitively through visualization tools, such as path highlighting, time scale plots, and bottleneck heatmaps, to help understand the time dependency structure of computational tasks.

[0155] Finally, based on the critical path analysis results and task execution status data, task state snapshots containing computation context, intermediate results, and memory state are generated, resulting in a set of timestamped task state snapshots. A task state snapshot is a complete record of the computation task execution state at a specific point in time, containing all the information needed to rebuild or restore task execution. The snapshot generation strategy is based on several factors: critical path analysis (subtasks on the critical path receive higher snapshot frequencies), state change rate (tasks with frequent changes require more frequent snapshots), and recovery cost (tasks with high recovery costs are prioritized for state saving). Computation context refers to the state information of the task execution environment, including the program counter, call stack, local variables, and control flow state. Context capture is implemented through the checkpointing mechanism provided by the framework or a custom state tracker, employing different capture techniques for different computing frameworks: for MapReduce-like frameworks, it records the processing progress and state of Map / Reduce tasks; for Spark, it saves the RDD family tree and transformation state; for iterative algorithms, it records the current iteration count and convergence parameters. Intermediate results are temporary data generated during computation, serving as both the output of completed computations and the input for subsequent computations. A selective storage strategy is adopted for intermediate results: small but costly results are fully preserved, while large but easily reconstructable results only retain metadata and reconstruction instructions. Result storage employs a hierarchical architecture: frequently accessed data is stored in memory or a fast storage layer, while infrequently accessed data is migrated to an economical storage layer. Memory state refers to the current values ​​of data structures and variables maintained in memory by the computation task, and is a crucial part of task recovery. Memory state capture uses an incremental approach: the first snapshot saves the complete state, and subsequent snapshots only record the changed parts, reducing storage overhead and generation time. For non-serializable states (such as local resource handles or external connections), reconstruction information is recorded instead of direct state values. Snapshot serialization uses an efficient binary format and applies compression algorithms to reduce storage size while preserving structural information for easy partial access. The timestamped task state snapshot set consists of multiple snapshot instances organized chronologically, forming a time series of task execution states. Each snapshot is accompanied by a precise generation timestamp, recording the exact time the snapshot was created, facilitating the selection of an appropriate recovery baseline at each time point. Snapshot collection management includes lifecycle control (automatic cleanup of expired snapshots), version management (maintaining dependencies between snapshots), and an indexing mechanism (supporting efficient querying and location). A snapshot consistency guarantee mechanism is implemented to ensure that multi-node snapshots generated in a distributed environment can form a globally consistent state view, providing a reliable foundation for subsequent task recovery.

[0156] In one embodiment, the step of analyzing resource requirements through the resource manager, reserving spare resources, identifying key intermediate results to formulate caching strategies, deploying a distributed caching system through the cache manager, and outputting a resource pre-allocation scheme and a cache deployment plan includes: analyzing the bandwidth requirements and key data transmission paths between computing nodes based on the directed dependency graph and the network topology graph, and outputting network resource requirement analysis results; reserving spare optical paths and wavelength resources in the physical network through the resource manager based on the network resource requirement analysis results, and outputting a wavelength resource pre-allocation scheme; selecting suitable spare computing nodes and reserving computing resources based on the directed dependency graph and the network topology graph, and outputting a computing resource reservation scheme; identifying key intermediate results and state data in the computing process based on the directed dependency graph, formulating a multi-level caching strategy, and outputting an intermediate result caching strategy; and deploying a distributed caching system on selected nodes through the cache manager based on the wavelength resource pre-allocation scheme, the computing resource reservation scheme, and the intermediate result caching strategy, establishing a data version control and consistency protocol, and obtaining the resource pre-allocation scheme and the cache deployment plan.

[0157] In this embodiment, firstly, based on the directed dependency graph and network topology graph, the bandwidth requirements and key data transmission paths between computing nodes are analyzed, and the network resource requirement analysis results are output. The directed dependency graph is a data structure representing the dependencies between subtasks within a computing task, where nodes represent subtasks and directed edges represent data or control dependencies. The network topology graph is a graphical representation describing the physical network connection structure, including network devices (such as routers and switches), links, and their attributes (such as bandwidth and latency). Bandwidth requirement analysis is a method for evaluating the data transmission volume and rate requirements during distributed computing, achieved by examining data flow patterns in the directed dependency graph. Using data flow modeling techniques, data transmission edges in the dependency graph are first identified (representing data flowing from one subtask to another), and then the data transmission volume of each edge is estimated. The data transmission volume estimation is based on multiple information sources: historical execution statistics (the actual data volume recorded during previous runtimes), static code analysis (the data size inferred from the source code), and runtime monitoring (the data flow patterns observed during current execution). For complex data processing patterns, the system considers data amplification and data shrinkage factors, representing the growth and reduction ratios of output data volume relative to input data volume, respectively. Critical data transmission paths refer to data flow channels that significantly impact overall performance during computation, typically manifested as large-volume data transfers or data dependencies located on critical execution paths. Critical paths are identified using a three-step method: first, all data dependency edges on critical computation paths are marked; second, edges with data transmission volumes exceeding predetermined thresholds are identified; and finally, considering network topology constraints, the actual transmission path and performance characteristics of each data dependency edge on the physical network are evaluated. The network resource demand analysis results comprise four key components: a bandwidth demand matrix (expected data transmission volume and rate between each pair of computing nodes), a critical path set (a list of data transmission paths ordered by importance), a time distribution map (the pattern of data transmission demand over time), and bottleneck analysis (identifying potential network congestion points and resource contention areas). The analysis results are presented intuitively using visualization techniques, including heatmaps (showing the intensity of bandwidth demand between node pairs), path highlighting (highlighting critical data transmission paths on the network topology), and time series graphs (showing the trend of bandwidth demand over time).

[0158] Secondly, based on the network resource demand analysis results, a wavelength resource pre-allocation scheme is output by reserving spare optical paths and wavelength resources in the physical network through a resource manager. The Resource Pre-allocation Manager (RPM) is a core component in a distributed system responsible for resource allocation, reservation, and management, maintaining a global resource view and executing resource allocation decisions. An optical path is an end-to-end optical channel connection in an optical network, providing a transparent data transmission channel with fixed bandwidth and low latency. Each optical path consists of a source node, a destination node, and intermediate optical switching nodes, and is implemented at the physical layer by optical signals of a specific wavelength. Wavelength resources refer to different wavelength channels available for data transmission in an optical fiber communication system, forming the basis of Wavelength Division Multiplexing (WDM) technology. In a WDM system, a single optical fiber can simultaneously transmit multiple optical signals of different wavelengths, each forming an independent transmission channel. The reservation strategy is based on a risk assessment method, evaluating the failure risk of each network link, considering historical failure records, physical environmental factors, and link usage, and allocating more spare resources to high-risk links. Optical path planning employs a multi-constraint path calculation algorithm, comprehensively considering bandwidth requirements, latency requirements, physical layer constraints (such as transmission distance and dispersion limitations), and available wavelength resources to calculate the optimal and suboptimal optical paths for each pair of critical nodes. A wavelength allocation strategy is implemented, assigning a specific wavelength to each optical path after determining the physical route, adhering to wavelength continuity (using the same wavelength throughout the optical path) and wavelength uniqueness (different wavelengths for different optical paths on the same link). For larger networks, a heuristic algorithm for routing and wavelength allocation is used to find a near-optimal solution within a reasonable timeframe. Backup resource reservation employs a hierarchical strategy: critical service paths use 1+1 protection (data is transmitted simultaneously on two physically isolated paths), important service paths use 1:1 protection (a backup path is reserved but not transmitted simultaneously), and ordinary service paths use shared protection (multiple service paths share backup resources). The wavelength resource pre-allocation scheme comprises four parts: primary optical path configuration (physical path and wavelength allocation for each primary optical path), backup optical path configuration (corresponding backup paths and wavelengths), resource reservation time window (validity period of reserved resources), and protection switching strategy (triggering conditions and processes for fault detection and optical path switching). Through integration with the SDN controller and optical network management system, the scheme transforms logical resource allocation into actual network configuration, achieving effective resource reservation.

[0159] Third, based on the directed dependency graph and network topology graph, suitable backup computing nodes are selected and computing resources are reserved, outputting a computing resource reservation scheme. Computing resources include hardware resources such as processors (CPUs), memory, storage, and accelerators, which are the infrastructure for executing computing tasks. Backup computing nodes are replacement nodes that can take over the computing tasks of the primary computing node when it fails, and need to have similar performance characteristics and resource configurations as the primary node. Node selection adopts a multi-objective optimization method, balancing multiple selection criteria: resource compatibility (backup nodes should have similar resource configurations to the primary node), network connectivity (backup nodes should have good network connections, especially data paths with related nodes), physical isolation (backup nodes should be located in different physical isolation domains from the primary node to avoid being affected by the same failure), and load balancing (the selection of backup nodes should consider the overall load distribution of the system). A resource matching algorithm is implemented to match the resource requirements of subtasks with the resource configurations of candidate backup nodes and calculate a compatibility score. Resource requirements include the number and architecture of CPU cores, memory capacity and bandwidth, storage type and capacity, special hardware requirements (such as GPUs or FPGAs), and software environment dependencies. Node scoring is based on a weighted model, assigning different weights to each dimension of indicators according to their importance. The overall score is calculated and ranked to select the optimal set of backup nodes. A hierarchical reservation strategy is adopted to provide differentiated protection for subtasks of different importance: critical subtasks use dedicated reservation (exclusive backup resources), important subtasks use priority reservation (granting priority in resource allocation but allowing resource sharing), and ordinary subtasks use elastic reservation (dynamically allocating resources based on availability). Reservation implementation interacts with the resource management system through a resource reservation protocol to create resource reservation records and set trigger conditions. The resource reservation scheme consists of five parts: subtask-backup node mapping (a list of backup nodes corresponding to each subtask), resource reservation details (the specific resource type and quantity reserved for each backup node), reservation priority (the priority order of different reservation requests), trigger conditions (conditions for activating reserved resources), and release strategy (the release mechanism after reserved resources are used). The scheme integrates with cluster management systems (such as Kubernetes, Mesos, or YARN) through standard interfaces, translating logical resource reservation into actual system configuration.

[0160] Fourth, based on the directed dependency graph, identify key intermediate results and state data in the computation process, formulate a multi-level caching strategy, and output the intermediate result caching strategy. Intermediate results refer to temporary data generated during the computation process, serving as both the output of previous computation steps and the input of subsequent computation steps. State data refers to the runtime state information maintained during the computation process, including the state variables of the iterative algorithm, the state of data structures in memory, and the control flow state. Key intermediate result identification is based on impact analysis. The system assesses the importance of each intermediate result, considering four key factors: recomputation cost (the computational resources and time required to regenerate the result), usage frequency (the number of times the result is referenced in subsequent computations), data volume (the storage volume of the result), and location (its strategic position in the dependency graph, especially results located on the critical path). Cut points and key nodes are marked on the dependency graph using graph algorithms; intermediate results generated at these locations have high retention value. The multi-level caching strategy is a hierarchical management method for cache resources, placing different types of data on different levels of storage media according to data characteristics and access patterns. The system employs a three-tiered caching structure: in-memory cache (high-speed access, suitable for frequently accessed, small-volume data), local storage cache (medium access speed, suitable for data accessed at a moderate frequency), and distributed storage cache (large capacity, suitable for important but infrequently accessed data). The caching strategy definition comprises five key components: cache location (which cache level the data should be stored in), cache granularity (the size of the cached data unit), cache frequency (the time interval between cache operations), cache triggering conditions (when to perform cache operations), and cache replacement strategy (how to decide which data to replace when cache space is insufficient). An adaptive caching algorithm is implemented, dynamically adjusting caching strategy parameters based on real-time observed data access patterns and system load. Cache cost-benefit analysis is used to evaluate the value of each potential cache point, calculating the ratio of cache benefits (saving recompile time) to cache costs (storage overhead and cache operation overhead), prioritizing the caching of high-value data. The intermediate result caching strategy specifically includes three aspects: time-driven caching (performing cache operations at fixed time intervals), event-driven caching (performing caching when a specific event, such as the completion of a critical computation phase), and a hybrid strategy (combining time and event triggering mechanisms). The policy document defines in detail the processing methods for each type of intermediate result, including storage format, compression method, metadata tagging, and lifecycle management rules.

[0161] Finally, based on the wavelength resource pre-allocation scheme, computing resource reservation scheme, and intermediate result caching strategy, the system deploys a distributed caching system on selected nodes through the cache manager, establishes data version control and consistency protocols, and obtains the resource pre-allocation scheme and cache deployment plan. The State Cache Manager (SCM) is a dedicated component responsible for the configuration, deployment, and operation of the distributed caching system, coordinating cache resource allocation and data management. A distributed caching system is a temporary data storage mechanism across multiple nodes, providing high-performance data access, fault tolerance, and scalability. Considering various mainstream distributed caching technologies, such as Redis, Memcached, and Apache Ignite, the most suitable caching implementation is selected based on application characteristics and performance requirements. The deployment process adopts the Infrastructure as Code (IaC) approach, using automated scripts and configuration templates to achieve rapid deployment and consistent configuration of the caching system. Deployment configuration includes node selection (determining cache node locations based on the computing resource reservation scheme), topology design (defining the organizational structure between cache nodes, such as master-slave replication or sharded clusters), and resource allocation (allocating CPU, memory, and network resources to the caching process). Data version control is a mechanism for tracking and managing the history of data changes, ensuring access to specific historical states of data. The system implements Multi-Version Concurrency Control (MVCC), allowing read operations to access stable snapshots of data without being blocked by concurrent write operations. Version control employs a dual mechanism of timestamps and version numbers; each data item is appended with a creation timestamp, modification timestamp, and version identifier, forming a complete version chain. Version storage uses an incremental approach, recording only the differences between versions rather than complete copies, reducing storage overhead. The consistency protocol defines the rules and processes for maintaining consistency among multiple data replicas in a distributed environment. The system supports multiple consistency models, from strong consistency (all read operations see the latest written data) to eventual consistency (the system guarantees that all replicas reach a consistent state after a period of time), allowing the selection of an appropriate consistency level based on application requirements and performance goals. The protocol implementation includes data propagation mechanisms (how to spread updates to all replicas), conflict detection (how to identify conflicts caused by concurrent updates), and conflict resolution (how to handle detected data conflicts). The resource pre-allocation scheme and cache deployment plan is a comprehensive document that integrates all the aforementioned planning results, including six core parts: resource pre-allocation summary (overall view of wavelength and computing resource reservation), cache system architecture (cache node topology and organizational structure), deployment configuration (detailed configuration parameters for each cache node), data management strategy (version control and consistency maintenance methods), fault handling mechanism (cache system fault detection and recovery process), and performance monitoring scheme (monitoring indicators and methods for cache system operation status).The deployment process is automated and includes four phases: environment preparation, software installation, application configuration, and verification testing, ensuring that the caching system is deployed correctly as planned and achieves the expected performance metrics.

[0162] In one embodiment, the step of analyzing the scope of the fault impact through the migration scheduler, extracting the affected computational subgraph, selecting the migration target node based on the resource pre-allocation scheme, and outputting the migration execution plan includes: based on the fault event report, combined with the network topology diagram and the subtask distribution mapping table, quickly analyzing the computational nodes and subtasks affected by the fault link through the migration scheduler, and outputting the fault impact analysis results; based on the fault impact analysis results, extracting the affected computational subgraph from the directed dependency graph, including directly affected subtasks and indirectly dependent upstream and downstream tasks, and outputting the computational subgraph model to be migrated; based on the... The process involves describing the computational subgraph model to be migrated and the computational resource reservation scheme, evaluating the adaptability, current load, and network connectivity of each backup node, selecting the optimal combination of migration target nodes, and outputting the migration target node allocation scheme. Based on the criticality and dependencies of each subtask in the computational subgraph model to be migrated, the migration operations are prioritized, and a subtask migration priority list is output. According to the migration target node allocation scheme and the subtask migration priority list, combined with the optical network protection switching time characteristics and the computational task fault tolerance time threshold, the available migration time window is evaluated, a time-sensitive migration schedule is formulated, and the migration execution plan is obtained.

[0163] In this embodiment, firstly, based on the fault event report, combined with the network topology map and subtask distribution mapping table, the migration scheduler quickly analyzes the computing nodes and subtasks affected by the faulty link, and outputs the fault impact analysis results. The fault event report is a structured document generated by the network monitoring system, recording detailed fault information detected in the fiber optic network, including fault type, location, occurrence time, and severity. The migration scheduler is the core component responsible for formulating and executing computing task migration strategies, coordinating all parts of the system to complete the entire process from fault detection to task recovery. The impact analysis uses a graph traversal algorithm, first marking the faulty link in the network topology map, and then identifying all network paths dependent on that link and directly connected computing nodes. The system implements a two-stage analysis method: the first stage is rapid assessment, completed within milliseconds after fault detection, mainly identifying the direct impact caused by physical connection interruption; the second stage is in-depth analysis, completed within seconds, assessing indirect impacts including bandwidth degradation and routing changes. Node impact determination is based on connectivity analysis, assessing the changes in the connection status between each computing node and the rest of the network. The system considers three levels of impact: complete isolation (nodes are completely disconnected from the network), partial isolation (nodes maintain partial connectivity but experience performance degradation), and indirect impact (nodes remain connected but depend on data from affected nodes). Subtask impact analysis is achieved by correlating a subtask distribution mapping table with changes in network connectivity, accurately identifying affected computing units. The system considers not only currently executing subtasks but also predicts and analyzes subtasks that have not yet started but are planned to be executed on affected nodes. Impact assessment employs multi-dimensional indicators: task criticality (the importance of the subtask to the overall computation), time urgency (how close the expected completion time of the subtask is to the deadline), and resource requirements (the computational, memory, and network resources required by the subtask). The failure impact analysis results include four key parts: a list of affected nodes (a set of computing nodes sorted by impact level), a list of affected subtasks (details of subtasks that need to be migrated on each node), a network impact map (visualization of the failure's impact on network connectivity), and a time estimate (predicted failure duration and its estimated impact on computing tasks). The analysis results are presented intuitively through visualization tools, using color coding to indicate different impact levels (e.g., red for severe impact, yellow for moderate impact, and green for slight impact), and providing an interactive interface for operators to explore detailed information.

[0164] Secondly, based on the failure impact analysis results, the affected computational subgraphs are extracted from the directed dependency graph, including directly affected subtasks and indirectly dependent upstream and downstream tasks, outputting a computational subgraph model to be migrated. A computational subgraph is a subset of the complete computational dependency graph, containing a set of interconnected subtasks and their dependencies. The extraction process employs a graph cutting algorithm, first marking all directly affected subtask nodes, then expanding upstream and downstream through dependencies until a suitable boundary is reached. An adaptive boundary determination strategy is implemented, dynamically adjusting the subgraph boundary based on multiple factors: migration cost (including state transfer cost and reconstruction cost), dependency density (the tightness of dependencies between subtasks), and execution stage (the position of the subtask in the overall computational flow). Directly affected subtasks refer to computational units that execute on failed nodes or heavily rely on failed links for data transmission. The identification process is based on node mapping tables and network path analysis to accurately locate computational components directly affected by network failures. For each directly affected subtask, the system assesses its current state (e.g., not started, in progress, or near completion) and importance, providing a basis for subsequent migration decisions. Indirectly dependent upstream and downstream tasks refer to computational units that, while not directly affected by network failures, have data or control dependencies on directly affected tasks. Upstream task identification is achieved through reverse dependency analysis, starting with directly affected subtasks and tracing back along the input edges of the dependency graph to find all subtasks whose outputs are used by the affected task. Downstream task identification is achieved through forward dependency analysis, starting with directly affected subtasks and tracing back along the output edges of the dependency graph to find all subtasks that depend on the outputs of the affected task. Dependency propagation depth control is used to limit the tracing range of upstream and downstream tasks, preventing excessively large subgraphs from leading to high migration costs. A decay model is employed, gradually reducing the probability of including a directly affected task as the distance increases, unless the task has special importance. The computational subgraph model to be migrated is a structured representation containing all computational components that must be migrated and their relationships, including five core elements: a set of nodes (subtasks to be migrated), a set of edges (dependencies between subtasks), an attribute map (characteristic descriptions of each subtask and its dependencies), a priority marker (indicating the migration priority of subtasks), and a state summary (the current execution status and completion rate of each subtask). The model uses a standard graph data structure, which supports efficient querying and operations, and is convenient for subsequent migration planning.

[0165] Third, based on the computational subgraph model to be migrated and the computational resource reservation scheme, the system evaluates the adaptability, current load, and network connectivity of each backup node, selects the optimal combination of target nodes for migration, and outputs the target node allocation scheme. Adaptability evaluation measures the degree to which backup nodes meet the resource requirements and environmental dependencies of subtasks, considering factors such as hardware compatibility, software environment, and performance characteristics. The system implements a multi-dimensional matching algorithm to create a resource requirement vector for each subtask, including CPU requirements (number of cores and architecture type), memory requirements (capacity and bandwidth), storage requirements (capacity, type, and access mode), and accelerator requirements (such as GPU or FPGA). Simultaneously, a resource supply vector is created for each backup node, describing its available resource configuration. Adaptability calculation is based on a similarity measure between the requirement vector and the supply vector, combined with environmental compatibility checks (such as operating system version, library dependencies, and network protocol support). Current load analysis assesses the resource usage of backup nodes to avoid migrating tasks to already heavily loaded nodes. The system collects resource usage metrics of backup nodes in real time through a monitoring agent, including CPU utilization, memory usage, I / O load, and network traffic. Load assessment employs time-series analysis, considering not only current snapshot values ​​but also recent load trends and fluctuation patterns to predict load conditions during migration. The system implements a load balancing heuristic algorithm to avoid uneven resource allocation or hotspot nodes during migration. Network connectivity assessment evaluates the network health of the target node and the quality of its connections with other relevant nodes. The system considers three key metrics: bandwidth availability (the network transmission capacity available to the node), connection reliability (the stability and redundancy of network connections), and topological location (the node's position in the network and its distance from dependent nodes). Network assessment pays particular attention to the connection paths between backup nodes and subtask-dependent nodes to ensure efficient data exchange after migration. Optimal node combination selection uses Integer Linear Programming (ILP), formalizing the node allocation problem as an optimization problem. The objective function comprehensively considers fitness, load balancing, and network efficiency, with constraints including resource capacity limitations, affinity requirements, and anti-affinity constraints. For larger-scale problems, the system uses heuristic methods such as genetic algorithms to find near-optimal solutions within a reasonable timeframe. The migration target node allocation scheme comprises four key parts: subtask-target node mapping (the target migration location for each subtask), resource allocation details (the specific resources reserved for the migration task on each target node), network configuration (network connections and routing adjustments after migration), and dependency handling strategy (methods for handling cross-node dependencies). The scheme is presented intuitively through visual graphics, using node-edge representation to show the changes in computational distribution before and after migration.

[0166] Fourth, based on the criticality and dependencies of each subtask in the computational subgraph model to be migrated, the system prioritizes migration operations and outputs a list of subtask migration priorities. Criticality assessment measures the importance of a subtask to the overall computational task completion, considering its position in the dependency graph, its contribution to the final result, and the difficulty of substitution. The system implements a criticality scoring model, comprehensively considering four dimensions: critical path impact (whether the subtask is located on the critical execution path), dependency impact scope (how many other subtasks depend on the subtask's output), computational complexity (the computational resource requirements and execution time of the subtask), and reconstruction cost (the cost of re-executing the subtask in case of migration failure). The scoring uses a weighted method, assigning different weights to different dimensions based on their importance in specific application scenarios. Dependency analysis checks the pre- and post-constraints between subtasks to ensure that the migration order does not violate computational logic. The system identifies strong dependencies (tasks that must be executed strictly in sequence) and weak dependencies (tasks that allow partial parallelism) through dependency graph analysis. Particular attention is paid to circular dependency detection, identifying and handling potential circular structures in the dependency graph, typically resolved by identifying breakpoints. The priority ranking algorithm is a variant of topological sorting. It first ensures that dependency constraints are met (the priority of a dependent task is no higher than that of the task it depends on), and then further adjusts priorities based on criticality. The system implements a hierarchical priority mechanism, dividing subtasks into multiple priority levels: highest priority (tasks on the critical path with broad dependencies), high priority (tasks on the critical path or with broad dependencies), medium priority (non-critical tasks with specific importance), and basic priority (other tasks). For tasks within the same priority level, fine-grained ranking is further considered based on execution status (tasks with longer running times receive higher priority) and expected completion time (tasks nearing completion are processed first). Priority adjustment also considers group dependencies, i.e., subtask groups with close data or control relationships are typically assigned similar priorities and migrated together as much as possible. The subtask migration priority list contains three core parts: priority ranking (a sequence of subtasks ordered by migration urgency), priority grouping (a set of subtasks with similar priorities), and priority adjustment rules (conditions for dynamically adjusting priorities based on runtime status). The list is represented in a hierarchical structure for easy querying and updating during migration execution.

[0167] Finally, based on the migration target node allocation scheme and the subtask migration priority list, combined with the optical network protection switchover time characteristics and the computation task fault tolerance time threshold, the available migration time window is evaluated, a time-sensitive migration schedule is formulated, and a migration execution plan is obtained. Optical network protection switchover time refers to the time required for an optical network to switch to a backup path after detecting a fault; it is a key indicator for evaluating network resilience. Switchover time typically consists of three parts: fault detection time (the time to detect link or device anomalies), protection path calculation time (the time to determine the backup path), and protection path activation time (the time to configure and enable the backup path). In modern optical networks, the switchover time of standard protection mechanisms ranges from milliseconds to seconds, depending on the type of protection mechanism and network size. The computation task fault tolerance time threshold refers to the maximum time a computation task can tolerate network interruption without causing permanent data loss or unrecoverable errors. The threshold is affected by various factors: the application's internal checkpointing mechanism, data cache persistence, timeout settings, and retry logic. The system estimates the fault tolerance threshold by analyzing application configurations and historical operating data and sets differentiated thresholds for different types of subtasks. The migration time window assessment comprehensively considers network recovery time and task fault tolerance thresholds to calculate the time range available for performing migration operations. The system implements time-critical path analysis to identify time-critical operation sequences during the migration process and estimates the execution time of each operation: state capture time (time to create a snapshot of the subtask's state), data transfer time (time to transfer state data to the target node), and task restart time (time to resume subtask execution on the target node). Time-sensitive migration scheduling is a task scheduling method that considers time constraints, ensuring that the most important migration operations are completed within a limited time window. The system employs a multi-stage scheduling strategy: an emergency phase (completing the migration of the most critical subtasks in a very short time), a standard phase (completing the migration of most subtasks within a normal time window), and a delayed phase (completing the migration of low-priority subtasks within an extended time window). The scheduling algorithm considers parallel migration opportunities, identifying subtask groups that can be migrated simultaneously without causing resource contention, maximizing migration throughput. The system implements an adaptive scheduling mechanism that can dynamically adjust the scheduling plan based on actual migration progress and resource availability, handling abnormal situations during execution. The migration execution plan is a detailed operational guide that directs the system through the entire process from fault detection to task recovery. It comprises five core components: a timeline (a detailed sequence of operations), resource allocation (resource usage plans for each migration stage), an operation sequence (specific migration steps for each subtask), monitoring points (verification and confirmation steps for key nodes), and contingency plans (to address potential anomalies during the migration process). The execution plan integrates with task scheduling and resource management systems through standard interfaces, enabling automated execution and monitoring of migration operations.

[0168] In one embodiment, migration decision optimization based on a matroid-constrained optimization algorithm includes: based on the fault impact analysis results and the directed dependency graph, formalizing the distributed computing migration problem into a matroid-constrained submodulus maximization problem, constructing a matroid M=(N,I), where N is the set of all possible subtask-target node pairs, and I is the set of feasible migration schemes that satisfy resource constraints and dependencies, and outputting a matroid-constrained problem model; based on the matroid-constrained problem model, designing a fixed-parameter algorithm to handle the offline computation of the migration planning stage and the dynamic response of the real-time migration execution stage, and outputting a fixed-parameter algorithm scheme; based on the fixed-parameter algorithm scheme, establishing a dynamic adjustment mechanism for the migration scheme, adjusting the migration decision in real time when the network state or computing node state is updated, and outputting a dynamically optimized migration scheme; based on the dynamically optimized migration scheme, designing a unified migration optimization algorithm to handle different scenarios of independent and competitive tasks, and outputting a unified migration optimization strategy; based on the unified migration optimization strategy, realizing the theoretical guarantee and performance evaluation of the migration scheme, and obtaining the optimized migration execution plan.

[0169] In this embodiment, firstly, based on the failure impact analysis results and the directed dependency graph, the distributed computing migration problem is formalized as a matroid-constrained submodular maximization problem, constructing a matroid M=(N,I) and outputting the matroid-constrained problem model. A matroid is a mathematical structure that can be represented as M=(N,I), where N is the set of basic elements, and I is a family of subsets of N, satisfying the properties of independent systems: the empty set belongs to I; subsets of any independent set are also independent; if two independent sets are of different sizes, the smaller set can add an element from the larger set while maintaining independence. In the context of computing migration, N represents the set of all possible subtask-target node pairs, and each element represents the decision to "migrate subtask t to node n"; I represents the set of feasible migration schemes that satisfy resource constraints and dependencies. Submodular Function f:2 N →R is a set function with diminishing marginal returns, for any set A B N and elements e ∈ N\B satisfy f(A∪{e})-f(A)≥f(B∪{e})-f(B). A transfer utility function is designed, comprehensively considering the computational performance, network efficiency, and fault recovery degree after the transfer. This function satisfies the submodularity property, reflecting the diminishing marginal utility characteristic of the transfer decision. The submodularity maximization problem with matroid constraints is formalized as: maximizing f(S), where S The problem involves finding the set of decisions that maximizes the utility function while satisfying matroid constraints, where N ∈ I. The system identifies three key constraints and represents them as matroids: resource constraints (resource capacity limits for each target node), dependency constraints (maintaining the integrity of dependencies between subtasks), and conflict constraints (avoiding the simultaneous execution of mutually exclusive migration decisions). The matroid constraint problem model defines in detail the mathematical representation of the problem, the formal description of the constraints, and the precise quantification of the optimization objective, providing a theoretical foundation for subsequent algorithm design.

[0170] Secondly, based on the matroid constraint problem model, a fixed-parameter algorithm is designed to handle the offline computation of the migration planning phase and the dynamic response of the real-time migration execution phase, outputting the fixed-parameter algorithm solution. Fixed-parameter algorithms are a special type of algorithm design technique that parameterizes the complexity of NP-hard problems, expressing the algorithm complexity as f(k)·n. O(1) The form is: where k is the problem parameter, n is the input size, and f is a function only related to k. In the migration problem, key parameters were identified: the maximum number of migration decisions, the number of resource types, and the maximum depth of dependencies. In the offline computation stage, a branch-and-bound algorithm is used to accurately solve the submodular maximization problem under matroid constraints, suitable for the planning stage before migration, generating an initial optimal solution. The algorithm designs an efficient branching strategy, prioritizing decisions based on their marginal benefits and utilizing the characteristics of the submodular function to design a compact upper bound estimate, quickly pruning ineffective search spaces. In the real-time migration execution stage, a parameterized variant of the greedy algorithm is used. By maintaining a priority queue of candidate decisions, the algorithm selects the decision that satisfies the matroid constraints and brings the maximum marginal benefit each time. The algorithm guarantees a solution with an approximation ratio of (1-1 / e) under resource constraints. The system implements a problem kernelization technique, identifying and removing elements that have no impact on the optimal solution in the preprocessing stage, reducing the original problem to a smaller equivalent problem kernel, significantly reducing real-time computational complexity. The fixed-parameter algorithm scheme details the parameter selection strategy, algorithm flow, theoretical guarantees, and complexity analysis, providing a theoretically feasible and practically efficient solution for migration decisions.

[0171] Third, based on a fixed-parameter algorithm, the system establishes a dynamic adjustment mechanism for the migration plan. When the network state or compute node state updates, the migration decision is adjusted in real time, outputting a dynamically optimized migration plan. This dynamic adjustment mechanism is a method that corrects decisions in real time based on the latest observed system state and environmental changes during execution. The system implements an incremental computation framework, making local adjustments based on the original solution, avoiding complete recalculation and significantly reducing computational overhead. The framework maintains a decision dependency graph, tracking the preconditions and scope of influence of each decision. When the state changes, only the affected subset of decisions is re-evaluated. State update processing adopts an event-driven model. The system defines four types of key events: changes in resource availability, network state updates, task state changes, and new fault discovery. For each type of event, a specific response strategy and priority rules are designed to ensure that the most critical changes are handled promptly. Real-time decision adjustment uses a rolling optimization method, performing adjustment calculations at fixed time intervals and setting a computation time cap to ensure the real-time nature of the adjustment decisions. The system implements adaptive batch processing, dynamically adjusting the batch size based on system load and event frequency to balance response time and processing efficiency. The dynamic optimization migration scheme defines status monitoring points, triggering conditions, adjustment rules, and priority strategies to ensure that the system can flexibly respond to various changes and abnormal situations during the execution process.

[0172] Fourth, based on a dynamic optimization migration scheme, a unified migration optimization algorithm is designed to handle different scenarios of independent and competitive tasks, outputting a unified migration optimization strategy. The unified migration optimization algorithm is a general-purpose algorithm framework capable of handling various task relationship patterns, adapting to scenarios ranging from completely independent to highly competitive. The algorithm adopts a layered design: the bottom layer implements the optimization core under matroid constraints, the middle layer handles special constraints for different scenarios, and the top layer provides a unified interface and coordination mechanism. For independent task scenarios, a parallel optimization strategy is adopted, decomposing the large-scale problem into multiple independent sub-problems for parallel solution, and then merging the results, significantly improving computational efficiency. For competitive task scenarios, the system implements a game theory-based resource allocation mechanism, modeling competition as a cooperative game, using Shapley values ​​to calculate the fairness of resource allocation, and designing incentive-compatible allocation rules to promote the formation of a globally optimal solution. The algorithm integrates online learning technology, continuously adjusting the parameters of the decision model by observing the actual effects of historical migration operations, improving the accuracy of subsequent decisions. The system pays special attention to edge case handling, such as extreme resource scarcity, severe network congestion, or task priority conflicts, designing specific coping strategies for each type of situation. The unified migration optimization strategy defines the algorithm framework, scene recognition rules, strategy selection logic, and parameter adjustment mechanism, providing the system with a consistent and flexible optimization method.

[0173] Finally, based on a unified migration optimization strategy, the theoretical guarantee and performance evaluation of the migration scheme are achieved, resulting in an optimized migration execution plan. The theoretical guarantee is a mathematical proof of the algorithm's performance in the worst case, providing a reliability foundation for the system. It is proven that under matroid constraints, the designed algorithm can guarantee at least (1-1 / e) times (approximately 63%) the optimal solution, a guarantee derived from the theoretical properties of the submodular function maximization problem under matroid constraints. For dynamic scenarios, the system analyzes the algorithm's competition ratio, proving that under finite state change rates, the dynamic adjustment mechanism can maintain an approximation ratio no less than 80% of the static algorithm. Performance evaluation is conducted through both theoretical analysis and simulation experiments. Theoretical analysis includes computational complexity evaluation (time and space complexity), convergence analysis, and stability analysis. Simulation experiments design various typical scenarios, including small-scale failures, large-scale failures, cascading failures, and resource contention, to evaluate the algorithm's performance under different conditions. Evaluation metrics include migration completion time, computational performance loss, resource utilization, and task completion rate. Comparison with benchmark algorithms verifies the superiority of the proposed method. The optimized migration execution plan integrates the results of theoretical analysis and performance evaluation, and makes fine adjustments to the original plan, including parameter optimization, threshold adjustment and strategy fine-tuning, forming a final execution solution that is theoretically sound and highly efficient in practice.

[0174] In one embodiment, a fault-tolerant path optimization mechanism for fiber optic networks based on Gomory-trees includes: constructing a weighted undirected graph G=(V,E) according to the physical topology and wavelength resource allocation of the fiber optic network, where vertices V represent network nodes, edges E represent fiber optic links, and the weight of the edges represents the bandwidth capacity or reliability index of the links, and outputting a network graph model; calculating the Gomory-tree T for the network graph model, retaining the minimum cut information between all node pairs in the graph, designing an incremental update mechanism, and outputting a network capacity analysis model; and, based on the network capacity analysis model, when it is necessary to find the maximum flow path between any two computing nodes, directly querying the path between nodes in the Gomory-tree. The minimum weighted edge on the path is used to calculate the maximum available bandwidth between key node pairs in the distributed computing task, and the maximum traffic information between nodes is output. Based on the maximum traffic information between nodes and the bandwidth requirements of the computing task, an optimal backup path resource allocation strategy is formulated. For each pair of key nodes, a set of wavelength resources that meets the corresponding minimum bandwidth requirements and minimizes total resource consumption is allocated, and an optimized backup path resource allocation scheme is output. Based on the optimized backup path resource allocation scheme, for the multi-point communication mode in distributed computing, the network graph model is extended to a hypergraph, and the Gomory-Hu tree extension algorithm on the hypergraph is applied to optimize the path planning and resource allocation of group communication, and the wavelength resource pre-allocation scheme is obtained.

[0175] In this embodiment, firstly, based on the physical topology and wavelength resource allocation of the optical fiber network, the system constructs a weighted undirected graph G=(V,E), where vertices V represent network nodes, edges E represent optical fiber links, and the weight of the edges represents the bandwidth capacity or reliability index of the link, outputting a network graph model. A weighted undirected graph is a mathematical model used to represent the connection relationships and attributes between nodes in a network. In optical fiber network modeling, each vertex represents a physical network device, such as an optical switching node, router, or computing server; each edge represents a direct optical fiber connection between two devices. Weight assignment uses a multi-attribute weighting method, comprehensively considering three key indicators: bandwidth capacity (the maximum data transmission rate that the link can provide), link reliability (a link stability index evaluated based on historical fault data and physical characteristics), and transmission delay (the time required for data to pass through the link). The model construction process includes network discovery (obtaining physical device and connection information through the network management system), topology mapping (mapping physical devices as vertices in the graph and physical connections as edges in the graph), and attribute collection (obtaining the performance parameters of each link). For wavelength division multiplexing (WDM) networks, the model is further extended to a multi-layer graph to represent the availability of different wavelength channels. The network graph model is represented using a standard graph data structure, supporting efficient path lookup and traffic analysis, and providing a foundation for subsequent Gomory-tree computation.

[0176] Secondly, for the network graph model, the system calculates its Gomory-Orlin tree T, retaining the minimum cut information between all node pairs in the graph, designs an incremental update mechanism, and outputs a network capacity analysis model. A Gomory-Orlin tree is a data structure used to efficiently represent the minimum cut information between all vertex pairs in an undirected graph. A minimum cut is the set of edges in a graph that removes the edge with the minimum total weight, making two specified vertices no longer connected. In traditional methods, calculating the minimum cut for all vertex pairs in a graph with n vertices requires O(n²) maximum flow algorithms; while the Gomory-Orlin tree constructs a tree that retains all minimum cut information by performing only n-1 maximum flow calculations. The construction algorithm uses a recursive approach: select two unprocessed vertices, calculate the minimum cut between them, divide the vertex set into two parts according to the cut, recursively process each part, and finally form a tree, where the minimum weight edge on the path between any two points in the tree corresponds to the minimum cut value between these two points in the original graph. The system implements a variant of the Hao-Orlin algorithm, significantly improving the computational efficiency of large-scale networks. Incremental update is a technique that selectively updates affected parts of a Gomory-tree when network topology or link states change, without recalculating the entire Gomory-tree. The system defines three key update operations: edge weight changes (adjusting affected paths in the tree), edge additions (merging affected subtrees), and edge deletions (recalculating the minimum cut of the affected region). The update algorithm employs a lazy evaluation strategy, batching multiple small changes to reduce computational overhead. The network capacity analysis model includes a Gomory-tree data structure, incremental update rules, and a query interface, providing efficient support for subsequent maximum traffic calculation.

[0177] Third, based on the network capacity analysis model, when it is necessary to find the maximum flow path between any two computing nodes, the system directly queries the minimum weight edge on the path between nodes in the Gomory-tree, calculates the maximum available bandwidth between key node pairs in the distributed computing task, and outputs the maximum flow information between nodes. Maximum flow refers to the maximum amount of data that can be transmitted simultaneously between two points in a network. According to the minimum cut theorem, the maximum flow is equal to the minimum cut. Traditional methods for calculating the maximum flow, such as the Ford-Fulkerson algorithm or the Dinic algorithm, have high complexity. However, by using the Gomory-tree, the maximum flow value between any two points can be queried in O(1) time, which significantly improves efficiency. The query process is very straightforward: find the unique path connecting the target node in the Gomory-tree, and the edge with the smallest weight on the path is the maximum flow value between these two points in the original graph. The system not only calculates the maximum flow value, but also stores the cutting information through the improved Gomory-tree, which can quickly determine the specific path combination that achieves the maximum flow. Critical node pair identification is based on distributed computing dependency graph analysis. The system identifies node pairs that frequently exchange data or transmit large amounts of data during the computation process and prioritizes calculating the maximum available bandwidth between these node pairs. Bandwidth requirement analysis considers three key factors: data transmission volume (the total amount of data that needs to be exchanged between nodes), time constraints (the maximum allowable time to complete data transmission), and reliability requirements (the requirements for link stability during transmission). Maximum traffic information between nodes is organized in matrix form, recording the maximum available bandwidth between all critical node pairs, the path combinations to achieve this bandwidth, and link utilization, providing a basis for subsequent resource allocation.

[0178] Fourth, based on the maximum traffic information between nodes and the bandwidth requirements of the computing task, an optimal backup path resource allocation strategy is formulated. For each pair of critical nodes, a set of wavelength resources that meets the corresponding minimum bandwidth requirements and minimizes total resource consumption is allocated, and an optimized backup path resource allocation scheme is output. Backup path resource allocation is a mechanism that can quickly switch to a pre-planned backup path to ensure data transmission continuity when the primary path fails. The allocation strategy is based on multi-objective optimization, balancing three key objectives: meeting bandwidth requirements (ensuring that the allocated resources meet the minimum bandwidth requirements of the application), minimizing resource consumption (using the fewest wavelength resources to meet the requirements), and maximizing path separation (ensuring that the primary and backup paths do not share links that may fail simultaneously). The system implements a Shared Backup Path Protection (SBPP) mechanism, allowing multiple primary paths to share backup resources, provided that these primary paths do not fail simultaneously. The resource allocation algorithm adopts a heuristic approach. First, it constructs a set of all candidate paths that meet the bandwidth requirements based on Gomory-tree information, and then uses an integer linear programming model to select the optimal path combination to minimize overall resource consumption. For wavelength resource allocation, the system adopts a hybrid strategy combining First-Fit and minimum load to balance resource utilization and load balancing. The optimized backup path resource allocation scheme includes a complete wavelength channel allocation plan, protection switching trigger conditions, and path priority definitions, providing efficient fault tolerance for the network.

[0179] Finally, based on the optimized backup path resource allocation scheme, and targeting multi-point communication modes in distributed computing, the network graph model is extended to a hypergraph. The Gomory-Hu tree extension algorithm on the hypergraph is applied to optimize path planning and resource allocation for group communication, resulting in a wavelength resource pre-allocation scheme. A hypergraph is a generalization of a graph, where a hyperedge can connect two or more vertices, representing relationships between multiple points. In distributed computing, multi-point communication such as broadcasting, multicasting, and aggregation are common data exchange modes, and traditional point-to-point optimization cannot effectively handle these scenarios. The system models multi-point communication modes as hyperedges in a hypergraph, with edge weights representing the bandwidth requirement or priority of that communication mode. The Gomory-Hu tree hypergraph extension uses a recursive reduction method, converting hyperedges into ordinary edge networks with special structures, and then applying a modified Gomory-Hu tree algorithm. The multi-point communication optimization strategy employs different methods depending on the communication type: for broadcasting, a minimum spanning tree is constructed; for multicasting, the Steiner tree algorithm is used; and for aggregation, a shortest path tree is designed. Resource allocation considers wavelength continuity constraints (using the same wavelength throughout the optical path) and wavelength conflict constraints (different wavelengths are used for different optical paths on the same link). A multi-tree protection mechanism is implemented, constructing multiple trees with different structures for critical multi-point communication, enabling rapid switching in the event of a main tree failure. The wavelength resource pre-allocation scheme integrates resource planning for point-to-point and multi-point communication, defining complete optical path establishment, wavelength allocation, and protection strategies to ensure high reliability of distributed computing in the event of fiber optic network failures.

[0180] In one embodiment, adaptive migration optimization of distributed computing tasks based on stochastic reuse includes: decomposing the migration optimization problem into sub-problems of node selection, path planning, resource allocation, and scheduling optimization based on the characteristics of the distributed computing task and the network state; designing a corresponding stochastic optimization algorithm framework for each type of sub-problem; outputting a set of stochastic algorithm models; introducing the concept of pseudo-independent algorithms based on the set of stochastic algorithm models; constructing a stochastic seed management system to ensure that stochasticity can be appropriately shared at different migration decision stages while maintaining independence, including stochastic pool design, stochastic mapping mechanism, and pseudo-independence evaluation; and outputting a stochastic management strategy based on the stochastic management strategy and the network state of the distributed computing task. For network status, an adaptive switching mechanism is designed to switch between full batch query and sample query. Sample query is prioritized in normal conditions, full batch query is switched for critical decisions, and a hybrid strategy is used in emergency situations, outputting an adaptive query strategy. Based on this adaptive query strategy, a stochastic reuse strategy across subproblems is implemented, including gradient reuse, sampling reuse, and Markov model reuse. An iterative convergence acceleration mechanism is designed, outputting an optimized migration decision scheme. Based on the optimized migration decision scheme, a dynamic trade-off mechanism between accuracy and efficiency is implemented. The degree of stochastic reuse, the full batch sampling ratio, and the iteration termination condition are automatically adjusted according to the severity of network faults and the available migration time window, outputting an optimized migration execution plan.

[0181] In this embodiment, firstly, based on the characteristics of distributed computing tasks and network conditions, the migration optimization problem is decomposed into sub-problems of node selection, path planning, resource allocation, and scheduling optimization. For each sub-problem, a corresponding stochastic optimization algorithm framework is designed, outputting a set of stochastic algorithm models. Distributed computing task migration is the process of transferring computing tasks from the affected area to a healthy area in the event of a network failure, involving multiple interrelated optimization sub-problems. Stochastic optimization algorithms are a class of methods that utilize random sampling and probabilistic models to solve complex optimization problems, particularly suitable for handling high-dimensional, nonlinear, and discrete space optimization. The node selection sub-problem employs a stochastic exploration method, using simulated annealing to search for the optimal combination of target nodes in the candidate node space. The algorithm explores extensively during the high-temperature phase and gradually concentrates on promising areas as the temperature decreases. The path planning sub-problem employs a stochastic path construction strategy, using ant colony optimization to construct multiple candidate paths on the network graph. The algorithm guides the search process towards high-quality paths through a virtual pheromone mechanism. The resource allocation sub-problem employs a stochastic fractional programming method, using particle swarm optimization to find the optimal allocation scheme that balances resource utilization and task performance in the resource space. The scheduling optimization subproblem employs a randomized sorting technique, using a genetic algorithm to evolve a task execution order that satisfies dependency constraints and minimizes completion time. These algorithms collectively constitute a set of randomized algorithm models. Each algorithm utilizes randomness to avoid local optima, improve search efficiency, and maintain the independence between models, laying the foundation for subsequent reuse of randomness.

[0182] Secondly, based on the set of randomized algorithm models, the concept of pseudo-independent algorithms is introduced, and a randomness seed management system is constructed to ensure that randomness can be appropriately shared at different migration decision stages while maintaining independence. This includes randomness pool design, randomness mapping mechanism, and pseudo-independence evaluation, outputting a randomness management strategy. A pseudo-independent algorithm refers to an algorithm instance that appears to run independently but actually shares a portion of the randomness base. Through a carefully designed random number generation mechanism, efficient utilization of computing resources is achieved while maintaining algorithm independence. The randomness seed management system is a central mechanism controlling the generation and allocation of random numbers. Its core is the complete lifecycle management of building, allocating, and recycling random seeds. The randomness pool is a pre-generated set of high-quality random numbers. The system adopts a hierarchical structure design: a global pool (basic randomness shared by all algorithms), a problem pool (randomness dedicated to specific sub-problem types), and an instance pool (randomness used by individual algorithm instances). The randomness mapping mechanism defines the mapping rules from the randomness pool to specific algorithms, using hash functions and offset techniques to ensure that algorithms obtain sufficient and non-overlapping randomness fragments. Pseudo-independence assessment uses statistical tests, such as correlation analysis and independence tests, to evaluate the degree of stochastic correlation between algorithms, ensuring that shared randomness does not lead to undue correlation in algorithmic behavior. The randomness management strategy defines in detail the complete process of randomness generation, allocation, use, and recycling, providing a unified randomness management framework for different transition stages and sub-problems, achieving a balance between computational efficiency and algorithmic independence.

[0183] Third, based on the randomness management strategy and the network state of the distributed computing task, the system designs an adaptive switching mechanism to switch between full batch query and sample query. In normal states, sample query is prioritized; for critical decisions, full batch query is used; and in emergency states, a hybrid strategy is employed, outputting an adaptive query strategy. Full batch query uses the complete dataset for computation, providing high-precision results but incurring high computational costs. Sample query uses a subset of data for computation, offering speed but lower accuracy. The adaptive switching mechanism dynamically selects the query method based on network state and decision importance, achieving a balance between accuracy and efficiency. The system defines three network states: normal state (network functionality is intact, no urgent migration needs), critical state (partial failure occurs, requiring precise decision-making), and emergency state (widespread failure, limited time window). State identification is based on multi-indicator fusion, comprehensively considering the scope of the failure, available time, and task urgency. In normal states, the system primarily uses sample query, with a sampling rate typically between 10% and 30%, ensuring sample representativeness through stratified sampling and importance sampling techniques. In critical states, the system switches to full batch query, using the complete dataset for computation, especially for critical decisions such as node selection and resource allocation. In emergency situations, the system employs a hybrid strategy: full-batch queries are used for decisions on the critical path, while more aggressive sample queries are used for other decisions, with a sampling rate as low as 5%. Decision importance assessment is based on sensitivity analysis to identify the decision points that have the greatest impact on the final migration outcome. The adaptive query strategy defines in detail the state transition conditions, query mode selection rules, and sampling parameter adjustment mechanisms, providing the system with the ability to flexibly adjust the calculation strategy according to the actual situation.

[0184] Fourth, based on an adaptive query strategy, a stochastic reuse strategy across subproblems is implemented, including gradient reuse, sampling reuse, and Markov model reuse. An iterative convergence acceleration mechanism is designed, and an optimization transfer decision scheme is output. Stochastic reuse is a technique for sharing and transferring random number sequences or random states among multiple related computational tasks, which can significantly reduce computational overhead and accelerate convergence. Gradient reuse is suitable for gradient-based stochastic optimization algorithms, reducing redundant computation by sharing gradient calculation results among similar problems. The system implements a gradient caching mechanism to store gradient information for commonly used solution space regions and determines reuse conditions through similarity evaluation. Sampling reuse shares sampling points and evaluation results in Monte Carlo methods, particularly suitable for computationally expensive evaluation functions. The system designs a sampling point library, retrieves and adapts historical sampling results based on problem characteristics, and corrects reuse biases through weighted adjustments. Markov model reuse shares state transition models in Markov process-based algorithms, accelerating the learning process for new problems. The system implements a state mapping function to uniformly transform the state spaces of different subproblems to a standard representation, achieving model transfer across problems. The iterative convergence acceleration mechanism accelerates algorithm convergence by intelligently combining historical search trajectories and current search information. The system employs techniques such as adaptive step size adjustment, dynamic neighborhood reduction, and early stopping conditions to dynamically adjust the search strategy based on convergence behavior. The optimized migration decision scheme integrates the optimal solutions to each sub-problem, ensuring overall coordination of decisions through consistency checks and conflict resolution, and providing detailed operational guidelines for migration execution.

[0185] Finally, based on the optimized migration decision-making scheme, the system implements a dynamic trade-off mechanism between accuracy and efficiency. According to the severity of network faults and the available migration time window, it automatically adjusts the degree of random reuse, the full batch sampling ratio, and the iteration termination condition, outputting an optimized migration execution plan. The dynamic trade-off mechanism is an adaptive control system that adjusts algorithm behavior in real time based on environmental constraints and performance requirements. The system defines an accuracy-efficiency curve to describe the quantitative relationship between algorithm performance under different parameter settings, and uses this to achieve precise trade-off control. Fault severity assessment uses a multi-level classification model, comprehensively considering the fault scope, the number of affected nodes, the degree of network connection interruption, and the expected recovery time, classifying faults into four levels: minor, moderate, severe, and catastrophic. Time window analysis, based on network protection mechanisms and application fault tolerance, estimates the time constraints available for migration decision-making and execution. Adjusting the degree of random reuse, by controlling the sharing ratio and reuse threshold, can increase the reuse rate to over 80% under extreme time pressure, significantly reducing computation time. The full-batch sampling ratio is dynamically set based on the time budget, ranging from 100% (full data) under sufficient time to 1% (extremely sparse sampling) under extreme conditions. The system uses a binary search method to quickly determine the optimal sampling rate. The iteration termination conditions are dynamically adjusted based on the convergence speed and time constraints, including three conditions: maximum number of iterations, minimum improvement threshold, and time limit. The optimized migration execution plan is an adaptive execution framework that includes basic decision-making schemes, parameter adjustment rules, exception handling strategies, and execution monitoring points. It can self-adjust according to the actual situation during execution to ensure optimal migration results under various constraints.

[0186] In one embodiment, the steps of preloading the computing environment on the target node, obtaining the timestamped task state snapshot set and reconstructing the execution context, redirecting the data communication path, initiating state consistency verification, and outputting task recovery confirmation information include: based on the migration execution plan and the migration target node allocation scheme, preloading the computing environment on the selected target node, including libraries, configuration files, and runtime dependencies, and outputting an environment ready state; based on the environment ready state and the cache deployment plan, obtaining the most recent snapshot and key intermediate results from the timestamped task state snapshot set from the distributed cache system, loading them into the target node memory, and outputting a data loading complete state; based on the data loading complete state, reconstructing the execution context of the computing task, including memory data structures, iteration states, and computing progress information, and outputting a context reconstruction complete state; based on the context reconstruction complete state, modifying the task scheduling information and data flow, redirecting subsequent computing and data communication paths to new nodes and network paths, and outputting a process redirection complete state; based on the process redirection complete state, initiating a verification program to check the consistency and correctness of the computing state, confirming the integrity of task recovery, and obtaining the task recovery confirmation information.

[0187] In this embodiment, firstly, based on the migration execution plan and the target node allocation scheme, the system preloads the computing environment on the selected target node, including libraries, configuration files, and runtime dependencies, and outputs an environment ready state. Computing environment preloading is the process of preparing the necessary software environment on the target node before the actual task migration. The system adopts a layered loading strategy, first deploying the basic environment layer (operating system settings, network configuration), then the middleware layer (distributed computing framework, message queue system), and finally the application layer (libraries and configurations required for specific computing tasks). Preloading employs a parallel pipeline mechanism, with multiple software components being transferred and installed simultaneously, significantly reducing environment preparation time. For frequently used environment configurations, the system maintains a pre-built image library, achieving second-level environment replication through lightweight container technology. An environment consistency checker ensures that the target environment matches the source environment's key parameters, including library versions, system settings, and network parameters. The environment ready state includes a detailed verification report confirming that all necessary components have been correctly loaded and passed basic functional tests.

[0188] Secondly, based on the environment readiness status and the cache deployment plan, the system retrieves the most recent snapshot and key intermediate results from the timestamped task state snapshot set in the distributed cache system, loads them into the target node's memory, and outputs a data loading completion status. A timestamped task state snapshot is a complete computational state captured at a specific point in time during the execution of a distributed computing task, including task progress, memory data, and intermediate computational results. The system employs incremental snapshot technology, storing only the data that has changed compared to the previous snapshot, reducing storage overhead. Snapshot retrieval uses a multi-level strategy: first, it locates the most recent complete snapshot, then applies subsequent incremental snapshots until the latest state. Data transmission uses a priority queue mechanism, loading first the core data necessary for task recovery, then data that may be used immediately, and finally data needed only under specific conditions. For large datasets, the system uses data sharding and parallel loading techniques, with multiple threads simultaneously acquiring different data shards. The data loading completion status includes detailed loading statistics and data integrity verification results.

[0189] Third, based on the data loading completion status, the system reconstructs the execution context of the computation task, including the in-memory data structure, iteration state, and computation progress information, and outputs the context reconstruction completion status. Execution context reconstruction is the process of converting loaded state data into an executable computation environment. The system implements a three-stage reconstruction process: data structure reconstruction (recreating the in-memory data structure based on metadata), state restoration (populating the reconstructed data structure with snapshot data), and pointer repair (updating all internal references and pointers). To handle platform differences, the system application environment adaptation layer dynamically adjusts the data representation and memory layout. Computation progress restoration uses a transaction model to ensure accurate recovery from the most recent consistency point. The dependencies of concurrent tasks are reconstructed through a dependency graph to ensure correct restoration of the synchronization state between tasks. The context reconstruction completion status includes the reconstruction verification results and a summary of the ready execution context.

[0190] Fourth, based on the context reconstruction completion status, the system modifies the task scheduling information and data flow direction, redirecting subsequent computation and data communication paths to new nodes and network paths, and outputs the process redirection completion status. Communication path redirection is the process of updating the routing tables and connection information of all relevant components in the distributed system, causing data flow to redirect to the new node. The system implements a two-phase commit protocol to ensure that all relevant nodes synchronously update routing information, avoiding data loss or duplication. For synchronous communication mode, the system establishes a temporary buffer to temporarily store messages during the transition. For stream processing tasks, the system implements a seamless switching mechanism, completing path switching while maintaining data order. The task scheduler updates the task allocation table and priority queue containing the new node, ensuring that subsequent tasks are correctly assigned to the new node. The process redirection completion status includes routing table update confirmation and initial communication test results.

[0191] Finally, based on the completed redirection status, a verification program is initiated to check the consistency and correctness of the computational state, confirm the completeness of task recovery, and obtain the task recovery confirmation information. State consistency verification is a process that uses multiple detection methods to confirm that the computational state after migration is consistent with that before migration. The system performs three levels of verification: hash verification (comparing the digest values ​​of key data structures), logical verification (performing specific verification calculations and comparing the results), and behavioral testing (simulating normal operation and monitoring responses). For uncertain computations, the system applies a fault-tolerant verification model, allowing state differences within an acceptable range. The verification result analyzer evaluates all test results, generating a comprehensive score and a detailed anomaly report. The task recovery confirmation information includes a verification result summary, a recovery quality score, and a list of exceptions that may require manual intervention, providing administrators with a clear recovery status report.

[0192] In one embodiment, the step of setting up a data change capture mechanism to record incremental data, formulating a state merging strategy, synchronizing the incremental data to the target node and performing consistency verification, and outputting a migration completion confirmation for the task recovery confirmation information includes: setting up a data change capture mechanism on the source node to record new data and state changes generated during the migration process, and outputting an incremental data record set; analyzing the data differences between the incremental data record set and the current computing state of the target node, formulating a state merging strategy to resolve potential conflicts, and outputting a state merging scheme; synchronizing the incremental data captured by the source node to the target node according to the state merging scheme, applying change operations to update the computing state, and outputting an incremental synchronization completion status; performing a data consistency verification based on the incremental synchronization completion status to verify the data consistency between the source node and the target node, and outputting a consistency verification report; and confirming that the task has been successfully recovered and reached a consistent state on the target node based on the consistency verification report, arranging resource reclamation and cleanup of the source node, and obtaining the migration completion confirmation.

[0193] In this embodiment, firstly, regarding the task recovery confirmation information, the system sets up a data change capture mechanism on the source node to record new data and state changes generated during the migration process, outputting an incremental data record set. Data Change Capture (DCC) is a technology for real-time monitoring and recording of data modifications, ensuring that data changes occurring on the source node are not lost during the computation task migration process. The system implements a three-layer capture architecture: storage layer capture (intercepting underlying storage operations and recording data block changes), transaction layer capture (monitoring database transactions and recording logical modifications), and application layer capture (inserting hook functions in application code to capture high-level operations). To ensure performance, the system adopts a lightweight log structure, recording only the changed data items, operation types, and timestamps, rather than complete data copies. Log compression uses differential coding and incremental compression algorithms to reduce storage overhead. To handle high-frequency changes, the system designs a batch merging mechanism, merging multiple consecutive modifications to the same data item into a single record. The incremental data record set is organized by transaction, with each transaction containing a set of atomic operations to ensure data consistency. For applications that process ultra-large-scale data, the system implements a sharding record strategy, which distributes change logs across multiple storage nodes to improve parallel processing capabilities.

[0194] Secondly, based on the incremental data record set and the current computational state of the target node, the system analyzes the data differences between them, formulates a state merging strategy to resolve potential conflicts, and outputs a state merging scheme. The state merging strategy is a systematic method for handling data differences between the source and target nodes, especially when both nodes may modify the same data items simultaneously. First, a difference analysis is performed, identifying all data items that need to be merged by comparing the incremental records with the target node's state. Conflict detection identifies potential write conflicts through version vector and timestamp comparisons. For different types of conflicts, the system applies different resolution strategies: a time-based strategy (last write wins), a priority-based strategy (source or target node takes precedence), and a semantic-based strategy (merging method is determined based on operation type and data meaning). For complex data structures, the system implements a three-way merging algorithm, comparing changes in the source and target nodes based on the common ancestor state. The merging decision tree automatically selects the best merging method based on data type, operation type, and business rules. The state merging scheme contains a detailed operation sequence describing how to apply source node changes to the target node while maintaining data consistency and business logic integrity.

[0195] Third, according to the state merging scheme, the system synchronizes the incremental data captured by the source node to the target node, updates the computational state by applying change operations, and outputs the incremental synchronization completion status. Incremental data synchronization is the process of transmitting and applying the change records collected by the source node to the target node. The system adopts a phased synchronization strategy: preparation phase (verifying the target node's preparation state), transmission phase (efficiently transmitting incremental data), and application phase (orderly executing change operations). To ensure transmission efficiency, the system implements adaptive compression and batch processing mechanisms, dynamically adjusting the data packet size and sending frequency according to network conditions. Change application adopts a transactional execution model, with each atomic change group committed as a single transaction, ensuring that intermediate states are not exposed. For change sequences with complex dependencies, the system constructs a dependency graph to ensure that changes are applied in the correct order. Parallel application technology allows changes without dependencies to be executed simultaneously, improving processing efficiency. Error handling during the synchronization process employs retry and rollback mechanisms, isolating and retrying failed operations, and rolling back to the previous consistent state when necessary. The incremental synchronization completion status includes detailed synchronization statistics, such as the number of successfully applied changes, processing time, and possible warning messages.

[0196] Fourth, based on the incremental synchronization completion status, the system performs data consistency verification to confirm the data consistency between the source node and the target node, and outputs a consistency verification report. Data consistency verification is the process of confirming the consistency of data states between the source node and the target node by comparing key data indicators. The system implements a multi-level verification framework: hash verification (comparing the encrypted hash values ​​of key data structures), sampling verification (randomly selecting data items for detailed comparison), aggregation checking (comparing statistical summaries such as record count, sum, and average), and integrity verification (checking the integrity of relationships and compliance with business rules). To handle large-scale datasets, the system adopts a hierarchical sampling strategy, performing full verification on key data and statistical sampling on secondary data. Incremental verification technology only verifies the most recently modified data, reducing verification overhead. The parallel verification engine distributes verification tasks to multiple processing units, accelerating the verification process. An error tolerance mechanism allows for acceptable small differences in non-critical data, avoiding overall migration failure due to minor inconsistencies. The consistency verification report includes a summary of the verification results, a consistency score, and a detailed list of inconsistencies (if any).

[0197] Finally, based on the consistency verification report, the system confirms that the task has been successfully restored and reached a consistent state on the target node, and arranges for the source node's resources to be reclaimed and cleaned up, thus obtaining the migration completion confirmation. Migration completion confirmation is the final stage of the migration process, confirming that the target node has completely taken over the computing tasks and released the source node's resources. The system first performs a final verification to confirm that there are no serious problems in the consistency verification report. The switchover control flow redirects all remaining requests to the target node, ensuring that no new requests are sent to the source node. Resource reclamation adopts a phased strategy: observation period (keeping the source node in a ready state but not processing new requests), soft cleanup (releasing non-critical resources such as cache and temporary storage), and hard cleanup (completely reclamating all computing and storage resources). The observation period typically lasts 15-30 minutes to ensure that the target node runs stably without anomalies. Audit logs record key events and decision points throughout the migration process, providing complete traceability. Migration completion confirmation includes a final status report, resource reclamation confirmation, and potential optimization suggestions, formally marking the successful completion of the migration process.

[0198] like Figure 2 As shown, the present invention also provides a distributed computing task migration system that is imperceptible under fiber optic network interruption, comprising:

[0199] System building module 10 is used to build a distributed task migration system. It outputs the system architecture by deploying a fault detection agent, a task monitor, a migration scheduler, a resource manager, and a cache manager.

[0200] The fault detection module 20 is used to collect network status indicators, construct a network topology map, analyze the status of optical fiber links and perform fault prediction based on the fault detection agent, and output a fault event report.

[0201] The task monitoring module 30 is used to obtain the running status of distributed computing tasks based on the task monitor, establish the mapping relationship between subtasks and computing nodes, analyze data dependency relationships to construct a directed dependency graph, generate task status snapshots, and output a set of task status snapshots with timestamps.

[0202] The resource pre-allocation management module 40 is used to analyze resource requirements, reserve spare resources, identify key intermediate results and formulate caching strategies based on the directed dependency graph and the network topology graph through the resource manager, deploy a distributed caching system through the cache manager, and output a resource pre-allocation scheme and a cache deployment plan.

[0203] The migration scheduling module 50 is used to analyze the scope of the fault impact through the migration scheduler based on the fault event report and the directed dependency graph, extract the affected computation subgraph, select the migration target node based on the resource pre-allocation scheme, and output the migration execution plan.

[0204] The migration and reconstruction module 60 is used to preload the computing environment on the target node based on the migration execution plan, obtain the set of timestamped task status snapshots and reconstruct the execution context, redirect the data communication path, start the status consistency verification, and output task recovery confirmation information.

[0205] The data synchronization module 70 is used to set up a data change capture mechanism to record incremental data for the task recovery confirmation information, formulate a state merging strategy, synchronize the incremental data to the target node and perform consistency verification, and output a migration completion confirmation.

[0206] Although the embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for seamless migration of distributed computing tasks under fiber optic network interruption, characterized in that, include: Build a distributed task migration system, and output the system architecture by deploying a fault detection agent, task monitor, migration scheduler, resource manager and cache manager; Based on the fault detection agent, network status indicators are collected, a network topology map is constructed, the status of optical fiber links is analyzed and fault prediction is performed, and a fault event report is output. Based on the task monitor, the running status of distributed computing tasks is acquired, a mapping relationship between subtasks and computing nodes is established, data dependencies are analyzed to construct a directed dependency graph, task status snapshots are generated, and a set of timestamped task status snapshots is output. This includes: acquiring the running status of distributed computing tasks in real time, including execution progress, resource consumption, and data processing volume, based on the task monitor, and outputting task running status data; analyzing the internal structure of the task based on the task running status data, dividing the overall task into multiple subtask units, establishing a mapping relationship between subtasks and physical computing nodes, and outputting a subtask distribution mapping table; analyzing the data dependencies between subtasks by tracking data flow and computational logic based on the subtask distribution mapping table, and constructing a directed dependency graph; applying a critical path analysis algorithm to the directed dependency graph to identify critical execution paths and performance bottlenecks, and outputting critical path analysis results; and generating task status snapshots containing computational context, intermediate results, and memory status based on the critical path analysis results and the task running status data, thus obtaining the set of timestamped task status snapshots. Based on the directed dependency graph and the network topology graph, the resource manager analyzes resource requirements, reserves spare resources, identifies key intermediate results to formulate caching strategies, deploys a distributed caching system through the cache manager, and outputs a resource pre-allocation scheme and a cache deployment plan. Based on the fault event report and the directed dependency graph, the migration scheduler analyzes the scope of the fault impact, extracts the affected computation subgraph, selects the migration target node based on the resource pre-allocation scheme, and outputs the migration execution plan. Based on the migration execution plan, the computing environment is preloaded on the target node, the set of timestamped task status snapshots is obtained and the execution context is reconstructed, the data communication path is redirected, the status consistency verification is started, and the task recovery confirmation information is output. For the task recovery confirmation information, a data change capture mechanism is set up to record incremental data, a state merging strategy is formulated, the incremental data is synchronized to the target node and consistency verification is performed, and a migration completion confirmation is output.

2. The method according to claim 1, characterized in that, The constructed distributed task migration system, through the deployment of a fault detection agent, task monitor, migration scheduler, resource manager, and cache manager, outputs a system architecture including: Deploy the fault detection agent, the task monitor, the migration scheduler, the resource manager, and the cache manager to establish the basic component architecture; Based on the aforementioned basic component architecture, a communication mechanism between components is constructed using a message bus and publish-subscribe pattern to enable collaborative work among components and output component communication protocols. Based on the aforementioned component communication protocol, it interfaces with the SDN controller and optical network management system to achieve optical network control plane integration and output network control interface; Based on the network control interface, a distributed computing framework adaptation module is designed to obtain the system architecture.

3. The method according to claim 1, characterized in that, The process, based on the fault detection agent, involves collecting network status indicators, constructing a network topology map, analyzing fiber optic link status, predicting faults, and outputting fault event reports, including: Based on the fault detection agent, network status indicators including optical power, bit error rate, latency, and jitter are collected in real time, and raw monitoring data is output. Based on the original monitoring data, a real-time network topology map is dynamically constructed using the link layer discovery protocol, and the network topology map is output. For the network topology, the health status of each optical fiber link is assessed using a multi-index threshold and health scoring algorithm, and the link health status assessment results are output. Based on the link health status assessment results, combined with historical fault data and machine learning models, fault prediction analysis is performed, and fault warning information is output. Based on the fault warning information and real-time monitoring data, when an optical fiber link interruption event is detected, key information including fault type, location, time, and switching path is recorded to obtain the fault event report.

4. The method according to claim 1, characterized in that, The process of analyzing resource requirements through the resource manager, reserving spare resources, identifying key intermediate results to formulate caching strategies, deploying a distributed caching system through the cache manager, and outputting a resource pre-allocation scheme and cache deployment plan includes: Based on the directed dependency graph and the network topology graph, analyze the bandwidth requirements and key data transmission paths between computing nodes, and output the network resource requirement analysis results. Based on the network resource demand analysis results, the resource manager reserves spare optical paths and wavelength resources in the physical network and outputs a wavelength resource pre-allocation scheme. Based on the directed dependency graph and the network topology graph, select suitable backup computing nodes and reserve computing resources, and output a computing resource reservation scheme; Based on the directed dependency graph, identify key intermediate results and state data in the computation process, formulate a multi-level caching strategy, and output the intermediate result caching strategy. Based on the wavelength resource pre-allocation scheme, the computing resource reservation scheme, and the intermediate result caching strategy, a distributed caching system is deployed on selected nodes through the cache manager to establish data version control and consistency protocols, thereby obtaining the resource pre-allocation scheme and the cache deployment plan.

5. The method according to claim 4, characterized in that, The process of analyzing the scope of the fault impact through the migration scheduler, extracting the affected computational subgraph, selecting the migration target node based on the resource pre-allocation scheme, and outputting the migration execution plan includes: Based on the fault event report, combined with the network topology diagram and the subtask distribution mapping table, the migration scheduler quickly analyzes the computing nodes and subtasks affected by the faulty link and outputs the fault impact analysis results. Based on the failure impact analysis results, the affected computational subgraphs are extracted from the directed dependency graph, including directly affected subtasks and indirectly dependent upstream and downstream tasks, and the computational subgraph model to be migrated is output. Based on the computing subgraph model to be migrated and the computing resource reservation scheme, the adaptability, current load and network connection status of each backup node are evaluated, the optimal combination of migration target nodes is selected, and the migration target node allocation scheme is output. Based on the criticality and dependency of each subtask in the computational subgraph model to be migrated, the migration operations are prioritized and a list of subtask migration priorities is output. Based on the migration target node allocation scheme and the subtask migration priority list, combined with the optical network protection switching time characteristics and the calculation task fault tolerance time threshold, the available migration time window is evaluated, a time-sensitive migration schedule is formulated, and the migration execution plan is obtained.

6. The method according to claim 5, characterized in that, Migration decision optimization based on matroid-constrained optimization algorithms includes: Based on the failure impact analysis results and the directed dependency graph, the distributed computing migration problem is formalized as a matte-constrained submodulus maximization problem. A matte M=(N,I) is constructed, where N is the set of all possible subtask-target node pairs and I is the set of feasible migration schemes that satisfy resource constraints and dependencies. The matte-constrained problem model is then output. Based on the aforementioned matroid constraint problem model, a fixed-parameter algorithm is designed to handle the offline computation of the migration planning stage and the dynamic response of the real-time migration execution stage, and outputs a fixed-parameter algorithm scheme. Based on the fixed parameter algorithm scheme, a dynamic adjustment mechanism for the migration scheme is established. When the network status or computing node status is updated, the migration decision is adjusted in real time, and a dynamically optimized migration scheme is output. Based on the aforementioned dynamic optimization migration scheme, a unified migration optimization algorithm is designed to handle different scenarios of independent and competitive tasks, and output a unified migration optimization strategy. Based on the unified migration optimization strategy, the theoretical guarantee and performance evaluation of the migration scheme are realized, and the optimized migration execution plan is obtained.

7. The method according to claim 4, characterized in that, The fault-tolerant path optimization mechanism for fiber optic networks based on the Gomory-Hu tree includes: Based on the physical topology and wavelength resource allocation of the optical fiber network, a weighted undirected graph G=(V,E) is constructed, where vertex V represents a network node, edge E represents an optical fiber link, and the weight of the edge represents the bandwidth capacity or reliability index of the link. The network graph model is then output. For the network graph model, calculate its Gomory-tree T, retain the minimum cut information between all node pairs in the graph, design an incremental update mechanism, and output the network capacity analysis model. According to the network capacity analysis model, when it is necessary to find the maximum traffic path between any two computing nodes, the minimum weight edge on the path between nodes in the Gomory-tree is directly queried, the maximum available bandwidth between key node pairs in the distributed computing task is calculated, and the maximum traffic information between nodes is output. Based on the maximum traffic information between the nodes and the bandwidth requirements of the computing task, an optimal backup path resource allocation strategy is formulated. For each pair of critical nodes, a set of wavelength resources that meets the corresponding minimum bandwidth requirements and minimizes total resource consumption is allocated, and an optimized backup path resource allocation scheme is output. Based on the optimized backup path resource allocation scheme, for the multi-point communication mode in distributed computing, the network graph model is extended into a hypergraph, and the Gomory-Hu tree extension algorithm on the hypergraph is applied to optimize the path planning and resource allocation of group communication, thus obtaining the wavelength resource pre-allocation scheme.

8. The method according to claim 5, characterized in that, Adaptive migration optimization for distributed computing tasks based on stochastic reuse includes: Based on the characteristics of distributed computing tasks and network conditions, the migration optimization problem is decomposed into sub-problems of node selection, path planning, resource allocation and scheduling optimization. For each type of sub-problem, a corresponding stochastic optimization algorithm framework is designed, and a set of stochastic algorithm models is output. Based on the set of random algorithm models, the concept of pseudo-independent algorithms is introduced to construct a randomness seed management system. This system ensures that randomness can be appropriately shared at different migration decision stages while maintaining independence. It includes randomness pool design, randomness mapping mechanism and pseudo-independence evaluation, and outputs a randomness management strategy. Based on the randomness management strategy and the network state of the distributed computing task, an adaptive switching mechanism is designed to switch between full batch query and sample query. In normal state, sample query is used first, critical decision is switched to full batch query, and a hybrid strategy is used in emergency state. The adaptive query strategy is output. Based on the adaptive query strategy, a stochastic reuse strategy across subproblems is implemented, including gradient reuse, sampling reuse and Markov model reuse. An iterative convergence acceleration mechanism is designed, and an optimized transfer decision scheme is output. Based on the optimized migration decision scheme, a dynamic trade-off mechanism between accuracy and efficiency is realized. According to the severity of network failure and the available migration time window, the degree of random reuse, the full batch sampling ratio and the iteration termination condition are automatically adjusted to output an optimized migration execution plan.

9. The method according to claim 5, characterized in that, The steps include preloading the computing environment on the target node, obtaining the set of timestamped task state snapshots and reconstructing the execution context, redirecting the data communication path, initiating state consistency verification, and outputting task recovery confirmation information, including: Based on the migration execution plan and the migration target node allocation scheme, the computing environment, including libraries, configuration files and runtime dependencies, is preloaded on the selected target node, and the environment is output as ready. Based on the environment ready state and the cache deployment plan, the most recent snapshot and key intermediate results in the timestamped task status snapshot set are obtained from the distributed cache system, loaded into the target node memory, and the data loading completed status is output. Based on the data loading completion status, reconstruct the execution context of the computing task, including the memory data structure, iteration status, and computing progress information, and output the context reconstruction completion status. Based on the context reconstruction completion status, modify the task scheduling information and data flow direction, redirect the subsequent calculation and data communication paths to the new node and network path, and output the process redirection completion status; Based on the completed process redirection status, a verification program is initiated to check the consistency and correctness of the calculation status, confirm the integrity of task recovery, and obtain the task recovery confirmation information.

10. The method according to claim 1, characterized in that, The process for confirming task recovery includes setting up a data change capture mechanism to record incremental data, formulating a state merging strategy, synchronizing incremental data to the target node and performing consistency checks, and outputting a migration completion confirmation, including: For the task recovery confirmation information, a data change capture mechanism is set on the source node to record new data and status changes generated during the migration process and output an incremental data record set; Based on the incremental data record set and the current computing state of the target node, analyze the data differences between the two, formulate a state merging strategy to resolve potential conflicts, and output a state merging scheme. According to the state merging scheme, the incremental data captured by the source node is synchronized to the target node, the change operation is applied to update the calculation state, and the incremental synchronization completion status is output. Based on the incremental synchronization completion status, perform data consistency verification to verify the data consistency between the source node and the target node, and output a consistency verification report. Based on the consistency verification report, it is confirmed that the task has been successfully restored and reached a consistent state on the target node. Resource reclamation and cleanup of the source node are arranged, and the migration completion is confirmed.

11. A distributed computing task migration system that is seamless under fiber optic network interruption, characterized in that, include: The system building module is used to build a distributed task migration system. It outputs the system architecture by deploying a fault detection agent, a task monitor, a migration scheduler, a resource manager, and a cache manager. The fault detection module is used to collect network status indicators, construct a network topology map, analyze the status of optical fiber links and perform fault prediction based on the fault detection agent, and output a fault event report. The task monitoring module is used to acquire the running status of distributed computing tasks based on the task monitor, establish the mapping relationship between subtasks and computing nodes, analyze data dependencies to construct a directed dependency graph, generate task status snapshots, and output a set of timestamped task status snapshots. This includes: acquiring the running status of distributed computing tasks in real time, including execution progress, resource consumption, and data processing volume, based on the task monitor, and outputting task running status data; analyzing the internal structure of the task based on the task running status data, dividing the overall task into multiple subtask units, establishing the mapping relationship between subtasks and physical computing nodes, and outputting a subtask distribution mapping table; analyzing the data dependencies between subtasks based on the subtask distribution mapping table by tracking data flow and computational logic, and constructing a directed dependency graph; applying a critical path analysis algorithm to the directed dependency graph to identify critical execution paths and performance bottlenecks, and outputting critical path analysis results; and generating task status snapshots containing computational context, intermediate results, and memory status based on the critical path analysis results and the task running status data, thus obtaining the set of timestamped task status snapshots. The resource pre-allocation management module is used to analyze resource requirements based on the directed dependency graph and the network topology graph, reserve spare resources, identify key intermediate results to formulate caching strategies, deploy a distributed caching system through the caching manager, and output a resource pre-allocation scheme and a caching deployment plan. The migration scheduling module is used to analyze the scope of the fault impact based on the fault event report and the directed dependency graph, extract the affected computation subgraph, select the migration target node based on the resource pre-allocation scheme, and output the migration execution plan. The migration and reconstruction module is used to preload the computing environment on the target node based on the migration execution plan, obtain the set of timestamped task status snapshots and reconstruct the execution context, redirect the data communication path, start the status consistency verification, and output task recovery confirmation information. The data synchronization module is used to set up a data change capture mechanism to record incremental data for the task recovery confirmation information, formulate a state merging strategy, synchronize the incremental data to the target node and perform consistency verification, and output a migration completion confirmation.

Images