Wavelength granularity task partitioning and scheduling method and system for large-scale graph computation

Abstract

The application provides a wavelength granularity task segmentation scheduling method and system for large-scale graph computation, comprising: obtaining a user-submitted graph computation task and network topology information, establishing a system architecture, and completing system initialization; performing feature analysis on the graph computation task; applying a multi-level graph segmentation algorithm to divide the graph data into multiple subgraph units; analyzing the available wavelength resource state in the optical network, establishing a mapping relationship between the subgraph communication demand and the wavelength resource; configuring an optical cross-connect device to establish a dedicated wavelength channel, and distributing the subgraph task to the corresponding computing node; and performing data exchange between the subgraphs through the dedicated wavelength channel, while performing parallel computation within the subgraphs. The application combines graph segmentation and wavelength resource allocation, realizes the collaborative optimization of the computation task and the communication resource, solves the performance bottleneck problem caused by the separation of computation and communication in the traditional distributed graph computation, and significantly improves the efficiency and scalability of large-scale graph computation.

Description

Technical Field

[0001] This invention relates to the fields of distributed computing and optical network communication technology, and in particular to a wavelength-granular task partitioning and scheduling method and system for large-scale graph computing, applicable to large-scale graph data processing and analysis scenarios, especially large-scale graph data processing and analysis scenarios in optical transmission-based computing networks. Background Technology

[0002] In the era of big data, graph computing has become an important technical means for processing large-scale, complex, and interconnected data, and is widely used in fields such as social network analysis, recommendation systems, knowledge graphs, and bioinformatics. With the exponential growth of graph data, single-machine systems and even single data centers can no longer meet the computing demands, making distributed graph computing the mainstream solution for large-scale graph processing problems.

[0003] Currently, mainstream distributed graph computing frameworks such as Pregel, GraphX, and PowerGraph typically employ edge or vertex partitioning strategies to distribute large-scale graph data across multiple computing nodes. These systems rely on traditional TCP / IP networks for inter-node communication, exchanging data through message passing mechanisms. For example, Pregel uses a Bulk Synchronous Parallel (BSP) computing model, performing computation and communication within each hyperstep, while GraphX, based on the Spark platform, implements more flexible graph parallel processing capabilities.

[0004] Existing distributed graph computing systems typically separate computation scheduling from network resource allocation. That is, after graph partitioning at the computation layer, communication relies on the best-effort service of traditional IP networks. This design leads to frequent data exchange between computing nodes when handling large-scale graph computations. Each communication requires processing through multiple layers of network protocol stacks, including packet encapsulation, decapsulation, buffering, routing, and forwarding. Furthermore, when multiple graph computation tasks execute simultaneously, network resource contention intensifies, resulting in communication congestion and unpredictable delays.

[0005] Traditional distributed graph computing methods face two main technical problems: First, the separation of computation and communication leads to low resource utilization efficiency, and graph partitioning does not consider the underlying network topology and resource status, making cross-node communication a performance bottleneck; Second, the packet switching mechanism of IP networks introduces a large amount of protocol processing overhead and queue latency, which is particularly unfavorable for the frequent small packet exchanges in graph computing, seriously affecting computational efficiency and scalability. Summary of the Invention

[0006] The purpose of this invention is to overcome the problems existing in the prior art and provide a wavelength-granular task partitioning and scheduling method and system for large-scale graph computing. By combining graph partitioning with wavelength resource scheduling, a direct mapping relationship between computing tasks and optical network resources is established, reducing communication overhead and improving the efficiency and scalability of large-scale graph computing.

[0007] To achieve the above objectives, this invention provides a wavelength-granularity task partitioning and scheduling method for large-scale graph computation, comprising:

[0008] The system acquires user-submitted graph computation tasks and network topology information, establishes a system architecture consisting of a task analysis module, a graph segmentation engine, a wavelength resource manager, and a scheduler, completes system initialization, and obtains an initialized system runtime environment. Based on the initialized system runtime environment, the system identifies the algorithm type, analyzes the communication mode, and evaluates the computational complexity of the user-submitted graph computation tasks, obtaining a graph computation task feature vector. According to the graph computation task feature vector, a multi-level graph segmentation algorithm is applied, considering the communication cost between nodes and the balance of computational load, to divide the large-scale graph data into multiple subgraph units, obtaining a set of subgraphs and their dependency matrix. Based on the set of subgraphs and their dependency matrix, the system analyzes the available wavelength resource status in the optical network, establishes a mapping relationship between subgraph communication requirements and wavelength resources, and obtains an optimal wavelength allocation scheme. For the optimal wavelength allocation scheme, an optical cross-connect device is configured to establish an end-to-end dedicated wavelength channel, distributing subgraph tasks to corresponding computing nodes, obtaining a ready computing environment. Based on the ready computing environment, data exchange between subgraphs is performed through the dedicated wavelength channel, while parallel computation within the subgraphs is executed simultaneously, obtaining computation results and performance statistics.

[0009] Furthermore, the process of acquiring user-submitted graph computing tasks and network topology information, establishing a system architecture consisting of a task analysis module, a graph segmentation engine, a wavelength resource manager, and a scheduler, completing system initialization, and obtaining the initialized system operating environment includes: performing system requirement analysis based on the characteristics of the graph computing scenario and the properties of optical networks, determining the system functional boundaries and performance targets, and obtaining a system requirement specification; based on the system requirement specification, designing the interface specifications of the task analysis module, graph segmentation engine, wavelength resource manager, and scheduler using modular design principles, and obtaining a system component design document; based on the system component design document, deploying each functional module in a distributed environment, completing system startup configuration and resource registration, and obtaining the initialized system operating environment.

[0010] Further, the step of identifying the algorithm type, analyzing the communication pattern, and evaluating the computational complexity of the graph computation task submitted by the user to obtain the graph computation task feature vector includes: analyzing the computational model and algorithm type based on the user-submitted graph computation task, identifying the computational characteristics of the graph computation task, and obtaining a task type identifier; analyzing the topological characteristics of the input graph data based on the task type identifier and the input graph data of the user-submitted graph computation task, identifying the structural features of the input graph data, and obtaining a graph structure feature vector; predicting the data exchange frequency and communication volume during the computation process based on the graph structure feature vector and the task type identifier, and obtaining a communication pattern description; estimating the computational, storage, and network resources required for task execution based on the communication pattern description and the scale of the input graph data, and obtaining a resource requirement vector; and generating a comprehensive description of task features through a multi-dimensional feature fusion algorithm based on the task type identifier, the graph structure feature vector, the communication pattern description, and the resource requirement vector, thus obtaining the graph computation task feature vector.

[0011] Furthermore, the multi-level graph segmentation algorithm considers the communication cost and computational load balancing between nodes, dividing large-scale graph data into multiple subgraph units to obtain a set of subgraphs and their dependency matrix. This includes: calculating task feature vectors based on the graph; selecting a graph segmentation strategy using a decision tree algorithm to obtain segmentation strategy parameters; generating an initial subgraph partition using a greedy allocation algorithm based on the segmentation strategy parameters and the original graph data to obtain an initial subgraph set; optimizing the connections between subgraphs using an iterative boundary node migration algorithm to reduce cross-subgraph communication costs, resulting in an optimized subgraph structure; adjusting the computational complexity distribution of each subgraph using a computational load estimation and rebalancing algorithm for the optimized subgraph structure to obtain a load-balanced set of subgraphs; and analyzing the data dependencies and communication requirements between subgraphs based on the load-balanced set of subgraphs to construct a subgraph communication matrix, thus obtaining the set of subgraphs and their dependency matrix.

[0012] Furthermore, the analysis of the available wavelength resource status in the optical network, establishing a mapping relationship between subgraph communication requirements and wavelength resources, and obtaining the optimal wavelength allocation scheme includes: identifying optical cross-connect devices and available wavelength resources in the network to obtain an optical network topology map; monitoring the wavelength occupancy on each optical link in real time for the optical network topology map to obtain a wavelength resource status table; estimating communication frequency and data volume based on the subgraph set and its dependency matrix, quantifying the communication requirements between subgraphs to obtain a subgraph communication requirement matrix; calculating the optimal subgraph-wavelength mapping relationship using a multi-objective optimization algorithm based on the wavelength resource status table and the subgraph communication requirement matrix to obtain a wavelength allocation scheme; and verifying the feasibility of the allocation results using a conflict detection algorithm based on the wavelength allocation scheme, adjusting for conflicts, and obtaining the optimal wavelength allocation scheme.

[0013] Furthermore, the multi-level graph segmentation algorithm employs an enhanced partition set coordination algorithm, including: calculating the task feature vector based on the graph, selecting the enhanced partition set coordination algorithm as the graph segmentation strategy, setting difference threshold parameters and coordination factors to obtain the parameter set of the enhanced partition set coordination algorithm; constructing the graph's correlation matrix and extracting key node features based on the original graph data and the parameter set of the enhanced partition set coordination algorithm, establishing a hierarchical representation of the graph, and obtaining the graph's feature representation and hierarchical structure; constructing a coordination tree structure by recursively applying the enhanced partition set coordination algorithm to obtain an initial coordination tree for the graph's feature representation and hierarchical structure; and calculating the initial coordination tree. The differences between nodes in the tree are measured to identify the optimal split point, resulting in a coordination tree with split labels. Based on the coordination tree with split labels, the connection characteristics of boundary nodes are analyzed, and a selective node replication strategy is adopted to optimize the cross-subgraph communication mode, resulting in an optimized subgraph set. Based on the optimized subgraph set, a communication cost model is established to evaluate the data exchange volume between subgraphs. Subgraph pairs with communication costs higher than a preset threshold are selectively merged to obtain a subgraph set that balances communication and computation. Based on the subgraph set that balances communication and computation, the data dependencies between subgraphs are analyzed, and a fine-grained dependency matrix is ​​constructed to obtain the subgraph set and its dependency matrix.

[0014] Furthermore, the recursive application of the enhanced partition set coordination algorithm to construct the coordination tree structure includes: recursively partitioning the graph structure starting from the root node based on the graph's feature representation and hierarchical structure, generating two child nodes each time to obtain a binary coordination tree; for each node in the binary coordination tree, calculating the ratio of internal connection density to external connection cost to obtain a node difference metric; comparing the node difference metric with a preset difference threshold parameter to determine whether to continue further partitioning the node, obtaining a partitioning decision; and based on the partitioning decision, continuing recursively until the difference metric of all leaf nodes is lower than the preset difference threshold parameter, thus obtaining the initial coordination tree.

[0015] Furthermore, the mapping relationship between subgraph communication requirements and wavelength resources is established using a locality-sensitive hashing (LSH) mechanism, including: designing an L-bit unique identifier encoding scheme based on the optical network topology and the wavelength resource status table, assigning a unique identifier to each available wavelength resource, and obtaining a wavelength resource identifier mapping table; mapping the communication requirement characteristics between subgraphs to a hash space using a family of LSH functions based on the subgraph set and its dependency matrix, constructing a hash representation of the communication requirements, and obtaining a subgraph communication requirement hash table; analyzing the synchronization status and message passing mode between computing nodes based on the system operating status and task scheduling mode, determining whether the current environment is synchronous or asynchronous, and obtaining an environment mode identifier; based on the environment mode identifier, using the LC-LSH algorithm when in synchronous mode and the LC-LSH4 algorithm when in asynchronous mode, combining the wavelength resource identifier mapping table and the subgraph communication requirement hash table, calculating the optimal subgraph and wavelength matching scheme, and obtaining a wavelength allocation scheme; based on the wavelength allocation scheme, identifying resource allocation conflicts using a conflict detection algorithm, and applying a priority-based conflict resolution strategy to obtain the optimal wavelength allocation scheme.

[0016] Furthermore, when the environment mode is identified as asynchronous mode, the method further includes: based on the preliminary LC-LSH4 algorithm allocation results, processing asymmetric communication requirements through the ASYM-LC-LSH4 algorithm to obtain an asymmetric optimization scheme; according to the asymmetric optimization scheme, applying the QR-LC-LSH4 algorithm to introduce quasi-randomness to guarantee a finite maximum intersection time, resulting in an optimized asynchronous environment wavelength allocation scheme; and evaluating key indicators including expected intersection time and maximum intersection time through a simulation model for the optimized asynchronous environment wavelength allocation scheme, resulting in a wavelength allocation scheme with performance indicators.

[0017] Furthermore, it also includes: based on the calculation results and performance statistics, continuously collecting system operating status through a distributed monitoring agent to obtain a system performance monitoring data stream; based on the system performance monitoring data stream, identifying performance bottlenecks in the system through anomaly detection and bottleneck identification algorithms to obtain a bottleneck analysis report; based on the bottleneck analysis report, generating adjustment strategies for the current state through an optimization strategy library and decision model to obtain an adjustment strategy instruction set; and based on the adjustment strategy instruction set, dynamically adjusting through wavelength resource reallocation algorithms and subgraph repartitioning algorithms to obtain optimized system performance indicators.

[0018] Furthermore, the fault tolerance mechanism based on local rerouting includes: pre-calculating multiple shortest backup paths for each optical link based on the optical network topology map and the wavelength resource status table, and storing the backup path information in the optical cross-connect devices at both ends of the link to obtain a link backup path table; setting up a multi-level fault detection mechanism at the optical layer and IP layer according to the link backup path table to achieve millisecond-level link fault perception and obtain a fault detection configuration; designing SRv6 segment routing strategies for various fault scenarios based on the fault detection configuration and the optimal wavelength allocation scheme, defining head node behavior and fault response rules to obtain an SRv6 routing strategy library; and, according to the SRv6 routing strategy library, switching to the backup path through the local rerouting mechanism to achieve fault recovery and obtain a fault tolerance performance report.

[0019] Furthermore, the local rerouting mechanism also includes: establishing a mapping relationship between optical link failures and the impact of subgraph communication based on the SRv6 routing policy library and the subgraph set and its dependency matrix, to obtain a fault response mapping table; designing a wavelength resource dynamic adjustment algorithm based on the fault response mapping table to reallocate available wavelength resources when a fault occurs, to obtain a wavelength resource dynamic adjustment strategy; and designing a checkpoint mechanism for the subgraph computation state based on the wavelength resource dynamic adjustment strategy to realize the periodic saving of the computation state and recovery after a fault, to obtain a computation state protection scheme.

[0020] Furthermore, the configuration of the optical cross-connect device to establish an end-to-end dedicated wavelength channel includes: based on the optimal wavelength allocation scheme and available computing resources, selecting the most suitable computing node for each subgraph using a matching algorithm to obtain the mapping relationship between the subgraph and the computing node; according to the mapping relationship between the subgraph and the computing node and the optimal wavelength allocation scheme, setting the cross-connection status of the optical cross-connect device by issuing configuration commands through the controller to obtain the configured optical path; and verifying the connectivity and bandwidth of the wavelength channel through end-to-end testing for the configured optical path to obtain a list of available wavelength channels.

[0021] Further, the data exchange between subgraphs via the dedicated wavelength channel includes: based on the ready computing environment, initiating computational processing within the subgraph on each node using a parallel computing framework, executing a local graph algorithm, and obtaining local computational results; exchanging boundary node data between subgraphs using a direct optical layer data transmission mechanism based on the local computational results and the available wavelength channel list, and obtaining updated boundary data; ensuring consistency of computation among subgraphs using a synchronization fence mechanism for the updated boundary data, controlling computation progress, and obtaining synchronization status information; managing the iteration process of computation using an iteration controller based on the synchronization status information, deciding whether to continue the next round of computation, and obtaining an iteration control signal; returning to the subgraph internal computational processing steps when the iteration control signal indicates continued execution, and entering the result integration stage when the iteration control signal indicates termination, and obtaining an iteration execution decision; merging the distributed computational results after the iteration execution decision indicates termination using a result integration algorithm, while collecting performance statistics, and obtaining the computational results and performance statistics.

[0022] This invention also provides a wavelength-granular task partitioning and scheduling system for large-scale graph computation, comprising: an architecture establishment module, used to acquire user-submitted graph computation tasks and network topology information, establish a system architecture consisting of a task analysis module, a graph partitioning engine, a wavelength resource manager, and a scheduler executor, complete system initialization, and obtain an initialized system operating environment; a task analysis module, used to perform algorithm type identification, communication mode analysis, and computational complexity evaluation on the user-submitted graph computation tasks based on the initialized system operating environment, and obtain graph computation task feature vectors; and a graph partitioning module, used to apply a multi-level graph partitioning algorithm considering the balance between inter-node communication costs and computational load based on the graph computation task feature vectors. The system divides large-scale graph data into multiple subgraph units, obtaining a set of subgraphs and their dependency matrix. A wavelength resource management module analyzes the available wavelength resources in the optical network based on the subgraph set and its dependency matrix, establishes a mapping relationship between subgraph communication needs and wavelength resources, and obtains the optimal wavelength allocation scheme. A scheduling and execution module configures optical cross-connect devices to establish end-to-end dedicated wavelength channels for the optimal wavelength allocation scheme, distributes subgraph tasks to corresponding computing nodes, and obtains a ready computing environment. A data exchange and computing module, based on the ready computing environment, performs data exchange between subgraphs through the dedicated wavelength channels, while simultaneously executing parallel computing within the subgraphs to obtain computation results and performance statistics.

[0023] The beneficial effects of this invention are as follows:

[0024] The wavelength-granular task partitioning and scheduling method and system for large-scale graph computing provided by this invention tightly integrates graph computing task partitioning with optical network wavelength resource scheduling, achieving collaborative optimization of computation and communication, and solving the problem that cross-node communication becomes a performance bottleneck in traditional distributed graph computing. By establishing a dedicated wavelength channel at the optical layer, the overhead of multi-layer protocol stack processing in traditional IP networks is avoided, effectively reducing communication latency and overhead. At the same time, a multi-level graph partitioning algorithm is adopted to consider the balance between inter-node communication costs and computational load, improving the overall efficiency of the system. In addition, this invention also provides dynamic optimization and fault recovery mechanisms, enhancing the adaptability and reliability of the system, making it particularly suitable for large-scale graph computing scenarios. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of the wavelength-granularity task segmentation and scheduling method for large-scale graph computation according to the present invention.

[0027] Figure 2 This is a structural diagram of the wavelength-granularity task segmentation and scheduling system for large-scale graph computation according to the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0029] Example 1:

[0030] like Figure 1As shown, this invention provides a wavelength-granular task segmentation and scheduling method for large-scale graph computing. The method includes acquiring user-submitted graph computing tasks and network topology information; establishing a system architecture consisting of a task analysis module, a graph segmentation engine, a wavelength resource manager, and a scheduler; completing system initialization to obtain an initialized system runtime environment; based on the initialized system runtime environment, performing algorithm type identification, communication mode analysis, and computational complexity evaluation on the user-submitted graph computing tasks to obtain graph computing task feature vectors; and applying a multi-level graph segmentation algorithm, considering inter-node communication costs and computational load balancing, to segment the large-scale graph data into multiple subgraph units, obtaining a set of subgraphs and their dependencies. The system is structured as follows: Based on the subgraph set and its dependency matrix, the available wavelength resources in the optical network are analyzed, and a mapping relationship between subgraph communication requirements and wavelength resources is established to obtain the optimal wavelength allocation scheme. For the optimal wavelength allocation scheme, optical cross-connect devices are configured to establish end-to-end dedicated wavelength channels, and subgraph tasks are distributed to corresponding computing nodes to obtain a ready computing environment. The dedicated wavelength channel is exclusively allocated to each subgraph pair and is used only for data transmission between the subgraph pairs during the allocated time period, and is not shared with other subgraph pairs. Based on the ready computing environment, data exchange between subgraphs is performed through the dedicated wavelength channel, and parallel computing within the subgraph is executed simultaneously to obtain computing results and performance statistics.

[0031] Step S1: This invention first obtains the graph computation task and network topology information submitted by the user. This information is the basic data for system initialization. Graph computation tasks refer to the algorithmic processing performed on graph data structures, including typical graph algorithms such as PageRank, community detection, and shortest path algorithms. These tasks are usually characterized by tight coupling between computation and communication. When submitting a task, the user needs to specify parameters such as algorithm type, graph data source location, computational accuracy requirements, and time constraints. Network topology information refers to the connection relationships and attributes of each node and link in the optical network, including the location distribution of optical cross-connect devices, the wavelength range supported by each device, link bandwidth capacity, and fiber distance. The system adopts a distributed architecture design and establishes four core functional modules: the task analysis module is responsible for extracting task features and predicting resource requirements; the graph partitioning engine realizes the optimal partitioning of graph data; the wavelength resource manager is responsible for wavelength resource status monitoring and allocation optimization; and the scheduler is responsible for task deployment and runtime management. The system initialization process includes loading configuration files, establishing communication channels between modules, connecting to the resource management database, and starting monitoring services. After initialization is completed, the system enters a ready state and can receive and process user graph computation tasks.

[0032] Step S2: After the system runtime environment is ready, perform multi-dimensional analysis on the graph computation tasks submitted by the user to provide a basis for subsequent segmentation and scheduling. Algorithm type identification refers to determining the type of graph algorithm by analyzing the task code or description, such as iterative algorithms (PageRank, trust propagation), exploratory algorithms (breadth-first search, depth-first search), or clustering algorithms (community detection, connected component analysis). Different types of algorithms have different computational and communication characteristics, requiring different optimization strategies. Communication pattern analysis refers to predicting the data exchange method during algorithm execution, including communication frequency (communication in every iteration or triggered by specific conditions), communication scope (point-to-point, local broadcast, or global broadcast), and communication data volume (boundary data or all states). This analysis usually combines algorithm type and historical execution data for inference. Computational complexity assessment refers to estimating the computational resources required for task execution, typically using a combination of time and space complexity methods, considering factors such as graph size (number of nodes and edges), algorithm characteristics, and number of iterations. The system integrates these analysis results into a graph computing task feature vector, which is a multi-dimensional data structure containing components such as task type, graph structure features, communication mode description, and resource requirement vector, providing a scientific basis for subsequent task segmentation and scheduling decisions.

[0033] Step S3: Based on the extracted graph calculation feature vector, apply a multi-level graph segmentation algorithm to divide the large-scale graph data into multiple subgraph units. Multi-level graph segmentation is an efficient graph segmentation technique, comprising three main stages: coarsening, initial segmentation, and refinement. In the coarsening stage, the algorithm gradually reduces the graph size by merging neighboring nodes, forming a hierarchical representation. In the initial segmentation stage, the coarsest-grained graph is initially partitioned. In the refinement stage, the algorithm maps the segmentation results back to the original fine-grained graph and optimizes the segmentation quality through boundary node migration. This invention introduces a dual-objective optimization mechanism of communication cost and computational load balancing based on standard multi-level graph segmentation. Communication cost refers to the amount of data that needs to be exchanged between subgraphs, usually proportional to the number of edges across subgraphs; computational load balancing refers to whether the computational load distribution of each subgraph is balanced, usually measured by the variance of the number of nodes or edges in the subgraph. The system continuously optimizes the subgraph partitioning through iterative boundary node migration and load rebalancing algorithms, ultimately forming a set of subgraphs with high cohesion (tight internal connections within subgraphs) and low coupling (sparse connections between subgraphs). Simultaneously, a dependency matrix between subgraphs is constructed, which describes the data dependencies and communication requirements between subgraphs, providing key input for subsequent wavelength resource allocation.

[0034] Step S4: After obtaining the subgraph set and its dependency matrix, analyze the wavelength resource status in the optical network and establish a mapping relationship between subgraph communication requirements and wavelength resources. Wavelength resources refer to different wavelength channels in the optical network that can be used for data transmission. Each wavelength channel can transmit data independently, and optical signals of different wavelengths can be transmitted in parallel in the same optical fiber without interfering with each other, which is the basis of wavelength division multiplexing (WDM) technology. First, by interacting with the network controller, obtain the optical network topology map and the real-time wavelength resource status table. Then, based on the subgraph dependency matrix, quantify the communication requirements between subgraphs to form a subgraph communication requirement matrix. Next, use a multi-objective optimization algorithm to calculate the optimal subgraph-wavelength mapping relationship. This optimization process considers multiple objectives, including minimizing communication latency, maximizing wavelength utilization, and balancing link load. The optimization algorithm can be a heuristic algorithm (such as genetic algorithm, simulated annealing) or an exact algorithm (such as integer linear programming), depending on the problem size and time constraints. After optimization, the system verifies the feasibility of the allocation results through a conflict detection algorithm to ensure that there are no wavelength resource conflicts (the same wavelength is simultaneously allocated to multiple conflicting communication requirements). In case of conflict, adjustments are made based on communication priority and resource availability to ultimately arrive at a feasible optimal wavelength allocation scheme.

[0035] Step S5: After obtaining the optimal wavelength allocation scheme, configure the optical cross-connect device to establish an end-to-end dedicated wavelength channel and distribute the subgraph tasks to the corresponding computing nodes.

[0036] Optical cross-connect (OXC) devices are key components in optical networks, enabling wavelength-based switching of optical signals between different ports, achieving optical layer exchange without photoelectric conversion. The system first selects the most suitable computing node for each subgraph, considering factors such as node computing power, memory capacity, network location, and the subgraph's computational requirements and communication mode. Then, the system sends configuration commands to each OXC device through the network controller, setting its cross-connect status and establishing end-to-end dedicated wavelength channels.

[0037] The most suitable computing node is selected through a resource matching algorithm that comprehensively considers the computing node's processing power, memory capacity, network location, and the computational requirements and communication patterns of the subgraph, thereby optimizing computational efficiency, communication efficiency, and load balancing. For example, when the system needs to select computing nodes for a subgraph containing 5 million nodes and 20 million edges, it first assesses the subgraph's computational requirements as needing an 8-core CPU and 16GB of memory, then filters the candidate nodes to find those that meet the hardware requirements. Assume node A has a 16-core CPU and 32GB of memory but a network latency of 20ms, node B has an 8-core CPU and 16GB of memory with a network latency of 5ms, and node C has a 12-core CPU and 24GB of memory with a network latency of 10ms. The system calculates a score for each node using a weighted scoring function, with computing power weighted at 0.4, memory weighted at 0.3, and network location weighted at 0.3. Ultimately, node C receives the highest overall score and is selected as the execution node for the subgraph.

[0038] These configuration instructions specify parameters such as input ports, output ports, and wavelengths to ensure that optical signals can be transmitted along the planned path. After configuration, the system verifies the connectivity and bandwidth of the wavelength channels through end-to-end testing to ensure that the channel quality meets requirements. Finally, the subgraph data and computation tasks are distributed to the corresponding computing nodes, including subgraph structure data, algorithm code, execution parameters, and communication configuration information. Through these steps, a ready computing environment is established, preparing for distributed graph computing.

[0039] Step S6: Based on the ready computing environment, data exchange between subgraphs is performed through dedicated wavelength channels, while parallel computation within the subgraphs is executed simultaneously. Parallel computing frameworks are software platforms that support distributed graph computing, such as Pregel and GraphX, capable of managing the execution and coordination of parallel tasks. First, computational processing within the subgraph is initiated on each computing node, executing local graph algorithms. During computation, when a node needs to access data from boundary nodes (belonging to other subgraphs), communication is achieved through a direct data transmission mechanism at the optical layer. This mechanism transmits data directly at the optical layer, avoiding the processing overhead of multi-layered protocol stacks in traditional IP networks and significantly reducing communication latency. The synchronization fence mechanism is a synchronization tool in distributed computing, requiring all computing nodes to reach a certain execution point before proceeding to the next step, ensuring computational consistency. This mechanism is used to control the computation progress and prevent inconsistencies caused by some nodes being too fast or too slow. The iteration controller manages the iteration process of the computation based on the algorithm type and convergence conditions, deciding whether to continue to the next round of computation. When the computation converges or reaches the maximum number of iterations, the result integration phase begins, merging the computation results of each subgraph to form the final output. Simultaneously, performance statistics are collected, including metrics such as computation time, communication latency, and resource utilization, providing a basis for performance analysis and optimization. Through this close collaboration between computation and communication, the system significantly improves the efficiency and scalability of large-scale graph computation.

[0040] Example 2:

[0041] In this embodiment, the process of obtaining user-submitted graph computing tasks and network topology information, establishing a system architecture consisting of a task analysis module, a graph segmentation engine, a wavelength resource manager, and a scheduler, completing system initialization, and obtaining the initialized system operating environment includes: performing system requirement analysis based on the characteristics of the graph computing scenario and the properties of optical networks to determine the system functional boundaries and performance targets, and obtaining a system requirement specification; designing the interface specifications of the task analysis module, graph segmentation engine, wavelength resource manager, and scheduler based on the system requirement specification using modular design principles, and obtaining a system component design document; and deploying each functional module in a distributed environment based on the system component design document, completing system startup configuration and resource registration, and obtaining the initialized system operating environment.

[0042] Step S1.1: The first step in system initialization is to conduct system requirements analysis, which is a process of deeply understanding the problem domain and clarifying the system boundaries. Graph computing scenarios have several significant characteristics: First, the data scale is large; modern graph datasets typically contain billions of nodes and hundreds of billions of edges, far exceeding the processing capacity of a single machine. Second, computation and communication are tightly coupled; node states are frequently updated and propagated during the execution of graph algorithms, resulting in complex communication modes. Third, iterative computation is a key feature; many graph algorithms, such as PageRank and trust propagation, require multiple iterations until convergence. Regarding optical network characteristics, wavelength division multiplexing (WDM) technology allows the simultaneous transmission of multiple optical signals of different wavelengths over a single optical fiber, with each wavelength providing bandwidths up to 100Gbps or even higher. Optical cross-connect (OXC) devices can directly switch between different wavelengths of optical signals at the optical layer, avoiding photoelectric conversion overhead. Wavelength resources are limited and discrete; a typical WDM system supports 40-80 wavelength channels. Based on these characteristics, several key functional boundaries were determined: supporting automatic identification of graph algorithm types and communication modes; implementing graph data partitioning that considers communication costs; providing fine-grained allocation and management of wavelength resources; and ensuring collaborative optimization of computation and communication. Regarding performance targets, quantitative indicators were defined, including the processing graph size (number of nodes and edges), computational speedup, communication latency, resource utilization, and system scalability. These analyses and definitions form the system requirements specification, serving as the basis for subsequent design.

[0043] Step S1.2: Based on the system requirements specification, adopt the modular design principle to design the interface specifications for each core functional module. Modular design is a software architecture method that decomposes the system into functionally independent modules, and achieves interaction between modules through clearly defined interfaces, thereby improving the maintainability and scalability of the system. The interfaces of the task analysis module include a task submission interface (receiving user input of graph computation task descriptions and parameters), a graph data interface (accessing and analyzing the structural features of graph data), and a feature extraction interface (generating and outputting task feature vectors). The interfaces of the graph segmentation engine include a segmentation strategy interface (selecting and configuring segmentation algorithms), a segmentation execution interface (performing graph data segmentation), and a result output interface (providing subgraph sets and dependencies). The interfaces of the wavelength resource manager include a resource discovery interface (obtaining network topology and wavelength status), a resource allocation interface (compiling and outputting wavelength allocation schemes), and a configuration distribution interface (distributing configurations to network devices). The interfaces of the scheduler executor include a task distribution interface (distributing subgraph tasks to computation nodes), an execution control interface (managing the computation process), and a result collection interface (integrating computation results). The modules exchange information through standardized data structures and APIs, such as task feature vectors, subgraph sets, dependency matrices, and wavelength allocation schemes. These interface specifications ensure low coupling and high cohesion between modules, facilitating parallel development and testing by different teams and forming system component design documents.

[0044] Step S1.3: Based on the system component design document, deploy each functional module in a distributed environment. A distributed environment refers to a network system composed of multiple computers that work collaboratively to complete tasks. The deployment process first determines the deployment location of each module: the task analysis module is deployed on the control node, responsible for receiving user tasks and performing preliminary analysis; the graph segmentation engine is typically deployed on dedicated nodes with high computing power and large memory to handle the segmentation calculation of large-scale graph data. A dedicated node is a computing node allocated for executing a specific subgraph calculation, and its computing resources are dedicated to processing the calculation task of that subgraph during task execution; the wavelength resource manager needs to be integrated with the network controller and deployed on the network management node to acquire and control network resources in real time; the scheduler executor adopts a master-slave architecture, with the master part deployed on the control node and the slave parts deployed on each computing node to coordinate task execution. After deployment, the system performs startup configuration, including loading configuration files, setting running parameters, and establishing communication channels between modules. Resource registration refers to each functional module registering its capabilities and status with the system, such as computing nodes registering their processing power and memory capacity, and network devices registering their wavelength resources and connection status. Through these steps, a working distributed architecture is established, enabling the modules to work together to form a fully initialized system operating environment, providing a foundation for subsequent task processing.

[0045] Example 3:

[0046] In this embodiment, the step of identifying the algorithm type, analyzing the communication pattern, and evaluating the computational complexity of the graph computation task submitted by the user to obtain a graph computation task feature vector includes: analyzing the computational model and algorithm type based on the user-submitted graph computation task, identifying the computational characteristics of the graph computation task, and obtaining a task type identifier; analyzing the topological characteristics of the input graph data based on the task type identifier and the input graph data of the user-submitted graph computation task, identifying the structural features of the input graph data, and obtaining a graph structure feature vector; predicting the data exchange frequency and communication volume during the computation process based on the graph structure feature vector and the task type identifier, and obtaining a communication pattern description; estimating the computational, storage, and network resources required for task execution based on the communication pattern description and the scale of the input graph data, and obtaining a resource requirement vector; and generating a comprehensive description of task features using a multi-dimensional feature fusion algorithm based on the task type identifier, the graph structure feature vector, the communication pattern description, and the resource requirement vector, thus obtaining the graph computation task feature vector.

[0047] Step S2.1: In the task analysis phase, the computational model and algorithm type of the graph computation task submitted by the user are analyzed first. The computational model refers to the abstract representation of graph computation, mainly including vertex-centric models, edge-centric models, and graph-centric models. Vertex-centric models (such as Pregel) organize computation as vertex programs, with each vertex performing computation independently and communicating with neighbors through message passing; edge-centric models (such as X-Stream) organize computation around edges, suitable for handling sparse graphs; graph-centric models (such as PowerGraph) allow operations on the global view of the graph, suitable for intensive computation. Algorithm type analysis identifies which category of classic graph algorithms it belongs to by examining the task code or description, such as path search algorithms (breadth-first search, depth-first search, shortest path), iterative convergence algorithms (PageRank, trust propagation, collaborative filtering), clustering analysis algorithms (connected component analysis, community detection, graph clustering), and centrality analysis algorithms (degree centrality, betweenness centrality, proximity centrality), etc. Different algorithm types have different computational characteristics, such as the number of iterations, convergence speed, and locality features. The system identifies algorithm types using techniques such as decision trees or pattern matching, and combines this with information on similar algorithms in the historical execution database to obtain a task type identifier. This identifier is structured data containing information such as the computational model, algorithm category, and iterative characteristics, providing a foundation for subsequent analysis.

[0048] Step S2.2: Based on the task type identifier and input graph data, analyze the topological characteristics of the graph data and identify its structural features. The topological characteristics of a graph refer to its structural attributes, which significantly impact computational performance. The system first calculates basic statistical indicators, including the number of nodes, number of edges, average degree, maximum degree, degree distribution, and graph density. Graph density is the ratio of the number of edges to the maximum possible number of edges, reflecting the sparseness or density of the graph. Then, the system analyzes high-level graph features, such as diameter (the maximum value of the shortest path between any two points), clustering coefficient (reflecting the tightness of connections between neighboring nodes), community structure (detecting dense subgraphs or modules in the graph), and power-law distribution characteristics (checking whether the degree distribution conforms to a power law). For large-scale graphs, sampling or approximation algorithms are used to quickly estimate these features. For example, Monte Carlo sampling is used to estimate the diameter, and local clustering coefficients are used to approximate global clustering coefficients. Furthermore, the system analyzes the dynamic characteristics of the graph, such as time-varying characteristics (whether the graph structure changes over time) and update frequency (the frequency and pattern of graph data updates). These analyses yield graph structure feature vectors, which contain multidimensional feature descriptions of the graph topology, providing a basis for communication pattern prediction and resource demand estimation.

[0049] Step S2.3: Based on the graph structure feature vector and task type identifier, predict the data exchange frequency and communication volume during the computation process to form a communication pattern description. The communication pattern refers to the method and characteristics of data exchange between nodes during graph computation, which has a decisive impact on system performance. The system first analyzes the communication frequency, i.e., the timing and frequency of data exchange. For iterative algorithms (such as PageRank), global synchronization is usually required after each iteration; for asynchronous algorithms (such as asynchronous stochastic gradient descent), communication may occur at any time; for exploratory algorithms (such as breadth-first search), communication occurs as the search front expands. Then, the system analyzes the communication scope, i.e., the spatial distribution of data exchange. Communication may be point-to-point (such as path updates in shortest path algorithms), local broadcasts (such as community merging in community detection), or global broadcasts (such as global convergence checks in PageRank). Next, estimate the communication data volume, i.e., the size of data transmitted in each communication. This depends on the algorithm type, graph structure, and data representation. For example, PageRank transmits a single floating-point number in each communication; while graph matching algorithms may need to transmit complex subgraph structures. Finally, communication dependencies, i.e., the impact of communication on computational progress, are analyzed. Some algorithms (such as the BSP model) require strict synchronization, while others may allow a degree of asynchronous communication. These analyses generate a communication pattern description that details the data exchange characteristics during computation, providing crucial input for wavelength resource allocation.

[0050] Step S2.4: Based on the communication pattern description and the scale of the input graph data, estimate the computational, storage, and network resources required for task execution, and generate a resource requirement vector. Resource requirement estimation is a crucial basis for task scheduling and influences the system's resource allocation decisions. Computational resource requirements are estimated based on algorithm complexity and graph size. For algorithms with a time complexity of O(V+E) (such as breadth-first search), the computational cost is proportional to the number of nodes V and edges E; for algorithms with a complexity of O(V²) (such as some pairwise shortest path algorithms), the computational cost is proportional to the square of the number of nodes. The system combines historical execution data and uses regression models or performance predictors to estimate the actual execution time. Storage resource requirements include graph data storage and temporary storage during computation. Graph data storage is directly related to the graph size, while temporary storage depends on the algorithm characteristics. For example, full graph analysis algorithms may require storing intermediate results proportional to the number of nodes. Network resource requirements are estimated based on the communication pattern description, including bandwidth requirements and latency requirements. Bandwidth requirements are related to the amount and frequency of communication data; latency requirements are related to the synchronization characteristics of the algorithm, with strictly synchronized algorithms being more sensitive to network latency. The system integrates these requirements to form a resource requirement vector, which includes multiple dimensions such as the number of CPU cores, memory capacity, bandwidth requirements, and latency requirements, providing comprehensive guidance for resource allocation.

[0051] Step S2.5: Finally, based on the task type identifier, graph structure feature vector, communication mode description, and resource requirement vector, a comprehensive description of task features is generated using a multi-dimensional feature fusion algorithm, resulting in a graph computing task feature vector. Multi-dimensional feature fusion refers to integrating features of different dimensions and scales into a unified representation, facilitating subsequent processing and decision-making. Weighted fusion or feature embedding techniques are used to integrate the various features obtained from the aforementioned analysis. Weighted fusion assigns weights based on the degree of influence of different features on segmentation and scheduling decisions, forming a weighted combination; feature embedding uses dimensionality reduction techniques (such as principal component analysis and autoencoders) to map high-dimensional features to a low-dimensional space, preserving key information. The fusion process also considers the correlation and complementarity between features to avoid redundancy or contradictions. The final graph computing task feature vector is a structured data object containing various features of the task, such as computational characteristics (algorithm type, iteration characteristics), data characteristics (graph size, topology), communication characteristics (frequency, range, data volume), and resource requirements (computation, storage, network). This feature vector provides a comprehensive and accurate basis for subsequent graph segmentation and resource allocation, and is a key input for achieving efficient task scheduling.

[0052] Example 4:

[0053] In this embodiment, the multi-level graph segmentation algorithm considers the communication cost between nodes and the balance of computational load, dividing large-scale graph data into multiple subgraph units to obtain a set of subgraphs and their dependency matrix. This includes: calculating task feature vectors based on the graph; selecting a graph segmentation strategy using a decision tree algorithm to obtain segmentation strategy parameters; generating an initial subgraph partition using a greedy allocation algorithm based on the segmentation strategy parameters and the original graph data to obtain an initial subgraph set; optimizing the connection relationships between subgraphs using an iterative boundary node migration algorithm to reduce cross-subgraph communication costs, resulting in an optimized subgraph structure; adjusting the computational complexity distribution of each subgraph using a computational load estimation and rebalancing algorithm for the optimized subgraph structure to obtain a load-balanced set of subgraphs; and analyzing the data dependencies and communication requirements between subgraphs based on the load-balanced set of subgraphs to construct a subgraph communication matrix, thus obtaining the set of subgraphs and their dependency matrix.

[0054] Step S3.1: In the graph segmentation stage, the task feature vector is first calculated based on the graph, and a suitable graph segmentation strategy is selected using the decision tree algorithm. The decision tree algorithm is a supervised learning method that classifies samples into different leaf nodes through a series of conditional judgments, with each leaf node corresponding to a classification result. In this system, the feature nodes of the decision tree include graph structure features (such as graph density, degree distribution, and clustering coefficients), algorithm type (such as iterative, exploratory, and clustering), and communication mode (such as peer-to-peer, local broadcast, and global broadcast). Based on these features, the system maps the task to the most suitable segmentation strategy, mainly including three types: edge-cut, vertex-cut, and hybrid-cut. Edge partitioning distributes graph nodes to different subgraphs, severing boundary edges (edges connecting nodes in different subgraphs). This strategy is suitable for graphs with many node attributes or uniform degree distribution. Vertex partitioning distributes graph edges to different subgraphs, copying shared vertices across multiple subgraphs. This strategy is suitable for graphs with power-law distributions (a few nodes connected to a large number of edges). Hybrid partitioning dynamically selects the partitioning method based on node degree, using vertex partitioning for high-degree nodes and edge partitioning for low-degree nodes, balancing communication costs and computational load. In addition to partitioning types, the system also determines partitioning strategy parameters such as the number of partitions (number of subgraphs), balance constraints (allowed range of subgraph size differences), and boundary optimization parameters (controlling the intensity of boundary optimization). These parameters directly affect partitioning quality and system performance. The system adjusts parameter values ​​based on historical execution data and performance models, outputting the optimal set of partitioning strategy parameters.

[0055] Step S3.2: Based on the determined partitioning strategy parameters and the original graph data, an initial subgraph partition is generated using a greedy allocation algorithm. The greedy allocation algorithm is a locally optimal selection strategy, making the best choice at each step without considering the global optimum. In graph partitioning, greedy allocation typically starts with a set of seed nodes and gradually assigns adjacent nodes to the subgraph. Seed node selection is a crucial step in the algorithm; common methods include random selection, selecting height nodes, or selecting key nodes in the graph (such as nodes with high centrality). Combining breadth-first search (BFS) or depth-first search (DFS) strategies, the subgraph is expanded from the seed nodes. In edge partitioning mode, nodes closely connected to nodes within the subgraph are prioritized for allocation to the same subgraph; in vertex partitioning mode, edges associated with edges within the subgraph are prioritized for allocation to the same subgraph. During the allocation process, the system tracks the size of each subgraph in real time to ensure balance constraints are met. To handle large-scale graph data, stream processing technology is used, eliminating the need to load the entire graph into memory and processing the graph data in a streaming manner, thus reducing memory consumption. Furthermore, the system employs a parallel allocation strategy, with multiple subgraphs expanding simultaneously from different seed nodes, accelerating the partitioning process. Through these techniques, the system quickly generates initial subgraph partitions, forming an initial subgraph set, which provides a foundation for subsequent optimization.

[0056] Step S3.3: Based on the initial subgraph set, optimize the connectivity between subgraphs using an iterative boundary node migration algorithm to reduce cross-subgraph communication costs. Boundary nodes are nodes connected to multiple subgraphs; these nodes are key points for inter-subgraph communication. The iterative boundary node migration algorithm is a local search optimization technique that improves segmentation quality by repeatedly moving boundary nodes. The algorithm first identifies all boundary nodes and, for each boundary node, calculates the benefit of moving it from the current subgraph to an adjacent subgraph. The benefit is typically defined as the weighted sum of the reduction in the number of cross-subgraph edges and the increase in subgraph imbalance after the move. The system selects the move operation with the highest benefit, then updates the boundary node set and the benefit value, and continues to the next round of migration. To avoid local optima, metaheuristic algorithms such as simulated annealing or tabu search are used, allowing for a certain probability of accepting unfavorable moves to explore a larger solution space. The iterative process continues until termination conditions are met, such as reaching the maximum number of iterations, no improvement for multiple consecutive rounds, or an improvement margin below a threshold. To accelerate the optimization process, a parallel boundary optimization strategy is adopted, dividing the graph into multiple non-overlapping regions, with boundary node migrations performed in parallel in each region. Furthermore, a multi-level optimization technique is employed, first optimizing the coarse-grained graph and then mapping the results back to the fine-grained graph for further optimization. These optimization techniques effectively reduce the communication cost across subgraphs, resulting in an optimized subgraph structure.

[0057] Step S3.4: For the optimized subgraph structure, the computational complexity distribution of each subgraph is adjusted through load estimation and rebalancing algorithms to obtain a load-balanced set of subgraphs. Load estimation refers to predicting the computational workload of each subgraph during actual execution, which is closely related to the subgraph's size (number of nodes and edges), structural characteristics (such as density and clustering degree), and algorithm characteristics. A load prediction model is constructed based on task type and historical execution data, and computational load weights are assigned to each subgraph. The rebalancing algorithm is a technique for balancing computational load by adjusting subgraph boundaries. First, the subgraph pairs with the largest computational load differences are identified, and nodes or edges are migrated from the high-load subgraph to the low-load subgraph to reduce load imbalance. Migration selection is based on multi-objective optimization principles, aiming to reduce load differences while maintaining low communication costs. In specific implementations, techniques such as diffusion redistribution or stream computing are used to achieve efficient rebalancing. Diffusion redistribution simulates the process of load spreading from high-load areas to low-load areas; stream computing transforms the load balancing problem into a network flow problem, solved using the minimum-cost maximum-flow algorithm. Furthermore, considering the characteristics of heterogeneous computing environments, nodes with higher computing power are allowed to be allocated more load, achieving weighted load balancing. The rebalancing process is iterative, reassessing the load distribution after each migration until the balance requirements are met or the maximum number of iterations is reached. Through these techniques, a set of load-balanced subgraphs is obtained, ensuring the efficient execution of distributed computing.

[0058] Step S3.5: Finally, based on the load-balanced subgraph set, the system analyzes the data dependencies and communication requirements between subgraphs, constructs a subgraph communication matrix, and obtains the subgraph set and its dependency matrix. Data dependency refers to the fact that the computation of one subgraph requires data from another subgraph; this dependency determines the communication requirements between subgraphs. The system first identifies all cross-subgraph edges (edges connecting different subgraphs), which represent data dependencies. For each pair of subgraphs with cross edges (Si, Sj), the system calculates the communication volume between them, which is usually proportional to the number of cross edges, but also considers algorithm characteristics and data representation. For example, in the PageRank algorithm, each cross edge transmits a single ranking value; while in graph matching algorithms, complex structural information may need to be transmitted. In addition, the system analyzes the communication frequency, i.e., the number of times data needs to be exchanged between subgraphs during the computation process. For iterative algorithms, the communication frequency is usually related to the number of iterations; for exploratory algorithms, it may be related to the search depth or graph diameter. The system also considers the directionality of communication, distinguishing between unidirectional communication (such as parent-child node communication in tree algorithms) and bidirectional communication (such as neighbor exchange in iterative algorithms). Finally, the system integrates this information into a subgraph communication matrix, which is an n×n matrix (where n is the number of subgraphs). Each element Cij represents the communication requirement from subgraph Si to Sj, including information such as communication volume, frequency, and direction. This matrix, together with the subgraph set, forms a complete subgraph set and its dependency matrix, providing crucial input for subsequent wavelength resource allocation.

[0059] Example 5:

[0060] In this embodiment, the analysis of available wavelength resources in the optical network, establishing a mapping relationship between subgraph communication requirements and wavelength resources, and obtaining the optimal wavelength allocation scheme includes: identifying optical cross-connect devices and available wavelength resources in the network to obtain an optical network topology map; monitoring the wavelength occupancy on each optical link in real time for the optical network topology map to obtain a wavelength resource status table; estimating communication frequency and data volume based on the subgraph set and its dependency matrix, quantifying the communication requirements between subgraphs to obtain a subgraph communication requirement matrix; calculating the optimal subgraph-wavelength mapping relationship using a multi-objective optimization algorithm based on the wavelength resource status table and the subgraph communication requirement matrix to obtain a wavelength allocation scheme; and verifying the feasibility of the allocation results using a conflict detection algorithm based on the wavelength allocation scheme, adjusting for conflicts, and obtaining the optimal wavelength allocation scheme.

[0061] Step S4.1: In the wavelength resource allocation phase, the optical cross-connect devices and available wavelength resources in the network are first identified, and an optical network topology map is constructed. Optical cross-connect devices are key components in an optical network, capable of switching optical signals by wavelength between different input / output ports, achieving optical layer switching without photoelectric conversion. Information on all OXC devices in the network, including device identification, location, type, and capabilities, is obtained through device discovery protocols (such as LLDP, Link Layer Discovery Protocol) or the Network Management System (NMS) interface. Device capabilities include supported wavelength ranges (typically in the C-band 1530-1565nm), wavelength spacing (such as 100GHz or 50GHz), maximum number of ports, and cross-connection capacity. The system also identifies fiber optic connections between devices, including fiber type (such as single-mode or multimode), length, attenuation characteristics, and maximum capacity. For each fiber optic link, the system determines its supported wavelength set, which is related to fiber characteristics and the capabilities of the connected OXC devices. Wavelength resources refer to the different wavelength channels available for data transmission. Each wavelength channel can transmit data independently. A typical DWDM system supports 40-80 wavelength channels, each providing 100Gbps or higher bandwidth. The system integrates this information into an optical network topology graph, which is a directed graph with attributes. Nodes represent OXC devices, and edges represent fiber optic links. Detailed attribute information is attached to nodes and edges. The topology graph also contains wavelength resource information, recording the set of wavelengths available on each link. This topology graph provides a comprehensive network view for subsequent resource allocation.

[0062] Step S4.2: For the optical network topology, monitor the wavelength occupancy status of each optical link in real time to obtain a wavelength resource status table. Wavelength occupancy monitoring refers to continuously tracking the usage status of each wavelength resource in the network to ensure that resource allocation is based on the latest network status. Real-time wavelength occupancy information is obtained through the northbound interface with the optical network controller (such as an SDN-based optical network controller). In specific implementations, periodic polling or event-triggered notification mechanisms can be used. Periodic polling queries the controller at fixed intervals (such as a few seconds or tens of seconds) to obtain the latest resource status; event-triggered notifications are initiated by the controller to notify the system when the resource status changes (such as wavelengths being allocated or released). Wavelength occupancy information can also be obtained directly through network monitoring equipment (such as an optical power monitor), which has a faster response but requires additional monitoring equipment. The obtained wavelength status information includes the occupancy status (idle, occupied, or reserved) of each wavelength on each link, user identifier, allocation time, and expected release time. This information is integrated into a wavelength resource status table, which is a multi-dimensional data structure. The index includes link identifier, wavelength identifier, and time, and the value is the occupancy status and related attributes. To handle large-scale networks, the system employs a distributed state management architecture, partitioning the network into zones, each monitored by an independent state manager, thus improving system scalability. The wavelength resource status table is updated in real time, providing the latest resource availability information for wavelength allocation decisions.

[0063] Step S4.3: Based on the subgraph set and its dependency matrix, estimate the communication frequency and data volume, quantify the communication requirements between subgraphs, and obtain the subgraph communication requirement matrix. Communication requirement quantification refers to converting the data dependencies between subgraphs into specific communication requirement indicators, providing a precise basis for wavelength allocation. First, extract the communication relationships between each pair of subgraphs from the dependency matrix, including whether communication exists, the direction of communication, and its importance. Then, the system estimates the communication frequency based on the characteristics of the graph computation task. For iterative algorithms (such as PageRank), the communication frequency is usually equal to the expected number of iterations; for asynchronous algorithms, the system predicts the communication frequency based on historical execution data and task characteristics; for exploratory algorithms (such as BFS), the communication frequency is related to the diameter or search depth of the graph. Next, estimate the data volume for each communication. The data volume is related to the number of edges across the subgraph, the size of node attributes, and the algorithm's data requirements. For example, in PageRank, each edge transmits a single floating-point value; while in graph matching algorithms, more complex structured data may need to be transmitted. The temporal distribution of communication, i.e., the distribution of communication requirements during the computation process, is also considered. Some algorithms (such as the BSP model) perform all communication simultaneously at the end of each iteration, creating a communication peak; while other algorithms may distribute communication throughout the computation process. The system integrates these estimates into a subgraph communication demand matrix, an n×n matrix (where n is the number of subgraphs). Each element contains communication demand metrics from subgraph i to subgraph j, such as communication frequency, data volume, peak bandwidth requirement, and time distribution. This matrix provides a detailed description of communication requirements for wavelength resource allocation.

[0064] Step S4.4: Based on the wavelength resource status table and the subgraph communication demand matrix, the optimal subgraph-wavelength mapping relationship is calculated using a multi-objective optimization algorithm to obtain the wavelength allocation scheme. Multi-objective optimization refers to the technique of simultaneously optimizing multiple potentially conflicting objective functions. In the wavelength allocation problem, the system considers multiple optimization objectives, including minimizing communication delay, maximizing wavelength utilization, balancing link load, and meeting priority requirements. Communication delay is related to optical path length and the number of intermediate nodes; wavelength utilization measures the effective utilization of wavelength resources by the allocation scheme; link load balancing avoids certain links becoming bottlenecks; priority requirements ensure that important communication needs receive resources first. The system integrates these objectives into a weighted objective function or uses the Pareto optimality method to find the non-dominated solution set. Various algorithms can be used to solve this multi-objective optimization problem, including heuristic algorithms (such as genetic algorithms, particle swarm optimization, simulated annealing), exact algorithms (such as integer linear programming), or hybrid algorithms. For large-scale problems, the system typically uses heuristic algorithms to find an approximate optimal solution within an acceptable time. In the specific implementation, the subgraph communication requirements are first converted into optical path requests, each request containing attributes such as source node, target node, bandwidth requirement, and priority. Then, the system performs two phases: path calculation and wavelength allocation, or uses an integrated routing and wavelength allocation algorithm to solve the problem in one step. The system considers wavelength continuity constraints (all links on the optical path use the same wavelength) and wavelength uniqueness constraints (a wavelength on a link can only be used by one optical path at a time) to ensure the physical feasibility of the allocation scheme. Finally, the system outputs a wavelength allocation scheme, specifying the optical path and the wavelength used between each pair of communication subgraphs.

[0065] Step S4.5: Based on the wavelength allocation scheme, verify the feasibility of the allocation results through a conflict detection algorithm, adjust the conflict situation, and obtain the final optimal wavelength allocation scheme. Conflict detection refers to the process of checking whether there are resource conflicts in the wavelength allocation scheme. In optical networks, the main conflict types include wavelength conflict (the same wavelength is used by multiple optical paths on the same link), path conflict (the optical path passes through a non-existent link or node), and capacity conflict (the allocated wavelength exceeds the device's supported range). First, a wavelength usage map is constructed to record the allocation of each wavelength on each link. Then, each optical path is checked to verify whether the wavelengths used on all links are available and meet the continuity constraint. If a conflict is found, the system needs to be adjusted. The adjustment strategy is based on the conflict type and resource priority, mainly including path reselection (calculating a new route for the conflicting optical path), wavelength reallocation (allocating a new wavelength for the conflicting optical path), and request degradation (reducing the service quality requirements of conflicting requests). According to predefined priority rules, it is determined which optical paths need to be adjusted, prioritizing the resource allocation of high-priority communication needs. During the adjustment process, multiple iterations may be required, re-detecting after resolving some conflicts each time, until all conflicts are resolved or the maximum number of iterations is reached. If certain conflicts cannot be resolved, alternative solutions are provided, such as time-division multiplexing (multiple communication needs share the same wavelength but are used at different times) or wavelength conversion (wavelength conversion at intermediate nodes in the optical path). Through these conflict detection and adjustment steps, a final optimal wavelength allocation scheme is obtained, which is physically feasible and meets the quality of service requirements of each communication need as much as possible.

[0066] Example 6:

[0067] In this embodiment, the multi-level graph segmentation algorithm employs an enhanced partition set coordination algorithm, including: calculating the task feature vector based on the graph, selecting the enhanced partition set coordination algorithm as the graph segmentation strategy, setting a difference threshold parameter and a coordination factor to obtain the enhanced partition set coordination algorithm parameter set; constructing the graph's correlation matrix and extracting key node features based on the original graph data and the enhanced partition set coordination algorithm parameter set, establishing a hierarchical representation of the graph, and obtaining the graph's feature representation and hierarchical structure; constructing a coordination tree structure by recursively applying the enhanced partition set coordination algorithm to obtain an initial coordination tree; and calculating the initial coordination tree... The difference measurement of each node in the computation tree is used to identify the optimal split point and obtain a coordination tree with split labels. Based on the coordination tree with split labels, the connection characteristics of the boundary nodes are analyzed, and a selective node replication strategy is adopted to optimize the cross-subgraph communication mode, resulting in an optimized subgraph set. Based on the optimized subgraph set, a communication cost model is established to evaluate the data exchange volume between subgraphs. Subgraph pairs with communication costs higher than a preset threshold are selectively merged to obtain a subgraph set that balances communication and computation. Based on the subgraph set that balances communication and computation, the data dependencies between subgraphs are analyzed, and a fine-grained dependency matrix is ​​constructed to obtain the subgraph set and its dependency matrix.

[0068] Step S5.1: In this embodiment, based on the graph calculation task feature vector, the Enhanced Partition Set Coordination (EPSR) algorithm is selected as the graph segmentation strategy. EPSR is an efficient graph segmentation technique derived from the partition set coordination divide-and-conquer strategy and incorporating the idea of ​​the random access protocol tree algorithm, making it particularly suitable for processing large-scale complex graph data. The core idea of ​​the PSR strategy is to continuously partition a set of sets that need coordination until the differences contained in these subsets are below a predetermined threshold, thereby achieving efficient segmentation. EPSR introduces adaptive difference control and hierarchical coordination mechanisms on top of PSR, significantly improving segmentation quality and efficiency. The system first calculates the task feature vector based on the graph, especially graph structural features (such as density and clustering degree) and communication patterns, to determine whether to use the EPSR algorithm. If the graph has a complex community structure or highly localized communication patterns, EPSR usually provides better segmentation results. Once EPSR is adopted, key parameters need to be set: the difference threshold parameter θ controls the fine-grainedness of the segmentation; a smaller θ value results in finer-grained segmentation, while a larger θ value generates coarser-grained segmentation; the coordination factor α controls the partition balance; a higher α value emphasizes subgraph size balance, while a lower α value emphasizes internal connection density. These parameters are automatically adjusted based on task characteristics and historical execution data, or users can manually specify parameter values. In addition, the system also sets parameters such as maximum recursion depth, minimum subgraph size, and segmentation evaluation metrics. These parameters together constitute the parameter set of the Enhanced Partition Set Coordination Algorithm, guiding the subsequent segmentation process.

[0069] Step S5.2: Based on the original graph data and the parameter set of the enhanced partitioning set coordination algorithm, construct the graph's correlation matrix and extract key node features to establish a hierarchical representation of the graph. The correlation matrix describes the strength of the connections between nodes in the graph and is an important foundation for graph segmentation. First, calculate the correlation between nodes. Correlation is defined as the strength of the connection between node pairs and can be based on various metrics: structural correlation considers the connection patterns between nodes, such as the number of shared neighbors, edge weights, and path distances; attribute correlation considers the similarity of node attributes, such as the Euclidean distance or cosine similarity of attribute values; functional correlation considers the functional similarity of nodes in algorithm execution, such as the similarity of computational patterns or communication behaviors. Combine these metrics into a unified correlation index to construct an n×n correlation matrix (n is the number of nodes). Since the correlation matrix of a large-scale graph is too large, the system uses sparse representation or low-rank approximation techniques to reduce storage and computational overhead. Next, extract key node features, including centrality indices (such as degree centrality, betweenness centrality, and eigenvector centrality), local structural features (such as clustering coefficients and neighborhood density), and attribute features. These features help identify key nodes and structural patterns in the graph. Based on the correlation matrix and node features, a hierarchical representation of the graph is established, organizing the graph into a multi-layered structure, with each layer representing a view at a different granularity. Nodes at higher levels represent sets or communities of nodes in the original graph, and the layers are connected by aggregation relationships. This representation is constructed using hierarchical clustering or community detection algorithms to obtain the graph's feature representation and hierarchical structure, providing a structured view for subsequent recursive segmentation.

[0070] Step S5.3: Based on the feature representation and hierarchical structure of the graph, a coordination tree structure is constructed by recursively applying the enhanced partition set coordination algorithm. The coordination tree is a tree structure representing a hierarchical partitioning process, where each tree node corresponds to a sub-region of the graph, and parent-child relationships represent the partitioning level. The system starts from the root node (representing the entire graph) and recursively executes the partitioning process. At each recursive level, a bidirectional partitioning strategy is applied to divide the current node into two child nodes. The partitioning algorithm considers the correlation between nodes and the structural features of the current level, optimizing the ratio of internal connection density to external connection cost. Specifically, spectral partitioning, minimum cut, or multi-level clustering algorithms can be used. Spectral partitioning maps nodes to a low-dimensional space based on the eigenvectors of the correlation matrix before partitioning; minimum cut directly minimizes the sum of weights of cross-partition edges; multi-level clustering combines hierarchical representation for fast partitioning. To improve efficiency, initial partitioning is performed at a coarse-grained level, and then the results are mapped to a fine-grained level for local optimization. After each partition, the difference metric of child nodes, i.e., the ratio of internal connection density to external connection cost, is calculated. If the difference metric is higher than the preset difference threshold parameter θ, it indicates that the internal structure of the child node has significant differences, and the system continues to recursively segment the child node; otherwise, the child node becomes a leaf node of the coordination tree. Through this recursive process, a complete initial coordination tree is constructed, representing the hierarchical segmentation structure of the graph.

[0071] Step S5.4: For the initial coordination tree, calculate the dissimilarity metric for each node in the tree and identify the optimal split point. The dissimilarity metric is an indicator that evaluates the tightness of internal connections and the looseness of external connections in a subgraph. A high dissimilarity metric indicates that the subgraph is a natural clustering unit. Multiple metrics are used to calculate the dissimilarity metric: modularity measures the degree of deviation of the internal connections of the subgraph from the random graph model; higher modularity indicates a significant community structure. Conductivity measures the ratio of the boundary to the volume of the subgraph; lower conductivity indicates a better split. Standardized cut considers both the weight of the splitting edge and the size of the subgraph, balancing split quality and balance. The system integrates these metrics and assigns a dissimilarity metric value to each node in the coordination tree. Then, the optimal split point is identified, i.e., the node where the dissimilarity metric value changes significantly. These nodes represent natural boundaries in the graph structure and are ideal splitting locations. Adaptive thresholding techniques can be used to identify these points, such as calculating the gradient of the dissimilarity metric and marking nodes with gradients greater than a certain threshold as split points. Alternatively, dynamic programming can be used to find the optimal set of splits, i.e., using the fewest split points to divide the graph into subgraphs with dissimilarity metrics below the threshold. These analyses yield a coordination tree with splitting labels, where each label represents a splitting decision point that guides subsequent subgraph generation.

[0072] Step S5.5: Based on the coordination tree with splitting tags, analyze the connectivity characteristics of boundary nodes and adopt a selective node replication strategy to optimize the cross-subgraph communication mode. Boundary nodes are nodes located at the boundaries of subgraphs. These nodes have direct connections to nodes in other subgraphs and are the main source of cross-subgraph communication. The system first identifies all boundary nodes and analyzes their connectivity characteristics, including the number of subgraphs they connect to, the strength of their connections to each subgraph, and their importance in computation. Based on these characteristics, a selective node replication strategy is adopted, which involves retaining copies of the same node in multiple subgraphs to reduce the need for cross-subgraph communication. Node replication is a space-for-time strategy, reducing communication overhead by increasing storage overhead. The replication benefit is calculated for each boundary node, which is the ratio of the reduced communication cost after replication to the increased storage overhead. Only nodes with benefits exceeding a certain threshold are replicated. The replication decision also considers the computational complexity and update frequency of nodes, avoiding the replication of computationally intensive or frequently updated nodes, as the replication of these nodes may lead to high synchronization overhead. For nodes selected for replication, copies are created in the relevant subgraphs, and a synchronization mechanism between copies is established to ensure data consistency. By employing this selective replication strategy, the system optimizes the cross-subgraph communication pattern, reduces communication requirements, and obtains an optimized set of subgraphs.

[0073] Step S5.6: Based on the optimized subgraph set, establish a communication cost model to evaluate the data exchange volume between subgraphs, and selectively merge subgraph pairs with communication costs exceeding a preset threshold. The communication cost model is a mathematical model that quantifies the communication overhead between subgraphs, considering various factors such as data exchange volume, communication frequency, network latency, and bandwidth requirements. First, based on the characteristics of graph algorithms and subgraph structures, the data exchange volume between subgraphs is estimated. For iterative algorithms, the data exchange volume is related to the number of edges across the subgraph, edge weights, and the number of iterations; for exploratory algorithms, it is related to access patterns and search depth. The system also considers the temporal distribution and priority of communication, constructing a comprehensive communication cost matrix, where each element represents the communication cost between a pair of subgraphs. Then, subgraph pairs with communication costs exceeding a preset threshold are identified, as frequent communication between these pairs may become performance bottlenecks. A selective merging strategy is adopted to merge communication-intensive subgraph pairs into larger subgraphs, reducing communication requirements. The merging decision is based on a multi-factor evaluation, including the benefits of reduced communication, the size of the merged subgraph, and the impact on computational load balancing. Prioritize merging subgraph pairs with high communication costs that will not lead to severe load imbalance after merging. The merging process is iterative, with communication costs reassessed after each merge until no subgraph pair's communication cost exceeds a threshold, or the maximum number of merges is reached. This selective merging strategy significantly reduces communication overhead while maintaining a relatively balanced computational load, resulting in a set of subgraphs with balanced communication and computation.

[0074] Step S5.7: Finally, based on the set of subgraphs balancing communication and computation, analyze the data dependencies between subgraphs and construct a fine-grained dependency matrix. Data dependency refers to the requirement that computation in one subgraph requires data input from another subgraph; this dependency determines the execution order of computation and communication requirements. First, analyze the data flow pattern of the algorithm to identify the dependency types between subgraphs. Dependency types include direct dependency (the output of subgraph A is directly used as the input of subgraph B), indirect dependency (dependency is passed through intermediate subgraphs), and bidirectional dependency (subgraphs depend on each other). For each dependency type, further analyze its characteristics, such as the size of the dependent data, update frequency, and timeliness requirements. Based on these analyses, construct a fine-grained dependency matrix, which is an n×n matrix (n is the number of subgraphs), where each element contains detailed dependency information from subgraph i to subgraph j. This information includes dependency type, data volume, update frequency, priority, and quality of service requirements. Compared to a simple communication matrix, the dependency matrix provides richer semantic information, enabling more precise resource allocation and scheduling decisions. For example, computation order can be arranged based on dependencies, prioritizing the execution of key subgraphs that are depended upon by multiple subgraphs; or appropriate communication resources can be allocated based on the size and frequency of dependent data. Through this fine-grained analysis, a set of subgraphs and their dependency matrix are obtained, providing a comprehensive and accurate basis for subsequent wavelength resource allocation and task scheduling.

[0075] Example 7:

[0076] The recursive application of the enhanced partition set coordination algorithm to construct a coordination tree structure includes recursively partitioning the graph structure from the root node based on the graph's feature representation and hierarchical structure, generating two child nodes at each partition to obtain a binary coordination tree; for each node in the binary coordination tree, calculating the ratio of internal connection density to external connection cost to obtain a node difference metric; comparing the node difference metric with a preset difference threshold parameter to determine whether to continue further partitioning the node, obtaining a partitioning decision; based on the partitioning decision, recursively continuing until the difference metric of all leaf nodes is lower than the preset difference threshold parameter, obtaining the initial coordination tree.

[0077] Step S5.3.1: In constructing the coordination tree structure, the graph structure is recursively partitioned starting from the root node based on the graph's feature representation and hierarchical structure. A coordination tree is a special hierarchical data structure used to represent the multi-level partitioning process of graph data, with each tree node corresponding to a sub-region of the original graph. Recursive partitioning is a top-down divide-and-conquer strategy, starting from the overall graph and gradually refining the partitions until the desired granularity is achieved. The system first creates the root node of the coordination tree, which represents the entire input graph. Then, a binary partitioning technique is applied to divide the graph region corresponding to the current node into two relatively independent parts. Binary partitioning is the core operation of the EPSR algorithm; it attempts to find a partitioning method that makes the two subgraphs tightly connected internally, while the connections between subgraphs are sparse. There are several methods to implement binary partitioning, including spectral partitioning, core decomposition, and random walks. Spectral partitioning uses the eigenvectors of the graph's Laplacian matrix to map nodes to a low-dimensional space before partitioning; core decomposition divides the graph into core and non-core regions based on the number of cores in each node; random walks simulate a random walking process and identify "bottleneck" locations as partitioning points. Based on the graph's characteristics and computational requirements, the most suitable binary partitioning algorithm is selected. A partitioning operation is performed on each non-leaf node, generating two child nodes. These child nodes represent two sub-regions of the original graph, and they are added to the coordination tree as children of their parent nodes. Through this recursive binary partitioning process, the system constructs a complete binary coordination tree, where each internal node has two child nodes, representing one binary partitioning operation, and leaf nodes represent the final subgraph units.

[0078] The most suitable binary segmentation algorithm is the one that performs optimally among spectral segmentation, core decomposition, and random walk algorithms, based on the graph's characteristics (such as density, degree distribution, and clustering coefficient) and computational requirements (such as segmentation quality and computation time). For example, for a social network graph with a density of 0.001 (sparse graph), a degree distribution exhibiting a clear power-law characteristic (a few nodes have very high degrees), and a clustering coefficient of 0.35 (with community structure), decision trees show that spectral segmentation provides the best segmentation quality when the graph density is less than 0.01 and the degree distribution follows a power-law distribution; when the clustering coefficient is greater than 0.3, the random walk algorithm based on community detection is more efficient. Considering all factors, spectral segmentation is chosen for initial segmentation, followed by local optimization using a random walk algorithm. This approach ensures both segmentation quality and keeps computation time within an acceptable range.

[0079] Step S5.3.2: For each node in the binary coordination tree, calculate the ratio of internal connection density to external connection cost to obtain the node difference metric. The node difference metric is a key indicator for evaluating subgraph quality. It measures the tightness of internal connections and the sparsity of connections between subgraphs, and is an important basis for deciding whether to continue splitting. Internal connection density quantifies the tightness of connections between nodes within a subgraph, usually defined as the ratio of the sum of edge weights within the subgraph to the number of nodes in the subgraph. High internal connection density indicates that nodes within the subgraph are tightly connected, forming a cohesive unit. External connection cost quantifies the connection cost between a subgraph and other subgraphs, usually defined as the sum of edge weights from the subgraph to other subgraphs. Low external connection cost indicates that the subgraph has fewer connections to other parts, and the splitting is more reasonable. The system calculates the ratio of these two indicators, i.e., internal connection density divided by external connection cost, as the node difference metric. The higher this ratio, the tighter the internal connections and the sparser the external connections, indicating a high-quality subgraph. In addition to the basic ratio, the system can also use other measurement methods, such as modularity, conductivity, or normalized cut. Modularity measures the difference between the actual internal connectivity of a subgraph and the expected connectivity of a random graph model; conductivity considers the ratio of the subgraph's "surface area" to its "volume"; and standardized cuts combine cut edge weights with subgraph size. The system can select the most suitable metric based on the characteristics of the graph computation task, or combine multiple metrics to form a comprehensive score. By calculating the difference metric value for each node, the quality of the current segmentation can be quantitatively evaluated, providing a scientific basis for subsequent segmentation decisions.

[0080] The most suitable metric is the one that most accurately reflects the quality of the subgraph, selected from metrics such as modularity, conductivity, and normalized cut, based on the characteristics of the graph computation task. For community detection algorithms, the main goal is to identify dense subgraph structures within the graph. In this case, modularity is the most accurate metric because it directly measures the deviation of the internal connection density of the subgraph from the random graph model. For distributed graph computation tasks with high load balancing requirements, normalized cut is more suitable because it considers both the weight of the splitting edges and the balance of subgraph size.

[0081] Step S5.3.3: Compare the node difference metric with a preset difference threshold parameter to determine whether to continue further segmentation of the node, thus obtaining the segmentation decision. The difference threshold parameter is a key control parameter of the EPSR algorithm, determining the termination condition and granularity of segmentation. Comparing the difference metric of each node with the preset difference threshold parameter θ is the core step of the segmentation decision. If the difference metric of a node is higher than the threshold θ, it indicates that the subgraph represented by the node has significant differences in its internal structure and may contain multiple natural clustering units, so segmentation should continue; if the difference metric is lower than the threshold θ, it indicates that the subgraph represented by the node is relatively uniform with little internal difference and can be considered as a basic unit without further segmentation. This threshold-based decision-making mechanism ensures that the segmentation process can adaptively adapt to the local structural characteristics of the graph, performing more detailed segmentation in structurally complex regions and maintaining a coarser granularity in structurally simple regions. An adaptive threshold strategy can be adopted, dynamically adjusting the threshold value according to the global or local characteristics of the graph. For example, for graphs with obvious community structures, a higher threshold can be used to quickly identify the main communities; for graphs with uniform structures, a lower threshold can be used to ensure appropriate segmentation. Furthermore, other constraints, such as minimum subgraph size, maximum segmentation depth, or computational resource limitations, can be combined to comprehensively determine whether to continue segmenting. Through this threshold comparison and comprehensive decision-making, the system generates a segmentation decision for each node—whether to continue segmenting or stop segmenting—guiding the subsequent recursive process.

[0082] Step S5.3.4: Based on the splitting decision, recursively continue until the difference metric values ​​of all leaf nodes are lower than the preset difference threshold parameter, obtaining the initial harmonic tree. Continuous recursion means that the system repeatedly performs the splitting operation on nodes that need further splitting based on the splitting decision until the termination condition is met. For each node with a splitting decision of "continue," the system reapplies the binary splitting algorithm, dividing it into two child nodes, and calculates the difference metric value and splitting decision for these two child nodes. This process is recursively performed, forming a tree structure with uneven depth, reflecting the local complexity differences of the graph. The termination condition is that the difference metric values ​​of all leaf nodes are lower than the preset difference threshold parameter θ, indicating that the current splitting has reached the expected granularity and quality requirements. To prevent infinite recursion or over-splitting, additional termination conditions are usually set, such as maximum recursion depth (limiting the maximum height of the tree), minimum subgraph size (ensuring that the subgraph contains enough nodes), or a global subgraph number limit (controlling the total number of splits). When one of these termination conditions is met, the system stops further splitting the corresponding node and marks it as a leaf node. Post-processing strategies may also be employed, such as merging small subgraphs (merging subgraphs smaller than a certain size into adjacent subgraphs) or balancing subgraph sizes (adjusting subgraph boundaries to make the sizes of each subgraph closer), to further optimize the segmentation results. Through this continuous recursion and termination control, an initial coordination tree is finally obtained, which is a complete hierarchical structure representing the multi-level segmentation process of the graph. Each leaf node of the coordination tree corresponds to a final subgraph unit, and these leaf nodes together constitute a complete segmentation of the original graph; while the internal nodes record the hierarchical relationships and historical processes of the segmentation, which can be used to understand the structure of the graph and optimize the segmentation strategy.

[0083] Example 8:

[0084] The mapping relationship between subgraph communication requirements and wavelength resources is established using a locality-sensitive hashing (LSH) mechanism, including: designing an L-bit unique identifier encoding scheme based on the optical network topology and the wavelength resource status table, assigning a unique identifier to each available wavelength resource, and obtaining a wavelength resource identifier mapping table; mapping the communication requirement characteristics between subgraphs to a hash space using a family of LSH functions based on the subgraph set and its dependency matrix, constructing a hash representation of communication requirements, and obtaining a subgraph communication requirement hash table; analyzing the synchronization status and message passing mode between computing nodes based on the system operating status and task scheduling mode, determining whether the current environment is synchronous or asynchronous, and obtaining an environment mode identifier; based on the environment mode identifier, using the LC-LSH algorithm when in synchronous mode and the LC-LSH4 algorithm when in asynchronous mode, combining the wavelength resource identifier mapping table and the subgraph communication requirement hash table, calculating the optimal subgraph and wavelength matching scheme, and obtaining a wavelength allocation scheme; based on the wavelength allocation scheme, identifying resource allocation conflicts using a conflict detection algorithm, and applying a priority-based conflict resolution strategy to obtain the optimal wavelength allocation scheme.

[0085] Step S6.1: In the wavelength resource allocation phase, based on the optical network topology and wavelength resource status table, an L-bit unique identifier encoding scheme is designed to assign a unique identifier to each available wavelength resource. The unique identifier encoding scheme is a technique for assigning unique identifiers to network resources. In an optical network, each wavelength resource requires a unique identifier for management and allocation. An L-bit identifier refers to an encoding using L binary bits, theoretically capable of representing 2^365 microseconds. L Each resource has a different encoding scheme. The designed encoding scheme typically includes multiple fields, each representing an attribute or dimension of the resource. For wavelength resources, typical encoding fields include: link identifier (identifying the fiber optic link), wavelength identifier (identifying the frequency or channel number of the wavelength), direction identifier (identifying the direction of optical signal transmission), time identifier (identifying the valid time period of the resource), and status identifier (identifying the current status of the resource), etc. The bit length of each field is determined based on the network size and resource characteristics to ensure a sufficiently large encoding space and high encoding efficiency. For example, in a network with 100 links and 80 wavelengths, the link identifier may require 7 bits (2^35 bits). 7=128), the wavelength identifier requires 7 bits, and other fields may require several bits, for a total encoding length of 20-30 bits. The encoding design also considers the principle of locality, ensuring that physically or logically proximate resources have similar codes, facilitating locality-sensitive hashing operations. This encoding scheme is applied to each available wavelength resource in the network to generate a unique identifier, and these identifiers are associated with detailed attributes of the resource (such as physical location, bandwidth capacity, latency characteristics, etc.) to form a wavelength resource identifier mapping table. This mapping table is a set of key-value pairs, where the key is the unique identifier of the resource and the value is a detailed description of the resource's attributes, providing an efficient index for subsequent resource matching and allocation.

[0086] Step S6.2: Based on the subgraph set and its dependency matrix, the communication requirement features between subgraphs are mapped to the hash space using a family of Locality Sensitive Hash (LSH) functions, constructing a hash representation of the communication requirements. LSH is a dimensionality reduction technique that maps high-dimensional data to a low-dimensional space, ensuring that similar data points in the original space are mapped to the same or similar positions in the hash space with a higher probability. Unlike traditional hash functions, LSH preserves the similarity relationships of the data, making it an effective tool for handling similarity searches in high-dimensional data. The LSH function family is a set of hash functions, each mapping input data to a hash value. Combining multiple functions can improve the accuracy of similarity judgment. In this system, the communication requirement feature is a multi-dimensional vector containing multiple dimensions such as communication frequency, data volume, priority, and time distribution. First, the communication requirements in the dependency matrix are converted into feature vectors, each representing the communication characteristics between a pair of subgraphs. Then, the system selects a suitable family of LSH functions, commonly including MinHash (suitable for set similarity), SimHash (suitable for vector cosine similarity), and p-stable distribution (suitable for Euclidean distance). The system applies a selected family of LSH functions to map each communication requirement feature vector to one or more hash values, which constitute the hash representation of the communication requirement. An index structure is established to group communication requirements with similar hash values ​​together, forming a communication requirement hash table. This hash table is an efficient data structure where the keys are hash values ​​or combinations of hash values, and the values ​​are sets of subgraph pairs with similar communication characteristics. This hash representation allows for the rapid identification of subgraph pairs with similar communication requirements, providing a clustered view for wavelength resource allocation, significantly reducing the search space, and improving allocation efficiency.

[0087] Step S6.3: Based on the system's operating status and task scheduling mode, analyze the synchronization status and message passing mode between computing nodes to determine whether the current environment is in synchronous or asynchronous mode. Synchronous and asynchronous modes are two basic operating modes of distributed computing systems, with different communication characteristics and resource requirements, necessitating different wavelength allocation strategies. Synchronous mode refers to all computing nodes performing computation and communication according to a unified clock or fence mechanism, with data exchange between nodes occurring at predetermined synchronization points. Typical synchronous computing models include batch synchronous parallel models, which organize computation into a series of hypersteps, each containing three stages: computation, communication, and synchronization. Communication in synchronous mode is predictable, with relatively fixed communication times and data volumes, making it suitable for deterministic resource allocation strategies. Asynchronous mode refers to computing nodes working independently, communicating as needed based on their own progress and requirements, without a global synchronization point. Asynchronous mode is more flexible, reducing synchronization overhead and improving resource utilization, but the communication mode is uncertain and difficult to predict. The system analyzes multiple factors to determine the current environment mode: task scheduling mode (iterative scheduling based on BSP or asynchronous scheduling based on events), algorithm characteristics (strictly synchronous iterative algorithm or exploratory algorithm that allows asynchronous execution), system state (clock synchronization degree between nodes, communication congestion status), and user configuration (whether synchronous or asynchronous mode is forcibly specified). The system integrates these factors to derive an environment mode identifier, i.e., synchronous or asynchronous mode, providing a basis for subsequent algorithm selection. For complex environments, a mixed mode may be identified, where some subgraph groups use synchronous mode while others use asynchronous mode. In this case, resource allocation for different modes needs to be handled separately.

[0088] Step S6.4: Based on the environment mode identifier, the system uses the LC-LSH algorithm in synchronous mode and the LC-LSH4 algorithm in asynchronous mode. Combining the wavelength resource identifier mapping table and the subgraph communication demand hash table, the optimal subgraph and wavelength matching scheme is calculated. LC-LSH and LC-LSH4 are two wavelength allocation algorithms based on locality-sensitive hashing, optimized for synchronous and asynchronous environments respectively. The LC-LSH algorithm is suitable for synchronous environments, utilizing the predictability of communication demands to allocate wavelength resources for communication demands within a defined time window. The core steps of the algorithm include: hash matching (matching communication demands with wavelength resources in the hash space), similarity scoring (calculating the degree of matching between demands and resources), optimized allocation (solving many-to-one or one-to-many matching problems), and resource reservation (reserving resources for recurring communication patterns). LC-LSH leverages the periodicity and regularity of communication patterns in synchronous environments to plan resource allocation in advance, reducing conflicts and reconfiguration overhead. The LC-LSH4 algorithm is an extended version of LC-LSH, specifically designed for asynchronous environments. It adds four key mechanisms: dynamic time window (dynamically adjusting the allocation period based on communication frequency), probabilistic prediction (predicting future communication needs based on historical data), resource sharing (allowing multiple low-frequency communication needs to share the same wavelength), and fast reconfiguration (rapidly adjusting allocation when communication needs change). LC-LSH4 can handle the uncertainty and dynamic changes in asynchronous environments, providing more flexible resource allocation. The system selects the appropriate algorithm based on the environment mode (e.g., the D algorithm and the BF (Bellman-Ford) algorithm, where the D algorithm is Dijkstra's algorithm with a time complexity of O(VlogV+E), and the BF algorithm has a time complexity of O(VE)). Then, it combines a wavelength resource identifier mapping table (providing available resource information) and a subgraph communication demand hash table (providing a demand clustering view) to perform matching calculations. The matching process considers multiple objectives, including maximizing resource utilization, minimizing communication latency, balancing link load, and meeting quality of service requirements. This complex matching problem can be solved using greedy algorithms, graph matching algorithms, or multi-objective optimization algorithms to obtain a preliminary wavelength allocation scheme, namely the mapping relationship between subgraph communication needs and wavelength resources.

[0089] Step S6.5: Based on the wavelength allocation scheme, resource allocation conflicts are identified using a conflict detection algorithm, and a priority-based conflict resolution strategy is applied to obtain the optimal wavelength allocation scheme. Conflict detection is a crucial step in verifying the feasibility of the resource allocation scheme, checking for situations where multiple communication requests are allocated to the same resource, leading to conflicts. In optical networks, the main conflict types include wavelength conflicts (the same wavelength on the same link is used by multiple optical paths at the same time), path conflicts (optical paths pass through non-existent or unavailable links), capacity conflicts (allocating wavelengths beyond the equipment's support range), and time conflicts (the same resource is used by multiple requests in overlapping time periods). First, a resource usage graph is constructed, recording the allocation of each resource, including users, time periods, and usage patterns. Then, conflicts are checked for in the allocation of each resource; if conflicts are found, they are recorded in the conflict list. For the detected conflicts, a priority-based conflict resolution strategy is applied. This strategy first assigns a priority to each communication request, with priority factors including communication importance (impact on algorithm convergence), data volume, time urgency, and service quality requirements. Conflicting requests are sorted according to priority; high-priority requests retain their original allocation, while low-priority requests need adjustment. The adjustment strategies include resource reallocation (allocating alternative resources to low-priority demands), time rescheduling (adjusting communication times to avoid conflicts), path reselection (selecting alternative paths for communication demands), and demand degradation (reducing quality of service requirements, such as accepting lower bandwidth or higher latency). Multiple rounds of conflict detection and resolution may be required, as each adjustment may introduce new conflicts. After resolving all conflicts, the final optimal wavelength allocation scheme is obtained. This scheme is physically feasible and, as far as possible, satisfies the quality of service requirements of each communication demand, providing a reliable communication foundation for efficient distributed graph computation.

[0090] Example 9:

[0091] When the environment mode is identified as asynchronous mode, the process further includes: based on the preliminary LC-LSH4 algorithm allocation results, processing asymmetric communication requirements through the ASYM-LC-LSH4 algorithm to obtain an asymmetric optimization scheme; according to the asymmetric optimization scheme, applying the QR-LC-LSH4 algorithm to introduce quasi-randomness to guarantee a finite maximum rendezvous time, resulting in an optimized asynchronous environment wavelength allocation scheme; for the optimized asynchronous environment wavelength allocation scheme, evaluating key indicators including expected rendezvous time and maximum rendezvous time through a simulation model to obtain a wavelength allocation scheme with performance indicators.

[0092] Step S6.6: When the system determines that the current environment is in asynchronous mode, based on the preliminary allocation results of the LC-LSH4 algorithm, the ASYM-LC-LSH4 algorithm is used to handle asymmetric communication requirements. Asymmetric communication requirements refer to the situation where the communication volume between subgraph pairs is not equal in two directions, that is, there is a significant difference between the communication volume from subgraph A to subgraph B and the communication volume from B to A. This asymmetry is common in many graph algorithms, such as directed graph analysis, parallel processing of unbalanced data distribution, or master-slave computational models. The ASYM-LC-LSH4 algorithm is an important extension of LC-LSH4, specifically designed to handle this asymmetric communication scenario. The algorithm first performs directional analysis on the communication requirements, identifying the dominant and subordinate directions of communication. The dominant direction refers to the direction with larger communication volume, which usually has a greater impact on system performance; the subordinate direction refers to the direction with smaller communication volume, which can appropriately reduce the quality of service requirements. Then, the algorithm constructs a direction-aware hash representation, generating an independent hash value for the communication requirements in each direction, enabling the algorithm to distinguish the characteristics of requirements in different directions. During the resource allocation phase, ASYM-LC-LSH4 employs an asymmetric allocation strategy: prioritizing high-quality wavelength resources, such as low-latency paths or high-bandwidth channels, for the dominant direction; allocating suboptimal resources to subordinate directions, or allowing them to reuse resources from the dominant direction (if there is no time conflict). The algorithm also implements a dynamic direction adjustment mechanism, continuously monitoring the actual communication pattern and adjusting resource allocation promptly when communication directions change (e.g., role swaps or changes in traffic). ASYM-LC-LSH4 considers the balance between resource efficiency and fairness, ensuring that even subordinate directions receive sufficient resources through techniques such as direction priority weighting, minimum service guarantees, and elastic resource allocation. These optimizations result in an asymmetric optimization scheme, significantly improving resource utilization efficiency and system performance in asymmetric communication scenarios.

[0093] Step S6.7: Based on the asymmetric optimization scheme, the QR-LC-LSH4 algorithm is applied to introduce quasi-randomness to ensure a finite maximum rendezvous time. Rendezvous time refers to the waiting time required for two nodes to successfully establish communication in an asynchronous communication environment. In wavelength hopping or multi-wavelength access systems, if the node's wavelength selection strategy is too deterministic, it may cause two nodes that need to communicate to be unable to "encounter" the same wavelength for a long time, resulting in communication failure or severe delay. Maximum rendezvous time is a key indicator of communication reliability; it represents the maximum time required for two nodes to establish communication in the worst-case scenario. The QR-LC-LSH4 algorithm solves this problem by introducing quasi-randomness. Quasi-randomness is a property between determinism and complete randomness; it maintains a certain structured pattern while introducing sufficient uncertainty to avoid the persistence of the worst-case scenario. The algorithm first constructs a quasi-random sequence generator. Commonly used methods include low-discrepancy sequences, pseudo-random number generators, or prime-number-based periodic sequences. These sequences have good coverage and uniformity, and can cover all possible states within a finite number of steps. The algorithm then assigns a quasi-random sequence to each communication node to guide its wavelength selection behavior. During wavelength selection, nodes access different wavelengths according to their quasi-random sequences. The sequence design ensures that any two nodes will inevitably "meet" (select the same wavelength) within a finite number of steps. By selecting a suitable quasi-random sequence, the algorithm can precisely control the MTTR, ensuring worst-case performance without significantly affecting average performance. Furthermore, QR-LC-LSH4 implements a sequence adaptive adjustment mechanism, dynamically adjusting the parameters of the quasi-random sequence based on historical communication success rates and latency data to optimize rendezvous performance. Through these techniques, an optimized wavelength allocation scheme for asynchronous environments is obtained, which retains the ability of ASYM-LC-LSH4 to handle asymmetric communication while increasing the controllability of rendezvous time and improving communication reliability in asynchronous environments. Specifically, the selection of the quasi-random sequence is based on rendezvous time requirements and system scale, choosing from low-discrepancy sequences, pseudo-random number sequences, or prime-number-based periodic sequences that guarantee node meeting within a finite number of steps and possess good coverage and uniformity. In an asynchronous communication system with 80 wavelength channels, the maximum intersection time between any two nodes must not exceed 100 time slots. The system evaluates the performance of various sequences. Low-dispersion sequences (such as Halton sequences) can guarantee intersection within 64 time slots but have high generation complexity. Pseudo-random number sequences have an average intersection time of 40 time slots, but the maximum intersection time cannot be guaranteed. Periodic sequences based on prime numbers (such as choosing prime number 41) can guarantee intersection within 82 time slots and are simple to implement. After comprehensive consideration, the system selects a periodic sequence based on prime number 41, which satisfies both the maximum intersection time requirement and maintains implementation efficiency.

[0094] Step S6.8: For the optimized asynchronous environment wavelength allocation scheme, evaluate key indicators including expected meeting time and maximum meeting time (MTTR) through a simulation model to obtain a wavelength allocation scheme with performance metrics. A simulation model is a tool for evaluating system performance before actual deployment, simulating system behavior under various conditions and providing a basis for performance prediction and scheme comparison. The constructed simulation model includes multiple components: a network model (simulating optical network topology, link characteristics, and device behavior), a communication model (simulating communication patterns, data flow, and protocol behavior between nodes), a computation model (simulating the execution process and resource consumption of graph computation tasks), and a scheduling model (simulating wavelength resource allocation and management strategies). Run Monte Carlo simulations on this model, simulating various possible scenarios through multiple random samplings to obtain statistical performance data. Key evaluation metrics include expected meeting time and maximum meeting time (MTTR). ETTR is the average time required for two nodes to establish communication, reflecting the average performance of the system; MTTR is the worst-case meeting time, reflecting the lower limit of system performance and reliability. Other metrics are also evaluated, such as throughput (the amount of communication completed per unit time), wavelength utilization (the effective use of wavelength resources), load balancing (the distribution of load across links), and quality of service (QoS) satisfaction rate (the proportion of communication that meets QoS requirements). Performance metrics under different algorithms and parameter configurations are compared, such as the performance of basic LC-LSH4, ASYM-LC-LSH4, and QR-LC-LSH4 under various network loads and communication modes, to identify the optimal configuration. Simulation results are typically presented as performance curves, statistical tables, or heatmaps, visually demonstrating the advantages and disadvantages of the proposed solution. The simulation evaluation results are correlated with the allocation scheme to form a wavelength allocation scheme with performance metrics. These metrics provide performance expectations and reliability guarantees for actual deployment, helping users understand the system's performance under various conditions and supporting performance-based decision-making and optimization.

[0095] Example 10:

[0096] It also includes: based on the calculation results and performance statistics, continuously collecting system operating status through a distributed monitoring agent to obtain a system performance monitoring data stream; based on the system performance monitoring data stream, identifying performance bottlenecks in the system through anomaly detection and bottleneck identification algorithms to obtain a bottleneck analysis report; based on the bottleneck analysis report, generating adjustment strategies for the current state through an optimization strategy library and decision model to obtain an adjustment strategy instruction set; and based on the adjustment strategy instruction set, dynamically adjusting through wavelength resource reallocation algorithms and subgraph repartitioning algorithms to obtain optimized system performance indicators.

[0097] Step S7.1: In the graph computing task feature extraction stage, a graph computing task identification model is first constructed. The code or description of the graph computing task is taken as input, processed by a feature extractor and a multi-layer classifier, and outputs an algorithm type identifier. The graph computing task identification model is an intelligent system that automatically analyzes and identifies graph algorithm types. It combines natural language processing, code analysis, and machine learning techniques to extract key features from task descriptions or code and determine the algorithm type. The feature extractor is the front-end component of the model, responsible for extracting discriminative features from the input. For code input, the feature extractor uses static code analysis techniques to identify key syntactic structures (such as nested loops and conditional statements), API call patterns (such as graph traversal functions and matrix operations), and data flow features (such as iterative update patterns and message passing patterns). The system may use techniques such as abstract syntax tree analysis, program dependency graphs, or code semantic vectorization to achieve code feature extraction. For text description input, the feature extractor uses natural language processing techniques, such as the bag-of-words model, TF-IDF features, or more advanced deep learning models (such as BERT and GPT), to extract semantic features. These models can understand information such as technical terms, algorithm intent, and performance requirements in the task description. A multi-level classifier is a backend component of the model that maps extracted features to a predefined set of algorithm types. The system typically employs a hierarchical classification strategy: the first layer distinguishes broad algorithm categories (e.g., pathfinding, iterative convergence, clustering); the second layer further subdivides specific algorithms (e.g., breadth-first search, depth-first search, shortest path algorithms, PageRank, community detection); and the third layer identifies algorithm variants and implementation methods (e.g., synchronous or asynchronous versions, shared memory or distributed implementations). The classifier can be based on various machine learning models, such as Support Vector Machines (SVM), Random Forests, Neural Networks, or ensemble models. The system is trained using labeled algorithm libraries containing code examples and descriptions of various graph algorithms, along with corresponding type labels. To improve recognition accuracy, the system may also incorporate historical execution data and expert rules, such as performance characteristics or resource usage patterns of specific algorithms. After processing, the system outputs an algorithm type identifier, a structured data set containing information such as the algorithm category, specific algorithm name, implementation method, and possible optimization strategies, providing a foundation for subsequent analysis.

[0098] Step S7.2: Based on the algorithm type identification results, and combined with the input graph data scale, topological characteristics, and node / edge attribute information of the graph computation task, analyze the data exchange pattern during the computation process to obtain a communication requirement profile. The communication requirement profile is a multi-dimensional representation describing the data exchange characteristics during graph computation, characterizing the frequency, range, data volume, and temporal distribution of communication between computation nodes. The system first analyzes the basic characteristics of the input graph data: the graph data scale refers to the order of magnitude of the number of nodes and edges, directly affecting computation and communication volume; topological characteristics include graph density (the ratio of the number of edges to the maximum possible number of edges), degree distribution (the distribution of the number of node connections), clustering coefficient (reflecting the tightness of connections between node neighbors), and community structure (densely connected subgroups in the graph), which affect the locality and distribution of communication; node / edge attribute information includes attribute type (such as numerical, categorical, and textual), data size, and update frequency, directly affecting the amount of communication data. Then, combined with the algorithm type identification results, predict the data exchange pattern during the computation process. Different types of graph algorithms exhibit different communication characteristics: iterative algorithms (such as PageRank) typically perform global synchronization after each iteration, with communication frequency related to the number of iterations; exploratory algorithms (such as breadth-first search) dynamically change communication as the search front expands; clustering algorithms (such as community detection) usually involve dense communication within local regions, with relatively little global communication. This study quantifies communication characteristics across multiple dimensions: communication frequency (number of communications per unit time), communication range (point-to-point, local broadcast, or global broadcast), data volume (size of data transmitted in each communication), and temporal distribution (distribution of communication during computation, such as uniform or bursty distribution). Furthermore, it analyzes communication directionality (one-way or two-way), priority (the degree of impact on computation progress), and latency (the impact of communication delay on algorithm correctness). These analytical results are integrated into a communication requirement profile, a multi-dimensional vector or structured description that details the communication requirements of graph computation tasks, providing crucial information for subsequent resource allocation and scheduling.

[0099] Step S7.3: Based on the algorithm type identification results and communication requirement profile, analyze the computational load distribution characteristics and resource consumption patterns during task execution to obtain the load feature vector. The load feature vector is a multi-dimensional representation describing the resource requirements and execution characteristics of graph computation tasks, characterizing aspects such as computational density, memory usage, parallelism, and load balancing. The system first estimates the computational complexity based on the algorithm type and graph data characteristics: time complexity represents the number of computational operations required for algorithm execution, such as O(V+E) for breadth-first search, O(VlogV+E) for shortest path algorithm, or O(V²) for community detection algorithm, where V is the number of nodes and E is the number of edges; space complexity represents the memory space required for algorithm execution, including input data storage, intermediate results, and auxiliary data structures.

[0100] Specifically, this embodiment transforms the complexity expression into a concrete resource requirement estimate through the following steps: First, identify the variables in the complexity expression, such as the number of nodes V and the number of edges E, and obtain the actual values ​​of these variables from the input graph data; Second, substitute the variable values ​​into the complexity expression to calculate the theoretical number of operations. For example, for a breadth-first search algorithm with a time complexity of O(V+E), if V=10 million and E=100 million, the theoretical number of operations is approximately 110 million; Third, based on a performance model established from historical execution data, convert the theoretical number of operations into actual execution time. This model considers factors such as processor performance, cache efficiency, and parallelism. For example, regression analysis shows that each million operations require approximately 10 milliseconds of execution time; Fourth, based on the algorithm's memory access pattern and data structure size, estimate the actual memory requirement corresponding to the space complexity. For example, if the algorithm needs to store a state value for each node (assuming 8 bytes), then 10 million nodes require approximately 76MB of memory; Finally, combining the time and space requirements with the system resource status, generate a resource requirement vector containing information such as the number of CPU cores, memory capacity, and estimated execution time.

[0101] The system transforms complexity expressions into concrete resource requirement estimates through theoretical analysis and historical execution data. Then, it analyzes the characteristics of computational load distribution, i.e., how computation is distributed across different parts of the graph: uniform distribution indicates that computation is evenly distributed across all nodes, suitable for a balanced resource allocation strategy; degree-dependent distribution indicates that computation is positively correlated with node degree (number of connections), with higher-degree nodes requiring more computational resources; community-dependent distribution indicates that computation mainly occurs within specific communities or subgraphs, suitable for a locally clustered resource allocation strategy. The system also analyzes resource consumption patterns, including computationally intensive (high CPU utilization), memory-intensive (high memory utilization), or communication-intensive (high network bandwidth utilization), as well as the temporal characteristics of resource usage, such as stable (stable resource utilization), phased (different resources used in different phases), or fluctuating (frequent fluctuations in resource utilization). Furthermore, the system assesses load balancing and predicts load distribution under different partitioning strategies, providing a reference for subsequent task partitioning. The system analyzes parallelism characteristics, including inherent parallelism (the degree to which the algorithm itself can be parallelized), data dependencies (dependencies between computational units), and scalability (performance improvement with increased computational resources). The system integrates these analysis results into a load feature vector, which is a multi-dimensional vector that describes in detail the resource requirements and execution modes of graph computing tasks, providing a scientific basis for resource allocation and performance optimization.

[0102] Step S7.4: Finally, integrate the algorithm type identification results, communication requirement profile, and load feature vector, and generate a multi-dimensional task representation through a feature fusion network. Extract the essential features of the graph computation task to obtain the graph computation task feature vector. The feature fusion network is a deep learning architecture that integrates multi-source heterogeneous features into a unified representation, effectively capturing the interrelationships and synergistic effects between different features. The feature fusion network typically includes the following core components: a feature transformation layer that converts features from different sources and formats into a unified representation, such as converting categorical features into dense vectors through embedding and normalizing numerical features to the same scale; a feature interaction layer that captures the interactions between different features, such as the correlation between algorithm type and communication mode, and the dependency between computational load and graph structure; commonly used interaction modeling methods include attention mechanisms, adaptive gating, and bilinear pooling; a feature compression layer that reduces feature dimensions and removes redundant information using dimensionality reduction techniques (such as principal component analysis, autoencoders, or nonlinear projection), while retaining the essential features of the task; and a feature calibration layer that adjusts feature weights and representations based on historical execution data and expert knowledge to improve the accuracy and representativeness of the features. An end-to-end training approach is adopted, using the target task (such as task execution time prediction, resource utilization prediction, or optimal segmentation strategy identification) as a supervision signal to optimize the parameters of the feature fusion network. Training data comes from historical execution records, containing the execution status and performance metrics of different graph algorithms on various graph datasets. To handle newly emerging algorithm types or graph structures, transfer learning and few-shot learning techniques are employed to transfer knowledge from similar tasks. After processing by the feature fusion network, a graph computation task feature vector is obtained. This is a highly condensed yet information-rich representation that captures the essential characteristics of the task, including computational patterns, communication requirements, resource dependencies, and performance expectations. This feature vector provides a comprehensive and accurate basis for subsequent graph segmentation, resource allocation, and task scheduling, and is a key input for achieving efficient task processing.

[0103] Example 11:

[0104] The fault tolerance mechanism based on local rerouting includes: pre-calculating multiple shortest backup paths for each optical link based on the optical network topology map and the wavelength resource status table, and storing the backup path information in the optical cross-connect devices at both ends of the link to obtain a link backup path table; setting up a multi-level fault detection mechanism at the optical layer and IP layer according to the link backup path table to achieve millisecond-level link fault awareness and obtain a fault detection configuration; designing SRv6 segment routing policies for various fault scenarios based on the fault detection configuration and the optimal wavelength allocation scheme, defining head node behavior and fault response rules to obtain an SRv6 routing policy library; and, according to the SRv6 routing policy library, switching to the backup path through the local rerouting mechanism to achieve fault recovery and obtain a fault tolerance performance report when an optical link fault is detected.

[0105] Step S8.1: In the dedicated wavelength channel establishment phase, configuration instructions for optical cross-connect devices are first generated based on the optimal wavelength allocation scheme and then sent to the optical cross-connect devices in the network to establish dedicated wavelength channels. Optical cross-connect devices are core devices in optical networks, capable of switching optical signals by wavelength between different ports, achieving optical layer switching without photoelectric conversion. Modern OXCs are typically implemented based on technologies such as microelectromechanical systems (MEMS), liquid crystal switches, or wavelength selective switches (WSS), possessing high-speed, low-loss, and large-scale port switching capabilities. Configuration instruction generation is the process of converting the wavelength allocation scheme into executable commands for the devices. First, the abstract wavelength allocation scheme (the mapping between communication requirements and wavelength resources between subgraphs) is converted into a specific optical path description, including source nodes, target nodes, intermediate nodes, used wavelengths, and quality of service parameters. Then, the system generates specific configuration instructions for each OXC device on each optical path. These instructions specify the input port, output port, wavelength, and processing operation (such as pass-through, wavelength conversion, or splitting) of the optical signal. The configuration instructions use command formats supported by the devices, such as TL1, SNMP, NETCONF, or SDN-based OpenFlow. Considering the coordination between devices, the continuity and consistency of the end-to-end optical path are ensured. The command generation process also considers security and robustness, including command verification (checking the validity and consistency of commands), rollback mechanisms (restoring to the original state in case of configuration failure), and progressive configuration (configuring in stages to minimize impact). Command issuance is the process of transmitting commands to network devices and ensuring their execution. Commands are issued through the network management system, SDN controller, or direct device interfaces, and the configuration progress is monitored based on device responses and network status. A transactional configuration method is employed to ensure the atomicity of multi-device configuration (either all succeed or all fail). Through these steps, a dedicated wavelength channel, i.e., an end-to-end optical path, is established, providing the physical basis for efficient communication between subgraphs.

[0106] Step S8.2: Perform connectivity testing and bandwidth verification on the end-to-end optical path to ensure that the optical path meets communication requirements and obtain a verified optical network environment. Connectivity testing verifies whether the optical path is correctly established and reachable end-to-end. A multi-layered testing strategy is adopted: physical layer testing uses techniques such as optical power measurement, spectral analysis, or optical time domain reflectometer (OTDR) to detect the transmission quality of the optical signal, including signal strength, wavelength accuracy, optical signal-to-noise ratio (OSNR), and dispersion parameters; link layer testing uses optical layer loopback or optical layer probing techniques to verify the connectivity and integrity of the optical path; end-to-end testing sends test data packets between the devices at both ends of the optical path to verify the availability of the complete communication path. Bandwidth verification confirms that the optical path can provide the expected transmission capacity. First, the theoretical bandwidth capacity of the optical path is calculated using a theoretical model, based on factors such as fiber type, wavelength characteristics, signal modulation format, and transmission distance. Then, bandwidth performance is verified through actual data transmission tests, including constant bit rate (CBR) transmission tests, burst transmission tests, and stepped load tests. The system collects key performance indicators, such as effective bandwidth (actually available data transmission rate), latency (time delay in data transmission), jitter (the magnitude of latency variation), and packet loss rate (the proportion of data lost during transmission). The test results are compared with communication requirements to ensure that the performance of the optical path meets the needs of communication between subgraphs. If problems are found during testing, a fault diagnosis and repair process is initiated, such as reconfiguring the optical path, adjusting signal parameters, or allocating alternative resources. Through these tests and verifications, the established optical network environment is ensured to reliably support distributed graph computing, providing a high-quality communication foundation for subsequent task execution.

[0107] Step S8.3: Based on the subgraph set and its dependency matrix, and considering the processing power and memory capacity of available computing nodes, a resource matching algorithm is used to select the most suitable computing node for each subgraph, thus obtaining the mapping relationship between the subgraph and the computing node. The resource matching algorithm is an optimization technique that allocates computing tasks to appropriate computing resources, considering the matching degree between task requirements and resource capabilities, with the goal of maximizing system performance and resource utilization. The system first collects and analyzes the resource characteristics of available computing nodes: processing power includes CPU type, number of cores, clock speed, and architecture, directly affecting computing speed; memory capacity includes physical memory size, cache hierarchy, and bandwidth, affecting data processing capabilities and the performance of large-scale graph operations; the availability of acceleration hardware such as GPUs or FPGAs can significantly improve the execution efficiency of specific graph algorithms; network location includes the node's position in the network topology and its connectivity with other nodes, affecting communication efficiency. Then, the resource requirements of each subgraph are analyzed: computational requirements are estimated based on the subgraph size (number of nodes and edges), structural complexity, and algorithm characteristics; memory requirements consider the subgraph data size, intermediate result storage, and algorithm working set size; communication requirements are based on the dependency matrix, considering the communication frequency and data volume with other subgraphs. The resource matching process employs a multi-objective optimization method, simultaneously considering multiple objectives: computational efficiency (execution speed of the subgraph on the assigned nodes), communication efficiency (latency and bandwidth of communication between subgraphs), load balancing (the degree of load balance among computing nodes), and resource utilization (the effective use of system resources). The system may employ various optimization algorithms to achieve resource matching, such as the Hungarian algorithm (for optimal matching of bipartite graphs), genetic algorithms (for multi-constraint optimization), or simulated annealing (for avoiding local optima). The matching process also considers constraints, such as maximum node load limits, specific subgraph location requirements, or node fault tolerance. The matching result is a mapping relationship between subgraphs and computing nodes, specifying which computing node each subgraph is assigned to for execution. This mapping provides guidance for subsequent task distribution.

[0108] Step S8.4: Based on the mapping relationship between the subgraph and computing nodes, distribute the subgraph data and computing tasks to the corresponding computing nodes, configure communication parameters and synchronization strategies, and obtain a ready computing environment. Task distribution is the process of transmitting subgraph data and computing tasks to the assigned computing nodes. A multi-stage distribution strategy is adopted: first, the subgraph structure data, including the node set, edge set, and attribute data, is transmitted, which is the basic data for computation; then, the algorithm code and execution parameters are transmitted, including the algorithm implementation, configuration parameters, and initialization settings; finally, execution control information, such as start conditions, termination conditions, and monitoring strategies, is transmitted. Data transmission uses efficient data movement protocols, such as parallel data transmission, incremental updates, or compressed transmission, to reduce transmission overhead. A pre-distribution strategy may be adopted to distribute data in advance before the actual computation begins, reducing startup latency. Communication parameter configuration is the process of setting up the communication mechanism between subgraphs. The system configures communication parameters for each computing node based on the wavelength allocation scheme and optical path characteristics. These parameters include communication endpoints (which nodes to communicate with), communication interfaces (which network interfaces to use), communication protocols (how to format and transmit data), and communication strategies (when and how to trigger communication). The system also configures communication buffer size, retransmission strategies, and congestion control parameters to optimize communication performance. Synchronization strategy configuration is the process of setting up coordination mechanisms between nodes. Based on algorithm characteristics and communication requirements, the system selects and configures appropriate synchronization strategies: global synchronization strategies use a global fence mechanism, requiring all nodes to reach a specific execution point before continuing, suitable for strictly synchronous iterative algorithms; local synchronization strategies only require synchronization between interdependent nodes, reducing global coordination overhead; asynchronous strategies allow nodes to execute at their own speed, achieving coordination through message passing or shared state, suitable for asynchronous iterative or exploratory algorithms. Synchronization trigger conditions, timeout handling, and consistency guarantee mechanisms are also configured. Furthermore, monitoring and recovery configurations are implemented, setting up performance monitoring, fault detection, and recovery mechanisms to ensure the reliability of the computation process. Through these configuration and distribution steps, a complete computing-ready environment has been established. Each computing node has loaded the corresponding subgraph data and task code, and communication channels and synchronization mechanisms have been established, making it ready to start computing immediately.

[0109] Example 12:

[0110] The local rerouting mechanism further includes: establishing a mapping relationship between optical link failures and the impact of subgraph communication based on the SRv6 routing policy library and the subgraph set and its dependency matrix, to obtain a fault response mapping table; designing a wavelength resource dynamic adjustment algorithm based on the fault response mapping table to reallocate available wavelength resources when a fault occurs, to obtain a wavelength resource dynamic adjustment strategy; and designing a checkpoint mechanism for the subgraph computation state based on the wavelength resource dynamic adjustment strategy to realize periodic saving of the computation state and recovery after a fault, to obtain a computation state protection scheme.

[0111] Step S8.5: During the distributed graph computation phase, the subgraph data structure and computation environment are first initialized on each computation node, and the parallel computation framework is started to obtain the initialized subgraph task. Subgraph data structure initialization is the process of constructing a graph data representation suitable for efficient computation. Based on algorithm characteristics and access patterns, the system selects a graph data structure (such as a spaced tree, quadtree, or attribute graph model): an adjacency list represents each node and its list of neighboring nodes, suitable for sparse graphs and frequent neighbor access operations; an adjacency matrix uses a matrix to represent the connection relationships between nodes, suitable for dense graphs and frequent edge existence queries; compressed sparse row or compressed sparse column formats use compact array representations, providing memory-efficient and cache-friendly access patterns; specialized graph data structures such as spaced trees, quadtrees, or attribute graph models are optimized for specific types of graph algorithms. Subgraph data, including node sets, edge sets, and attribute data, is loaded, the selected data structure is constructed, and the data is preprocessed, such as creating indexes, sorting edge lists, or pre-calculating frequently used values, to improve subsequent computation efficiency. Computation environment initialization prepares the software environment and resources required for algorithm execution. Configure memory management strategies, such as memory pool allocation, large page support, or NUMA-aware memory layout, to optimize memory access performance; set thread or process parameters, such as thread count, affinity settings, or priority strategies, to fully utilize multi-core processors; configure exception handling mechanisms, such as error detection, fault recovery, or timeout handling, to ensure computational reliability. Parallel computing framework startup initializes and starts the software platform supporting distributed graph computing. Various parallel computing frameworks may be used, such as Pregel (a synchronous iterative framework based on the BSP model), GraphX ​​(a graph processing framework based on Spark), PowerGraph (a framework optimized for natural graphs), or a custom distributed graph processing system. This involves loading framework components, establishing inter-node communication channels, initializing synchronization mechanisms, loading algorithm code, and setting execution parameters. The framework startup process also includes role assignment (such as master node, worker nodes), state consistency checks, and ready-to-work status broadcasting. Through these initialization steps, a complete execution environment is established on each computing node; data is loaded, the framework is started, and subgraph tasks are ready to begin computation immediately.

[0112] Step S8.6: Based on the initialized subgraph task, perform parallel computation within the subgraph, updating the states of nodes and edges according to the graph algorithm to obtain local computation results. When the subgraph computation requires access to boundary node data, a request is sent and a response is received through the optical layer direct data transmission mechanism to complete cross-subgraph data exchange and obtain updated boundary data. Parallel computation within the subgraph is the process of executing graph algorithms in parallel on a single computation node using a multi-core processor. Appropriate parallel strategies are adopted based on algorithm characteristics and subgraph structure: node parallelism distributes computations of different nodes to different threads, suitable for highly independent computations; edge parallelism distributes operations on different edges to different threads, suitable for edge-up-centric algorithms; task parallelism decomposes the algorithm into subtasks for parallel execution, suitable for complex multi-stage algorithms; and data parallelism applies the same operations to different data partitions, suitable for regular computation patterns. Various parallel optimization techniques are used, such as work-stealing to balance thread load, task batching to reduce scheduling overhead, and vectorized computation using SIMD instructions for acceleration. The computation process strictly follows the logic of the graph algorithm, updating the states of nodes and edges. For example, in the PageRank algorithm, each node calculates a new ranking based on the ranking values ​​of its incoming edge nodes; in the shortest path algorithm, a node updates its distance estimate to the source node; and in the community detection algorithm, a node may change its community affiliation. These updates can be synchronous (all nodes update using the values ​​from the previous round simultaneously) or asynchronous (nodes update using the latest available values), depending on the algorithm design. Direct optical layer data transmission is an efficient cross-subgraph communication mechanism that utilizes dedicated wavelength channels to directly transmit data at the optical layer, avoiding the processing overhead of multi-layered protocol stacks in traditional IP networks. When computation requires access to data from boundary nodes located in other subgraphs, a data request message is constructed, containing information such as the target node identifier, the requested data type, and the request priority. The request message is directly encapsulated as an optical transmission unit and sent to the target node through a pre-configured wavelength channel. The receiving node processes the request, obtains the required data, constructs a response message, and sends it back to the requesting node through a return channel. This mechanism significantly reduces communication latency and improves bandwidth utilization, making it particularly suitable for frequent small data exchange scenarios. It also implements several optimization techniques, such as request batching (merging multiple requests into a single transmission unit), data caching (caching frequently accessed remote data), prefetching (predicting future data needs based on access patterns and retrieving them in advance), and asynchronous communication (not blocking computation threads while waiting for responses), further improving communication efficiency. Through this collaborative execution mode of computation and communication, the system efficiently completes distributed graph computation, obtaining updated node and edge states.

[0113] Step S8.7: Determine the iteration termination condition based on the feature vector of the graph computation task. Use distributed state synchronization to determine if the termination condition is met, complete multiple rounds of iterative computation, and obtain the final computation result. During the execution of the graph computation task, the system collects key indicators such as computation time, communication latency, and resource utilization, generates a performance report, and obtains computation results and performance statistics. The iteration termination condition is the standard for determining when to stop iterative computation. Based on the feature vector of the graph computation task, the system determines suitable termination condition types: convergence conditions require that the state change between consecutive iterations is less than a preset threshold, suitable for iterative convergence algorithms such as PageRank; fixed iterations simply execute a predetermined number of iterations, suitable for algorithms with theoretical guarantees or time-sensitive applications; goal achievement conditions terminate when a specific goal is met (such as finding all nodes that meet the conditions), suitable for search or optimization algorithms; combined conditions combine multiple condition logics to provide more flexible control. Distributed state synchronization is a mechanism for coordinating the states of each subgraph and checking termination conditions in a distributed environment. Global aggregation operations are used to collect state summaries of each subgraph, such as state change, current error, or goal completion. These aggregation operations are usually implemented as tree reduction or butterfly networks to minimize communication overhead. A global condition check is performed to determine if the termination condition is met. If the condition is met, a termination signal is broadcast, and each node completes its final calculation and prepares to output the result. If the condition is not met, the system begins a new round of iteration. This process is repeated until the termination condition is met, completing multiple rounds of iterative calculation. Performance monitoring and statistics are the process of collecting and analyzing system operating metrics to provide a basis for performance optimization and problem diagnosis. During execution, the system continuously monitors various performance metrics: computation metrics include computation time (execution time of each iteration), CPU utilization, memory usage, and cache hit rate; communication metrics include communication latency (time from data request to response), bandwidth utilization, message quantity and size distribution, etc.; algorithm metrics include convergence speed (progress in each iteration), algorithm-specific metrics (such as the L1 norm of PageRank), and intermediate result distribution, etc. The system uses lightweight monitoring techniques, such as sampling monitoring, event counters, or hardware performance counters, to minimize monitoring overhead. The collected data is processed and aggregated to form a comprehensive performance report, including an overall performance overview (such as total execution time and resource utilization), detailed performance breakdown (such as computation time vs. communication time), bottleneck analysis (identifying performance limiting factors), and optimization suggestions (improvement schemes targeting specific bottlenecks). The final calculation results, along with performance statistics, are output for user analysis and application. This performance data can also be fed back to the system's task analysis and resource allocation components to optimize future task processing strategies.

[0114] Example 13:

[0115] The configuration of the optical cross-connect device to establish an end-to-end dedicated wavelength channel includes: based on the optimal wavelength allocation scheme and available computing resources, selecting the most suitable computing node for each subgraph using a matching algorithm to obtain the mapping relationship between the subgraph and the computing node; according to the mapping relationship between the subgraph and the computing node and the optimal wavelength allocation scheme, setting the cross-connection status of the optical cross-connect device through the controller to obtain the configured optical path; and verifying the connectivity and bandwidth of the wavelength channel through end-to-end testing for the configured optical path to obtain a list of available wavelength channels.

[0116] Step S9.1: In the wavelength resource mapping stage, based on the optical network topology and communication requirements, K candidate paths that meet the bandwidth and delay requirements are first identified using a multi-constraint path computation algorithm, forming a path pool and obtaining a multi-constraint candidate path set. Multi-constraint path computation is a technique for finding paths in a network that satisfy multiple resource constraints, extending the traditional shortest path algorithm by considering multiple constraints simultaneously. In an optical network environment, the main path constraints include bandwidth constraints (the path must provide sufficient transmission capacity), delay constraints (end-to-end delay does not exceed a specific threshold), reliability constraints (the path must meet specific availability requirements), and resource constraints (the path must use available network resources). The multi-constraint path computation problem has been proven NP-complete in computational complexity theory, meaning that the computational complexity of finding an exact solution increases exponentially with network size; therefore, systems typically employ heuristic or approximate algorithms. The multi-constraint path computation algorithm implemented in the system typically includes the following core components: constraint preprocessing, which analyzes the original constraints, removes redundant constraints, simplifies constraint expression, and improves computational efficiency; network graph transformation, which converts the multi-constraint problem into a form suitable for specific algorithms, such as converting certain constraints into edge weights or node attributes; path search, which is the core computational step, may employ various algorithms, such as a modified Dijkstra's algorithm (considering multiple weights), the k-shortest path algorithm (finding k shortest paths and then filtering for paths that satisfy the constraints), Lagrangian relaxation (converting the multi-constraint problem into a sequence of single-constraint problems), or genetic algorithms (using evolutionary computation to find an approximate optimal solution); and feasibility checks, which verify whether the found paths satisfy all constraints and eliminate infeasible paths. When performing multi-constraint path computation, the communication requirements between each subgraph pair are first defined, including bandwidth requirements (based on data exchange volume), latency requirements (based on computational dependencies and time sensitivity), and priorities (based on the impact of communication on algorithm performance). Then, path calculation is performed on the optical network topology graph to find at most K candidate paths that satisfy the constraints for each pair of subgraphs that need to communicate. K is a system parameter, usually with a value of 3-5, which is sufficient to provide diverse options without excessive computation. These K paths form a path pool, providing candidate schemes for subsequent wavelength allocation, increasing the flexibility and success rate of resource allocation.

[0117] Step S9.2: For each path in the multi-constraint candidate path set, check the wavelength resource availability on each link, and apply the first-fit wavelength allocation algorithm to allocate the corresponding wavelength to each path, obtaining a path-wavelength combination scheme. Wavelength resource checking is the process of verifying the wavelength availability of each link on the candidate path. Each fiber link in an optical network supports multiple wavelength channels, and each wavelength can only be used by one optical path at a time. Query the wavelength resource status table to obtain the current status (idle, occupied, or reserved) of each wavelength on each link. In a distributed graph computing environment, wavelength allocation needs to consider wavelength continuity constraints, i.e., an optical path must use the same wavelength on all its traversed links (unless the network supports wavelength conversion). This constraint significantly increases the complexity of allocation because wavelengths available on all links of the path must be found. Perform a wavelength availability check on each candidate path to identify the set of available wavelengths that satisfy the continuity constraint. The first-fit wavelength allocation algorithm is a commonly used wavelength selection strategy that allocates the lowest indexed wavelength that meets the conditions to the optical path. The algorithm first sorts all available wavelengths in a predefined order (usually ascending order of wavelength indices), then selects the first wavelength in the sequence that is available on all links of the path. This simple greedy strategy has advantages such as ease of implementation, fast decision-making, and minimal resource fragmentation, and is widely used in practical systems. Besides the basic first-fit strategy, several variations or alternative algorithms can be implemented: minimum load selects the wavelength with the lightest current load to balance network load; random adaptation randomly selects from available wavelengths to avoid resource contention; minimum use selects the wavelength used least in the network to improve wavelength utilization balance; maximum sum selects the wavelength that maximizes the total weight of the path, suitable for weighted networks. A wavelength allocation algorithm is applied to each candidate path. If an available wavelength is successfully found, a path-wavelength combination scheme is formed; if a path cannot find an available wavelength that satisfies the continuity constraint, the path is marked as infeasible and removed from the candidate set. Each path-wavelength combination scheme contains complete routing information (nodes and links traversed) and wavelength allocation decisions (wavelength channels used), representing a feasible end-to-end optical communication channel. Through this process, the system obtains multiple feasible path-wavelength combination schemes, providing a candidate set for subsequent scheme evaluation and selection.

[0118] Step S9.3: Based on the performance metrics of the path-wavelength combination scheme, including end-to-end delay, link utilization, and wavelength continuity, a weighted scoring function is used to comprehensively evaluate each scheme and select the optimal scheme to obtain the optimal wavelength allocation scheme. Performance metric evaluation is a process of quantifying the quality of each path-wavelength combination scheme, considering various factors affecting communication efficiency and system performance. The main performance metrics for system evaluation include: end-to-end latency, which is the time required for an optical signal to propagate from the source node to the target node. It consists of propagation delay (dependent on fiber length and light speed), switching delay (processing time in OXC equipment), and queuing delay (occurring during network congestion). Lower latency is crucial for time-sensitive graph computations. Link utilization measures the ratio of the number of wavelengths allocated to a link to the total number of wavelengths, reflecting resource utilization efficiency and load distribution. Balanced link utilization helps avoid network bottlenecks. Wavelength continuity assesses the consistency of wavelength allocation along a path. Completely continuous wavelength allocation (all links using the same wavelength) is ideal, followed by partially continuous (requiring a finite number of wavelength transitions), while discontinuous allocation (frequent wavelength transitions) increases complexity and cost. Path length is the number of links or physical distance traversed by the path. Shorter paths generally mean lower latency and higher reliability. Resource consumption assesses the total network resources consumed by the scheme, including the number of wavelengths and links. Resource-efficient schemes are more scalable. The weighted scoring function is a mathematical model that combines multiple performance metrics into a single score, allowing for direct comparison of schemes. The weighted scoring function implemented in the system typically adopts a linear weighted sum form. Each performance indicator is first normalized (converted to the 0-1 range), then multiplied by its corresponding weight coefficient, and finally all weighted indicators are summed to obtain the total score. The weight coefficients reflect the relative importance of different indicators and can be dynamically adjusted based on the characteristics of the graph computation task and communication requirements. For example, for time-sensitive tasks, the weight of latency indicators will increase; for large-scale data exchange, the weights of bandwidth and resource efficiency will increase. The weighted scoring function is applied to all feasible path-wavelength combination schemes to calculate the comprehensive score of each scheme. Then, the scheme with the highest score is selected as the optimal wavelength allocation scheme. This scheme represents the best choice under the comprehensive consideration of multiple performance indicators, providing efficient and reliable optical network connections for inter-subgraph communication. If multiple schemes have similar scores, secondary factors such as scheme complexity, implementation difficulty, or compatibility with existing allocations may be considered to make the final decision.

[0119] Example 14:

[0120] The data exchange between subgraphs via the dedicated wavelength channel includes: based on the ready computing environment, initiating computational processing within the subgraph on each node using a parallel computing framework, executing a local graph algorithm to obtain local computation results; exchanging boundary node data between subgraphs using a direct optical layer data transmission mechanism based on the local computation results and the available wavelength channel list to obtain updated boundary data; ensuring consistency of computation across subgraphs using a synchronization fence mechanism to control computation progress and obtain synchronization status information; managing the iteration process of computation using an iteration controller based on the synchronization status information to determine whether to continue to the next round of computation and obtain an iteration control signal; returning to the subgraph's internal computational processing steps when the iteration control signal indicates termination, and entering the result integration stage to obtain an iteration execution decision; merging the distributed computation results using a result integration algorithm based on the computation results of each subgraph after the iteration execution decision indicates termination, while simultaneously collecting performance statistics to obtain the computation results and performance statistics.

[0121] Step S10.1: In the optical cross-connect equipment configuration phase, the optimal wavelength allocation scheme is first parsed, extracting the node sequence, link information, and wavelength allocation decisions for each optical path to generate a network-level configuration plan and obtain a configuration instruction set. Scheme parsing is the process of converting the abstract wavelength allocation scheme output by the optimization algorithm into specific network configuration information. The optimal wavelength allocation scheme is usually represented by a mathematical model or graph structure, containing the mapping relationship between subgraph communication requirements and network resources. The system parses this scheme, extracting complete information for each optical path: the node sequence lists all network nodes traversed by the optical path, from the source node to the target node, including all intermediate forwarding nodes; the link information details the fiber optic links connecting these nodes, including link identification, physical characteristics (such as length, attenuation), and capacity limitations; the wavelength allocation decision specifies the wavelength channels used on each link, considering wavelength continuity constraints and resource availability. The network-level configuration plan is a global view of the configuration blueprint, integrating the configuration requirements of all optical paths, resolving potential resource conflicts, and determining the timing and dependencies of the configuration. Several factors are considered when generating the configuration plan: configuration order optimization considers the dependencies between nodes and links to determine the optimal configuration sequence and avoid deadlocks and resource contention; resource conflict detection identifies and resolves concurrent configuration requests from multiple optical paths to the same device or port to ensure configuration consistency; network impact minimization arranges configuration operations to minimize their impact on existing services, such as prioritizing configurations that do not affect existing traffic; rollback point setting sets checkpoints in the configuration sequence to allow orderly rollback in case of configuration failure, ensuring network state consistency. The configuration instruction set is a set of device-specific configuration commands that translates the network-level configuration plan into device-executable instructions. For each optical cross-connect device, specific configuration instructions are generated, including port configuration (setting the operating mode and parameters of input / output ports), cross-connect configuration (establishing internal connections between ports), wavelength configuration (setting wavelength selection and filtering parameters), and monitoring configuration (setting performance monitoring and alarm thresholds), etc. These instructions use command languages ​​and protocol formats supported by the device, such as TL1, CLI, SNMP, or NETCONF. Configuration verification instructions are also generated to confirm the correct application and effectiveness of the configuration. Through this parsing and conversion process, the system obtains a complete set of configuration instructions, providing detailed guidance for subsequent device configuration.

[0122] Step S10.2: Based on the configuration instruction set, create a distributed signaling session for each wavelength channel, and establish optical channels segment by segment according to the node dependency order to obtain end-to-end wavelength channels. A distributed signaling session is a mechanism for coordinating multiple network devices to jointly complete end-to-end resource allocation and channel establishment. In optical networks, commonly used signaling protocols include Resource Reservation Protocol-Traffic Engineering and Constrained Routing Label Distribution Protocol, which provide standardized methods for requesting, allocating, and releasing network resources. An independent signaling session is created for each wavelength channel, containing a unique identifier, path information, resource requirements, and quality of service parameters. The core advantage of distributed signaling lies in its reliability and scalability, enabling efficient coordination of resource allocation even in large-scale networks. Node dependency order is a strategy for determining the execution order of configuration operations; it analyzes the dependencies between nodes based on network topology and optical path characteristics. During optical path establishment, signaling and configuration typically follow a specific order pattern: the source-destination pattern starts from the source node and configures each node sequentially along the path until the target node; the destination-source pattern starts from the target node and configures in reverse to the source node; the bidirectional pattern starts configuration at the source and target nodes simultaneously, progressing towards intermediate nodes. A flexible configuration mode is selected based on network status and optical path characteristics. For example, the configuration mode that prioritizes reducing configuration time and resource contention is preferred. Segment-by-segment establishment is a progressive channel establishment method that decomposes the end-to-end optical path into multiple consecutive segments, configuring and verifying them one by one. After each segment is configured, the system performs local verification to confirm the correct establishment of that segment before proceeding to the next segment. The advantage of this method lies in error isolation and rapid recovery capabilities; if a segment configuration fails, only that segment needs to be reconfigured without affecting the successfully established parts. During the configuration process, the execution status is monitored in real time, recording the completion status, return code, and execution time of each configuration operation. If a configuration error or timeout is detected, the system triggers an exception handling process, such as retrying the configuration, attempting to replace resources, or rolling back the completed configuration. When all segments are successfully configured, the system performs end-to-end verification to confirm the establishment of the complete channel. Through this distributed signaling and segment-by-segment establishment method, an end-to-end wavelength channel is established efficiently and reliably, providing a physical connection for communication between subgraphs.

[0123] Step S10.3: Initiate bidirectional connectivity verification. This involves sending a test optical signal to detect channel quality and end-to-end reachability, recording optical power, bit error rate, and delay data, and obtaining a wavelength channel quality report. Bidirectional connectivity verification is a test process that confirms the optical channel can transmit data normally in both directions. In optical networks, bidirectional communication typically requires two independent optical paths (one in each direction) or one optical path supporting bidirectional transmission. The connectivity verification ensures that both directions are functioning correctly, avoiding unidirectional connectivity (i.e., data can only be sent from A to B, but not from B to A). The test optical signal is a specialized optical signal used for network testing and verification. It has specific patterns and characteristics that facilitate identification and analysis at the receiving end. The generated test signal typically contains predefined data patterns (such as pseudo-random bit sequences, PRBS), specific frame formats, and identification markers. During the test, the test signal is injected at the source end of the optical path, and the signal characteristics are received and analyzed at the target end to evaluate channel performance. Collect and record various performance indicators: Optical power measures the intensity of the received optical signal, usually measured in dBm (decibel-milliwatt). Too low a power may result in incorrect signal detection, while too high a power may lead to nonlinear effects. Optical signal-to-noise ratio (OSNR) measures the ratio of signal strength to noise level; a higher OSNR indicates better signal quality. Bit error rate (BER) is calculated as the ratio of erroneous bits to total transmitted bits, a key performance indicator for digital communication systems; modern optical systems typically require a BER below 10. -12 End-to-end delay measures the transmission time of a signal from the source node to the target node, including propagation delay, processing delay, and queuing delay. Jitter measures the magnitude of delay variation; excessive jitter can lead to synchronization problems and difficulties in data recovery. Dispersion measures the difference in propagation speed between different frequency components; excessive dispersion can cause signal broadening and distortion. The system collects these metrics and performs statistical analysis, such as calculating the average, standard deviation, maximum / minimum values, and frequencies exceeding thresholds. The analysis results are integrated into a wavelength channel quality report, a structured document that details the results, analysis data, and conclusive evaluations for each test item. This report provides crucial information for subsequent channel quality assessment and problem diagnosis.

[0124] Step S10.4: Analyze the wavelength channel quality report to determine whether the channel performance meets the preset quality of service (QoS) requirements. For channels that do not meet the requirements, reconfigure or reallocate resources to obtain verified wavelength channels. QoS requirements are the minimum standards that optical channel performance must meet; different types of applications and services may have different requirements. In a distributed graph computing environment, QoS requirements typically include bandwidth requirements (ensuring sufficient data transmission capacity), latency requirements (limiting end-to-end transmission time), reliability requirements (ensuring the correctness and continuity of data transmission), and stability requirements (limiting performance fluctuations). The system first loads the preset QoS requirements, which are determined based on the characteristics and communication needs of the graph computing task and may vary depending on the task type and priority. Then, the performance indicators in the wavelength channel quality report are compared with these requirements to evaluate whether the channel meets all requirements. Performance evaluation is the process of comparing measurement results with required thresholds. Acceptance standards are defined for each performance indicator, such as optical power must be within a specific range (e.g., -25dBm to -5dBm) and bit error rate must be below a specific threshold (e.g., 10). -12 The criteria include requirements such as latency being less than a specific value (e.g., 10 milliseconds). Each channel's eligibility is evaluated using methods such as rule engines or decision trees. Evaluation results are categorized into three types: fully qualified (all indicators meet requirements), partially qualified (some indicators meet requirements, some are close to the threshold), and unqualified (one or more key indicators do not meet requirements). For unqualified channels, a reconfiguration or resource reallocation process is initiated. Reconfiguration is the process of adjusting the parameters of existing channels to improve performance, including adjusting signal power, optimizing equipment parameters, or modifying transmission modes. For example, for a channel with insufficient power, the system may increase transmit power or adjust amplifier gain; for a channel with excessive dispersion, a dispersion compensation module may be enabled. Resource reallocation is the process of allocating new network resources to channels to resolve performance issues, including selecting new paths, allocating different wavelengths, or using higher-quality equipment ports.

[0125] Step S10.5: Based on the verified wavelength channels, configure the protection switching mechanism of the optical network, set the optical path recovery strategy and backup resources, improve the reliability of the wavelength channels, and obtain an end-to-end optical path with protection capabilities. The protection switching mechanism is a technology that quickly switches to a backup path when the primary path fails, ensuring service continuity and data transmission reliability. Common protection switching architectures in optical networks include: 1+1 protection: each primary path is equipped with a backup path, data is sent simultaneously to both paths, and the receiver selects the signal with better quality; 1:1 protection: each primary path is equipped with a backup path, but the backup path can transmit low-priority services under normal circumstances, and only transmits protection services when the primary path fails; 1:N protection: multiple primary paths share a backup path, which is activated when any primary path fails; ring network protection: utilizes the characteristics of a ring topology to achieve protection by transmitting data in different directions of the ring. Based on the importance of communication requirements and network resource conditions, select an appropriate protection architecture for each wavelength channel. For critical communication links, the more reliable 1+1 protection may be used; for ordinary links, the more resource-efficient 1:N protection may be used. Optical path recovery strategies define the specific methods and steps for restoring service after a fault is detected. Implemented recovery strategies typically include the following elements: a fault detection mechanism that quickly identifies path faults by continuously monitoring indicators such as optical signal quality, received power, or bit error rate (BER), with typical detection times in milliseconds; switching trigger conditions that define the specific conditions for initiating protection switching, such as signal loss, synchronization loss, or continuous degradation of service quality; a switching execution process that includes steps such as notifying all relevant nodes, activating the backup path, redirecting traffic, and verifying the new path; a recovery strategy that specifies whether to switch back to the primary path or continue using the backup path after the primary path is restored; and a coordination mechanism that ensures multiple protection domains in the network work together to avoid contention and instability. Backup resources are network resources specifically reserved for protection switching, including backup wavelengths, backup fiber paths, or backup equipment ports. These backup resources are allocated and managed to ensure they are available and in good condition when needed. Backup resource allocation considers risk diversification principles, such as avoiding primary and backup paths sharing the same physical facilities or fault domains, enhancing the system's resilience to single points of failure. Resource sharing strategies may also be implemented, allowing multiple primary paths that are unlikely to fail simultaneously to share backup resources, improving resource utilization efficiency. By configuring these protection mechanisms and strategies, the reliability and fault recovery capability of the wavelength channel are significantly improved, enabling it to maintain service continuity in abnormal situations such as equipment failure or link interruption, and providing a stable and reliable communication foundation for distributed graph computing.

[0126] Step S10.6: Map the protected end-to-end optical path to the upper-layer application, providing an optical layer communication interface for each communication subgraph pair, abstracting the underlying complexity to obtain the end-to-end optical path. The optical layer communication interface is an abstraction layer connecting the physical optical network and the upper-layer application, hiding the complexity of the underlying network and providing simple and consistent communication services for the application. The implemented optical layer communication interface typically includes the following components: a service abstraction layer that abstracts the physical optical channel into logical communication services, such as point-to-point links or bandwidth-guaranteed connections, hiding the underlying wavelength, routing, and protection details; communication primitives that provide basic communication operations, such as connection establishment, data transmission, reception acknowledgment, and connection release, through which the application interacts with the communication system; a quality of service interface that allows the application to specify and monitor the quality requirements of the communication service, such as bandwidth requirements, latency limits, or reliability levels; and a monitoring and management interface that provides functions such as communication status query, performance monitoring, and anomaly notification, enabling the application to perceive changes in the communication environment. The optical layer communication interface is a communication interface customized for a specific subgraph pair, optimized for the communication characteristics and requirements of that subgraph pair, providing the most suitable service experience. A dedicated interface is provided for each pair of subgraphs requiring communication. This interface corresponds one-to-one with the underlying optical path, ensuring dedicated communication resources and performance guarantees. The dedicated interface is a customized communication interface for specific communication needs, optimized for the communication characteristics and requirements of a particular subgraph pair. The advantages of the dedicated interface lie in quality of service assurance, security isolation, and performance optimization, avoiding resource contention and interference that may arise from multiple applications sharing the same interface. Underlying complexity abstraction is a technique that hides network details and provides a simplified view. The implemented abstraction mechanisms include: topology abstraction, which simplifies the physical network topology into a logical connection graph, hiding intermediate nodes and physical link details; resource abstraction, which converts physical resources such as wavelength and bandwidth into application-level concepts such as capacity and rate; fault abstraction, which converts network faults and recovery processes into service state changes, such as temporary unavailability or performance degradation; and protocol abstraction, which hides the underlying signaling and transport protocols, providing unified communication semantics. Through this mapping and abstraction, the complex optical network infrastructure is successfully transformed into an easy-to-use communication service. Applications only need to focus on the logical requirements of data exchange, without needing to understand and handle the underlying optical network details. This separation of concerns design greatly simplifies the development and optimization of distributed graph computing applications, allowing developers to focus on algorithms and data processing logic while delegating communication details to a dedicated optical network subsystem.

[0127] like Figure 2 As shown, the present invention also provides a wavelength-granularity task partitioning and scheduling system for large-scale graph computation, comprising:

[0128] Architecture building module 10 is used to obtain the graph computing tasks and network topology information submitted by the user, build a system architecture consisting of a task analysis module, a graph segmentation engine, a wavelength resource manager and a scheduler executor, complete system initialization, and obtain the initialized system running environment;

[0129] Task analysis module 20 is used to identify the algorithm type, analyze the communication mode, and evaluate the computational complexity of the graph computing task submitted by the user based on the initialized system operating environment, and obtain the graph computing task feature vector.

[0130] Graph segmentation module 30 is used to calculate task feature vectors based on the graph, apply a multi-level graph segmentation algorithm to consider the communication cost between nodes and the balance of computational load, and divide the large-scale graph data into multiple subgraph units to obtain a set of subgraphs and their dependency matrix.

[0131] The wavelength resource management module 40 is used to analyze the status of available wavelength resources in the optical network based on the subgraph set and its dependency matrix, establish the mapping relationship between subgraph communication requirements and wavelength resources, and obtain the optimal wavelength allocation scheme.

[0132] The scheduling and execution module 50 is used to configure the optical cross-connect device to establish an end-to-end dedicated wavelength channel for the optimal wavelength allocation scheme, distribute the subgraph tasks to the corresponding computing nodes, and obtain a ready computing environment.

[0133] The data exchange and calculation module 60 is used to perform data exchange between subgraphs through the dedicated wavelength channel based on the ready computing environment, and simultaneously perform parallel calculations within the subgraphs to obtain calculation results and performance statistics.

[0134] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A wavelength granularity task partitioning and scheduling method for large-scale graph computation, characterized in that, include: The system obtains user-submitted graph computing tasks and network topology information, establishes a system architecture consisting of a task analysis module, a graph segmentation engine, a wavelength resource manager, and a scheduler, completes system initialization, and obtains the initialized system runtime environment. Based on the initialized system operating environment, the algorithm type identification, communication mode analysis, and computational complexity evaluation of the graph computing tasks submitted by the user are performed to obtain the graph computing task feature vector. Based on the graph, a task feature vector is calculated. A multi-level graph segmentation algorithm is applied, considering inter-node communication costs and computational load balancing, to divide the large-scale graph data into multiple subgraph units, resulting in a set of subgraphs and their dependency matrices. This includes: calculating the task feature vector based on the graph; selecting an enhanced partition set coordination algorithm as the graph segmentation strategy; setting a difference threshold parameter and a coordination factor to obtain an enhanced partition set coordination algorithm parameter set; constructing a graph correlation matrix and extracting key node features based on the original graph data and the enhanced partition set coordination algorithm parameter set; establishing a hierarchical representation of the graph to obtain its feature representation and hierarchical structure; and constructing a coordination tree by recursively applying the enhanced partition set coordination algorithm to the graph's feature representation and hierarchical structure. The structure is constructed to obtain an initial coordination tree; for the initial coordination tree, the difference metric of each node in the tree is calculated, the optimal split point is identified, and a coordination tree with split labels is obtained; based on the coordination tree with split labels, the connection characteristics of the boundary nodes are analyzed, and a selective node replication strategy is adopted to optimize the cross-subgraph communication mode, resulting in an optimized subgraph set; based on the optimized subgraph set, a communication cost model is established to evaluate the data exchange volume between subgraphs, and subgraph pairs with communication costs higher than a preset threshold are selectively merged to obtain a subgraph set that balances communication and computation; based on the subgraph set that balances communication and computation, the data dependencies between subgraphs are analyzed, a fine-grained dependency matrix is ​​constructed, and the subgraph set and its dependency matrix are obtained; Based on the subgraph set and its dependency matrix, the available wavelength resources in the optical network are analyzed, a mapping relationship between subgraph communication requirements and wavelength resources is established, and the optimal wavelength allocation scheme is obtained. This includes: estimating the communication frequency and data volume based on the subgraph set and its dependency matrix, quantifying the communication requirements between subgraphs, and obtaining the subgraph communication requirement matrix. Based on the wavelength resource status table and the subgraph communication demand matrix, the optimal subgraph-wavelength mapping relationship is calculated using a multi-objective optimization algorithm to obtain the wavelength allocation scheme. For the optimal wavelength allocation scheme, an optical cross-connect device is configured to establish an end-to-end dedicated wavelength channel, and the subgraph tasks are distributed to the corresponding computing nodes to obtain a ready computing environment. Based on the prepared computing environment, data exchange between subgraphs is performed through the dedicated wavelength channel, while parallel computing within the subgraph is executed to obtain computing results and performance statistics.

2. The method of claim 1, wherein, The process involves acquiring user-submitted graph computation tasks and network topology information, establishing a system architecture consisting of a task analysis module, a graph segmentation engine, a wavelength resource manager, and a scheduler, completing system initialization, and obtaining the initialized system runtime environment, including: Based on the characteristics of graph computing scenarios and optical network features, system requirements analysis is conducted to determine the system functional boundaries and performance targets, resulting in a system requirement specification. Based on the system requirements specification, the interface specifications of the task analysis module, graph segmentation engine, wavelength resource manager and scheduling executor were designed using the modular design principle, resulting in the system component design document; Based on the system component design document, each functional module is deployed in a distributed environment to complete system startup configuration and resource registration, thereby obtaining the initialized system operating environment.

3. The method according to claim 1, characterized in that, The process of identifying the algorithm type, analyzing the communication pattern, and evaluating the computational complexity of the graph computation task submitted by the user yields a feature vector for the graph computation task, including: Based on the graph computing tasks submitted by users, analyze the computing models and algorithm types, identify the computing characteristics of the graph computing tasks, and obtain task type identifiers. Based on the task type identifier and the input graph data of the graph computation task submitted by the user, the topological characteristics of the input graph data are analyzed, the structural features of the input graph data are identified, and a graph structure feature vector is obtained. Based on the graph structure feature vector and the task type identifier, the data exchange frequency and communication volume during the calculation process are predicted to obtain a communication mode description. Based on the communication mode description and the scale of the input graph data, the computational, storage, and network resources required for task execution are estimated to obtain a resource requirement vector; Based on the task type identifier, the graph structure feature vector, the communication mode description, and the resource requirement vector, a comprehensive description of task features is generated through a multi-dimensional feature fusion algorithm to obtain the graph computation task feature vector.

4. The method according to claim 1, characterized in that, The applied multi-level graph segmentation algorithm considers the communication cost between nodes and the balance of computational load, dividing large-scale graph data into multiple subgraph units to obtain a set of subgraphs and their dependency matrix, including: The task feature vector is calculated based on the graph, and the graph segmentation strategy is selected through the decision tree algorithm to obtain the segmentation strategy parameters; Based on the segmentation strategy parameters and the original graph data, an initial subgraph partition is generated using a greedy allocation algorithm, resulting in an initial subgraph set. Based on the initial set of subgraphs, the connection relationship between subgraphs is optimized by an iterative boundary node migration algorithm to reduce the communication cost across subgraphs and obtain the optimized subgraph structure. For the optimized subgraph structure, the computational complexity distribution of each subgraph is adjusted by calculating load estimation and rebalancing algorithms to obtain a load-balanced set of subgraphs; Based on the load-balanced subgraph set, the data dependencies and communication requirements between subgraphs are analyzed, a subgraph communication matrix is ​​constructed, and the subgraph set and its dependency matrix are obtained.

5. The method according to claim 1, characterized in that, The analysis of the available wavelength resources in the optical network, establishing a mapping relationship between subgraph communication needs and wavelength resources, and obtaining the optimal wavelength allocation scheme also includes: Identify optical cross-connect devices and available wavelength resources in the network to obtain an optical network topology map; For the aforementioned optical network topology, the wavelength occupancy status on each optical link is monitored in real time to obtain a wavelength resource status table; Based on the wavelength allocation scheme, the feasibility of the allocation results is verified by a conflict detection algorithm, and adjustments are made to the conflict situation to obtain the optimal wavelength allocation scheme.

6. The method according to claim 1, characterized in that, The recursive application of the enhanced partition set coordination algorithm to construct the coordination tree structure includes: Based on the feature representation and hierarchical structure of the graph, the graph structure is recursively partitioned starting from the root node, generating two child nodes each time, resulting in a binary coordination tree. For each node in the binary coordination tree, calculate the ratio of internal connection density to external connection cost to obtain a node difference metric. The node difference metric is compared with a preset difference threshold parameter to determine whether to continue further segmentation of the node, thus obtaining a segmentation decision. Based on the splitting decision, the process is recursively continued until the difference metric of all leaf nodes is lower than the preset difference threshold parameter, thus obtaining the initial coordination tree.

7. The method according to claim 5, characterized in that, The mapping relationship between subgraph communication requirements and wavelength resources is established using a locality-sensitive hashing mechanism, including: Based on the optical network topology and the wavelength resource status table, an L-bit unique identifier encoding scheme is designed to assign a unique identifier to each available wavelength resource, thereby obtaining a wavelength resource identifier mapping table. Based on the set of subgraphs and their dependency matrix, the communication requirement characteristics between subgraphs are mapped to the hash space using a family of locality-sensitive hash functions, and a hash representation of the communication requirements is constructed to obtain a hash table of subgraph communication requirements. Based on the system running status and task scheduling mode, analyze the synchronization status and message passing mode between computing nodes, determine whether the current environment is in synchronous or asynchronous mode, and obtain the environment mode identifier. Based on the environment mode identifier, the LC-LSH algorithm is used when it is in synchronous mode and the LC-LSH4 algorithm is used when it is in asynchronous mode. Combining the wavelength resource identifier mapping table and the subgraph communication requirement hash table, the optimal subgraph and wavelength matching scheme is calculated to obtain the wavelength allocation scheme. Based on the wavelength allocation scheme, resource allocation conflicts are identified through a conflict detection algorithm, and a priority-based conflict resolution strategy is applied to obtain the optimal wavelength allocation scheme.

8. The method according to claim 7, characterized in that, When the environment mode is identified as asynchronous mode, it also includes: Based on the preliminary allocation results of the LC-LSH4 algorithm, the asymmetric communication requirements are handled by the ASYM-LC-LSH4 algorithm, resulting in an asymmetric optimization scheme. Based on the aforementioned asymmetric optimization scheme, the QR-LC-LSH4 algorithm is applied to introduce quasi-randomness to ensure a finite maximum intersection time, resulting in an optimized asynchronous environment wavelength allocation scheme. For the optimized asynchronous environment wavelength allocation scheme, key indicators including expected rendezvous time and maximum rendezvous time are evaluated through simulation models to obtain a wavelength allocation scheme with performance indicators.

9. The method according to claim 1, characterized in that, Also includes: Based on the calculation results and performance statistics, the system operating status is continuously collected through a distributed monitoring agent to obtain the system performance monitoring data stream; Based on the system performance monitoring data stream, performance bottlenecks in the system are identified through anomaly detection and bottleneck identification algorithms, resulting in a bottleneck analysis report. Based on the bottleneck analysis report, an adjustment strategy for the current state is generated by optimizing the strategy library and decision model, resulting in an adjustment strategy instruction set. Based on the aforementioned adjustment strategy instruction set, dynamic adjustments are made using wavelength resource reallocation algorithms and subgraph repartitioning algorithms to obtain optimized system performance indicators.

10. The method according to claim 5, characterized in that, Fault tolerance mechanisms based on local rerouting include: Based on the optical network topology and the wavelength resource status table, multiple shortest backup paths are pre-calculated for each optical link, and the backup path information is stored in the optical cross-connect devices at both ends of the link to obtain the link backup path table. Based on the link backup path table, a multi-level fault detection mechanism is set at the optical layer and IP layer to achieve millisecond-level link fault perception and obtain the fault detection configuration. Based on the fault detection configuration and the optimal wavelength allocation scheme, SRv6 segment routing policies for various fault scenarios are designed, head node behavior and fault response rules are defined, and an SRv6 routing policy library is obtained. According to the SRv6 routing policy library, when an optical link failure is detected, the system switches to a backup path through a local rerouting mechanism to achieve fault recovery and obtain a fault tolerance performance report.

11. The method according to claim 10, characterized in that, The local rerouting mechanism also includes: Based on the SRv6 routing policy library and the subgraph set and its dependency matrix, a mapping relationship between optical link failures and the impact of subgraph communication is established to obtain a fault response mapping table. Based on the fault response mapping table, a dynamic wavelength resource adjustment algorithm is designed to reallocate available wavelength resources when a fault occurs, thus obtaining a dynamic wavelength resource adjustment strategy. Based on the aforementioned wavelength resource dynamic adjustment strategy, a checkpoint mechanism for the subgraph computation state is designed to realize the periodic saving of the computation state and recovery after a fault, thus obtaining a computation state protection scheme.

12. The method according to claim 1, characterized in that, The configuration of the optical cross-connect device establishes an end-to-end dedicated wavelength channel, including: Based on the optimal wavelength allocation scheme and available computing resources, the most suitable computing node is selected for each subgraph using a matching algorithm to obtain the mapping relationship between the subgraph and the computing node; Based on the mapping relationship between the subgraph and the computing nodes and the optimal wavelength allocation scheme, the cross-connection status of the optical cross-connection device is set by the controller through the issuance of configuration instructions, and the configured optical path is obtained; For the configured optical path, the connectivity and bandwidth of the wavelength channels are verified through end-to-end testing to obtain a list of available wavelength channels.

13. The method according to claim 12, characterized in that, The data exchange between sub-graphs via the dedicated wavelength channel includes: Based on the prepared computing environment, the computational processing inside the subgraph is initiated on each node through a parallel computing framework, the local graph algorithm is executed, and the local computation results are obtained. Based on the local calculation results and the available wavelength channel list, boundary node data is exchanged between subgraphs through the optical layer direct data transmission mechanism to obtain updated boundary data; For the updated boundary data, a synchronization fence mechanism is used to ensure the consistency of calculations in each subgraph, control the calculation progress, and obtain synchronization status information. Based on the synchronization state information, the iterative process of the calculation is managed by the iterative controller, which determines whether to continue to the next round of calculation and obtains the iterative control signal. When the iteration control signal indicates to continue, the process returns to the internal calculation and processing steps of the execution subgraph; when the iteration control signal indicates to terminate, the process enters the result integration stage to obtain the iteration execution decision. Based on the calculation results of each subgraph after the termination of the iterative execution decision instruction, the distributed calculation results are merged through the result integration algorithm, and performance statistics are collected to obtain the calculation results and performance statistics.

14. A wavelength-granularity task partitioning and scheduling system for large-scale graph computation, characterized in that, include: The architecture establishment module is used to obtain the graph computing tasks and network topology information submitted by the user, establish the system architecture consisting of the task analysis module, graph segmentation engine, wavelength resource manager and scheduler executor, complete system initialization, and obtain the initialized system running environment; The task analysis module is used to identify the algorithm type, analyze the communication mode, and evaluate the computational complexity of the graph computing tasks submitted by the user based on the initialized system operating environment, and obtain the graph computing task feature vector. The graph segmentation module is used to calculate the task feature vector based on the graph, and apply a multi-level graph segmentation algorithm considering the communication cost between nodes and the balance of computational load to divide the large-scale graph data into multiple subgraph units, obtaining a set of subgraphs and their dependency matrix. This includes: calculating the task feature vector based on the graph; selecting an enhanced partition set coordination algorithm as the graph segmentation strategy; setting a difference threshold parameter and a coordination factor to obtain an enhanced partition set coordination algorithm parameter set; constructing a graph correlation matrix and extracting key node features based on the original graph data and the enhanced partition set coordination algorithm parameter set; establishing a hierarchical representation of the graph to obtain the graph's feature representation and hierarchical structure; and recursively applying the enhanced partition set coordination algorithm to construct... A coordination tree structure is constructed to obtain an initial coordination tree. For the initial coordination tree, the difference metric of each node in the tree is calculated, and the optimal split point is identified to obtain a coordination tree with splitting labels. Based on the coordination tree with splitting labels, the connection characteristics of boundary nodes are analyzed, and a selective node replication strategy is adopted to optimize the cross-subgraph communication mode, resulting in an optimized subgraph set. Based on the optimized subgraph set, a communication cost model is established to evaluate the data exchange volume between subgraphs. Subgraph pairs with communication costs exceeding a preset threshold are selectively merged to obtain a subgraph set that balances communication and computation. Based on the subgraph set that balances communication and computation, the data dependencies between subgraphs are analyzed, and a fine-grained dependency matrix is ​​constructed to obtain the subgraph set and its dependency matrix. The wavelength resource management module is used to analyze the status of available wavelength resources in the optical network based on the subgraph set and its dependency matrix, establish a mapping relationship between subgraph communication requirements and wavelength resources, and obtain the optimal wavelength allocation scheme. This includes: estimating communication frequency and data volume based on the subgraph set and its dependency matrix, quantifying communication requirements between subgraphs, and obtaining a subgraph communication requirement matrix; and calculating the optimal subgraph-wavelength mapping relationship using a multi-objective optimization algorithm based on the wavelength resource status table and the subgraph communication requirement matrix to obtain the wavelength allocation scheme. The scheduling and execution module is used to configure the optical cross-connect device to establish an end-to-end dedicated wavelength channel for the optimal wavelength allocation scheme, distribute the subgraph tasks to the corresponding computing nodes, and obtain a ready computing environment. The data exchange and computing module is used to exchange data between subgraphs through the dedicated wavelength channel based on the ready computing environment, and simultaneously perform parallel computing within the subgraphs to obtain computing results and performance statistics.

Images