Optimization method, device, electronic device and medium for multimodal network global routing

Abstract

The present invention relates to the field of network communication and intelligent optimization technology, and in particular to a method, device, electronic device and medium for optimizing the global routing of a multimodal network. The method comprises: constructing a data plane and a control plane according to the hierarchical structure of a software-defined network, performing a multimodal analysis on the actual network topology of the data plane according to the mode types supported by the switch, and dividing the actual network topology into logical subnets of different modes based on the multimodal analysis results; based on the network topology relationship between the logical subnets of different modes, using a preset path finding algorithm to generate a set of N paths between the source switch and the destination switch that meet preset conditions; based on the N path sets and the global controller of the control plane, using a preset global policy model to generate the global routing of the multimodal network. In this way, the problems of insufficient coordination and low utilization of global features in a multi-domain data environment of traditional network optimization methods are solved, and network transmission efficiency and cross-domain coordination capabilities are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of network communication and intelligent optimization, and in particular to a method, device, electronic device and medium for optimizing global routing in a multimodal network. Background Art

[0002] With the continued expansion of network scale and the surge in demand for cross-domain data transmission, traditional network architectures are facing numerous bottlenecks in routing optimization. In multimodal networks, different modes (such as traditional IP (Internet Protocol) networks, content-driven networks, location-based networks, and emerging FlexIP (Flexible Variable Length Internet Protocol Addressing) networks) have distinct identification and addressing mechanisms, making traditional routing protocols difficult to adapt to these heterogeneous networks. Furthermore, seamless switching and data interoperability across heterogeneous identification spaces are key challenges, as the efficiency of inter-network collaboration directly determines the performance of cross-domain path optimization. The diversity and dynamic nature of multimodal traffic place higher demands on the real-time nature of routing decisions, privacy protection, and resource scheduling capabilities.

[0003] In related technologies, a deep deterministic policy gradient algorithm is introduced to make the optimal routing selection based on the current state of the network. Combined with asynchronous federated learning, the parameters uploaded by the deep reinforcement learning agents in each domain are integrated to obtain a deep reinforcement agent with stronger decision-making ability. This agent is then applied to the decision-making process of cross-domain routing. Each domain forwards cross-domain data based on the inter-domain routing and intra-domain routing strategies.

[0004] However, this method still has problems such as insufficient coordination and low utilization of global features in multi-domain data environments, which need to be solved urgently. Summary of the Invention

[0005] The present invention provides a method, device, electronic device and medium for optimizing global routing in a multimodal network to address the problems of insufficient coordination and low global feature utilization in traditional network optimization methods in a multi-domain data environment, significantly improving the overall transmission efficiency and cross-domain coordination capabilities of the network.

[0006] To achieve the above objectives, a first embodiment of the present invention provides a method for optimizing global routing in a multimodal network, comprising the following steps:

[0007] Building a data plane and a control plane according to the hierarchical structure of the software-defined network, wherein the data plane includes the actual network topology and routing environment, and the control plane includes at least one domain controller and a global controller;

[0008] Performing a multimodal analysis on the actual network topology according to the modality types supported by the switch, and dividing the actual network topology into logical subnets of different modalities based on the multimodal analysis results;

[0009] Based on the network topology relationship between the logical subnets of different modes, a preset path finding algorithm is used to generate a set of N paths between the source switch and the destination switch that meet preset conditions, where N is a positive integer;

[0010] Based on the N path sets, the actual network topology, the routing environment, the at least one domain controller and the global controller, a global route of the multimodal network is generated using a preset global policy model.

[0011] According to one embodiment of the present invention, generating a global route for the multimodal network using a preset global policy model based on the N path sets, the actual network topology, the routing environment, the at least one domain controller, and the global controller includes:

[0012] Acquire boundary nodes and boundary link information of the multimodal network, transmission parameters of each domain controller, and routing link information based on the N path sets, the actual network topology, and the routing environment;

[0013] Based on the boundary nodes, the boundary link information, the transmission parameters of each domain controller and the routing link information, the global controller and the preset global policy model are used to generate a global route of the multimodal network.

[0014] According to one embodiment of the present invention, before generating a global route of the multimodal network using a preset global policy model based on the N path sets, the actual network topology, the routing environment, the at least one domain controller, and the global controller, the method further includes:

[0015] Acquire performance parameters of links within the domain based on the N path sets, train a preset local model based on the performance parameters of the links within the domain, and obtain local policy parameters for each domain based on the preset local model;

[0016] Based on a dynamic weight allocation mechanism, the global controller is used to fuse the local policy parameters of each domain, and the preset global policy model is generated based on the fused parameters.

[0017] According to one embodiment of the present invention, acquiring performance parameters of intra-domain links based on the set of N paths, and training a preset local model based on the performance parameters of the intra-domain links, includes:

[0018] Based on the N path sets, using in-band network telemetry technology to obtain performance parameters of links within the domain in real time, and aggregating the performance parameters according to a preset time window to construct at least one short-term traffic state matrix;

[0019] Input each short-term traffic state matrix into the current Actor network to obtain the initial path selection corresponding to each current short-term traffic state;

[0020] Inputting the initial path selection corresponding to each short-term traffic state and each current short-term traffic state into the current critic network to obtain the expected reward corresponding to each current short-term traffic state;

[0021] Inputting the initial path selection corresponding to each current short-term traffic state and the expected return corresponding to each current short-term traffic state into the target network for training to obtain an optimized Actor network and an optimized Critic network;

[0022] The preset local model is obtained based on the optimized Actor network and the optimized Critic network using a preset feedback mechanism.

[0023] According to one embodiment of the present invention, after fusing the local policy parameters of each domain using the global controller based on the dynamic weight allocation mechanism and generating the preset global policy model based on the fused parameters, the method further includes:

[0024] The preset local model is updated based on the model parameters of the preset global policy model.

[0025] According to one embodiment of the present invention, after performing multimodal analysis on the actual network topology according to the modality types supported by the switch and dividing the actual network topology into logical subnets of the different modalities based on the multimodal analysis results, the method further includes:

[0026] Determining bandwidth resources for each logical subnet based on the service requirements and resource requirements of the logical subnets of different modes;

[0027] Bandwidth is allocated to each logical subnet according to the bandwidth resources of each logical subnet.

[0028] According to one embodiment of the present invention, the generating of a set of N paths between a source switch and a destination switch that meet preset conditions using a preset path finding algorithm based on the network topology relationship between the logical subnets of different modes includes:

[0029] Determine the source node and target node;

[0030] Based on the network topology relationship between the logical subnets of different modes, the preset path finding algorithm is used to calculate the path planning status from the source node to the target node;

[0031] determining at least one parent node and a length of each path based on each path in the path planning scenario;

[0032] Based on the length of each path, trace back from the target node along at least one parent node of each path until returning to the source node, to obtain a set of N paths between the source switch and the destination switch that meet preset conditions.

[0033] According to the optimization method for global routing of a multimodal network proposed in an embodiment of the present invention, a data plane and a control plane can be constructed based on the hierarchical structure of a software-defined network. Then, a multimodal analysis is performed on the actual network topology in the data plane according to the mode types supported by the switch. Based on the multimodal analysis results, the actual network topology is divided into logical subnets of different modes. Based on the network topological relationship between the logical subnets of different modes, a preset path finding algorithm is used to generate a set of N paths between the source switch and the destination switch that meet preset conditions. Based on the N path sets and the global controller of the control plane, a preset global policy model is used to generate the global routing of the multimodal network. This solves the problems of insufficient coordination and low utilization of global features in traditional network optimization methods in a multi-domain data environment, and significantly improves the overall transmission efficiency and cross-domain coordination capabilities of the network.

[0034] To achieve the above objectives, a second embodiment of the present invention provides an optimization device for global routing in a multimodal network, comprising:

[0035] A construction module, configured to construct a data plane and a control plane according to a hierarchical structure of a software-defined network, wherein the data plane includes an actual network topology and a routing environment, and the control plane includes at least one domain controller and a global controller;

[0036] a partitioning module, configured to perform a multimodal analysis on the actual network topology according to the modal types supported by the switch, and to partition the actual network topology into logical subnets of different modalities based on the multimodal analysis results;

[0037] A first generating module is configured to generate, based on the network topology relationship between the logical subnets of different modes, a set of N paths between the source switch and the destination switch that meet preset conditions using a preset path finding algorithm, where N is a positive integer;

[0038] The second generation module is configured to generate a global route of the multimodal network using a preset global policy model based on the N path sets, the actual network topology, the routing environment, the at least one domain controller, and the global controller.

[0039] According to one embodiment of the present invention, the second generating module is specifically configured to:

[0040] Acquire boundary nodes and boundary link information of the multimodal network, transmission parameters of each domain controller, and routing link information based on the N path sets, the actual network topology, and the routing environment;

[0041] Based on the boundary nodes, the boundary link information, the transmission parameters of each domain controller and the routing link information, the global controller and the preset global policy model are used to generate a global route of the multimodal network.

[0042] According to one embodiment of the present invention, before generating a global route of the multimodal network using a preset global policy model based on the N path sets, the actual network topology, the routing environment, the at least one domain controller, and the global controller, the second generation module further includes:

[0043] an obtaining unit, configured to obtain performance parameters of links within the domain based on the set of N paths, train a preset local model based on the performance parameters of the links within the domain, and obtain local policy parameters for each domain based on the preset local model;

[0044] A generating unit is configured to fuse the local policy parameters of each domain using the global controller based on a dynamic weight allocation mechanism, and generate the preset global policy model based on the fused parameters.

[0045] According to one embodiment of the present invention, the obtaining unit is specifically configured to:

[0046] Based on the N path sets, using in-band network telemetry technology to obtain performance parameters of links within the domain in real time, and aggregating the performance parameters according to a preset time window to construct at least one short-term traffic state matrix;

[0047] Input each short-term traffic state matrix into the current Actor network to obtain the initial path selection corresponding to each current short-term traffic state;

[0048] Inputting the initial path selection corresponding to each short-term traffic state and each current short-term traffic state into the current critic network to obtain the expected reward corresponding to each current short-term traffic state;

[0049] Inputting the initial path selection corresponding to each current short-term traffic state and the expected return corresponding to each current short-term traffic state into the target network for training to obtain an optimized Actor network and an optimized Critic network;

[0050] The preset local model is obtained based on the optimized Actor network and the optimized Critic network using a preset feedback mechanism.

[0051] According to one embodiment of the present invention, after fusing the local policy parameters of each domain using the global controller based on the dynamic weight allocation mechanism and generating the preset global policy model based on the fused parameters, the generating unit is further configured to:

[0052] The preset local model is updated based on the model parameters of the preset global policy model.

[0053] According to one embodiment of the present invention, after performing multimodal analysis on the actual network topology according to the modality types supported by the switch, and dividing the actual network topology into logical subnets of different modalities based on the multimodal analysis results, the dividing module is further configured to:

[0054] Determining bandwidth resources for each logical subnet based on the service requirements and resource requirements of the logical subnets of different modes;

[0055] Bandwidth is allocated to each logical subnet according to the bandwidth resources of each logical subnet.

[0056] According to one embodiment of the present invention, the first generating module is specifically configured to:

[0057] Determine the source node and target node;

[0058] Based on the network topology relationship between the logical subnets of different modes, the preset path finding algorithm is used to calculate the path planning status from the source node to the target node;

[0059] determining at least one parent node and a length of each path based on each path in the path planning scenario;

[0060] Based on the length of each path, trace back from the target node along at least one parent node of each path until returning to the source node, to obtain a set of N paths between the source switch and the destination switch that meet preset conditions.

[0061] According to the optimization device for global routing of a multimodal network proposed in an embodiment of the present invention, a data plane and a control plane can be constructed based on the hierarchical structure of a software-defined network. Then, a multimodal analysis is performed on the actual network topology in the data plane according to the mode types supported by the switch. Based on the multimodal analysis results, the actual network topology is divided into logical subnets of different modes. Based on the network topological relationship between the logical subnets of different modes, a preset path finding algorithm is used to generate a set of N paths between the source switch and the destination switch that meet preset conditions. Based on the N path sets and the global controller of the control plane, a preset global policy model is used to generate the global routing of the multimodal network. This solves the problems of insufficient coordination and low utilization of global features in traditional network optimization methods in a multi-domain data environment, and significantly improves the overall transmission efficiency and cross-domain coordination capabilities of the network.

[0062] To achieve the above-mentioned objectives, the third aspect of the present invention proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for optimizing global routing of a multimodal network as described in the above-mentioned embodiment.

[0063] To achieve the above objectives, a fourth embodiment of the present invention proposes a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the method for optimizing global routing of a multimodal network as described in the above embodiments.

[0064] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0066] Figure 1 A flowchart of a method for optimizing global routing in a multimodal network according to an embodiment of the present invention;

[0067] Figure 2 A schematic diagram of a multimodal network global routing optimization architecture based on federated reinforcement learning according to one embodiment of the present invention;

[0068] Figure 3 Schematic diagram of multimodal network slicing resource partitioning based on an SDN (Software-Defined Networking controller) controller according to one embodiment of the present invention;

[0069] Figure 4 A schematic diagram of a data packet format for network data measurement based on INT (Inband Network Telemetry) according to an embodiment of the present invention;

[0070] Figure 5 A schematic diagram of a reinforcement learning principle according to an embodiment of the present invention;

[0071] Figure 6 A block diagram of an apparatus for optimizing global routing in a multimodal network according to an embodiment of the present invention;

[0072] Figure 7 A schematic structural diagram of an electronic device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0073] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present invention, but are not to be construed as limiting the present invention.

[0074] The following describes a method, device, electronic device, and medium for optimizing global routing in a multimodal network according to embodiments of the present invention with reference to the accompanying drawings.

[0075] Figure 1 The present invention is a flowchart of a method for optimizing global routing in a multimodal network according to an embodiment of the present invention.

[0076] For example, Figure 1 As shown, the optimization method for the multimodal network global routing includes the following steps:

[0077] In step S101, a data plane and a control plane are constructed according to the hierarchical structure of the software-defined network, wherein the data plane includes the actual network topology and routing environment, and the control plane includes at least one domain controller and a global controller.

[0078] It is understandable that software-defined networking is a new type of network architecture that uses software to centrally control network behavior and traffic, making the network more flexible and programmable.

[0079] Specifically, if Figure 2As shown in Figure 1, based on the hierarchical structure of software-defined networking, the network can be divided into two main planes: the data plane and the control plane. The data plane is primarily responsible for forwarding network traffic and includes various physical or virtual devices in the network, such as switches and routers. These devices form the actual network topology and perform routing decisions, determining how packets move within the network. The data plane can utilize the Mininet simulation platform to deploy the actual network topology and routing environment. The network is divided into three domains based on geographic distribution and traffic characteristics and deployed in three virtual machines. The control plane is responsible for intelligent decision-making and management of the entire network. It comprises an ONOS (Open Network Operating System) domain controller and an ONOS global controller, forming a distributed hierarchical management structure. Each domain has a domain controller responsible for management and optimization within the domain. The ONOS domain controller maintains complete network topology and performance data within that domain. The global controller coordinates domain controllers and path selection across domains to ensure the coordinated operation of the entire network. The control plane determines network configuration and policy, formulates routing rules and policies, and then distributes them to the data plane for execution.

[0080] For example, the actual network topology of the data plane is represented by Indicates that, is a node set, is a set of links. Each domain The node set is , the link set is The control plane consists of three domain controllers and one global controller, forming a distributed hierarchical control architecture. Responsible for topology discovery, performance monitoring and optimization within the domain, and mastering the node set within the domain and link set Complete information of the global controller. Master cross-domain border switch information and communicate with domain controllers through federated learning mechanism Collaboration,for cross-domain path optimization.

[0081] By separating the control layer and data forwarding layer of network devices, flexible network management and optimization can be achieved.

[0082] In step S102, a multimodal analysis is performed on the actual network topology according to the modal types supported by the switch, and the actual network topology is divided into logical subnets of different modalities based on the multimodal analysis results.

[0083] As you can understand, a switch is a network device used to connect computers and other network devices on a network and forward data packets to the correct port based on the packet's destination information. Different switches support different modes, including IP, Geo (the switch's geographic location mode), ID (Virtual Local Area Network Identifier), MF (Multi-Factor Authentication), NDN (Named Data Networking), and others.

[0084] Specifically, when designing and optimizing a network, an important operation is to identify the different modal types supported by switches in the actual network topology, so as to perform multimodal analysis on the actual network topology (physical network). Based on the results of the multimodal analysis, the actual network topology can be further divided into multiple logical subnets, each of which corresponds to a specific modality, such as Figure 3 This division helps to better manage and optimize network resources, ensuring that network traffic of different modes can be transmitted efficiently and securely, and is also conducive to network expansion and maintenance.

[0085] In step S103 , based on the network topology relationship between logical subnets of different modes, a preset path finding algorithm is used to generate a set of N paths between the source switch and the destination switch that meet preset conditions, where N is a positive integer.

[0086] In other words, based on the network topology between logical subnets of different modes, a preset path-finding algorithm can be used to generate a set of N paths from the source switch to the destination switch that meet the preset conditions (the shortest path), and this set is used as the candidate path set. Here, N is a positive integer, indicating the number of paths.

[0087] For ease of understanding, the following describes in detail how to obtain a set of N paths that meet preset conditions.

[0088] As a possible implementation method, in some embodiments, based on the network topology relationship between logical subnets of different modes, a preset path finding algorithm is used to generate a set of N paths between the source switch and the destination switch that meet preset conditions, including: determining a source node and a destination node; based on the network topology relationship between the logical subnets of different modes, using a preset path finding algorithm to calculate the path planning situation from the source node to the destination node; based on each path in the path planning situation, determining at least one parent node and the length of each path; based on the length of each path, tracing back from the destination node according to at least one parent node of each path until returning to the source node, to obtain a set of N paths between the source switch and the destination switch that meet the preset conditions.

[0089] Specifically, embodiments of the present invention first use INT technology to monitor key performance parameters such as traffic and latency of switch ports in real time. Then, based on this collected performance data, an adjacency matrix for the local network is generated. This adjacency matrix is ​​a two-dimensional array, where each element represents the link weight between two nodes. Link weights can be latency, bandwidth utilization, or other performance indicators. Finally, the connection information between different local networks is integrated to construct a global network topology across multiple domains (i.e., the network topology relationships between logical subnets of different modalities).

[0090] Furthermore, based on the global network topology, a preset path-finding algorithm (such as the Dijkstra algorithm, a classic shortest path algorithm that finds the path with the smallest total weight (e.g., total delay) between two points in the network) is used to calculate all shortest paths between the source and destination nodes. During the calculation process, the Dijkstra algorithm records the parent node and path length of each node in each day's path. By backtracking the parent node information, it traces back from the destination node to the source node, extracting N shortest paths and storing them in an alternative path pool, providing a set of alternative paths for route optimization. This set of alternative paths can be used to quickly select the optimal path for data transmission when a network failure occurs or when route optimization is required, thereby improving network reliability and efficiency.

[0091] The following describes in detail how to obtain a set of alternative paths by backtracking parent node information.

[0092] Specifically, the actual network topology based on the data plane is determined by Indicates that, is a node set, is a set of links, consider a weighted directed graph , each edge Has a weight To extract the s To the target nodet N shortest paths, first given the source node s And run a standard Dijkstra algorithm. The function of Dijkstra algorithm is to calculate the s The shortest path to all other nodes. In this process, the parent node and path length of each node are recorded, and the path can be extracted by backtracking the parent node information. Then extract the path from the source node. s To the target node t The shortest path is found and recorded. In order to avoid repeatedly selecting the same path, the selected path can be removed from the graph and the weight of each edge in the selected path can be increased by a large value to ensure that the path will not be selected again. Then, the Dijkstra algorithm is run again on the modified graph to calculate the new shortest path and extract the new path for recording. At this time, since the weight of the selected path has been adjusted, the calculation result no longer contains the selected path, thus achieving path deduplication. Repeat this process until the selected path is found from the source node. s To the target node t The N shortest paths are extracted. Finally, these N shortest paths are returned to complete the s To the target node t Extract multiple shortest paths.

[0093] In step S104, a global route of the multimodal network is generated using a preset global policy model based on a set of N paths, an actual network topology, a routing environment, at least one domain controller, and a global controller.

[0094] The pre-trained global policy model guides the global controller on how to generate and adjust routes. Global routing refers to a network-wide routing solution. It is not targeted at a single path, but rather a routing strategy covering the entire network.

[0095] In other words, after obtaining a set of N paths that meet pre-defined conditions, a pre-defined global policy model is used to determine how to generate and optimize routes within the network based on these N paths, the actual network topology, the routing environment, at least one domain controller, and the global controller of the control plane. Ultimately, this mechanism generates a global routing solution for a multimodal network—a routing solution that can adapt to various network conditions and requirements.

[0096] As a possible implementation method, in some embodiments, based on N path sets, actual network topology, routing environment, at least one domain controller and a global controller, a preset global policy model is used to generate a global route for a multimodal network, including: obtaining boundary nodes, boundary link information, transmission parameters and routing link information of each domain controller of the multimodal network based on N path sets, actual network topology and routing environment; based on boundary nodes, boundary link information, transmission parameters and routing link information of each domain controller, using a global controller and a preset global policy model, generating a global route for the multimodal network.

[0097] It can be understood that boundary nodes refer to nodes connecting different domains, acting as bridges in multimodal networks. Boundary links refer to links connecting different domains, serving as channels for cross-domain data transmission. Transmission parameters can include link bandwidth, latency, packet loss rate, queue depth, and other parameters, reflecting the performance and transmission capacity of a link. Routing links refer to links within a network used for data transmission, including both intra-domain and inter-domain links.

[0098] Specifically, by analyzing the actual network topology, we can identify the boundary nodes, boundary links, and routing link information connecting different domains. Using INT technology, we monitor the routing environment (i.e., link performance) in real time and collect data such as link utilization, packet loss rate, and latency (i.e., the transmission parameters of each domain controller). The global controller collects and integrates information about the boundary nodes and boundary links of the multimodal network, as well as the transmission parameters and routing link information of each domain controller, to generate a complete network view. Based on this network view, a pre-defined global policy model is used for comprehensive analysis and intelligent decision-making to generate an efficient and reliable global routing solution for the multimodal network, thereby optimizing the performance of the entire network.

[0099] Next, we will explain in detail how to obtain the preset global strategy model.

[0100] As a possible implementation method, in some embodiments, before generating the global route of the multimodal network using a preset global policy model based on a set of N paths, an actual network topology, a routing environment, at least one domain controller and a global controller, it also includes: obtaining the performance parameters of the links within the domain based on the set of N paths, and training a preset local model based on the performance parameters of the links within the domain, and obtaining the local policy parameters of each domain based on the preset local model; based on a dynamic weight allocation mechanism, using the global controller to fuse the local policy parameters of each domain, and generating a preset global policy model based on the fused parameters.

[0101] Specifically, the embodiment of the present invention can use INT technology to monitor the performance parameters of the links in the domain in real time, and use these performance parameter data to train the preset local model. After obtaining the preset local model, the local policy parameters of each domain output by the preset local model can be further obtained. The global controller can regularly receive local policy parameters uploaded by each domain. , and based on the dynamic weight distribution mechanism, the local policy parameters uploaded by each domain Perform fusion and comprehensively consider factors such as domain data quality and model convergence speed to generate global strategy parameters , based on the global policy parameters A preset global policy model can be obtained. By issuing this preset global policy model, the global controller can assist domain controllers in achieving collaborative routing optimization within and across domains. Furthermore, using a federated learning algorithm, the preset global policy model can be optimized based on the local policy parameters of each regional controller, thereby optimizing cross-domain routing policies. The global routing policy generated by the global controller can be used to perform cross-region routing selection and performance optimization, thereby improving overall network transmission efficiency.

[0102] It should be noted that, given the high feature similarity between different agents, the embodiment of the present invention adopts a horizontal federated learning strategy and deploys a set of additional evaluation neural networks. As the core model of federated learning, the network is located on the federated learning server During the training process, the agent Network Each time you train, you first copy The network parameters of , and then the agent uses the sliding average update method: To adjust your own parameters .

[0103] In the At the time iteration, the status information Input the agent Actor network, combined with the noise generated by the OU (OrnsteinUhlenbeck, a random process), and you can output a deterministic action , aggregate models through federated learning mechanisms and perform joint actions in the environment After the overall routing strategy of the environment is updated, a simulation communication process will be carried out. After the simulation communication is completed, the reinforcement learning agent obtains new local observation results by perceiving the state of the environment. , and calculate the corresponding reward function based on the feedback of the simulation communication , the agent converts the current state, action, next state and reward value into a quadruple Stored in the experience replay pool, it provides sample data for subsequent strategy updates and training, and then conducts further training.

[0104] When the agent When the amount of data M in its own experience pool is larger than the preset batch size, the agent can extract the corresponding batch size of experience samples for training. Can be input to the target policy network , to select the next action The Q target value of the state-action pair is further obtained from the Bellman equation (dynamic programming equation) :

[0105]

[0106] in, is the immediate reward function, is the discount factor, is the target Critic network, is the target Actor network, and are the parameters of the target critic and target actor networks respectively.

[0107] By learning the impact of different path selections on the end-to-end performance of the network, evaluation feedback can be provided, and the network parameters can be updated using the minimized loss function. :

[0108]

[0109] Loss function evaluates network parameters Take the derivative:

[0110]

[0111] After each domain reinforcement learning agent completes its local update, it can send the updated parameters and reward information to the federated learning global server. The server first evaluates the Normalize , and then judge the quality of the parameters based on the normalized results and make more updates in the direction of better results. These normalized values ​​are used as aggregation weights to update the federated global network: , and then sent to each domain reinforcement learning agent.

[0112] When the number of iterations Frequency of training for the Actor network In order to enable the agent to select the optimal path under different network conditions, the server can use the policy gradient method to adjust the policy network To update:

[0113]

[0114] Right now:

[0115]

[0116] When the number of iterations reaches the target network update frequency When performing a task, the target evaluation network and the target policy network can also be updated using a soft update method. The actor network is used to select paths, and the critic network is used to evaluate the quality of path selection. The two are optimized together, allowing the agent to learn the strategy that optimizes multi-dimensional QoS (Quality of Service) performance indicators.

[0117] Throughout the training process, the target network will be Regular updates help stabilize the training process. Both the Critic network and the Actor network can adjust parameters based on their current state and actions to achieve optimal routing. Through global aggregation and reinforcement learning feedback mechanisms, each agent in each domain can continuously adjust its strategy without sharing raw data. Gradient aggregation improves overall routing decision-making capabilities, achieving multi-dimensional QoS optimization with low latency, low packet loss, and high resource utilization.

[0118] Furthermore, in some embodiments, after the local policy parameters of each domain are fused using a global controller based on a dynamic weight allocation mechanism and a preset global policy model is generated based on the fused parameters, it also includes: updating the preset local model based on the model parameters of the preset global policy model.

[0119] As you can understand, when the network environment changes, such as changes in link status or traffic fluctuations, each domain controller can use the model parameters of the preset global policy model to update the preset local model. Through such updates, the domain controller can adjust the routing policy in real time, that is, the rules that determine how data is transmitted in the network, to cope with dynamically changing network conditions and further ensure the stability and efficiency of communication between different domains in the network.

[0120] Next, we will explain in detail how to obtain the preset local model, namely the DDPG (Deep Deterministic Policy Gradient) model.

[0121] As a possible implementation method, in some embodiments, performance parameters of intra-domain links are obtained based on a set of N paths, and a preset local model is obtained based on the performance parameters of the intra-domain links. This includes: based on the set of N paths, using in-band network telemetry technology to obtain the performance parameters of the intra-domain links in real time, and aggregating the performance parameters according to a preset time window to construct at least one short-term traffic state matrix; inputting each short-term traffic state matrix into the current Actor network to obtain an initial path selection corresponding to each current short-term traffic state; inputting each short-term traffic state and the initial path selection corresponding to each current short-term traffic state into the current Critic network to obtain an expected return corresponding to each current short-term traffic state; inputting the initial path selection corresponding to each current short-term traffic state and the expected return corresponding to each current short-term traffic state into the target network for training to obtain an optimized Actor network and an optimized Critic network; using a preset feedback mechanism, a preset local model is obtained based on the optimized Actor network and the optimized Critic network.

[0122] It is understandable that the INT (In-band Network Telemetry) mechanism implements real-time, packet-level, end-to-end monitoring of network status by embedding telemetry information in each data packet. First, the format of the data packet needs to be changed to implement the INT measurement solution, such as Figure 4 As shown in the figure, during network monitoring, an INT header can be embedded in the packet structure. The INT header specifically includes the telemetry identifier field If_probe and the type identifier field EtherType, as well as a series of fields for recording network status data. The If_probe flag indicates whether the packet has been tagged, allowing the switch to check this field to determine whether data telemetry is required. When If_probe == 1, the packet is tagged and the switch must append telemetry metadata to the INT header. Conversely, if If_probe == 0, the packet does not participate in telemetry at the current switch, and no telemetry data is appended. The original packet type is retained in the EtherType field. If the packet's routing is controlled by a source routing policy, the INT header can also include the bos (bottom of stack) field to indicate whether the current switch is the first node in the path. This allows each node in the network to autonomously determine whether to perform telemetry tagging on the packets passing through it without control plane intervention, ensuring continuous data measurement and transparent data communication.

[0123] Run the send.py file on host terminal h1. This file uses the Scapy tool to send INT packets to simulate the transmission of multimodal traffic. Furthermore, the send.py file sets an appropriate sending rate to ensure that packet transmission meets the requirements of the simulated traffic flow without placing excessive burden on the network. Run the receive.py file on receiving host terminal h2. This file receives and parses INT packets to extract raw measurement data, as shown in Table 1.

[0124]

[0125] The original data packet contains three layers: Ethernet, Probe, and ProbeData. The data in Table 1-3 is generated when the host switch transmits the INT packet to the telemetry server. After receiving the INT packet, the telemetry server parses the telemetry data, allowing it to measure and analyze the data packets from all switches along the path. This complete packet parsing enables the telemetry server to obtain end-to-end latency parameters.

[0126] In the configuration of this embodiment of the present invention, the control plane comprises a distributed hierarchical management structure composed of domain controllers and a global controller. Each domain is managed and optimized by a domain controller. The aforementioned multi-domain distribution based on INT data means that each domain controller can only access INT data within its own domain (such as traffic matrix, latency, packet loss rate, and link load) and cannot directly access INT data from other domains. The global controller is responsible for cross-domain coordination and path selection. Multiple domain controllers execute the global controller's global routing decisions while also monitoring network status.

[0127] By using INT technology to obtain the performance parameters of the links in the domain in real time, including link delay (end-to-end transmission delay), queue depth (cache queue status of the switch port), etc., these performance parameters can be aggregated according to the preset time window (such as 1s) to build a short-term traffic state matrix , whose dimensions are .in, is the number of switch nodes in the network; is the number of links on each switch node; The matrix represents the number of performance indicators for each link (such as latency, bandwidth utilization, and packet loss rate). Each element of the matrix represents the real-time performance of a link, such as link utilization, packet loss rate, and latency. This performance data is collected at a specific time (for example, every second) and therefore reflects the traffic status of each link in the network over a short period of time.

[0128] Obtaining the short-term flow state matrix After that, we enter the model parameter configuration phase. First, we deploy local agents for the three domain controllers and initialize each agent. Neural networks, including Critic networks (i.e., evaluation networks) and Actor network (ie, policy network) , global controller core evaluation network Located on the server As a benchmark reference for the federated model, the learning rate of the Critic network is set to , the learning rate of the Actor network is , and configure the discount factor , experience pool capacity M and training batch size batch size, is the sliding average weight. The Actor network training frequency is , the target network update frequency is , initialize the core Critic of the federated learning server, the parameters are .

[0129] The training process is in time steps Expand for the iteration unit, such as Figure 5 As shown, for each agent , the short-term traffic state matrix Input to the Actor network, this data represents the "observation" of the current network environment, reflecting the traffic status of network nodes and links in the network in a short period of time. The agent can decide the next action based on this information. As a set, it can be represented as: .in, For the moment The traffic state vector from the switch node To Node , is the bandwidth utilization, is the link delay, is the packet loss rate, is the link load (such as queue depth), It is an adjacency matrix, including the connection mode of network nodes and link topology.

[0130] Short-term traffic state matrix After input into the Actor network, the Actor network can Output an action , that is, select a path from the set of alternative paths (that is, initial path selection) Then, the current flow status and actions The input is fed into the Critic network, and the output is a Q-value, which is used to evaluate the long-term expected benefit (i.e., expected reward) for a given state and action combination. The Q-value estimate can guide the Actor network to adjust its strategy to select the optimal path and optimize overall network performance.

[0131] In order to further improve the decision-making ability of the agent and accelerate convergence, the embodiment of the present invention proposes a reward function. The calculation method of the reward function can be based on real-time INT data to more accurately measure the impact of the current strategy on network performance. Select Action After that, you will receive immediate rewards from the environment The instant reward function combines three key factors: network delay, packet loss, and resource utilization. The instant reward function is as follows:

[0132]

[0133] in, is the immediate reward function, 、 、 To adjust the coefficients of each weight, In the current traffic state and actions The network delay under In the current traffic state and actions The packet loss rate under In the current traffic state and actions The resource utilization rate is as follows.

[0134] This immediate reward function not only ensures low latency and low packet loss rate, but also takes into account multimodal service traffic management. Combined with the bandwidth allocation optimization goal, it optimizes resource utilization and avoids excessive occupation of a single path.

[0135] During the training process, the target network is used to stabilize the training. The target network can be updated regularly to help stabilize the training process. The Critic network and the Actor network can be updated according to the current traffic status. and corresponding actions To adjust parameters, we obtain an optimized actor network and critic network. Based on these optimized actor and critic networks, the agent continuously adjusts its strategy through a feedback mechanism, ensuring that routing decisions achieve multi-objective optimization, such as minimizing latency, improving bandwidth utilization, and reducing packet loss. In other words, during training, the preset local model continuously optimizes the parameters of the actor network. These parameters represent local policy decisions (i.e., local policy parameters). The final output of these local policy parameters serves as the foundational data for subsequent federated learning.

[0136] In addition, in some embodiments, after performing multimodal analysis on the actual network topology according to the modality types supported by the switch, and dividing the actual network topology into logical subnets of different modalities based on the multimodal analysis results, it also includes: determining the bandwidth resources of each logical subnet based on the business needs and resource requirements of the logical subnets of different modalities; and allocating bandwidth to each logical subnet according to the bandwidth resources of each logical subnet.

[0137] Specifically, each modal logical subnet has independent resources (such as bandwidth, computing power, storage, etc.). The ONOS domain controller in the domain schedules the bandwidth resources of the physical network, makes decisions based on the traffic rate of the logical subnet of the current modality and the set control strategy, and sends it to the bottom layer to update the bandwidth of each logical subnet, thereby coping with competition for resources such as bandwidth.

[0138] The resource allocation model is as follows: The bandwidth pre-allocated based on the statistical demand results of the logical subnet is , the current logical subnet The bandwidth is The resource utilization of the logical subnet in the current mode is:

[0139]

[0140] In order to ensure the operation of each modal business, ( represents the upper limit of resource utilization), that is, the logical subnet of this mode is in a congested state and more bandwidth resources can be allocated to it; and when When , that is, in the waste state, the bandwidth resources allocated to the logical subnet of the mode can be reduced by adjusting the bandwidth; when ( represents the lower limit of resource utilization), it indicates that the network is in a feasible state and there is no need to adjust resource allocation.

[0141] When resource utilization is outside the normal range, resource scheduling requests can be sent to the ONOS domain controller within the domain. Consider the number of network slices is , when you need to adjust network resources, use Indicates the adjusted bandwidth, Indicates the total amount, is the traffic of the current subnet, then the resource utilization of the logical subnet is:

[0142]

[0143] The objective function is to maximize resource utilization after bandwidth resource allocation, that is:

[0144]

[0145] At the same time, the adjustment of network resources will inevitably cause losses, such as the delay caused by updating the logical subnet. To indicate the adjustment of logical subnet The cost per unit bandwidth is , and the objective function is to minimize the cost of bandwidth adjustment, that is:

[0146]

[0147] Combining the above two objectives, the final objective function can be obtained as:

[0148]

[0149] The constraints are:

[0150]

[0151] It can be seen that through resource partitioning, it is possible to ensure that each logical subnet operates independently and meets its specific needs, thus providing a basis for subsequent routing optimization and federated learning.

[0152] In summary, the optimization method for global routing in a multimodal network proposed in an embodiment of the present invention combines the characteristics of heterogeneous identification and traffic in a multimodal network environment, fully utilizes the computing power and data distribution characteristics of local controllers, realizes efficient training of local strategies through distributed reinforcement learning, and realizes collaborative optimization of global strategies through a federated learning mechanism. On the one hand, this method reduces the amount of communication between local controllers and the global controller, reducing the risk of privacy leakage; on the other hand, through the distributed aggregation of model parameters and the unified optimization of the global model, it realizes efficient global routing decisions in the multimodal network, overcomes the shortcomings of the traditional centralized architecture in efficiency and privacy protection, and provides a new solution for intelligent routing optimization in a multimodal network environment.

[0153] According to the optimization method for global routing of a multimodal network proposed in an embodiment of the present invention, a data plane and a control plane can be constructed based on the hierarchical structure of a software-defined network. Then, a multimodal analysis is performed on the actual network topology in the data plane according to the mode types supported by the switch. Based on the multimodal analysis results, the actual network topology is divided into logical subnets of different modes. Based on the network topological relationship between the logical subnets of different modes, a preset path finding algorithm is used to generate a set of N paths between the source switch and the destination switch that meet preset conditions. Based on the N path sets and the global controller of the control plane, a preset global policy model is used to generate the global routing of the multimodal network. This solves the problems of insufficient coordination and low utilization of global features in traditional network optimization methods in a multi-domain data environment, and significantly improves the overall transmission efficiency and cross-domain coordination capabilities of the network.

[0154] Next, a device for optimizing global routing in a multimodal network according to an embodiment of the present invention will be described with reference to the accompanying drawings.

[0155] Figure 6 The figure is a block diagram of an apparatus for optimizing global routing in a multimodal network according to an embodiment of the present invention.

[0156] like Figure 6 As shown, the multimodal network global routing optimization device 10 includes: a construction module 100, a division module 200, a first generation module 300 and a second generation module 400.

[0157] The construction module 100 is configured to construct a data plane and a control plane according to the hierarchical structure of the software-defined network, wherein the data plane includes the actual network topology and routing environment, and the control plane includes at least one domain controller and a global controller;

[0158] A division module 200 is configured to perform multimodal analysis on the actual network topology according to the mode types supported by the switch, and divide the actual network topology into logical subnets of different modes based on the multimodal analysis results;

[0159] The first generation module 300 is configured to generate a set of N paths between a source switch and a destination switch that meet preset conditions based on the network topology relationship between logical subnets of different modes and using a preset path finding algorithm, where N is a positive integer;

[0160] The second generation module 400 is configured to generate a global route of the multimodal network using a preset global policy model based on a set of N paths, an actual network topology, a routing environment, at least one domain controller, and a global controller.

[0161] Furthermore, in some embodiments, the second generating module 400 is specifically configured to:

[0162] Based on the N path sets, the actual network topology and routing environment, the boundary nodes and boundary link information of the multimodal network, the transmission parameters of each domain controller and the routing link information are obtained;

[0163] Based on the boundary nodes, boundary link information, transmission parameters of each domain controller and routing link information, the global controller and the preset global policy model are used to generate the global routing of the multimodal network.

[0164] Furthermore, in some embodiments, before generating a global route of the multimodal network using a preset global policy model based on the N path sets, the actual network topology, the routing environment, at least one domain controller, and the global controller, the second generation module 400 further includes:

[0165] An obtaining unit, configured to obtain performance parameters of links within the domain based on the set of N paths, train a preset local model based on the performance parameters of the links within the domain, and obtain local policy parameters for each domain based on the preset local model;

[0166] The generation unit is used to fuse the local policy parameters of each domain using the global controller based on the dynamic weight allocation mechanism, and generate a preset global policy model based on the fused parameters.

[0167] Furthermore, in some embodiments, the obtaining unit is specifically configured to:

[0168] Based on a set of N paths, in-band network telemetry technology is used to obtain the performance parameters of the links within the domain in real time. The performance parameters are aggregated according to a preset time window to construct at least one short-term traffic state matrix.

[0169] Input each short-term traffic state matrix into the current Actor network to obtain the initial path selection corresponding to each current short-term traffic state;

[0170] Input the initial path selection corresponding to each short-term traffic state and each current short-term traffic state into the current critic network to obtain the expected return corresponding to each current short-term traffic state;

[0171] The initial path selection corresponding to each current short-term traffic state and the expected return corresponding to each current short-term traffic state are input into the target network for training to obtain the optimized Actor network and the optimized Critic network;

[0172] Using the preset feedback mechanism, the preset local model is obtained based on the optimized Actor network and the optimized Critic network.

[0173] Furthermore, in some embodiments, after fusing the local policy parameters of each domain using a global controller based on a dynamic weight allocation mechanism and generating a preset global policy model based on the fused parameters, the generation unit is further configured to:

[0174] The preset local model is updated based on the model parameters of the preset global policy model.

[0175] Furthermore, in some embodiments, after performing multimodal analysis on the actual network topology according to the modality types supported by the switch and dividing the actual network topology into logical subnets of different modalities based on the multimodal analysis results, the dividing module 200 is further configured to:

[0176] Determine the bandwidth resources for each logical subnet based on the business and resource requirements of the logical subnets of different modes;

[0177] Bandwidth is allocated to each logical subnet based on its bandwidth resources.

[0178] Furthermore, in some embodiments, the first generating module 300 is specifically configured to:

[0179] Determine the source node and target node;

[0180] Based on the network topology relationship between logical subnets of different modes, the preset path finding algorithm is used to calculate the path planning from the source node to the target node;

[0181] determining, based on each path in the path planning scenario, at least one parent node and a length of each path;

[0182] Based on the length of each path, the path is traced back from the target node to at least one parent node of each path until it returns to the source node, and a set of N paths between the source switch and the destination switch that meet the preset conditions is obtained.

[0183] It should be noted that the aforementioned explanation of the embodiment of the method for optimizing global routing in a multimodal network is also applicable to the device for optimizing global routing in a multimodal network in this embodiment, and will not be repeated here.

[0184] According to the optimization device for global routing of a multimodal network proposed in an embodiment of the present invention, a data plane and a control plane can be constructed based on the hierarchical structure of a software-defined network. Then, a multimodal analysis is performed on the actual network topology in the data plane according to the mode types supported by the switch. Based on the multimodal analysis results, the actual network topology is divided into logical subnets of different modes. Based on the network topological relationship between the logical subnets of different modes, a preset path finding algorithm is used to generate a set of N paths between the source switch and the destination switch that meet preset conditions. Based on the N path sets and the global controller of the control plane, a preset global policy model is used to generate the global routing of the multimodal network. This solves the problems of insufficient coordination and low utilization of global features in traditional network optimization methods in a multi-domain data environment, and significantly improves the overall transmission efficiency and cross-domain coordination capabilities of the network.

[0185] Figure 7 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device may include:

[0186] Memory 701 , processor 702 , and computer programs stored in the memory 701 and executable on the processor 702 .

[0187] When the processor 702 executes the program, the method for optimizing the multimodal network global routing provided in the above embodiment is implemented.

[0188] Furthermore, the electronic device further includes:

[0189] The communication interface 703 is used for communication between the memory 701 and the processor 702 .

[0190] The memory 701 is used to store computer programs that can be run on the processor 702 .

[0191] The memory 701 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0192] If the memory 701, processor 702, and communication interface 703 are implemented independently, the communication interface 703, memory 701, and processor 702 can be connected to each other via a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0193] Optionally, in a specific implementation, if the memory 701, the processor 702 and the communication interface 703 are integrated on a chip, the memory 701, the processor 702 and the communication interface 703 can communicate with each other through an internal interface.

[0194] The processor 702 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0195] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for optimizing global routing in a multimodal network.

[0196] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0197] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0198] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for optimizing global routing in a multimodal network, characterized in that: The following steps are involved: Building a data plane and a control plane according to the hierarchical structure of the software-defined network, wherein the data plane includes the actual network topology and routing environment, and the control plane includes at least one domain controller and a global controller; Performing a multimodal analysis on the actual network topology according to the modality types supported by the switch, and dividing the actual network topology into logical subnets of different modalities based on the multimodal analysis results; Based on the network topology relationship between the logical subnets of different modes, a preset path finding algorithm is used to generate a set of N paths between the source switch and the destination switch that meet preset conditions, where N is a positive integer; Generate a global route for the multimodal network using a preset global policy model based on the N path sets, the actual network topology, the routing environment, the at least one domain controller, and the global controller; After performing a multimodal analysis on the actual network topology according to the mode types supported by the switch and dividing the actual network topology into the logical subnets of different modes based on the multimodal analysis results, the method further includes: determining bandwidth resources for each logical subnet based on the service requirements and resource requirements of the logical subnets of different modes, and allocating bandwidth to each logical subnet according to the bandwidth resources of each logical subnet; When resource utilization is outside the normal range and network resources need to be adjusted, Indicates the number of network slices, represents the adjusted bandwidth, Indicates the total amount, represents the traffic of the current subnet, then the resource utilization of the logical subnet is: in, For the current logical subnet of traffic, For the current logical subnet Resource utilization, For the current logical subnet bandwidth; The objective function is to maximize resource utilization after bandwidth resource allocation. : use Indicates adjusting the logical subnet The cost per unit bandwidth, the objective function is to minimize the cost of bandwidth adjustment : in, Logical subnet The bandwidth pre-allocated based on the statistical demand results; The final objective function is: The constraints are: in, is the lower limit of resource utilization, The upper limit of resource utilization.

2. The method according to claim 1, characterized in that The generating of a global route of the multimodal network using a preset global policy model based on the N path sets, the actual network topology, the routing environment, the at least one domain controller, and the global controller includes: Acquire boundary nodes and boundary link information of the multimodal network, transmission parameters of each domain controller, and routing link information based on the N path sets, the actual network topology, and the routing environment; Based on the boundary nodes, the boundary link information, the transmission parameters of each domain controller and the routing link information, the global controller and the preset global policy model are used to generate a global route of the multimodal network.

3. The method according to claim 1, characterized in that Before generating a global route of the multimodal network using a preset global policy model based on the N path sets, the actual network topology, the routing environment, the at least one domain controller, and the global controller, the method further includes: Acquire performance parameters of links within the domain based on the N path sets, train a preset local model based on the performance parameters of the links within the domain, and obtain local policy parameters for each domain based on the preset local model; Based on a dynamic weight allocation mechanism, the global controller is used to fuse the local policy parameters of each domain, and the preset global policy model is generated based on the fused parameters.

4. The method according to claim 3, characterized in that The acquiring performance parameters of the intra-domain links based on the N path sets, and training a preset local model based on the performance parameters of the intra-domain links, includes: Based on the N path sets, using in-band network telemetry technology to obtain performance parameters of links within the domain in real time, and aggregating the performance parameters according to a preset time window to construct at least one short-term traffic state matrix; Input each short-term traffic state matrix into the current Actor network to obtain the initial path selection corresponding to each current short-term traffic state; Inputting the initial path selection corresponding to each short-term traffic state and each current short-term traffic state into the current critic network to obtain the expected reward corresponding to each current short-term traffic state; Inputting the initial path selection corresponding to each current short-term traffic state and the expected return corresponding to each current short-term traffic state into the target network for training to obtain an optimized Actor network and an optimized Critic network; The preset local model is obtained based on the optimized Actor network and the optimized Critic network using a preset feedback mechanism.

5. The method according to claim 3, characterized in that After fusing the local policy parameters of each domain using the global controller based on the dynamic weight allocation mechanism and generating the preset global policy model based on the fused parameters, the method further includes: The preset local model is updated based on the model parameters of the preset global policy model.

6. The method according to claim 1, wherein The method of generating a set of N paths between a source switch and a destination switch that meet preset conditions by using a preset path finding algorithm based on the network topology relationship between the logical subnets of different modes includes: Determine the source node and target node; Based on the network topology relationship between the logical subnets of different modes, the preset path finding algorithm is used to calculate the path planning status from the source node to the target node; determining at least one parent node and a length of each path based on each path in the path planning scenario; Based on the length of each path, trace back from the target node along at least one parent node of each path until returning to the source node, to obtain a set of N paths between the source switch and the destination switch that meet preset conditions.

7. A device for optimizing global routing in a multimodal network, characterized in that: include: A construction module, configured to construct a data plane and a control plane according to a hierarchical structure of a software-defined network, wherein the data plane includes an actual network topology and a routing environment, and the control plane includes at least one domain controller and a global controller; a partitioning module, configured to perform a multimodal analysis on the actual network topology according to the modal types supported by the switch, and to partition the actual network topology into logical subnets of different modalities based on the multimodal analysis results; A first generating module is configured to generate, based on the network topology relationship between the logical subnets of different modes, a set of N paths between the source switch and the destination switch that meet preset conditions using a preset path finding algorithm, where N is a positive integer; A second generation module is configured to generate a global route of the multimodal network using a preset global policy model based on the N path sets, the actual network topology, the routing environment, the at least one domain controller, and the global controller; After performing a multimodal analysis on the actual network topology according to the mode types supported by the switch and dividing the actual network topology into the logical subnets of different modes based on the multimodal analysis results, the division module is further configured to: determine bandwidth resources for each logical subnet based on the service requirements and resource requirements of the logical subnets of different modes, and allocate bandwidth to each logical subnet according to the bandwidth resources of each logical subnet; When resource utilization is outside the normal range and network resources need to be adjusted, Indicates the number of network slices, represents the adjusted bandwidth, Indicates the total amount, represents the traffic of the current subnet, then the resource utilization of the logical subnet is: in, For the current logical subnet of traffic, For the current logical subnet Resource utilization, For the current logical subnet bandwidth; The objective function is to maximize resource utilization after bandwidth resource allocation. : use Indicates adjusting the logical subnet The cost per unit bandwidth, the objective function is to minimize the cost of bandwidth adjustment : in, Logical subnet The bandwidth pre-allocated based on the statistical demand results; The final objective function is: The constraints are: in, is the lower limit of resource utilization, The upper limit of resource utilization.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for optimizing global routing in a multimodal network according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method for optimizing multimodal network global routing according to any one of claims 1 to 6.

Images