Efficient routing method for ultra-high bandwidth service flow in time-sensitive network

By combining Transformer's improved deep reinforcement learning and NSGA-II algorithm in a time-sensitive network, the routing path is optimized, and the routing optimization problem under the hybrid deployment of ultra-high bandwidth flows and conventional service flows is solved, achieving more balanced network resource allocation and Performance improvement.

CN119030914AActive Publication Date: 2024-11-26NANJING UNIV OF AERONAUTICS & ASTRONAUTICS

Patent Information

Application Number
CN202411517233.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-29
Publication Date
2024-11-26
Estimated Expiration
2044-10-29

AI Technical Summary

Technical Problem

The existing technology has failed to effectively solve the routing optimization problem in the hybrid deployment of ultra-high bandwidth streams and conventional service streams in time-sensitive networks, resulting in large network delays and unbalanced link loads, which cannot meet the high bandwidth and high bandwidth of ultra-high bandwidth streams. Low latency demand.

Method used

Using a multi-objective optimization method, the routing path is optimized to ensure that the path of ultra-high bandwidth streams meets the paths of ultra-high bandwidth streams through a routing calculation method based on Transformer improved deep reinforcement learning (T-DRL) and non-dominant sorting genetic algorithm (NSGA-II). High bandwidth and low latency requirements, and achieve balanced allocation of network resources.

Benefits of technology

Effectively reduces the risk of network congestion, improves overall network performance, solves the service load balancing and low latency requirements of local area networks, improves routing optimization efficiency and success rate, and can better meet ultra-high bandwidth flows and conventional service flows. QoS requirements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119030914A_ABST
    Figure CN119030914A_ABST
Patent Text Reader

Abstract

The invention discloses an efficient routing method for an ultrahigh bandwidth service flow in a time-sensitive network, and aims to solve the problems of load balancing of a UBF, improvement of network performance and the like. The method mainly comprises the following steps: distinguishing the UBF from other data streams so as to meet respective requirements; and then, a multi-objective optimization efficient routing algorithm is adopted to select an optimal path for each service flow for transmission. According to the invention, the operation efficiency and performance of the TSN can be improved, and particularly, more reliable service is provided for real-time application in a mixed transmission scene of ultrahigh bandwidth flow and conventional service flow.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of time-sensitive network of Internet communication network, and in particular to an efficient routing method for ultra-high bandwidth service flow in a time-sensitive network. Background Art

[0002] With the development of AR / VR technology, the high-speed interaction and precise control of immersive video transmission networks require data synchronization, deterministic latency, and ultra-high bandwidth. At the same time, the more complex communication mode of large-scale cluster computing technology has put forward extremely high requirements on the transmission capacity of the network. The emergence of immersive video transmission networks and the development of large-scale cluster computing technology have further promoted the continuous expansion of the application scope of time-sensitive networking (TSN). However, the communication data model of large-scale distributed intelligent computing systems presents the characteristics of multi-point mutual communication and "elephant flow". The changes in this new extreme business scenario with high bandwidth requirements put forward higher requirements on network performance such as real-time, reliability, and security. In order to meet these challenges, deterministic real-time communication has been introduced into fields such as aerospace and industrial automation to ensure the performance of time-critical flows and the security of systems. TSN provides deterministic low-latency transmission services for Layer 2 networks. Specifically, the 802.1 TSN standard is designed to support the transmission of multiple types of traffic in the same network. However, the large number of periodic control flows, reserved ultra-high bandwidth flows, and ordinary IT flows continuously generated by IoT devices have posed new challenges to the TSN network management, routing, and ability to accommodate data flows.

[0003] The characteristics of regular business flow (RSF) and ultra-high bandwidth streaming (UBF) are significantly different. RSF is a large class of traffic that is often optimized in TSN networks, including time-triggered (TT) flows, audio and video bridging (AVB) flows, and best effort (BE) flows. Typically, TT traffic is generated by control or command applications with strict real-time requirements and requires strict latency constraints. AVB traffic comes from applications that are not so demanding in terms of real-time requirements. BE traffic is a basic traffic type with no specific low latency or bandwidth requirements. UBF refers to traffic that requires extremely high bandwidth and deterministic transmission in the network. It is often used in scenarios such as big data processing in cloud computing, real-time video streaming, and ultra-low latency communication in telecommunications networks. Its transmission can cause load imbalance problems such as network congestion and bandwidth bottlenecks, affecting the performance of real-time applications. Therefore, UBF needs to be distinguished from other data flows to meet their respective needs. Compared with traditional networks, TSN not only focuses on the transmission timing of traffic, but also supports the transmission of key traffic by optimizing paths at the network level. Effective routing strategies can reduce transmission delays, improve resource utilization efficiency, and ensure the service quality of key traffic. Therefore, in a feasible and practical TSN system, it is crucial to support the hybrid transmission of UBF and RSF at the same time.

[0004] Routing optimization is the core of ensuring the transmission quality of UBF. In TSN systems, heuristic solutions are usually used to route service flows on different paths to achieve spatial isolation. However, this heuristic solution often results in longer solution time, and only considers RSF while ignoring the needs and impact of UBF. Driven by services such as immersive video transmission networks and large-scale cluster computing technology, UBF with ultra-high bandwidth and deterministic latency has become the most typical new service feature. The huge difference in its traffic characteristics can lead to network congestion and bandwidth bottlenecks. Therefore, it is necessary to comprehensively consider the routing problem of mixed deployment of RSF and UBF in the deterministic domain. However, due to the mutual influence between various types of flows in TSN and the existence of various constraints, it is very challenging to design an efficient routing solution.

[0005] Routing methods and limitations of existing technologies for solving TSN multi-target mixed service flows:

[0006] (1) In view of the new extreme business scenarios with ultra-high bandwidth requirements, improving the network operation efficiency has become a challenging problem in complex TSN networks with multiple paths and multiple inputs and outputs. So far, in the scheduling and routing mechanisms of the TSN standard, the design scheme for joint path optimization of conventional business flows and ultra-high bandwidth flows has not been effectively solved. Therefore, more in-depth research is still needed on the existing time-sensitive network routing management and communication scheduling methods. These issues are directly related to and affect the real-time and reliability performance of the network.

[0007] (2) In the TSN local area scenario, existing solutions fail to effectively solve the load balancing problem when UBF exists. Considering the significant differences in traffic characteristics between UBF and RSF, traditional load balancing routing algorithms fail to distinguish the needs of UBF and RSF, ignore the specific routing requirements of UBF, and cause conflicts between service flows, increase the latency of time-sensitive flows, and thus violate strict flow deadlines. In addition, existing methods limit the diversity of solutions and fail to fully utilize the characteristics of TSN.

[0008] (3) To solve the routing and scheduling problems of various types of flows based on machine learning or deep learning, the optimization goal is to minimize the end-to-end delay of RSF and construct an objective function consisting of the weighted sum of three indicators. However, it is difficult to accurately set the weight vector to obtain the Pareto optimal solution using the linear weighting method. In addition, traditional multi-objective optimization algorithms, such as deep reinforcement learning (DRL), are prone to fall into local optimal solutions and find it difficult to fully explore the solution space and find the global optimal solution. The non-dominated sorting genetic algorithm (NSGA-II) takes a long time to find the optimal solution and has limited local optimization capabilities. Summary of the invention

[0009] The present invention aims at the routing optimization problem under the mixed deployment of ultra-high bandwidth business flow and conventional business flow in a time-sensitive network, and provides an efficient routing method for ultra-high bandwidth business flow in a time-sensitive network, so as to solve the problems of large delay and unbalanced link load in the network.

[0010] The technical solution to achieve the purpose of the present invention is: an efficient routing method for ultra-high bandwidth service flows in a time-sensitive network, comprising the following steps:

[0011] S1: The centralized user configuration CUC initiates a request to retrieve the network physical topology through the user network interface UNI to the network centralized controller CNC to obtain TSN network topology information; the CNC discovers the TSN network topology, abstracts it into a network directed graph and returns the result to the CUC;

[0012] S2: The terminal system sends TSN connection requests and resource requirements to CUC through the user configuration protocol, including information such as the communication terminal, the period T, size and deadline delay of the service flow, to obtain all traffic information in the network. CUC passes these connection requests and resource requirements to CNC through UNI;

[0013] S3: CNC performs service modeling at the switch based on service flow attributes. First, the controller CNC divides these service flows into TT flows, AVB flows, and BE flows based on the QoS delay requirements of different flows. Then, when the bandwidth of the data flow reaches a certain threshold of the link bandwidth, it will be identified and classified as UBF. At the same time, PCP and VID are assigned to each service flow and mapped to different priority queues in the switch.

[0014] S4: Obtain basic information of the entire network, including TSN network topology and data flow information set, and establish a multi-objective optimization mathematical model considering multiple routing constraints based on this basic information;

[0015] S5: CNC performs routing calculations based on the network physical topology, network requirements, and the established multi-objective optimization mathematical model containing multiple routing constraints to obtain relevant path information that meets the connection requirements and is feasible. The first stage of routing calculation is to use the Transformer-based improved deep reinforcement learning T-DRL to obtain a feasible initial solution as an alternative path.

[0016] S6: Execute the second stage of route calculation, the CNC optimizes the candidate paths selected in step S5 using the non-dominated sorting genetic algorithm II (NSGA-II) to select the optimal path from the sender to the receiver that meets the service flow requirements;

[0017] S7: CNC traverses to see if there are any uncalculated paths between the sender and the receiver. If yes, it returns to execute steps S3-S6. If no, it executes step S8.

[0018] S8: CNC encapsulates the calculated path results into a transmission table, configures it to the TSN switch, and then sends it to the TSN terminal device through CUC.

[0019] Furthermore, step S1 specifically includes: before the network starts running, CUC initiates a request to retrieve the network physical topology to CNC through UNI; CNC discovers TSN network topology information and node information according to LLDP (Link Layer Discovery Protocol), and models it into a TSN directed graph through a network modeling method And return the results to CUC; the specific construction process of the network model is as follows:

[0020] The TSN network topology is modeled as a directed graph ,in represents a set of n switch SW nodes and a set of λ terminal equipment ES nodes, It is a set of physical links. TSN supports full-duplex, and each full-duplex link is regarded as two separate directed links. and .let represents the adjacency matrix, which consists of elements 0 and 1, where if , ,if , ; Each link Defined by triples ,in Yes Link The bandwidth capacity of It represents the link delay in µsec, which is the transmission delay of i , queuing delay and Link The propagation delay composition, For link The number of TSN streams.

[0021] Further, step S2 includes: the terminal system configures the communication terminal through the user configuration protocol , the base period of business flow , Deadline , bandwidth size , Priority , population size Size, maximum number of iterations of NSGA-II , T-DRL maximum number of iterations The information is sent to CUC to obtain all the information in the network, and then CUC passes these communication requirements to CNC through UNI, as follows:

[0022] Each communication task is abstracted as a business flow, and these business flows are defined by tuples. ,in and Business Flow The source and destination nodes, Representation Flow The bandwidth required, and Respectively represent the flow The deadline and base period, Representation Flow For m service flows, m paths that meet their needs are selected for transmission.

[0023] Further, as an improvement of the efficient routing method for ultra-high bandwidth service flow in a time-sensitive network provided by the present invention, the specific process of performing service modeling in step S3 is as follows:

[0024] S31: After the data flow enters the switch, the switch will check the PCP field and VLAN ID field in the IEEE 802.1Q VLAN tag in the Ethernet frame header. By identifying these fields, the attribute information or statistical information of the data flow is determined to further classify and determine its traffic type and priority information.

[0025] S32: According to the delay attribute information of the data stream, the data stream is first classified into different traffic types, including TT stream, AVB stream and BE stream.

[0026] S33: Based on step S32, the data flows among TT flows, AVB flows, and BE flows whose bandwidth reaches a certain threshold of link bandwidth are additionally classified as ESF, and finally the data flows are divided into TT, ESF, AVB, and BE flows. Here, assume Represents a collection of TSN service flows, including subset TT flows, ESF flows, AVB flows, and BE flow sets: .

[0027] S34: The classified data flows enter their respective priority queues in the switch. Each queue represents a traffic type and priority. These queues are used for priority queuing to ensure that high-priority traffic (TT flows) can be processed and transmitted faster.

[0028] S35: Finally, the CNC establishes its switch queue information table according to the transmission rules of each type of business flow. The switch selects a non-conflicting egress port according to the queue information table to queue forward each flow, which ensures that the transmission of various types of flows will not conflict, so as to maintain smooth transmission of the flow.

[0029] Furthermore, in step S4, a multi-objective optimization mathematical model considering multiple routing constraints is established, including minimizing the end-to-end delay of RSF, minimizing the available bandwidth rate, and considering finding a path with a large remaining bandwidth and a small number of flows for the ultra-high bandwidth flow UBF as the goal for multi-objective optimization. The specific objective function is as follows:

[0030] S41: Objective function 1 is to minimize the total end-to-end delay of TT stream, AVB stream, and BE stream

[0031]

[0032] In the formula, is the weighted sum of the total end-to-end delay, and its value is between [0, 1]. , and Represents the weight value, which has , , and They represent the normalized average delays of TT stream, AVB stream and BE stream respectively;

[0033] Furthermore, the normalized average delay of the TT stream, the AVB stream, and the BE stream in step S41 can be specifically expressed as:

[0034] S411: The normalized average delay of the TT flow is defined as follows:

[0035]

[0036] In the formula, Indicates the number of TT flows, is a constant, indicating the maximum acceptable delay of the TT flow. Express demand end-to-end latency;

[0037] S412: The normalized average delay of the AVB stream is defined as follows:

[0038]

[0039] In the formula, Indicates the number of AVB streams, is a constant representing the worst-case end-to-end delay of the AVB stream;

[0040] S413: Focus on limiting the worst-case BE traffic delay without degrading the performance of high priority traffic. The normalized average delay of BE flows is defined as follows:

[0041]

[0042] In the formula, represents the number of BE flows, is a constant, indicating the acceptable delay threshold of BE flow. In addition, ;

[0043] Furthermore, the requirement in step S411 Represents TSN streams, including TT streams, AVB streams, BE streams, and UBF stream sets, so there is a set of TSN streams: ,in , then the demand End-to-end delay It can be specifically expressed as follows:

[0044]

[0045] In the formula, Indicates link delay, including transmission delay , Propagation Delay and queuing delay ; Indicates that the TSN flow is at the node and The transmission direction between and Specifically, it can be expressed as:

[0046] (1) ;

[0047] (2) ;

[0048] S42: Objective function 2 is to maximize the available bandwidth rate

[0049]

[0050] Furthermore, some relevant definitions are given in step S42 for easy understanding and use. The specific definitions are as follows:

[0051] (1) Effective routing: For For example, an effective route Defined as meeting its deadline Requested route. Can be represented as an ordered list of physical links , From its source address To the destination address Traverse these physical links.

[0052] (2) Link load: For each link , defines the link The load, Indicates that through the link The sum of the sizes of all flows of . Formally, its load can be defined as:

[0053]

[0054] In the formula, Representing data flow The bandwidth size.

[0055] (3) Link bandwidth utilization: For each link , defines the link The bandwidth utilization rate is Indicates. Link The bandwidth utilization is:

[0056]

[0057] In the formula Indicates the bandwidth of the link. The link bandwidth is the same.

[0058] (4) Remaining bandwidth of routing: The remaining bandwidth is used It is defined as The minimum value of the remaining bandwidth of all links in can be expressed as:

[0059]

[0060] (5) Number of TSN flows routed: Number of TSN flows It is defined as The maximum number of TSN flows on all links in , which can be specifically expressed as:

[0061]

[0062] In the formula Indicates that the link The number of TSN flows is obtained .

[0063] (6) The access degree of each node: represents the kth stream Whether the node is visited , which can be defined as:

[0064]

[0065] In order to prevent loops, each node All in the same stream Visit once. Therefore, for each node ,have:

[0066]

[0067] (7) Bandwidth utilization: Routing Bandwidth utilization It is defined as The maximum bandwidth utilization of all links in can be expressed as:

[0068]

[0069] S43: Objective function 3 is to minimize the path blocking rate of UBF. Specifically, UBF is transmitted by selecting a path with a large remaining bandwidth and a small number of flows. The specific objective function is as follows:

[0070]

[0071] In the formula, Indicates the maximum value of the remaining bandwidth among all valid paths. Indicates the conversion value of the number of TSN streams, as follows:

[0072]

[0073] In the formula, Indicates the minimum number of TSN flows in all valid paths.

[0074] Furthermore, the objective function established in steps S41, S42 and S43 needs to consider multiple routing constraints to implement a multi-objective routing optimization model, and the constraints are specifically as follows:

[0075] 1): Latency constraints of TSN data streams

[0076] ①:

[0077] ②:

[0078] ③:

[0079] ④:

[0080] Among them, constraints ① and ② limit the maximum end-to-end delay allowed for TT flow and AVB flow respectively; constraint ③ states that the transmission of BE flow should be completed within the worst-case deadline. Although BE traffic is a delay-insensitive data flow, it needs to be guaranteed to be transmitted within the current time period; constraint ④ ensures that the end-to-end delay of flows with higher priority is lower, where and is the delay of two different paths.

[0081] 2): Access degree constraints of nodes

[0082] ①:

[0083] ②:

[0084] Among them, constraint ① avoids loops, and a node can only be accessed once by the same data flow. Constraint ② restricts m data flows to reach the destination node.

[0085] 3): Bandwidth constraints

[0086] ①:

[0087] ②:

[0088] ③:

[0089] in, Indicates link Constraint ① prevents the current link load from exceeding the link bandwidth; constraint ② limits the bandwidth utilization of each link to no more than 10% of its capacity to ensure link load balancing; constraint ③ indicates that the ESF bandwidth meets its link bandwidth requirements and does not exceed 10% of the remaining bandwidth.

[0090] 3): Constraints on the number of TSN flows on the path

[0091] ①:

[0092] Among them, constraint ① prevents the number of flows on the path allocated by UBF from exceeding This threshold ensures the service quality of UBF and avoids network congestion.

[0093] Furthermore, in step S5, the Transformer-based deep reinforcement learning (TDRL) is used to obtain a feasible initial solution and use it as an alternative path. Specifically, the TSN multi-objective hybrid service flow routing problem (MOHSFR) is first modeled as a Markov decision problem (MDP); then the policy network uses the Transformer architecture to model the agent that solves MOHSFR. The model mainly includes an encoding process, a decoding process, and a training process. By inputting the network directed graph, the characteristics and requirements of the flow (known), and the trained Transformer-based DRL model, a set of initial flow paths is output to represent the individuals in the solution, and then the initial solution path is used as the input of step S6;

[0094] It includes the following specific execution steps:

[0095] S51: Modeling the MOHSFR problem as an MDP:

[0096] S511: Define MDP elements:

[0097] (1) Agent: The DRL model acts as an agent and is responsible for making decisions on path selection actions.

[0098] (2) State set : The state set includes the network state S t All possible values ​​of .

[0099] (3) Action set : The action set includes all possible path selection actions A t .

[0100] (4) Reward Function : Define a reward function and calculate the reward based on the performance of the path (such as delay weighted value, available bandwidth rate, and path blocking degree of UBF).

[0101] S512: MDP modeling, after defining the MDP elements, establish the MDP model, including state transition probability, reward function, etc. Among them, the state transition function finally outputs the probability of all possible next states based on the given current state and action. Because the system state changes, the node state, link state and flow state will change accordingly;

[0102] S52: The encoding process based on the transformer architecture first uses a linear layer to embed the network topology features and each TSN service flow into a high-dimensional space and obtain the initial embedding information ; then After L identical layers, the final information embedding is obtained Each layer consists of a multi-head attention layer (MHA), a skip connection layer, a batch normalization (BN) layer, and a fully connected feed-forward (FF) layer;

[0103] S53: Based on the decoding process of the transformer architecture, the decoder uses the context information from the encoder to decode at each step and generate a probability vector for selecting the next hop node for each service flow. Here, the agent selects the path and action at each time step according to the strategy of the DRL model, constructs the final path solution, and then outputs the initial solution and returns a set of initial paths generated by the encoder process, which constitute part of the initial population;

[0104] S54: Training process: Using policy gradients to train parameters in neural networks , used to update the parameters of the DRL model, and the model provides feedback based on the generated paths and performance indicators to optimize the policy network;

[0105] S55: repeating steps S51-S55 to generate different initial solutions or optimization paths;

[0106] S56: After the training is completed, a set of initial flow paths is output to represent individuals in the solution;

[0107] Further, in step S6, the CNC uses the candidate path selected in step S5 for NSGA-II optimization to select a preferred path from the sending end to the receiving end that meets various flow requirements, which specifically includes the following steps:

[0108] S61: First set the initialization parameters, including the population size and the maximum number of iterations of NSGA-II , crossover rate , mutation rate ;

[0109] S62: Convert the initial solution obtained in step S5 into the initial population of NSGA-II to create a parent population of size Size ;

[0110] S63: Parent populations according to non-dominance level Sort and assign each chromosome a value equal to its non-dominated level. And let the initial cycle number be ;

[0111] S64: Use selection (based on non-dominated levels), crossover, and mutation operators in Create a descendant population on ;

[0112] S65: Determine whether the constraint conditions are met. If the constraint conditions are met, stop and return ; If the constraint condition is not met, execute step S66;

[0113] S66: Initial parent population and offspring population Merge to generate a merged population ;

[0114] S67: From Identifying non-dominated frontiers , and set ;

[0115] S68: When When The crowding distance of the intermediate solution; if , then let Otherwise, according to the crowding distance Sort and add arrive The least crowded solution; then let ;

[0116] S69: Determine whether the termination condition is met. If not, set , return to step S64, Apply selection (based on non-dominated level), crossover, and mutation operators to create the offspring population If satisfied, output the solution obtained after NSGA-II optimization. That is, the optimal path from the sender to the receiver that meets various flow requirements is selected.

[0117] Compared with the prior art, the present invention has the following significant advantages: (1) The present invention specifically optimizes the routing selection of ultra-high bandwidth flows to ensure that they meet the requirements of high bandwidth and low latency, so as to achieve more balanced network resource allocation, reduce the risk of network congestion, and improve the overall network performance. (2) The present invention can solve the requirements of service load balancing and low latency in TSN local area networks; through intelligent path selection and routing strategies, ultra-high bandwidth flows and conventional service flows are distinguished to meet their respective QoS requirements. This innovative strategy is very useful for processing applications such as large data transmission, immersive video interaction, and generative large model data synchronization. (3) The present invention uses Transformer-based deep reinforcement learning (TDRL) to generate initial solutions, effectively avoiding the problem of long training time of traditional DRL. The TDRL model is generated quickly, providing a high-quality initial solution for the NSGA-II algorithm, and improving the overall optimization efficiency and success rate. This multi-objective optimization method can comprehensively consider different aspects of network performance, so that the network performs well under various traffic requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0118] Figure 1 The figure is a schematic diagram of communication forwarding initiated by a service flow in the TSN network architecture of the present invention.

[0119] Figure 2 A schematic diagram of the structure of a CNC of the present invention for identifying and classifying TSN hybrid service flows.

[0120] Figure 3 The present invention is a flowchart of a multi-objective optimized efficient routing method for mixed flow transmission in a time-sensitive network.

[0121] Figure 4 Schematic diagram of the first stage process of solving a feasible initial solution for the transformer-based DRL of the present invention and using it for optimization in NSGA-II.

[0122] Figure 5 It is a schematic diagram of the process of obtaining a feasible initial solution for optimization of NSGA-II according to the present invention. DETAILED DESCRIPTION

[0123] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0124] Embodiment 1: A communication forwarding process initiated by a service flow in a TSN network structure.

[0125] Since ultra-high bandwidth flows occupy a high amount of resources, this new scenario change has brought tremendous pressure to reliable communications. In addition, the current network's switching capacity and cache space are limited, which can easily lead to network congestion and packet loss. In order to solve the above problems, the present invention proposes an efficient routing method for ultra-high bandwidth service flows in time-sensitive networks. The core idea of ​​realizing the transmission requirements of hybrid services in TSN is: first, distinguish UBF from other data flows to meet their respective needs. Then, a new routing algorithm is used to select the optimal path for each service flow for transmission. The process of initiating communication forwarding by service flows in TSN is as follows: Figure 1 shown.

[0126] The workflow of the service flow from the source node to the destination node through the relay node is as follows:

[0127] Step 1: Before the network starts running, the Central User Configuration (CUC) initiates a request to retrieve the network physical topology to the network centralized controller CNC through the user network interface UNI;

[0128] Step 2: CNC discovers the TSN network topology, abstracts the TSN network topology into a network directed graph and returns the result to CUC;

[0129] Step 3: The terminal device sends a TSN connection request and requirements to the CUC through the user configuration protocol, such as which terminal devices need to communicate, the period, size and latency of the TSN flow, etc. The CUC sends the connection request and requirements to the CNC through the UNI;

[0130] Step 4: The controller CNC divides these service flows into TT flows, AVB flows, and BE flows based on the QoS delay requirements of different flows. At the same time, when the bandwidth of the data flow reaches a certain threshold of the link bandwidth, it will be classified as UBF and mapped to different priority queues of the switch.

[0131] Step 5: According to the network physical topology and network requirements, CNC uses the multi-objective optimization and efficient routing method for mixed flow transmission proposed in this solution to perform routing calculations to obtain relevant path information that meets the connection requirements and is feasible. The routing calculation is based on T-DRL to obtain a feasible initial solution, which is then used for NSGA-II optimization;

[0132] Step 6: If such a path and transmission table exist, CNC configures all TSN switches along the calculated path with the calculated transmission table;

[0133] Step 7: CNC sends the transmission table to CUC, and CUC forwards it to the sending device;

[0134] Step 8: The sending device starts sending the TSN stream to the receiving device along the selected path according to the calculated transmission table.

[0135] Example 2: Process of CNC identifying and classifying TSN hybrid service flows

[0136] In order to make the expression of step 4 in Example 1 clearer, this embodiment takes the schematic diagram of the process of controlling the CNC to identify and classify the TSN hybrid service flow as an example, such as Figure 2 As shown in the figure, the identification and classification process of TSN hybrid service flows is introduced to meet the transmission requirements of various service flows. The specific identification and classification process is as follows:

[0137] Step 1: CNC receives the TSN service flow connection requirements sent by CUC through UNI, identifies the QoS attribute requirements of the service flow, and then extracts the characteristic requirements of the service flow, including latency and bandwidth size;

[0138] Step 2: CNC defines the classification rules for service flows and assigns PCP and VID to each service flow. The specific classification rules are as follows: first, the delay attribute of the service flow is extracted and classified for the first time; then, based on the service flow after the first classification, the bandwidth attribute is extracted and classified again.

[0139] Step 3: According to the classification rules, the classifier performs specific classification operations on the service flow. First, based on the information extracted from the features, it is classified into TT flow, AVB flow and BE flow according to the delay attribute; then, based on this, it is classified again into TT flow, UBF, AVB flow and BE flow according to the bandwidth attribute.

[0140] Step 4: Finally, the controller maps the classified service flows to different priority queues of the switch according to the assigned PCP and VID, waits for further processing, and selects a non-conflicting switch egress port to forward the flows.

[0141] Step 5: The service flows enter their respective priority queues for queuing, waiting for subsequent processing to select non-conflicting switch egress ports to forward the flows.

[0142] Embodiment 3: Specific process of a multi-objective optimization efficient routing method for mixed flow transmission in a time-sensitive network

[0143] In order to make the expression of step 5 in Example 1 clearer, this embodiment takes the flow chart of the multi-objective optimization efficient routing method for mixed flow transmission in a time-sensitive network as an example. Figure 3 As shown in the figure, this paper introduces the efficient routing algorithm process of TSN mixed business flow to meet the transmission requirements of various business flows. The proposed algorithm mainly includes three stages:

[0144] The first stage is the network modeling stage, where the controller CNC performs network modeling based on the network topology and service flow attributes as input for the subsequent two stages;

[0145] The second stage is the transformer-based deep reinforcement learning stage. When CNC receives the TSN service flow connection requirements sent by CUC through UNI, CNC first executes the T-DRL algorithm to select the optimal path that meets the requirements of various service flows according to the network topology and service flow requirements. Then, the solution generated by T-DRL is used as the initial solution of the NSGA-II algorithm optimization stage to improve the quality of the solution and obtain an ideal Pareto front. At the same time, this method can also solve the problem of time-consuming initial solution when traditional heuristic algorithms encounter complex constraint problems.

[0146] The third stage is the optimization stage of the NSGA-II algorithm. CNC encodes the solution generated by T-DRL in the second stage into the chromosome form of NSGA-II to create the initial population ; Then, the parent population is sorted according to the non-dominated level Sort and assign each chromosome a value equal to its non-dominated level, and set the initial cycle number to ; Use selection (based on non-dominated levels), crossover, and mutation operators in Create a descendant population on ; Determine whether the constraints are met. If the constraints are met, stop and return ; If the constraints are not met, the initial parent population and offspring population Merge to generate a merged population ; Then, a new population is created through selection, crossover and mutation operations , until the maximum number of iterations is met and an ideal Pareto optimal solution is obtained; finally, the transmission table results calculated for each business flow are sent to the TSN switch.

[0147] The specific algorithm flow of the efficient routing method for ultra-high bandwidth service flows in time-sensitive networks is as follows:

[0148] Step 1: CNC discovers the TSN physical network topology based on LLDP and models it as a network directed graph using the network modeling method And return the results to CUC;

[0149] Step 2: The terminal system uses the user configuration protocol to configure the communication terminal , the base period of business flow , Deadline , bandwidth size , Priority , population size Size, NSGA-II maximum number of generations , T-DRL maximum number of iterations The information is sent to CUC to obtain all the information in the network, and then CUC passes these communication requirements to CNC through UNI;

[0150] Step 3: When the CNC receives a service demand, it first generates a feasible solution based on the T-DRL algorithm according to the network topology and the requirements of various flows, that is, it selects a path that meets the requirements for each service flow for transmission;

[0151] Step 4: CNC uses the solution generated by the T-DRL algorithm in step 3 as the initial solution of NSGA-II, and then performs subsequent optimization through selection, crossover and mutation operations to obtain the ideal Pareto optimal solution;

[0152] Step 5: CNC checks whether there is a transmission path from the source end to the destination end of the service flow that has not been calculated. If so, it returns and executes steps 3 to 6, and saves the calculated preferred path of each service flow into the transmission path table. If there is no such situation, it continues to execute step 6;

[0153] Step 6: The controller CNC configures the calculated service flow transmission table to the switch through the NETCONF protocol, and sends its transmission table to the terminal device through the CUC.

[0154] Example 4: Transformer-based DRL solves a feasible initial solution and uses it for further optimization of the NSG-II algorithm

[0155] In order to make the second stage in Example 3 more clearly expressed, the first stage flow diagram of solving a feasible initial solution based on transformer DRL and using it for subsequent optimization by the NSG-II algorithm is taken as an example, as shown in FIG. Figure 4 As shown. First, the steps to solve the initial solution include modeling process, encoding process, decoding process and training process. The specific description is as follows:

[0156] Process 1: When solving MOHSFR using deep reinforcement learning, we model it as an MDP, where the state S, action A, state transition function, and reward R are defined as follows:

[0157] 1): State space

[0158] The routing process includes multiple system states. The system state in step t consists of the service flow state, SW node state and link state, which are described as follows. , destination node , Current location node , Required bandwidth , cumulative delay and priority Composition, expressed as The node state consists of the remaining bandwidth of all links connected to the SW node and can be expressed as The node state includes the node's M priority queues, which can be expressed as Each element represents a node The length of the Mth priority queue. Therefore, the system state space can be expressed as

[0159] 2): Action Space

[0160] In the MOHSFR problem, the agent must assign a path for each traffic flow from the source node to the destination node. First, the agent selects the next hop from the neighboring nodes for each traffic flow in the network. Then, the traffic flow is transferred to the next node and enters the corresponding priority queue. Therefore, the state Representation Flow will reach a node at step t where Is a vector representing the business flow The next hop is selected from the neighbor nodes of Representation Node Being Streamed Select as the next hop. Representation Node Not selected. The action sequence from step 0 to the last step T can be expressed as , where n represents the number of SWs. Here, each service flow Initially located at the source node ,use Indicates that the destination node is finally reached ,use express. Indicates business flow Flows to the SW node.

[0161] 3): Reward function

[0162] By performing actions , the routing paths of all data flows, as well as the arrival time and link bandwidth usage can be obtained. The total reward function is calculated according to the objective function established in steps S41, S42 and S43, which is specifically represented by the following formula:

[0163]

[0164] in , , and There are three objective functions respectively.

[0165] 4): State transfer function

[0166] Current system status The currently executed action will be Update to the next state Once the service flow selects the next hop, the link state will also be updated as the action is executed, which can be expressed as:

[0167]

[0168] In addition, the transmission delay of the TSN service flow and the number of flows on its path will also increase, as shown below:

[0169]

[0170]

[0171] Process 2: Information encoding process based on transformer architecture:

[0172] 1): The encoder consists of a linear layer and L identical layers. First, the encoder uses the linear layer to embed the network topology features and each TSN service flow into a high-dimensional space and obtain the initial embedding information. And we use separate linear projections to process different information of TSN business flows and network topology, as shown below:

[0173]

[0174] The network topology needs to embed node information, link bandwidth capacity and maximum load information, while the TSN service flow needs to embed source address, destination address, priority, deadline and bandwidth requirement information. It means connecting information of different dimensions together, and are trainable linear projection parameters, where express In the algorithm of the present invention, The value of is set to 128.

[0175] 2): Then, the initial information is embedded After L identical layers of processing, the final information embedding is obtained Each layer consists of a multi-head attention layer (MHA), a skip connection (SC) layer, a batch normalization (BN) layer, and a fully connected feed-forward (FF) layer. The input of layer l is the output of the previous layer, expressed as . First, the attention value is calculated by MHA, which uses the dimension H head, and set H=8, For each head , first query ,key , and value , which is calculated as follows:

[0176]

[0177] in , , is a trainable parameter of the l-sublayer.

[0178] 3): Get the Head attention value through the softmax function , then Connect projection to dimension Get multi-head attention value , the main formula is as follows:

[0179]

[0180] in are the trainable parameters of sub-layer l.

[0181] 4): Further process the attention value through SC and BN to obtain Finally, the output of layer l information embedding is obtained through the FF layer and the SC layer. The calculation process is shown in the following formula:

[0182]

[0183] Process 3: Information decoding process based on transformer architecture

[0184] During the decoder process, the decoder outputs a probability vector for each node to be selected. Here, various constraints are established, which indicates that the existence of these constraints will cause some incoming links or nodes to be inaccessible. Therefore, a shielding rule is used to exclude nodes or incoming links that cannot be accessed in the current step due to constraints. The main shielding rules include: (1) switch nodes that have been accessed by the same data flow are shielded; (2) nodes whose data flow bandwidth exceeds the remaining bandwidth of the link are shielded; (3) nodes whose data flow delay to the node exceeds the deadline are shielded. Whether the i-th fixed point is shielded in step t is defined as follows:

[0185]

[0186] Furthermore, the main decoding process of process 3 includes:

[0187] 31): The decoder uses the context information from the encoder to decode at each step and generates a probability vector for selecting the next hop node for each service flow. At each step t, the agent selects the next hop node according to the current state Making decisions , where the decoder obtains the context Contains the entire graph The embedded information of the business flow at the previous node position , specifically expressed as: .

[0188] 32): Then, the current state is calculated through the MHA layer Contextual information under Among them, the query vector of MHA is The context information is obtained by , and the key vector and value vector are defined as the embedding of the node:

[0189]

[0190] 33): Calculate the probability vector of each node being selected through the attention mechanism, that is, calculate Compatibility with all nodes:

[0191]

[0192] 34): Finally, combining the data flow and the compatibility vector between nodes, the probability vector of each hop node selection is output through the softmax function:

[0193]

[0194] Here C is a large negative number. The decoder will repeat and select the next hop node at each step until the requirements of all flows are met and they all reach the destination node.

[0195] Process 4: Training process The baseline policy gradient method is applied to the neural network to train the parameters θ. The algorithm consists of a policy network and a baseline network. The two networks have the same network structure. The only difference is that the policy network selects actions based on the sampling of the probability vector, while the baseline network selects the action with the highest probability (greedy strategy). The gradient of the loss function can be defined as: ,in and Represent the rewards obtained by the policy network and the baseline network respectively. During each training process, the model will undergo a paired t-test, and if the test result is significant at 95% confidence, the parameters of the policy network will replace the parameters of the baseline network. This step ensures that the updated parameters achieve statistically significant improvements over the previous parameters, thereby further improving model performance.

[0196] The solution generated based on TDRL is used as the initial solution of the NSGA-II algorithm for further optimization.

[0197] In order to make the expression of the third stage in Example 3 clearer, a flow chart of further optimizing the solution generated based on TDRL as the initial solution of the NSGA-II algorithm is taken as an example, as shown in FIG. Figure 5 The specific steps of the algorithm are as follows:

[0198] Step 1: Initialization of NSGA-II population

[0199] A set of feasible initial solutions for routing and path selection is obtained through the T-DRL algorithm. Each solution represents the routing path of the service flow in TSN to meet the given optimization goal. The initial solution generated by DRL is converted into a population representation suitable for NSGA-II. That is, the routing path is encoded as a chromosome. Usually the NSGA-II population is a group of individuals, each of which represents a solution. In this case, a single chromosome represents a complete solution to the problem, which can be defined as follows:

[0200]

[0201] where path1, path2, path3 and path mIt represents the path allocation of different flows. Simply put, the genes in the chromosome can be regarded as a set of routing paths. Each gene represents a data flow and encodes the path information of the data flow. The combination of these path selections constitutes the chromosome, which determines the data flow routing configuration in the entire network. The gene encoding method adopts integer encoding, where each integer represents a node on the path, which is expressed as follows:

[0202]

[0203] Through the above process, a set of solutions generated by T-DRL can be integrated into the initial population of NSGA-II to create a parent population of size Gen: .

[0204] Step 2: Calculate the fitness value of different targets for each chromosome, compare each chromosome with other chromosomes to check whether it is dominant, and classify the parent population according to the non-dominance level of the chromosome. Sort them and assign each chromosome a fitness value equal to its non-dominated level. And let the number of initial cycles be Specific:

[0205] First, the fitness function algorithm needs to be defined: the quality of the chromosome is generally evaluated by the fitness value. In the present invention, the focus is on minimizing the end-to-end delay and maximum link bandwidth utilization of GSF in the TSN network, as well as minimizing the ESF path degree. Therefore, a separate fitness function value is defined for each optimization objective, and the quality of each solution is evaluated according to different objectives, and they are set to be the same as the corresponding objective function. Therefore, the definition of the fitness function is as follows:

[0206]

[0207] Then define the fast non-dominated sorting rule: After generating the initial population and evaluating multiple fitness values ​​of the chromosomes, apply the non-dominated sorting algorithm to the population to identify all non-dominated frontiers , that is, all non-dominant frontiers are classified according to the dominance level of chromosomes. In the present invention, two chromosomes and The dominance of chromosomes is first identified by comparing their end-to-end delays and path degrees. dominate , if the following conditions are met.

[0208] (1): Objective function and Better than and ,regardless In other words, ;

[0209] (2): Objective function and reaches the minimum value, and and At least one of them does not reach the minimum value, regardless of In other words, , (here and The targets are and Otherwise

[0210] (3): In all targets Better than or equal to , and in Strictly dominate at least one of the targets .

[0211] Step 3: Use selection (based on non-dominated level), crossover and mutation operators in Create a descendant population with a population size of Size Specific:

[0212] (1) Selection operator: The main idea of ​​the selection operator is to give priority to the most fit individuals in the population during the crossover process. In the selection phase, a pair of chromosomes is selected from the population for crossover and mutation operations. In traditional single-objective problems, selection usually depends on the fitness value, that is, chromosomes with higher fitness values ​​are more likely to be selected. However, for the proposed algorithm, since the problem is multi-objective, the selection process adopts a method based on dominance level and crowding distance, that is, a higher crowding distance and a lower non-dominated level are selected. In addition, a sufficient number of chromosomes should be selected in the selection phase so that they can generate Size number of sub-chromosomes after the crossover operation.

[0213] (2) Crossover operator: The crossover operator combines the genes of two individuals by extracting information from both parents and recombining it into a new offspring, and allows the inheritance of this information. The crossover operator proposed in the present invention adopts a single-point crossover method, allowing non-repeating routing groups of different individuals at the same location to be exchanged to avoid the same routing being repeated again. The crossover rate here is set to 90%.

[0214] (3) Mutation operator: The mutation operator allows the population to acquire new genes to maintain the diversity of the population and prevent the algorithm from converging prematurely. The algorithm randomly selects two positions within the chromosome from the corresponding routing set and exchanges the information of these positions. The mutation rate here is set to 10%.

[0215] Step 4: Determine whether the constraints are met. If so, stop and return. ; If the constraint condition is not met, execute step 5;

[0216] Step 5: Initial parent population and offspring population Merge to generate a merged population ;

[0217] Step 6: Then for the group Apply fast non-dominated sorting to sort non-dominated frontiers with different non-dominated levels Classify and then create the next generation population of size Size based on the sorted crowding distance , the specific operations are as follows:

[0218] The first step is to Identifying non-dominated frontiers , and set ;

[0219] The second step is when When Crowding distance of the intermediate solution;

[0220] Step 3: If , then let ;

[0221] Step 4: Otherwise ( ), according to the crowding distance Sort and add arrive The least crowded A solution;

[0222] Step 5, then set ; Return to execute steps 2 to 5 until the new population The process ends when the size of meets the required population size Size;

[0223] Step 6: Generate a new population with a population size of Size , and set ;

[0224] Step 7: Determine whether the termination condition is met If not, return to step 3. Apply selection (based on non-dominated level), crossover, and mutation operators to create the offspring population ; If satisfied, output the Parto optimal solution obtained after NSGA-II optimization, that is, select the optimal path from the sender to the receiver that meets various flow requirements.

[0225] The present invention can minimize the path blocking rate of UBF, minimize the average end-to-end delay of RSF, and maximize the available bandwidth rate under the premise of meeting the service flow requirements and constraints, thereby obtaining a set of Pareto optimal solutions. Compared with the traditional method, the method of the present invention can significantly improve network performance in the scenario of mixed deployment of UBF and RSF, and meet the high requirements of time-sensitive networks.

[0226] In summary, the present invention significantly improves the solution quality and convergence performance through a multi-objective optimization method, and solves the problem that the initial solution of the traditional heuristic algorithm takes a long time; when UBF exists, the method of the present invention can achieve better load balancing, reduce network congestion and path blocking, and improve the stability and reliability of the overall network; effectively reduce the average end-to-end delay of RSF, meet the delay requirements of different types of business flows, and improve user experience; by maximizing the available bandwidth rate, the utilization efficiency of network resources is improved, bandwidth waste is avoided, and it is ensured that the high bandwidth demand of UBF is fully met; the method of the present invention can be flexibly applied to ultra-high bandwidth business flow routing optimization in various scenarios such as cloud computing, communication and network services, and has broad application prospects.

[0227] It should be noted that the above-mentioned embodiments only express several implementation methods of the present application, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present application, which all belong to the protection scope of the present application. Therefore, the protection scope of the patent of this application shall be subject to the attached claims.

Claims

1. An efficient routing method for ultra-high bandwidth traffic in a time-sensitive network, characterized in that: include: S1: The centralized user configuration CUC initiates a request to retrieve the network physical topology to the network centralized controller CNC. The CNC discovers the TSN network topology and abstracts it as a network directed graph and returns it to the CUC; S2: The terminal system sends TSN connection requests and resource requirements to CUC through the user configuration protocol, and CUC passes these connection requests and resource requirements to CNC; S3: CNC performs service modeling at the switch based on service flow attributes. CNC classifies service flows based on service characteristics, assigns PCP and VID to each service flow, and maps them to different priority queues of the switch. S4: Obtain basic information of the entire network, including TSN network topology and data flow information set, and establish a multi-objective optimization mathematical model considering multiple routing constraints based on this basic information; S5: CNC performs routing calculations based on the network physical topology, network requirements, and the established multi-objective optimization mathematical model containing multiple routing constraints to obtain relevant path information that meets the connection requirements and is feasible; The routing calculation first uses the Transformer-based improved deep reinforcement learning T-DRL to obtain a feasible initial solution as an alternative path; S6: CNC selects the optimal path from the sender to the receiver that meets the service flow requirements from the candidate paths selected by performing routing calculation through the NSGA-II algorithm; S7: CNC traverses to see if there are any uncalculated paths between the sender and the receiver. If yes, it returns to execute steps S3-S6. If no, it executes step S8. S8: CNC encapsulates the calculated path results into a transmission table and configures it to the TSN switch, and sends it to the TSN terminal device through CUC.

2. The efficient routing method for ultra-high bandwidth traffic in a time-sensitive network according to claim 1, characterized in that: The step S1 comprises: Before the network starts running, CUC initiates a request to retrieve the network physical topology to the network centralized controller CNC through the user network interface UNI to obtain TSN network topology information; CNC discovers TSN network topology information and node information based on the link layer discovery protocol LLDP, and models it as a TSN directed graph through the network modeling method. ,in and Represent the node set and edge set respectively, and return the results to CUC; the specific modeling process is as follows: The TSN network topology is modeled as a directed graph ,in represents a set of n switch SW nodes and a set of λ terminal equipment ES nodes, is the set of physical links, where Representation Node and nodes The connected links; TSN supports full-duplex, and each full-duplex link is regarded as two separate directed links and ; represents an adjacency matrix consisting of elements 0 and 1, where if ,but ,if ,but ; Each link Defined by triples ,in Yes Link The bandwidth capacity of The link delay is expressed in µsec, given by The transmission delay , queuing delay and Link The propagation delay composition, For link The number of TSN streams.

3. The efficient routing method for ultra-high bandwidth service flows in a time-sensitive network according to claim 1, characterized in that: The step S2 includes: the terminal system configures the communication terminal through the user configuration protocol , the base period of business flow , Deadline , bandwidth size , Priority , population size Size, maximum number of iterations of NSGA-II , T-DRL maximum number of iterations This information is sent to CUC, which passes these connection requests and resource requirements to CNC via UNI as follows: Each communication task is abstracted as a business flow, which is defined by tuples. ,in and Business Flow The source and destination nodes, Representation Flow The bandwidth required, and Respectively represent the flow The deadline and base period, Representation Flow priority; select m paths that meet the needs of m business flows for transmission.

4. The efficient routing method for ultra-high bandwidth traffic in a time-sensitive network according to claim 1, characterized in that: The step S3 includes: the CNC performs service modeling at the switch according to the service flow attributes. First, the controller CNC divides these service flows into TT flows, AVB flows and BE flows based on the QoS delay requirements of different flows; then, when the bandwidth of the data flow reaches a certain threshold of the link bandwidth, it will be identified and classified as UBF; at the same time, PCP and VID are allocated to each service flow, and they are mapped to different priority queues of the switch; The specific process of business modeling is as follows: S31: After the data flow enters the switch, the switch checks the PCP field and the VLAN ID field in the IEEE 802.1Q VLAN tag in the Ethernet frame header; by identifying these fields, the switch determines the attribute information or statistical information of the data flow to further classify and determine its traffic type and priority information; S32: according to the delay attribute information of the data flow, the data flow is classified into different traffic types, including TT flow, AVB flow and BE flow; S33: Based on step S32, the data flows whose bandwidth reaches a certain threshold of link bandwidth among the TT flow, AVB flow and BE flow are additionally classified as ESF, and finally the data flows are divided into TT, ESF, AVB and BE flows; Represents a collection of TSN service flows. An integer, including subset TT streams, ESF streams, AVB streams and BE stream sets: ; S34: The classified data flows enter respective priority queues in the switch, each queue represents a traffic type and priority, and these queues are used for queuing by priority; S35: The CNC establishes a switch queue information table according to the transmission rules of each type of service flow, and the switch selects a non-conflicting egress port according to the queue information table to forward each flow.

5. The efficient routing method for ultra-high bandwidth traffic in a time-sensitive network according to claim 1, characterized in that: In step S4, a multi-objective optimization mathematical model considering multiple routing constraints is established, including minimizing the end-to-end delay of RSF, minimizing the available bandwidth rate, and considering finding a path with a large remaining bandwidth and a small number of flows for the ultra-high bandwidth flow UBF as the goal for multi-objective optimization. The specific objective function is as follows: S41: Objective function 1 is to minimize the total end-to-end delay of TT stream, AVB stream and BE stream , In the formula, is the weighted sum of the total end-to-end delay, and its value is between [0, 1]. , and represents the weight value, , , and They represent the normalized average delays of TT stream, AVB stream and BE stream respectively; S42: Objective function 2 is to maximize the available bandwidth rate , is the bandwidth utilization, Represented as an ordered list of physical links , business flow From the source node To the destination node All nodes passed through are included in this list; S43: Objective function 3 is to minimize the path blocking rate of UBF by selecting a path with a remaining bandwidth greater than a set threshold and a flow number less than a set threshold to transmit UBF. The specific objective function is as follows: , In the formula, and represents the weight, and there is ; Indicates the maximum value of the remaining bandwidth among all valid paths. Indicates the conversion value of the number of TSN streams, as follows: , In the formula, Indicates the minimum number of TSN flows in all valid paths.

6. The efficient routing method for ultra-high bandwidth traffic in a time-sensitive network according to claim 5, characterized in that: The normalized average delay of the TT stream, the AVB stream and the BE stream in step S41 is specifically expressed as: S411: The normalized average delay of the TT flow is defined as follows: , In the formula, Indicates the number of TT flows, is a constant, indicating the maximum acceptable delay of the TT flow. Express demand end-to-end latency; S412: The normalized average delay of the AVB stream is defined as follows: , In the formula, Indicates the number of AVB streams, is a constant representing the worst-case end-to-end delay of the AVB stream; S413: Focus on the limited worst-case BE traffic delay without degrading the performance of high-priority traffic; the normalized average delay of BE flows is defined as follows: , In the formula, represents the number of BE flows, is a constant, indicating the acceptable delay threshold of BE flow. In addition, ; The requirement in step S411 Represents TSN streams, including TT streams, AVB streams, BE streams, and UBF stream sets, a collection of TSN streams: ,in , then the demand End-to-end delay It can be expressed as follows: , In the formula, Indicates link delay, including transmission delay , Propagation Delay and queuing delay ; Indicates that the TSN flow is at the node and The transmission direction between and Specifically, it can be expressed as: (1) ; (2) 。 7. The efficient routing method for ultra-high bandwidth service flows in a time-sensitive network according to claim 5, characterized in that: The step S42 is defined as follows: (1) Effective routing: For For example, an effective route Defined as meeting its deadline Requested routing, Represented as an ordered list of physical links , From its source address To the destination address Traverse these physical links; (2) Link load: For each link ,use Indicates link The load through the link The sum of the sizes of all flows of ; formally, its load is defined as: , In the formula, Representing data flow The bandwidth size; (3) Link bandwidth utilization: For each link ,use Indicates link Bandwidth utilization, link The bandwidth utilization is: , In the formula Indicates the bandwidth of the link. The bandwidth of all links is the same. (4) Remaining bandwidth of routing: The remaining bandwidth is used It means that The minimum value of the remaining bandwidth of all links in is expressed as: , (5) Number of TSN flows routed: Number of TSN flows It means that The maximum number of TSN flows on all links in , which is specifically expressed as: , In the formula Indicates that the link The number of TSN flows is obtained ; (6) The access degree of each node: represents the kth stream Whether the node is visited , defined as: , Among them, each node All in the same stream Visit once; for each node ,have: , (7) Bandwidth utilization: Routing Bandwidth utilization It means that The maximum bandwidth utilization of all links in is expressed as: 。 8. The efficient routing method for ultra-high bandwidth service flows in a time-sensitive network according to any one of claims 5 to 7, characterized in that: The objective function established in steps S41, S42 and S43 needs to consider multiple routing constraints to implement a multi-objective routing optimization model, and the constraints are specifically as follows: 1): Latency constraints of TSN data streams ①: , ②: , ③: , ④: , Among them, constraints ① and ② limit the maximum end-to-end delay allowed for TT flow and AVB flow respectively; constraint ③ stipulates that the transmission of BE flow should be completed within the deadline in the worst case, and it needs to be guaranteed that it is transmitted within the current time period; constraint ④ ensures that the end-to-end delay of the flow with higher priority is lower, among which and is the delay of two different paths; 2): Access degree constraints of nodes ①: , ②: , Among them, constraint ① avoids loops, a node can only be accessed once by the same data flow, and constraint ② limits Each data stream must reach the destination node; 3): Bandwidth constraints ①: , ②: , ③: , in, Indicates link The sum of the bandwidths of the flows transmitted on the link. Constraint ① prevents the current link load from exceeding the link bandwidth; Constraint ② limits the bandwidth utilization of each link to no more than 10% of its capacity; Constraint ③ indicates that the ESF bandwidth meets its link bandwidth requirements and does not exceed 10% of the remaining bandwidth; 3): Constraints on the number of TSN flows on the path ①: , Among them, constraint ① prevents the number of flows on the path allocated by UBF from exceeding This threshold.

9. The efficient routing method for ultra-high bandwidth service flows in a time-sensitive network according to claim 1, characterized in that: In step S5, the Transformer-based improved deep reinforcement learning T-DRL is used to obtain a feasible initial solution as an alternative path, specifically: The TSN multi-objective hybrid business flow routing problem MOHSFR is modeled as a Markov decision problem MDP; then the policy network uses the Transformer architecture to model the agent that solves MOHSFR. The model includes an encoding process, a decoding process, and a training process; by inputting the network directed graph, flow characteristics, known requirements, and a trained Transformer-based DRL model, a set of initial flow paths is output to represent individuals in the solution, and then the initial solution path is used as the input of step S6; The specific implementation steps include: S51: Modeling the MOHSFR problem as an MDP: S511: Define MDP elements: (1) Agent: The DRL model acts as an agent and is responsible for making decisions on path selection actions; (2) State set : The state set includes the network state S t All possible values ​​of ; (3) Action set : The action set includes all possible path selection actions A t ; (4) Reward Function : Define a reward function to calculate the reward based on the performance of the path; S512: MDP modeling. After defining the MDP elements, establish the MDP model, including the state transition probability and reward function. The state transition function finally outputs the probability of all possible next states based on the given current state and action. S52: The encoding process based on the transformer architecture first uses a linear layer to embed the network topology features and each TSN service flow into a high-dimensional space and obtain the initial embedding information ; then go through The same layers are processed to obtain the final information embedding ; Each layer consists of a multi-head attention layer MHA, a skip connection layer, a batch normalization BN layer and a fully connected feed-forward FF layer; S53: Based on the decoding process of the transformer architecture, the decoder uses the context information from the encoder to decode at each step and generates a probability vector for selecting the next hop node of each service flow; the agent selects the path and action at each time step according to the strategy of the DRL model, constructs the final path solution, and then outputs the initial solution and returns a set of initial paths generated by the encoder process, which constitute part of the initial population; S54: Training process: Using policy gradients to train parameters in neural networks , used to update the parameters of the DRL model, and the model provides feedback based on the generated paths and performance indicators to optimize the policy network; S55: repeating steps S51-S55 to generate different initial solutions or optimization paths; S56: After the training is completed, a set of initial flow paths is output to represent individuals in the solution.

10. The efficient routing method for ultra-high bandwidth service flows in a time-sensitive network according to claim 1, characterized in that: In step S6, the CNC optimizes the candidate paths selected in the routing calculation using a non-dominated sorting genetic algorithm NSGA-II to select a preferred path from the sending end to the receiving end that meets the service flow requirements; Specifically include the following steps: S61: First set the initialization parameters, including the population size and the maximum number of iterations of NSGA-II , crossover rate , mutation rate ; S62: Convert the initial solution obtained in step S5 into the initial population of NSGA-II to create a parent population of size Size ; S63: Parent populations according to non-dominance level Sort and assign each chromosome a value equal to its non-dominated level; and let the initial cycle number be ; S64: Using non-dominated level-based selection, crossover, and mutation operators in Create a descendant population on ; S65: Determine whether the constraint conditions are met. If the constraint conditions are met, stop and return ; If the constraint condition is not met, execute step S66; S66: Initial parent population and offspring population Merge to generate a merged population ; S67: From Identifying non-dominated frontiers , and set ; S68: When When The crowding distance of the intermediate solution; if , then let ; Otherwise, according to the crowding distance Sort and add arrive The least crowded solution; then let ; S69: Determine whether the termination condition is met. If not, set , return to step S64, The offspring population is created by applying selection, crossover and mutation operators based on non-dominated level. ; If satisfied, output the solution obtained after NSGA-II optimization; that is, select the optimal path from the sender to the receiver that meets various flow requirements.

Citation Information

Patent Citations

  • Industrial heterogeneous network scheduling method oriented to TSN and non-TSN interconnection

    CN111600754A

  • Multi-path joint scheduling method in time-sensitive network

    CN115460130A

  • Routing scheduling optimization method and system based on time-sensitive network

    CN118400313A

  • Inner door installation structure for electric control box

    KR1020250028870A

Cited By

  • Bandwidth resource allocation method and system based on GPON optical access network

    CN119402785A

  • Multi-path optimization method, device and system for network routing

    CN119449693A

  • Multi-path optimization method, device and system for network routing

    CN119449693B

  • Multi-substream parallel forwarding scheduling method for novel intelligent computing center network

    CN119520378A

  • Multi-subflow parallel forwarding scheduling method for new intelligent computing center network

    CN119520378B