An intelligent routing method based on maximizing the transmission value of multiple types of services

Through deep neural network combining GNN+LSTM and DRL architecture, business value and reward functions are set, which solves the problem that existing routing solutions fail to balance the needs of multiple types of services, and maximizes the system transmission value, throughput and important services. Balance of delivery rates.

CN116346702BActive Publication Date: 2024-11-19THE 54TH RESEARCH INSTITUTE OF CHINA ELECTRONICS TECHNOLOGY GROUP CORPORATION
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202310405416.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-04-17
Publication Date
2024-11-19
Estimated Expiration
2043-04-17

AI Technical Summary

Technical Problem

The existing routing scheme based on reinforcement learning failed to effectively consider the different needs of multiple services, especially the balance between delivery rate and throughput of important services, resulting in the failure to maximize the transmission value.

Method used

Design an intelligent routing method based on deep neural networks. By setting business value and new reward functions, combining GNN+LSTM and DRL architectures, multiple types of service traffic forwarding paths are optimized to balance throughput and important service delivery rates.

Benefits of technology

The optimization of multi-category business traffic forwarding paths has been achieved, which has improved the system transmission value, met the needs of different types of services, and has improved the maximization of business value.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116346702B_ABST
    Figure CN116346702B_ABST
Patent Text Reader

Abstract

The present invention proposes an intelligent routing method based on maximizing the transmission value of multiple types of services, which belongs to the field of network artificial intelligence. The present invention is based on a centralized network architecture. First, the value is set for different types of services, and the service characteristics, service value, and topology information are input into a deep neural network structure. The neural network outputs the next hop node of the path, and then uses the node as the source node for the neural network to continue to output the next hop until the complete path is output. The reward is calculated, and the network is trained according to feedback until the system converges. The present invention can solve the traffic forwarding path problem of multiple types of services and meet the needs of different types of services by designing a deep neural network architecture for different types of services; at the same time, a new routing reward scheme is designed to maximize the transmission value of the system and balance the contradiction between throughput and important service delivery rate.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of network artificial intelligence, and in particular to an intelligent routing method based on maximizing the transmission value of multiple types of services. Background Art

[0002] In recent years, with the rapid development of mobile communication technology, cloud computing technology and Internet applications, a large number of mobile terminals and smart IoT devices are constantly connected to the network, resulting in the switching system in the communication network experiencing an explosive growth in network service traffic. At the same time, emerging services have also put forward different QoS transmission requirements for the Internet. Therefore, it is urgent to solve the problem of forwarding paths for multiple types of service traffic. In addition, considering that the reward function of the current reinforcement learning-based routing scheme is generally based on a single indicator such as throughput, delay, packet loss rate, or a weighted combination of several indicators, it does not take into account the delivery rate of important services. Therefore, it is necessary to design a new reward scheme to measure the pros and cons of routing path selection. Summary of the invention

[0003] In view of this, the present invention proposes an intelligent routing method based on maximizing the transmission value of multiple types of services based on a centralized network architecture. By designing a deep neural network structure, the traffic forwarding path problem of multiple types of services in the network is solved, and the throughput and important business delivery rate indicators can be optimized through a new reward scheme, thereby improving the system transmission value.

[0004] Based on the above purpose, the technical solution provided by the present invention is:

[0005] An intelligent routing method based on maximizing the transmission value of multiple types of services includes the following steps:

[0006] Step 1: Set business value for different types of businesses;

[0007] Step 2: Input the current traffic characteristics, business value and topology information into the deep neural network structure, and the deep neural network outputs the next hop node of the path;

[0008] Step 3: The next hop node is used as the source node and the deep neural network continues to output the next hop until the complete path is output, the reward is calculated, and the network is trained until convergence based on the feedback.

[0009] Furthermore, the specific method for step 1 is:

[0010] The service value v is set according to the service priority and bandwidth requirements. The definition of v is as follows:

[0011] v=v baseline +ln(bandwidth demand )

[0012] Among them, vbaseline The value benchmark is set based on different business priorities. The higher the priority, the higher the value. baseline The larger the value, the bandwidth demand is the business bandwidth demand. The greater the bandwidth demand, the greater v is. When the bandwidth demand reaches a set level, v tends to be stable.

[0013] Furthermore, the deep neural network structure in step 2 is specifically as follows:

[0014] Deep neural network structures include GNN+LSTM architecture and DRL architecture;

[0015] The input of the GNN+LSTM architecture is topology information, business flow features, and current traffic features, and the output is the extracted business features, which are used as the input of the DRL architecture. Among them, the business flow features include source nodes, destination nodes, business value, bandwidth requirements, delay requirements, and packet loss requirements.

[0016] The input of the DRL architecture is the output of the GNN+LSTM architecture in the first part, and the output is the next hop of the route.

[0017] Furthermore, the reward function for calculating the reward in step 3 is specifically:

[0018] The reward function is as follows:

[0019] r=v+C 1 ×u(delay demand -delay)+C 2 ×u(loss demand -loss)

[0020] Where delay demand To meet the service delay requirements, loss demand is the service packet loss rate requirement, delay is the transmission delay, loss is the packet loss rate, u(xx 0 ) is a step function. When x>x 0 When u(xx 0 )=1, when x <x 0 When u(xx 0 )=1,C 1 ×u(delay demand -delay) indicates that when the transmission delay is less than the service delay requirement, C 1 Reward, C 2 ×u(loss demand -loss) means that when the packet loss rate is less than the service packet loss rate requirement, C 2 The reward is v, and v is the business value.

[0021] It can be seen from the above description that the beneficial effects of the technical solution of the present invention are:

[0022] 1. Designed a deep neural network architecture for different types of services, which can solve the traffic forwarding path problems of multiple types of services and meet the needs of different types of services;

[0023] 2. A new routing reward scheme has been designed to maximize the transmission value of the system and balance the contradiction between throughput and important business delivery rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Schematic diagram of the neural network architecture of the present invention.

[0025] Figure 2 This is the training flow chart of the neural network. DETAILED DESCRIPTION

[0026] In order to facilitate technical personnel in this field to understand the technical solution of this patent, and at the same time, to make the technical purpose, technical solution and beneficial effects of this patent clearer and to fully support the scope of protection of the claims, the technical solution of this patent is further and more detailedly explained in the form of specific cases.

[0027] An intelligent routing method based on maximizing the transmission value of multiple types of services. In this method, values ​​are first set for different types of services, and service characteristics, service value, and topology information are input into a deep neural network structure. The neural network outputs the next hop node of the path, and then the node is used as the source node by the neural network to continue to output the next hop. After the complete path is output, the reward is calculated, and the network is trained according to feedback until the system converges.

[0028] The specific steps include:

[0029] Step 1: Set the value v of the service according to the service priority and bandwidth requirements. The definition of v is as follows:

[0030] v=v baseline +ln(bandwidth demand )

[0031] Among them, v baseline The value benchmark is set based on different business priorities. The higher the priority, the higher the value. baseline The larger the value, the bandwidth demand is the service bandwidth demand in kbps. The greater the bandwidth demand, the greater v is. However, when the bandwidth demand reaches a certain level, v will tend to be stable.

[0032] Step 2: Construct a deep neural network structure such as Figure 1As shown, the first module is the input information, which mainly includes topology information, business flow characteristics (source node, destination node, value, bandwidth requirement, delay requirement, packet loss requirement) and current traffic characteristics, and outputs the common characteristics of the business; the second module is the GNN+LSTM architecture, which uses the input information of module one to extract business features and uses it as the input of the DRL architecture; the third module is the DRL architecture for different business types, the input is the output of the GNN+LSTM architecture, and the output is the next-hop router.

[0033] Step 3: The next hop node is used as the source node and the deep neural network continues to output the next hop until the complete path is output, the reward is calculated, and the network is trained until convergence based on the feedback.

[0034] The reward function is defined as follows:

[0035] r=v+C 1 ×u(delay demand -delay)+C 2 ×u(loss demand -loss)

[0036] Among them, delay demand To meet the service delay requirements, loss demand is the packet loss rate requirement of the service. u(xx 0 ) is a step function. When x>x 0 When u(xx 0 )=1, when x <x 0 When u(xx 0 )=1. The second item on the right indicates that when the transmission delay is less than the service delay requirement, C is given 1 The third item on the right indicates that when the packet loss rate is less than the service packet loss rate requirement, C is given 2 When the transmission delay is greater than the delay requirement and the packet loss rate is greater than the packet loss rate requirement, no reward is given; v is the value of the successfully transmitted service. By considering both the important service value and the throughput, the contradiction between the two can be balanced, helping the system to make the decision with the maximum reward.

[0037] The training process of the neural network is as follows Figure 2 As shown, the specific steps are as follows:

[0038] Step 1: Set the number of episodes and steps. Here, it is set to episode. 0 and step 0 ;

[0039] Step 2: episode = 1;

[0040] Step 3: step = 1; R = 0;

[0041] Step 4: Generate new business;

[0042] Step 5: Input topology information, business flow characteristics (source node, destination node, value, bandwidth requirement, delay requirement, packet loss requirement), and current traffic characteristics into the neural network, and output the next hop of the route;

[0043] Step 6: Determine whether the complete path is obtained. If not, go to step 5 and continue to output the next hop route. If the complete path is obtained, calculate the reward r, and then calculate R = R + r.

[0044] Step 7: step = step + 1, determine whether step is less than or equal to step 0 , if yes, go to step 4, otherwise episode = episode + 1;

[0045] Step 8: Determine whether episode is less than or equal to episode 0 , if yes, go to step 3, otherwise the training ends.

[0046] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An intelligent routing method based on maximizing the transmission value of multiple types of services, It is characterized in that The following steps are involved: Step 1: Set business value for different types of businesses; Step 2: Input the current traffic characteristics, business value and topology information into the deep neural network structure, and the deep neural network outputs the next hop node of the path; Step 3: The next hop node is used as the source node and the deep neural network continues to output the next hop until the complete path is output, the reward is calculated, and the network is trained until convergence based on the feedback; Among them, the specific method for step 1 is: The service value v is set according to the service priority and bandwidth requirements. The definition of v is as follows: v=v baseline +ln(bandwidth demand ) Among them, v baseline The value benchmark value is set based on different business priorities. The higher the priority, the higher the value. baseline The larger the value, the bandwidth demand is the business bandwidth demand. The larger the bandwidth demand, the larger v is. When the bandwidth demand reaches the set level, v tends to be stable. Among them, the deep neural network structure in step 2 is specifically: Deep neural network structures include GNN+LSTM architecture and DRL architecture; The input of the GNN+LSTM architecture is topology information, business flow features, and current traffic features, and the output is the extracted business features, which are used as the input of the DRL architecture. Among them, the business flow features include source nodes, destination nodes, business value, bandwidth requirements, delay requirements, and packet loss requirements. The input of the DRL architecture is the output of the GNN+LSTM architecture in the first part, and the output is the next hop of the route; Among them, the reward function for calculating the reward in step 3 is specifically: The reward function is as follows: r=v+C 1 ×u(delay demand -delay)+C 2 ×u(loss demand -loss) Where delay demand To meet the business latency requirements, loss demand is the service packet loss rate requirement, delay is the transmission delay, loss is the packet loss rate, u(xx 0 ) is a step function. When x>x 0 When u(xx 0 )=1, when x <x 0 When u(xx 0 )=0,C 1 ×u(delay demand -delay) indicates that when the transmission delay is less than the service delay requirement, C 1 Reward, C 2 ×u(loss demand -loss) means that when the packet loss rate is less than the service packet loss rate requirement, C 2 The reward is v, and v is the business value.

Citation Information

Patent Citations

  • SDN (Software Defined Network) core network QoS (Quality of Service) routing optimization algorithm based on reinforcement learning

    CN112822109A

  • Heterogeneous network slice scheduling method based on deep reinforcement learning

    CN113891401A

  • Routing traffic adjustment method, device and controller

    WO2015021615A1

  • Algorithm model determining method, path determining method, electronic device, SDN controller, and medium

    WO2022116957A1