Routing optimization method, control device and medium

Abstract

The invention relates to the technical field of communication, in particular to a routing optimization method, a control device and a medium, and the method comprises the steps: obtaining real-time data of a current network state; a simulation environment is established according to the real-time data, multiple routing strategies are simulated in the simulation environment, the effect of the multiple routing strategies is determined through a reinforcement learning algorithm, and the utilized reinforcement learning algorithm comprises the steps of constructing a network structure, optimizing parameters and establishing a reward function; and optimizing the current network routing table and updating the routing information according to the effects of the determined multiple routing strategies. Therefore, the problems that in the prior art, static configuration and manual intervention are relied on, real-time performance and intellectualization are lacked, and network topology changes and dynamic environment requirements cannot be coped with are solved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a routing optimization method, a control device and a medium. Background Art

[0002] In modern large-scale enterprise networks, with the expansion of business scope and the acceleration of technology iteration, the need to connect numerous branches and data centers has become more urgent. However, the increasing complexity and dynamism of the network environment has brought many challenges to data transmission. Frequent changes in network topology such as equipment failure, link interruption or traffic peaks often lead to a significant decrease in communication efficiency and a significant increase in data transmission delay.

[0003] The existing technology cannot quickly adapt to changes in the network environment, which leads to network congestion, data loss and other problems. In addition, traditional routing optimization methods often rely on manual intervention, lack real-time and intelligence, and are difficult to meet the needs of dynamic environments. This not only increases the workload of network administrators, but also makes it difficult for network optimization to achieve the desired effect. Summary of the invention

[0004] The present application provides a routing optimization method, a control device and a medium to solve the problems in the prior art of relying on static configuration and manual intervention, lacking real-time performance and intelligence, and being unable to cope with network topology changes and dynamic environment requirements.

[0005] The first aspect of the present application provides a routing optimization method, comprising the following steps: obtaining real-time data of the current network status; establishing a simulation environment based on the real-time data, simulating multiple routing strategies in the simulation environment, and using a reinforcement learning algorithm to determine the effects of the multiple routing strategies, wherein the reinforcement learning algorithm used includes constructing a network structure, optimizing parameters, and establishing a reward function; based on the determined effects of the multiple routing strategies, optimizing the current network routing table and updating the routing information.

[0006] Optionally, the real-time data is at least one of traffic load, link status, delay and packet loss rate.

[0007] Optionally, the multiple routing strategies include a shortest path strategy, a load balancing strategy, etc.

[0008] Optionally, the effects of the multiple routing strategies are determined using a reinforcement learning algorithm, including: using the reinforcement learning algorithm and based on a reward function, calculating the cumulative rewards of the multiple routing strategies; comparing the cumulative rewards of the multiple routing strategies, and selecting a target routing strategy.

[0009] Optionally, the target routing strategy includes target path selection, backup paths, and corresponding routing priorities.

[0010] Optionally, the reward function includes: ,in, is the reward function, is the weight of the delay time, is the throughput weight, is the weight of the congestion degree.

[0011] Optionally, based on the effects of the determined multiple routing strategies, the current network routing table is optimized and the routing information is updated, including: obtaining a target routing strategy and a routing table, wherein the routing table includes at least one entry, and the entry includes a destination address, a next hop address, interface information, and routing priority, etc.; based on the target routing strategy and the routing table, entries that need to be updated are identified, and the routing information is updated simultaneously.

[0012] Optionally, the updating of routing information includes adding target entries, modifying existing entries and deleting redundant entries. After updating the routing information according to the recommendation of the decision engine, it includes: notifying other nodes of the updated routing information, and monitoring the updated routing information and converging the routing status.

[0013] A second aspect of the present application provides a control device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any one of the routing optimization methods described.

[0014] The third aspect of the present application provides a machine-readable storage medium, on which instructions are stored, and the instructions enable a machine to execute any one of the routing optimization methods described.

[0015] Therefore, the present application includes the following beneficial effects: the embodiment of the present application obtains real-time data of the current network status, establishes a simulation environment based on these data and simulates multiple routing strategies, and uses a reinforcement learning algorithm to evaluate the effect of each strategy to ensure the scientificity and accuracy of the evaluation. According to the evaluation results, the routing table is optimized and the routing information is updated, so as to realize the automation and intelligence of the routing configuration, significantly improve the network communication efficiency, reduce delays and congestion, enhance the stability and reliability of the network, and reduce the need for manual intervention, so as to effectively respond to changes in network topology and dynamic environmental requirements. Thus, the technical problems of relying on static configuration and manual intervention, lacking real-time and intelligence, and being unable to respond to changes in network topology and dynamic environmental requirements in the prior art are solved.

[0016] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which: Figure 1 A flowchart of a route optimization method provided according to an embodiment of the present application; Figure 2 This is an example diagram of a route optimization device according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0019] The following describes the routing optimization method, control device and medium of the embodiment of the present application with reference to the accompanying drawings. In view of the problem of reliance on static configuration and manual intervention mentioned in the above background technology, the present application provides a routing optimization method, in which real-time data of the current network status is obtained, a simulation environment is established based on these data and multiple routing strategies are simulated, and the effect of each strategy is evaluated using a reinforcement learning algorithm to ensure the scientificity and accuracy of the evaluation. According to the evaluation results, the routing table is optimized and the routing information is updated, the automation and intelligence of the routing configuration are realized, the network communication efficiency is significantly improved, the delay and congestion are reduced, the stability and reliability of the network are enhanced, and the need for manual intervention is reduced, thereby effectively responding to changes in network topology and dynamic environment requirements. As a result, the problems of reliance on static configuration and manual intervention, lack of real-time and intelligence, and inability to respond to changes in network topology and dynamic environment requirements in the prior art are solved.

[0020] Figure 1 A schematic diagram of a route optimization method provided in an embodiment of the present application. Figure 1 As shown, the route optimization method includes the following steps: In step S101, real-time data of the current network status is obtained.

[0021] The real-time data may be at least one of traffic load, link status, delay and packet loss rate.

[0022] It can be understood that the embodiments of the present application achieve comprehensive monitoring and dynamic analysis of the network status by acquiring real-time data of the current network status, wherein the real-time data includes key indicators such as traffic load, link status, delay and packet loss rate, which are used to monitor network performance in real time, promptly discover and respond to topology changes in the network, such as link interruption or equipment failure, and provide an accurate basis for subsequent routing policy adjustments.

[0023] In the embodiment of the present application, a sensor is first installed on each network node and configured to collect data using a network protocol. A central server is set up to receive and store this data. By running the data analysis algorithm regularly, the system can monitor the network status in real time and issue an alarm in time when a link is interrupted or a device fails. For example, when the packet loss rate of a link exceeds 5%, the system will automatically record the event and display the status change of the link to the administrator through a visual interface, which is convenient for rapid response and processing.

[0024] It should be noted that traffic load can refer to the traffic usage of each node in the network, usually in bits per second. By monitoring traffic load, it is possible to identify which nodes or links are in a high load state, which may cause congestion.

[0025] The preferred link status of the embodiment of the present application may include the availability and health of the link. The monitoring system will regularly check whether the link is working properly and whether there is a fault or interruption. The link status can be evaluated by sending a detection packet and analyzing its return time.

[0026] In the embodiments of the present application, latency may refer to the time required for a data packet to reach a destination node from a source node, usually in milliseconds. Latency monitoring can help identify bottlenecks in the network, especially during high traffic periods; packet loss rate may refer to the proportion of data packets lost during data transmission, usually expressed as a percentage. A high packet loss rate may indicate a problem in the network, such as a link failure or overload.

[0027] You can monitor the network status in real time by the following methods: Data collection, using network protocols to collect real-time data from various nodes and send status reports regularly through sensors, or send alerts immediately when anomalies are detected; Data storage and processing, the collected data will be stored in a centralized database for subsequent analysis. Time series databases can be used to efficiently store and query time series data; Data processing, which processes the collected data through data analysis algorithms to identify trends and anomalies. For example, thresholds can be set so that when the traffic load exceeds a certain level, the system automatically triggers an alarm; Data presentation uses a graphical user interface to display network status and updates visual charts of various indicators in real time to help network administrators quickly identify problems.

[0028] In step S102, a simulation environment is established according to real-time data, multiple routing strategies are simulated in the simulation environment, and the effects of the multiple routing strategies are determined using a reinforcement learning algorithm, wherein the reinforcement learning algorithm used includes constructing a network structure, optimizing parameters, and establishing a reward function.

[0029] The various routing strategies preferred in the embodiments of the present application may include the shortest path strategy, load balancing strategy, etc.

[0030] Among them, the shortest path strategy can be a routing strategy that selects the best path based on the path distance, and the load balancing strategy can be a strategy that reasonably distributes the workload to multiple servers or computing resources in network communication, cloud computing, data centers, etc., to improve system performance, reliability and scalability. Among them, load balancing can dynamically adjust the route selection according to factors such as traffic size and link utilization.

[0031] It is understandable that the embodiments of the present application use real-time data to build an accurate simulation environment, simulate the key elements of the real network environment, and provide a solid foundation for routing strategy evaluation. By supporting the simulation and comparison of multiple routing strategies and optimizing the reinforcement learning algorithm, the routing efficiency is improved, the network stability is enhanced, the operating cost is reduced, and the network is promoted to develop in the direction of intelligence.

[0032] For example, select NS3 as the network simulation tool and set a network containing, for example, 100 nodes, with each node having a bandwidth of, for example, 10 Gbps and a latency of 5 ms. Use the NS3 API to create network nodes and configure a unique identifier for each node. Configure the links between the nodes to ensure that the bandwidth and latency of each link meet the set parameters.

[0033] Implement dynamic topology changes in the simulation environment, such as randomly disconnecting certain links during the simulation peak period to observe changes in network performance. By sending test data packets and recording the delay and packet loss rate of each node, ensure that the simulation environment can accurately reflect the behavior of the real network.

[0034] As described above, a scalable simulation environment can be built to provide a solid foundation for subsequent routing strategy evaluation and optimization.

[0035] It should be noted that the network structure may include an input layer, a hidden layer, and an output layer, and the parameters include a learning rate, a discount factor, and the size of the experience replay. Among them, the learning rate can control the degree of influence of new information on old information; the discount factor can be used to balance the weight of current rewards and future rewards.

[0036] In the embodiment of the present application, the learning rate determines the speed of model update, and the discount factor affects the weight of future rewards, wherein a higher learning rate can speed up the learning speed but may cause instability; a lower learning rate may cause the learning speed to be too slow; a higher discount factor makes the algorithm pay more attention to long-term returns, while a lower discount factor makes it pay more attention to short-term returns.

[0037] For example, choose a suitable reinforcement learning algorithm, such as DQN (Deep Q-Learning Network); DQN combines the advantages of Q-learning and deep learning and can handle high-dimensional state space.

[0038] Set the network structure of DQN, including the input layer, hidden layer and output layer; optimize the hyperparameters of DQN, such as the learning rate, discount factor and the size of experience replay, among which the learning rate determines the speed of model update and the discount factor affects the weight of future rewards.

[0039] In an embodiment of the present application, a reinforcement learning algorithm is used to determine the effects of multiple routing strategies, including: using the reinforcement learning algorithm and based on the reward function, calculating the cumulative rewards of multiple routing strategies; comparing the cumulative rewards of multiple routing strategies, and selecting a target routing strategy.

[0040] The target routing strategy includes target path selection, backup paths, and corresponding routing priorities.

[0041] It can be understood that the embodiment of the present application uses a reinforcement learning algorithm and, based on a carefully designed reward function, accurately calculates and compares the cumulative rewards of multiple routing strategies, thereby intelligently selecting the optimal target routing strategy. This not only significantly improves network transmission efficiency and stability, but also effectively reduces operating costs, providing an efficient and flexible routing optimization solution for complex and changeable network environments.

[0042] In this embodiment of the present application, the reward function includes: ,in, is the reward function, is the weight of the delay time, is the throughput weight, is the weight of the congestion degree.

[0043] For example, design a reward function to reflect the effectiveness of the routing strategy; the reward function can be designed based on latency, throughput, and congestion: ,in, is the current state, For the action to be taken.

[0044] For example, to set the weight of the delay , the throughput weight , the congestion weight After each routing strategy evaluation, the delay, throughput, and congestion values ​​of the current strategy are calculated and substituted into the reward function to obtain the reward value of the current strategy; DQN is selected as the reinforcement learning algorithm, and the initial learning rate is set , discount factor During the training process, with each update of the strategy, the reward value is recorded and the strategy is adjusted. Through multiple iterations, the changing trend of the reward value is observed, and the learning rate and discount factor are gradually optimized to ensure that the algorithm can effectively learn the optimal routing strategy; thus, an effective reward mechanism can be obtained, and the parameters of the reinforcement learning algorithm can be optimized, thereby improving the performance of the dynamic routing optimization system.

[0045] It should be noted that latency can be the time required for a data packet to reach the destination node from the source node in the network. The lower the latency, the higher the reward; throughput can be the amount of data successfully transmitted per unit time. The higher the throughput, the higher the reward. Congestion can be the density of data traffic in the network. The lower the congestion, the higher the reward.

[0046] In step S103, according to the determined effects of the multiple routing strategies, the current network routing table is optimized and the routing information is updated.

[0047] The various routing strategies preferred in this application may include shortest path strategy, load balancing strategy, etc.

[0048] It is understandable that the embodiment of the present application can select the routing strategy that best suits the current network conditions by comprehensively evaluating the effects of multiple routing strategies, thereby significantly improving the adaptability and flexibility of the network. At the same time, based on the evaluation results of real-time data and simulation environment, the bottlenecks and potential problems in the network are accurately identified and solved to ensure the stable improvement of network performance. In addition, by optimizing the routing table, network resources can be used more effectively, unnecessary transmission delays and bandwidth waste can be reduced, network operating costs can be further reduced, and resource utilization efficiency can be improved.

[0049] For example, a network with 50 nodes is built in a simulation environment, and the bandwidth of each node is set to 1Gbps and the delay is 10ms. DQN is selected as the reinforcement learning algorithm, the learning rate is set to 0.001, the discount factor is set to 0.99, and the reward function is designed to reflect the delay and packet loss rate.

[0050] In the simulation environment, different routing strategies are run, such as the shortest path strategy and the load balancing strategy, and the average delay and packet loss rate of each strategy are recorded. The effects of these strategies are evaluated through the DQN model, the cumulative reward of each strategy is calculated, and their performance is compared.

[0051] As a result, the best performing strategy can be selected, and the routing configuration can be updated in real time when the network topology changes to ensure efficient data transmission. For example, when the average latency of a strategy is less than 50 ms and the packet loss rate is less than 2%, the system will automatically switch to that strategy to optimize network performance.

[0052] It should be noted that based on the evaluation results, the routing strategy with the best performance can be selected. A threshold can be set. When the performance index of a strategy exceeds the threshold, the strategy is considered to be the best strategy. When a network topology change is detected, the best strategy is quickly switched. By dynamically adjusting the routing table, the efficiency of data transmission is ensured.

[0053] In an embodiment of the present application, based on the effects of the determined multiple routing strategies, the current network routing table is optimized and the routing information is updated, including: obtaining a target routing strategy and a routing table, wherein the routing table includes at least one entry; identifying entries that need to be updated based on the target routing strategy and the routing table, and updating the routing information at the same time.

[0054] The routing table may consist of multiple entries, each of which contains the destination address, next hop address, interface information, and routing priority.

[0055] It can be understood that the embodiment of the present application accurately identifies and updates routing information through the target routing strategy and routing table output by the intelligent decision-making engine, significantly improves network performance and enhances stability, while improving resource utilization efficiency and simplifying network management.

[0056] It should be noted that the intelligent decision engine can be a list containing multiple paths and their performance indicators, such as latency, bandwidth, and packet loss rate. In addition, it is necessary to identify whether the destination address matches the path output by the decision engine; whether the priority of the current path is lower than the priority of the new policy.

[0057] In an embodiment of the present application, updating routing information includes adding target entries, modifying existing entries, and deleting redundant entries. After updating the routing information according to the recommendation of the decision engine, it includes: notifying other nodes of the updated routing information, and monitoring the updated routing information and converging the routing status.

[0058] It is understandable that the embodiments of the present application identify and add new valid paths as routing entries, update the next hop addresses and priorities of existing entries to reflect the latest network policies, and delete redundant entries that are no longer valid to avoid routing conflicts and unnecessary delays. In addition, the updated routing information can be promptly notified to other nodes in the network through the routing protocol to ensure that the entire network can obtain the latest and accurate routing information. At the same time, the response of other nodes in the network to the new routing information is monitored to ensure that the network can quickly converge to the new routing state.

[0059] For example, the current optimal routing policy is first obtained from the decision engine. Assume that the path recommended by the policy is "192.168.1.1 -> 192.168.1.2 -> 192.168.1.3". The system parses the current routing table to check whether there is an entry matching "192.168.1.1".

[0060] If the entry "192.168.1.1 -> 192.168.1.4" exists in the current routing table and its priority is lower than the new policy, the system will update the entry to "192.168.1.1 -> 192.168.1.2" and increase the priority. At the same time, the system will mark the "192.168.1.4" entry as redundant and delete it.

[0061] The updated routing information is notified to other nodes in the network through the OSPF protocol to ensure that all nodes can obtain the latest routing information in a timely manner.

[0062] According to a routing optimization method proposed in an embodiment of the present application, by acquiring real-time data of the current network status, a simulation environment is established based on these data and multiple routing strategies are simulated, and the effect of each strategy is evaluated using a reinforcement learning algorithm to ensure the scientificity and accuracy of the evaluation. According to the evaluation results, the routing table is optimized and the routing information is updated, which realizes the automation and intelligence of routing configuration, significantly improves the efficiency of network communication, reduces latency and congestion, enhances the stability and reliability of the network, and reduces the need for manual intervention, thereby effectively responding to changes in network topology and dynamic environmental requirements. As a result, the problems of relying on static configuration and manual intervention, lack of real-time and intelligence, and inability to respond to changes in network topology and dynamic environmental requirements in the prior art are solved.

[0063] The route optimization method is described below through an embodiment, and the description content is as follows: S1, monitor network status in real time, detect topology changes such as link interruption or device failure, and collect data such as traffic load, link status, latency, and packet loss rate; S2, simulate multiple routing strategies in a simulation environment, use reinforcement learning algorithms to evaluate strategy effects, and select the optimal strategy; Among them, the basic parameters of the network are clarified in the simulation environment, including network scale, number of nodes, bandwidth and delay. For example, a network with 100 nodes can be set, with each node having a bandwidth of 10Gbps and a delay of 5ms. Select a suitable network topology, such as star, ring, tree or mesh structure, to simulate different network scenarios. Different topologies will have a significant impact on the efficiency and reliability of data transmission; Choosing a suitable network simulation tool, such as NS3 or OMNeT++, can provide flexible network models and rich functions to support the simulation of dynamic topology changes.

[0064] In the simulation tool, build a simulation environment according to the set parameters. The specific steps include: Create network nodes: Define 100 nodes in the simulation tool and assign a unique identifier to each node. Configure links: Configure bandwidth and latency for connections between nodes. For example, you can set the link bandwidth between node A and node B to 10Gbps and the latency to 5ms. Implement dynamic topology: Simulate dynamic changes in links by writing scripts or using the API provided by the tool, such as disconnecting certain links or adding new nodes under certain conditions.

[0065] After the simulation environment is set up, perform preliminary verification tests to ensure that the basic functions of the network are normal. You can send test packets to check the transmission delay and packet loss rate of the packets to ensure that these indicators are within the expected range. Record the preliminary test results and adjust the network parameters as needed to optimize the performance of the simulation environment.

[0066] Choose a suitable reinforcement learning algorithm, such as DQN (Deep Q-Learning Network); DQN combines the advantages of Q-learning and deep learning and can handle high-dimensional state space.

[0067] Set the network structure of DQN, including the input layer, hidden layer and output layer; optimize the hyperparameters of DQN, such as the learning rate, discount factor and the size of experience replay, among which the learning rate determines the speed of model update and the discount factor affects the weight of future rewards.

[0068] Design a reward function to reflect the effectiveness of the routing strategy; the reward function can be designed based on delay, throughput, and congestion level, as shown in formula (1).

[0069] The parameters of the reinforcement learning algorithm are optimized to reflect the effect of the routing strategy, including indicators such as delay, throughput and congestion. The reward function is shown in formula (2).

[0070] Select a suitable reinforcement learning algorithm and optimize its parameters. Adjust the learning rate and discount factor through experiments, observe the impact of different parameter combinations on the learning effect, and select the best combination.

[0071] S3. Automatically adjust the routing table based on the recommendations of the decision engine, implement the selected optimal strategy, and dynamically update the routing information.

[0072] In summary, the embodiments of the present application can significantly improve the communication efficiency of the network, reduce latency and congestion, and ensure the reliability and stability of data transmission by monitoring the network status in real time, automatically detecting topology changes, and using advanced reinforcement learning algorithms to simulate and evaluate multiple routing strategies, thereby selecting the optimal routing solution. By automatically updating the routing configuration, the system can quickly respond to network changes, reduce manual intervention, and improve the intelligent level of network management, which has important practical application value.

[0073] Next, the routing optimization device proposed according to the embodiment of the present application is described with reference to the accompanying drawings.

[0074] Figure 2 It is a block diagram of a route optimization device according to an embodiment of the present application.

[0075] like Figure 2 As shown, the route optimization device 10 includes: an acquisition module 100 , an evaluation module 200 and an optimization module 300 .

[0076] Among them, the acquisition module 100 is used to obtain real-time data of the current network status; the evaluation module 200 is used to establish a simulation environment based on real-time data, simulate multiple routing strategies in the simulation environment, and use a reinforcement learning algorithm to evaluate the effects of multiple routing strategies, wherein the use of the reinforcement learning algorithm includes constructing a network structure, optimizing parameters and establishing a reward function; the optimization module 300 is used to optimize the routing table and update the routing information based on the evaluation of the effects of multiple routing strategies.

[0077] It should be noted that the above explanation of the embodiment of the route optimization method is also applicable to the route optimization device of this embodiment, and will not be repeated here.

[0078] According to a routing optimization device proposed in an embodiment of the present application, by acquiring real-time data of the current network status, a simulation environment is established based on these data and multiple routing strategies are simulated, and the effect of each strategy is evaluated using a reinforcement learning algorithm to ensure the scientificity and accuracy of the evaluation. According to the evaluation results, the routing table is optimized and the routing information is updated, which realizes the automation and intelligence of routing configuration, significantly improves the efficiency of network communication, reduces latency and congestion, enhances the stability and reliability of the network, and reduces the need for manual intervention, thereby effectively responding to changes in network topology and dynamic environmental requirements. As a result, the problems of relying on static configuration and manual intervention, lack of real-time and intelligence, and inability to respond to changes in network topology and dynamic environmental requirements in the prior art are solved.

[0079] Based on the same general inventive concept, the present invention also protects a control device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement any one of the routing optimization methods in the above-mentioned embodiments.

[0080] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0081] On the other hand, the present invention also protects a machine-readable storage medium, on which instructions are stored, and the instructions enable a machine to execute any one of the routing optimization methods in the above embodiments.

[0082] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0083] In addition, the terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0084] It should be understood that in the embodiment of the present invention, "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0085] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0086] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0087] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0088] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0089] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0090] Through the description of the above embodiments, it can be clearly understood by those skilled in the art that the present invention can be implemented in hardware, firmware, or a combination thereof. When implemented using software, the above functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication media include any medium that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a computer. Taking this as an example but not limited to: a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer. In addition. Any connection can be appropriately a computer-readable medium. For example, if the software is transmitted from a website, server or other remote source using a coaxial cable, optical fiber cable, twisted pair, digital subscriber line (DSL) or wireless technologies such as infrared, radio and microwave, then the coaxial cable, optical fiber cable, twisted pair, DSL or wireless technologies such as infrared, wireless and microwave are included in the fixing of the medium. As used in the present invention, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and blue-ray disc, wherein disk usually reproduces data magnetically, while disc uses laser to reproduce data optically. The above combination should also be included in the scope of protection of computer-readable media.

[0091] In short, the above is only a preferred embodiment of the technical solution of the present invention, and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A routing optimization method, characterized in that: The following steps are involved: Get real-time data of current network status; Establishing a simulation environment according to the real-time data, simulating multiple routing strategies in the simulation environment, and determining the effects of the multiple routing strategies using a reinforcement learning algorithm, wherein the reinforcement learning algorithm used includes building a network structure, optimizing parameters, and establishing a reward function; According to the determined effects of the multiple routing strategies, the current network routing table is optimized and routing information is updated.

2. The routing optimization method according to claim 1, characterized in that: The real-time data is at least one of traffic load, link status, delay and packet loss rate.

3. The routing optimization method according to claim 1, characterized in that: The multiple routing strategies include the shortest path strategy and the load balancing strategy.

4. The routing optimization method according to claim 1, characterized in that: The effects of the multiple routing strategies are determined using a reinforcement learning algorithm, including: Calculating cumulative rewards of the plurality of routing strategies based on a reward function using a reinforcement learning algorithm; The cumulative rewards of the multiple routing strategies are compared, and a target routing strategy is selected.

5. The routing optimization method according to claim 4, characterized in that: The target routing strategy includes target path selection, backup paths and corresponding routing priorities.

6. The route optimization method according to claim 1 or 4, characterized in that: The reward function includes: in, is the reward function, is the weight of the delay time, is the throughput weight, is the weight of the congestion degree.

7. The route optimization method according to claim 1, characterized in that: According to the determined effects of the multiple routing strategies, the current network routing table is optimized and routing information is updated, including: Obtaining a target routing strategy and a routing table, wherein the routing table includes at least one entry, and the entry includes a destination address, a next hop address, interface information, and a routing priority, etc.; According to the target routing policy and the routing table, entries that need to be updated are identified, and routing information is updated simultaneously.

8. The route optimization method according to claim 7, characterized in that: The updating of routing information includes adding a target entry, modifying an existing entry, and deleting a redundant entry. After the routing information is updated according to the suggestion of the decision engine, the updating includes: The updated routing information is notified to other nodes, and the updated routing information is monitored and the routing status is converged.

9. A control device, characterized in that: The control device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the route optimization method according to any one of claims 1-8.

10. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores instructions, which enable the machine to execute the route optimization method according to any one of claims 1-8.

Images