Intelligent routing control method and system based on improved deep reinforcement learning model
Through the improved deep reinforcement learning model, the network performance score is calculated and routing strategies are determined based on the optimal performance of traditional intelligent routing control technology in dynamic network environments, and more efficient, stable and reliable data transmission is achieved.
Patent Information
- Application Number
- CN202410975366.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-26
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional intelligent routing control technology lacks the ability to adapt to dynamic changes in network state, is difficult to deal with complexity and uncertainty in large-scale network environments, and is unable to extract patterns or knowledge from historical data to optimize routing, resulting in dynamic changes in modern times. poor results in network environment.
Using an improved deep reinforcement learning model, the network performance score is calculated by obtaining key network metrics of the data source of the smart router, such as data traffic, delay time and packet loss rate, and determining the data source of the next data detection cycle based on the score value. and control strategies to enhance the network's adaptability and intelligent level of routing decisions.
It realizes more efficient, stable and reliable data transmission in complex and changeable network environments, and improves network performance optimization and response efficiency.
Smart Images

Figure CN119030910A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent routing control technology, and in particular to an intelligent routing control method and system based on an improved deep reinforcement learning model. Background Art
[0002] Intelligent routing control technologies usually rely on traditional algorithms and heuristic methods to achieve packet path selection and network traffic management. These technologies include routing protocols based on shortest path algorithms, such as Dijkstra and Bellman-Ford algorithms, and strategies based on load balancing and congestion avoidance, such as polling and random routing. Although the above methods are effective in some cases, they often lack the ability to adapt to dynamic changes in network status, have difficulty handling the complexity and uncertainty in large-scale network environments, and may not be able to make optimal decisions when faced with problems such as network congestion and high latency. In addition, traditional methods usually do not have the ability to learn and cannot extract patterns or knowledge from historical data to optimize routing selection. Therefore, in modern dynamic and ever-changing network environments, the limitations of these technologies are gradually revealed and cannot meet actual needs. Summary of the invention
[0003] The purpose of the embodiments of the present invention is to provide an intelligent routing control method and system based on an improved deep reinforcement learning model, by obtaining the key network indicators of each data source of the intelligent router, calculating the network performance score value of the corresponding data source, and determining the control strategy of the intelligent router in the next data detection cycle based on the improved deep reinforcement learning model, thereby enhancing the network's adaptive ability, improving the intelligence level of routing decision-making, and continuously optimizing network performance through a learning mechanism, thereby achieving more efficient, more stable and more reliable data transmission in a complex and changeable network environment.
[0004] To solve the above technical problems, a first aspect of an embodiment of the present invention provides an intelligent routing control method based on an improved deep reinforcement learning model, comprising the following steps:
[0005] In the current data detection cycle, receiving port test data from several data sources of the intelligent router, and obtaining data flow, delay time and packet loss rate in the port test data;
[0006] Based on the data traffic, the delay duration and the packet loss rate, a network performance score of each of the data sources is calculated by an improved deep reinforcement learning model;
[0007] According to the network performance score values of the plurality of data sources, the data source and control strategy of the intelligent router in the next data detection cycle are preferentially determined.
[0008] Further, the preferentially determining the data source and control strategy of the intelligent router in the next data detection cycle includes:
[0009] Sorting the network performance scores from large to small;
[0010] The data source with the highest network performance score is selected as the data source for the intelligent router in the next data detection cycle.
[0011] Furthermore, after sorting the network performance score values from large to small, the method further includes:
[0012] When the difference between the network performance score values of the two data sources ranked first is less than a preset threshold, the two data sources are selected as data sources for the intelligent router in the next data detection cycle, and traffic distribution is performed based on a load balancing strategy.
[0013] Furthermore, the calculation formula for the network performance score of the data source is:
[0014] NPS=ω T ×g T (T×α T )-ω L ×g L (L×α L )-ω P ×g P (P×α P );
[0015] Wherein, T is the data flow, ω T is the data traffic weight coefficient, g T is the nonlinear transformation function of data flow, α T is the data flow adjustment factor, L is the delay time, ω L is the delay duration weight coefficient, g L is the nonlinear transformation function of the delay time, α L is the delay adjustment factor, P is the packet loss rate, ω P is the packet loss rate weight coefficient, g P is the nonlinear transformation function of packet loss rate, α P is the adjustment factor for packet loss rate.
[0016] Furthermore, the nonlinear transformation function g of the data flow is T The calculation formula is:
[0017]
[0018] Where k is the nonlinear transformation function g TThe coefficient value that controls the steepness of the curve.
[0019] Furthermore, the nonlinear transformation function g of the delay time is L The calculation formula is:
[0020]
[0021] Among them, γ1, γ2, and γ3 are the weight coefficient values of the corresponding items, L0×a L0 is the nonlinear transformation function g L Controls the center position value of the curve.
[0022] Furthermore, the packet loss rate nonlinear transformation function g P The calculation formula is:
[0023]
[0024] Among them, θ1, θ2, θ3, θ4, θ5, θ6, θ7, and θ8 are weight coefficients of the corresponding items.
[0025] Furthermore, the data sources include: satellite data, 5G mobile data, wired network and / or digital radio data.
[0026] Furthermore, the intelligent router acquires data in accordance with the data source priority in the initial state;
[0027] The priorities of the data sources are from high to low: wired network, satellite data, 5G mobile data, and digital radio data.
[0028] Accordingly, a second aspect of an embodiment of the present invention provides an intelligent routing control system based on an improved deep reinforcement learning model, comprising:
[0029] A data acquisition module, which is used to receive port test data from several data sources of the intelligent router within the current data detection cycle, and obtain data flow, delay time and packet loss rate in the port test data;
[0030] A performance calculation module, which is used to calculate the network performance score of each of the data sources through an improved deep reinforcement learning model based on the data traffic, the delay duration and the packet loss rate;
[0031] A strategy formulation module is used to preferentially determine the data source and control strategy of the intelligent router in the next data detection cycle according to the network performance score values of the plurality of data sources.
[0032] Correspondingly, the third aspect of an embodiment of the present invention also provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the above-mentioned intelligent routing control method based on the improved deep reinforcement learning model.
[0033] In addition, the fourth aspect of an embodiment of the present invention also provides a computer-readable storage medium on which computer instructions are stored, which, when executed by a processor, implement the above-mentioned intelligent routing control method based on the improved deep reinforcement learning model.
[0034] The above technical solution of the embodiment of the present invention has the following beneficial technical effects:
[0035] By obtaining the key network indicators of each data source of the smart router, calculating the network performance score of the corresponding data source, and determining the control strategy of the smart router in the next data detection cycle based on the improved deep reinforcement learning model, the network's adaptability is enhanced, the intelligence level of routing decisions is improved, and the network performance is continuously optimized through the learning mechanism, thereby achieving more efficient, stable and reliable data transmission in a complex and changeable network environment. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 It is a flow chart of an intelligent routing control method based on an improved deep reinforcement learning model provided by an embodiment of the present invention;
[0037] Figure 2 It is a schematic diagram of the principle of an improved deep reinforcement learning model provided by an embodiment of the present invention;
[0038] Figure 3 It is a block diagram of an intelligent routing control system module based on an improved deep reinforcement learning model provided in an embodiment of the present invention.
[0039] Reference numerals:
[0040] 1. Data acquisition module, 2. Performance calculation module, 3. Strategy formulation module. DETAILED DESCRIPTION
[0041] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. In addition, in the following description, the description of well-known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention.
[0042] Please refer to Figure 1A first aspect of an embodiment of the present invention provides an intelligent routing control method based on an improved deep reinforcement learning model, comprising the following steps:
[0043] Step S100, receiving port test data from several data sources of the intelligent router within the current data detection cycle, and obtaining data flow, delay time and packet loss rate in the port test data.
[0044] During the current data detection cycle, the key task performed by the smart router is to receive port test data from different data sources. This step involves real-time monitoring and analysis of data packets passing through each port of the router to collect key indicators of network performance. Specifically, the smart router needs to obtain data throughput, latency, and packet loss rate in the port test data. Data throughput reflects the amount of data passing through the port within a certain period of time, which helps to identify hot spots of network congestion; latency measures the time required for data packets to travel from the source to the destination, which is particularly important for applications with high real-time requirements; and packet loss rate indicates the proportion of data packets lost during transmission. Monitoring of packet loss rate can reveal potential connection problems or hardware failures in the data transmission network. Through real-time monitoring of these indicators, the smart router can provide accurate data support for subsequent network performance scoring and routing decisions, ensuring that network traffic is routed in the best way, thereby improving overall data transmission network efficiency and user satisfaction.
[0045] Step S200, based on data traffic, delay time and packet loss rate, the network performance score of each data source is calculated through an improved deep reinforcement learning model.
[0046] The improved deep reinforcement learning model can dynamically learn historical data, learn to consider multiple factors (such as data traffic, delay time and packet loss rate) simultaneously in different environments, obtain the rules of how the above factors jointly affect network performance, and after training with historical data, predict the network performance scores of various data sources in different environments, and then determine the optimal data source and control strategy, helping smart routers to adjust data sources in response to environmental changes in a timely and rapid manner.
[0047] Step S300, based on the network performance score values of several data sources, the control strategy of the intelligent router in the next data detection cycle is determined optimally.
[0048] Based on the network performance score values of several data sources from the smart router, the above score values reflect the key performance indicators such as data flow, delay time and packet loss rate of each data source in the current detection cycle. The data source is selected to achieve efficient data packet routing, minimize transmission delay, reduce packet loss rate, and maximize network throughput, thereby ensuring the optimal allocation of network resources and the overall improvement of network performance.
[0049] The step S300 of preferentially determining the data source and control strategy of the intelligent router in the next data detection cycle includes: sorting the network performance score values from large to small; selecting the data source with the highest network performance score as the data source of the intelligent router in the next data detection cycle.
[0050] Furthermore, when the difference in network performance scores between the two data sources ranked first is less than a preset threshold, the two data sources are selected as data sources for the intelligent router in the next data detection cycle, and traffic distribution is performed based on a load balancing strategy.
[0051] Based on the data source selection method and traffic distribution control strategy of the above intelligent router,
[0052] The above method not only improves the adaptability and intelligence level of the network, but also enables the intelligent router to respond more accurately to dynamic changes in the network environment through continuous learning and optimization.
[0053] For details, please refer to Figure 2 In an optional implementation of an embodiment of the present invention, the improved deep reinforcement learning model includes: an actor network, a first critic network and a second critic network. The actor network performs policy function fitting based on a policy gradient method according to the data flow, delay duration and packet loss rate of a data source, and selects the optimal action; the first critic network and the second critic network respectively evaluate the optimal action to obtain a first target Q value and a second target Q value, and select the smaller value of the first target Q value and the second target Q value to update the action-value function of the improved deep reinforcement learning model.
[0054] The above improved deep reinforcement learning model adopts the method of combining the actor network with two critic networks, that is, setting up the first critic network and the second critic network to realize the dynamic optimization of the intelligent routing control strategy. In this model, the actor network adopts the policy gradient method to fit the policy function according to the key performance indicators such as the traffic, delay and packet loss rate of each data source, and selects the optimal action to make routing decisions. At the same time, the two critic networks independently evaluate the optimal action selected by the actor network, and generate the first target Q value and the second target Q value respectively. By comparing the two Q values, the model selects the smaller one as the update basis of the action-value function (i.e., Q value function), so as to guide the policy update of the actor network and ensure that the intelligent router makes more accurate and efficient routing selection in the ever-changing network environment. Through this method of integrating multiple evaluation perspectives and continuous optimization, the intelligent router can realize intelligent decision-making on the control strategy of the next data detection cycle, further improving network performance and response efficiency.
[0055] Furthermore, the actor network update frequency is lower than the update frequency of the first critic network and the second critic network.
[0056] In this improved deep reinforcement learning model, the actor network's update strategy adopts an improved update frequency control method, that is, its update frequency is lower than that of the first critic network and the second critic network; the above control method slows down the speed of change of the actor network parameters in order to maintain the stability and reliability of the strategy in a rapidly changing environment. In addition, since the critic network is responsible for evaluating actions and providing Q values, they can be updated more frequently to quickly adapt to environmental changes, while the actor network relies on these evaluation results to adjust its strategy at a slower rate, thereby reducing the instability that may be introduced by too frequent updates. This balanced update mechanism helps to achieve a better balance between developing new control strategies and using known effective strategies, ensuring that the control strategy of the smart router can maintain long-term stability and robustness while pursuing performance optimization.
[0057] In a specific implementation of the embodiment of the present invention, the calculation formula for the network performance score of the data source is:
[0058] NPS=ω T ×g T (T×α T )-ω L ×g L (L×α L )-ω P ×g P (P×α P );
[0059] Where T is the data flow, ω T is the data traffic weight coefficient, g T is the nonlinear transformation function of data flow, α T is the data flow adjustment factor, L is the delay time, ω L is the delay duration weight coefficient, g L is the nonlinear transformation function of the delay time, α L is the delay adjustment factor, P is the packet loss rate, ω P is the packet loss rate weight coefficient, g P is the nonlinear transformation function of packet loss rate, α P is the adjustment factor for packet loss rate.
[0060] The above calculation formula takes into account the three key indicators of data flow (Throughput, T), delay time (Latency, L) and packet loss rate (Packet Loss Rate, P), and assigns weight coefficients and adjustment factors to each indicator, as well as corresponding non-linear transformation functions. Combined with the weighting and adjustment of the three key network performance indicators, the above formula can calculate the comprehensive score of the network performance of a data source of the intelligent router, which can reflect the overall performance of the network transmission quality of a data source. The weight coefficient determines the relative importance of each indicator in the final score, while the non-linear transformation function allows for more detailed regulation of the impact of each indicator. The adjustment factor is used to further fine-tune the score contribution of each indicator to adapt to specific network environments or business needs.
[0061] The improved deep reinforcement learning model can dynamically learn historical data, learn to consider multiple factors (such as data flow, delay time and packet loss rate) in different environments, and obtain the rules of the above factors affecting network performance together. After training with historical data, the improved deep reinforcement learning model can optimize multiple calculation factors in the network performance score of the data source, such as: data flow weight coefficient, data flow adjustment factor, delay time weight coefficient, delay time adjustment factor, packet loss rate weight coefficient and packet loss rate adjustment factor. After the above optimization process, the accuracy and robustness of the scoring model are improved to ensure that the score more truly reflects the actual performance; the adaptability of the model to different network environments and usage scenarios is enhanced, so that it can make more reasonable scoring and routing decisions under dynamically changing network conditions; the user experience is improved, and data transmission is guided by more accurate network performance evaluation, reducing delays and packet loss, thereby providing smoother and more stable network services. In addition, the self-learning and adaptive capabilities of the model enable it to continue to improve, and as time goes by and data accumulates, the scoring strategy is continuously optimized to maintain the long-term effectiveness and advancement of the model.
[0062] The weight coefficients, adjustment factors and nonlinear transformation functions in the above formulas need to be adjusted and optimized according to actual business requirements and network environment.
[0063] Specifically, the nonlinear transformation function g of data flow is T The calculation formula is:
[0064]
[0065] Where k is the nonlinear transformation function g T The coefficient value that controls the steepness of the curve.
[0066] Using the S-shaped transformation (Logistic function) as a nonlinear transformation function for data traffic has many advantages in network performance scoring. First, the Logistic function can compress the original value of the data traffic into the interval (0,1), which allows data of different dimensions or orders of magnitude to be reasonably compared and weighted. Secondly, the S-shaped transformation has an asymptotic characteristic. As the data traffic increases, its contribution to the score gradually tends to saturation, which helps to avoid excessive influence of extreme values on the overall score. In addition, the smoothness of the Logistic function can reduce the sharp fluctuations in the score and provide more stable and continuous score changes. Finally, because the Logistic function is easy to differentiate, it allows more efficient parameter updates and policy learning in optimization algorithms that require gradient descent, such as deep reinforcement learning.
[0067] Specifically, the nonlinear transformation function g of the delay time is L The calculation formula is:
[0068]
[0069] Among them, γ1, γ2, and γ3 are the weight coefficient values of the corresponding items, L0×a L0 is the nonlinear transformation function g L Controls the center position value of the curve.
[0070] The nonlinear transformation function of the delay time sets the logarithmic transformation term, square term, sigmoid function term, cubic term and weight coefficient. The composite transformation can more carefully and flexibly reflect the impact of delay on network performance. The logarithmic transformation term helps to reduce the impact of extreme delay values, making the score more sensitive to small delay changes; the square term increases the penalty for larger delay values because the square operation amplifies the effect of the delay value; the sigmoid function term provides a smooth transition interval to ensure the gradualness and stability of the score when the delay value changes; the cubic term increases the complexity of the transformation, allowing the model to capture more subtle delay changes. The weight coefficient allows the importance of different delay characteristics to be adjusted according to actual business needs. Overall, this comprehensive nonlinear transformation function provides a highly customizable evaluation tool for network performance scoring that can accurately reflect the impact of delay, helping smart routers to make more accurate and effective routing decisions.
[0071] Specifically, the packet loss rate nonlinear transformation function g P The calculation formula is:
[0072]
[0073] Among them, θ1, θ2, θ3, θ4, θ5, θ6, θ7, and θ8 are weight coefficients of the corresponding items.
[0074] Packet loss rate nonlinear transformation function g P The calculation formula adopts a comprehensive calculation method including sine transformation terms, logarithmic transformation terms, logic function terms, and reciprocal transformation terms, which realizes a detailed and dynamic evaluation of the impact of packet loss rate. The sine transformation term introduces periodic changes, which increases the sensitivity of the model to packet loss rate fluctuations; the logarithmic transformation term reduces the negative impact of extremely high packet loss rate values, making the score more balanced; the logic function term provides an S-shaped smooth transition to ensure the stability of the score when the packet loss rate changes; the reciprocal transformation term emphasizes the importance of low packet loss rate because it is directly related to the high reliability of the network. Combining these transformation terms, the packet loss rate nonlinear transformation function can accurately capture the comprehensive impact of packet loss rate on network performance, providing a flexible and responsive scoring mechanism for intelligent routers, which helps to achieve better routing selection and network traffic management, thereby improving the stability and efficiency of overall network transmission.
[0075] Optionally, the data sources of the smart router may include: at least 2 of satellite data, 5G mobile data, wired network and digital radio data. Based on the above control method, the smart router can select the most appropriate routing path from different data sources according to network conditions and user needs to ensure the efficiency and reliability of data transmission. For example, if the 5G network provides the fastest speed and lowest latency, the smart router may give priority to the 5G data source; while in areas where the 5G signal is unstable, satellite data or digital radio data may become a better choice. By comprehensively considering the performance scores of different data sources, the smart router is able to achieve multi-mode access and intelligent switching, thereby providing more stable and efficient data transmission services.
[0076] After the smart router is initially powered on, data is obtained according to the priorities of multiple data sources; optionally, the priorities of the data sources are from high to low: wired network, satellite data, 5G mobile data, and digital radio data.
[0077] Accordingly, a second aspect of an embodiment of the present invention provides an intelligent routing control system based on an improved deep reinforcement learning model, comprising:
[0078] Data acquisition module 1, which is used to receive port test data from several data sources of the intelligent router within the current data detection cycle, and obtain data flow, delay time and packet loss rate in the port test data;
[0079] Performance calculation module 2, which is used to calculate the network performance score of each data source based on data traffic, delay time and packet loss rate through an improved deep reinforcement learning model;
[0080] The strategy formulation module 3 is used to determine the control strategy of the intelligent router in the next data detection cycle based on the network performance score values of several data sources.
[0081] The functional modules in the above-mentioned spatial data analysis and processing system correspond to the steps in the spatial data analysis and processing method. Each functional module can be further defined as each functional unit according to the detailed steps of the analysis and processing method to be consistent with the method, and will not be described in detail here.
[0082] Correspondingly, the third aspect of an embodiment of the present invention also provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the above-mentioned intelligent routing control method based on the improved deep reinforcement learning model.
[0083] In addition, the fourth aspect of an embodiment of the present invention also provides a computer-readable storage medium on which computer instructions are stored, which, when executed by a processor, implement the above-mentioned intelligent routing control method based on the improved deep reinforcement learning model.
[0084] The embodiment of the present invention aims to protect an intelligent routing control method and system based on an improved deep reinforcement learning model, wherein the control method includes the following steps: receiving port test data from several data sources of the intelligent router in the current data detection cycle, obtaining data flow, delay time and packet loss rate in the port test data; calculating the network performance score of each data source through the improved deep reinforcement learning model based on the data flow, delay time and packet loss rate; and determining the data source and control strategy of the intelligent router in the next data detection cycle based on the network performance score values of several data sources. The above technical solution has the following effects:
[0085] By obtaining the key network indicators of each data source of the smart router, calculating the network performance score of the corresponding data source, and determining the control strategy of the smart router in the next data detection cycle based on the improved deep reinforcement learning model, the network's adaptability is enhanced, the intelligence level of routing decisions is improved, and the network performance is continuously optimized through the learning mechanism, thereby achieving more efficient, stable and reliable data transmission in a complex and changeable network environment.
[0086] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0087] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0088] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0089] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.
Claims
1. An intelligent routing control method based on an improved deep reinforcement learning model, characterized in that: The steps include: In the current data detection cycle, receiving port test data from several data sources of the intelligent router, and obtaining data flow, delay time and packet loss rate in the port test data; Based on the data traffic, the delay duration and the packet loss rate, a network performance score of each of the data sources is calculated by an improved deep reinforcement learning model; According to the network performance score values of the plurality of data sources, the data source and control strategy of the intelligent router in the next data detection cycle are preferentially determined.
2. The intelligent routing control method based on the improved deep reinforcement learning model according to claim 1 is characterized in that: The step of preferentially determining the data source and control strategy of the intelligent router in the next data detection cycle includes: Sorting the network performance scores from large to small; The data source with the highest network performance score is selected as the data source for the intelligent router in the next data detection cycle.
3. The intelligent routing control method based on the improved deep reinforcement learning model according to claim 2 is characterized in that: After sorting the network performance score values from large to small, the method further includes: When the difference between the network performance score values of the two data sources ranked first is less than a preset threshold, the two data sources are selected as data sources for the intelligent router in the next data detection cycle, and traffic distribution is performed based on a load balancing strategy.
4. The intelligent routing control method based on the improved deep reinforcement learning model according to claim 1 is characterized in that: The calculation formula for the network performance score of the data source is: NPS=ω T ×g T (T×a T )-oh L ×g L (L×α L )-oh P ×g P (P×α P ); Wherein, T is the data flow, ω T is the data traffic weight coefficient, g T is the nonlinear transformation function of data flow, α T is the data flow adjustment factor, L is the delay time, ω L is the delay duration weight coefficient, g L is the nonlinear transformation function of the delay time, α L is the delay adjustment factor, P is the packet loss rate, ω P is the packet loss rate weight coefficient, g P is the nonlinear transformation function of packet loss rate, α P is the adjustment factor for packet loss rate.
5. The intelligent routing control method based on the improved deep reinforcement learning model according to claim 4 is characterized in that: The nonlinear transformation function g of the data flow T The calculation formula is: Where k is the nonlinear transformation function g T The coefficient value that controls the steepness of the curve.
6. The intelligent routing control method based on the improved deep reinforcement learning model according to claim 4 is characterized in that: The nonlinear transformation function g of the delay time L The calculation formula is: Among them, γ1, γ2, and γ3 are the weight coefficient values of the corresponding items, L0×a L0 is the nonlinear transformation function g L Controls the center position value of the curve.
7. The intelligent routing control method based on the improved deep reinforcement learning model according to claim 4 is characterized in that: The packet loss rate nonlinear transformation function g P The calculation formula is: Among them, θ1, θ2, θ3, θ4, θ5, θ6, θ7, and θ8 are weight coefficients of the corresponding items.
8. The intelligent routing control method based on the improved deep reinforcement learning model according to any one of claims 1 to 7, characterized in that: The data sources include: satellite data, 5G mobile data, cable network and / or digital radio data.
9. The intelligent routing control method based on the improved deep reinforcement learning model according to claim 8, characterized in that: The intelligent router acquires data in accordance with the data source priority in the initial state; The priorities of the data sources are from high to low: wired network, satellite data, 5G mobile data, and digital radio data.
10. An intelligent routing control system based on an improved deep reinforcement learning model, characterized in that: include: A data acquisition module, which is used to receive port test data from several data sources of the intelligent router within the current data detection cycle, and obtain data flow, delay time and packet loss rate in the port test data; A performance calculation module, which is used to calculate the network performance score of each of the data sources through an improved deep reinforcement learning model based on the data traffic, the delay duration and the packet loss rate; A strategy formulation module is used to preferentially determine the data source and control strategy of the intelligent router in the next data detection cycle according to the network performance score values of the plurality of data sources.
Citation Information
Patent Citations
Network autonomous intelligent management and control method based on deep reinforcement learning
CN113328938A
Optical fiber communication channel management method and system based on cloud platform
CN117811905A
Method and apparatus for constructing dispatching model of integrated energy system, medium, and electronic device
WO2022160705A1
