Network bandwidth prediction method and system

Abstract

The invention discloses a network bandwidth prediction method and system, and the method comprises the steps: collecting historical bandwidth data of a target network, carrying out the prediction through a deep reinforcement learning algorithm, and obtaining a reinforcement learning bandwidth prediction value at each moment; predicting by adopting a heuristic algorithm to obtain a heuristic bandwidth predicted value at each moment; carrying out weighted fusion on the reinforcement learning bandwidth prediction value and the heuristic bandwidth prediction value to obtain a fusion prediction bandwidth at each moment; carrying out serialization processing on the fusion prediction bandwidth at all moments to obtain a fusion prediction bandwidth sequence, and carrying out first-level moving smoothing processing on the fusion prediction bandwidth sequence by adopting an autoregression moving average model to obtain a prediction bandwidth first-level smoothing sequence; and performing secondary dynamic smoothing processing on the fused predicted bandwidth sequence by adopting a Kalman filtering algorithm to obtain a predicted bandwidth secondary smoothing sequence. According to the network bandwidth prediction method and system provided by the embodiment of the invention, the prediction accuracy of the network bandwidth is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of network bandwidth prediction, and in particular to a network bandwidth prediction method and system. Background Art

[0002] Network bandwidth prediction refers to the process of predicting the current network bandwidth size in real time. By predicting the network bandwidth, the network transmission capacity can be estimated in advance to avoid sending too much data when the network bandwidth is insufficient, which may lead to network congestion, delays and packet loss. This will optimize the data transmission strategy, ensure smooth data flow, reduce delays and packet loss, and improve user experience and system efficiency.

[0003] However, since network bandwidth is affected by many factors, these factors may change significantly when the network fluctuates, thereby affecting the bandwidth prediction results. As a result, existing network bandwidth prediction technologies are difficult to accurately predict network bandwidth when the network fluctuates greatly.

[0004] It can be seen that how to improve the accuracy of network bandwidth prediction when the network fluctuates greatly has become a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0005] The present invention provides a network bandwidth prediction method and system to solve the technical problem that the existing network bandwidth prediction technology is difficult to accurately predict the network bandwidth when the network fluctuates greatly.

[0006] In order to solve the above technical problem, an embodiment of the present invention provides a network bandwidth prediction method.

[0007] Collect historical bandwidth data of the target network, use a deep reinforcement learning algorithm to predict the bandwidth of the target network at each moment based on the historical bandwidth data, and obtain a reinforcement learning bandwidth prediction value at each moment; use a heuristic algorithm to predict the bandwidth of the target network based on the historical bandwidth data, and obtain a heuristic bandwidth prediction value at each moment;

[0008] Performing smoothing confidence processing on the reinforcement learning bandwidth prediction value and the heuristic bandwidth prediction value to obtain the deep reinforcement smoothing confidence weight and the corresponding heuristic prediction smoothing confidence weight at each moment;

[0009] Based on the heuristic prediction smooth confidence weight at each moment and the corresponding deep reinforcement smooth confidence weight, the reinforcement learning bandwidth prediction value at each moment and the corresponding heuristic bandwidth prediction value are fused and analyzed to obtain a fused prediction bandwidth at each moment;

[0010] The fused prediction bandwidth at all times is serialized to obtain a fused prediction bandwidth sequence; an autoregressive moving average model is used to perform a first-level moving smoothing process on the fused prediction bandwidth sequence to obtain a first-level smoothing sequence of prediction bandwidth;

[0011] The Kalman filter algorithm is used to perform secondary dynamic smoothing on the fused prediction bandwidth sequence to obtain a secondary smoothing sequence of the prediction bandwidth.

[0012] As one of the preferred solutions, the reinforcement learning bandwidth prediction value and the heuristic bandwidth prediction value are smoothed and confidence processed to obtain the deep reinforcement smooth confidence weight and the corresponding heuristic prediction smooth confidence weight at each moment, including:

[0013] Collecting the actual measured value of bandwidth at each moment, and calculating the deep reinforcement confidence weight at each moment and the heuristic prediction confidence weight at each moment based on the difference characteristics among the reinforcement learning bandwidth prediction value, the heuristic bandwidth prediction value and the actual measured value of bandwidth;

[0014] The heuristic prediction confidence weight at each moment is smoothed by the heuristic prediction confidence weight at the adjacent moment corresponding to each moment to obtain the heuristic prediction smoothed confidence weight at each moment;

[0015] The depth enhancement confidence weight at each moment is smoothed by the depth enhancement confidence weight at the adjacent moments corresponding to each moment to obtain the depth enhancement smoothed confidence weight at each moment.

[0016] As one of the preferred solutions, the deep reinforcement confidence weight at each moment and the heuristic prediction confidence weight at each moment are calculated based on the difference characteristics between the reinforcement learning bandwidth prediction value, the heuristic bandwidth prediction value and the actual bandwidth measurement value, respectively, including:

[0017] The absolute value of the difference between the reinforcement learning bandwidth prediction value and the corresponding actual bandwidth measurement value at each moment is used as the reinforcement learning bandwidth prediction error at each moment, and the absolute value of the difference between the heuristic bandwidth prediction value and the corresponding actual bandwidth measurement value at each moment is used as the heuristic bandwidth prediction error at each moment;

[0018] The sum of the reinforcement learning bandwidth prediction error and the corresponding heuristic bandwidth prediction error at each moment is used as the fused bandwidth prediction error at each moment, and the ratio of the reinforcement learning bandwidth prediction error and the corresponding fused bandwidth prediction error at each moment is used as the corresponding heuristic prediction confidence weight;

[0019] The difference between the value 1 and the heuristic prediction confidence weight at each moment is used as the deep reinforcement confidence weight at each moment.

[0020] As one of the preferred solutions, the reinforcement learning bandwidth prediction value at each moment and the corresponding heuristic bandwidth prediction value are fused and analyzed based on the heuristic prediction smoothing confidence weight at each moment and the corresponding deep reinforcement smoothing confidence weight to obtain the fused prediction bandwidth at each moment, including:

[0021] Comparing and analyzing the heuristic prediction smoothing confidence weight and the calibration threshold at each moment to determine the prediction mode at each moment;

[0022] The maximum value of the heuristic prediction smooth confidence weight and the corresponding deep enhancement smooth confidence weight at each moment is used as the prediction enhancement weight at each moment, and the minimum value of the heuristic prediction smooth confidence weight and the corresponding deep enhancement smooth confidence weight at each moment is used as the prediction weakening weight at each moment;

[0023] Based on the prediction mode at each moment, the reinforcement learning bandwidth prediction value at each moment and the corresponding heuristic bandwidth prediction value are weightedly fused with the prediction enhancement weight at each moment and the corresponding prediction weakening weight to obtain the fused prediction bandwidth at each moment.

[0024] As one of the preferred solutions, the prediction mode includes a strong heuristic mode and a weak heuristic mode;

[0025] The comparative analysis of the heuristic prediction smoothing confidence weight and the calibration threshold at each moment to determine the prediction mode at each moment includes:

[0026] When the heuristic prediction smoothing confidence weight at each moment is greater than or equal to the calibration threshold, determining that the prediction mode at the corresponding moment is a strong heuristic mode;

[0027] When the heuristic prediction smoothing confidence weight at each moment is less than the calibration threshold, the prediction mode at the corresponding moment is determined to be a weak heuristic mode.

[0028] As one of the preferred solutions, based on the prediction mode at each moment, the reinforcement learning bandwidth prediction value at each moment and the corresponding heuristic bandwidth prediction value are weightedly fused with the prediction reinforcement weight at each moment and the corresponding prediction weakening weight to obtain the fused prediction bandwidth at each moment, including:

[0029] When the prediction mode at each moment is a strong heuristic mode, the prediction reinforcement weight at each moment, the corresponding prediction weakening weight, the corresponding reinforcement learning bandwidth prediction value and the corresponding heuristic bandwidth prediction value are input into a strong heuristic prediction fusion expression to calculate the fusion prediction bandwidth at each moment;

[0030] The strong heuristic prediction fusion expression is:

[0031]

[0032] Among them, PCB i is the fusion prediction bandwidth at the i-th moment, is the predicted reinforcement weight at the i-th moment, is the predicted weakening weight at the i-th moment, Psn i is the heuristic bandwidth prediction value at the i-th moment, Pqf i is the predicted value of reinforcement learning bandwidth at the i-th moment.

[0033] As one of the preferred solutions, based on the prediction mode at each moment, the reinforcement learning bandwidth prediction value at each moment and the corresponding heuristic bandwidth prediction value are weightedly fused with the prediction enhancement weight at each moment and the corresponding prediction weakening weight to obtain the fused prediction bandwidth at each moment, further comprising:

[0034] When the prediction mode at each moment is a weak heuristic mode, the prediction reinforcement weight at each moment, the corresponding prediction weakening weight, the corresponding reinforcement learning bandwidth prediction value and the corresponding heuristic bandwidth prediction value are input into a weak heuristic prediction fusion expression to calculate the fusion prediction bandwidth at each moment;

[0035] The weak heuristic prediction fusion expression is:

[0036]

[0037] Among them, PCB i is the fusion prediction bandwidth at the i-th moment, is the predicted reinforcement weight at the i-th moment, is the predicted weakening weight at the i-th moment, Psn i is the heuristic bandwidth prediction value at the i-th moment, Pqf i is the predicted value of reinforcement learning bandwidth at the i-th moment.

[0038] As one of the preferred solutions, the Kalman filter algorithm is used to perform secondary dynamic smoothing on the fused prediction bandwidth sequence to obtain a secondary smoothed sequence of prediction bandwidth, including:

[0039] Using Kalman filter algorithm to perform state estimation on the fusion prediction bandwidth sequence;

[0040] Calculate the prediction residual and covariance matrix based on the state estimation result and the difference characteristics of the observation value;

[0041] Performing anomaly detection on the covariance matrix, when the prediction residual is greater than a preset threshold or the covariance matrix is ​​abnormal, switching to using an unscented Kalman filter algorithm to perform state estimation on the fusion prediction bandwidth sequence;

[0042] The state estimation result is serialized as a prediction bandwidth secondary smoothed sequence.

[0043] As one of the preferred solutions, before the first-level moving smoothing process, the fused prediction bandwidth sequence is subjected to time series stationarity processing;

[0044] The stationarity processing includes:

[0045] A time series stationarity check is performed on the fused prediction bandwidth sequence. If the result of the time series stationarity check is a non-stationary sequence, an iterative difference processing is performed on the fused prediction bandwidth sequence, and a time series stationarity check is performed on the time series obtained by each difference processing until the result of the time series stationarity check is a stationary sequence.

[0046] Yet another embodiment of the present invention provides a network bandwidth prediction system, the system comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the network bandwidth prediction method as described above when executing the computer program.

[0047] Compared with the prior art, the embodiments of the present invention have the following advantages:

[0048] (1) Collect historical bandwidth data of the target network, use a deep reinforcement learning algorithm to predict the bandwidth of the target network at each moment based on the historical bandwidth data, and obtain a reinforcement learning bandwidth prediction value at each moment; use a heuristic algorithm to predict the bandwidth of the target network based on the historical bandwidth data, and obtain a heuristic bandwidth prediction value at each moment;

[0049] (2) Perform smooth confidence processing on the reinforcement learning bandwidth prediction value and the heuristic bandwidth prediction value to obtain the deep reinforcement smooth confidence weight and the corresponding heuristic prediction smooth confidence weight at each moment; introduce an adaptive fusion mechanism to perform fusion analysis on the reinforcement learning bandwidth prediction value and the corresponding heuristic bandwidth prediction value at each moment based on the heuristic prediction smooth confidence weight and the corresponding deep reinforcement smooth confidence weight at each moment to obtain the fused prediction bandwidth at each moment, thereby improving the reliability and accuracy of network bandwidth prediction;

[0050] (3) The fused prediction bandwidth at all times is serialized to obtain a fused prediction bandwidth sequence. Considering that in a complex network environment, excessive network fluctuations may lead to poor smoothing effects, the autoregressive moving average model is first used to perform a first-level moving smoothing process on the fused prediction bandwidth sequence to obtain a first-level smoothed sequence of the prediction bandwidth; then the Kalman filter algorithm is used to perform a second-level dynamic smoothing process on the fused prediction bandwidth sequence to obtain a second-level smoothed sequence of the prediction bandwidth. A two-level smoothing model is introduced to perform a two-level smoothing process on the prediction bandwidth sequence obtained after fusion, and a dynamic switching mechanism of the smoothing model is added to the second-level dynamic smoothing process. Dynamic switching is performed between different smoothing models according to the linear characteristics of the system. In the case of dynamic network fluctuations, the network bandwidth is adaptively smoothed in real time to avoid the prediction of the network bandwidth being affected by sudden network fluctuations, thereby improving the prediction accuracy of the network bandwidth.

[0051] (4) By introducing an adaptive fusion mechanism and a two-level smoothing model, the prediction results of the heuristic algorithm and the deep reinforcement learning algorithm are integrated, and the fused prediction results are dynamically smoothed to avoid the prediction of network bandwidth being affected by sudden network fluctuations, thereby comprehensively improving the prediction accuracy of network bandwidth. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A flowchart of a network bandwidth prediction method in one embodiment of the present invention;

[0053] Figure 2 It is a schematic diagram of obtaining a secondary smoothed sequence of predicted bandwidth in one embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] In the description of this application, the terms "first", "second", "third", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", "third", etc. may explicitly or implicitly include one or more of the feature. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0056] In the description of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be a connection between the two elements. The terms "vertical", "horizontal", "left", "right", "upper", "lower" and similar expressions used herein are only for illustrative purposes, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0057] In the description of this application, it should be noted that, unless otherwise defined, all technical and scientific terms used in the present invention have the same meaning as those commonly understood by those skilled in the art. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood by specific circumstances.

[0058] An embodiment of the present invention provides a network bandwidth prediction method. For details, see Figure 1-2 , Figure 1 FIG. 1 is a flow chart of a network bandwidth prediction method in one embodiment of the present invention. Figure 2 It is a schematic diagram showing the acquisition of the predicted bandwidth secondary smoothing sequence in one embodiment of the present invention.

[0059] A flowchart of a network bandwidth prediction method in one embodiment of the present invention includes the following steps S1 to S4, which are specifically as follows:

[0060] Step S1: Collect historical bandwidth data of the target network.

[0061] It should be noted that network bandwidth prediction refers to the process of predicting the current network bandwidth in real time. By predicting the network bandwidth, the network transmission capacity can be estimated in advance to avoid sending too much data when the network bandwidth is insufficient, which will lead to network congestion, delays, packet loss and other problems. Before predicting the network bandwidth, it is necessary to collect historical bandwidth data, and then analyze the historical bandwidth data to obtain the hidden patterns and change trends in the historical bandwidth data, so as to predict the bandwidth of the target network.

[0062] Specifically, the bandwidth of the target network is collected and recorded at regular intervals to form historical bandwidth data of the target network, and data cleaning and outlier processing are performed on the historical bandwidth data.

[0063] Step S2: Use a deep reinforcement learning algorithm to predict the bandwidth of the target network at each moment based on historical bandwidth data, and obtain a reinforcement learning bandwidth prediction value at each moment; use a heuristic algorithm to predict the bandwidth of the target network based on historical bandwidth data, and obtain a heuristic bandwidth prediction value at each moment.

[0064] It should be noted that the core idea of ​​the deep reinforcement learning algorithm is to use neural networks to represent and learn complex data relationships, and learn how to make the best decisions by interacting with the environment. It combines the perception ability of deep learning and the decision-making ability of reinforcement learning, enabling computer systems to process high-dimensional state and action spaces and maximize cumulative rewards in long-term behaviors. Heuristic algorithms estimate network states through predefined rules or formulas. Deep reinforcement learning algorithms and heuristic algorithms have their own advantages and disadvantages in network bandwidth prediction. Deep reinforcement learning algorithms are suitable for processing complex and dynamic network environments, but require a large amount of data and computing resources and are sensitive to parameter selection. Heuristic algorithms have the advantages of high efficiency, adaptability, and robustness, but may not be able to find the global optimal solution, and it is difficult to select heuristic information.

[0065] Therefore, by integrating the prediction results of deep reinforcement learning algorithm and heuristic algorithm and combining the advantages of both, the accuracy of network bandwidth prediction can be improved.

[0066] Specifically, a deep reinforcement learning algorithm is used to predict the bandwidth of the target network at each moment based on historical bandwidth data to obtain a reinforcement learning bandwidth prediction value at each moment, and a heuristic algorithm is used to predict the bandwidth of the target network based on historical bandwidth data to obtain a heuristic bandwidth prediction value at each moment.

[0067] As an embodiment of the present application, a deep reinforcement learning algorithm adopts an asymmetric Actor-Critic architecture.

[0068] Step S3: Perform smooth confidence processing on the reinforcement learning bandwidth prediction value and the heuristic bandwidth prediction value to obtain the deep reinforcement smooth confidence weight and the corresponding heuristic prediction smooth confidence weight at each moment; perform fusion analysis on the reinforcement learning bandwidth prediction value and the corresponding heuristic bandwidth prediction value at each moment based on the heuristic prediction smooth confidence weight and the corresponding deep reinforcement smooth confidence weight at each moment to obtain the fused prediction bandwidth at each moment.

[0069] It should be noted that in order to take advantage of the advantages of different models while reducing the bias of a single model, the prediction results of the deep reinforcement learning algorithm and the heuristic algorithm are fused and analyzed, and the combined weight of the two is dynamically adjusted.

[0070] Specifically, the actual measured value of the bandwidth at each moment is collected, and the deep reinforcement confidence weight at each moment and the heuristic prediction confidence weight at each moment are calculated based on the difference characteristics among the reinforcement learning bandwidth prediction value, the heuristic bandwidth prediction value and the actual measured value of the bandwidth.

[0071] Furthermore, the heuristic prediction confidence weight at each moment is smoothed with the heuristic prediction confidence weight of the adjacent moments corresponding to each moment to obtain the heuristic prediction smoothed confidence weight at each moment; the depth enhancement confidence weight at each moment is smoothed with the depth enhancement confidence weight of the adjacent moments corresponding to each moment to obtain the depth enhancement smoothed confidence weight at each moment.

[0072] In this step, the specific calculation method of the deep reinforcement confidence weight and the heuristic prediction confidence weight at each moment is as follows: the absolute value of the difference between the reinforcement learning bandwidth prediction value and the corresponding actual bandwidth measurement value at each moment is taken as the reinforcement learning bandwidth prediction error at each moment, and the absolute value of the difference between the heuristic bandwidth prediction value and the corresponding actual bandwidth measurement value at each moment is taken as the heuristic bandwidth prediction error at each moment; the sum of the reinforcement learning bandwidth prediction error and the corresponding heuristic bandwidth prediction error at each moment is taken as the fusion bandwidth prediction error at each moment, and the ratio of the reinforcement learning bandwidth prediction error and the corresponding fusion bandwidth prediction error at each moment is taken as the corresponding heuristic prediction confidence weight; the difference between the value 1 and the heuristic prediction confidence weight at each moment is taken as the deep reinforcement confidence weight at each moment.

[0073] It should be noted that when the reinforcement learning bandwidth prediction error is relatively larger, it means that the prediction effect of the deep reinforcement learning algorithm is worse, and the prediction results of the heuristic algorithm should be given a larger confidence weight.

[0074] Specifically, a comparative analysis is performed on the heuristic prediction smoothing confidence weight and the calibration threshold at each moment to determine the prediction mode at each moment; the maximum value of the heuristic prediction smoothing confidence weight at each moment and the corresponding deep enhancement smoothing confidence weight is used as the prediction enhancement weight at each moment, and the minimum value of the heuristic prediction smoothing confidence weight at each moment and the corresponding deep enhancement smoothing confidence weight is used as the prediction weakening weight at each moment.

[0075] In this step, the prediction mode includes a strong heuristic mode and a weak heuristic mode. The heuristic prediction smoothing confidence weight and the calibration threshold at each moment are compared and analyzed, and the specific method for determining the prediction mode at each moment is: when the heuristic prediction smoothing confidence weight at each moment is greater than or equal to the calibration threshold, the prediction mode at the corresponding moment is determined to be a strong heuristic mode; when the heuristic prediction smoothing confidence weight at each moment is less than the calibration threshold, the prediction mode at the corresponding moment is determined to be a weak heuristic mode.

[0076] Furthermore, when the prediction mode at each moment is the strong heuristic mode, the prediction reinforcement weight at each moment, the corresponding prediction weakening weight, the corresponding reinforcement learning bandwidth prediction value, and the corresponding heuristic bandwidth prediction value are input into the strong heuristic prediction fusion expression to calculate the fusion prediction bandwidth at each moment;

[0077] The strong heuristic prediction fusion expression is:

[0078]

[0079] Among them, PCB i is the fusion prediction bandwidth at the i-th moment, is the predicted reinforcement weight at the i-th moment, is the predicted weakening weight at the i-th moment, Psn i is the heuristic bandwidth prediction value at the i-th moment, Pqf i is the predicted value of reinforcement learning bandwidth at the i-th moment.

[0080] When the prediction mode at each moment is the weak heuristic mode, the prediction reinforcement weight at each moment, the corresponding prediction weakening weight, the corresponding reinforcement learning bandwidth prediction value and the corresponding heuristic bandwidth prediction value are input into the weak heuristic prediction fusion expression to calculate the fusion prediction bandwidth at each moment;

[0081] The weak heuristic prediction fusion expression is:

[0082]

[0083] Among them, PCB i is the fusion prediction bandwidth at the i-th moment, is the predicted reinforcement weight at the i-th moment, is the predicted weakening weight at the i-th moment, Psn i is the heuristic bandwidth prediction value at the i-th moment, Pqf i is the predicted value of reinforcement learning bandwidth at the i-th moment.

[0084] Step S4: Serialize the fused prediction bandwidth at all times to obtain a fused prediction bandwidth sequence; use the autoregressive moving average model to perform a first-level moving smoothing process on the fused prediction bandwidth sequence to obtain a first-level smoothing sequence of the prediction bandwidth; use the Kalman filter algorithm to perform a second-level dynamic smoothing process on the fused prediction bandwidth sequence to obtain a second-level smoothing sequence of the prediction bandwidth.

[0085] It should be noted that the Kalman filter algorithm is a recursive algorithm for estimating the state of a linear dynamic system. It estimates the system state and minimizes the uncertainty of the estimate by combining the system model and observation data through two steps of prediction and update.

[0086] Specifically, an autoregressive moving average model is used to perform first-level moving smoothing processing on the fused prediction bandwidth sequence to obtain a first-level smoothed sequence of the prediction bandwidth.

[0087] It should be noted that before the first-level moving smoothing process, the fused prediction bandwidth sequence is subjected to time series stationarity processing.

[0088] As an embodiment of the present application, the stationarity processing includes: performing a time series stationarity check on the fused prediction bandwidth sequence; if the result of the time series stationarity check is a non-stationary sequence, performing iterative differential processing on the fused prediction bandwidth sequence, and performing a time series stationarity check on the time series obtained by each differential processing until the result of the time series stationarity check is a stationary sequence.

[0089] Furthermore, a Kalman filter algorithm is used to perform state estimation on the fused prediction bandwidth sequence; the prediction residual and covariance matrix are calculated based on the results of the state estimation and the difference characteristics of the observed values; the covariance matrix is ​​detected for anomalies, and when the prediction residual is greater than a preset threshold or the covariance matrix is ​​abnormal, the unscented Kalman filter algorithm is switched to perform state estimation on the fused prediction bandwidth sequence; the results of the state estimation are serialized as a secondary smoothed sequence of the prediction bandwidth.

[0090] Yet another embodiment of the present invention provides a network bandwidth prediction system, the system comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the network bandwidth prediction method as described above when executing the computer program.

[0091] Compared with the prior art, the network bandwidth prediction method and system provided by the embodiments of the present invention have the following beneficial effects:

[0092] (1) Collect historical bandwidth data of the target network, use a deep reinforcement learning algorithm to predict the bandwidth of the target network at each moment based on the historical bandwidth data, and obtain a reinforcement learning bandwidth prediction value at each moment; use a heuristic algorithm to predict the bandwidth of the target network based on the historical bandwidth data, and obtain a heuristic bandwidth prediction value at each moment;

[0093] (2) Perform smooth confidence processing on the reinforcement learning bandwidth prediction value and the heuristic bandwidth prediction value to obtain the deep reinforcement smooth confidence weight and the corresponding heuristic prediction confidence weight at each moment; introduce an adaptive fusion mechanism to fuse and analyze the reinforcement learning bandwidth prediction value and the corresponding heuristic bandwidth prediction value at each moment based on the heuristic prediction smooth confidence weight and the corresponding deep reinforcement confidence weight at each moment to obtain the fused prediction bandwidth at each moment, thereby improving the reliability and accuracy of network bandwidth prediction;

[0094] (3) The fused prediction bandwidth at all times is serialized to obtain a fused prediction bandwidth sequence. Considering that in a complex network environment, excessive network fluctuations may lead to poor smoothing effects, the autoregressive moving average model is first used to perform a first-level moving smoothing process on the fused prediction bandwidth sequence to obtain a first-level smoothed sequence of the prediction bandwidth; then the Kalman filter algorithm is used to perform a second-level dynamic smoothing process on the fused prediction bandwidth sequence to obtain a second-level smoothed sequence of the prediction bandwidth. A two-level smoothing model is introduced to perform a two-level smoothing process on the prediction bandwidth sequence obtained after fusion, and a dynamic switching mechanism of the smoothing model is added to the second-level dynamic smoothing process. Dynamic switching is performed between different smoothing models according to the linear characteristics of the system. In the case of dynamic network fluctuations, the network bandwidth is adaptively smoothed in real time to avoid the prediction of the network bandwidth being affected by sudden network fluctuations, thereby improving the prediction accuracy of the network bandwidth.

[0095] (4) By introducing an adaptive fusion mechanism and a two-level smoothing model, the prediction results of the heuristic algorithm and the deep reinforcement learning algorithm are integrated, and the fused prediction results are dynamically smoothed to avoid the prediction of network bandwidth being affected by sudden network fluctuations, thereby comprehensively improving the prediction accuracy of network bandwidth.

[0096] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A network bandwidth prediction method, characterized in that: The method comprises: Collect historical bandwidth data of the target network, use a deep reinforcement learning algorithm to predict the bandwidth of the target network at each moment based on the historical bandwidth data, and obtain a reinforcement learning bandwidth prediction value at each moment; use a heuristic algorithm to predict the bandwidth of the target network based on the historical bandwidth data, and obtain a heuristic bandwidth prediction value at each moment; Performing smoothing confidence processing on the reinforcement learning bandwidth prediction value and the heuristic bandwidth prediction value to obtain the deep reinforcement smoothing confidence weight and the corresponding heuristic prediction smoothing confidence weight at each moment; Based on the heuristic prediction smooth confidence weight at each moment and the corresponding deep reinforcement smooth confidence weight, the reinforcement learning bandwidth prediction value at each moment and the corresponding heuristic bandwidth prediction value are fused and analyzed to obtain a fused prediction bandwidth at each moment; The fused prediction bandwidth at all times is serialized to obtain a fused prediction bandwidth sequence; an autoregressive moving average model is used to perform a first-level moving smoothing process on the fused prediction bandwidth sequence to obtain a first-level smoothing sequence of prediction bandwidth; The Kalman filter algorithm is used to perform secondary dynamic smoothing on the fused prediction bandwidth sequence to obtain a secondary smoothing sequence of the prediction bandwidth.

2. A network bandwidth prediction method according to claim 1, characterized in that: The step of performing smoothing confidence processing on the reinforcement learning bandwidth prediction value and the heuristic bandwidth prediction value to obtain the deep reinforcement smoothing confidence weight and the corresponding heuristic prediction smoothing confidence weight at each moment includes: Collecting the actual measured value of bandwidth at each moment, and calculating the deep reinforcement confidence weight at each moment and the heuristic prediction confidence weight at each moment based on the difference characteristics among the reinforcement learning bandwidth prediction value, the heuristic bandwidth prediction value and the actual measured value of bandwidth; The heuristic prediction confidence weight at each moment is smoothed by the heuristic prediction confidence weight at the adjacent moment corresponding to each moment to obtain the heuristic prediction smoothed confidence weight at each moment; The depth enhancement confidence weight at each moment is smoothed by the depth enhancement confidence weight at the adjacent moments corresponding to each moment to obtain the depth enhancement smoothed confidence weight at each moment.

3. A network bandwidth prediction method according to claim 2, characterized in that: The calculating of the deep reinforcement confidence weight at each moment and the heuristic prediction confidence weight at each moment based on the difference characteristics among the reinforcement learning bandwidth prediction value, the heuristic bandwidth prediction value and the actual bandwidth measurement value respectively includes: The absolute value of the difference between the reinforcement learning bandwidth prediction value and the corresponding actual bandwidth measurement value at each moment is used as the reinforcement learning bandwidth prediction error at each moment, and the absolute value of the difference between the heuristic bandwidth prediction value and the corresponding actual bandwidth measurement value at each moment is used as the heuristic bandwidth prediction error at each moment; The sum of the reinforcement learning bandwidth prediction error and the corresponding heuristic bandwidth prediction error at each moment is used as the fused bandwidth prediction error at each moment, and the ratio of the reinforcement learning bandwidth prediction error and the corresponding fused bandwidth prediction error at each moment is used as the corresponding heuristic prediction confidence weight; The difference between the value 1 and the heuristic prediction confidence weight at each moment is used as the deep reinforcement confidence weight at each moment.

4. A network bandwidth prediction method according to claim 1, characterized in that: The step of fusing and analyzing the reinforcement learning bandwidth prediction value at each moment and the corresponding heuristic bandwidth prediction value based on the heuristic prediction smoothing confidence weight at each moment and the corresponding deep reinforcement smoothing confidence weight to obtain the fused prediction bandwidth at each moment includes: Comparing and analyzing the heuristic prediction smoothing confidence weight and the calibration threshold at each moment to determine the prediction mode at each moment; The maximum value of the heuristic prediction smooth confidence weight and the corresponding deep enhancement smooth confidence weight at each moment is used as the prediction enhancement weight at each moment, and the minimum value of the heuristic prediction smooth confidence weight and the corresponding deep enhancement smooth confidence weight at each moment is used as the prediction weakening weight at each moment; Based on the prediction mode at each moment, the reinforcement learning bandwidth prediction value at each moment and the corresponding heuristic bandwidth prediction value are weightedly fused with the prediction enhancement weight at each moment and the corresponding prediction weakening weight to obtain the fused prediction bandwidth at each moment.

5. A network bandwidth prediction method according to claim 4, characterized in that: The prediction mode includes a strong heuristic mode and a weak heuristic mode; The step of comparing and analyzing the heuristic prediction smoothing confidence weight and the calibration threshold at each moment to determine the prediction mode at each moment includes: When the heuristic prediction smoothing confidence weight at each moment is greater than or equal to the calibration threshold, determining that the prediction mode at the corresponding moment is a strong heuristic mode; When the heuristic prediction smoothing confidence weight at each moment is less than the calibration threshold, the prediction mode at the corresponding moment is determined to be a weak heuristic mode.

6. A network bandwidth prediction method according to claim 5, characterized in that: The prediction mode at each moment is based on the prediction strengthening weight at each moment and the corresponding prediction weakening weight at each moment to weightedly fuse the reinforcement learning bandwidth prediction value and the corresponding heuristic bandwidth prediction value to obtain the fused prediction bandwidth at each moment, including: When the prediction mode at each moment is a strong heuristic mode, the prediction reinforcement weight at each moment, the corresponding prediction weakening weight, the corresponding reinforcement learning bandwidth prediction value and the corresponding heuristic bandwidth prediction value are input into a strong heuristic prediction fusion expression to calculate the fusion prediction bandwidth at each moment; The strong heuristic prediction fusion expression is: Among them, PCB i is the fusion prediction bandwidth at the i-th moment, is the predicted reinforcement weight at the i-th moment, is the predicted weakening weight at the i-th moment, Psn i is the heuristic bandwidth prediction value at the i-th moment, Pqd i is the predicted value of reinforcement learning bandwidth at the i-th moment.

7. A network bandwidth prediction method according to claim 6, characterized in that: The prediction mode at each moment is based on the prediction strengthening weight at each moment and the corresponding prediction weakening weight at each moment to weightedly fuse the reinforcement learning bandwidth prediction value and the corresponding heuristic bandwidth prediction value to obtain the fused prediction bandwidth at each moment, and further includes: When the prediction mode at each moment is a weak heuristic mode, the prediction reinforcement weight at each moment, the corresponding prediction weakening weight, the corresponding reinforcement learning bandwidth prediction value and the corresponding heuristic bandwidth prediction value are input into a weak heuristic prediction fusion expression to calculate the fusion prediction bandwidth at each moment; The weak heuristic prediction fusion expression is: Among them, PCB i is the fusion prediction bandwidth at the i-th moment, is the predicted reinforcement weight at the i-th moment, is the predicted weakening weight at the i-th moment, Psn i is the heuristic bandwidth prediction value at the i-th moment, Pqf i is the predicted value of reinforcement learning bandwidth at the i-th moment.

8. A network bandwidth prediction method according to claim 1, characterized in that: The Kalman filter algorithm is used to perform secondary dynamic smoothing on the fused prediction bandwidth sequence to obtain a secondary smoothed sequence of prediction bandwidth, including: Using Kalman filter algorithm to perform state estimation on the fusion prediction bandwidth sequence; Calculate the prediction residual and covariance matrix based on the state estimation result and the difference characteristics of the observed value; Performing anomaly detection on the covariance matrix, when the prediction residual is greater than a preset threshold or the covariance matrix is ​​abnormal, switching to using an unscented Kalman filter algorithm to perform state estimation on the fusion prediction bandwidth sequence; The state estimation result is serialized as a prediction bandwidth secondary smoothed sequence.

9. A network bandwidth prediction method according to claim 1, characterized in that: Before the first-level moving smoothing process, performing time series stationarity processing on the fused prediction bandwidth sequence; The stationarity processing includes: A time series stationarity check is performed on the fused prediction bandwidth sequence. If the result of the time series stationarity check is a non-stationary sequence, an iterative difference processing is performed on the fused prediction bandwidth sequence, and a time series stationarity check is performed on the time series obtained by each difference processing until the result of the time series stationarity check is a stationary sequence.

10. A network bandwidth prediction system, characterized in that: The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, a network bandwidth prediction method according to any one of claims 1 to 9 is implemented.

Images