Bandwidth prediction method based on LSTM and XGBoost fusion

Abstract

A bandwidth prediction method that integrates LSTM and XGBoost includes the following steps: 1) collecting bandwidth-related time series feature data; 2) using an LSTM model to process the time series feature data and capture the dynamic changes in historical bandwidth; introducing an attention mechanism into the LSTM model to calculate the weights of time steps, enabling the LSTM+Attention model to focus on the time steps that have the greatest impact on prediction and ignore irrelevant information; 3) feature fusion: using the output of the LSTM+Attention model as one of the features, while also introducing other network state features; 4) XGBoost model: using the XGBoost algorithm to model the concatenated high-dimensional feature space matrix; and 5) weighted summing the prediction results of all XGBoost trees to obtain the final bandwidth prediction value. This method improves the accuracy and robustness of bandwidth prediction.

Description

Technical Field

[0001] The present invention relates to network bandwidth prediction technology, and in particular to a bandwidth prediction method for virtual reality (VR) video streaming scenarios. Combining a long short-term memory network (LSTM), an attention mechanism, and an XGBoost algorithm, a fusion model is proposed to improve the accuracy and robustness of bandwidth prediction in a dynamic network environment. Background Art

[0002] In VR video streaming scenarios, users connect to servers via wireless networks to receive high-quality video content. Users' bandwidth requirements are not only affected by network conditions but also by factors such as user location, device performance, and signal quality. Especially when users are wearing VR headsets (HMDs) and moving around, network bandwidth fluctuates. Accurately predicting bandwidth fluctuations is crucial for optimizing video streaming quality. Traditional bandwidth prediction methods rely primarily on simple time series models or statistical algorithms, which often overlook the complexity of network environments and the dynamic changes in user behavior. Therefore, developing a more robust and efficient bandwidth prediction method by combining time series data, feature information, and network fluctuations has become a key research topic. Summary of the Invention

[0003] To overcome the shortcomings of existing technologies, the present invention provides a bandwidth prediction method that integrates LSTM and XGBoost. By combining the LSTM model (for capturing temporal dependencies) and the attention mechanism (for focusing on the key time steps that have the greatest impact on bandwidth prediction), as well as the XGBoost model (for feature selection and optimizing nonlinear relationships), the accuracy and robustness of bandwidth prediction are improved. This method can adapt to changes in the network environment and user mobility in real time, thereby optimizing the bitrate adaptive control of VR video streams.

[0004] The technical solution adopted by the present invention to solve its technical problem is:

[0005] A bandwidth prediction method integrating LSTM and XGBoost includes the following steps:

[0006] 1) Data collection: Collect bandwidth-related timing characteristic data, including:

[0007] Historical bandwidth: records the user's bandwidth usage in the past time window;

[0008] User location information: Obtain user location data through GPS to capture the impact of user movement on bandwidth requirements;

[0009] Delay data: records network transmission delays and evaluates network performance;

[0010] Network load: Statistics on the current network traffic load and analysis of network congestion;

[0011] Clean, standardize, and normalize time series feature data to ensure consistent scales for different features and facilitate input into the model. Divide the data into training and test sets for model training and performance evaluation.

[0012] 2) LSTM Model and Attention Mechanism: The LSTM model is used to process time series feature data and capture dynamic changes in historical bandwidth. The attention mechanism is introduced into the LSTM model. By calculating the weights of time steps, the LSTM + Attention model can focus on the time steps that have the greatest impact on prediction and ignore irrelevant information.

[0013] 3) Feature Fusion: The output of the LSTM+Attention model, that is, the time series prediction value weighted by the attention mechanism, is used as one of the features. Other network state features are also introduced, including the following:

[0014] RSSI (Received Signal Strength Indicator): indicates the received signal strength and reflects the quality of the wireless signal;

[0015] RSRP (Reference Signal Received Power): represents the reference signal received power, which is used to evaluate the base station signal strength;

[0016] RSRQ (Reference Signal Received Quality): indicates the reference signal received quality, which comprehensively reflects the signal strength and interference level;

[0017] SINR (Signal-to-Interference-plus-Noise Ratio): represents the signal-to-noise ratio, which measures the relationship between signal quality and interference level;

[0018] The above features are concatenated with the output of the LSTM+Attention model into a new feature matrix to form a high-dimensional feature space matrix.

[0019] 4) XGBoost Model: Use the XGBoost algorithm to model the concatenated high-dimensional feature space matrix. XGBoost gradually captures the complex nonlinear relationships between features by constructing multiple decision trees. During the splitting process of each tree, XGBoost automatically evaluates the importance of each feature and selects the optimal split point, thereby achieving effective feature screening and modeling.

[0020] The XGBoost model is trained using a standard regression loss function to minimize the prediction error. Each new tree is trained and optimized based on the residual of the previous tree using the gradient boosting method.

[0021] 5) Output results: The prediction results of all XGBoost trees will be weighted and summed to obtain the final bandwidth prediction value.

[0022] The beneficial effects of the present invention are mainly manifested in:

[0023] (1) High accuracy: Combining the LSTM to capture the long-term dependency of time series data with the attention mechanism of the attention mechanism, the key factors of bandwidth changes can be captured more accurately.

[0024] (2) Strong robustness: XGBoost can improve the adaptability of the model in complex environments by selecting features and processing nonlinear relationships.

[0025] (3) Strong adaptability: The method of the present invention can cope with the influence of factors such as slow user movement and network environment fluctuations, adjust the bandwidth prediction results in real time, and improve the quality of video streaming.

[0026] (4) High efficiency: It integrates two powerful prediction models (LSTM+Attention and XGBoost), which can provide faster prediction response speed while ensuring prediction accuracy, and is suitable for real-time bandwidth prediction tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Flowchart of the bandwidth prediction method integrating LSTM and XGBoost of the present invention;

[0028] Figure 2 This is the LSTM network structure diagram of the present invention;

[0029] Figure 3 Graph showing the construction and training of the XGBoost model of this invention. DETAILED DESCRIPTION

[0030] The present invention will be further described below with reference to the accompanying drawings.

[0031] Reference Figures 1 to 3 , a bandwidth prediction method integrating LSTM and XGBoost, comprising the following steps:

[0032] 1) Data collection: Collect bandwidth-related timing characteristic data, including:

[0033] Historical bandwidth: records the user's bandwidth usage in the past time window;

[0034] User location information: Obtain user location data through GPS to capture the impact of user movement on bandwidth requirements;

[0035] Delay data: records network transmission delays and evaluates network performance;

[0036] Network load: Statistics on the current network traffic load and analysis of network congestion;

[0037] Clean, standardize, and normalize time series feature data to ensure consistent scales for different features and facilitate input into the model. Divide the data into training and test sets for model training and performance evaluation.

[0038] In this embodiment, in order to verify the effectiveness of the proposed bandwidth prediction method, a set of typical devices and network environments were used to conduct experiments. The experimental setup process is as follows:

[0039] Step 1.1) Use six devices: three mobile devices (smartphones) to simulate slow user movement. The positions and states of these devices are adjusted according to the experimental design to simulate the bandwidth requirements of users in different network environments; three head-mounted displays (HMDs) to simulate high-performance devices to ensure device diversity and meet experimental accuracy.

[0040] Step 1.2) Each device connects to the server via WiFi to transmit VR video data. Real-time data such as historical bandwidth, latency, and network load are collected via Android APIs at a 1-second granularity to collect key metrics for subsequent bandwidth prediction model training. User location information for each device is collected via GPS signals to help model user mobility behavior.

[0041] Step 1.3) Data cleaning: Ensure data quality and integrity and reduce the impact of noise and outliers on model training. The steps are as follows:

[0042] 1.3.1) Detect and remove missing values ​​or invalid data points;

[0043] 1.3.2) Process outliers and filter out extreme values ​​by setting thresholds;

[0044] 1.3.3) Interpolate incomplete time series data to ensure data continuity;

[0045] Step 1.4) Standardize and normalize all features, adjust the scales of different features to make them on the same level, which is convenient for model input and training, as follows:

[0046] 1.4.1) Standardization: Convert the data into a distribution with a mean of 0 and a standard deviation of 1. The formula is:

[0047]

[0048] Where x is the original data, μ is the mean, and σ is the standard deviation;

[0049] 1.4.2) Normalization: Scale the data to the [0,1] interval. The formula is:

[0050]

[0051] Among them, min(x) and max(x) are the minimum and maximum values ​​of the data respectively;

[0052] Step 1.5) Data Partitioning: Divide the data into training and test sets for model training and performance evaluation. The steps are as follows:

[0053] 1.5.1) Divide the data into chronological order, using 80% of the data as the training set and 20% of the data as the test set;

[0054] 1.5.2) Ensure the temporal continuity of the training and test sets to avoid data leakage;

[0055] 2) LSTM Model and Attention Mechanism: The LSTM model is used to process time series feature data and capture dynamic changes in historical bandwidth. The attention mechanism is introduced into the LSTM model. By calculating the weights of time steps, the LSTM + Attention model can focus on the time steps that have the greatest impact on prediction and ignore irrelevant information.

[0056] In this embodiment, the LSTM network structure construction and training process is as follows:

[0057] Step 2.1) Input Layer: Input features include historical bandwidth, user location information, latency data, and network load data. The input for each time step is a vector consisting of multiple features. A batch size of 32 is used, and 32 samples are used for each training run.

[0058] Step 2.2) LSTM layer: This layer consists of two LSTM layers (num_layers = 2) to learn short-term and long-term dependencies in time series data. The number of neurons in each layer is 64 and 32 respectively.

[0059] Step 2.3) Attention Mechanism: After the LSTM output layer, add an additive attention mechanism, which assigns a weight to the LSTM output at each time step, focusing on moments that have a greater impact on bandwidth prediction.

[0060] Step 2.4) Output layer: The output of the LSTM and attention mechanism is passed to a fully connected layer to predict the bandwidth value at the current moment;

[0061] Step 2.5) LSTM model training: Use mean squared error (MSE) as the loss function to train the model. The loss function is defined as:

[0062]

[0063] Where n is the number of samples, y i is the true bandwidth value of the i-th sample, is the bandwidth value predicted by the model for the i-th sample;

[0064] Use the gradient descent method (Adam optimizer) to update the model parameters to ensure that the model can gradually reduce the prediction error. The Adam optimizer update rule is:

[0065]

[0066] Where η is the learning rate; m t is the first moment estimate of the gradient, m t =β1·m t-1 +(1-β1)·g t ;v t is the second moment estimate of the gradient, ∈ is a very small constant 1e-8, which is used to prevent the stability constant from dividing by zero; the specific value of this embodiment is η is 0.001; g t is the gradient at the current moment, β1 is the decay rate of the first-order moment, which is set to 0.9; is the square of the current gradient, β2 is the decay rate of the second-order moment, set to 0.999;

[0067] Step 2.6) Attention mechanism model training: Using the Additive Attention method, the model's focus is adjusted by calculating the importance weight of each time step, automatically selecting important time segments for more detailed learning. The output of each moment is weighted summed to obtain the final bandwidth prediction result.

[0068] Step 2.7) During training, the LSTM+Attention model uses the training data for forward and backward propagation, continuously adjusting the LSTM weights via gradient descent to minimize the loss function until the model converges. During training, early stopping is used to avoid overfitting, i.e., early_stopping_patience = 5: training is stopped when the validation set loss does not improve within 5 consecutive epochs.

[0069] 3) Feature Fusion: This strategy combines the time series prediction capabilities of the LSTM+Attention model with the feature optimization capabilities of XGBoost to form a complementary fusion strategy. The output of the LSTM+Attention model, namely the time series prediction value weighted by the attention mechanism, is used as one of the features. Other network state features are also introduced, including the following:

[0070] RSSI (Received Signal Strength Indicator): indicates the received signal strength and reflects the quality of the wireless signal;

[0071] RSRP (Reference Signal Received Power): represents the reference signal received power, which is used to evaluate the base station signal strength;

[0072] RSRQ (Reference Signal Received Quality): indicates the reference signal received quality, which comprehensively reflects the signal strength and interference level;

[0073] SINR (Signal-to-Interference-plus-Noise Ratio): represents the signal-to-noise ratio, which measures the relationship between signal quality and interference level;

[0074] The above features are concatenated with the output of the LSTM+Attention model into a new feature matrix to form a high-dimensional feature space matrix.

[0075] In this embodiment, the prediction results of the LSTM model are used as one of the input features of XGBoost. That is, the LSTM+Attention model provides a time series prediction of historical bandwidth, and XGBoost further optimizes the prediction using RSSI, RSRP, RSRQ, and SINR network status features. Ultimately, the output of LSTM+Attention and other network status features form a high-dimensional feature space matrix, which is used as the input of the XGBoost model. For each time step, XGBoost will process the following features: [LSTM prediction output, RSSI, RSRP, RSRQ, SINR].

[0076] 4) XGBoost Model: Use the XGBoost algorithm to model the concatenated high-dimensional feature space matrix. XGBoost gradually captures the complex nonlinear relationships between features by constructing multiple decision trees. During the splitting process of each tree, XGBoost automatically evaluates the importance of each feature and selects the optimal split point, thereby achieving effective feature screening and modeling.

[0077] The XGBoost model is trained using a standard regression loss function to minimize the prediction error. Each new tree is trained and optimized based on the residual of the previous tree using the gradient boosting method.

[0078] In this embodiment, Figure 3 This is the XGBoost model construction and training diagram. Based on the LSTM+Attention model, the XGBoost model is further used to optimize the bandwidth prediction results, especially to model the complex nonlinear relationship between multi-dimensional features. The process is as follows:

[0079] Step 4.1) XGBoost uses decision trees to model the features of the input data. In each iteration, XGBoost selects the best features and split points by splitting nodes, gradually building trees. Each tree is modeled on the residuals of the previous tree. XGBoost automatically evaluates which features are most important for bandwidth prediction and builds a prediction model based on these features.

[0080] Step 4.2) Training of the XGBoost model: Use the standard regression loss function to minimize the prediction error, and use the XGBoost tree model to capture the complex nonlinear relationship between features. In this embodiment, XGBoost uses an additive model to iteratively generate trees. After each tree is generated, the residuals of the previously generated tree are fitted; the nodes of each tree are split according to the different values ​​of the features, with the goal of minimizing the mean square error after the split; in this embodiment, regularization is used to control the complexity of the model to prevent overfitting; XGBoost uses L1 and L2 regularization to control the flexibility of the model by penalizing the complexity of the tree; in this embodiment, the L1 regularization strength alpha = 0.5, and the L2 regularization strength lambda = 1.0.

[0081] Step 4.3) XGBoost model optimization: Through the gradient boosting method, each new tree is trained and optimized based on the residual of the previous tree; the hyperparameters of the model are adjusted through cross-validation. The hyperparameters include tree depth, learning rate, and subsampling ratio. Each tree reduces the error by adjusting the prediction of the previous tree, thereby improving the accuracy of the model; in this embodiment, the learning rate learning_rate = 0.05, the tree depth max_depth = 6, and the subsampling ratio subsample = 0.8.

[0082] Step 4.4) Use the mean square error (MSE), root mean square error (RMSE), and mean absolute error (MAE) regression evaluation indicators to evaluate the model. The formula is:

[0083]

[0084] Where n is the number of samples, y i is the true value of the i-th sample, is the predicted value of the i-th sample.

[0085] 5) Output results: The prediction results of all XGBoost trees will be weighted and summed to obtain the final bandwidth prediction value.

[0086] In this embodiment, the bandwidth prediction value can be used for adaptive bit rate control of VR video stream scenarios, helping the video stream to dynamically adjust under different network environments.

[0087] The above embodiments are merely exemplary embodiments of the present invention and are not intended to limit the scope of the present invention. The scope of protection of the present invention is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present invention within the spirit and scope of protection of the present invention, and such modifications or equivalent substitutions shall also be deemed to fall within the scope of protection of the present invention.

[0088] The embodiments of this specification are merely examples of implementations of the invention and are provided for illustrative purposes only. The scope of protection of the present invention should not be considered limited to the specific embodiments described in these embodiments. The scope of protection of the present invention also extends to equivalent technical means that can be conceived by a person of ordinary skill in the art based on the invention.

Claims

1. A bandwidth prediction method integrating LSTM and XGBoost, characterized in that the method comprises the following steps: 1) Data Collection: Collect bandwidth-related timing characteristic data, including: Historical bandwidth: records the user's bandwidth usage in the past time window; User location information: Obtain user location data through GPS to capture the impact of user movement on bandwidth requirements; Delay data: records network transmission delays and evaluates network performance; Network load: Statistics on the current network traffic load and analysis of network congestion; Clean, standardize, and normalize time series feature data to ensure consistent scales for different features and facilitate input into the model. Divide the data into training and test sets for model training and performance evaluation. 2) LSTM Model and Attention Mechanism: The LSTM model is used to process time series feature data and capture dynamic changes in historical bandwidth. The attention mechanism is introduced into the LSTM model. By calculating the weights of time steps, the LSTM + Attention model can focus on the time steps that have the greatest impact on prediction and ignore irrelevant information. 3) Feature Fusion: The output of the LSTM+Attention model, namely the time series prediction value weighted by the attention mechanism, is used as one of the features. Other network status features, including RSSI, RSRP, RSRQ, and SINR, are also introduced. These features are combined with the output of the LSTM+Attention model to form a new feature matrix, forming a high-dimensional feature space matrix. 4) XGBoost Model: Use the XGBoost algorithm to model the concatenated high-dimensional feature space matrix. XGBoost gradually captures the complex nonlinear relationships between features by constructing multiple decision trees. During each tree split, XGBoost automatically evaluates the importance of each feature and selects the optimal split point, thereby achieving effective feature screening and modeling. The XGBoost model is trained using a standard regression loss function to minimize the prediction error. Each new tree is trained and optimized based on the residual of the previous tree using the gradient boosting method. 5) Output: The prediction results of all XGBoost trees are weighted and summed to obtain the final bandwidth prediction value; The process of 1) is as follows: Step 1.1) Use six devices: three mobile devices to simulate slow user movement. The positions and states of these devices are adjusted according to the experimental design to simulate the bandwidth requirements of users in different network environments; and three head-mounted displays (HMDs) to simulate high-performance devices, ensuring device diversity to ensure experimental accuracy. Step 1.2) Each device connects to the server via WiFi to transmit VR video data. Historical bandwidth, latency, and real-time network load data are collected via Android APIs at a 1-second granularity to facilitate bandwidth prediction model training. Each device's user location information is collected via GPS signals to help model user mobility behavior. Step 1.3) Data cleaning: 1.3.1) Detect and remove missing values ​​or invalid data points; 1.3.2) Handle outliers and filter out extreme values ​​by setting thresholds; 1.3.3) Interpolate incomplete time series data to ensure data continuity; Step 1.4) Standardize and normalize all features as follows: 1.4.1) Standardization: Convert the data into a distribution with a mean of 0 and a standard deviation of 1. The formula is: ; in, is the original data, is the mean, is the standard deviation; 1.4.2) Normalization: Scale the data to the interval [0, 1], the formula is: ; in, and are the minimum and maximum values ​​of the data respectively; Step 1.5) Data Partitioning: Divide the data into training and test sets for model training and performance evaluation. The steps are as follows: 1.5.1) Divide the data into chronological order, using 80% of the data as the training set and 20% of the data as the test set; 1.5.2) Ensure the temporal continuity of the training and test sets to avoid data leakage.

2. The bandwidth prediction method of LSTM and XGBoost fusion according to claim 1, characterized in that: In 2), the LSTM network structure construction and training process are as follows: Step 2.1) Input layer: Input features include historical bandwidth, user location information, latency data, and network load data. The input for each time step is a vector consisting of multiple features. Step 2.2) LSTM layer: This layer consists of num_layers LSTM layers, which are used to learn long-term and short-term dependencies in time series data. The number of neurons in each layer is set. Step 2.3) Attention Mechanism: After the output layer of the LSTM, add an additive attention mechanism that assigns a weight to the LSTM output of each time step, focusing on those moments that have a greater impact on bandwidth prediction; Step 2.4) Output layer: The output of the LSTM and attention mechanism is passed to a fully connected layer to predict the bandwidth value at the current moment; Step 2.5) LSTM model training: The model is trained using the mean squared error (MSE) as the loss function, which is defined as: ； in, is the sample size, is the true bandwidth value of the i-th sample, is the bandwidth value predicted by the model for the i-th sample; Use the gradient descent method to update the model parameters to ensure that the model can gradually reduce the prediction error. The Adam optimizer update rule is: ； Where η is the learning rate; is the first moment estimate of the gradient, ; is the second moment estimate of the gradient, ; ϵ is a very small constant 1e-8, used to prevent division by zero stability constant; is the gradient at the current moment, is the decay rate of the first-order moment, is the square of the current gradient, is the decay rate of the second-order moment; Step 2.6) Attention Mechanism Model Training: Using the Additive Attention method, the model's focus is adjusted by calculating the importance weight of each time step, automatically selecting important time segments for more refined learning. The output of each moment is weighted summed to obtain the final bandwidth prediction result. Step 2.7) During training, the LSTM+Attention model uses the training data for forward and backward propagation, continuously adjusting the LSTM weights via gradient descent to minimize the loss function until the model converges. During training, early stopping is used to avoid overfitting. Training is terminated when the validation set loss does not improve for a set number of consecutive epochs.

3. The bandwidth prediction method of LSTM and XGBoost fusion according to claim 1, characterized in that: In 4), the complex nonlinear relationship between multi-dimensional features is modeled as follows: Step 4.1) XGBoost uses decision trees to model the features of the input data. In each iteration, XGBoost selects the best features and split points by splitting nodes, gradually building trees. Each tree is modeled on the residuals of the previous tree. XGBoost automatically evaluates which features are most important for bandwidth prediction and builds a prediction model based on these features. Step 4.2) XGBoost model training: Use a standard regression loss function to minimize prediction error. XGBoost uses a tree model to capture complex nonlinear relationships between features. XGBoost uses an additive model to iteratively generate trees. After each generated tree, the residuals of the previous tree are fitted. Each tree node is split based on the value of the feature, with the goal of minimizing the mean squared error after the split. Regularization is used to control model complexity and prevent overfitting. XGBoost uses L1 and L2 regularization to control model flexibility by penalizing tree complexity. Step 4.3) XGBoost model optimization: Using gradient boosting, each new tree is trained and optimized based on the residuals of the previous tree. Cross-validation is used to adjust the model's hyperparameters, including tree depth, learning rate, and subsampling ratio. Each tree reduces the error by adjusting the predictions of the previous tree, thereby improving the model's accuracy. Step 4.4) Use the mean square error (MSE), root mean square error (RMSE), and mean absolute error (MAE) regression evaluation indicators to evaluate the model. The formula is: ， ， ； in, is the sample size, is the true value of the i-th sample, is the predicted value of the i-th sample.

Images