Secure wireless traffic prediction method using federated learning

Abstract

Wireless traffic prediction is crucial for improving the performance of an intelligent cellular network, and influences the optimal configuration of network resources, the service quality and the satisfaction degree of final users. The traditional method is limited in precision when processing dynamic flow data, and has the problems of privacy leakage and response delay. Therefore, the invention provides a wireless traffic prediction method which is combined with the federated learning and the Kolmogorov-ArnoldNetworks (KAN). The method comprises the following steps: firstly, acquiring differential flow data of different geographic positions and preprocessing the differential flow data; then, edge computing and federated learning techniques are employed to allow a plurality of edge devices to locally train the LSTM model. On the basis, the KAN is combined to deeply mine the internal complexity of the flow data, and the adaptability of the model is enhanced. Meanwhile, a FedNova model updating mechanism is introduced, and data heterogeneity between devices is considered. Compared with the evaluation of the traditional method, the method provided by the invention is proved to have superiority in the aspects of prediction precision, calculation efficiency and data privacy protection.

Description

Technical Field

[0001] The present invention relates to a secure wireless traffic prediction method using federated learning, which belongs to a data prediction technology combining federated learning and deep learning technology in the field of network traffic analysis. Technical Background

[0002] As we move towards 6G networks, the importance of mobile traffic prediction is becoming increasingly prominent. With the improvement of smart device performance and users' continuous pursuit of service quality, the demand for network management and control is also growing rapidly. Future 6G networks will integrate AI technology to support innovative applications such as autonomous driving, augmented reality (AR) and virtual reality (VR). These applications have extremely high requirements for network reliability and responsiveness, and accurate traffic prediction will be the key to meeting these requirements. Currently, traffic prediction usually adopts a centralized learning method, which requires a large amount of raw data to be transferred to the data center to train a unified prediction model. However, frequent data transmission and signal overhead may quickly consume network resources and affect the effective transmission of data. Therefore, there is an urgent need to develop new traffic prediction methods to address these challenges.

[0003] Federated learning provides an innovative solution to this challenge. Under the framework of federated learning, multiple clients, such as mobile devices, base stations, and enterprises, can jointly participate in model training under the control of a central server. Unlike traditional methods, federated learning only requires the client to upload the model parameters or gradients obtained from local training to the server without transmitting the original data. In the field of next-generation communication technology, the application of federated learning has shown obvious advantages. The rapid development of edge computing has provided strong support for federated learning. As the computing power of edge devices increases, they reduce their dependence on central data centers and effectively reduce the delay of data transmission.

[0004] The present invention uses the advantages of federated learning to propose a secure wireless traffic prediction framework. In order to more accurately capture and predict the traffic characteristics of different base stations, the invention cleverly combines the LSTM model and KAN technology. This combination not only enhances the model's sensitivity to traffic changes, but also provides strong support for processing heterogeneous data. Based on this foundation, the invention proposes the FedNova-LSTM-KAN model, introduces the FedNova framework, and fully utilizes its advantages in a distributed learning environment. Through a unique normalized averaging strategy, the heterogeneity problem of different devices in optimization objectives is successfully solved, and the generalization ability and convergence speed of the model are significantly improved. Using the FedNova-LSTM-KAN model for global traffic prediction can not only improve the accuracy and reliability of the prediction, but also facilitate data sharing and model training across multiple participants while protecting user privacy and data security. The development and application of this technology marks an important step forward in the intelligence and automation of the telecommunications industry. With the continuous improvement and advancement of technology, we look forward to seeing more accurate and efficient SMS traffic prediction solutions in the near future. Summary of the invention

[0005] In order to improve the accuracy of global wireless traffic prediction, the present invention proposes a secure wireless traffic prediction method using federated learning, the method comprising the following steps:

[0006] Step 1, obtain and filter three sets of geographically dispersed historical traffic data;

[0007] Step 2: preprocess the historical data and divide it into training set and test set;

[0008] Step 3: On the edge server, use the training set to perform local training on the LSTM-KAN model.

[0009] Step 4: The central server coordinates the local training results of each mobile edge computing server, aggregates parameters, generates a universal traffic prediction model FedNova-LSTM-KAN, and saves the optimal model;

[0010] Step 5: Evaluate and compare this method with the long short-term memory network traffic prediction method.

[0011] Furthermore, in step 1, three groups of geographically dispersed historical traffic data are obtained and screened, and the specific steps include:

[0012] Traffic data of the selected SMS type is aggregated into hourly data at 10-minute intervals, with timestamp parameters added and information such as call traffic, network traffic, country code, and weekdays removed.

[0013] Furthermore, in step 2, the historical data is preprocessed and divided into a training set and a test set. The specific implementation process includes:

[0014] (1) Fill missing values ​​with a specified constant value and optionally specify an identifier to identify missing values.

[0015] (2) Perform outlier processing on the data in each region. The outlier processing threshold range is (10, 90). The outliers that exceed the threshold are corrected according to the specified threshold range. If the outlier value is greater than the upper limit of the threshold, it is set to 90.

[0016] The outliers are less than the lower limit of the threshold, which is set to 10. The corrected data will replace the outliers in the original data.

[0017] (3) Extract feature variables and target variables, and scale the feature variables to ensure that they have similar scales or ranges. This can improve the performance and convergence speed of the machine learning model and ensure that the weights of different features are reasonably balanced during model training. Scale the target variable to make it suitable for model training and prediction.

[0018] The selected three groups of traffic data set samples from base stations in different areas of the city and geographically dispersed are divided into training sets and test sets, where the training set is the first 80% of the data and the test set is the remaining 20% ​​of the data.

[0019] Furthermore, in step 3, the LSTM-KAN model is locally trained using the training set on the edge server. The specific implementation process includes:

[0020] (1) Initialize the LSTM-KAN model and set the initial parameters.

[0021] (2) The input data and target labels are loaded into the model, and the output is calculated by forward propagation. The implementation formula of forward propagation is:

[0022] Forgetgate:f t =σ(W f [h (t-1) , x t ]+b f )

[0023] Inputgate:i t =σ(W i [h (t-1) , x t ]+b i )

[0024] g t =tanh(W g [h(t-1) , x t ]+b g )

[0025] Cellstate: c t =f t ×c (t-1) +i t ×g t

[0026] Outputgate: o t =σ(W o [h (t-1) , x t ]+b o )

[0027] h t =o t ×tanh(c t )

[0028]

[0029] Φ={φ q,p}

[0030]

[0031] At time step t, the activation value of the input gate is given by i t Represents. The weight matrix W i The input x t Mapped to the input gate. The hidden state h at the previous time step t-1 The weight matrix to the input gate is U i The bias term of the input gate is b i The activation function is denoted by σ. The activation value of the forget gate at time step t is determined by f t W f and U f The input x is t and the previous hidden state h t-1 To the weight matrix of the forget gate. The bias term of the forget gate is b f . is the candidate state of the cell, b C is the bias term of the candidate cell state. C t is the cell state. t is the hidden state. l It is KAN's l Layer function matrix, Φ is the function matrix, q is the input dimension, p is the output dimension, is a linear combination parameterized as a B-spline, b(h t ) represents the activation function.

[0032] (3) Calculate the loss value and use it as the basis for back propagation. The loss formula is:

[0033]

[0034] Where M is the number of samples, N is the number of categories, and y ij One-hot encoding of the target value, Represents input x t The probability of belonging to class j.

[0035] (4) Call the optimizer's backward method to calculate the gradient, perform iterative optimization according to the specified number of training rounds, and call the step method to gradually update the model parameters;

[0036] (5) Repeat steps 2 to 4 above until the predetermined number of training iterations is reached.

[0037] Furthermore, in step 4, the central server coordinates the local training results of each mobile edge computing server and aggregates the parameters to generate a general traffic prediction model FedNova-LSTM-KAN, and saves the optimal model including:

[0038] The FedNova algorithm used in this invention applies stochastic gradient descent and its variant optimization algorithms to federated learning. By normalizing the average strategy, it solves the "target inconsistency" problem caused by differences in client data distribution and computing power. This means that the global model no longer converges to a point that does not match the true objective function, but is closer to the true target. The specific implementation steps and formulas are:

[0039] (1) The client initializes the local model parameters θ k,0 is the global model parameter θ0;

[0040] (2) Initialize the client's first-order momentum m k,0 and the second-order momentum v k,0 is the zero vector;

[0041] (3) For client k, perform the following steps E times (E is the number of local training rounds);

[0042] (4) Use the local data set to train the model and calculate the gradient g k,e ;

[0043] (5) Update the first-order momentum: m k,e =β1×m k,e-1 +(1-β1)×g k,e , where β1 is the first-order momentum decay factor;

[0044] (6) Update the second-order momentum: v k,e =vk,e-1 +g k,e ⊙g k,e , where ⊙ represents element-wise multiplication;

[0045] (7) Update local model parameters: Where η is the learning rate and τ is the numerical stability constant;

[0046] (8) Client k calculates the local model update Δ k,E =θ k,E -θ k,0 And upload to the server;

[0047] (9) The server receives model updates from all clients Δ 1,E ,…,Δ k,E ;

[0048] (10) Calculate the weighted average update: where ω k is the weight of client k, which is usually proportional to the amount of data of the client;

[0049] (11) Update global model parameters: θ new =θ0+Δ E ;

[0050] (12) Send the updated global model parameters back to all clients and repeat steps 3-11 until the preset stop condition is met.

[0051] Furthermore, in step 5, the evaluation and comparison of FedNova-LSTM-KAN and the long short-term memory network traffic prediction method includes:

[0052] The test set results predicted by the method of the present invention are compared with the results of the long short-term memory network model, and two evaluation indicators, mean absolute error (MAE) and root mean square error (RMSE), are used to measure the accuracy of the model prediction performance. MAE provides a quantitative measure of the prediction error by calculating the average of the absolute values ​​of the difference between the predicted value and the actual value. RMSE calculates the average of the sum of squares of the prediction errors and then takes the square root to obtain the Euclidean distance of the error. Specifically, the indicators considered are defined as follows:

[0053]

[0054] where y i is the actual value of the ith SMS traffic, is the predicted value of the ith SMS traffic, and n is the length of the data contained in the test set. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1Global traffic prediction framework based on federated learning

[0056] Figure 2 Traffic prediction architecture diagram based on LSTM-KAN model

[0057] Figure 3 Federated learning workflow

[0058] Figure 4 Comparison of predicted and true values ​​of the secure wireless traffic prediction method based on federated learning DETAILED DESCRIPTION

[0059] In order to improve the accuracy of network traffic and the security of traffic management, the present invention proposes a secure wireless traffic prediction method using federated learning. The present invention first preprocesses the client data and divides it into three data sets. The server initializes the global model and distributes the initial parameters to the client. The client independently trains the LSTM-KAN method and uploads the updated parameters to the server. The server uses the FedNova algorithm to integrate the parameters, update the global model, and then send the new parameters to the client. After multiple rounds of iterations, when the model performance reaches the predetermined standard, the client uses a local test data set to evaluate the prediction accuracy of the global model. In order to better understand the method of the present invention, the specific implementation method of the method is further described in conjunction with the accompanying drawings. For details of the traffic prediction framework, see Figure 1 .

[0060] Step 1: Obtain and filter three sets of geographically dispersed historical traffic data from the Telecom Italia Big Data Challenge website. Select SMS traffic data, aggregate it into hourly data at 10-minute intervals, add timestamp parameters, and remove information such as call traffic, network traffic, country code, and weekdays.

[0061] Step 2: In the preprocessing stage of historical traffic data, the data set is first divided into a training set and a test set, of which the first 80% is used for training and the last 20% is used for testing. Next, a specified constant value is used to fill the missing values ​​in the data, and an identifier can be selected to identify these missing values. Then, the data of each area is processed for outliers, and the threshold range is set to (10,90). Outliers exceeding this range will be corrected: values ​​greater than 90 are set to 90, and values ​​less than 10 are set to 10. The corrected values ​​will replace the outliers in the original data. Finally, feature variables and target variables are extracted and scaled to ensure that they have similar scales or ranges to improve the performance and convergence speed of the machine learning model and ensure a reasonable balance of different feature weights during model training. The target variable also needs to be scaled to accommodate model training and prediction.

[0062] Step 3: Use the LSTM-KAN method for training on the client. The structure of the LSTM-KAN method is as follows: Figure 2 shown.

[0063] The specific implementation process of LSTM-KAN method training is as follows:

[0064] (1) Initialize the LSTM-KAN method and set the initial parameters.

[0065] (2) The input data and target labels are loaded into the model, and the output is calculated by forward propagation. The implementation formula of forward propagation is:

[0066] Forgetgate:f t =σ(W f [h (t-1) , x t ]+b f )

[0067] Inputgate:i t =σ(W i [h (t-1) , x t ]+b i )

[0068] g t =tanh(W g [h (t-1) , x t ]+b g )

[0069] Cellstate: c t =f t ×c (t-1) +i t ×g t

[0070] Outputgate: o t =σ(W o [h (t-1) , x t ]+b o )

[0071] h t =o t ×tanh(c t )

[0072]

[0073] Φ={φ q,p}

[0074]

[0075] At time step t, the activation value of the input gate is given by i t Represents. The weight matrix W i The input x t Mapped to the input gate. The hidden state h at the previous time step t-1 The weight matrix to the input gate is U i The bias term of the input gate is b i The activation function is denoted by σ. The activation value of the forget gate at time step t is determined by f t W f and U f The input x is t and the previous hidden state h t-1 To the weight matrix of the forget gate. The bias term of the forget gate is b f . is the candidate state of the cell, b C is the bias term of the candidate cell state. C t is the cell state. t is the hidden state. l It is KAN's l Layer function matrix, Φ is the function matrix, q is the input dimension, p is the output dimension, is a linear combination parameterized as a B-spline, b(h t ) represents the activation function.

[0076] (3) Calculate the loss value and use it as the basis for back propagation. The loss formula is:

[0077]

[0078] Where M is the number of samples, N is the number of categories, and y ij One-hot encoding of the target value, Represents input x t The probability of belonging to class j.

[0079] (4) Call the optimizer's backward method to calculate the gradient, perform iterative optimization according to the specified number of training rounds, and call the step method to gradually update the model parameters;

[0080] (5) Repeat steps 2 to 4 above until the predetermined number of training iterations is reached.

[0081] Step 4: The central server coordinates the local training results of each mobile edge computing server and aggregates the parameters to generate a universal traffic prediction model FedNova-LSTM-KAN. The workflow of this federated learning is as follows: Figure 3 shown.

[0082] The FedNova algorithm used in this invention applies stochastic gradient descent and its variant optimization algorithms to federated learning. By normalizing the average strategy, it solves the "target inconsistency" problem caused by differences in client data distribution and computing power. This means that the global model no longer converges to a point that does not match the true objective function, but is closer to the true target. The specific implementation steps and formulas are:

[0083] (1) The client initializes the local model parameters θ k,0 is the global model parameter θ0;

[0084] (2) Initialize the client's first-order momentum m k,0 and the second-order momentum v k,0 is the zero vector;

[0085] (3) For client k, perform the following steps E times (E is the number of local training rounds);

[0086] (4) Use the local data set to train the model and calculate the gradient g k,e ;

[0087] (5) Update the first-order momentum: m k,e =β1×m k,e-1 +(1-β1)×g k,e , where β1 is the first-order momentum decay factor;

[0088] (6) Update the second-order momentum: v k,e =v k,e-1 +g k,e ⊙g k,e , where ⊙ represents element-wise multiplication;

[0089] (7) Update local model parameters: Where η is the learning rate and τ is the numerical stability constant;

[0090] (8) Client k calculates the local model update Δ k,E =θ k,E -θ k,0 And upload to the server;

[0091] (9) The server receives model updates from all clients Δ 1,E ,…,Δ k,E ;

[0092] (10) Calculate the weighted average update: where ω k is the weight of client k, which is usually proportional to the amount of data of the client;

[0093] (11) Update global model parameters: θ new =θ0+Δ E;

[0094] (12) Send the updated global model parameters back to all clients and repeat steps 3-11 until the preset stop condition is met.

[0095] Step 5: Compare the predicted results of the method of the present invention with the real data, such as Figure 4 As shown. Two evaluation indicators, mean absolute error (MAE) and root mean square error (RMSE), are used to measure the accuracy of the model's prediction performance. Specifically, the indicators considered are defined as follows:

[0096]

[0097] where y i is the actual value of the ith SMS traffic, is the predicted value of the ith SMS traffic, and n is the length of the data contained in the test set.

[0098] Table 1: Shows the MAE and RMSE evaluation indicators on the training data set (MAE and RMSE are in Mbps)

[0099]

Claims

1. A secure wireless traffic prediction method using federated learning, the method comprising the following steps: Step 1, obtain and filter three sets of geographically dispersed historical traffic data; Step 2: preprocess the historical data and divide it into training set and test set; Step 3: On the edge server, use the training set to perform local training on the LSTM-KAN model. Step 4: The central server coordinates the local training results of each mobile edge computing server, aggregates parameters, generates a universal traffic prediction model FedNova-LSTM-KAN, and saves the optimal model; Step 5: Evaluate and compare this method with the long short-term memory network traffic prediction method. In step 1, three groups of geographically dispersed historical traffic data are obtained and filtered. The specific steps include: Traffic data of the selected SMS type is aggregated into hourly data at 10-minute intervals, with timestamp parameters added and information such as call traffic, network traffic, country code, and weekdays removed. In step 2, the historical data is preprocessed and divided into a training set and a test set. The specific implementation process includes: (1) Fill missing values ​​with a specified constant value and optionally specify an identifier to identify missing values. (2) Perform outlier processing on the data in each region. The outlier processing threshold range is (10, 90). The outliers that exceed the threshold are corrected according to the specified threshold range. Outliers greater than the upper threshold are set to 90. Outliers less than the lower threshold are set to 10. The corrected data replaces the outliers in the original data. (3) Extract feature variables and target variables, and scale the feature variables to ensure that they have similar scales or ranges. This can improve the performance and convergence speed of the machine learning model and ensure that the weights of different features are reasonably balanced during model training. Scale the target variable to make it suitable for model training and prediction. The selected three groups of traffic data set samples from base stations in different areas of the city and geographically dispersed are divided into training sets and test sets, where the training set is the first 80% of the data and the test set is the remaining 20% ​​of the data. In step 3, on the edge server, the LSTM-KAN model is trained locally using the training set. The specific implementation process includes: (1) Initialize the LSTM-KAN model and set the initial parameters. (2) Load the input data and target label into the model, and calculate the output by forward propagation. The implementation formula of propagation is: Forget gate:f t =σ(W f [h (t-1) ,x t ]+b f ) Input gate:i t =σ(W i [h (t-1) ,x t ]+b i ) g t =tanh(W g [h (t-1) ,x t ]+b g ) Cell state:c t =f t ×c (t-1) +i t ×g t Output gate:o t =σ(W o [h (t-1) ,x t ]+b o ) h t =o t ×tanh(c t ) Kolmogorov-Arnold Networks: Φ={φ q,p } At time step t, the activation value of the input gate is given by i t Represents. The weight matrix W i The input x t Mapped to the input gate. The hidden state h at the previous time step t-1 The weight matrix to the input gate is U i The bias term of the input gate is b i The activation function is denoted by σ. The activation value of the forget gate at time step t is given by f t W f and U f The input x is t and the previous hidden state h t-1 To the weight matrix of the forget gate. The bias term of the forget gate is b f . is the candidate state of the cell, b C is the bias term of the candidate cell state. C t is the cell state. t is the hidden state. l It is KAN's l Layer function matrix, Φ is the function matrix, q is the input dimension, p is the output dimension, is a linear combination parameterized as a B-spline, b(h t ) represents the activation function. (3) Calculate the loss value and use it as the basis for back propagation. The loss formula is: Where M is the number of samples, N is the number of categories, and y ij One-hot encoding of the target value, Represents input x t The probability of belonging to class j. (4) Call the optimizer's backward method to calculate the gradient, perform iterative optimization according to the specified number of training rounds, and call the step method to gradually update the model parameters; (5) Repeat steps 2 to 4 above until the predetermined number of training iterations is reached. In step 4, the central server coordinates the local training results of each mobile edge computing server and aggregates the parameters to generate a general traffic prediction model FedNova-LSTM-KAN, and saves the optimal model including: The FedNova algorithm used in this invention applies stochastic gradient descent and its variant optimization algorithms to federated learning. By normalizing the average strategy, it solves the "target inconsistency" problem caused by differences in client data distribution and computing power. This means that the global model no longer converges to a point that does not match the true objective function, but is closer to the true target. The specific implementation steps and formulas are: (1) The client initializes the local model parameters θ k,0 is the global model parameter θ0; (2) Initialize the client's first-order momentum m k,0 and the second-order momentum v k,0 is the zero vector; (3) For client k, perform the following steps E times (E is the number of local training rounds); (4) Use the local data set to train the model and calculate the gradient g k,e ; (5) Update the first-order momentum: m k,e =β1×m k,e-1 +(1-β1)×g k,e , where β1 is the first-order momentum decay factor; (6) Update the second-order momentum: v k,e =v k,e-1 +g k,e ⊙g k,e , where ⊙ represents element-wise multiplication; (7) Update local model parameters: Where η is the learning rate and τ is the numerical stability constant; (8) Client k calculates the local model update Δ k,E =θ k,E -θ k,0 And upload to the server; (9) The server receives model updates from all clients Δ 1,E ,…,Δ k,E ; (10) Calculate the weighted average update: where ω k is the weight of client k, which is usually proportional to the amount of data of the client; (11) Update global model parameters: θ new =θ0+Δ E ; (12) Send the updated global model parameters back to all clients and repeat steps 3-11 until the preset stop condition is met. In step 5, the evaluation and comparison of FedNova-LSTM-KAN and the long short-term memory network traffic prediction method include: The test set results predicted by the method of the present invention are compared with the results of the long short-term memory network model, and two evaluation indicators, mean absolute error (MAE) and root mean square error (RMSE), are used to measure the accuracy of the model prediction performance. MAE provides a quantitative measure of the prediction error by calculating the average of the absolute values ​​of the difference between the predicted value and the actual value. RMSE calculates the average of the sum of squares of the prediction errors and then takes the square root to obtain the Euclidean distance of the error. Specifically, the indicators considered are defined as follows: where y i is the actual value of the ith SMS traffic, is the predicted value of the ith SMS traffic, and n is the length of the data contained in the test set.

Images