A wireless traffic prediction method based on dynamic gradient sparsification federated learning

Abstract

The application relates to a wireless traffic prediction method based on dynamic gradient sparsification federated learning; a training process comprises the following steps: a central server broadcasts a global model to each local base station device participating in local training; the local base station device trains and updates the received global model on a local data set; then, gradient information screened out according to a dynamic gradient sparsification rule is uploaded to the central server, and the rest of the gradient information is discarded on the spot and does not perform a communication task; after receiving the gradient information uploaded by the local base station device, the central server performs federated aggregation to generate a new global model; the obtained new global model is broadcast to the local base station device again; a prediction process comprises the following steps: wireless traffic prediction is realized through the new global model. Before the central server and the local base station device communicate, the dynamic gradient sparsification strategy is taken into account, so that the high communication cost phenomenon caused by frequent communication can be reduced.

Description

Technical Field

[0001] The present invention relates to the fields of wireless communications and artificial intelligence, and in particular to a wireless traffic prediction method based on dynamic sparse federated learning. Background Art

[0002] Wireless traffic forecasting is a critical task in wireless communication networks to accurately predict future traffic flows. Accurate traffic forecasting can help network operators optimize network resource allocation, improve network performance, and meet user needs.

[0003] However, traditional wireless traffic prediction methods have several limitations that restrict their effectiveness and scalability in practical applications. Traditional wireless traffic prediction methods typically require centralized data collection, which faces challenges with uneven data distribution and large-scale data processing. Centralized methods pose privacy and security risks and are not effective for large-scale networks. Non-IID data is difficult to process, and frequent model updates are also a challenge.

[0004] To overcome these issues, federated learning, a specialized distributed machine learning approach, has been applied to wireless traffic prediction. It allows participants to train models locally while protecting data privacy, and enables collaborative learning and parameter updates through global model aggregation. This approach improves prediction accuracy and scalability while protecting data privacy. It also significantly reduces communication overhead compared to centralized approaches.

[0005] However, federated learning still requires a high communication frequency in wireless traffic prediction, so reducing communication overhead and improving communication efficiency remain key tasks. Therefore, there is an urgent need to develop a wireless network prediction method that balances prediction performance with reduced communication overhead. Summary of the Invention

[0006] In view of the shortcomings of the existing technology, the present invention provides a wireless traffic prediction method based on dynamic gradient sparsification federated learning;

[0007] This invention aims to provide a solution that accurately predicts wireless traffic while protecting user privacy and reducing communication overhead. It addresses the challenges of traditional traffic prediction methods in protecting user privacy and reducing data transmission volume. It can be directly applied to network resource allocation and optimizing network decisions.

[0008] The embodiment of the application obtains the feasibility of realizing wireless traffic prediction under the federated learning framework through analysis of the challenges faced by wireless traffic service and the advantages of the federated learning framework. On the basis of the conclusion that most of the information transmission is redundant in each communication process between the central server and the local base station device under the federated learning framework, the application proposes a dynamic gradient sparsification strategy. The strategy not only aims to improve the prediction performance, but also reduces the communication volume between the central server and the local base station device each time, thereby achieving the purpose of reducing communication overhead.

[0009] Term explanation:

[0010] 1. Stochastic gradient descent is the most commonly used optimization algorithm for solving machine learning algorithms. For the objective function to be optimized, the stochastic gradient descent algorithm samples part of the data sample through a loop, calculates the gradient information of the objective function, and uses the gradient information to update the model parameters. After reaching the maximum number of loops, the loop ends, and the stochastic gradient descent returns the last model parameter information.

[0011] 2. Adaptive moment estimation is based on the stochastic gradient descent algorithm and introduces high-order gradient information and momentum terms. Adaptive moment estimation uses first-order and second-order gradient information to dynamically adjust the learning rate of each parameter. The advantage of adaptive moment estimation is that after bias correction of the gradient, the learning rate of each iteration has a certain range, making the parameters more stable.

[0012] 3. Momentum method is an extension of the traditional gradient descent algorithm, which is more efficient than the stochastic gradient descent algorithm. Momentum method, also known as momentum-based gradient descent method, is a method that accelerates the change of gradient vector in the relevant direction and ultimately achieves accelerated convergence. Momentum method is a very popular optimization algorithm and is used in the optimization of many deep learning models.

[0013] The technical solution of the application is:

[0014] A wireless traffic prediction method based on dynamic gradient sparsification federated learning, comprising:

[0015] The training process comprises:

[0016] The central server broadcasts the global model to each local base station device participating in local training;

[0017] The local base station device trains and updates the received global model on the local data set; then, the gradient information selected according to the dynamic gradient sparsification rule is uploaded to the central server, and the remaining gradient information is discarded on the spot without performing the communication task;

[0018] The central server performs federated aggregation to generate a new global model after receiving the gradient information uploaded by the local base station device, and broadcasts the new global model to the local base station device again.

[0019] The training process is repeatedly performed until a training end condition is met.

[0020] The prediction process includes:

[0021] The wireless traffic prediction is realized through the new global model.

[0022] According to the application, the wireless traffic prediction method is preferably run on a wireless traffic prediction system, which includes a plurality of local base station devices, i.e., base stations, and a central server.

[0023] According to the application, the global model is preferably a linear regression model or a neural network model.

[0024] According to the application, the local base station device trains the received global model on the local data set, which includes:

[0025] The local data set preprocessing stage: data cleaning is performed to remove invalid or missing data, and abnormal values or noise data are processed; then, data standardization or normalization processing is performed.

[0026] The preprocessed local data set is divided into a training data set and a test data set.

[0027] Both the training data set and the test data set are set with a sliding window.

[0028] According to the application, the local base station device updates the received global model on the local data set, which includes:

[0029] A. Determine the optimization method.

[0030] B. Select the batch size and gradient calculation algorithm according to the training set and test set size and hardware device conditions.

[0031] C. Update the global model through the calculated gradient information.

[0032] D. According to the training settings, repeat steps B and C until the maximum training loop number is reached and the relevant parameters are updated.

[0033] According to the application, the gradient information is screened according to a dynamic gradient sparsification rule, as shown in formula (1):

[0034]

[0035] In formula (1), represents the gradient information screened out after gradient sparsification; g(·) is a generalization of the gradient sparsification strategy; and represents the gradient sparsification ratio; the condition met by is: 0 < p < 1, which is adjusted according to the training round.

[0036] According to the application, preferably, the central server generates a new global model through federated aggregation after receiving the gradient information uploaded by the local base station device; as shown below:

[0037]

[0038] wherein, w represents the global model, and represents the learning rate of the optimization algorithm.

[0039] According to the application, preferably, the wireless traffic prediction is realized through the new global model; including:

[0040] The local base station device uses the final prediction model, i.e., the new global model, to perform wireless traffic prediction on the test data set.

[0041] A computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps of a wireless traffic prediction method based on dynamic gradient sparsification federated learning when executing the computer program.

[0042] A computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of a wireless traffic prediction method based on dynamic gradient sparsification federated learning.

[0043] The application has the following beneficial effects:

[0044] The embodiment of the application realizes wireless traffic prediction on a federated learning framework and proposes a dynamic gradient sparsification federated learning algorithm. After the central server receives the gradient information uploaded by the local base station device, the central server generates a new global model through federated aggregation. Compared with the traditional federated learning, the method of the application example takes into account the dynamic gradient sparsification strategy before the central server and the local base station device communicate, which can reduce the high communication overhead phenomenon caused by frequent communication. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is a schematic diagram of the wireless traffic prediction system of the application;

[0046] Figure 2 is a schematic diagram of the training process of the prediction model of the application;

[0047] Figure 3 is a schematic diagram of the prediction performance comparison of the algorithm proposed in the embodiment of the application on two data sets and the traditional algorithm;

[0048] Figure 4 is a comparison between the predicted value of an embodiment of the present application and the real value of a certain local base station device and an error analysis schematic diagram;

[0049] Figure 5 is a communication overhead analysis schematic diagram of an embodiment of the present application. DETAILED DESCRIPTION

[0050] The present application will be further limited by the accompanying drawings and examples, but not limited thereto.

[0051] Embodiment 1

[0052] A wireless traffic prediction method based on dynamic gradient sparsification federated learning, comprising:

[0053] The training process comprises:

[0054] The central server broadcasts the global model to each local base station device participating in local training;

[0055] The local base station device trains and updates the received global model on the local data set; then, according to the dynamic gradient sparsification rule, a small part of important gradient information is uploaded to the central server, and the remaining gradient information is discarded in place and not executed by the communication task;

[0056] After the central server receives the gradient information uploaded by the local base station device, it performs federated aggregation to generate a new global model; the new global model obtained is broadcast to the local base station device again;

[0057] Repeat the training process until the training end condition is met;

[0058] The prediction process comprises:

[0059] The new global model is used to realize wireless traffic prediction.

[0060] Compared with the traditional federated learning, the method of the present application counts the dynamic gradient sparsification strategy before the central server and the local base station device communicate, which can reduce the high communication overhead phenomenon caused by frequent communication and improve the overall communication efficiency of wireless traffic prediction work.

[0061] Embodiment 2

[0062] The wireless traffic prediction method based on dynamic gradient sparsification federated learning according to Embodiment 1 is different in that:

[0063] The wireless traffic prediction method runs in a wireless traffic prediction system; for example Figure 1As shown in Figure 1, the wireless traffic prediction system includes multiple (K) local base station devices, i.e., base stations, and a central server. At the same time, each base station has q wireless traffic data and a local prediction model w from the central server. k .

[0064] A wireless traffic prediction method based on dynamic gradient sparsification federated learning is used to solve the problems of excessive communication overhead and inability to protect data privacy in traditional centralized methods. This method achieves wireless traffic prediction tasks by combining the ideas of gradient sparsification and federated learning. Figure 1 The system model is presented. Figure 2 The core flow chart of the system is shown. It includes K base stations, K = 100. It is shown as follows:

[0065] (1) Initialize all local base station devices, assuming that there are K local base station devices. The initialization process includes: numbering the local base station devices as 1; ...; k; ...; K; numbering all local data sets as d k ;…;d k ;…;d K ;

[0066] (2) The central server initializes a random neural network model w for training, namely the global model w, and broadcasts it to the learning unit of each local base station device;

[0067] (3) After receiving the global model w from the central server, the local base station device k is renumbered as w k , and use the local dataset to train the local model w k Train and update the local model w k Parameters, after t local training rounds, the new local model w is obtained k ;

[0068] (4) Each local base station device filters the updated local model according to a specific gradient sparsification strategy, and finally sends the compressed gradient information of the local model to the central server;

[0069] (5) After receiving the gradient information from the local base station device, the central server obtains a new global model w through the federated aggregation method and broadcasts the obtained new global model to the local base station device;

[0070] (6) Repeat steps (3) to (5) until the training end condition is met;

[0071] When the central server broadcasts the global model to the local base station device for the first time, the local base station device needs to determine the prediction model type and initialize the local dataset after receiving the global model. The specific steps include:

[0072] A, for determining the prediction model type: since the wireless traffic data has strong time dependence, this patent selects to use long short-term memory network that can handle long-term dependence; since the edge base station device recollects a large amount of data, in order to reduce the computing performance threshold of the edge device, this patent selects to use a lightweight long short-term memory network with only two layers; since the wireless traffic data has long-term trend and periodicity, the model input includes two parts, which are used to represent trend characteristics and periodic characteristics respectively;

[0073] B, for the initialization of the local data set stage, first data cleaning, removing invalid or missing data, processing outliers or noise data. After that, the data is standardized;

[0074] C, for the test set and the training interface, this patent selects to set the last week as the test set, and the remaining data as the training set. This patent adopts the mechanism of sliding window, the sliding window size is set to p = 3, the sliding step size is set to z period = 24, z close = 1. The training data set has 1104 entries, and the test data set has a total of 168 entries.

[0075] The global model is a linear regression model or a neural network model. For example: linear regression method, multilayer perception, long short-term neural network model, etc.

[0076] The local base station device trains the received global model on the local data set; including:

[0077] The local data set preprocessing stage: data cleaning, removing invalid or missing data, processing outliers or noise data; then, standardizing or normalizing the data;

[0078] For the preprocessed local data set, divide it into a training data set and a test data set according to a certain proportion;

[0079] Both the training data set and the test data set are set with a sliding window. The sliding window size is set to p, and the sliding step is set to z.

[0080] The local base station device updates the received global model on the local data set; including:

[0081] A, determine the optimization method; including but not limited to stochastic gradient descent, adaptive matrix estimation, momentum method;

[0082] B. Selecting appropriate batch size and gradient calculation algorithm according to the size of training set and test set and the condition of hardware device; the actual training is carried out at the base station side, and the training can determine the batch size (the batch size represents how many samples are sampled for calculation at one time, and the more the number of samples in the batch, the higher the requirement for computing resources) and the gradient calculation method (the traditional stochastic gradient descent only needs to calculate the gradient information, the calculation amount is small, and the complexity is low; the advanced gradient calculation method has large calculation amount and high complexity) according to the size of the training set and the test set (if the number of training and test samples is small, the samples can be put into the video memory at one time, then a large batch size is selected to save processing time; on the contrary, a small batch size must be selected to make the training successfully) and the current load of the base station (idle state, few users are served, more computing resources can be used, and the CPU temperature is low; busy state, many users are served, few computing resources can be used, and the CPU temperature is high).

[0083] C. Updating the global model by the calculated gradient information;

[0084] The gradient information is the gradient value obtained by deriving the model parameters of the target function, which can be expressed as where f represents the loss function corresponding to the model, w represents the model parameters, and represents the derivation operation of the target function f on w.

[0085] D. According to the training setting, repeating steps B and C until the maximum training cycle number is reached and the related parameters are updated.

[0086] According to the dynamic gradient sparsification rule, a small part of important gradient information is screened out, and the required uploaded local gradient information is calculated according to formula (1) and the other redundant information is discarded:

[0087]

[0088] In formula (1), represents the gradient information screened out after gradient sparsification; g(·) is a generalization of the gradient sparsification strategy; g(·) represents a strategy for sparsifying the gradient vector; in actual execution, a gradient vector is given, and its value is sorted; when a value of p is given, such as p = 0.2, the largest p n values and the corresponding positions are selected, and the gradient values at other positions are filtered out and only the p n gradient values are retained. p represents the gradient sparsification ratio; p satisfies the condition: 0 ≤ p ≤ 1, and p is not constant, but is adjusted according to the training round.

[0089] After receiving the gradient information uploaded by the local base station device, the central server performs federal aggregation to generate a new global model; as follows:

[0090]

[0091] wherein w represents a global model, and represents a learning rate of an optimization algorithm.

[0092] Wireless traffic prediction is realized through a new global model, which comprises:

[0093] The local base station device uses the final prediction model, i.e., the new global model, to perform wireless traffic prediction on a test data set.

[0094] By comparing the predicted value with the true value, relevant evaluation indexes are calculated to evaluate the prediction performance.

[0095] The total compression rate of all local base station devices is calculated to evaluate the total communication overhead.

[0096] After the evaluation is completed, the local base station device continues to collect local data to enrich the database.

[0097] With the wireless traffic prediction method based on the dynamic gradient sparsification federated learning, the following changes may occur over time: the traffic pattern near the base station may change; the wireless traffic data collected by the local base station continuously accumulates; and the working state of the local base station device may change. This may cause the performance of the original prediction model to continuously decrease, and the training overhead of the prediction model to continuously increase. To solve this problem, the wireless traffic prediction method based on the dynamic gradient sparsification federated learning further comprises the following steps:

[0098] A new prediction model is periodically trained according to the working state of the base station. When training the prediction model each time, the size of the data set is fixed, and the data set is updated in real time. According to the working state of the local base station device, it is determined whether to participate in the training task of the prediction model. If the number of base stations participating in the training does not meet the set threshold value at a certain time of updating, the updating is skipped and the request for the next updating is waited.

[0099] After the training target is achieved, the performance of the prediction model is evaluated, which specifically comprises:

[0100] A. The local base station device uses the final prediction model to perform wireless traffic prediction on a test data set.

[0101] B. By comparing the predicted value with the true value, relevant evaluation indexes are calculated to evaluate the prediction performance. Since it is time series data, the accuracy cannot be compared, so the average absolute error and the mean square error are used to evaluate the prediction performance of the model.

[0102] C. The total compression rate of all local base station devices is calculated to evaluate the total communication overhead.

[0103] D. After the evaluation is completed, the local base station device continues to collect the local data enrichment database.

[0104] The result of the wireless service traffic prediction based on the dynamic gradient sparsification federated learning proposed in the embodiment is as shown in Figure 3 、 Figure 4 、 Figure 5 Figure 3 The schematic diagram of the prediction performance comparison of the algorithm proposed in the embodiment and the traditional algorithm on two data sets is shown; the two data sets are the Milan data set and the Tronto data set. The traditional federated representation is an algorithm without gradient information compression; the traditional centralized representation adopts the same prediction algorithm as the federated representation, but has a different training strategy. For a prediction task, there is a data set and a prediction model. If the model is directly trained on the data set, this category is centralized learning; if the data set is distributed on K local devices instead of existing in a data center, and the K devices train the model in a collaborative manner, it is traditional federated; if the data is distributed on K local devices, and during collaborative training, each local device first compresses the gradient information and then sends it to the server for aggregation, it is the algorithm proposed in the embodiment.

[0105] Figure 4 The schematic diagram of the prediction value of a certain local base station device of the embodiment of the application and the true value and error analysis is shown; the more consistent the prediction value and the true value are, the more accurate the prediction is. From Figure 4 it can be seen that the algorithm proposed in the embodiment is more consistent with the true value. Figure 4 The cumulative probability distribution (CDR) of the prediction absolute error is shown below. If the CDR curve of the prediction absolute error quickly reaches the probability 1, it means that the prediction is good. From Figure 4 it can also be concluded that the algorithm proposed in the embodiment is better than the traditional federated and centralized.

[0106] From Figure 3 and Figure 4 it can be observed that the use of the dynamic gradient sparsification strategy improves the prediction model performance. Therefore, the wireless service traffic prediction scheme proposed in the application can effectively improve the prediction performance.

[0107] Figure 5 The schematic diagram of the communication overhead analysis of the embodiment of the application is shown. Since the traditional federated algorithm transmits all the gradient information, its communication overhead is 100%; and the gradient information to be transmitted by the local base station of the algorithm proposed in the embodiment is greatly compressed, and the communication overhead is only 1.929% of the traditional algorithm. From Figure 5 the results, it can be concluded that after using the dynamic gradient sparsification strategy, the communication burden can be greatly reduced, and the communication efficiency can be improved.

[0108] Embodiment 3​

[0109] A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the steps of the wireless traffic prediction method based on dynamic gradient sparsification federated learning of embodiment 1 or 2 when executing the computer program.

[0110] Embodiment 4

[0111] A computer readable storage medium having stored thereon a computer program, the computer program implementing the steps of the wireless traffic prediction method based on dynamic gradient sparsification federated learning of embodiment 1 or 2 when executed by a processor.

Claims

1. A wireless traffic prediction method based on dynamic gradient sparsification federated learning, characterized in that: include: The training process includes: The central server broadcasts the global model to each local base station device participating in local training; The local base station device trains and updates the received global model on the local dataset. Afterwards, the gradient information filtered out according to the dynamic gradient sparsification rule is uploaded to the central server, and the remaining gradient information is discarded on the spot and no longer performs the communication task. After receiving the gradient information uploaded by the local base station device, the central server performs federation aggregation to generate a new global model; the obtained new global model is broadcasted to the local base station device again; Repeat the training process until the training end condition is met; The forecasting process includes: Wireless traffic prediction is achieved through a new global model.

2. A wireless traffic prediction method based on dynamic gradient sparsification federated learning according to claim 1, characterized in that: The wireless traffic prediction method is run on a wireless traffic prediction system; the wireless traffic prediction system includes a plurality of local base station devices, namely base stations, and a central server.

3. The wireless traffic prediction method based on dynamic gradient sparsification federated learning according to claim 1 is characterized in that: The global model is a linear regression model or a neural network model.

4. The wireless traffic prediction method based on dynamic gradient sparsification federated learning according to claim 1 is characterized in that: The local base station device trains the received global model on the local dataset; including: Local dataset preprocessing stage: data cleaning, removal of invalid or missing data, processing of outliers or noise data; then, standardization or normalization of the data; For the pre-processed local dataset, divide it into training dataset and test dataset; A sliding window setting is performed on both the training dataset and the test dataset.

5. The wireless traffic prediction method based on dynamic gradient sparsification federated learning according to claim 1, characterized in that: The local base station device updates the received global model on the local data set; including: A. Determine the optimization method; B. Select the batch size and gradient calculation algorithm based on the size of the training and test sets and the hardware conditions; C. Update the global model using the calculated gradient information; D. Repeat steps B and C according to the training settings until the maximum number of training cycles is reached and the relevant parameters are updated.

6. The wireless traffic prediction method based on dynamic gradient sparsification federated learning according to claim 1 is characterized in that: Gradient information is filtered out according to the dynamic gradient sparsification rule, as shown in formula (1): In formula (1), represents the gradient information filtered after gradient sparsification; g(·) is a generalized representation of the gradient sparsification strategy; ρ represents the gradient sparsification ratio; ρ satisfies the condition: 0≤ρ≤1 and is adjusted according to the training round.

7. The wireless traffic prediction method based on dynamic gradient sparsification federated learning according to claim 1 is characterized in that: After receiving the gradient information uploaded by the local base station device, the central server performs federation aggregation to generate a new global model; as shown below: Here, w represents the global model and ∈ represents the learning rate of the optimization algorithm.

8. A wireless traffic prediction method based on dynamic gradient sparsification federated learning according to any one of claims 1 to 7, characterized in that: Wireless traffic prediction is achieved through a new global model; including: The local base station device uses the final prediction model, i.e., the new global model, on the test dataset to perform wireless traffic prediction.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of any one of claims 1-8 of the wireless traffic prediction method based on dynamic gradient sparsification federated learning are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of any one of claims 1-8 of the wireless traffic prediction method based on dynamic gradient sparsification federated learning are implemented.

Images