Federated learning network flow prediction method based on multi-scale time feature enhancement, storage medium and equipment

Abstract

The invention discloses a federated learning network traffic prediction method based on multi-scale time feature enhancement, a storage medium and equipment, and belongs to the technical field of network traffic prediction. The problem that an existing traffic prediction network is large in prediction error is solved. According to the invention, prediction is carried out based on a federated learning-based multi-scale time network traffic prediction framework; during prediction of a local client, on the basis of an iTransform model, a traffic data sequence is integrally coded into a token, meanwhile, time features and comprehensive influence factors corresponding to different time granularities are independently coded into respective tokens, and then a traffic prediction value of a future time period is obtained; in the process of global training based on federated learning, local clients are responsible for training a local model by using local data, a plurality of local clients train a local model respectively in a round of training, and then respective local model parameters are transmitted to a central server for aggregation to obtain a global model.

Description

Technical Field

[0001] The present invention relates to a federated learning network traffic prediction method, storage medium and device based on multi-scale time feature enhancement. Background Art

[0002] With the iterative upgrades of mobile communication technologies and the widespread penetration of smart devices, global network traffic is experiencing exponential growth, leading to multi-dimensional challenges in wireless networks in terms of data transmission scale, service quality assurance, and energy efficiency management. In this context, network traffic prediction has become a core technology for achieving dynamic resource scheduling, ensuring user experience, and building green and sustainable communication networks. Effective traffic prediction is crucial for optimizing network resource allocation, improving user experience, reducing energy consumption, and ultimately achieving intelligent and efficient network management. Therefore, building an efficient and accurate network traffic prediction model has far-reaching practical and strategic value for network operators and service providers.

[0003] Network traffic forecasting is essentially a time series forecasting problem. Early research primarily employed statistical methods, such as the Autoregressive Integrated Moving Average (ARIMA) method. However, with the significant increase in network architecture complexity, traffic data exhibits complex characteristics such as nonlinearity, long-range correlation, time-varying behavior, self-similarity, and burstiness. Traditional methods have gradually exposed limitations in terms of feature capture and prediction accuracy.

[0004] With the rapid development of artificial intelligence (AI) technology, network traffic prediction methods have undergone a paradigm shift from traditional models to deep learning. Earlier research primarily relied on shallow models such as linear regression and support vector regression (SVR) for traffic modeling. In recent years, deep learning, with its powerful feature extraction capabilities, has driven breakthrough progress in this field: Zhang et al. proposed an end-to-end online prediction framework (LNTP) based on LSTM and developed a sliding window gradient descent (SWGD) algorithm to optimize network weights; Ramakrishnan et al. systematically compared recursive network structures such as RNN, LSTM, and GRU, verifying the effectiveness of deep models in traffic prediction; and enhanced models such as DCRNN further improved prediction accuracy by improving nonlinear feature extraction capabilities. While these deep learning models have made significant progress in centralized training scenarios, existing research still faces three key limitations: centralized training models are expensive to transmit data and pose privacy risks; existing models tend to focus on the autocorrelation characteristics of historical data, lacking attention to periodic temporal features such as hour of day, day of week, month of month, and year of year; and they generally ignore the persistent disruptions to traffic patterns caused by holiday events. Consequently, network traffic forecasts can suffer from significant errors, especially during holidays. Summary of the Invention

[0005] The present invention aims to solve the problem of large prediction errors in existing traffic prediction networks.

[0006] A federated learning network traffic prediction method based on multi-scale temporal feature enhancement includes the following steps:

[0007] Obtain network traffic data and record it as the original data set; map the discrete time information corresponding to the network traffic data into continuous value features. In this process, the corresponding periodicity is obtained according to different time granularities to obtain time features; regard the influencing factor of each festival as an independent feature dimension, and regard the distance d between the i-th day and the j-th festival as the distance ... ij Converted into impact factor value X through kernel function ij ; For a single time point t i , will affect the factor value X ij Perform weighted summation to obtain a single numerical comprehensive impact factor F i ';

[0008] A multi-scale temporal network traffic prediction framework based on federated learning is used for prediction; the multi-scale temporal network traffic prediction framework based on federated learning includes a prediction step of a local client and a step of global training based on federated learning, taking each local client as a participant;

[0009] The prediction steps of the local client include: based on the iTransformer network model, encoding the entire traffic data sequence into a token, and independently encoding the time characteristics and comprehensive influencing factors corresponding to different time granularities into their own tokens, thereby obtaining the traffic forecast value for the future period;

[0010] Taking each local client as a participant, the steps of global training based on federated learning include: Federated learning includes the training process of local clients and central servers, wherein the local client is responsible for training a local model using local data, and multiple local clients each train a local model in a round of training, and then transmit their local model parameters to the central server. The central server is responsible for aggregating the local models uploaded by each local client to obtain a global model. In the next round of training, the central server distributes the global model to each local client. Each local client uses the obtained global model as a basis and performs a new round of training using its own local data to obtain an updated local model. The new local model is then uploaded to the central server and trained in a cycle until the specified training round is reached, and finally a final global model is obtained.

[0011] Furthermore, the process of mapping discrete time information into continuous value features includes:

[0012] For a given time point t, its feature construction follows a general paradigm: first extract the original sequence number g(t) of its time dimension, then divide it by the period length N of this dimension, and finally constrain the value range to the symmetric interval [-0.5, 0.5] through a translation operation, denoted as f(t):

[0013]

[0014] Among them, g(t) is the original serial number of time in the dimension of time granularity, which is adjusted according to the time granularity; N is the period length corresponding to different time granularities.

[0015] Furthermore, in the process of obtaining the time features according to the periodicities corresponding to different time granularities, the time features are obtained according to the periodicities corresponding to "hour", "week", "month" and "year".

[0016] Furthermore, the original serial numbers of the time in the dimension of time granularity of "hour", "week", "month", and "year" and the corresponding period lengths of different time granularities are as follows:

[0017] For the hour feature, g(t) corresponds to the hour number within a day, starting with 0, and its cycle length N is set to 23 hours; for the week feature, g(t) represents the date number within the week, starting with 0, and its cycle length N is set to 6 days; for the month feature, g(t) corresponds to the date number within a month, and the maximum number of days in the month is dynamically selected as its cycle length; in the construction of the year feature, g(t) is the cumulative date number within the year, starting with 0, and its cycle length N is set to 364.

[0018] Furthermore, the impact factor of each festival is regarded as an independent feature dimension, and the distance d between the i-th day and the j-th festival is ij Converted into impact factor value X through kernel function ij The process is as follows:

[0019] The time information in the original data set is listed separately and converted into a standard timestamp format; let T = {t1, t2, ..., t m} represents the set of all time points in the time series dataset; for each time point t i ∈T, calculate the distance d between it and each important festival j ij ; Then, the distance d ij Input kernel function K(d ij ;α)=exp(-α|d ij |), and get the corresponding impact factor value X ij ; where α is a parameter that controls the speed of influence decay.

[0020] Furthermore, the comprehensive impact factor Among them, w j is the weight corresponding to the j-th holiday.

[0021] Furthermore, the local client is responsible for training a local model using local data. The local client i calculates the gradient g corresponding to the small batch sample based on the local data. i (y i ), and perform gradient correction while controlling the variable c i Updates.

[0022] Furthermore, the central server is responsible for aggregating the local models uploaded by each local client, and the central server updates the weighted average aggregation parameters. At the same time, the global control variables are adjusted according to the updates provided by each local client; where ΔW global It is the global model obtained by aggregating multiple local client models in federated learning, ΔW k The meaning is the local model on the kth local client, and there are n local clients in total.

[0023] A computer storage medium, characterized in that: the storage medium stores a computer program, and the computer program is loaded and executed by a processor to implement the federated learning network traffic prediction method based on multi-scale time feature enhancement.

[0024] A federated learning network traffic prediction device based on multi-scale temporal feature enhancement, the device comprising a processor and a memory, wherein the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the federated learning network traffic prediction method based on multi-scale temporal feature enhancement.

[0025] Beneficial Effects: This invention not only integrates periodic time features such as "hour," "week," "month," and "year," but also innovatively incorporates the persistent influence of holiday events. Furthermore, during the model aggregation phase of federated learning, a stochastic controlled averaging algorithm is introduced to dynamically correct the update direction of local models, effectively improving the convergence speed and generalization capability of the federated model. This invention achieves improved accuracy in network traffic prediction while ensuring data privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is the autocorrelation coefficient of network traffic in the typical functional area of ​​Milan (cell number is 4456).

[0027] Figure 2 is the autocorrelation coefficient of network traffic in the typical functional area of ​​Milan (cell number is 4529).

[0028] Figure 3 is the autocorrelation coefficient of network traffic in typical functional areas of Milan (cell number is 5060).

[0029] Figure 4 This is a line graph analyzing the network traffic pattern of a typical functional area in Milan (cell number is 4456) over a “weekly” period.

[0030] Figure 5 This is a line graph analyzing the network traffic pattern of a typical functional area in Milan (cell number 4529) over a “weekly” period.

[0031] Figure 6 This is a line graph analyzing the network traffic pattern of a typical functional area in Milan (cell number is 5060) over a “weekly” period.

[0032] Figure 7 This is a comparison chart of network traffic changes and change rates from December 16 to December 27 compared with the previous week.

[0033] Figure 8 This is a comparison chart of network traffic changes and change rates from December 16 to December 27 with the previous month.

[0034] Figure 9 This is the overall framework diagram of traffic prediction.

[0035] Figure 10 This is the model structure diagram of iTransformer.

[0036] Figure 11 Schematic diagram of client drift phenomenon in federated learning.

[0037] Figure 12 A diagram showing the steps for updating SCAFFOLD on an edge client.

[0038] Figure 13 This is a comparison chart of the predicted value, actual value and error analysis of Traffic value in a certain area of ​​Milan.

[0039] Figure 14 This is a CDF comparison chart of the predicted value, actual value and error analysis for a certain area in Milan.

[0040] Figure 15 This is a comparison chart of the predicted value, actual value and error analysis of Traffic value in a certain area of ​​Trento.

[0041] Figure 16 This is a CDF comparison chart of the predicted value, actual value and error analysis for a certain area in Trento. DETAILED DESCRIPTION

[0042] To address the challenges presented in the prior art, this paper proposes a Federated Learning Framework for Network Traffic Prediction with Multi-Scale Temporal Awareness (FedMTA) framework. Compared to traditional centralized approaches, this framework utilizes federated learning, a distributed collaborative training mechanism. Specifically, federated learning provides a distributed training architecture that can be applied in conjunction with many machine learning algorithms (especially deep neural networks). This algorithm effectively learns from local data, and a global model is obtained by iteratively aggregating local training models. Client data information can also be shared while protecting the privacy of training client data, resulting in more accurate prediction results. During the model training and prediction phases, not only are periodic temporal features such as "hour," "week," "month," and "year" integrated, but the ongoing impact of holiday events is also innovatively incorporated. During the model aggregation phase of federated learning, a stochastic controlled averaging algorithm is introduced to dynamically correct the update direction of local models, effectively improving the convergence speed and generalization capability of the federated model. Through these innovations, the accuracy of network traffic prediction is improved while ensuring data privacy. This is explained below with reference to specific implementations. Specific implementation method one:

[0044] This embodiment is a federated learning network traffic prediction method based on multi-scale temporal feature enhancement, which includes the following steps:

[0045] S1. Network traffic analysis:

[0046] S101. Analysis of network traffic autocorrelation characteristics:

[0047] Changes in people's pace of life and daily activities significantly impact network traffic patterns. These changes not only reflect the dynamic adjustments in users' online behavior but also directly shape the distribution of peak and trough times for network traffic. Typically, network traffic is relatively high during the daytime on weekdays, as people are out working or engaging in various activities. Meanwhile, during the evening hours, data usage decreases significantly. Weekend network traffic patterns also differ from weekdays, reflecting people's more active online entertainment and social interactions during their off-hours. Therefore, these temporal factors not only increase the complexity of network traffic forecasting but also are crucial to achieving accurate forecasts.

[0048] This paper uses a public dataset released by the "Telecom Italia Big Data Challenge" project, referred to as the original dataset. This dataset records historical network traffic data for the Italian cities of Milan and Trento, with a temporal resolution of hours. Milan is divided into 10,000 grid cells, each containing complete time-series observations. To further explore the correlation between urban functional zoning and traffic patterns, three typical functional areas in Milan were selected for comparative analysis: Navigli Nightlife District (area code 4456), representing entertainment functions, accounts for 62% of the total daily traffic; Bocconi University (area code 4259), representing an educational district, exhibits periodic fluctuations during the semester and a single peak during the day; and Duomo Centro (area code 5060), a core business district, maintains a high traffic plateau for 12 hours between 9:00 AM and 9:00 PM.

[0049] By calculating the autocorrelation coefficient (ACF) of the time series, it is found that there are significant periodic patterns in each region, such as Figures 1 to 3 As shown, at 24-hour and 48-hour lags, the autocorrelation coefficients are very high—close to 1—and exceed the confidence intervals in the shaded area, confirming the statistical significance of the cyclical nature of the cyclical nature of the 24-hour and 48-hour lags. Notably, secondary peaks appear in the educational and commercial districts after a lag of 168 hours, or seven days, suggesting the presence of a weekly cyclical behavior pattern.

[0050] S102. Characteristic analysis of periodic time:

[0051] Through in-depth analysis of network traffic data, we found that time of day significantly influences network traffic fluctuation patterns, and that relying solely on historical traffic data to predict future trends is limited. For example, the Navigli region exhibits increased internet connection activity in the evenings, while Bocconi University's network connection decreases on weekends. These observations demonstrate that accurately predicting base station network traffic requires comprehensive consideration of multiple time-related factors.

[0052] like Figure 4-Figure 6 As shown, after segmenting the data from the three different areas into weekly segments, significant differences in network traffic data can be observed at different times of the week, particularly between weekdays (Monday to Friday) and weekends (Saturday and Sunday). In the Bocconi University area, weekend traffic drops significantly, likely due to reduced activity by students and staff. In contrast, weekend traffic increases significantly in the Navigli nightlife district and Duomo city center, reflecting these areas' role as entertainment and tourism centers, where people tend to go out for entertainment and sightseeing on weekends.

[0053] S103. Analysis of holiday event effects:

[0054] We further analyzed the non-periodic disturbance characteristics of network traffic caused by major holiday events. For example, during important holidays like Christmas, users tend to interact with friends and family online or offline, sending greetings and well wishes, which further affects network traffic during specific time periods.

[0055] like Figure 7-Figure 8 As shown, a comparison of network traffic changes in the Bocconi University area from December 16th to 27th reveals a significant delayed and persistent Christmas effect. Compared to the same period the previous week, traffic on December 25th, Christmas Day, decreased by 54.0%, and the drop widened to 55.4% the following day. Compared to the same period the previous month, the decline was even more dramatic, with traffic dropping by 60.1% and 62.9% on December 25th and 26th, respectively. Notably, the traffic trough did not occur on the holiday itself, but delayed until 24 hours afterward. This may be related to a phased shift in user activity during the holiday season: Christmas Day was dominated by offline gatherings, while online activity lagged behind the following day due to the extended holiday. In contrast, traffic fluctuations during the non-holiday period of December 16th to 21st were limited, with drops of less than 30% on these days and no significant persistent impact.

[0056] The above observations show that the festival effect not only directly suppresses instantaneous traffic, but also triggers continuous disturbances through chain reactions of social behavior. Such disturbances exceed the prediction boundaries of traditional periodic models and need to be quantified as festival impact factors through feature engineering and added to the model input to improve the accuracy and reliability of the prediction.

[0057] Based on the above analysis, network traffic forecasting relies not only on the basic characteristics of historical traffic data but also requires comprehensive consideration of temporal cyclical patterns and the persistent impact of holiday events on traffic. Specifically, the weekly cyclical pattern is one of the core features that captures the difference between weekday and weekend traffic and effectively reflects user behavior patterns on weekdays and weekends. However, major holidays such as Christmas, acting as non-cyclical disturbances, significantly alter network traffic patterns during the holiday period, with persistent effects in the days surrounding the holiday. This impact goes beyond the typical cyclical pattern, manifesting as a significant decrease in traffic or a delayed recovery. Therefore, when building forecasting models, it is important to consider quantifying the holiday effect as an independent feature variable and combining it with cyclical features to fully capture the complexity of traffic fluctuations during special periods. Incorporating these feature variables significantly improves the accuracy and reliability of model forecasts, allowing them to better adapt to the dynamic changes in network traffic.

[0058] S2. Build a multi-scale temporal network traffic prediction framework based on federated learning:

[0059] This prediction framework is built on the iTransformer model. It not only integrates periodic time information to capture the periodic changes in network traffic, but also pays special attention to the persistent non-periodic disturbances caused by festivals on network traffic. In the federated learning architecture, considering the heterogeneity of network traffic trends and correlations between different base stations, the SCAFFOLD algorithm is introduced to correct the "client drift" phenomenon in local updates. This ensures that the consistency and stability of model parameters can be maintained even in a distributed training environment, achieving better prediction performance. The overall framework is as follows: Figure 9 shown.

[0060] S3. Network traffic prediction:

[0061] S31. Calculation of multi-scale temporal features:

[0062] S311. Periodic time characteristic calculation:

[0063] Periodic time features can help the model capture repetitive patterns in network traffic data. By converting time information into specific periodic features, the model can learn these patterns more easily, thereby improving prediction performance. The present invention proposes a time feature representation method based on symmetric intervals, which maps discrete time information into continuous value features to enhance the model's sensitivity to periodic patterns. Specifically, for a given time point t, its feature construction follows a general paradigm: first, the original serial number g(t) within the dimension of its time granularity is extracted, and then it is divided by the period length N of this dimension, and finally the value range is constrained to a symmetric interval of [-0.5, 0.5] through a translation operation. This process can be formally expressed as:

[0064]

[0065] g(t) is dynamically adjusted based on the time granularity. For the hourly feature, g(t) corresponds to the hour number within a day, starting at 0, and its cycle length N is set to 23 hours to eliminate sudden changes in values ​​during the diurnal cycle. For the weekly feature, g(t) represents the day number within the week. By mapping it to a maximum cycle length of 6 days, the characteristic values ​​of Monday and Sunday are symmetrically distributed at both ends of the interval, thus maintaining the continuity of the weekly cycle boundary. When representing the "week" cycle, Monday is represented as 0, Tuesday as 1, and so on, with Sunday as 6, which is the maximum cycle length of the "week" cycle. For the monthly date feature, g(t) corresponds to the day number within a month, and its cycle length is dynamically selected from the maximum number of days in that month. In the construction of the yearly date feature, g(t) is expanded to the cumulative date number within the year. When representing the yearly date, it also starts at 0 and ranges from 0 to 364. The year-date feature is obtained by performing the same processing as the above method. This feature not only contains the composite information of month and day, but also enables the model to identify seasonal patterns across months, such as the cyclical pattern of the same quarter in different years.

[0066] This method achieves the collaborative expression of multi-granularity temporal features through a unified mathematical framework. The choice of symmetric intervals has dual advantages: on the one hand, zero-centering reduces the model's sensitivity to absolute time values ​​and avoids the gradient offset problem; on the other hand, the continuous and smooth mapping relationship enhances the differentiability of periodic boundaries, enabling neural networks based on sinusoidal activation functions to fit periodic functions more efficiently.

[0067] S312. Calculation of festival impact factors:

[0068] Holidays often cause non-periodic disturbances in network traffic, which cannot be fully described by traditional periodic features. Therefore, this paper proposes a method based on distance metrics and kernel functions to calculate the holiday impact factor and use it as an additional feature input for time series models.

[0069] The kernel function can effectively convert date distance into numerical impact factor, and as the distance increases, the impact factor gradually approaches zero. Assume d i Indicates the distance between a certain day and an important festival in days. We define a kernel function K(d i ; α), the formula of the kernel function is expressed as:

[0070] K(d i ;α)=exp(-α|d i |)

[0071] Among them, α is the parameter that controls the attenuation speed. i = 0, that is, the current date happens to be a holiday, and the impact factor reaches the maximum value; and when di When increases, the impact factor decreases rapidly, reflecting that the network data on dates far from holidays are less disturbed by the holidays.

[0072] In order to fully consider the impact of multiple important holidays, we regard the impact factor of each holiday as an independent feature dimension. For a time series dataset containing n important holidays, we can construct an m×n feature matrix X, where m represents the length of the time series and n represents the number of holidays. Each element in the matrix X ij The distance between the i-th day and the j-th festival is converted by the kernel function. Specifically, the present invention lists the time information in the original data set separately and converts it into a standard timestamp format. Let T = {t1, t2, ..., t m} represents the set of all time points in the time series dataset, T = {t1, t2, ..., t m} is a collection of time points in timestamp format, such as 2013 / 11 / 41:00:00. For each time point t i ∈T, calculate the distance d between it and each important festival j ij Then, the distance d ij Input kernel function K(d i ; α), get the corresponding impact factor value X ij .

[0073] In order to further improve the performance of the model, the present invention combines these features X ij Perform weighted summation. By weighted summation of multi-dimensional impact factors, a single numerical comprehensive impact factor F is synthesized. i ′, thus simplifying the subsequent analysis process. j is the weight corresponding to the jth holiday, then the comprehensive impact factor F i ′ can be expressed as:

[0074]

[0075] Among them, the weight w j It can be set manually. Comprehensive impact factor F i ′Represent all holidays to a single time point t i The impact of the impact is summarized into a single value, thus simplifying the subsequent analysis process.

[0076] Using this method, we can not only accurately quantify the impact of periodic and holiday events on network traffic data, but also incorporate these influencing factors as additional features into the model. Combined with the multi-attention module in the iTransformer model, we can more effectively capture the changing patterns of network traffic data and improve prediction accuracy.

[0077] S32. Prediction based on the prediction framework:

[0078] The Transformer architecture has been widely used in the field of time series forecasting due to its superior sequence modeling capabilities and high scalability. However, traditional multivariate time series modeling methods have significant limitations: they typically compress multidimensional variables at the same timestamp into a single embedding vector, resulting in loss of interactive information between variables and semantic confusion. To address this, iTransformer proposes an inverted variable sequence embedding strategy. Through this inverted embedding strategy, the complete time series of each variable is treated as an independent processing unit, achieving efficient modeling of multivariate time series.

[0079] iTransformer is an improved Transformer architecture designed specifically for multivariate time series forecasting. Its core concept is to restructure the model through "dimensional inversion." Without modifying native Transformer components, it significantly improves forecasting performance through an encoder-only architecture consisting of an embedding layer, a projector layer, and several Transformer modules (TrmBlocks). The following module-by-module detailed descriptions of its design principles and functionality.

[0080] (1) Embedding Layer

[0081] Function: Convert raw time series data into high-dimensional feature representation.

[0082] Design features:

[0083] Independent Variable Embedding: The entire time series of each variable is independently embedded as a "variable token," rather than combining multiple variables at the same time point into a "temporal token" as in traditional Transformers. In network traffic forecasting, traditional methods concatenate the traffic value and time characteristics at a specific moment into a single time token; iTransformer, on the other hand, encodes the entire traffic data sequence into a single token, while also independently encoding different time characteristics (including periodic time characteristics such as "hour," "week," "month," and "year," as well as comprehensive holiday influencing factors) into their own tokens.

[0084] Global receptive field: The time series of each variable is mapped into a high-dimensional vector through MLP (multi-layer perceptron) to capture its global temporal pattern.

[0085] No positional encoding: Since the temporal order is implicit in the arrangement of the sequence, iTransformer does not require explicit positional encoding.

[0086] (2) Multivariate Attention

[0087] Function: Model the correlation between variables.

[0088] Design features:

[0089] Cross-variable interaction: The attention mechanism acts on variable tokens to calculate the association weights of different variable tokens. In network traffic prediction, the correlation between network traffic usage and temporal features can be intuitively presented through attention maps.

[0090] (3) Feed-Forward Network (FFN)

[0091] Function: Encode nonlinear characteristics of time series and predict future values.

[0092] Design features:

[0093] Time dimension modeling: FFN processes the time dimension of each variable token independently and extracts temporal patterns (such as mutations and periodicity) through nonlinear transformations.

[0094] Universal representation capability: Based on the universal representation theorem of the multi-layer perceptron, the feedforward network acts on the entire sequence and can extract the intrinsic properties of the sequence.

[0095] Prediction output: Finally, the high-dimensional features are mapped to the predicted values ​​of the future time steps through the projection layer (MLP).

[0096] (4) Layer Normalization

[0097] Function: Stabilize the training process and reduce the distribution differences between variables.

[0098] Design features:

[0099] Variable centering normalization: Layer normalization acts on the variable tokens, making the characteristics of all variables under a relatively uniform distribution and reducing the differences in measurement units.

[0100] Alleviate non-stationarity: When dealing with non-stationary time series, normalized variable tokens are more likely to capture statistical regularities and avoid the over-smoothing problem caused by time step normalization in traditional methods.

[0101] The final prediction result is directly generated by the projection layer implemented by the fully connected network.

[0102] The prediction model iTransformer structure is as follows Figure 10As shown in , the model treats each complete sequence containing time features, comprehensive influencing factors or traffic data as an independent processing unit. Figure 10 As shown in (a), specifically, for a sequence of T steps containing time features, comprehensive influencing factors, or traffic data, the entire sequence of each variable is independently embedded as a "variable token" in the embedding layer, and a high-dimensional embedding representation is generated through a multi-layer perceptron (MLP). This inverted embedding strategy avoids the mixing of different variables in the time dimension and retains the independent semantics of each variable. Then, the multi-attention module is used to explicitly model cross-variable dependencies. The attention calculation follows the formula:

[0103]

[0104] Among them, the query matrix Q and the key value matrix K, V are generated by embedding the variable sequence through independent parameters. Figure 10 As shown in (b), the multi-attention mechanism can quantify the impact of temporal features on network traffic data and capture nonlinear correlations across variables.

[0105] Then through the feedforward network, such as Figure 10 As shown in (c), in the feedforward network, based on the universal representation theorem of the multi-layer perceptron, the temporal pattern is extracted. Finally, the layer normalization is performed independently on each variable sequence output by the feedforward network, as shown in Figure 10 As shown in (d), the characteristics of all variables are kept under a relatively uniform distribution, which reduces the differences in measurement units and avoids the non-stationary problem of time series. Finally, the traffic forecast value for the future period is generated.

[0106] In summary, among network traffic prediction methods based on multi-scale temporal information, iTransformer's inverted embedding strategy can effectively preserve the characteristics of network traffic sequence data. It leverages a multi-attention mechanism to comprehensively explore the relationship between network traffic data and temporal information, enabling the model to better understand the correlation and importance between different parts of the input sequence. By assigning higher weights to highly correlated variables, more accurate network traffic predictions are achieved. This design not only improves the model's ability to understand and represent input sequences, but also enhances its performance in capturing complex patterns in multivariate time series data, ultimately making network traffic predictions more accurate and reliable.

[0107] S33. Federal training process based on random control average:

[0108] Step S32 is the model processing process of the local client (participant), and S33 is the global training process of federated learning. The structure of federated learning is divided into two parts: the local client (participant) and the central server. The client is responsible for training a local model using local data. Multiple clients each train a local model in a round of training, and then transmit their local model parameters to the central server. The central server is responsible for aggregating the local models uploaded by each client to obtain a global model. In the next round of training, the central server distributes the global model to each client. Each client uses the obtained global model as a basis and uses its own local data for a new round of training to obtain an updated local model. The new local model is then uploaded to the central server and trained in a cycle until the specified training round is reached, and finally a final global model is obtained.

[0109] As an innovative distributed learning framework, federated learning is unique in that it allows multiple devices or servers to collaborate on training models while maintaining data privacy, effectively processing network traffic data scattered across different locations. Due to differences in the functional attributes of base station coverage areas (such as educational areas, commercial areas, and entertainment areas), different base stations may exhibit different network traffic patterns, resulting in highly heterogeneous traffic data, that is, network traffic data in different areas are non-independent and identically distributed (non-IID). Therefore, each base station is regarded as a participant in federated learning, that is, a client. Studies have shown that non-IID data distribution can cause client drift, that is, the direction of local model update deviates from the global optimal solution, such as Figure 11 As shown: The blue dot is the local update y i Move to single client optimal value The orange square and red dot represent server updates, which are moved to That is, the black square, not the truly optimal green square x * .

[0110] To this end, the present invention introduces the Stochastic Controlled Averaging (SCAFFOLD) algorithm. The complete training process is shown in Algorithm 1. Its key stages include:

[0111] Initialization phase: The server initializes the global model parameters w o Together with the global control variable c0, each client initializes the client control variable synchronously

[0112] Local training phase: Client i calculates the mini-batch gradient g based on local data i (y i ), and perform gradient correction and control variable c iUpdates. Figure 12 As shown, the dotted hollow circle representing the direction of the local model gradient update points to the orange square, but the correction term with the red arrow ensures that the update moves to the true optimal green square.

[0113] Global aggregation phase: The server updates the aggregation parameters by weighted average At the same time, global control variables are adjusted according to the updates provided by each client.

[0114] Where ΔW global It is the global model obtained by aggregating multiple local client models in federated learning, ΔW k It means the local model on the k-th client, and there are n local clients in total.

[0115] By applying the randomized controlled averaging algorithm, we performed gradient correction on the model of each edge client, effectively dealt with the problem of data heterogeneity, and solved the client drift phenomenon. This method significantly improved the convergence speed and stability of the model, while enhancing the generalization ability of the final global model. Specifically, the algorithm ensures model consistency under different data distributions by precisely adjusting the local models on each edge device, thereby preventing deviations during training. In addition, the dynamic adjustment of the gradient update strategy further optimizes the convergence process of the model, so that the global model can better adapt to diverse and complex practical application scenarios while maintaining efficiency and robustness. The process of SCAFFOLD is shown in Table 1

[0116] Table 1

[0117] Specific implementation method 2:

[0119] This embodiment is a computer storage medium, which stores a computer program. The computer program is loaded and executed by a processor to implement the federated learning network traffic prediction method based on multi-scale time feature enhancement.

[0120] It should be understood that the storage medium described in this embodiment includes but is not limited to magnetic storage media and optical storage media; the magnetic storage medium includes but is not limited to RAM, ROM, and other storage media such as hard disks and USB flash drives. Specific implementation method three:

[0122] This embodiment is a federated learning network traffic prediction device based on multi-scale time feature enhancement. The device includes a processor and a memory. The memory stores a computer program. The computer program is loaded and executed by the processor to implement the federated learning network traffic prediction method based on multi-scale time feature enhancement.

[0123] It should be understood that the devices described in this embodiment include, but are not limited to, devices including a processor and memory, and may also include devices corresponding to other units or modules with information collection, information exchange, and control functions. For example, the devices may also include signal acquisition devices. Such devices include, but are not limited to, PCs, workstations, mobile devices, etc.

[0124] Example

[0125] We randomly selected 100 regions from the datasets of the two cities. The first seven weeks of network traffic data served as the training set for training the prediction model, while the last week of data served as the test set for evaluating model performance. To ensure consistency of model input and processing efficiency, we preprocessed the dataset using min-max normalization. The mean squared error (MSE) was chosen as the model's loss function, which helps accurately quantify the prediction error and thus improve model performance and accuracy. The data was also converted to a sliding window format, generating experimental samples containing multidimensional arrays from continuous time series data. In each round of communication, 10% of the regions were randomly selected for model training. Stochastic gradient descent (SGD) was used as the optimizer to update the model parameters. The specific hyperparameter settings for the model are detailed in Table 2.

[0126] Table 2 Model parameter settings

[0127]

[0128] In order to evaluate the effectiveness of the proposed method, the following baseline models were selected for comparative experiments:

[0129] Lasso: A linear model used in regression analysis.

[0130] LSTM: LSTM (Long Short-Term Memory) is a common method used in deep learning for time series data forecasting. It has powerful modeling capabilities for time series datasets, often outperforming linear models and shallow machine learning models in forecasting performance. This network structure effectively captures complex patterns and long-term dependencies in time series, thereby providing more accurate forecasts.

[0131] MVSTGN: MVSTGN regards network traffic data as graph-structured data and uses a model based on the attention mechanism to capture the local and global spatial dependencies and long-term temporal dependencies of urban network traffic from local and global spatial perspectives as well as global temporal perspectives, thereby comprehensively mining spatiotemporal characteristics.

[0132] GCN-T: GCN-T is a spatiotemporal traffic prediction model based on graph convolutional networks and Transformer. It replaces the traditional adjacency matrix with geographic distance scalars, integrates the spatiotemporal features of geographic coordinates and traffic data, and uses the attention mechanism to capture the long-term dependencies and dynamic weight distribution of user behaviors, ultimately achieving multi-node collaborative prediction.

[0133] This paper uses Mean Square Error (MSE) and Mean Absolute Error (MAE) as indicators for evaluating model performance. By using these two evaluation indicators, we can comprehensively and accurately evaluate the prediction performance of the model. The calculation formula is as follows:

[0134]

[0135] The experimental results, shown in Table 3, show that the FedMTA framework performs well across all evaluation metrics on both the Milan and Trento datasets. In particular, FedMTA achieves the lowest MSE value of 0.1080 and MAE of 0.2250 on the Milan dataset, demonstrating its highest level of prediction accuracy on this dataset. Further observation of the results on the Trento dataset reveals that FedMTA not only maintains the best performance but also achieves improvements of approximately 4.14% and 1.82% in MSE and MAE, respectively, compared to the next-best performing method, MVSTGN. This improvement demonstrates FedMTA's significant advantage in improving network traffic prediction accuracy.

[0136] Table 3 Comparison of experimental results

[0137]

[0138] These results strongly demonstrate the effectiveness of FedMTA in improving the accuracy of network traffic prediction. Its success is mainly attributed to the following aspects: By adopting the iTransformer model within the federated learning framework, FedMTA can accurately quantify the impact of periodic and holiday events on network traffic data and integrate these influencing factors as additional features into the network traffic prediction model. Combined with the inverted architecture and multi-attention mechanism in the model, the complete time series of each variable is treated as an independent processing unit, thereby achieving efficient modeling of multivariate time series data, more effectively capturing the changing patterns of network traffic data, and improving prediction accuracy. By applying the stochastic controlled averaging algorithm to correct the gradient of the model of each edge client, the client drift problem is solved, and robust processing of data heterogeneity is achieved, which significantly improves the convergence speed and stability of the model, thereby enhancing the generalization ability of the final global model.

[0139] Ablation Experiments: To comprehensively evaluate the importance and effectiveness of the proposed periodic time features and holiday information in network traffic prediction, we conducted ablation experiments using two variants, FedMTA-HD and FedMTA-PT. The following is a detailed description of these two methods:

[0140] FedMTA-HD (Historical Data Only): A variant of the FedMTA prediction framework. This version of the FedMTA framework only uses historical data of network traffic for prediction and does not include additional time information.

[0141] FedMTA-PT (Periodic Time Features): A forecasting framework based on another variant of FedMTA. In this variant, FedMTA integrates periodic time features, such as hourly, weekly, monthly, and annual patterns, but does not take holiday impact factors into consideration.

[0142] The experimental results are shown in Table 3. A comparison of FedMTA-HD and FedMTA-PT on the Milan dataset clearly demonstrates the importance of periodic temporal features. Specifically, the MSE and MAE of FedMTA-HD are 0.1135 and 0.2327, respectively, while those of FedMTA-PT are reduced to 0.1120 and 0.2267, respectively. This means that after introducing periodic temporal features, the MSE is reduced by approximately 1.32% and the MAE is reduced by approximately 2.58%. Similarly, on the Trento dataset, the MSE and MAE of FedMTA-HD are 2.6688 and 0.7827, respectively, while those of FedMTA-PT are significantly improved to 2.2022 and 0.7357, representing a reduction of approximately 17.48% in MSE and 5.99% in MAE. These results demonstrate that introducing periodic temporal features can effectively capture the periodic fluctuations of network traffic, significantly improving prediction accuracy.

[0143] Table 3. Comparison of experimental results

[0144]

[0145] Further analysis revealed that while FedMTA-PT performs well in leveraging periodic temporal features, it still fails to account for the impact of special events such as holidays on network traffic. For example, in the Milan dataset, the final version of FedMTA reduced the mean square error (MSE) from 0.1120 to 0.1080, a decrease of approximately 3.57%, compared to FedMTA-PT. In the Trento dataset, the MSE decreased from 2.2022 to 2.1578, a decrease of approximately 2.02%, and the mean square error (MAE) decreased from 0.7357 to 0.7190, a decrease of approximately 2.27%. This demonstrates that, in addition to periodic temporal features, the impact of holiday events on network traffic is also a significant factor. During holiday periods, network traffic often exhibits significant fluctuations, which may not be accurately captured by periodic patterns alone. Therefore, combining periodic temporal features with the holiday effect can provide a more comprehensive understanding and prediction of network traffic trends, further improving the performance of the prediction model.

[0146] Predictive performance analysis: e.g. Figure 13-14 and Figure 15-16 As shown in the figure, we randomly selected one region each from the 100 regions used for training in the Milan and Trento datasets and visualized the trends in the true and predicted values ​​over the week-long test data, as well as the cumulative distribution function (CDF) of the absolute prediction error. By comparing the true values ​​(red line) with the predicted values ​​(black line), we can intuitively see that FedMTA accurately predicts peak traffic in both datasets and effectively captures the overall changing trends in network traffic over the entire week-long test data.

[0147] Specifically, in the Milan dataset, FedMTA not only successfully predicted multiple significant traffic peaks but also maintained high prediction accuracy during periods of low traffic. Similarly, in the Trento dataset, FedMTA demonstrated excellent prediction capabilities, particularly during periods of significant traffic fluctuations, with predicted values ​​highly consistent with true values. These results demonstrate that FedMTA excels in network traffic forecasting tasks, accurately predicting peak traffic and comprehensively capturing overall trends, demonstrating its strong potential and reliability in practical applications.

[0148] Through analysis Figure 14 and Figure 16The CDF graphs in Figure 1 provide a visual assessment of FedMTA's prediction accuracy on these two datasets. Specifically, in the Milan dataset, the CDF curve for the absolute prediction error shows that the prediction errors for most samples are concentrated at a low level, and the curve rises rapidly to near 1.0, demonstrating that FedMTA is able to accurately predict most traffic values. Similarly, the CDF curve for the Trento dataset exhibits similar characteristics: the prediction errors for most samples are low, and the curve quickly approaches 1.0, further validating FedMTA's high accuracy across different datasets.

[0149] It is worth noting that Figure 15-16 The points on the right with larger prediction errors reflect the complex dynamic characteristics of network traffic, especially when the traffic pattern changes dramatically.

[0150] In summary, FedMTA not only accurately captures peak traffic and overall trends in the week-long test data, but its prediction error CDF also demonstrates the model's high accuracy and reliability when processing the Milan and Trento datasets. These experimental results fully demonstrate FedMTA's superior performance in network traffic forecasting tasks and its broad application potential, demonstrating its powerful ability to cope with complex and dynamic network traffic changes.

[0151] The present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A federated learning network traffic prediction method based on multi-scale temporal feature enhancement, characterized by: The following steps are involved: Obtain network traffic data, recorded as the original data set; The discrete time information corresponding to the network traffic data is mapped into continuous value features. In this process, the corresponding periodicity is obtained according to different time granularities to obtain the time features. The impact factor of each festival is regarded as an independent feature dimension, and the distance d between the i-th day and the j-th festival is ij Converted into impact factor value X through kernel function ij ; For a single time point t i , will affect the factor value X ij Perform weighted summation to obtain a single numerical comprehensive impact factor F i '; A multi-scale temporal network traffic prediction framework based on federated learning is used for prediction; the multi-scale temporal network traffic prediction framework based on federated learning includes a prediction step of a local client and a step of global training based on federated learning, taking each local client as a participant; The prediction steps of the local client include: based on the iTransformer network model, encoding the entire traffic data sequence into a token, and independently encoding the time characteristics and comprehensive influencing factors corresponding to different time granularities into their own tokens, thereby obtaining the traffic forecast value for the future period; Taking each local client as a participant, the steps of global training based on federated learning include: Federated learning includes the training process of local clients and central servers, wherein the local client is responsible for training a local model using local data, and multiple local clients each train a local model in a round of training, and then transmit their local model parameters to the central server. The central server is responsible for aggregating the local models uploaded by each local client to obtain a global model. In the next round of training, the central server distributes the global model to each local client. Each local client uses the obtained global model as a basis and performs a new round of training using its own local data to obtain an updated local model. The new local model is then uploaded to the central server and the training is repeated until the specified training round is reached, and finally a final global model is obtained.

2. The method for predicting network traffic using federated learning based on multi-scale temporal feature enhancement according to claim 1 is characterized by: The process of mapping discrete time information into continuous-valued features includes: For a given time point t, its feature construction follows a general paradigm: first extract the original sequence number g(t) of its time dimension, then divide it by the period length N of this dimension, and finally constrain the value range to the symmetric interval [-0.5, 0.5] through a translation operation, denoted as f(t): Among them, g(t) is the original serial number of time in the dimension of time granularity, which is adjusted according to the time granularity; N is the period length corresponding to different time granularities.

3. The method for predicting network traffic using federated learning based on multi-scale temporal feature enhancement according to claim 2 is characterized by: In the process of obtaining the time features according to the periodicity corresponding to different time granularities, the time features are obtained according to the periodicity corresponding to "hour", "week", "month" and "year".

4. The method for predicting network traffic using federated learning based on multi-scale temporal feature enhancement according to claim 3 is characterized by: The original serial numbers of "hour", "week", "month", and "year" in the dimension of time granularity and the corresponding period lengths of different time granularities are as follows: For the hour feature, g(t) corresponds to the hour number within a day, starting with 0, and its cycle length N is set to 23 hours; for the week feature, g(t) represents the date number within the week, starting with 0, and its cycle length N is set to 6 days; for the month feature, g(t) corresponds to the date number within a month, and the maximum number of days in the month is dynamically selected as its cycle length; in the construction of the year feature, g(t) is the cumulative date number within the year, starting with 0, and its cycle length N is set to 364.

5. The method for predicting network traffic using federated learning based on multi-scale temporal feature enhancement according to any one of claims 1 to 4, characterized in that: The impact factor of each festival is regarded as an independent feature dimension, and the distance d between the i-th day and the j-th festival is ij Converted into impact factor value X through kernel function ij The process is as follows: The time information in the original data set is listed separately and converted into a standard timestamp format; let T = {t1, t2, ..., t m } represents the set of all time points in the time series dataset; for each time point t i ∈T, calculate the distance d between it and each important festival j ij ; Then, the distance d ij Input kernel function K(d ij ;α)=exp(-α|d ij |), and get the corresponding impact factor value X ij ; where α is a parameter that controls the speed of influence decay.

6. The method for predicting network traffic using federated learning based on multi-scale temporal feature enhancement according to claim 5 is characterized by: The comprehensive impact factor Among them, w j is the weight corresponding to the j-th holiday.

7. The method for predicting network traffic using federated learning based on multi-scale temporal feature enhancement according to claim 6, characterized in that: The local client is responsible for training a local model using local data. The local client i calculates the gradient g corresponding to the small batch sample based on the local data. i (y i ), and perform gradient correction while controlling the variable c i Updates.

8. The method for predicting network traffic using federated learning based on multi-scale temporal feature enhancement according to claim 7, characterized in that: The central server is responsible for aggregating the local models uploaded by each local client. The central server updates the weighted average aggregation parameters. At the same time, the global control variables are adjusted according to the updates provided by each local client; where ΔW global It is the global model obtained by aggregating multiple local client models in federated learning, ΔW k The meaning is the local model on the kth local client, and there are n local clients in total.

9. A computer storage medium, characterized in that: The storage medium stores a computer program, which is loaded and executed by a processor to implement a federated learning network traffic prediction method based on multi-scale time feature enhancement as described in any one of claims 1 to 8.

10. A federated learning network traffic prediction device based on multi-scale temporal feature enhancement, characterized by: The device includes a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement a federated learning network traffic prediction method based on multi-scale time feature enhancement as described in any one of claims 1 to 8.

Images