A communication base station traffic prediction method and system based on deep learning
Optimizing the number of attention heads of the Transformer model through Gray Wolf algorithm and AOA algorithm, solving the problem of insufficient model complexity and data dependency capture, and improving the accuracy and generalization ability of communication base station traffic prediction.
Patent Information
- Application Number
- CN202410798641.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-20
- Publication Date
- 2024-12-13
- Estimated Expiration
- 2044-06-20
AI Technical Summary
In the prior art, the Transformer model has improperly set the attention head count in the traffic prediction of communication base stations, resulting in the model being too complex or unable to fully capture data dependencies, affecting the prediction accuracy and generalization ability.
The Gray Wolf algorithm optimizes the number of attention heads of the Transformer model, combines the AOA algorithm to update the position of the Gray Wolf, and automatically adjusts the number of attention heads to improve model performance.
Improve the accuracy of traffic prediction and generalization capabilities of the model, reduce the risk of overfitting, and achieve more refined optimization and automated attention head count settings.
Smart Images
Figure CN118632277B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of traffic prediction, and in particular to a communication base station traffic prediction method and system based on deep learning. Background Art
[0002] Against the backdrop of the rapid development of mobile communication technology and the increasing popularity of multimedia applications, with the widespread use of smart phones, IoT devices, and services such as high-definition video streaming, the network traffic carried by communication base stations is growing exponentially. Therefore, communication base station traffic prediction is an important part of modern communication network management and optimization.
[0003] The use of the Transformer model is one of the important means to predict the traffic of communication base stations, because the multi-head attention mechanism in the Transformer allows the model to simultaneously consider input information from multiple representation subspaces, which enhances the model's ability to capture different types of correlations and is very helpful for the multi-dimensional features in traffic prediction.
[0004] However, in a multi-head attention layer, too many attention heads will make the model too complex, increase the difficulty of training, and may also cause overfitting of the training data and poor generalization ability. Moreover, when the number of attention heads reaches a certain level, adding additional heads may not bring significant performance improvements, and may even lead to diminishing returns.
[0005] However, a model with too few attention heads may not be able to fully capture the complex dependencies and patterns in the input data, especially for data such as communication base station traffic that may contain multiple periodicities and randomness. It may also not be able to learn enough features, resulting in poor performance of the model on unseen data. At the same time, it limits the model's ability to explore data in different subspaces, thereby affecting the diversity and robustness of the model.
[0006] Therefore, how to find the optimal number of attention heads to achieve the best prediction accuracy has become a problem that needs to be solved urgently. Based on this, the present invention provides a communication base station traffic prediction method and system based on deep learning. Summary of the invention
[0007] In order to solve the technical problems existing in the prior art, the present invention provides a communication base station traffic prediction method based on deep learning, comprising the following steps:
[0008] S01. Acquire historical traffic data of a communication base station, divide the historical traffic data into encoder input historical data and decoder input historical data, and perform preprocessing;
[0009] S02, construct a Transformer model and train it, and during the training process, optimize the Transformer model using the Grey Wolf Algorithm to obtain an optimized Transformer model;
[0010] S03, inputting the encoder input historical data into the encoder of the optimized Transformer model, and inputting the decoder input historical data into the decoder of the optimized Transformer model, to complete the prediction;
[0011] The Transformer model is constructed and trained, and during the training process, the Transformer model is optimized by the Grey Wolf Algorithm to obtain an optimized Transformer model, which specifically includes the following steps:
[0012] S1, initialize the population parameters of the gray wolf algorithm;
[0013] The population parameters include: the number of gray wolves, the range of attention heads, and the maximum number of iterations;
[0014] Each gray wolf has a different number of attention heads corresponding to the range of attention heads;
[0015] S2. For each gray wolf, use the same training data to train a Transformer model with the corresponding number of attention heads, and obtain the corresponding prediction data;
[0016] S3, evaluate the fitness value F of the corresponding gray wolf according to the error between the verification data and the prediction data, and take the gray wolf with the largest fitness value as the Alpha wolf;
[0017] S4. Update the position vector X of each gray wolf through the AOA algorithm. The specific formula is:
[0018] ;
[0019] ;
[0020] ;
[0021] ;
[0022] ;
[0023] Among them, y represents the current number of iterations, B represents the simulated buoyancy of the current gray wolf, and ρ represents the simulated density of the current gray wolf. represents the current position of the Alpha wolf, a represents the simulated acceleration of the gray wolf, represents the fitness value corresponding to the current Alpha wolf, D represents the Euclidean distance between the current gray wolf and the current Alpha wolf, C represents the control parameter, and V1 represents the simulation volume of the current gray wolf;
[0024] S5, repeat steps S2 to S4 for each gray wolf until the current number of iterations reaches the maximum number of iterations, and then execute step S6;
[0025] S6. The number of attention heads corresponding to the gray wolf with the largest final fitness value is used as the number of attention heads of the Transformer model.
[0026] Furthermore, using the same training data to respectively train the Transformer models with corresponding numbers of attention heads includes the following steps:
[0027] S21, obtaining first historical traffic data and second historical traffic data as training data;
[0028] S22. Input the first historical traffic data into the Transformer model encoder, and input the second historical traffic data into the Transformer model decoder.
[0029] Furthermore, the second historical traffic data is actual future data corresponding to the first historical traffic data; the period length of the second historical traffic data is obtained according to the time window length of the fully connected output layer of the Transformer model.
[0030] Furthermore, the verification data is the second historical traffic data.
[0031] Furthermore, the fitness value F of the gray wolf corresponding to the error evaluation of the verification data and the prediction data can be evaluated according to the root mean square error RMSE difference between the verification data and the prediction data:
[0032] ;
[0033] ;
[0034] Among them, n represents the number of predicted data, f0 represents the flow value of verification data, and f1 represents the flow value of predicted data.
[0035] The present invention also provides a communication base station traffic prediction system based on deep learning, comprising:
[0036] A data acquisition module, used to acquire training data and verification data;
[0037] Transformer prediction module, used to obtain prediction data through prediction of training data;
[0038] An attention head number optimization module, used to optimize the number of attention heads of the multi-head attention layer of the Transformer prediction module by using the gray wolf algorithm;
[0039] The gray wolf position update module is used to update the position vector X of each gray wolf in the gray wolf algorithm of the attention head number optimization module through the AOA algorithm.
[0040] Furthermore, the attention head number optimization module includes a parameter setting module, a fitness value calculation module and an optimization module;
[0041] The parameter setting module is used to set the population parameters of the gray wolf algorithm, and the population parameters include: the number of gray wolves, the range of the number of attention heads, and the maximum number of iterations;
[0042] The fitness value calculation module is used to calculate the fitness value F of the corresponding gray wolf according to the root mean square error RMSE between the verification data and the prediction data;
[0043] The optimization module is used to use the gray wolf with the largest fitness value as the Alpha wolf, and set the number of attention heads corresponding to the Alpha wolf to the number of attention heads of the multi-head attention layer of the Transformer prediction module.
[0044] Furthermore, the fitness value F of the corresponding gray wolf is calculated based on the root mean square error RMSE of the verification data and the prediction data, specifically:
[0045] ;
[0046] ;
[0047] Among them, n represents the number of predicted data, f0 represents the flow value of verification data, and f1 represents the flow value of predicted data.
[0048] Furthermore, the gray wolf position updating module includes: an updating module and a terminating module;
[0049] The updating module is used to update the position vector X of each gray wolf;
[0050] The termination module is used to determine whether the current number of iterations has reached the maximum number of iterations. If so, the updating module is terminated to perform the update.
[0051] Furthermore, the updating module updates the position vector X of each gray wolf, specifically:
[0052] ;
[0053] ;
[0054] ;
[0055] ;
[0056] ;
[0057] Among them, y represents the current number of iterations, B represents the simulated buoyancy of the current gray wolf, and ρ represents the simulated density of the current gray wolf. represents the current position of the Alpha wolf, a represents the simulated acceleration of the gray wolf, represents the fitness value corresponding to the current Alpha wolf, D represents the Euclidean distance between the current gray wolf and the current Alpha wolf, C represents the control parameter, and V1 represents the simulation volume of the current gray wolf.
[0058] Compared with the prior art, the present invention has the following beneficial effects:
[0059] 1. Through the self-attention mechanism of the multi-head attention layer of the Transformer model, all elements in the sequence data are paid attention to at the same time, which can better capture the long-term dependencies between different positions in the sequence data. For communication base station traffic data with periodicity and trend, the prediction results are more accurate. At the same time, multi-head attention allows the model to pay attention to different information at the same time, which is very important in communication base station traffic prediction, because traffic data may be affected by many factors, such as time, location, weather conditions, etc. The multi-head attention mechanism enables the model to analyze data from different angles, thereby improving prediction accuracy;
[0060] 2. The number of attention heads of the transformer model is optimized through the gray wolf algorithm, so that the transformer model can learn the dependencies in the input sequence more effectively, thereby improving the accuracy of traffic prediction. At the same time, optimizing the number of attention heads helps to avoid the model being too complex, thereby reducing the risk of overfitting. The appropriate number of attention heads can make the model more generalizable and have better prediction capabilities for unseen data. In addition, using the gray wolf algorithm to optimize the number of attention heads can achieve a certain degree of automation, reduce human intervention, and improve prediction accuracy.
[0061] 3. The AOA algorithm, namely the Archimedean optimization algorithm, is used to optimize the position update of the gray wolf in the gray wolf algorithm. The concepts of simulated buoyancy and simulated density are introduced, so that the movement of the gray wolf is not only guided by the Alpha wolf, but also affected by its own buoyancy and density, which helps the gray wolf algorithm to better balance global search and local refinement and avoid premature convergence.
[0062] At the same time, the calculation of simulated buoyancy and simulated density is adaptive, dynamically adjusted according to the fitness value of the gray wolf, and adaptively adapted to different search stages, encouraging gray wolves with low fitness values (i.e., farther away from the optimal solution) to have greater buoyancy, thereby promoting their exploration and increasing the possibility of finding new solutions. On the contrary, gray wolves with higher fitness values (close to or near the optimal solution) will have smaller buoyancy, which helps them conduct deeper searches in local areas, thereby achieving more refined optimization;
[0063] Simulated acceleration is also introduced to enable the gray wolf to respond more quickly to changes in the optimal solution, thereby performing a more refined search when approaching the optimal solution. BRIEF DESCRIPTION OF THE DRAWINGS
[0064] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0065] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0066] Figure 1 It is a flow chart of a communication base station traffic prediction method based on deep learning of the present invention;
[0067] Figure 2 is a flow chart of step S02 of the method of the present invention;
[0068] Figure 3 is a flow chart of step S2 of the method of the present invention;
[0069] Figure 4 It is a structural block diagram of a communication base station traffic prediction system based on deep learning of the present invention;
[0070] Figure 5 is a structural block diagram of the attention head number optimization module of the system of the present invention;
[0071] Figure 6 It is a structural block diagram of the gray wolf position update module of the system of the present invention. DETAILED DESCRIPTION
[0072] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0073] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.
[0074] In addition, the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.
[0075] See also Figure 1 As shown, the present invention provides a communication base station traffic prediction method based on deep learning, comprising the following steps:
[0076] S01. Acquire historical traffic data of a communication base station, divide the historical traffic data into encoder input historical data and decoder input historical data, and perform preprocessing;
[0077] S02, construct a Transformer model and train it, and during the training process, optimize the Transformer model using the Grey Wolf Algorithm to obtain an optimized Transformer model;
[0078] S03. Input the encoder input historical data into the encoder of the optimized Transformer model, and input the decoder input historical data into the decoder of the optimized Transformer model to complete the prediction.
[0079] For step S1, the time corresponding to the historical data input by the decoder is later than the time corresponding to the historical data input by the encoder;
[0080] The period length of the decoder input historical data is intercepted according to the time window length of the fully connected output layer of the optimized Transformer model.
[0081] See also Figure 2 As shown, step S02 specifically includes the following steps:
[0082] S1, initialize the population parameters of the gray wolf algorithm;
[0083] The population parameters include: the number of gray wolves, the range of attention heads and the maximum number of iterations Y.
[0084] Each gray wolf corresponds to a different number of attention heads according to the range of attention heads.
[0085] S2. For each gray wolf, use the same training data to train a Transformer model with the corresponding number of attention heads, and obtain the corresponding prediction data.
[0086] For example, if the number of attention heads corresponding to gray wolf a is 3, then the number of attention heads in the multi-head attention layer in the Transformer model corresponding to gray wolf a is 3.
[0087] S3. Evaluate the fitness value F of the corresponding gray wolf based on the error between the verification data and the prediction data, and select the gray wolf with the largest fitness value as the Alpha wolf.
[0088] S4. Update the position vector X of each gray wolf through the AOA algorithm. The specific formula is:
[0089]
[0090]
[0091]
[0092]
[0093]
[0094] Among them, y represents the current number of iterations, B represents the simulated buoyancy of the current gray wolf, and ρ represents the simulated density of the current gray wolf. represents the current position of the Alpha wolf, a represents the simulated acceleration of the gray wolf, represents the fitness value corresponding to the current Alpha wolf, D represents the Euclidean distance between the current gray wolf and the current Alpha wolf, C represents the control parameter, and V1 represents the simulation volume of the current gray wolf.
[0095] S5. Repeat steps S2 to S4 for each gray wolf until the current number of iterations reaches the maximum number of iterations, and then execute step S6.
[0096] S6. Use the number of attention heads corresponding to the gray wolf with the largest final fitness value as the number of attention heads of the Transformer model to obtain the optimized Transformer model.
[0097] See also Figure 3 As shown, in step S2, the Transformer model is trained using the training data, specifically including:
[0098] S21. Acquire first historical traffic data and second historical traffic data as training data.
[0099] The second historical traffic data is the actual future data corresponding to the first historical traffic data. For example, the first historical traffic data is the historical traffic data of February of the current year. If the cycle length of the second historical traffic data is one week, the second historical traffic data is the historical traffic data of the first week of March of the current year.
[0100] The period length of the second historical traffic data is obtained according to the time window length of the fully connected output layer of the Transformer model. For example, if the time window length of the output layer is set to predict one week's traffic, the second historical traffic data takes one week's traffic data.
[0101] S22. Input the first historical traffic data into the Transformer model encoder, and input the second historical traffic data into the Transformer model decoder.
[0102] Before performing step S22, the method further includes performing a preprocessing step on the first historical flow data and the second historical flow data;
[0103] The preprocessing step may include data cleaning and data conversion;
[0104] The data cleaning may include:
[0105] Missing value processing: check whether there are missing values in the first historical traffic data and the second historical traffic data, and decide whether to delete the records containing missing values or use interpolation methods (such as linear interpolation and average interpolation) to fill the missing values;
[0106] Outlier detection: Identify and handle outliers. Statistical methods (such as IQR) or machine learning-based methods can be used to identify and handle outliers.
[0107] The data conversion may include:
[0108] Standardization and normalization, converting the flow data to the same scale. Commonly used methods include Z-score standardization and Min-Max scaling to avoid model skewness caused by differences in numerical ranges;
[0109] Temporal feature extraction, extracting useful temporal features such as hours, days of the week, holidays, etc. from timestamps to capture potential periodic and seasonal patterns.
[0110] In step S01, the encoder input historical data and the decoder input historical data are preprocessed, and the specific steps are the same as the preprocessing steps performed on the first historical traffic data and the second historical traffic data.
[0111] In step S22, the first historical traffic data is input into the Transformer model encoder, specifically:
[0112] S2211, inputting the first historical traffic data into an encoder for time series encoding to obtain first time series data;
[0113] S2212, inputting the first time series data into the multi-head attention layer of the encoder to obtain a first output result;
[0114] S2213. Input the first output result into the feedforward neural network of the encoder to obtain the encoder output result.
[0115] In step S22, the second historical traffic data is input into the Transformer model decoder, specifically:
[0116] S2221. Input the second historical traffic data into the decoder for time series encoding to obtain second time series data;
[0117] S2222, inputting the second time series data into the masked multi-head attention layer of the decoder to obtain a second output result;
[0118] S2223, inputting the second output result into the multi-head attention layer of the decoder to obtain a third output result;
[0119] S2224: The feedforward neural network of the decoder simultaneously receives the encoder output result and the third output result to obtain a fourth output result;
[0120] S2225. Pass the fourth output result through the fully connected output layer to obtain predicted data.
[0121] In step S2211 and step S2221, the time series coding is to convert the flow value at each time point into an embedded vector, and then add the corresponding time series coding vector to each embedded vector to form the final input vector, that is, time series data.
[0122] In step S2212 and step S2223, the specific workflow of the multi-head attention layer includes:
[0123] 1. Linear transformation: First, the input vector S is projected into three different spaces to obtain the query vector Q, key vector K and value vector V2 respectively:
[0124]
[0125]
[0126]
[0127] in, , and They are the weight matrices of Q, K and V2, obtained through model training.
[0128] 2. Attention score calculation: Next, the attention score between the query vector and the key vector is calculated. This is usually done through dot product attention, where the dot product of the query vector and the key vector is normalized to produce the attention weight:
[0129]
[0130] Among them, software() is an activation function. represents the dimension of the key vector, and As a scaling factor, used to prevent the dot product result from being too large, The T in represents the matrix transpose symbol.
[0131] 3. Multi-head parallel computing: The above linear transformation and attention score calculation process is performed in parallel on multiple heads, and each head has its own weight matrix , and :
[0132]
[0133] in, This is the output of the hth head.
[0134] 4. Result connection and transformation: The output of each head is connected together to form a multi-dimensional vector, and then an additional linear transformation is performed to obtain the final multi-head attention output:
[0135]
[0136] in, Represents the weight matrix used for the linear transformation.
[0137] In step S2221, the second historical traffic data is input into the decoder, and the second historical traffic data is the actual future data corresponding to the first historical traffic data. By observing the second historical traffic data, the decoder can learn the periodicity, trend and other statistical characteristics of the traffic data. At the same time, during the training process, the error between the decoder's prediction and the actual historical data will be calculated and back-propagated to update the weight of the model, which ensures that the model gradually improves its prediction ability;
[0138] At the same time, the decoder in the model of the present invention is not like other Transformer models, which use predicted data for subsequent training as input into the decoder. Instead, the actual future data corresponding to the first historical traffic data is input into the decoder as the second historical traffic data, which helps prevent model overfitting and avoids using predicted values during training, which may cause the model to learn to replicate its own mistakes, thereby causing performance degradation.
[0139] In step S2222, the mask multi-head attention layer functions as follows:
[0140] In base station traffic prediction, the masked multi-head attention layer helps the model to predict the traffic at the next moment by relying only on the traffic data at the previous moment and other related auxiliary information. This is because the masked multi-head attention layer in the decoder applies a special mask matrix when calculating the attention weight. This matrix covers the positions of all future time points, thereby ensuring that the model does not use traffic data that has not occurred for prediction. This helps the model learn the pattern of traffic changes over time without worrying about information leakage, that is, it will not use traffic information at future moments to predict traffic at the current moment.
[0141] For example, the masked multi-head attention layer ensures that when predicting the traffic at time t, the decoder does not inappropriately use data after time t. The decoder can only predict future data based on past and current data, but cannot know the actual traffic in the future in advance. The masked multi-head attention layer ensures that the decoder follows causality.
[0142] In step S2213 and step S2224, the specific structure of the feedforward neural network usually includes two fully connected layers (also called linear layers), and an activation function is sandwiched between the two fully connected layers. The activation function selects the ReLU activation function.
[0143] The feedforward neural network is designed to process all positions of the input sequence simultaneously, that is, each position of the input sequence is processed and calculated simultaneously, rather than waiting serially for the processing result of the previous position.
[0144] In step S3, the corresponding second historical data may be used as verification data.
[0145] In step S3, the fitness value F of the corresponding gray wolf is evaluated according to the error between the verification data and the prediction data. Specifically, the fitness value F of the corresponding gray wolf can be evaluated according to the root mean square error RMSE difference between the verification data and the prediction data.
[0146]
[0147]
[0148] Among them, n represents the number of predicted data, f0 represents the flow value of the validation data, and f1 represents the flow value of the predicted data. The smaller the RMSE value, the closer the predicted value of the model is to the true value, the better the performance of the model, and the larger the fitness value of the corresponding gray wolf.
[0149] In step S6, the number of attention heads corresponding to the gray wolf with the largest final fitness value is used as the number of attention heads of the Transformer model, specifically:
[0150] The number of attention heads corresponding to the gray wolf with the largest final fitness value is used as the number of attention heads of the multi-head attention layer of the encoder of the Transformer model, as well as the number of attention heads of the masked multi-head attention layer and the multi-head attention layer of the decoder.
[0151] See also Figure 4 As shown, the present invention also provides a communication base station traffic prediction system based on deep learning, comprising:
[0152] Data acquisition module, Transformer prediction module, attention head number optimization module, gray wolf position update module and central processing unit;
[0153] The data acquisition module, the Transformer prediction module, the attention head number optimization module and the gray wolf position update module are all connected to the central processing unit in communication.
[0154] A data acquisition module, used to acquire training data and verification data;
[0155] Transformer prediction module, used to obtain prediction data through prediction of training data;
[0156] An attention head number optimization module, used to optimize the number of attention heads of the Transformer prediction module by using the gray wolf algorithm;
[0157] The gray wolf position update module is used to update the position vector X of each gray wolf in the gray wolf algorithm of the attention head number optimization module through the AOA algorithm.
[0158] See also Figure 5As shown, the attention head number optimization module includes a parameter setting module, a fitness value calculation module and an optimization module.
[0159] The parameter setting module is used to set the population parameters of the gray wolf algorithm, and the population parameters include: the number of gray wolves, the range of attention heads and the maximum number of iterations; each gray wolf corresponds to a different number of attention heads according to the range of attention heads.
[0160] The fitness value calculation module is used to calculate the fitness value F of the corresponding gray wolf according to the root mean square error RMSE of the verification data and the prediction data, specifically:
[0161]
[0162]
[0163] Among them, n represents the number of predicted data, f0 represents the flow value of the validation data, and f1 represents the flow value of the predicted data. The smaller the RMSE value, the closer the predicted value of the model is to the true value, the better the performance of the model, and the larger the fitness value of the corresponding gray wolf.
[0164] The optimization module is used to use the gray wolf with the largest fitness value as the Alpha wolf, and set the number of attention heads corresponding to the Alpha wolf to the number of attention heads of the Transformer prediction module.
[0165] The attention head number optimization module optimizes the number of attention heads of the Transformer prediction module, specifically optimizes the number of attention heads of the multi-head attention layer of the encoder of the Transformer prediction module, and the number of attention heads of the mask multi-head attention layer and the multi-head attention layer of the decoder.
[0166] See also Figure 6 As shown, the gray wolf position updating module includes: an updating module and a terminating module.
[0167] The updating module is used to update the position vector X of each gray wolf, specifically:
[0168]
[0169]
[0170]
[0171]
[0172]
[0173] Among them, y represents the current number of iterations, B represents the simulated buoyancy of the current gray wolf, and ρ represents the simulated density of the current gray wolf. represents the current position of the Alpha wolf, a represents the simulated acceleration of the gray wolf, represents the fitness value corresponding to the current Alpha wolf, D represents the Euclidean distance between the current gray wolf and the current Alpha wolf, C represents the control parameter, and V1 represents the simulation volume of the current gray wolf.
[0174] The termination module is used to determine whether the current number of iterations has reached the maximum number of iterations. If so, the updating module is terminated to perform the update.
[0175] The beneficial effects of the present invention are:
[0176] 1. Through the self-attention mechanism of the multi-head attention layer of the Transformer model, all elements in the sequence data are paid attention to at the same time, which can better capture the long-term dependencies between different positions in the sequence data. For communication base station traffic data with periodicity and trend, the prediction results are more accurate. At the same time, multi-head attention allows the model to pay attention to different information at the same time, which is very important in communication base station traffic prediction, because traffic data may be affected by many factors, such as time, location, weather conditions, etc. The multi-head attention mechanism enables the model to analyze data from different angles, thereby improving prediction accuracy;
[0177] 2. The number of attention heads of the transformer model is optimized through the gray wolf algorithm, so that the transformer model can learn the dependencies in the input sequence more effectively, thereby improving the accuracy of traffic prediction. At the same time, optimizing the number of attention heads helps to avoid the model being too complex, thereby reducing the risk of overfitting. The appropriate number of attention heads can make the model more generalizable and have better prediction capabilities for unseen data. In addition, using the gray wolf algorithm to optimize the number of attention heads can achieve a certain degree of automation, reduce human intervention, and improve prediction accuracy.
[0178] 3. The AOA algorithm, namely the Archimedean optimization algorithm, is used to optimize the position update of the gray wolf in the gray wolf algorithm. The concepts of simulated buoyancy and simulated density are introduced, so that the movement of the gray wolf is not only guided by the Alpha wolf, but also affected by its own buoyancy and density, which helps the gray wolf algorithm to better balance global search and local refinement and avoid premature convergence.
[0179] At the same time, the calculation of simulated buoyancy and simulated density is adaptive, dynamically adjusted according to the fitness value of the gray wolf, and adaptively adapted to different search stages, encouraging gray wolves with low fitness values (i.e., farther away from the optimal solution) to have greater buoyancy, thereby promoting their exploration and increasing the possibility of finding new solutions. On the contrary, gray wolves with higher fitness values (close to or near the optimal solution) will have smaller buoyancy, which helps them conduct deeper searches in local areas, thereby achieving more refined optimization;
[0180] Simulated acceleration is also introduced to enable the gray wolf to respond more quickly to changes in the optimal solution, thereby performing a more refined search when approaching the optimal solution.
[0181] In the description of the specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.
[0182] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.
Claims
1. A communication base station traffic prediction method based on deep learning, It is characterized in that The following steps are involved: S01. Acquire historical traffic data of a communication base station, divide the historical traffic data into encoder input historical data and decoder input historical data, and perform preprocessing; S02, construct a Transformer model and train it, and during the training process, optimize the Transformer model using the Grey Wolf Algorithm to obtain an optimized Transformer model; S03, inputting the encoder input historical data into the encoder of the optimized Transformer model, and inputting the decoder input historical data into the decoder of the optimized Transformer model, to complete the prediction; The Transformer model is constructed and trained, and during the training process, the Transformer model is optimized by the Grey Wolf Algorithm to obtain an optimized Transformer model, which specifically includes the following steps: S1, initialize the population parameters of the gray wolf algorithm; The population parameters include: the number of gray wolves, the range of attention heads, and the maximum number of iterations; Each gray wolf has a different number of attention heads corresponding to the range of attention heads; S2. For each gray wolf, use the same training data to train a Transformer model with the corresponding number of attention heads, and obtain the corresponding prediction data; S3, evaluate the fitness value F of the corresponding gray wolf according to the error between the verification data and the prediction data, and take the gray wolf with the largest fitness value as the Alpha wolf; S4. Update the position vector X of each wolf through the AOA Archimedean optimization algorithm. The specific formula is: ； ； ； ； ； Among them, y represents the current number of iterations, B represents the simulated buoyancy of the current gray wolf, and ρ represents the simulated density of the current gray wolf. represents the current position of the Alpha wolf, a represents the simulated acceleration of the gray wolf, represents the fitness value corresponding to the current Alpha wolf, D represents the Euclidean distance between the current gray wolf and the current Alpha wolf, C represents the control parameter, and V1 represents the simulation volume of the current gray wolf; S5, repeat steps S2 to S4 for each gray wolf until the current number of iterations reaches the maximum number of iterations, and then execute step S6; S6. The number of attention heads corresponding to the gray wolf with the largest final fitness value is used as the number of attention heads of the Transformer model.
2. According to the method for predicting traffic flow of communication base stations based on deep learning according to claim 1, It is characterized in that The method of using the same training data to train the Transformer models with the corresponding number of attention heads includes the following steps: S21, obtaining first historical traffic data and second historical traffic data as training data; S22. Input the first historical traffic data into the Transformer model encoder, and input the second historical traffic data into the Transformer model decoder.
3. According to the deep learning-based communication base station traffic prediction method of claim 2, It is characterized in that The second historical traffic data is the actual future data corresponding to the first historical traffic data; the period length of the second historical traffic data is obtained according to the time window length of the fully connected output layer of the Transformer model.
4. The communication base station traffic prediction method based on deep learning according to claim 2, It is characterized in that The verification data is the second historical traffic data.
5. The communication base station traffic prediction method based on deep learning according to claim 1, It is characterized in that The fitness value F of the gray wolf corresponding to the error evaluation of the verification data and the prediction data is evaluated according to the root mean square error RMSE difference between the verification data and the prediction data: ； ； Among them, n represents the number of predicted data, f0 represents the flow value of verification data, and f1 represents the flow value of predicted data.
6. A communication base station traffic prediction system based on deep learning, It is characterized in that include: A data acquisition module, used to acquire training data and verification data; Transformer prediction module, used to obtain prediction data through prediction of training data; An attention head number optimization module, used to optimize the number of attention heads of the multi-head attention layer of the Transformer prediction module by using the gray wolf algorithm; A gray wolf position updating module, used to update the position vector X of each gray wolf in the gray wolf algorithm of the attention head number optimization module through the AOA Archimedean optimization algorithm; The attention head number optimization module includes: A parameter setting module is used to set the population parameters of the gray wolf algorithm, wherein the population parameters include the number of gray wolves, the range of the number of attention heads, and the maximum number of iterations; The fitness value calculation module is used to calculate the fitness value F of the corresponding gray wolf according to the root mean square error RMSE between the verification data and the prediction data; The optimization module is used to select the gray wolf with the largest fitness value as the Alpha wolf, and set the number of attention heads corresponding to the Alpha wolf to the number of attention heads of the multi-head attention layer of the Transformer prediction module; The gray wolf position updating module comprises: An updating module is used to update the position vector X of each gray wolf through the AOA Archimedean optimization algorithm; A termination module is used to determine whether the current number of iterations has reached the maximum number of iterations, and if so, terminate the updating module to perform the update; The position vector X of each wolf is updated by the AOA Archimedean optimization algorithm, specifically: ； ； ； ； ； Among them, y represents the current number of iterations, B represents the simulated buoyancy of the current gray wolf, and ρ represents the simulated density of the current gray wolf. represents the current position of the Alpha wolf, a represents the simulated acceleration of the gray wolf, represents the fitness value corresponding to the current Alpha wolf, D represents the Euclidean distance between the current gray wolf and the current Alpha wolf, C represents the control parameter, and V1 represents the simulation volume of the current gray wolf.
7. The communication base station traffic prediction system based on deep learning according to claim 6, It is characterized in that The fitness value F of the corresponding gray wolf is calculated based on the root mean square error RMSE of the verification data and the prediction data, specifically: ； ； Among them, n represents the number of predicted data, f0 represents the flow value of verification data, and f1 represents the flow value of predicted data.