A wireless traffic prediction method and system based on mGPT

Abstract

The application relates to an mGPT-based wireless traffic prediction method and system, and belongs to the technical fields of communication networks and artificial intelligence, and comprises the following steps: inputting preprocessed wireless traffic data to be predicted into a trained wireless traffic prediction model to realize wireless traffic data prediction; the wireless traffic prediction model comprises multiple layers of Block blocks, each Block block comprises a self-attention mechanism, a feedforward network, layer normalization and residual connection; the self-attention mechanism generates QKV, performs dimension reshaping and transposition, calculates attention through scaled dot product, and then performs projection; the self-attention mechanism assigns different weights to each element in sequence data; in the feedforward network, a linear layer expands the dimension, and the dimension is recovered through a GELU activation and a projection layer; layer normalization defines scaling and offset parameters and uses a normalization operation; the application can accurately predict wireless traffic data at future time steps and has a certain generalization ability.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of communication networks and artificial intelligence, and relates to a wireless traffic data prediction method and system based on GPT (Generative Pretrained Transformer). BACKGROUND

[0002] Traditional wireless traffic prediction models cannot capture the complex characteristics of wireless traffic data. In recent years, large language models have shown excellent capabilities in machine vision and text language. However, few studies have applied large language models to wireless traffic data prediction.

[0003] Based on large language models for wireless traffic prediction, not only can it make full use of its advantages in processing large-scale data and capturing temporal patterns, but also can improve the adaptability of the model in dynamic environments through context learning.

[0004] As a powerful generative pre-training model, GPT has proven to have strong generalization ability in various tasks due to its excellent performance in natural language processing. When applied to wireless traffic prediction, it can automatically identify and capture the potential patterns of traffic changes through its context understanding ability of historical data, and make accurate predictions.

[0005] However, in traditional models, only specific data can be used to train the model for specific tasks, and there is a lack of adaptability, so there is an urgent need for a model that can accurately predict data with slight modifications without retraining the model from scratch. SUMMARY

[0006] To overcome the shortcomings of the prior art, the application provides a wireless traffic prediction method based on mGPT, which solves the problem of limited nonlinear modeling capability of traditional models.

[0007] The application combines the self-supervised learning ability of the GPT model and the efficient context learning mechanism to predict the traffic in wireless communication networks. By introducing context information into wireless traffic data, the GPT model can automatically learn and generate the future trend of wireless traffic, significantly improving the accuracy and computational efficiency of the prediction.

[0008] The application provides a wireless traffic data prediction method based on an improved GPT, which divides data into a training set, a validation set and a test set by using a sliding window mechanism based on historical wireless traffic data, ensuring that the time sequence and context information of the data are fully utilized. The GPT model is trained to generate a prediction result of future traffic by combining historical data and context information. A periodic updating mechanism is used to fine-tune the model based on new historical data and prediction results to ensure that the model adapts to changes in traffic patterns. The prediction result is evaluated using indicators such as mean square error (MSE) to evaluate the prediction accuracy, and the model is updated based on the evaluation results to continuously optimize the prediction result.

[0009] The technical solution of the application is:

[0010] A wireless traffic prediction method based on mGPT, comprising:

[0011] The preprocessed wireless traffic data to be predicted is input into the trained wireless traffic prediction model to realize wireless traffic data prediction.

[0012] The wireless traffic prediction model includes multiple Block blocks, each Block block including a self-attention mechanism, a feedforward network, layer normalization and residual connection.

[0013] The self-attention mechanism includes a linear transformation layer that generates QKV, performs dimension reshaping and transposition, calculates attention through scaled dot product, and then projects. The self-attention mechanism assigns different weights to each element in the sequence data when processing sequence data in the wireless traffic prediction model. In the feedforward network, the linear layer expands the dimension, and the GELU activation and projection layer restore the dimension. Layer normalization defines scaling and offset parameters and uses normalization operations. Residual connection directly adds input to layer output, and the attention after layer normalization and the feedforward network result are connected in the Block class.

[0014] According to the application, the following operations are performed before training the wireless traffic prediction model, comprising:

[0015] A, obtaining historical data of a base station, dividing it into a training data set and a test data set, and selecting a sliding window size according to the sliding window mechanism on the training data set and the test data set , using the sliding window to generate a training sample data set and a test sample data set;

[0016] B, for the training sample data set, obtaining the maximum and minimum values of the traffic in the training sample data set, and using Min-Max normalization on the training sample data, the validation sample data and the test sample data to normalize them to the range [0, 1].

[0017] According to the preferred wireless traffic prediction model of the present application, the training process is as follows:

[0018] (1) In the initial stage, the input tensor, i.e. the training sample data set, is linearly transformed to convert each feature into a higher-dimensional embedding space, obtaining ;

[0019] (2) Position encoding is added ; Specifically, it refers to ;

[0020] (3) is sequentially sent into a multi-layer Block block, and the processing flow in the Block block is as follows:

[0021] (I);

[0022] (II);

[0023] (III);

[0024] (IV);

[0025] (V);

[0026] In formula (I), is the layer normalized tensor, and the input tensor is processed;

[0027] In formula (II), is the tensor after the self-attention mechanism;

[0028] Formula (III) is a residual connection, is the input tensor and the tensor processed by the self-attention mechanism are added to obtain;

[0029] Formula (IV) is a multi-layer perceptron and layer normalization, is layer normalized, and then input into the multi-layer perceptron (MLP) to obtain;

[0030] Formula (V) is another residual connection, which adds the tensor processed by the MLP to the tensor processed by the self-attention mechanism and the residual connection;

[0031] Steps (1) to (3) are repeatedly performed until the following training end conditions are met, to obtain a trained wireless traffic prediction model; the training end conditions are ① or ②:

[0032] ①: the wireless traffic prediction model parameters converge, that is, the update of the parameters is less than a set threshold value;

[0033] ②: the number of updates of the wireless traffic prediction model reaches a preset number of times.

[0034] According to the application, preferably, the wireless traffic prediction model updates parameters according to historical data, and the specific steps include:

[0035] A. Select an optimization algorithm; update the parameters of the wireless traffic prediction model based on a gradient descent strategy;

[0036] B. Select the number of samples according to the batch size from the training data set, and calculate the gradient through back propagation;

[0037] C. Update the parameters of the wireless traffic prediction model according to the gradient information of the current batch of samples; adjust the weight parameters through the optimization algorithm (AdamW);

[0038] D. Repeat steps B and C until the training end condition is met.

[0039] Further preferably, in step C, the parameters of the wireless traffic prediction model are updated according to the gradient information of the current batch of samples; the weight parameters are adjusted through the optimization algorithm (AdamW); which includes:

[0040] Calculate the first moment: calculate the exponentially weighted moving average of the gradient; the formula is:

[0041] (VI);

[0042] wherein, is the estimate of the first moment, is the gradient of the current batch, is a momentum hyperparameter;

[0043] Calculate the second moment: the formula is:

[0044] (VII);

[0045] wherein, is the estimate of the second moment, is another hyperparameter;

[0046] Update the parameters: update the parameters of the wireless traffic prediction model using the first moment and the second moment ; the formula is:

[0047] (VIII);

[0048] wherein, is a learning rate, is a constant to prevent division by zero errors.

[0049] According to the application, after wireless traffic data prediction is realized, the following operations are performed:

[0050] The predicted value is normalized to obtain the true scale of the predicted value, and the prediction performance is evaluated according to the corresponding evaluation index; after the evaluation is completed, the newly predicted data of the wireless traffic prediction model is stored in the historical database.

[0051] A computer device comprises a memory and a processor, the memory stores a computer program, and the processor realizes the steps of the wireless traffic data prediction method based on mGPT when executing the computer program.

[0052] A computer readable storage medium stores a computer program, and the computer program realizes the steps of the wireless traffic data prediction method based on mGPT when executed by a processor.

[0053] A wireless traffic prediction system based on mGPT comprises:

[0054] A preprocessing module is configured to preprocess the wireless traffic data to be predicted;

[0055] A wireless traffic data prediction module is configured to input the preprocessed wireless traffic data to be predicted into a trained wireless traffic prediction model to realize wireless traffic data prediction; the wireless traffic prediction model comprises multiple layers of Block blocks, each Block block comprises a self-attention mechanism, a feedforward network, layer normalization and residual connection; the self-attention mechanism comprises a linear transformation layer, the linear transformation layer generates QKV, performs dimension reshaping and transposition, calculates attention through scaled dot-product, and then performs projection; the self-attention mechanism assigns different weights to each element in the sequence data when the wireless traffic prediction model processes the sequence data; in the feedforward network, the linear layer expands the dimension, and the dimension is restored through the GELU activation and the projection layer; the layer normalization defines scaling and offset parameters and uses the normalization operation; the residual connection directly adds the input to the layer output, and the attention after the layer normalization and the feedforward network result are connected in the Block class respectively.

[0056] The application has the following beneficial effects:

[0057] 1、The embodiment of the application uses the pre-trained GPT for wireless traffic prediction, which can accurately predict the wireless traffic data at the future time step and has a certain generalization ability.

[0058] 2. Wireless traffic prediction using a pre-trained GPT model: By using a pre-trained GPT model for wireless traffic prediction, the present application can fully utilize the deep feature learning capability of a large language model, accurately capture the potential regularity in time series data, and predict future time step wireless traffic data.

[0059] 3. The present application replaces the original loss function in the text generation task with mean square error (MSE), and adjusts the output layer to a regression type linear layer, optimizing the accuracy and stability of wireless traffic prediction. The MSE loss function is suitable for regression problems, which can effectively reduce the error between the predicted value and the actual value, and improve the prediction accuracy.

[0060] 4. Using the few-shot learning ability of the GPT model, the present application effectively reduces the demand for a large amount of labeled data in wireless traffic prediction, reducing the cost of data preparation.

[0061] 5. By using the self-attention mechanism and deep nonlinear transformation of the GPT model, the present application can generalize in various wireless traffic data environments and adapt to different prediction scenarios, such as short-term and long-term traffic prediction, improving the adaptability and robustness of the model. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 is an architecture diagram of a wireless traffic prediction model;

[0063] Figure 2 is a comparison diagram of predicted values and true values for a certain area. DETAILED DESCRIPTION

[0064] The present application will be further limited by the description and examples in the specification and drawings, but not limited thereto.

[0065] Example 1

[0066] A wireless traffic prediction method based on mGPT (Modified Generative Pretrained Transformer) includes:

[0067] The pre-processed wireless traffic data to be predicted is input into the trained wireless traffic prediction model to realize wireless traffic data prediction, which is used to improve the accuracy of wireless traffic data prediction.

[0068] As shown in Figure 1 , the wireless traffic prediction model includes multiple layers of Block blocks, each Block block includes a self-attention mechanism, a feedforward network, layer normalization, and a residual connection;

[0069] The self-attention mechanism includes a linear transformation layer that generates QKV, performs dimension reshaping and transposition, calculates attention through a scaled dot product, and then projects; the self-attention mechanism assigns different weights to each element in the sequence data when the wireless traffic prediction model processes sequence data; so that the wireless traffic prediction model can better focus on the importance of different parts of the input sequence. In the feedforward network, the linear layer expands the dimension, and the GELU activation and projection layer restore the dimension; the feedforward network is usually located after the self-attention mechanism, and further nonlinearly transforms the output of the self-attention mechanism to increase the expression ability and complexity of the model, and help the model learn more complex feature representations. The layer normalization defines the scaling and offset parameters and uses the normalization operation; the layer normalization helps to stabilize the training process, speeds up the convergence speed, and reduces the internal covariate shift. It normalizes the input feature dimension to a distribution with a mean of 0 and a variance of 1, making the network training more stable and efficient. The residual connection directly adds the input to the layer output in the Block class, and connects the attention and feedforward network results after layer normalization. The residual connection allows gradients to flow more easily in the network, alleviating the vanishing gradient problem, and helps to train deeper networks. It adds the input directly to the output of the network layer, allowing the network to learn the residual between the input and the output, making it easier to optimize and train deep networks.

[0070] Embodiment 2

[0071] The wireless traffic prediction method based on mGPT according to Embodiment 1, the difference lies in:

[0072] Before training the wireless traffic prediction model, the following operations are performed, including:

[0073] A, obtain the historical data of the base station, divide it into a training data set and a test data set according to a certain proportion, and select a sliding window size on the training data set and the test data set according to the sliding window mechanism; the data set used by the present application is a public data set widely used in the field of cellular services, which provides real telecommunications data from the Telecom Italia Big Data Challenge. The data time span is from 00:00 on November 1, 2013 to 23:50 on January 1, 2014. In order to construct a data set suitable for training, the present application uses a sliding window technique on the data of the 88 districts of Milan. Each sliding window contains n (determined according to the periodicity of the data and the time span required by the prediction task) consecutive data units of time, and slides at a step of m time units. This configuration aims to ensure the continuity and comprehensive coverage of the data. The data processed by the sliding window is divided into a training set, a validation set and a test set according to the proportion 7:2:1.

[0074] B. For the training sample data set, get the maximum and minimum values of the traffic in the training sample data set, and use Min-Max normalization for the training sample data, validation sample data and test sample data in the training sample data set, normalized to the [0, 1] range. The formula is as follows:

[0075] ;

[0076] where, is the normalized data, is the minimum value in the training set, X is the current time data, is the maximum value of the training set.

[0077] The training process of the wireless traffic prediction model is as follows:

[0078] (1) In the initial stage, the input tensor, i.e. the training sample data set, is transformed by a linear transformation , which converts each feature into a higher-dimensional embedding space to obtain ; the linear transformation is defined as: ; where, is the weight matrix of the linear transformation; represents the input tensor, which is usually the training sample data set. is the input tensor after linear transformation and adding bias , i.e. the embedding representation. is the bias vector of the linear transformation. TE is token Embedding.

[0079] (2) Add position encoding ; specifically: ; when processing sequence data (such as time series), since the order of input usually has an important influence on the result, position encoding is used to add position information for each position in the sequence. is the result of adding the embedding representation and the position encoding . By adding position encoding to the embedding representation, the input not only contains the embedding information of the original features, but also contains their position information in the sequence, which helps the model consider the importance of position when processing sequence data.

[0080] (3) is sequentially sent into multiple layers of Block blocks, which constitute the core of the wireless traffic prediction model. Each Block block will process the input tensor through a series of steps, which aims to refine and transform the data to deepen the understanding and extract patterns;

[0081] The processing flow in Block is as follows:

[0082] (I);

[0083] (II);

[0084] (III);

[0085] (IV);

[0086] (V);

[0087] In formula (I), is the layer normalized tensor, and the input tensor is processed; layer normalization calculates the mean and variance of each sample in the feature dimension and normalizes it to a distribution with a mean of 0 and a variance of 1. It is different from batch normalization, which normalizes in the feature dimension of each sample, rather than in the batch dimension.

[0088] In formula (II), is the tensor after self-attention mechanism; SA is the self-attention mechanism that allows the model to assign different attention weights to each element in the input sequence, allowing the model to focus on the importance of different parts of the input sequence.

[0089] Specifically: first generate query, key and value, and input Through different weight matrices , , get Q, K, V,

[0090] Calculate the attention score through the scaled dot-product attention formula, where is the dimension of the key vector, used to scale the attention score.

[0091] ;

[0092] Finally, according to the calculated attention score, weight sum V to get .

[0093] Formula (III) is a residual connection, is the input tensor and the tensor processed by the self-attention mechanism are added to get; allows gradients to propagate more easily in the network, helping to solve the gradient vanishing problem in deep networks, making the network easier to train.

[0094] Formula (IV) is a multilayer perceptron (MLP) with layer normalization, is After layer normalization, it is input into a multilayer perceptron (MLP) to obtain; layer normalization is the same as formula (I). In the multilayer perceptron, first linear transformation, the input features are transformed through one or more linear layers. Then nonlinear activation, use a nonlinear activation function to introduce nonlinearity, so that the model can learn more complex function relationships, which helps to capture the nonlinear patterns in wireless traffic data. The traffic in different time periods may show a nonlinear trend, and the MLP can better fit this complex relationship.

[0095] Formula (V) is another residual connection, which adds the tensor processed by the MLP and the tensor processed by the self-attention mechanism and residual connection.

[0096] Since the model is composed of multiple blocks, i represents the processing in the ith block.

[0097] The superscript LN refers to the result after layer normalization, SA refers to the result after self-attention mechanism, and MLP refers to the tensor after multilayer perceptron processing.

[0098] Repeat steps (1) to (3) until the following training end conditions are met to obtain a trained wireless traffic prediction model; the training end conditions are ① or ②:

[0099] ①: The wireless traffic prediction model parameters converge, that is, the update of the parameters is less than a set threshold;

[0100] ②: The number of updates of the wireless traffic prediction model reaches a preset number of times.

[0101] The wireless traffic prediction model updates the parameters according to the historical data, and the specific steps include:

[0102] A, select an optimization algorithm; in this step, the AdamW optimizer is selected, and the gradient descent strategy is used to update the parameters of the wireless traffic prediction model;

[0103] B, select the appropriate (usually 32) number of samples from the training data set according to the batch size, and calculate the gradient through back propagation;

[0104] Forward propagation: first input the selected batch data into the wireless traffic prediction model to calculate the predicted value. For wireless traffic prediction, the input is data processed by a sliding window and normalized. The model performs a series of calculations based on the input and finally outputs the predicted wireless traffic value.

[0105] Compute loss: Compare the predicted values with the true values and compute the loss function. The formula is , where is the true wireless traffic value, is the predicted wireless traffic value, and n is the batch size.

[0106] Backpropagation: Use the chain rule to compute the gradient of the loss function with respect to each model parameter. Starting from the loss function, compute the gradient backward according to the computation graph (which is constructed by the forward propagation process of the model). For each operation, compute the gradient according to the derivative rule of that operation.

[0107] C. Update the parameters of the wireless traffic prediction model according to the gradient information of the current batch of samples; adjust the weight parameters through the optimization algorithm (AdamW); use hyperparameters such as learning rate and momentum to ensure the stability and convergence of the parameter update of the wireless traffic prediction model.

[0108] D. Repeat steps B and C until the training end condition is met.

[0109] In step C, update the parameters of the wireless traffic prediction model according to the gradient information of the current batch of samples; adjust the weight parameters through the optimization algorithm (AdamW); including:

[0110] AdamW combines the ideas of momentum and adaptive learning rate, and is particularly suitable for deep learning models. AdamW can adaptively adjust the learning rate according to the historical gradient of each parameter during training, thereby accelerating convergence and reducing oscillation.

[0111] The update formula of AdamW is roughly as follows:

[0112] Compute first moment (momentum): By computing the exponentially weighted moving average of the gradient, it reduces the fluctuation of the gradient. The formula is:

[0113] ;

[0114] where is the estimate of the first moment, is the gradient of the current batch, is the momentum hyperparameter; usually close to 1 (e.g. 0.9).

[0115] Compute second moment (variance of gradient): Adam also computes the variance of the gradient to scale the learning rate of different parameters so that parameters with smaller gradients can also be updated sufficiently. The formula is:

[0116] ;

[0117] where is the estimate of the second moment, is another hyperparameter; usually set close to 1 (e.g. 0.999).

[0118] Update parameters: use the first and second moments to update the parameters of the wireless traffic prediction model AdamW uses weight decay (L2 regularization), that is, a constraint on the model weights is added during the update process to avoid overfitting of the model. The formula is:

[0119]

[0120] where, is the learning rate, is a constant to prevent division by zero errors. Usually 1e-8.

[0121] Since the Adam optimization algorithm converges faster, this implementation uses this optimization algorithm.

[0122] The learning rate determines the step size of parameter updates. A larger learning rate will make the parameter update faster, but may cause the model to fail to converge or oscillate around the minimum value. A smaller learning rate will make the update slow and require more iterations.

[0123] Weight decay is a feature of AdamW that controls the complexity of the model and prevents overfitting. Weight decay will penalize larger weights during the optimization process, thereby limiting the size of the parameters.

[0124] During training, by adjusting the learning rate, momentum and other hyperparameters, ensure that the model converges stably during training.

[0125] After implementing wireless traffic data prediction, the following operations are performed:

[0126] The inverse operation of normalization is performed on the predicted value to obtain the true scale of the predicted value, and the prediction performance is evaluated according to the corresponding evaluation index. After the evaluation is completed, the newly predicted data of the wireless traffic prediction model is stored in the historical database.

[0127] The inverse operation of normalization is the process of inverse normalization: , where is the predicted value.

[0128] The evaluation index includes MSE, MAE and MAPE; including:

[0129] ;

[0130] ;

[0131] ;

[0132] In the formula, m is the number of predicted data, is the real value, is the predicted value.

[0133] For wireless traffic prediction in different areas, due to the pre-training of the GPT model, only a small amount of data is needed to make predictions for specific areas.

[0134] Figure 2 is the prediction using the trained model on the test set divided in the BICOCCA area (one of the 88 areas of the Milan dataset), using the trained model, the horizontal coordinate is the predicted data time step, and the vertical coordinate is the wireless traffic data. From Figure 2 It can be seen from the results of mGPT that the traffic data at the future time step can be accurately predicted, and the sudden events can also be well predicted. Therefore, the wireless traffic prediction scheme proposed by the present application can effectively improve the prediction performance.

[0135] Embodiment 3

[0136] A computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the wireless traffic prediction method based on mGPT in embodiment 1 or 2 when executing the computer program.

[0137] Embodiment 4

[0138] A computer readable storage medium, having a computer program stored thereon, the computer program is executed by a processor to implement the steps of the wireless traffic prediction method based on mGPT in embodiment 1 or 2.

[0139] Embodiment 5

[0140] A wireless traffic prediction system based on mGPT, comprising:

[0141] A preprocessing module configured to preprocess the wireless traffic data to be predicted;

[0142] The wireless traffic data prediction module is configured to: input preprocessed wireless traffic data to be predicted into a trained wireless traffic prediction model to realize wireless traffic data prediction; the wireless traffic prediction model comprises multiple layers of Block blocks, each Block block comprises a self-attention mechanism, a feedforward network, layer normalization and a residual connection; the self-attention mechanism comprises a linear transformation layer, the linear transformation layer generates QKV, performs dimension reshaping and transposition, calculates attention through scaled dot-product, and then performs projection; the self-attention mechanism assigns different weights to each element in sequence data when the wireless traffic prediction model processes the sequence data; in the feedforward network, a linear layer expands the dimension, and a GELU activation and a projection layer restore the dimension; the layer normalization defines scaling and offset parameters and uses a normalization operation; the residual connection directly adds the input to the layer output, and the layer normalization result and the feedforward network result are connected in the Block class respectively.

Claims

1. A wireless traffic data prediction method based on mGPT, characterized in that: include: The wireless traffic data to be predicted is pre-processed and input into the trained wireless traffic prediction model to realize wireless traffic data prediction; The wireless traffic prediction model consists of multiple layers of blocks, each of which includes a self-attention mechanism, a feedforward network, layer normalization, and a residual connection. The self-attention mechanism includes a linear transformation layer, which generates a QKV, performs dimensionality reshaping and transposition, calculates attention through scaled dot products, and then performs projection. When the wireless traffic prediction model processes sequence data, the self-attention mechanism assigns different weights to each element in the sequence data. In the feedforward network, the linear layer expands the dimension, and the GELU activation and projection layers restore the dimension. Layer normalization defines the scaling and offset parameters and uses normalization operations. The residual connection adds the input directly to the layer output. In the block, the layer normalization attention and the feedforward network results are connected separately. The training process of the wireless traffic prediction model is as follows: (1) In the initial stage, the input tensor, i.e., the training sample dataset, is transformed linearly by W. te , transform each feature into a higher-dimensional embedding space to obtain X EMB ; (2) Add position code X POs ; Specifically means: X0=X EMB +X POS ; (3) X0 is sent to multiple layers of Block blocks in sequence. The processing flow in the Block blocks is as follows: In formula (I), Is the layer normalized tensor, for the input tensor X i conduct; In formula (II), is the tensor after the self-attention mechanism; Formula (III) is the residual connection, Z i is the input tensor X i And the tensor processed by the self-attention mechanism obtained by addition; Formula (IV) is a multi-layer perceptron with layer normalization. It's Z i After layer normalization, it is input into the multi-layer perceptron; Formula (V) is another residual connection, which converts the tensor processed by MLP into And the tensor Z after the self-attention mechanism and residual connection i Addition; Repeat steps (1) to (3) until the following training end conditions are met to obtain a trained wireless traffic prediction model; the training end conditions are ① or ②: ①: The wireless traffic prediction model parameters converge, that is, the parameter update change is less than the set threshold; ②: The wireless traffic prediction model has been updated a preset number of times; The following operations are performed before training the wireless traffic prediction model, including: A. Obtain historical data from the base station and divide it into a training dataset and a test dataset. Then, based on the sliding window mechanism, select a sliding window size p on the training dataset and the test dataset, and use the sliding window to process the data to generate a training sample dataset and a test sample dataset. B. For the training sample data set, obtain the maximum and minimum flow rates in the training sample data set, and use Min-Max normalization on the validation sample data and the test sample data of the training sample data set to normalize them to the range of [0, 1]; The loss function in the text generation task is replaced with mean squared error, and the output layer is adjusted to a regression-type linear layer.

2. The method for predicting wireless traffic data based on mGPT according to claim 1, characterized in that: The wireless traffic prediction model updates parameters based on historical data. The specific steps include: A. Select an optimization algorithm; update the wireless traffic prediction model parameters based on the gradient descent strategy; B. Select the number of samples from the training dataset based on the batch size and calculate the gradient through backpropagation; C. Update the parameters of the wireless traffic prediction model based on the gradient information of the current batch of samples; adjust the weight parameters through the optimization algorithm; D. Repeat steps B and C until the training end condition is met.

3. The wireless traffic data prediction method based on mGPT according to claim 2 is characterized in that: In step C, the parameters of the wireless traffic prediction model are updated based on the gradient information of the current batch of samples; the weight parameters are adjusted through the optimization algorithm; including: Calculate the first-order moment: by calculating the exponentially weighted moving average of the gradient; the formula is: Among them, m t is the estimate of the first moment, is the gradient of the current batch, and β1 is the momentum hyperparameter; Calculate the second moment: The formula is: Among them, v t is the estimate of the second-order moment, and β2 is another hyperparameter; Update parameters: Use the first-order moment and second-order moment to update the parameters θ of the wireless traffic prediction model t ; The formula is: Here, η is the learning rate and ∈ is a constant to prevent division by zero errors.

4. The method for predicting wireless traffic data based on mGPT according to claim 1, characterized in that: After wireless traffic data prediction is implemented, perform the following operations: Perform the inverse normalization operation on the predicted value to obtain the true scale of the predicted value, and evaluate the prediction performance according to the corresponding evaluation indicators. After the evaluation is completed, the newly predicted data of the wireless traffic prediction model is stored in the historical database.

5. A wireless traffic data prediction system based on mGPT, characterized in that: include: The preprocessing module is configured to: preprocess the wireless traffic data to be predicted; The wireless traffic data prediction module is configured to: pre-process the wireless traffic data to be predicted and input it into the trained wireless traffic prediction model to realize wireless traffic data prediction; The wireless traffic prediction model consists of multiple layers of blocks, each of which includes a self-attention mechanism, a feedforward network, layer normalization, and residual connections. The self-attention mechanism includes a linear transformation layer, which generates a QKV, performs dimensionality reshaping and transposition, calculates attention through scaled dot products, and then performs projection. When the wireless traffic prediction model processes sequence data, the self-attention mechanism assigns different weights to each element in the sequence data. In the feedforward network, the linear layer expands the dimension, and the dimensionality is restored through GELU activation and projection layers. Layer normalization defines scaling and offset parameters and uses normalization operations. Residual connections add the input directly to the layer output. In the block, the attention after layer normalization and the feedforward network results are connected separately. The training process of the wireless traffic prediction model is as follows: (1) In the initial stage, the input tensor, i.e., the training sample dataset, is transformed linearly by W. te , transform each feature into a higher-dimensional embedding space to obtain X EMB ; (2) Add position code X POS ; Specifically means: X0=X EMB +X POS ; (3) X0 is sent to multiple layers of Block blocks in sequence. The processing flow in the Block blocks is as follows: In formula (I), Is the layer normalized tensor, for the input tensor X i conduct; In formula (II), is the tensor after the self-attention mechanism; Formula (III) is the residual connection, Z i is the input tensor X i And the tensor processed by the self-attention mechanism obtained by addition; Formula (IV) is a multi-layer perceptron with layer normalization. It's Z i After layer normalization, it is input into the multi-layer perceptron; Formula (V) is another residual connection, which converts the tensor processed by MLP into And the tensor Z after the self-attention mechanism and residual connection i Addition; Repeat steps (1) to (3) until the following training end conditions are met to obtain a trained wireless traffic prediction model; the training end conditions are ① or ②: ①: The wireless traffic prediction model parameters converge, that is, the parameter update change is less than the set threshold; ②: The wireless traffic prediction model has been updated a preset number of times; The following operations are performed before training the wireless traffic prediction model, including: A. Obtain historical data from the base station and divide it into a training dataset and a test dataset. Then, based on the sliding window mechanism, select a sliding window size p on the training dataset and the test dataset, and use the sliding window to process the data to generate a training sample dataset and a test sample dataset. B. For the training sample data set, obtain the maximum and minimum flow rates in the training sample data set, and use Min-Max normalization on the validation sample data and the test sample data of the training sample data set to normalize them to the range of [0,1]. The loss function in the text generation task is replaced with mean squared error, and the output layer is adjusted to a regression-type linear layer.

Images