A transformer-based smart grid 5G mobile terminal multi-trajectory prediction method

Abstract

The application discloses a kind of based on the intelligent power grid 5G mobile terminal multi-track prediction method of Transformer, comprising the following steps: first, set the deployment range of intelligent power grid, collect the mobile track in the range;Processing track data, by Gaussian algorithm to data is smoothed and eliminates abnormal data, with minute as minimum unit sampling, and according to 80% and 20% proportion division training sample and test sample;Finally, design overall double-layer Transformer model, the first layer Transformer encodes the history and current predicted track of each mobile terminal, the second layer Transformer captures and further models the relationship between mobile terminal and related movement mode;By using the architecture of Transformer to complete the terminal track prediction of 5G intelligent power grid, to provide effective support for custom or optimize the mobile, registration and handover management of network.

Description

TECHNICAL FIELD

[0001] The application relates to the field of trajectory prediction, in particular to a smart power grid 5G mobile terminal multi-trajectory prediction method based on a double-layer Transformer. BACKGROUND

[0002] With the continuous popularity and development of mobile Internet, the data traffic of mobile services presents an explosive growth. Subsequently, 4G mobile communication technology is difficult to meet the existing demand, and 5G emerges as the times require. The emerging 5G mobile communication technology has the advantages of super connection, ultra-low latency, super-high bandwidth, etc., which can greatly improve the network reliability and operation efficiency. With the continuous improvement of 5G coverage, 5G is increasingly widely used. As a key infrastructure for installing new base stations, 5G not only serves individual users, but also needs to meet the needs of many vertical industries. Among the many 5G application vertical industries, the power grid is a typical representative. The power grid needs to rely on the advanced communication technology of 5G to complete the digitalization and intelligentization transformation, which has important strategic significance for promoting the innovation of energy production and consumption mode, ensuring energy security, and promoting sustainable economic development.

[0003] The 5G power communication network is an important infrastructure to support the development of the smart grid, but at the present stage, the 5G network has high frequency bands and large transmission loss, which is difficult to achieve full coverage, while the 4G LTE private network of the power grid has the characteristics of wide coverage and low operating cost. The deep integration of the two networks is more practical. At the same time, we also need various technical means to support the terminal access management, resource allocation and state monitoring of the integrated network.

[0004] In addition to fixed smart grid terminals such as electric meter data acquisition, power differential protection, and power video monitoring, 5G power services also include power inspection robots, mobile charging piles, unmanned aerial vehicles, and electric operation vehicles. On the one hand, the moving speed range of these mobile terminals varies greatly. For example, an ordinary unmanned aerial vehicle can reach 30 meters per second, and the moving distance in one minute is 1800 meters. On the other hand, the coverage range of a 5G base station is only about 300 meters. In the process of moving the mobile terminal, it is inevitable to frequently switch between different base stations, causing dynamic changes in network load. Based on the trajectory data of 5G mobile terminals, their running state, business mode, resource utilization, etc. can be analyzed to provide data support for the optimization of integrated networks. In addition, based on existing data, the trajectory of 5G mobile terminals can be predicted to provide effective support for customizing or optimizing mobile, registration and handover management.

[0005] To study the trajectory prediction of individual, a trajectory prediction method based on hidden Markov model (HMTP) is proposed in the paper S. Qiao, D. Shen, X. Wang, N. Han and W. Zhu, “A Self-Adaptive Parameter Selection Trajectory Prediction Approach via Hidden Markov Models,” IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 1, pp. 284-296, 2015. The transition probability is obtained by calculating the state transition matrix to predict the next specific location of the moving user. A long horizon trajectory prediction method of surrounding vehicles based on dual LSTM networks is proposed in the paper L. Xin, P. Wang, C. Y. Chan, J. Chen, S. E. Li and B. Cheng, “Intention-aware Long Horizon Trajectory Prediction of Surrounding Vehicles using Dual LSTM Networks,” Proc. International Conference on Intelligent Transportation Systems (ITSC), pp. 1441-1446, 2018. The first LSTM is used to identify the intention of the subject as the intermediate hidden layer variable of the second LSTM to predict the final trajectory. The double-layer LSTM structure effectively improves the prediction accuracy.

[0006] However, this method mainly focuses on the prediction of individual motion trajectory, that is, the motion trajectory of each individual is independent of each other. In the vertical application of 5G smart grid, mobile terminals often complete the common task through cooperation or division of labor, and there is obvious correlation between the trajectories of the terminals. This method lacks multi-trajectory prediction and cannot fully utilize the related movement patterns of the terminals.

[0007] A model ITGTAIT is proposed in the paper F. Li, Q. Li, Z. Li, Z. Huang, X. Chang and J. Xia, "A Personal Location Prediction Method to Solve the Problem of Sparse Trajectory Data," Proc. IEEE International Conference on Mobile Data Management (MDM), pp. 329-336, 2019. The general motion patterns are obtained by clustering the group trajectories, which can be used to improve the prediction accuracy of individual trajectories. These methods are mainly to solve the sparsity of individual trajectories; in order to solve the problem of multi-trajectory prediction, a trajectory prediction method based on multivariate multi-order Markov model is proposed in the paper J. Ding, H. Liu, L. T. Yang, T. Yao and W. Zuo, "Multiuser Multivariate Multiorder Markov-Based Multimodal User Mobility Pattern Prediction," IEEE Internet of Things Journal, vol. 7, no. 5, pp. 4519-4531, 2020. However, it needs to construct the transition probability matrix artificially according to the statistical information, and needs to convert the predicted continuous position into a discrete state space. Obviously, the choice of discretization granularity will have a great influence on the result. SUMMARY

[0008] The present application mainly solves the problem that the existing trajectory prediction method independently inputs the trajectory without considering the correlation of the mobile terminal trajectory in the smart grid; a 5G mobile terminal multi-trajectory prediction method based on Transformer for the smart grid is provided, which adopts a double-layer Transformer structure, the first layer of Transformer encodes the historical and current predicted trajectory of each mobile terminal, and the second layer of Transformer captures the related motion patterns between terminals to improve the prediction performance.

[0009] The above technical problems of the present application are mainly solved by the following technical scheme:

[0010] A 5G mobile terminal multi-trajectory prediction method based on Transformer for the smart grid, comprising the following steps:

[0011] S1: Set the deployment range, collect the mobile trajectory data of the mobile terminals in the deployment range; including the trajectories of intelligent robots, drones, electric work vehicles, video monitoring in the 5G smart grid and other data;

[0012] S2: Process the mobile trajectory data; smooth the mobile trajectory data and eliminate abnormal data, divide the training samples and test samples;

[0013] S3: Build a double-layer Transformer model, the first layer of Transformer encodes the historical and current predicted trajectory of each mobile terminal; the second layer of Transformer captures the relationship and motion pattern between mobile terminals and further models them;

[0014] S4: Train and test the double-layer Transformer model, deploy the trained double-layer Transformer model to the processing center;

[0015] S5: Track the trajectory of the mobile terminal in the deployment range, output the trajectory prediction result through the double-layer Transformer model.

[0016] The scheme adopts a double-layer Transformer structure, the first layer of Transformer encodes the historical and current predicted trajectory of each mobile terminal, and the second layer of Transformer captures the related motion pattern between terminals to improve the prediction performance. The correlation of the mobile terminal trajectory in the smart grid is fully considered.

[0017] As a preferred, the step S1 specifically includes the following processes:

[0018] S101: Set the length and width of the area covered by the smart grid;

[0019] S102: Collect the mobile trajectory data of 5G terminal devices such as intelligent robots, drones, electric work vehicles, video monitoring, etc. respectively;

[0020] S103: Given g mobile terminal trajectory data with the same sampling interval and length p. The target is to simultaneously predict the coordinate positions of the g mobile terminals in the next h time steps.

[0021] Taking the 5G power communication network as an important infrastructure supporting the development of smart grid, through the efficient, safe and orderly operation of various businesses, the trajectories of various mobile terminals such as power inspection robots, mobile charging piles, drones and electric work vehicles are predicted, thereby providing support for dynamic management and allocation of network spectrum bandwidth resources, and finally improving the utilization efficiency of limited 5G network resources.

[0022] As preferred, in the step S2, the data is smoothed by Gaussian algorithm.

[0023] The filter function of Gaussian algorithm is:

[0024]

[0025] Wherein, (x, y) is the point coordinate of the moving trajectory data.

[0026] σ is the standard deviation.

[0027] As preferred, the first layer of Transformer comprises an encoder and a decoder;

[0028] The encoder comprises a plurality of first layers, each of which comprises a multi-head self-attention module and a feedforward network module, and each layer is connected in residual connection and normalized;

[0029] The decoder comprises a plurality of second layers, each of which comprises a masked multi-head self-attention module, a multi-head self-attention module and a feedforward network module, and each layer is connected in residual connection and normalized.

[0030] The first layer of Transformer self-encodes the historical and current predicted trajectory of each terminal, captures the important events on the time axis, and completes the internal feature extraction by using the self-attention mechanism and the position-based feedforward network.

[0031] As preferred, the operation process of the first layer of Transformer is:

[0032] A1: input sequence T = {t 1 , t 2 ,..., t p} is input to the first layer of Transformer; by linear transformation, the coordinates of each time step are mapped to a high-dimensional embedding space;

[0033] A2: for the position information in the input sequence, the position features are encoded by different frequency sine and cosine functions, and the position encoding is embedded at the bottom of the encoder and the decoder;

[0034] A3: add the outputs in step A1 and step A2 to obtain x, and convert it into Query vector, Key vector and Value vector through three weight matrices W q , W k and W v respectively;

[0035] A4: the feedforward network module comprises two linear layers and a nonlinear activation function, and the specific calculation process is:

[0036] FFN(x) = ReLU(W1x + b1) W2 + b2

[0037] where W1 and W2 are parameter matrices of two linear layers, and b1 and b2 are bias parameters of linear layers;

[0038] A5: The specific process of residual connection and normalization is as follows:

[0039] x = LayerNorm(x + Operation(x))

[0040] where Operation(·) represents a self-attention operation or a feedforward network, and LayerNorm(·) represents a layer normalization operation;

[0041] A6: The decoding network adopts two self-attention model structures; the first self-attention model encodes the input {t zero , t p +1 , t p+2 , t p+3 , t p+4} as a Query vector of the second self-attention model; the output of the second self-attention model adopts a feedforward network, a residual connection, and a normalization module, and the final output is a D-dimensional feature vector corresponding to each trajectory position point and is used as the input of the second layer Transformer.

[0042] The first layer Transformer self-encodes the historical and current predicted trajectories of each terminal, captures important events on the time axis, and completes intrinsic feature extraction by using a self-attention mechanism and a position-based feedforward network.

[0043] As preferred, the second layer Transformer includes a multi-head self-attention module, a feedforward network module, a residual connection module, and a normalization module, obtains an enhanced feature sequence, and then uses linear transformation to map the output enhanced feature vector to a two-dimensional coordinate position, thereby obtaining the final predicted trajectory coordinate position.

[0044] The second layer Transformer takes the intrinsic features of each terminal as input and further performs attention encoding, thereby effectively capturing the motion patterns related between terminals and improving the prediction accuracy.

[0045] As preferred, the operation process of the second layer Transformer is as follows:

[0046] B1: The input of the second layer Transformer is a high-dimensional embedding of each mobile terminal at each predicted time step , which is D-dimensional in total.

[0047] B2: three weight matrices W q , W k , and W v convert the input into Query vector, Key vector and Value vector respectively;

[0048] B3: embedding feed-forward network module, the feed-forward network module contains two linear layers and a nonlinear activation function, the specific calculation process is:

[0049] FFN(x) = ReLU(W1x + b1)W2 + b2

[0050] where W1 and W2 are parameter matrices of two linear layers, and b1 and b2 are bias parameters of linear layers;

[0051] B4: the specific process of residual connection and normalization is:

[0052] x = LayerNorm(x + Operation(x))

[0053] where Operation(·) represents self-attention operation or feed-forward network, and LayerNorm(·) represents layer normalization operation;

[0054] B5: for each mobile terminal's reinforced feature vector of the output, a linear transformation is used to map it to the two-dimensional coordinate position of the mobile terminal at this time step, so as to obtain the final predicted trajectory coordinate position.

[0055] The second layer of Transformer takes the internal features of each terminal as input and further encodes the attention, so as to effectively capture the motion patterns related between terminals and improve the prediction accuracy.

[0056] As preferred, the step S4 comprises the following steps:

[0057] S401: set the learning rate of training to 0.01, epoch to 300, and batch size to 100;

[0058] S402: during the training process, use MSE as the learning error function:

[0059]

[0060] where t i is the actual mobile terminal trajectory coordinate position, is the predicted mobile terminal trajectory coordinate position;

[0061] S403: Adopt the Adam optimizer to optimize the network parameters, and gradually reduce the error function loss value.

[0062] With the advantages of super large connection, super low latency, super high bandwidth of 5G, the trajectories of various 5G mobile terminals are predicted, thereby providing effective support for customizing or optimizing mobile, registration and handover management.

[0063] The beneficial effects of the present application are:

[0064] The present application provides a kind of intelligent power grid 5G mobile terminal multi-track prediction method based on double-layer Transformer.The first layer Transformer is self-encoding to each terminal's history and current predicted trajectory, capture important events on time axis, and utilize self-attention mechanism and position-based feedforward network to complete intrinsic feature extraction.The second layer Transformer takes each terminal's intrinsic feature as input, further carries out attention coding, thereby effectively capturing the motion pattern related between terminals, improve prediction accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0065] Figure 1 It is a kind of intelligent power grid 5G mobile terminal multi-track prediction method flow chart of the present application.

[0066] Figure 2 It is the schematic diagram of the 5G network architecture of the intelligent power grid of the present application.

[0067] Figure 3 It is the collaborative working mode diagram of the 5G intelligent power grid mobile terminal of the present application.

[0068] Figure 4 It is the division of labor working mode diagram of the 5G intelligent power grid mobile terminal of the present application.

[0069] Figure 5 It is the organization architecture diagram of the first layer Transformer of the present application.

[0070] Figure 6 It is the organization architecture diagram of the second layer Transformer of the present application. DETAILED DESCRIPTION

[0071] The technical solutions of the present application will be further specifically explained below by examples, and combined with the drawings.

[0072] Embodiment:

[0073] A kind of intelligent power grid 5G mobile terminal multi-track prediction method based on Transformer of the present application, as shown in Figure Figure 1 It includes the following steps:

[0074] S1: Set the deployment range, collect the mobile trajectory data of the mobile terminal in the deployment range.

[0075] The mobile trajectory data of the mobile terminal includes the trajectory of intelligent robots, unmanned aerial vehicles, electric work vehicles, video monitoring and other data in the 5G smart grid.

[0076] S101: Set the length and width of the area covered by the smart grid. As shown in the figure, a schematic diagram of the 5G network architecture of the smart grid is shown. Figure 2

[0077] S102: Collect the mobile trajectory data of 5G terminal devices such as intelligent robots, unmanned aerial vehicles, electric work vehicles, and video monitoring.

[0078] S103: Given g mobile terminal trajectory data with the same sampling interval and length p. The goal is to predict the coordinate position of the next h time step for the g mobile terminals at the same time

[0079] Taking 5G power communication network as an important infrastructure to support the development of smart grid, through the efficient, safe and orderly operation of various businesses, the trajectory prediction of power inspection robots, mobile charging piles, unmanned aerial vehicles, electric work vehicles and other mobile terminals is carried out, so as to provide support for dynamic management and allocation of network spectrum bandwidth resources, and finally improve the utilization efficiency of limited 5G network resources.

[0080] S2: Process the mobile trajectory data; smooth the mobile trajectory data and remove abnormal data, and divide the training samples and test samples.

[0081] The collected trajectory data is denoised. The data is smoothed by Gaussian algorithm. The Gaussian function is discretized, that is, the template coefficient of the Gaussian filter is obtained, and the Gaussian filtering function is:

[0082]

[0083] Where (x, y) is the point coordinate of the mobile trajectory data;

[0084] σ is the standard deviation.

[0085] Figure 3 、 Figure 4 Two common working modes of 5G mobile terminals in smart grid are demonstrated, Figure 3 The cooperative working mode of 5G mobile terminals in smart grid is shown, Figure 4 ​The division of labor mode of the 5G smart grid mobile terminal is shown. The collected trajectory data is sampled 100 times, and the training samples and test samples are divided according to the ratio of 80% and 20%, that is, 80% of the samples are used to constitute the training set, and the remaining 20% of the samples constitute the test set.

[0086] For cooperative tasks, a target position is first randomly generated in the region for the group, each terminal randomly generates V, and moves to the target position at different speeds. When all terminals reach the target position, P is randomly generated as the work time of the target position. When the work time P arrives, the next target position is randomly generated.

[0087] For division of labor tasks, a set of target positions is first randomly generated in the region for the group, each terminal selects a target position that has not been reached by other terminals from the target position group, and reaches the selected target position at a randomly generated V, and then stays at the randomly generated P to complete the work. After the work is completed, each terminal continues to select other target positions that have not been reached. When all target positions are reached, the mobile trajectory sampling ends.

[0088] S3: Build a double-layer Transformer model, the first layer of Transformer encodes the historical and current predicted trajectories of each mobile terminal; the second layer of Transformer captures the relationship and movement mode between mobile terminals and further models.

[0089] As shown in Figure 5 , the first layer of Transformer includes an encoder and a decoder. The first layer of Transformer self-encodes the historical and current predicted trajectories of each terminal, captures important events on the time axis, and uses the self-attention mechanism and the position-based feedforward network to complete the intrinsic feature extraction.

[0090] The encoder includes several first layers, each of which contains a multi-head self-attention module (Multi-Head Attention) and a feedforward network module (Feed Forward), and each layer performs residual connection and normalization (ADD & Norm).

[0091] The decoder includes several second layers, each of which contains a masked multi-head self-attention module (Masked Multi-Head Attention), a multi-head self-attention module (Multi-Head Attention), and a feedforward network module (Feed Forward), and each layer performs residual connection and normalization (ADD & Norm).

[0092] The operation process of the first layer Transformer is as follows:

[0093] A1: The input sequence of the Transformer is T = {t 1 , t 2 ,..., t p}, first map the coordinates of each time step to a high-dimensional embedding space, which is realized by a linear transformation, the input is 2-dimensional and the output is D-dimensional.

[0094] A2: For the position information in the input sequence, encode the position features by different frequencies of sine and cosine functions, and embed the position encoding at the bottom of the encoder and decoder.

[0095] The trajectory sequence t 1 , t 2 ,..., t p is strictly sorted according to the sampling point time, and for the position information 1, 2,..., p in the input sequence, "position encoding" is embedded at the bottom of the encoder and decoder, that is, the position features are encoded by different frequencies of sine and cosine functions, and the output is the position encoding corresponding to the input sequence, the specific calculation formula is:

[0096]

[0097] Where pos is the position information, taking integer values between {1, 2,..., p}, i is an integer between {1, Dm}, Dm is the encoding length, and k is an integer corresponding to i, indicating whether i is odd or even.

[0098] A3: Add the outputs in steps A1 and A2 to obtain x, which is converted into Query vector, Key vector and Value vector through three weight matrices W q , W k and W v .

[0099] Calculate the attention matrix representing the similarity between Query vector and Key vector, and get the value of Attention, in order to alleviate the defect of gradient disappearance, the attention model is normalized by d k .

[0100]

[0101] Where d k is the length of the Attention head, and Softmax is a normalized exponential function.

[0102] A4: The feedforward network module includes two linear layers and a nonlinear activation function, and the specific calculation process is as follows:

[0103] FFN(x)=ReLU(W i x+b1)W2+b2

[0104] Where W1 and W2 are the parameter matrices of the two linear layers, and b1 and b2 are the bias parameters of the linear layers;

[0105] A5: The specific process of residual connection and normalization is as follows:

[0106] x = LayerNorm(x + Operation(x))

[0107] Where Operation(·) represents self-attention operation or feedforward network, and LayerNorm(·) represents layer normalization operation.

[0108] To increase the model depth and achieve higher performance, residual connections are added, and layer normalization is performed.

[0109] A6: The decoding network employs a two-self-attention model structure; the first self-attention model applies input {t} zero , t p +1 , t p+2 , t p+3 , t p+4 The vector is encoded and used as the query vector for the second self-attention model. The output of the second attention model uses a feedforward network, residual connections, and a normalization module. The final output is a D-dimensional feature vector corresponding to each trajectory position point, which is used as the input of the second Transformer layer.

[0110] For the decoding module, during training, the input is {t} zero , t p+1 , t p+2 , t p+3 , t p+4}, where t zero This represents an empty vector, used to indicate the predicted first coordinate position, t. p+1 , t p+2 , t p+3 , t p+4 These are the actual trajectory coordinates. During prediction, t... p+1 , t p+2 , t p+3 , t p+4 This is the predicted position; a mask is used to limit attention to the already generated trajectory coordinates.

[0111] like Figure 6As shown, the second layer Transformer includes a multi-head self-attention module (Multi-Head Attention) and a feed forward network module (Feed Forward), a residual connection module and a normalization module (ADD&Norm), after obtaining the enhanced feature sequence, a linear transformation is used to map the output enhanced feature vector to a two-dimensional coordinate position, thereby obtaining the final predicted trajectory coordinate position.

[0112] The second layer Transformer takes the intrinsic features of each terminal as input and further performs attention encoding, thereby effectively capturing the motion patterns related between terminals and improving the prediction accuracy.

[0113] The operation process of the second layer Transformer is as follows:

[0114] B1: The input of the second layer Transformer is the high-dimensional embedding of each mobile terminal at each prediction time step , which has a total of D dimensions.

[0115] B2: Three weight matrices W q , W k and W v convert the input into Query vector, Key vector and Value vector respectively.

[0116] The attention matrix representing the similarity between Query vector and Key vector is calculated, and the value of Attention is obtained. In order to alleviate the defect of gradient disappearance, the attention model is normalized by d k .

[0117]

[0118] B3: Embedding feed forward network module, the feed forward network module includes two linear layers and a nonlinear activation function, the specific calculation process is as follows:

[0119] FFN(x) = ReLU(W1x + b1)W2 + b2

[0120] Where W1 and W2 are parameter matrices of the two linear layers, and b1 and b2 are bias parameters of the linear layer.

[0121] B4: The specific process of residual connection and normalization is as follows:

[0122] x = LayerNorm(x + Operation(x))

[0123] Where Operation(·) represents self-attention operation or feed forward network, and LayerNorm(·) represents layer normalization operation.

[0124] To deepen the model depth to achieve higher performance, add residual connections, and perform layer normalization.

[0125] B5: For each mobile terminal's enhanced feature vector of the output, use linear transformation to map it to the two-dimensional coordinate position of the mobile terminal at this time step, thereby obtaining the final predicted trajectory coordinate position.

[0126] S4: Train and test the double-layer Transformer model, and deploy the trained double-layer Transformer model to the processing center.

[0127] Step S4 includes the following steps:

[0128] S401: Set the learning rate of the training to 0.01, epoch to 300, and batch size to 100.

[0129] S402: During the training process, use MSE (mean square error) as the learning error function.

[0130]

[0131] where t i is the actual mobile terminal trajectory coordinate position, is the predicted mobile terminal trajectory coordinate position.

[0132] S403: Use the Adam optimizer to optimize the network parameters and gradually reduce the error function loss value.

[0133] S5: For mobile terminal trajectory tracking within the deployment range, output trajectory prediction results through the double-layer Transformer model.

[0134] The scheme of the present embodiment adopts a double-layer Transformer structure, the first layer Transformer encodes the history and current predicted trajectory of each mobile terminal, and the second layer Transformer captures the related motion patterns between terminals to improve the prediction performance. The correlation of the mobile terminal trajectory in the smart grid is fully considered.

[0135] Suppose the length and width of the area covered by the smart grid are 10000 meters. Referring to the flight speed of the unmanned aerial vehicle, the terminal moving speed V ranges from 10 to 25 meters per second, the longest working time P of the terminal in a place is 1 to 10 minutes, and the number of terminal moving groups Nt is set to the range of 5 to 10. V, Pmax, and Nt are all sampled using uniform distribution. Sampling is performed every minute, and a total of 180 times of sampling are performed, i.e., 3 hours of trajectory data are obtained.

[0136] For the cooperative task, a target position is first randomly generated for the group in the area range, each terminal randomly generates V, and moves to the target position at different speeds, and when all terminals arrive at the target position, P is randomly generated as the work time of the target position. When the work time P arrives, the next target position is randomly generated.

[0137] For the division task, a set of target positions is first randomly generated for the group in the area range, each terminal selects a target position that has not been reached by other terminals from the target position group, and reaches the selected target position at a randomly generated V, and then stays at a randomly generated P to complete the work. After the work is completed, each terminal continues to select other target positions that have not been reached. When all target positions are reached, the entire mobile trajectory sampling ends.

[0138] For the two cooperative types, we sample 100 times respectively, and the training samples and test samples are divided according to the ratio of 80% and 20%, that is, 80% of the samples are used to constitute the training set, and the remaining 20% of the samples constitute the test set.

[0139] The processor of the experimental platform is Intel I9 10900X, the gpu is Navida RTX3090, and the deep learning framework is PyTorch. The training learning rate is 0.1, the batch size is 240, and the epoch is 300.

[0140] For the first layer of transformer, the embedding size is set to 512, the header is 8, and the layer is 6; for the second layer of transformer, the embedding is set to 512, the header is 8, and the layer is 3.

[0141] In order to further evaluate the effectiveness, the method of the embodiment and the traditional method and the sequence prediction method based on deep learning are compared, which are Linear Regression and LSTM respectively. The above methods all take each user's trajectory as a separate training sample. The same training data and test data are used to compare the performance difference of different methods. For 5 different time steps, Table 1 gives the trajectory prediction error of the above 4 methods and the method of this paper, in meters.

[0142] Algorithms Step 1 Step 2 Step 3 Step 4 Step 5 Linear-Regression 207.6 770.2 1245.6 1842.3 2630.8 LSTM 188.3 539.6 958.3 1450.9 1967.1 The present embodiment 128.9 357.1 699.1 1025.7 1325.3

[0143] From the above table, it can be seen that the prediction accuracy of the Linear-Regression method in the next time window is not very different from that of the LSTM, and then its performance decreases greatly with the increase of the step, which shows that it cannot use long-term historical information, so it can only have good performance in a very small time window. Compared with Linear-Regression, the prediction accuracy of LSTM has a significant improvement. For smart grid mobile terminals, the time of turning, staying, and changing speed events often has a long time interval with the current time, and LSTM can better handle important events with very long delays through input, output, and forget gates.

[0144] The scheme of the present embodiment achieves the best performance, with a prediction error of 128.9 meters at the first time step and a prediction error of 1325.3 meters at the fifth time step, which is 31.6% and 32.6% higher than that of LSTM, respectively. This is mainly because the design of the double-layer Transformer makes the mobile terminal not only rely on the historical trajectory on the time axis, but also be able to integrate the movement trajectories of other terminals, so as to effectively utilize the shared group motion mode and improve the accuracy of trajectory prediction, providing support for the dynamic management and allocation of network spectrum bandwidth resources, thereby improving the utilization efficiency of limited 5G network resources.

[0145] It should be understood that the embodiments are only used to illustrate the present application and not to limit the scope of the present application. Furthermore, it should be understood that those skilled in the art can make various modifications or changes to the present application after reading the content taught by the present application, and these equivalent forms also fall within the scope defined by the appended claims of the present application.

Claims

1. A method for multi-trajectory prediction of a smart grid 5G mobile terminal based on a Transformer, characterized in that, The method comprises the following steps: S1: setting a deployment range, collecting mobile trajectory data of mobile terminals in the deployment range; S2: processing the mobile trajectory data; performing smoothing processing on the mobile trajectory data and eliminating abnormal data, and dividing training samples and test samples; S3: constructing a double-layer Transformer model, a first-layer Transformer encoding the historical and current predicted trajectory of each mobile terminal; A second-layer Transformer capturing and further modeling the relationship and motion mode between mobile terminals, including taking the intrinsic characteristics of each terminal as input and further performing attention encoding; S4: training and testing the double-layer Transformer model, and deploying the trained double-layer Transformer model to a processing center; S5: tracking the trajectory of the mobile terminal in the deployment range, and outputting the trajectory prediction result through the double-layer Transformer model. 2.The Transformer-based smart grid 5G mobile terminal multi-trajectory prediction method of claim 1, wherein, The step S1 specifically comprises the following processes: S101: setting the length and width of the area covered by the smart grid; S102: collecting mobile trajectory data of 5G terminal devices respectively; S103: given g mobile terminal trajectory data with a length of p sampled at the same sampling interval. In the step S2, the mobile trajectory data is smoothed by a Gaussian algorithm; 3.The Transformer-based smart grid 5G mobile terminal multi-trajectory prediction method of claim 1 or 2, wherein, The filtering function of the Gaussian algorithm is: Where (x, y) is the point coordinate of the mobile trajectory data; σ is the standard deviation. The first-layer Transformer comprises an encoder and a decoder; 4.The Transformer-based smart grid 5G mobile terminal multi-trajectory prediction method of claim 1, wherein, The encoder comprises a plurality of first layers, each first layer comprising a multi-head self-attention module and a feedforward network module, and each layer performing residual connection and normalization; The decoder comprises a plurality of second layers, each second layer comprising a masked multi-head self-attention module, a multi-head self-attention module and a feedforward network module, and each layer performing residual connection and normalization. The operation process of the first-layer Transformer is as follows: 5.The Transformer-based smart grid 5G mobile terminal multi-trajectory prediction method of claim 4, wherein, A2: For position information in the input sequence, position features are encoded by different frequencies of sine and cosine functions, and position encoding is embedded at the bottom of the encoder and the decoder; A1: Input sequence T = {t 1 ,t 2 ,...,t p} is input to the first layer of Transformer; through linear transformation, the coordinates of each time step are mapped to a high-dimensional embedding space; A4: The feedforward network module comprises two linear layers and a nonlinear activation function, and the specific calculation process is as follows: A3: add the outputs in step A1 and step A2 to obtain x, which is converted into Query vector, Key vector and Value vector through three weight matrices W q , k and W v respectively; FFN(x)=ReLU(W1x+b1)W2+b2 Where W1 and W2 are parameter matrices of the two linear layers, and b1 and b2 are bias parameters of the linear layers; A5: The specific process of residual connection and normalization is as follows: x=LayerNorm(x+Operation(x)) Where Operation(·) represents self-attention operation or feedforward network, and LayerNorm(·) represents layer normalization operation; The second-layer Transformer comprises a multi-head self-attention module, a feedforward network module, a residual connection module and a normalization module, and after obtaining the enhanced feature sequence, a linear transformation is used to map the output enhanced feature vector to a two-dimensional coordinate position, thereby obtaining the final predicted trajectory coordinate position. A6: The decoding network adopts two self-attention model structures; the first self-attention model encodes the input {t zero ,t p+1 ,t p+2 ,t p+3 ,t p+4} as the Query vector of the second self-attention model; the output of the second attention model adopts a feedforward network, a residual connection and a normalization module, and the final output is a D-dimensional feature vector corresponding to each trajectory position point, which is used as the input of the second layer of the Transformer.

6. The Transformer-based intelligent power grid 5G mobile terminal multi-trajectory prediction method of claim 1 or 4 or 5, wherein, ​ 7. The Transformer-based smart grid 5G mobile terminal multi-trajectory prediction method of claim 6, wherein, The operation process of the second layer Transformer is as follows: B1: The input to the second layer Transformer is a high-dimensional embedding for each mobile terminal at each prediction time step co-D dimensional; B2: three weight matrices W q , W k , and W v convert the inputs into Query vector, Key vector, and Value vector, respectively; B3: embedding a feedforward network module, the feedforward network module comprising two linear layers and a nonlinear activation function, and the specific calculation process being as follows: FFN(x) = ReLU(W1x + b1)W2 + b2 Wherein, W1 and W2 are parameter matrices of the two linear layers, and b1 and b2 are bias parameters of the linear layers; B4: the specific process of residual connection and normalization is as follows: x = LayerNorm(x + Operation(x)0 Wherein, Operation(·) represents self-attention operation or feedforward network, and LayerNorm(·) represents layer normalization operation; B5: for each mobile terminal reinforcement feature vector of the output, a linear transformation is used to map it to the two-dimensional coordinate position of the mobile terminal at this time step, so as to obtain the final predicted trajectory coordinate position. 8.The Transformer-based smart grid 5G mobile terminal multi-trajectory prediction method of claim 1, wherein, The step S4 comprises the following steps: S401: setting the learning rate of training to 0.01, epoch to 300, and batch size to 100; S402: in the process of training, using MSE as the error function of learning: where t i is the actual mobile terminal trajectory coordinate position, is the predicted mobile terminal trajectory coordinate position; S403: using the Adam optimizer to optimize the network parameters, and gradually reducing the error function loss value.

Images