Multi-node base station flow prediction method based on self-organizing graph neural network

Abstract

The invention discloses a multi-node base station flow prediction method based on a self-organizing graph neural network, and belongs to the technical field of space-time flow prediction, and the method comprises the steps: processing base station node data and evaluating the influence: carrying out the sub-graphing of a node set, generating an adjacent matrix, and evaluating the node influence; base station node recombination: performing linear combination on time sequence nodes through the mapping matrix to generate a linear independent vector base, and guiding the convergence direction of the mapping matrix according to the influence; gathering spatial features by using partial attention; sequentially carrying out static graph convolution and dynamic graph convolution on a node set to capture spatial relation; interpenetrating a long-short-term memory network in the convolution; capturing a time relation to carry out time sequence prediction; and finally, standardizing and generating a prediction result through a multi-layer perceptron. According to the multi-node base station flow prediction method based on the self-organizing graph neural network, a spatial feature extraction mechanism of the multi-node base station flow is provided, and an effective prediction method is provided for the change trend of the base station flow.

Description

Technical Field

[0001] The present invention relates to the technical field of spatiotemporal traffic prediction, and in particular to a multi-node base station traffic prediction method based on a self-organizing graph neural network. Background Art

[0002] With the development of devices and the emergence of emerging user demands such as metadata and large language models, cellular communication traffic is growing exponentially in a diverse and heterogeneous manner. The huge cellular traffic poses many challenges to the next generation of mobile communication systems, such as cellular traffic management, base station energy control, network signal delay, concurrent network requests, etc. For 5G base stations, the energy consumption problem is particularly serious. According to statistics, the energy consumption of 5G base stations is about three to four times that of 4G base stations. Therefore, energy-saving strategies are of great significance in mobile communication systems. As one of the efficient energy-saving strategies, cellular traffic resource optimization that relies on high-precision cellular traffic prediction in large-scale cities has attracted much attention.

[0003] In the field of deep learning, people initially applied recurrent neural networks (RNNs) to this task. RNNs are a common model architecture for time series prediction tasks composed of gated units. This architecture captures time series features by forgetting, adding, and updating memory units. The process is to input the data of each moment of the time series into the same model in sequence and output the prediction of the future time series at the last moment. But later people found that cellular network traffic has a strong spatial correlation. Considering only the temporal characteristics of cellular network traffic cannot well predict the trend of cellular network traffic.

[0004] The 5G traffic prediction problem can be modeled as a spatiotemporal sequence prediction problem. Graph convolutional network (GCN) is a neural network used to process abstract graph relationships such as social networks, knowledge graphs, and complex file systems. Combining graph convolutional networks with RNNs can better accomplish this task. However, GCN can only capture static and fixed node correlations, which is disastrous for cellular traffic with frequently changing correlations. In response to the shortcomings of previous methods based on graph convolution or its approximation, some researchers have tried to combine GCN with attention to improve the accuracy of GCN in cellular traffic prediction tasks. However, no one has carefully studied how GCN should be combined with attention to achieve the best results. The current common practice is to multiply the adjacency matrix of GCN as a static graph and the dynamic graph formed by attention element by element. In 2023, Wang et al. proposed a cellular traffic prediction method based on adaptive hybrid spatiotemporal graph neural network (AHSTGNN), which takes the approach of adding a gate to add the static graph and the dynamic graph according to the weight. However, these combinations still need to be improved, and they cannot take advantage of the attention mechanism to capture the complex spatiotemporal dependencies of base station traffic.

[0005] Therefore, it is necessary to design a better algorithm that can coordinate the capture of static features and dynamic features to achieve better prediction performance. Summary of the invention

[0006] The purpose of the present invention is to provide a multi-node base station traffic prediction method based on a self-organizing graph neural network, propose a spatial feature extraction mechanism for multi-node base station traffic, and provide an effective prediction method for the changing trend of base station traffic.

[0007] To achieve the above object, the present invention provides a multi-node base station traffic prediction method based on a self-organizing graph neural network, comprising the following steps:

[0008] S1. Divide an area into multiple sub-areas, divide the base stations in each sub-area into N nodes, collect node data, generate an adjacency matrix and evaluate the influence;

[0009] S2, build a neural network model;

[0010] S3. Define the loss function and optimizer, use Smooth_l1_loss as the loss function, and use Adam optimizer to update the model parameters;

[0011] S4, bringing the predicted result and the true value into the loss function to calculate the loss value, using the gradient descent algorithm for back propagation, and updating the model parameters;

[0012] S5. Repeat S4 until the neural network model converges, and input the test set into the neural network model to evaluate the prediction performance of the neural network model.

[0013] Preferably, the neural network model in S2 includes the following components:

[0014] Node fusion module: including randomly initialized mapping matrix, which can realize the self-organization and reorganization of node set, map the node set into vector basis, and guide the convergence of mapping matrix according to the influence of node;

[0015] Node space feature convolution module: includes static convolution and dynamic convolution. First, the attention score matrix is ​​calculated for the node set and the vector basis in time series. After the static graph convolution is performed on the node set, the mapping matrix is ​​used to map it to the vector basis again, and the attention score matrix is ​​used to perform dynamic convolution.

[0016] Node temporal feature extraction module: inserting a long short-term memory network between the multiple layers of the node spatial feature convolution module to learn temporal features, and merging the output of each layer together and aggregating them using a fully connected layer;

[0017] Output module: After the output of each layer is aggregated with a fully connected layer, it is standardized according to layer standardization, and then the prediction result is output through a multi-layer perceptron.

[0018] Preferably, the mapping matrix is ​​generated for base station node reorganization, and the node influence is used to guide the convergence direction of the mapping matrix.

[0019] Preferably, the specific process of the base station node reorganization includes normalizing the rows of the adjacency matrix in S1, and then calculating the column sum to represent the influence of each base station node; and in the node fusion module of S2, the influence of the base station node is converted into a diagonal matrix and multiplied with the mapping matrix to guide the convergence direction of the mapping matrix; then, the node set is mapped into K linearly independent vector bases through a mapping matrix of shape (S*N,K) to participate in the attention score matrix calculation in S3;

[0020] Among them, S is the number of sub-regions, N is the number of nodes in a node set, and K refers to the number of vector bases generated by node set concentration, which is between 1 and S*N.

[0021] Preferably, a partial attention mechanism is proposed by using a mapping matrix to improve the calculation strategy of attention, including that in the node space feature convolution module of S2, attention is sought between the node set and the vector basis to generate an attention score matrix, and after graph convolution is performed on the node set, it is multiplied by the transpose of the mapping matrix and the transpose of the attention score matrix in sequence to obtain the node set of the next layer.

[0022] Therefore, the present invention adopts the above-mentioned multi-node base station traffic prediction method based on self-organizing graph neural network, proposes a spatial feature extraction mechanism for multi-node base station traffic, and provides an effective prediction method for the changing trend of base station traffic.

[0023] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is an overall flow chart of a multi-node base station traffic prediction method based on a self-organizing graph neural network of the present invention;

[0025] Figure 2 It is a self-organizing map neural network structure diagram of an embodiment of a multi-node base station traffic prediction method based on a self-organizing map neural network of the present invention;

[0026] Figure 3 It is a schematic diagram of a node dynamic relationship generation process of an embodiment of a multi-node base station traffic prediction method based on a self-organizing graph neural network of the present invention;

[0027] Figure 4 It is a diagram showing the prediction effect of the final model of an embodiment of a multi-node base station traffic prediction method based on a self-organizing graph neural network in the present invention on cellular traffic. DETAILED DESCRIPTION

[0028] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0029] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0030] Embodiment 1

[0031] like Figure 1 As shown, the present invention provides a multi-node base station traffic prediction method based on a self-organizing graph neural network, comprising the following steps:

[0032] S1. Divide an area into multiple sub-areas, divide the base stations in each sub-area into N nodes, collect node data, generate an adjacency matrix and evaluate the influence;

[0033] S2. Construct a neural network model, including the following components:

[0034] Node fusion module: including randomly initialized mapping matrix, which can realize the self-organization and reorganization of node set, map the node set into vector basis, and guide the convergence of mapping matrix according to the influence of node;

[0035] Node space feature convolution module: includes static convolution and dynamic convolution. First, the attention score matrix is ​​calculated for the node set and the vector basis in time series. After the static graph convolution is performed on the node set, the mapping matrix is ​​used to map it to the vector basis again, and the attention score matrix is ​​used to perform dynamic convolution.

[0036] Node temporal feature extraction module: Long short-term memory networks are inserted between multiple layers of the node spatial feature convolution module to learn temporal features, and the output of each layer is merged together and aggregated using a fully connected layer;

[0037] Output module: After the output of each layer is aggregated with a fully connected layer, it is standardized according to layer standardization, and then the prediction result is output through a multi-layer perceptron.

[0038] S3. Define the loss function and optimizer, use Smooth_l1_loss as the loss function, and use Adam optimizer to update the model parameters;

[0039] S4, put the predicted result and the true value into the loss function to calculate the loss value, use the gradient descent algorithm for back propagation, and update the model parameters;

[0040] S5. Repeat S4 until the neural network model converges, and input the test set into the neural network model to evaluate the prediction performance of the neural network model.

[0041] Generate a mapping matrix for base station node reorganization, and use the node influence to guide the convergence direction of the mapping matrix. Base station node reorganization includes normalizing the rows of the adjacency matrix in S1, and then calculating the column sum to represent the influence of each base station node; and in the node fusion module of S2, the base station node influence is converted into a diagonal matrix and multiplied with the mapping matrix to guide the convergence direction of the mapping matrix; then the node set is mapped into K linearly independent vector bases through a mapping matrix of shape (S*N,K) to participate in the attention score matrix calculation in S3;

[0042] Among them, S is the number of sub-regions, N is the number of nodes in a node set, and K refers to the number of vector bases generated by node set concentration, which is between 1 and S*N.

[0043] A partial attention mechanism is proposed by using the mapping matrix to improve the calculation strategy of attention, including the node space feature convolution module of S2, which seeks attention between the node set and the vector basis to generate an attention score matrix, and after performing graph convolution on the node set, multiplying it by the transpose of the mapping matrix and the transpose of the attention score matrix in sequence to obtain the node set of the next layer.

[0044] Embodiment 2

[0045] A prediction method based on a self-organizing graph neural network comprises the following steps:

[0046] S1: Collecting data sets. Directly collect existing public data sets, such as Milan, Trentino, or regional traffic data sets uploaded by communication operators such as China Telecom and China Mobile on public data platforms. This embodiment uses the Milan public data set for experiments.

[0047] S2: Dataset preprocessing. The dataset is processed according to the dimensions of time axis, space length, space width, and feature type to obtain a standardized dataset. Then, sub-graphs are formed in the two dimensions of space length and space width, and multiple samples are divided according to the time axis dimension. Finally, the sample set is divided into a training set and a test set. Optionally, the training set accounts for 80% and the test set accounts for 20%.

[0048] S3: Generate auxiliary training information. For each sub-dataset after the standard data set is sub-graphed, the cosine similarity of the nodes is calculated pairwise. The specific formula is as follows:

[0049]

[0050] in, is the flow value of the ith node of the sth subgraph at time t, is the flow value of the jth node of the sth subgraph at time t, a s,i,j is the cosine similarity between the ith node and the jth node in the sth subgraph. The adjacency matrix is ​​constructed using the cosine similarity between nodes.

[0051] S4: Figure 2 As shown in Figure 1, the adjacency matrices and node sets of multiple subgraphs are input into the model. The model is mainly composed of a fusion module (FM), a static convolution module (SCM), a dynamic convolution module (DCM), and a recurrent neural network gated unit (RNN GatedStructure):

[0052] The fusion module includes diagonalizing the rows of the adjacency matrix, two linear layers, and a diagonalization operation. The specific operation can be expressed by the following formula:

[0053]

[0054]

[0055]

[0056] E=Diag({e s，j} s＝1，2，...S，j＝1,2，...N );

[0057] M=E×W r ;

[0058] Among them, A sis the adjacency matrix of subgraph s, a s，i，j Indicates A s The element in row i and column j in Indicates A s The elements in the row sum are calculated. Diag(*) represents the operation of converting a vector into a diagonal matrix. s Yes A s The degree matrix, A′ s Yes A s The result of row normalization using the degree matrix. a′ s，i，j Represents A′ s The element in the i-th row and j-th column of . A′ s Find the column sum of the elements in e. s，j is the influence of the jth node in the sth subgraph. e and b e is a learnable parameter. E is the node influence matrix, W r is the linear layer weight that generates the mapping matrix, which is also a learnable parameter. M is the mapping matrix that will eventually be passed to the dynamic convolution layer.

[0059] The static convolution module includes graph static convolution and position static convolution. Graph static convolution is to perform graph neural network convolution on each subgraph according to its own adjacency matrix, while position static convolution is to reconnect all subgraphs according to their original orientation relationship and perform convolution on adjacent nodes.

[0060] The dynamic convolution module includes the calculation of the attention score matrix and the attention convolution. The calculation of the attention score matrix is ​​to use the node set before the static convolution as the query side, and the vector basis (also called node representative) obtained by transforming the node set before the static convolution through the mapping matrix as the query side, and calculate the correlation between them. The attention convolution is to use the node set after the static convolution to perform convolution using the attention score matrix. The schematic diagram of the node connection changes in this process is shown in Figure 3 At the same time, the process can be expressed by the following formula:

[0061]

[0062]

[0063]

[0064]

[0065]

[0066] attn=softmax(attn1||attn2);

[0067]

[0068] Among them, W e is the weight matrix used to map the vector to the embedding dimension, W q is the weight matrix used to map the vector to the query dimension, W k It is a weight matrix used to map the vector to the queried dimension, which are all learnable parameters. allnodes represents the query matrix formed by mapping the node set to the query dimension, K allnodes represents the queried matrix formed by mapping the node set to the queried dimension, K keynodes represents the query matrix formed by mapping the node representatives to the query dimension. attn′1 is the attention matrix of the node set to the node representatives, attn′2 is the vector of attention of each node in the node set to itself, ∑ col (Q allnodes ·K allnodes ) indicates that Q allnodes Matrix and K allnodes The matrix is ​​dot-multiplied and then summed over the encoding dimension, so the result is a vector. model is the number of dimensions of the embedding layer. attn is the attention score matrix of each node in the node set about the node representatives and itself. The biggest difference from attn′1 is that attn′2 is used to take its own weight into account to avoid being completely affected by the node representatives and ignoring its original characteristics. softmax(attn′1||attn′2) means combining the attn′1 matrix and the attn′2 vector together, and then using the softmax function to convert the attention weight distribution into a score distribution. The total score of any node for other nodes is 1. Attn1 and attn2 are the reverse operation of combining the attn′1 matrix and the attn′2 vector together, re-splitting attn and performing convolution on the node representatives and the node itself respectively. X is the node set before the static convolution module, X′ is the node set after the static convolution module, and X″ is the output of the dynamic convolution module.

[0069] The recurrent neural network gating unit module uses a long short-term memory network (LSTM) to extract temporal features.

[0070] Finally, it is necessary to use a multi-layer stacking method to repeatedly and alternately extract spatial features and temporal features, and finally use a fully connected layer with an input dimension equal to the number of stacked layers to integrate the outputs of each layer, and then use layer normalization to standardize it, and output the prediction results through a three-layer perceptron with relu as the activation function.

[0071] S5: Select smooth_l1_loss as the loss function and Adam optimizer as the optimizer. The gradient of each model parameter is obtained by calculating the loss value between the predicted value and the true value, and the gradient descent method is used for back propagation to update the model parameters with Adam optimizer.

[0072] S6: Repeat S4 and S5 until the model converges. Test the model with the test set to ensure that the model prediction is valid and output the final model.

[0073] like Figure 4 The figure shows the prediction effect of this example experiment. The actual value of the prediction data comes from the traffic changes of a central area node in the Milan dataset over a period of time.

[0074] Therefore, the present invention adopts the above-mentioned multi-node base station traffic prediction method based on self-organizing graph neural network, proposes a spatial feature extraction mechanism for multi-node base station traffic, and provides an effective prediction method for the changing trend of base station traffic.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A multi-node base station traffic prediction method based on a self-organizing graph neural network, characterized in that: The following steps are involved: S1. Divide an area into multiple sub-areas, divide the base stations in each sub-area into N nodes, collect node data, generate an adjacency matrix and evaluate the influence; S2, build a neural network model; S3. Define the loss function and optimizer, use Smooth_l1_loss as the loss function, and use Adam optimizer to update the model parameters; S4, bringing the predicted result and the true value into the loss function to calculate the loss value, using the gradient descent algorithm for back propagation, and updating the model parameters; S5. Repeat S4 until the neural network model converges, and input the test set into the neural network model to evaluate the prediction performance of the neural network model.

2. According to claim 1, a multi-node base station traffic prediction method based on a self-organizing graph neural network is characterized in that: The neural network model in S2 includes the following components: Node fusion module: including randomly initialized mapping matrix, which can realize the self-organization and reorganization of node set, map the node set into vector basis, and guide the convergence of mapping matrix according to the influence of node; Node space feature convolution module: includes static convolution and dynamic convolution. First, the attention score matrix is ​​calculated for the node set and the vector basis in time series. After the static graph convolution is performed on the node set, the mapping matrix is ​​used to map it to the vector basis again, and the attention score matrix is ​​used to perform dynamic convolution. Node temporal feature extraction module: inserting a long short-term memory network between the multiple layers of the node spatial feature convolution module to learn temporal features, and merging the output of each layer together and aggregating them using a fully connected layer; Output module: After the output of each layer is aggregated with a fully connected layer, it is standardized according to layer standardization, and then the prediction result is output through a multi-layer perceptron.

3. A multi-node base station traffic prediction method based on a self-organizing graph neural network according to claim 2, characterized in that: The mapping matrix is ​​generated for base station node reorganization, and the node influence is used to guide the convergence direction of the mapping matrix.

4. The method for predicting multi-node base station traffic based on a self-organizing graph neural network according to claim 3 is characterized in that: The specific process of the base station node reorganization includes normalizing the rows of the adjacency matrix in S1, and then calculating the column sum to represent the influence of each base station node; and in the node fusion module of S2, the influence of the base station node is converted into a diagonal matrix and multiplied with the mapping matrix to guide the convergence direction of the mapping matrix; then, the node set is mapped into K linearly independent vector bases through a mapping matrix of shape (S*N,K) to participate in the attention score matrix calculation in S3; Among them, S is the number of sub-regions, N is the number of nodes in a node set, and K refers to the number of vector bases generated by node set concentration, which is between 1 and S*N.

5. The method for predicting multi-node base station traffic based on a self-organizing graph neural network according to claim 4 is characterized in that: A partial attention mechanism is proposed by using a mapping matrix to improve the calculation strategy of attention, including that in the node space feature convolution module of S2, attention is sought between the node set and the vector basis to generate an attention score matrix, and after performing graph convolution on the node set, it is multiplied by the transpose of the mapping matrix and the transpose of the attention score matrix in sequence to obtain the node set of the next layer.

Images