Subgraph-driven lightweight federated learning space-time cellular network prediction method
Through sub-graph-driven lightweight federated learning methods and adaptive pruning technology, the spatiotemporal correlation and cost problems in cellular networks are solved, the accuracy of traffic prediction is improved and the cost is reduced, and a more efficient federated learning framework is achieved.
Patent Information
- Application Number
- CN202411533393.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-31
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-10-31
AI Technical Summary
In cellular network traffic prediction, spatiotemporal correlation, complexity of distributed scenarios, and communication and training costs have become the main challenges of federated learning. Traditional methods are difficult to effectively extract and retain the spatiotemporality of nodes, resulting in prediction accuracy and cost issues.
Adopting lightweight federated learning method driven by subgraphs, the spatiotemporal correlation between nodes is extracted by constructing a sparse hybrid matrix, and combining the spatiotemporal graph convolution neural network, an adaptive pruning scheme is introduced, and the model transmission is optimized by reinforcement learning agents to reduce the Communication and computing costs.
It significantly improves the accuracy and model accuracy of traffic prediction, while reducing transmission and computing costs, and optimizes the performance of the federated learning framework.
Smart Images

Figure CN119047657A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of cellular network communications, and in particular relates to a subgraph-driven lightweight federated learning spatiotemporal cellular network prediction method. Background Art
[0002] With the advancement of 5G and the upcoming 6G technology, mobile cellular network traffic prediction has become a core element in modern and future cellular network management. The deployment of advanced network technologies has triggered the need for enhanced network capacity and more sophisticated resource allocation mechanisms to cope with the growing data service requirements. In particular, the heterogeneous business needs are gradually expanding in supporting applications that are highly dependent on network reliability and low latency, such as autonomous driving and augmented / virtual reality. In the future promotion of 6G technology, cellular traffic prediction, as a key component of telecommunications network management, is of great significance for achieving smarter distributed wireless network management. With the support of artificial intelligence solutions, advanced network traffic prediction strategies can help network operators make more informed decisions in many aspects such as network planning, traffic scheduling, fault diagnosis, and operating cost control. In the research field of cellular network traffic prediction, most studies have traditionally relied on centralized learning strategies. The technical paradigm has evolved from traditional machine learning methods to deep learning solutions, and advanced models based on graph neural networks have been introduced in recent years.
[0003] However, with the increasing demand for privacy protection and the increasing need for decentralized data management, the application of the FL (federated learning) framework in cellular network traffic prediction has gradually gained attention. Clients such as RSU (road side unit), BS (base station) or institutions can collaboratively train models under the coordination of a central server, which not only protects the privacy of local data but also realizes the advantages of resource integration. With the improvement of communication technology and device computing power, the next generation of edge communication technology supported by 6G networks or higher-order network technologies has made the implementation and implementation of federated learning a reality.
[0004] Nevertheless, the application of federated learning in the field of cellular traffic prediction faces multiple challenges: (1) Challenges of spatiotemporal correlation and distributed scenarios: Cellular network base stations are spread all over the city, and factors such as user mobility and dynamic changes in demand add complexity to cellular traffic prediction. Traffic patterns show a variety of spatiotemporal correlations between different base stations, which is far beyond the extraction of simple spatial adjacency or similar traffic pattern analysis methods. High-quality spatiotemporal correlations can help the model achieve higher prediction accuracy. In a distributed federated learning environment, the fact that data cannot leave the local client brings data privacy protection to customers, but at the same time brings greater challenges to achieving high-precision predictions, which requires higher-quality technology to maximize the extraction and retain node spatiotemporal properties to achieve accurate predictions.
[0005] (2) Consider communication costs and training costs: In a federated learning architecture, each client is usually an edge device, base station, or paid data provider with a certain computing capability. During the training process, the client needs to frequently interact with the central server to aggregate and distribute model parameters. The amount of communication data and communication costs have become factors that must be considered. The computing and storage resources of these clients are relatively limited, and the communication bandwidth is usually limited. How to reduce the training and data transmission costs while maintaining prediction accuracy has become an important consideration when designing a federated learning framework.
[0006] The present invention discloses a subgraph-driven lightweight federated learning spatiotemporal cellular network prediction method. We have designed a distributed subgraph partitioning strategy for the federated distributed environment with the spatiotemporal correlation between nodes as the core. This strategy extracts the adjacency distance between nodes and captures the traffic pattern to construct a sparse mixed matrix, from which a low-dimensional continuous vector representing the multi-hop neighbor structure of the node is extracted to represent the complex dependency relationship between nodes, and a subgraph node partitioning scheme that is closer to the actual FL application scenario is formulated. On this basis, we integrate the FL framework with the STGCN (Spatial-Temporal Graph Convolutional Neural Network) model, introduce an adaptive pruning scheme based on transmission parameter weights, design a reinforcement learning agent to make decisions in the model data transmission stage, and prune the lower weight parameters to reduce the model communication cost and computing cost, so as to achieve model lightweight while effectively maintaining prediction accuracy. Summary of the invention
[0007] The present invention provides a lightweight spatiotemporal network traffic prediction method under a subgraph-driven federated learning framework for cellular network scenarios. The method first adopts a subgraph partitioning strategy to solve the problem that spatiotemporal correlation is difficult to extract and retain in a distributed cellular network environment, thereby improving the accuracy of model training. On this basis, combined with federated learning and spatiotemporal graph convolutional neural networks, an adaptive pruning scheme based on transmission parameter weights is introduced to lightweight the training process and reduce the transmission and computing costs in practical applications. At the same time, through the designed reinforcement learning agent, the model adaptive pruning decision process is optimized, achieving the training effect of reducing the transmission cost while improving the model accuracy.
[0008] The technical solution adopted by the present invention is as follows: The subgraph-driven lightweight federated learning spatiotemporal cellular network prediction method includes the following steps: S1. Data preprocessing: Select the cellular network dataset for cleaning and remove invalid or missing data points; The data is processed into time series input, each sequence contains 12 time steps of historical data for predicting the traffic situation in the next time step; S2, subgraph partitioning: extract the adjacency distance matrix and node traffic pattern matrix between nodes to construct a hybrid sparse matrix, extract the low-dimensional continuous vector representing the multi-hop neighbor structure of the node through graph embedding, which is used to represent the complex dependency relationship between nodes and obtain the similarity matrix, and perform community detection on the similarity matrix to obtain the final subgraph; S3, model training: perform spatiotemporal graph convolutional neural network model training on the local client on the divided subgraphs to obtain local model parameters; S4, model parameter pruning: adaptively prune the weight parameters of the trained model, adjust the pruning strategy through the reinforcement learning agent, and obtain the pruned weight parameters; S5, model transmission: the client transmits the pruned model parameters to the server; S6, parameter aggregation: The server obtains the parameters passed by each client and performs federated average aggregation; S7, model update: The server sends the aggregated new parameters back to the client, and the client obtains the returned parameters for the next round of spatiotemporal graph convolutional neural network model training until the iteration round ends.
[0009] The hybrid sparse matrix regards the daily traffic of the BS record node as a vector, so the traffic generated within D days can be divided into a group of vectors by day. That is, there exists a vector group ; Consider the vector group as a probability distribution, and each vector as a probability mass of the probability distribution, and get the node The probability mass Probability distribution of components : The conversion cost between probability distributions of different nodes can be regarded as the difference distance between node traffic patterns. The Wasserstein probability distribution distance is used to calculate the traffic pattern distance between every two nodes, and the cost function It can be expressed as the difference between 1 and the cosine similarity between two vector groups: Further get the node With Node The Wasserstein probability distribution distance matrix between , the probability distribution distance between every two nodes for: in, is a joint probability distribution that satisfies the following marginal constraints: in, and The nodes are and nodes The probability distribution of Traffic pattern relationship matrix between nodes represented by probability distribution distance and the adjacency matrix dominated by the actual distances between nodes It is the external manifestation of implicit space-time relationship and explicit space-time relationship, and is the organic combination of the two; and Perform aggregation to obtain a new sparse matrix: in, is the normalization constant, Represents the sparsity matrix preservation threshold.
[0010] The similarity matrix means that each node in the global graph G has many neighbor nodes in both breadth and depth dimensions, and it is necessary to further extract the information of these neighbor nodes to ensure the maximum effectiveness of the information contained in the subgraph after node division; Use the Node2vec node embedding model to extract two features between nodes: Homogeneity, that is, the closer the distance between nodes on the graph, the more similar they are; Isomorphism, that is, if the structures around two nodes are similar, they are more similar; By extracting homogeneity and isomorphism, the low-dimensional node continuous vector contains the neighbor relationship between multi-hop nodes, which can help us further obtain the influence relationship between nodes; control the return parameters and in and out parameters To balance the DFS and BFS strategies, thus generating a node sequence, the node continuous vector extraction process is as follows: Given the current node and the previous node , select the next node The transition probability for: in, is the weight, is the normalization constant, the weight The calculation depends on and : Afterwards, for each node conduct The secondary length is The random walk generates node sequences and uses the generated node sequences to train the Skip-gram model. The Skip-gram model maximizes the node and context node (in The co-occurrence probability within the window size range is used to learn the embedding representation of the node: in, is the set of all generated node pairs, is the conditional probability, expressed using the softmax function: The final output of the model is the node embedding matrix , whose size is ,in is the number of nodes, is the dimension of the embedding vector, each row corresponds to the embedding vector of a node ; In the global graph, there are nodes, and the embedding vector dimension of each node is , then the node embedding matrix It can be expressed as: in, Is a node The embedding vector of These extracted continuous vectors effectively represent the connection relationship and edge weight of each node in a certain dimension. Through algorithm calculation, we can obtain the vector of each node , for the distance between two nodes in the graph Vector similarity calculation, then the node's neighbor similarity matrix can be obtained ,as follows: .
[0011] The community detection partitioning refers to using the similarity matrix We apply the Leiden algorithm for community detection. The Leiden algorithm maximizes the modularity To optimize the community structure and modularity Defined as: in, is the adjacency matrix, and The nodes are and nodes The degree, is the total number of edges in the graph, is the Kronecker delta, when the node and nodes When they belong to the same community, the value is 1; or Node and nodes When they do not belong to the same community, the value is 0; The Leiden algorithm iteratively optimizes the modularity and continuously refines the community structure to generate a densely connected subgraph.
[0012] In step S3, a network traffic prediction model using a spatiotemporal graph convolutional neural network in an acyclic network is constructed, which uses federated learning as a communication framework, an adaptive pruning module as a transmission cost optimization module, and a local training model taking into account the limited training resources of the client. The spatiotemporal graph convolutional neural network includes a spatiotemporal graph convolution block and a final convolution layer. Each spatiotemporal graph convolution block includes a Chebyshev graph convolution layer, a temporal convolution layer, and a residual connection layer. The input signal The shape is ,in, is the batch size, is the number of nodes, is the input feature dimension, is the time step, and the spatiotemporal graph convolution block is composed as follows: Chebyshev graph convolutional layer: We use the Chebyshev polynomial approximation method to calculate the graph convolution. The calculation formula of the graph convolution layer is: in, It is Chebyshev polynomial of order, is the normalized Laplacian operator, It is The filter parameters of the order are The shape is ,in is the output feature dimension; Output of the temporal convolutional layer to the graph convolutional layer Perform convolution operation, the calculation formula is: Output The shape is ,in is the time step after convolution; in the residual connection layer, the input signal And the signal after graph convolution and time convolution Add to retain the information of the input signal; the calculation formula of the residual connection layer is: in, Is the input signal The convolution operation is performed to make its shape Consistent; the shape of the residual connection layer output is ; The final convolutional layer further processes the output of the residual connection layer to obtain the final prediction result. The output shape of the final convolutional layer is ,in is the final time step.
[0013] The federated learning communication framework includes a central server and an edge client; After the model is initialized, it is distributed to each client. After training on the client, the model parameters are transmitted to the central server for aggregation, and then the parameters are distributed until the model converges; Before each model aggregation, an adaptive weight pruning module is set up in the client. The adaptive module has a built-in reinforcement learning model and a dynamic loss window. The model uses the lowest training loss and validation loss in the current round of the local client model and the sliding window round to adaptively adjust the pruning ratio through the reinforcement learning model to protect the model accuracy. The pruned weights of each client are uploaded to the central server.
[0014] The federated learning communication framework is as follows: Assume that there is clients, each client have data samples, the total amount of data is , each client trains a local model and updates its parameters; Then, these parameters are uploaded to the central server, which performs weighted averaging of the parameters based on the amount of data from each client and updates the global model as follows: Model parameters for each client According to the amount of data Weighted to get the global model parameters .
[0015] In step S4, in order to further reduce the computation and communication costs, an adaptive parameter pruning module is introduced; This module dynamically adjusts the pruning ratio according to the change of the loss function during model training, thereby minimizing the transmission and computing overhead while ensuring the accuracy of the model. The details are as follows: Assume that in During the training round, each client The local model parameters are , the pruning ratio is , the model parameters after pruning are ; M1: Calculate the number of weights that need to be retained as follows: in, is the total weight of the model, is the number of weights that need to be retained; M2: Sort by the absolute value of the weights, as follows: Sort by the absolute value of The weight value is used as the threshold ; M3: Apply threshold for pruning, as follows: in, Is an indicator function, when the absolute value of the weight is greater than or equal to the threshold The value is 1 when it is, otherwise it is 0; After each round of training, the minimum loss function memorized by the active loss window is compared with the loss function of the current round. The difference between and pruning interval The reinforcement learning agent model is input together to adaptively adjust the pruning ratio.
[0016] The reinforcement learning agent selects the Q-learning model, which can adaptively adjust the action to obtain the optimal solution based on the relationship between reward and state. The model update algorithm is as follows: in, Indicates the current state. Indicates the current action. is the learning rate, It is the reward of the next moment. is the discount factor.
[0017] The specific adjustment strategy of the adaptive adjustment action is as follows: Action 1: linearly increase the pruning rate and action 2: exponentially decrease the pruning rate, as follows: Action 3: Increase the pruning interval and fine-tune the pruning rate as follows: Action 4: Reduce the pruning interval and fine-tune the pruning rate as follows: Action 5: Increase the pruning rate linearly in small increments and Action 6: Decrease the pruning rate linearly in small increments, as follows: .
[0018] The technical effects achieved by the present invention are: Compared with the currently commonly used partitioning methods based on adjacency matrices or relying only on traffic patterns, the subgraph-driven lightweight federated learning spatiotemporal cellular network prediction method of the present invention proposes a subgraph partitioning strategy that more finely handles the complexity of spatiotemporal relationships in a federated distributed environment. This strategy uses a sparse hybrid relationship matrix, combined with the embedding extraction of homogeneity and isomorphism relationships between nodes and community detection technology, to effectively retain the complex connections between client nodes. Through this method, a high correlation is maintained within the subgraph of each client, significantly improving the accuracy of traffic prediction.
[0019] The subgraph-driven lightweight federated learning spatiotemporal cellular network prediction method of the present invention introduces an adaptive parameter pruning strategy that focuses on the data transmission cost problem faced by the model in a federated environment compared to other prediction models. At the same time, this strategy integrates a reinforcement learning agent to achieve real-time dynamic adjustment of pruning ratios and parameter updates, which not only significantly reduces transmission costs, but also maintains or even improves model prediction accuracy. By comparing and verifying with other models, the present invention shows better performance than existing technologies in capturing spatiotemporal data characteristics, improving prediction accuracy and transmission costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 It is a flowchart of the subgraph-driven lightweight federated learning spatiotemporal cellular network prediction method of the present invention; Figure 2 It is an architectural diagram of the centralized prediction model and the federated distributed prediction model of the present invention; Figure 3 It is a structural diagram of the overall model framework of the subgraph-driven lightweight federated learning spatiotemporal cellular network prediction method of the present invention; Figure 4 It is a bar chart comparing the evaluation indicators MAE and RMSE of the subgraph partitioning strategy of the present invention and the baseline method; Figure 5 It is a bar chart comparing the evaluation indexes MAE and RMSE of SMS in the Telecom Italia dataset of the present invention; Figure 6 It is a bar chart comparing the evaluation indexes MAE and RMSE of calls in the Telecom Italia dataset of the present invention; Figure 7 It is a bar chart comparing the evaluation indexes of MAE and RMSE of the flow in the Telecom Italia dataset of the present invention; Figure 8 It is a bar chart comparing the evaluation indicators of flow data on the Pems07 data set of the present invention; Fig. 9It is a line chart of the present invention based on the actual call data prediction comparison of the Telecom Italia data set. DETAILED DESCRIPTION
[0021] In order to make the purpose and advantages of the present invention more clearly understood, the present invention is specifically described below in conjunction with embodiments. It should be understood that the following text is only used to describe one or several specific embodiments of the present invention, and does not strictly limit the scope of protection of the specific claims of the present invention.
[0022] like Figure 1-9 As shown in the figure, the subgraph-driven lightweight federated learning spatiotemporal cellular network prediction method includes a subgraph partitioning strategy and an adaptive parameter pruning scheme, as follows: First, the subgraph partitioning strategy involves creating a hybrid sparse matrix that captures explicit and implicit spatiotemporal relationships between nodes, which is critical for effectively distributing workloads among clients in a federated learning environment while retaining the necessary relationship information for accurate predictions, as follows: (a) Hybrid sparse matrix; We preprocess the data to obtain a hybrid relationship matrix. The daily traffic of the BS recording node is regarded as a vector. The traffic generated within D days can be divided into a group of vectors by day. For any node That is, there exists a vector group ; Consider the vector group as a probability distribution, and each vector as a probability mass of the probability distribution, and get the node The probability mass Probability distribution of components : The conversion cost between probability distributions of different nodes can be regarded as the difference distance between node traffic patterns. The Wasserstein probability distribution distance is used to calculate the traffic pattern distance between every two nodes, and the cost function It can be expressed as the difference between 1 and the cosine similarity between two vector groups: Further get the node With Node The Wasserstein probability distribution distance matrix between , the probability distribution distance between every two nodes for: in, is a joint probability distribution that satisfies the following marginal constraints: in, and The nodes are and nodes The probability distribution of Traffic pattern relationship matrix between nodes represented by probability distribution distance and the adjacency matrix dominated by the actual distances between nodes It is the external manifestation of implicit and explicit space-time relationships, and is an organic combination of the two, which enables nodes to obtain more relationships; Adjacency Matrix and Perform aggregation to obtain a new sparse matrix: in, is the normalization constant, represents the sparsity matrix retention threshold; (b) extract low-dimensional information and obtain the adjacency similarity matrix; Each node in the global graph G has many neighbor nodes in both breadth and depth. It is necessary to further extract the information of these neighbor nodes to ensure the maximum effectiveness of the information contained in the subgraph after node division. We use the Node2vec node embedding model to focus on extracting two features between nodes: (I) Homogeneity, that is, the closer the distance between nodes on the graph, the more similar they are; (II) Isomorphism, that is, if the structures around two nodes are similar, they are more similar; By extracting homogeneity and isomorphism, the low-dimensional node continuous vector contains the neighbor relationship between multi-hop nodes, which can help us further obtain the influence relationship between nodes; control the return parameters and in and out parameters To balance the DFS (depth first search) and BFS (breadth first search) strategies, thus generating a node sequence, the node continuous vector extraction process is as follows: Given the current node and the previous node , select the next node The transition probability for: in, is the weight, is the normalization constant, the weight The calculation depends on and : Afterwards, for each node conduct The secondary length is The random walk generates node sequences and uses the generated node sequences to train the Skip-gram model. The Skip-gram model maximizes the node and context node (in The co-occurrence probability within the window size range is used to learn the embedding representation of the node: in, is the set of all generated node pairs, is the conditional probability, expressed using the softmax function: The final output of the model is the node embedding matrix , whose size is ,in is the number of nodes, is the dimension of the embedding vector, each row corresponds to the embedding vector of a node ; In the global picture There are nodes, and the embedding vector dimension of each node is , then the node embedding matrix It can be expressed as: in, Is a node The embedding vector of These extracted continuous vectors effectively represent the connection relationship and edge weight of each node in a certain dimension. Through algorithm calculation, we can obtain the vector of each node , for the distance between two nodes in the graph Vector similarity calculation, then the node's neighbor similarity matrix can be obtained ,as follows: (c) Community detection node division to obtain a subgraph with higher density; Using Similarity Matrix We apply the Leiden algorithm for community detection. The Leiden algorithm maximizes the modularity To optimize the community structure and modularity Defined as: in, is the adjacency matrix, and The nodes are and nodes The degree, is the total number of edges in the graph, is the Kronecker delta function, when the node and nodes When they belong to the same community, the value is 1, otherwise it is 0; The Leiden algorithm iteratively optimizes modularity and continuously refines the community structure, thereby generating densely connected subgraphs; Based on the divided communities, we obtain the subgraph ,in is the required number of clients; each subgraph Contains more tightly connected nodes; Traditional federated learning prediction frameworks are generally equipped with simple linear neural network models such as ANN or LSTM. However, with the development of technology, the requirements for prediction accuracy have brought more complex models. The basis of the graph neural network model is the graph structure between nodes, which is the basis for model training. In a distributed architecture, it is manifested in the importance of dividing the subgraph set, which has been achieved in the previous article. While giving better performance, the model based on the graph neural network emphasizes the importance of device computing power. In order to achieve better performance while minimizing computational and transmission costs, we introduced an adaptive fine-grained pruning module that can keep the granularity of the pruned part at the parameter level. In addition to reducing the parameter cost of data transmission between the client and the server, the pruned parameters can also effectively reduce the complexity of model training due to the parameter characteristics of the model, thereby simplifying the model training time. The adaptive mechanism based on the reinforcement learning agent supports the model to achieve the above advantages while ensuring training accuracy. Specifically, we built a network traffic prediction model with federated learning as the communication framework, an adaptive pruning module as the transmission cost optimization module, and a local training model that uses a spatiotemporal graph convolutional neural network in an acyclic network, considering that the client training resources are limited; The spatiotemporal graph convolutional neural network consists of a spatiotemporal graph convolutional block and a final convolutional layer. Each spatiotemporal graph convolutional block consists of a Chebyshev graph convolutional layer, a temporal convolutional layer, and a residual connection layer. The shape is ,in, is the batch size, is the number of nodes, is the input feature dimension, is the time step, and the spatiotemporal graph convolution block is composed as follows: Chebyshev graph convolutional layer: We use the Chebyshev polynomial approximation method to calculate the graph convolution. The calculation formula of the graph convolution layer is: in, It is Chebyshev polynomial of order, is the normalized Laplacian operator, It is The filter parameters of the order are The shape is ,in is the output feature dimension; Output of the temporal convolutional layer to the graph convolutional layer Perform convolution operation, the calculation formula is: Output The shape is ,in is the time step after convolution; in the residual connection layer, the input signal And the signal after graph convolution and time convolution Add to retain the information of the input signal; the calculation formula of the residual connection layer is: in, Is the input signal The convolution operation is performed to make its shape Consistent; the shape of the residual connection layer output is ; The final convolution layer further processes the output of the residual connection layer to obtain the final prediction result. The output shape of the final convolution layer is ,in is the final time step; We have built a federated learning communication framework with a central server and edge clients. After the model is initialized, it is distributed to each client. After training on the client, the model parameters are transmitted to the central server for aggregation, and then the parameters are distributed until the model converges. Before each model aggregation, an adaptive weight pruning module is set up in the client. The adaptive module has a built-in reinforcement learning model and a dynamic loss window. The model uses the local client's current round model and the lowest training loss and verification loss in the sliding window round. The reinforcement learning model adaptively adjusts the pruning ratio to protect the model accuracy. The pruned weights of each client are uploaded to the central server.
[0023] The federated learning communication framework is as follows: Assume that clients, each client have data samples, the total amount of data is , each client trains a local model and updates its parameters, and then uploads these parameters to the central server. The server performs a weighted average of the parameters based on the amount of data from each client and updates the global model as follows: Model parameters for each client According to the amount of data Weighted to get the global model parameters ; Second, in order to further reduce the computation and communication costs, an adaptive parameter pruning module is introduced; this module dynamically adjusts the pruning ratio according to the change of the loss function during the model training process, thereby minimizing the transmission and computation overhead while ensuring the accuracy of the model. The details are as follows: Assume that in During the training round, each client The local model parameters are , the pruning ratio is , the model parameters after pruning are ; 1. Calculate the number of weights that need to be retained: in, is the total weight of the model, is the number of weights that need to be retained; 2. Sort by the absolute value of the weight: model parameters Sort by the absolute value of The weight value is used as the threshold ; 3. Apply threshold for pruning: in, Is an indicator function, when the absolute value of the weight is greater than or equal to the threshold The value is 1 when it is, otherwise it is 0; After each round of training, the minimum loss function memorized by the active loss window is compared with the loss function of the current round. The difference between and pruning interval The reinforcement learning agent model is input together to adaptively adjust the pruning ratio; The reinforcement learning agent selects the Q-learning model, which can adaptively adjust the action to obtain the optimal solution based on the relationship between reward and state. The model update algorithm is as follows: in, Indicates the current state. Indicates the current action. is the learning rate, It is the reward of the next moment. is the discount factor; The specific adjustment strategies for the actions are as follows: Action 1: linearly increase the pruning rate and action 2: exponentially decrease the pruning rate, as follows: Action 3: Increase the pruning interval and fine-tune the pruning rate as follows: Action 4: Reduce the pruning interval and fine-tune the pruning rate as follows: Action 5: Increase the pruning rate linearly in small increments and Action 6: Decrease the pruning rate linearly in small increments, as follows: .
[0024] The above is only a preferred embodiment of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention. The structures, devices and operating methods not specifically described and explained in the present invention shall be implemented according to the conventional means in the art unless otherwise specified and limited.
Claims
1. A subgraph-driven lightweight federated learning spatiotemporal cellular network prediction method, characterized by: The following steps are involved: S1. Data preprocessing: Select the cellular network dataset for cleaning and remove invalid or missing data points; The data is processed into time series input, each sequence contains 12 time steps of historical data for predicting the traffic situation in the next time step; S2, subgraph partitioning: extract the adjacency distance matrix and node traffic pattern matrix between nodes to construct a hybrid sparse matrix, extract the low-dimensional continuous vector representing the multi-hop neighbor structure of the node through graph embedding, which is used to represent the complex dependency relationship between nodes and obtain the similarity matrix, and perform community detection on the similarity matrix to obtain the final subgraph; S3, model training: perform spatiotemporal graph convolutional neural network model training on the local client on the divided subgraphs to obtain local model parameters; S4, model parameter pruning: adaptively prune the weight parameters of the trained model, adjust the pruning strategy through the reinforcement learning agent, and obtain the pruned weight parameters; S5, model transmission: the client transmits the pruned model parameters to the server; S6, parameter aggregation: The server obtains the parameters passed by each client and performs federated average aggregation; S7, model update: The server sends the aggregated new parameters back to the client, and the client obtains the returned parameters for the next round of spatiotemporal graph convolutional neural network model training until the iteration round ends.
2. The subgraph-driven lightweight federated learning spatiotemporal cellular network prediction method according to claim 1, characterized in that: The hybrid sparse matrix regards the daily traffic of the BS record node as a vector, and the traffic generated within D days is divided into a group of vectors by day. That is, there exists a vector group ; Consider the vector group as a probability distribution, and each vector as a probability mass of the probability distribution, and get the node The probability mass Probability distribution of components : The conversion cost between probability distributions of different nodes is regarded as the difference distance between node traffic patterns. The Wasserstein probability distribution distance is used to calculate the traffic pattern distance between every two nodes, and the cost function Expressed as the difference between 1 and the cosine similarity between two vector groups: Further get the node With Node The Wasserstein probability distribution distance matrix between , the probability distribution distance between every two nodes for: in, is a joint probability distribution that satisfies the following marginal constraints: in, and The nodes are and nodes The probability distribution of; the traffic pattern relationship matrix between nodes represented by the probability distribution distance and the adjacency matrix dominated by the actual distances between nodes It is the external manifestation of implicit space-time relationship and explicit space-time relationship, and is the organic combination of the two; and Perform aggregation to obtain a new sparse matrix: in, is the normalization constant, Represents the sparsity matrix preservation threshold.
3. The subgraph-driven lightweight federated learning spatiotemporal cellular network prediction method according to claim 1, characterized in that: The similarity matrix means that each node in the global graph has many neighbor nodes in both breadth and depth. It is necessary to further extract the information of these neighbor nodes to ensure the maximum effectiveness of the information contained in the subgraph after node division. The Node2vec node embedding model is used to extract two characteristics between nodes: Homogeneity, where the closer the distance between nodes on the graph, the more similar they are; Isomorphism, which means that if the structures around two nodes are similar, they are more similar; by extracting homogeneity and isomorphism, the low-dimensional node continuous vector contains the neighbor relationship between multi-hop nodes, which helps to further obtain the influence relationship between nodes; control the return parameters and in and out parameters To balance the DFS and BFS strategies, thus generating a node sequence, the node continuous vector extraction process is as follows: Given the current node and the previous node , select the next node The transition probability for: in, is the weight, is the normalization constant, the weight The calculation depends on and : Afterwards, for each node conduct The secondary length is The random walk generates node sequences and uses the generated node sequences to train the Skip-gram model. The Skip-gram model maximizes the node and context node (in The co-occurrence probability within the window size range is used to learn the embedding representation of the node: in, is the set of all generated node pairs, is the conditional probability, expressed using the softmax function: The final output of the model is the node embedding matrix , whose size is ,in is the number of nodes, is the dimension of the embedding vector, each row corresponds to the embedding vector of a node ; In the global graph, nodes, and the embedding vector dimension of each node is , then the node embedding matrix It is expressed as: in, Is a node Embedded vectors of; these extracted continuous vectors effectively represent the connection relationship and edge weight of each node in a certain dimension. Through algorithm calculation, the vector of each node is obtained , for the distance between two nodes in the graph Vector similarity calculation is performed to obtain the node's neighbor similarity matrix ,as follows: 。 4. The subgraph-driven lightweight federated learning spatiotemporal cellular network prediction method according to claim 1, characterized in that: The community detection partitioning refers to using the similarity matrix , the Leiden algorithm is used for community detection. The Leiden algorithm maximizes the modularity To optimize the community structure and modularity Defined as: in, is the adjacency matrix, and The nodes are and nodes The degree, is the total number of edges in the graph, is the Kronecker delta, when the node and nodes When they belong to the same community, the value is 1; or and nodes When they do not belong to the same community, the value is 0; the Leiden algorithm iteratively optimizes the modularity and continuously refines the community structure to generate a densely connected subgraph.
5. The subgraph-driven lightweight federated learning spatiotemporal cellular network prediction method according to claim 1, characterized in that: In step S3, a network traffic prediction model using a spatiotemporal graph convolutional neural network in an acyclic network is constructed, which uses federated learning as a communication framework, an adaptive pruning module as a transmission cost optimization module, and a local training model taking into account the limited training resources of the client. The spatiotemporal graph convolutional neural network includes a spatiotemporal graph convolution block and a final convolution layer. Each spatiotemporal graph convolution block includes a Chebyshev graph convolution layer, a temporal convolution layer, and a residual connection layer. The input signal The shape is ,in, is the batch size, is the number of nodes, is the input feature dimension, is the time step, and the spatiotemporal graph convolution block is composed as follows: Chebyshev graph convolution layer: The Chebyshev polynomial approximation method is used to calculate graph convolution. The calculation formula of the graph convolution layer is: in, It is Chebyshev polynomial of order, is the normalized Laplacian operator, It is The filter parameters of the order are The shape is ,in is the output feature dimension; Output of the temporal convolutional layer to the graph convolutional layer Perform convolution operation, the calculation formula is: Output The shape is ,in is the time step after convolution; In the residual connection layer, the input signal And the signal after graph convolution and time convolution Add to retain the information of the input signal; the calculation formula of the residual connection layer is: in, Is the input signal The convolution operation is performed to make its shape Consistent; the shape of the residual connection layer output is ; The final convolutional layer further processes the output of the residual connection layer to obtain the final prediction result. The output shape of the final convolutional layer is ,in is the final time step.
6. The subgraph-driven lightweight federated learning spatiotemporal cellular network prediction method according to claim 5, characterized in that: The federated learning communication framework includes a central server and an edge client; After the model is initialized, it is distributed to each client. After training on the client, the model parameters are transmitted to the central server for aggregation, and then the parameters are distributed until the model converges; Before each model aggregation, an adaptive weight pruning module is set up in the client. The adaptive module has a built-in reinforcement learning model and a dynamic loss window. The model uses the lowest training loss and validation loss in the current round of the local client model and the sliding window round to adaptively adjust the pruning ratio through the reinforcement learning model to protect the model accuracy. The pruned weights of each client are uploaded to the central server.
7. The subgraph-driven lightweight federated learning spatiotemporal cellular network prediction method according to claim 1, characterized in that: The federated learning communication framework is as follows: Assume that there is clients, each client have data samples, the total amount of data is , each client trains a local model and updates its parameters; Then, these parameters are uploaded to the central server, which performs weighted averaging of the parameters based on the amount of data from each client and updates the global model as follows: Model parameters for each client According to the amount of data Weighted to get the global model parameters .
8. The subgraph-driven lightweight federated learning spatiotemporal cellular network prediction method according to claim 1, characterized in that: In step S4, in order to further reduce the computation and communication costs, an adaptive parameter pruning module is introduced; This module dynamically adjusts the pruning ratio according to the change of the loss function during model training, thereby minimizing the transmission and computing overhead while ensuring the accuracy of the model. The details are as follows: Assume that in During the training round, each client The local model parameters are , the pruning ratio is , the model parameters after pruning are ; M1: Calculate the number of weights that need to be retained as follows: in, is the total weight of the model, is the number of weights that need to be retained; M2: Sort by the absolute value of the weight, as follows: For model parameters Sort by the absolute value of The weight value is used as the threshold ; M3: Apply threshold for pruning, as follows: in, Is an indicator function, when the absolute value of the weight is greater than or equal to the threshold The value is 1 when it is, otherwise it is 0; After each round of training, the minimum loss function memorized by the active loss window is compared with the loss function of the current round. The difference between and pruning interval The reinforcement learning agent model is input together to adaptively adjust the pruning ratio.
9. The subgraph-driven lightweight federated learning spatiotemporal cellular network prediction method according to claim 1, characterized in that: The reinforcement learning agent selects the Q-learning model, which adaptively adjusts the action to obtain the optimal solution based on the relationship between reward and state. The model update algorithm is as follows: in, Indicates the current state. Indicates the current action. is the learning rate, It is the reward of the next moment. is the discount factor.
10. The subgraph-driven lightweight federated learning spatiotemporal cellular network prediction method according to claim 9, characterized in that: The specific adjustment strategy of the adaptive adjustment action is as follows: Action 1: linearly increase the pruning rate and action 2: exponentially decrease the pruning rate, as follows: Action 3: Increase the pruning interval and fine-tune the pruning rate as follows: Action 4: Reduce the pruning interval and fine-tune the pruning rate as follows: Action 5: Increase the pruning rate linearly in small increments and Action 6: Decrease the pruning rate linearly in small increments, as follows: 。
Citation Information
Patent Citations
Traffic prediction model training method and device based on federated learning, and traffic prediction method and device based on federated learning
CN115580891A
Network traffic prediction method based on time-space diagram attention mechanism
CN116170351A
Efficient federated learning system and method based on channel pruning
CN117592538A
Traffic state prediction method based on adaptive dynamic space-time diagram convolutional network
CN118116194A
Cooperative broadband spectrum sensing method based on federal pruning transfer learning and secondary Nyquist sampling
CN118555574A
Cited By
Network traffic anomaly detection method for vehicle-network interaction
CN119676747A
CP-GAT enterprise potential supply relation prediction method and related device
CN120317456A
