Node optimization method of communication network

Abstract

The embodiment of the invention provides a node optimization method for a communication network. The node optimization method comprises the following steps: optimizing a node utilization problem of the communication network into a maximum independent set problem; the maximum independent set problem is converted into a QUBO model; a corresponding QUBO graph is constructed for the QUBO model; a graph neural network is used to represent the structure and characteristics of the QUBO graph, and a deep learning model CONQUER is constructed; the deep learning model CONQUER is trained to explore the optimal solution of the utilization problem of the nodes of the communication network, and the trained deep learning model CONQUER is obtained. The independent nodes or groups in the network can be quickly and accurately optimized. And the connectivity of a communication network and the information transmission efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the field of node optimization of communication networks, and in particular to a node optimization method of communication networks. Background Art

[0002] As communication technology matures and people's reliance on wireless connections continues to increase, the configuration of independent nodes in communication networks may affect the flow of information and the maintenance of network quality. For example, in wireless network design, identifying and optimizing potential independent nodes can help engineers configure the network more accurately, improve data transmission efficiency, and reduce signal interference. In addition, in network structure analysis, by deeply studying the connection relationship and distribution between nodes, it is possible to optimize the setting of independent nodes and understand their impact on network performance, providing an important basis for the optimization and management of communication networks.

[0003] In this context, the optimization of independent nodes in communication networks has become a key task, which is directly related to the operation efficiency of communication networks and the reliability of information transmission, and aims to maximize the connectivity of communication networks and the efficiency of information transmission. By analyzing the network structure, optimizing the configuration of independent nodes and taking corresponding measures, the stability and reliability of the network can be improved, providing users with a better connection experience. Therefore, the optimization and analysis of independent nodes in communication networks has important practical significance and application prospects. Summary of the invention

[0004] The embodiment of the present invention provides a node optimization method for a communication network, which can solve the problem of optimizing the network configuration of independent nodes in the communication network in the prior art.

[0005] To achieve the above object, on the one hand, an embodiment of the present invention provides a node optimization method for a communication network, comprising:

[0006] Optimizing the utilization problem of nodes of a communication network as a maximum independent set problem, the nodes including: terminal nodes, and transit nodes for converting and exchanging information;

[0007] Convert the maximum independent set problem into a QUBO model;

[0008] Constructing a corresponding QUBO graph for the QUBO model;

[0009] A graph neural network is used to characterize the structure and characteristics of the QUBO graph, and a deep learning model CONQUER is constructed;

[0010] The deep learning model CONQUER is trained to explore the optimal solution to the node utilization problem of the communication network, and a trained deep learning model CONQUER is obtained; the trained deep learning model CONQUER is used to optimize the node utilization problem of any communication network to obtain a node optimization result, and the node optimization result includes: the nodes used, the number of nodes, and the connection relationship between the nodes.

[0011] On the other hand, an embodiment of the present invention provides a node optimization method for a communication network, further comprising:

[0012] Input the node utilization problem of the communication network to be solved into the previously trained deep learning model CONQUER;

[0013] Output node optimization results, which include: adopted nodes, number of nodes, and connection relationships between nodes.

[0014] The above technical solution has the following beneficial effects: As a framework widely used in quantum annealing and other optimization algorithms, the QUBO model shows great potential for solving the maximum independent set problem. Although the QUBO model can effectively handle static optimization problems, the dynamic nature of communication networks requires optimization algorithms to adapt to environmental changes and make adjustments in real time. The main advantage of CONQUER is that it does not need to adjust the deep learning architecture for the key communication line identification problem in a specific scenario, thereby reducing the cost of problem customization. Experimental results show that although CONQUER is only trained on small-scale synthetic graphs, it exhibits excellent generalization ability and can easily cope with the challenges of large-scale graphs. By adopting a simple fine-tuning strategy, CONQUER can be effectively adapted to real-world data sets. In practical applications, especially in the optimization task of independent nodes in communication networks, the CONQUER algorithm has demonstrated its excellent performance and can quickly and accurately optimize independent nodes or groups in the network. Improve the connectivity of communication networks and the efficiency of information transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 This is an embodiment of a node optimization method for a communication network according to an embodiment of the present invention;

[0017] Figure 2It is a solution framework of the QUBO model of the embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] like Figure 1 As shown, in combination with an embodiment of the present invention, a node optimization method for a communication network is provided, comprising:

[0020] S101: Optimizing the utilization problem of nodes of a communication network into a maximum independent set problem, wherein the nodes include: terminal nodes, and switching nodes for converting and exchanging information;

[0021] S102: Convert the maximum independent set problem into a QUBO model;

[0022] S103: constructing a corresponding QUBO graph for the QUBO model;

[0023] S104: Using a graph neural network to characterize the structure and characteristics of the QUBO graph, and constructing a deep learning model CONQUER;

[0024] S105: The deep learning model CONQUER is trained to explore the optimal solution to the node utilization problem of the communication network, and a trained deep learning model CONQUER is obtained; the trained deep learning model CONQUER is used to optimize the node utilization problem of any communication network to obtain a node optimization result, and the node optimization result includes: the nodes used, the number of nodes, and the connection relationship between the nodes.

[0025] Preferably, the utilization problem of nodes of the communication network is optimized as a maximum independent set problem, comprising:

[0026] The communication network is represented as G = (V, E), where V refers to the node set, including all nodes in the communication network, and E refers to the edge set, including all edges in the communication network. ij ∈E, edge e ij ∈E indicates that there is a connection relationship between nodes i and j;

[0027] The problem of utilizing the nodes of the communication network is transformed into a corresponding maximum independent set problem, so as to find a non-interfering node subset through the maximum independent set problem. The number of nodes in the non-interfering node subset is maximized, and the maximum independent set problem is expressed as:

[0028]

[0029] Where X = (x1, x2.....x n ) represents the decision vector of whether a node is selected as a non-interference node. A non-interference node refers to a node whose communications do not interfere with each other.

[0030] Preferably, the maximum independent set problem is transformed into a QUBO model, comprising:

[0031] By adding a penalty term to the maximum independent set problem constraint, the maximum independent set problem is converted into a QUBO model, and the QUBO model is expressed as:

[0032]

[0033] Where, X=(x1,x2……x n ) represents the decision vector composed of nodes, x i 、x j are the values ​​of the i-th and j-th nodes, respectively, indicating whether these nodes are selected into the non-interference node subset. The matrix Q is a symmetric matrix or an upper triangular matrix. i =Q ii ,so b ij =Q ij , a i represents the node weight in the QUBO model, b ij Represents the edge weight on the QUBO model.

[0034] Preferably, constructing a corresponding QUBO diagram for the QUBO model comprises:

[0035] A corresponding QUBO graph is constructed for the QUBO model, and the mathematical representation of the QUBO graph is:

[0036]

[0037] Where p represents the penalty function.

[0038] Preferably, a graph neural network is used to characterize the structure and characteristics of the QUBO graph, and a deep learning model CONQUER is constructed, including:

[0039] Constructing an encoder, wherein the encoder construction specifically includes: integrating a centrality encoding layer, a message passing layer, a Transformer layer, and an iterative calculation solution layer; wherein:

[0040] At the integrated centrality encoding layer:

[0041] Each node in the QUBO graph is enhanced by the corresponding degree to capture the importance of the node in the QUBO graph, and then encoded. Each node in the QUBO graph is enhanced by the corresponding degree and encoded to obtain the corresponding vector representation:

[0042]

[0043] Among them, ReLU refers to the activation function, The i in the equation represents the i-th node, 0 represents the initial feature, and x i is the initial feature of the node, z deg (v i ) encodes the degree of the node, v i It refers to the node with serial number i, W1 and W2 are the parameters of the neural network, and they are learnable parameters;

[0044] At the messaging layer:

[0045] The neighborhood aggregation process is adopted: edge feature aggregation and node feature aggregation to merge node features and edge features respectively. The edge feature aggregation and node feature aggregation are expressed as follows:

[0046]

[0047]

[0048] Among them, h ij =ReLU(W3x ij ) is the initial edge embedding vector, which is the intermediate output of the neural network, that is, embedding the initial features of the edge into the high-dimensional space, x ij is the initial feature of the edge, i.e., the input feature of the edge. W3, W4, W5, W6, and W7 are the parameters of the neural network, and they are all learnable parameters.

[0049] After obtaining the node features in step l After that, by splicing the features of the previous step of the node Residual connection, the residual connection is expressed as follows:

[0050]

[0051] Among them, W8 is the parameter of the neural network and is a learnable parameter;

[0052] In the Transformer layer:

[0053] Aggregate the output of the message passing layer with the output of the global attention layer in the Transformer layer to obtain updated node features:

[0054]

[0055]

[0056] Among them, MHA represents multi-head attention network, LN represents layer normalization, and FFN represents feed-forward neural network layer. represents the node embedding vector obtained at step l-1, represents the node embedding vector after the multi-head attention network;

[0057] In the iterative calculation solution layer:

[0058] When solving, the updated node features output from the Transformer layer are used as the input of the integrated centrality encoding layer, and the steps of the integrated centrality encoding layer, the message passing layer, and the Transformer layer are repeatedly executed in a loop to iterate. After the iteration is completed, the final embedding vector corresponding to each node is obtained, and the final embedding vector of each node is expressed as:

[0059] Sum the final embedding vectors of all nodes to get: s =∑ i∈V z i , the solution of the QUBO model is characterized by summation.

[0060] Preferably, a graph neural network is used to characterize the structure and characteristics of the QUBO graph, and a deep learning model CONQUER is constructed, which also includes:

[0061] Build the decoder, including:

[0062] Formally, a Markov decision process is formed, which is described by the tuple (S, A, P, R), where S is all possible states s i A collection of actions i ∈A represents the assignment of the binary variable X of the QUBO model for a given state si;

[0063] In status i Execute action a i Get reward i :r i =a i +∑ (i,j)∈E b ij x j ∈R,, represents the change of the objective function before and after the execution of the action Markov decision process;

[0064] Through the decoder, a pair of state-action pairs (s i , a i ) is converted to a scalar Q(s i ,a i ; Θ), to evaluate i Next action a i The quality of the decoder is expressed as follows:

[0065] Q(h,v;Θ)=W 11 ReLU(W 10 ReLU(W9[z s ,z i ]))

[0066] Among them, W9, W10, W11 are the parameters of the neural network and can be learned. s ,z i ] means to convert the vector z s and vector z i Join by rows.

[0067] Preferably, the deep learning model CONQUER is trained to explore the optimal solution to the problem of node utilization of the communication network, and the trained deep learning model CONQUER is obtained, including:

[0068] Obtain a sample of nodes of the communication network corresponding to the maximum independent set problem, and generate a trajectory based on the Markov decision process representation based on the QUBO graph; wherein a training round is a complete process of finding a solution that minimizes the Hamiltonian on the QUBO graph, requiring that the states of all decision variables are sequentially flipped from 0 to 1; the trajectory generated in an episode is a state-action-reward sequence: s0,a0,r0,s1,a1,r1,...,s T , s0 is the initial state where all decision variables are 0, s T is the terminal state where all decision variables are 1; completing one round requires a complete training round on the QUBO graph, and a complete training round includes the generation of trajectories and the collection of experience transformations. t ,a t ,r t,t+n ,s t+n , r t+n represents the cumulative reward at time t+n, the delayed reward step n is a hyperparameter, and the discount factor γ is a hyperparameter;

[0069] Using the experience transformations selected from the storage structure, the deep learning model CONQUER is enabled to learn and update the network parameters Θ of the deep learning model CONQUER to minimize the following loss function:

[0070]

[0071] Among them, E represents the maximum expectation, r t,t+n represents the immediate reward after n steps after executing the action at time t, s t+n represents the state at time t+n, a t+n Indicates the action to be performed at t+n. Indicates that in state s t+n The target network for execution is The action a with the largest return value t+n The reward estimate returned after is the target network parameter; Q(s t ,a t ; Θ) indicates that in state s t Execute action a t According to the reward valuation obtained by the original network Q(Θ), s t represents the state at time t, a t represents the action executed at time t, Θ is the parameter of the original network, which updates each parameter through real-time data; the original network Q(Θ) will periodically copy the parameter Θ to the target network Parameters To achieve network updates;

[0072] When the loss function value is less than the preset value, the trained deep learning model CONQUER can be obtained.

[0073] In particular, the node optimization method of the communication network further includes:

[0074] In the obtained sample of nodes of the communication network corresponding to the maximum independent set problem, noise is introduced into the sample when constructing the QUBO graph;

[0075] When the loss function is less than a preset value, it is also necessary to freeze the parameters of the encoder and fine-tune the parameters of the encoder for the corresponding deep learning model CONQUER to obtain a trained deep learning model CONQUER.

[0076] Preferably, the communication network node optimization method further includes:

[0077] When the trained deep learning model CONQUER is used to solve the node utilization problem of the communication network to be solved, when the obtained solution does not meet the feasibility conditions of the communication network to be solved, a greedy search method or a local search method is used to repair the solution.

[0078] In conjunction with an embodiment of the present invention, a method for optimizing nodes in a communication network is provided, further comprising:

[0079] Inputting the node utilization problem of the communication network to be solved into any of the trained deep learning models CONQUER described above;

[0080] Output node optimization results, which include: adopted nodes, number of nodes, and connection relationships between nodes.

[0081] The above technical solution of the embodiment of the present invention is described in detail below in conjunction with specific application examples. For technical details not introduced during the implementation process, please refer to the relevant description in the previous text.

[0082] The embodiment of the present invention provides a node optimization algorithm for a communication network based on QUBO and deep reinforcement learning. The efficiency and performance of the communication network depend on the optimization of its topological structure. The maximum independent set problem is used to find a maximum set of vertices from a graph so that any two vertices in the maximum set of vertices are not adjacent. In a communication network, the maximum independent set can be used to determine the maximum set of non-interfering nodes in a wireless communication network, thereby optimizing the spectrum utilization of each node in the communication network, thereby reducing conflicts in the use of each node. Although traditional algorithms such as greedy algorithms and local searches can find better solutions in some scenarios, they often perform poorly in large-scale complex networks. In recent years, the development of quantum computing and machine learning has provided new solutions to this problem.

[0083] The optimization of independent nodes in a communication network is usually modeled as a maximum independent set problem, which aims to optimize independent nodes in the network without affecting the connectivity of the network. The maximum independent set problem is a typical NP-hard problem. Traditional operations research optimization methods often fall into the so-called trade-off dilemma when solving the maximum independent set problem: algorithms with high solution quality often have slow solution speed, while algorithms with fast solution speed often have low solution quality.

[0084] In recent years, machine learning methods designed based on the special structure of the problem have shown outstanding advantages in solution speed and solution quality, and have strong generalization capabilities, gradually becoming the mainstream algorithm for solving the maximum independent set problem. However, in the actual modeling process, the maximum independent set problem in different situations will present different mathematical forms using different modeling methods, which makes it impossible for machine learning methods to take into account the maximum independent set problem in various mathematical forms.

[0085] In order to solve this problem, the maximum independent set problem in various situations is uniformly described as a QUBO model (quadratic unconstrained binary optimization), and the corresponding QUBO graph is constructed, so that the solution of the maximum independent set problem is converted into an energy minimization problem on the QUBO graph. Reinforcement learning provides a powerful framework for continuously improving decision-making strategies by learning from interactions with the environment to explore the optimal solution to the problem. Combining reinforcement learning with the QUBO model, an optimization algorithm that dynamically adjusts strategies can be constructed to respond to changes in network status in real time (network status refers to the ability of various parameters and conditions in the network to change over time, and these changes may include the number of nodes and the connection relationship between nodes). Therefore, the deep reinforcement learning framework of CONQUER is used to solve the maximum independent set problem, which mainly includes the following four key steps:

[0086] (1) Problem transformation: The maximum independent set problem in various situations is uniformly described as a QUBO model, and the corresponding QUBO diagram is constructed.

[0087] (2) Graph representation learning: Design a deep learning-based graph transformer model to characterize the structure and characteristics of the QUBO graph.

[0088] (3) Solution space exploration based on reinforcement learning: DQN is used to explore the solution space and minimize the energy on the QUBO graph.

[0089] (4) Checking and repairing the solution: Check and repair the found solution to ensure that it is the solution to the maximum independent set problem.

[0090] Compared with traditional methods, the QUBO model, as a framework widely used in quantum annealing and other optimization algorithms, shows great potential in solving the maximum independent set problem. Although the QUBO model can effectively handle static optimization problems, the dynamic nature of communication networks requires optimization algorithms to adapt to environmental changes and make real-time adjustments.

[0091] The main advantage of CONQUER is that it does not require the deep learning architecture to be adjusted for the critical communication line identification problem in a specific scenario, thus reducing the cost of problem customization. Experimental results show that although CONQUER is only trained on small-scale synthetic graphs, it exhibits excellent generalization ability and can easily cope with the challenges of large-scale graphs. By adopting a simple fine-tuning strategy, CONQUER can be effectively adapted to real-world datasets. In practical applications, especially in the optimization task of independent nodes in communication networks, the CONQUER algorithm has demonstrated its excellent performance and can quickly and accurately optimize independent nodes or groups in the network. In communication networks, identifying independent nodes is an important problem, which is directly related to the connectivity of the network and the efficiency of information transmission. This problem can be abstractly modeled as the maximum independent set problem, that is, finding a maximum set of nodes in a graph where no two nodes are adjacent. Due to the NP-hard nature of the maximum independent set problem, traditional heuristic methods and integer programming algorithms rely on hand-crafted heuristics to make decisions, which are computationally too time-consuming and have low feasibility.

[0092] 1. Maximum Independent Set Problem

[0093] Generally, the problem of identifying independent nodes in a communication network is often modeled as the maximum independent set problem, the purpose of which is to find the largest set of isolated nodes in the network while ensuring the connectivity of the communication network. Formally, the communication network G = (V, E) is usually composed of a node set V representing various types of communication units and an edge set E indicating that these nodes are connected (that is, there are edges when there is communication between nodes). Each edge e ij ∈E indicates that there is a connection between nodes i and j. The goal is to find a subset of isolated nodes (non-interfering nodes) Make the number of nodes maximum. Therefore, the mathematical form of this problem is as follows:

[0094]

[0095] Where, X=(x1,x2……x n ) represents the decision vector of whether a node is selected as an isolated node.

[0096] The connection relationship between nodes refers to whether there is an edge between two nodes, and the edge represents the connection between nodes. The maximum independent set means that two nodes will not be covered by the same edge.

[0097] Non-interference nodes in wireless networks usually refer to those nodes that do not send signals in a specific frequency band, thereby avoiding interference with other nodes that are sending or receiving signals. In wireless communications, spectrum resources are precious. In order to avoid mutual interference and improve the overall performance and spectrum efficiency of the network, some nodes in the network do not send data at a specific time or on a specific frequency band, which can reduce co-frequency interference and improve the capacity and coverage of the network. In a wireless local area network (WLAN), non-interference nodes may refer to access points or client devices that can automatically avoid sending data when they hear that other nodes are transmitting to reduce collisions and retransmissions. In a cellular network, non-interference nodes may refer to base stations or mobile devices that do not transmit data on a specific frequency. In fact, here "non-interference nodes" are nodes that do not interfere with each other in communication. Nodes include transfer nodes and terminal nodes that convert and exchange information. Independent nodes are arranged in the network so that the nodes of the entire network do not interfere with each other in communication.

[0098] 2. QUBO Form of the Maximum Independent Set Problem

[0099] QUBO, or quadratic unconstrained extreme optimization problem, contains linear terms and quadratic terms. By adding penalty terms to the constraints, the mathematical form of the combinatorial optimization problem can be converted to QUBO. Specifically, the QUBO problem is formalized as:

[0100]

[0101] Where, X=(x1,x2……x n ) represents the decision vector composed of nodes, x i 、x j Respectively represent the values ​​of the i-th and j-th nodes, indicating whether these nodes are selected into the non-interference node subset. The matrix Q is usually a symmetric or upper triangular matrix and a i =Q ii ,so b ij =Q ij , a i represents the node weight in the QUBO graph, b ij represents the edge weight on the QUBO graph, Q ij is the value in the matrix Q, and the objective function value of QUBO (i.e., maximizing the number of isolated nodes (non-interference nodes) in the line cutting problem) is recorded as H QUBO Since the maximum independent set problem has no additional constraints, no additional transformation of the maximum independent set problem is required.

[0102] 3. Deep reinforcement learning framework CONQUER

[0103] The overall framework is as follows Figure 2shown.

[0104] 1. Problem Transformation

[0105] The maximum independent set problem can be easily converted into a QUBO model using the mainstream software PyQUBO. Its mathematical representation is as follows:

[0106]

[0107] Where p represents the penalty function.

[0108] (II) Qubormer-based graph representation

[0109] From the construction of the QUBO graph, it can be known that the QUBO graph contains the objective function and constraint information of the problem. Therefore, ideally, it is hoped to find an embedding vector that not only effectively captures the properties of a single node (parameters attached to the node), but also fully represents the properties of nodes with remote correlations, thereby accurately mapping the objective function and constraints of the QUBO problem. Considering the unique structure of the QUBO graph, the deep learning model used in the embodiment of the present invention is a new graph transformer structure, called the Qubormer framework, which mainly includes two components: an encoder and a decoder.

[0110] 1. Encoder

[0111] In recent years, many studies have emphasized that traditional message passing networks encounter bottlenecks such as smoothing, which makes them less efficient when dealing with tasks with long-range dependencies. In order to overcome the shortcomings of traditional neural networks, it is proposed to integrate the transformer mechanism into the message passing network. Generally speaking, the encoder includes three key components.

[0112] (1) Centrality coding

[0113] By incorporating centrality encodings, the importance of nodes in the QUBO graph can be captured. Each node is enhanced by its degree, providing a measure of its relative importance in the network topology.

[0114]

[0115] Among them, ReLU refers to the activation function, The i in the equation represents the i-th node, 0 represents the initial feature, and x i is the initial feature of the node, z deg (v i ) encodes the degree of the node, v irefers to the node with serial number i, v is the abbreviation of the English word vertex, W1 and W2 are the parameters of the neural network, and they are learnable parameters. Degree refers to the degree of the node (degree is a professional term in complex networks, which refers to the node v i The number of adjacent nodes. Specifically for communication network problems, it refers to the adjacent entities with which the current network entity can communicate directly, that is, the node v i The number of directly connected entities), encoding refers to converting nodes into vectors.

[0116] (2) Message Passing Layer

[0117] To be precise, converting various problems into the QUBO model fundamentally integrates the target information and constraint information into the nodes (i.e., nodes in network communication) and edge weights (edge ​​weights) of the QUBO graph, making the edge information the primary information of the QUBO graph. Therefore, a two-step neighborhood aggregation process is adopted, including edge feature aggregation and node feature aggregation, to merge node features (aggregate node features to edges) and edge features (aggregate edge features back to node features) as follows:

[0118]

[0119]

[0120] Among them, h ij =ReLU(W3x ij ) is the initial edge embedding vector, which is the intermediate output of the neural network, that is, embedding the initial features of the edge into the high-dimensional space, x ij is the initial feature of the edge, i.e., the input feature of the edge. W3, W4, .... W7 are all learnable parameters. After obtaining the node feature in step l After that, you can concatenate the features of the previous step of the node Implementing residual connections can alleviate the smoothing problem of the transition model. As follows:

[0121]

[0122] Among them, W8 is a learnable parameter.

[0123] (3) Transformer layer

[0124] In previous studies, such as the GraphTrans model, researchers usually stacked Transformer layers on top of several layers of the message passing network. Although this method enhances the capabilities of the message passing network by introducing positional or structural encoding, it still needs to fundamentally solve core problems such as over-smoothing.

[0125] Therefore, the embodiment of the present invention proposes a new viewpoint: in each layer, the output of the message passing network is aggregated with the output of the global attention layer (which is part of the Transformer layer) to collaboratively update the node features.

[0126]

[0127]

[0128] Among them, MHA represents multi-head attention network, LN represents layer normalization, and FFN represents feed-forward neural network layer. represents the node embedding vector obtained at step l-1, Represents the node embedding vector after the multi-head attention network.

[0129] (4) Iterative calculation to solve

[0130] The above calculation is repeated for T layers to obtain the final embedding vector of each node (that is, the final learned node feature representation). After T layers, the final embedding vector of each node vi is represented as At the same time, the embedding information z of all nodes is aggregated by summing s =∑ i∈V z i , to represent the information related to the entire QUBO problem instance.

[0131] 2. Decoder

[0132] Before discussing the details of the decoder, we need to understand the Markov decision process (MDP) for solving the QUBO problem. Formally, consider an MDP described by a tuple (S, A, P, R), where S is the set of all possible states s i A collection of actions. i ∈A means that for a given state s i The QUBO problem example, the assignment of binary variables X, a i Represents an action i of selection, that is, selecting a node.

[0133] The obtained r i =a i +∑ (i,j)∈E b ij x j ∈R is in state s i Execute action a i Rewards received i , which indicates the change of the objective function before and after the action is executed.

[0134] The decoder essentially converts a state-action pair (s i , a i), converted to a scalar Q(s i ,a i ; Θ), used to evaluate i Next action a i The quality of (i.e., the value, the greater the value, the greater the possibility of the action being executed), Θ is a learnable parameter, and the Q function is expressed as follows:

[0135] Q(h,v;Θ)=W 11 ReLU(W 10 ReLU(W9[z s ,z i ]))

[0136] Among them, W9, W10, W11 are learnable parameters, [z s ,z i ] means to convert the vector z s and vector z i Join by rows.

[0137] (III) Minimizing the energy on the DQN-based QUBO graph, i.e. maximizing the number of nodes

[0138] 1. Training

[0139] After the network structure of the deep learning model CONQUER is built, the widely used DQN is used for model training. The process mainly includes the following steps: First, an instance of the original problem is generated and converted into QUBO form. Then a series of trajectories are generated based on the QUBO graph and stored in the experience replay area. Then the agent selects batches of trajectories from the experience replay area for learning, and updates the neural network parameters accordingly, forming the parameters of the learning model through training.

[0140] (1) Data preparation

[0141] In fact, it is necessary to generate instances of the maximum independent set problem. These instances are determined by the characteristics of the maximum independent set problem itself, and it is not necessarily required that the original problem instance must be a graph.

[0142] (2) Trajectory generation

[0143] As mentioned earlier, we map the optimization problem of the traffic network into a maximum clique solving problem on an abstract graph, where the nodes in the graph represent intersections, key points, etc. in the traffic network, while the edges can represent roads, lines, etc. between intersections, and the weights on the edges represent the traffic information of the route. At the same time, the maximum clique solving problem can be converted into a Hamiltonian minimization problem on the QUBO graph. We use a reinforcement learning algorithm to solve it, that is, we regard the process as a Markov decision process, in which an episode (a complete training round) is the complete process of finding a solution that minimizes the Hamiltonian on the QUBO graph, requiring all decision variable states to flip sequentially from 0 to 1. The trajectory (training trajectory data or training process data) generated in an episode is a state-action-reward sequence s0, a0, r0, s1, a1, r1, ..., s T , where s0 is the initial state where all decision variables are 0, s T is the terminal state where all decision variables are 1. Running a training process game requires completing a complete episode on the QUBO graph, and the model will collect training process data, i.e., s t ,a t ,r t,t+n ,s t+n , where s t represents the state set of all variables at time t, a t represents the action taken at time t, represents the cumulative reward at time t+n, s t+n The state set model representing all variables at time t+n stores them in the replay buffer (this is a data result similar to cache). Here, the delayed reward step n and the discount factor γ are both hyperparameters.

[0144] (3) Update Qubormer parameters

[0145] Using the empirical transitions selected from the replay buffer, CONQUER is able to learn and update the network parameters Θ to minimize the following loss function.

[0146]

[0147] Among them, E represents the maximum expectation, r t,t+n represents the immediate reward after n steps after executing the action at time t, s t+n represents the state at time t+n, a t+n represents the action to be performed at time t+n, γ is the discount factor (hyperparameter, usually 0.99), Indicates that in state s t+n The target network for execution is The action a with the largest return value t+n The reward estimate returned after is the target network parameter, Q(s t ,a t ; Θ) indicates that in state s t Execute action a t The reward valuation obtained according to the original network Q(Θ), where s t represents the state at time t, a t represents the action executed at time t, Θ is the parameter of the original network, which updates its parameters through real-time data. In addition, the original network Q(Θ) periodically copies its parameters Θ to the target network Parameters To update the network.

[0148] 2. Online application and fine-tuning

[0149] In the application, the state of each decision variable in the QUBO graph is iteratively flipped from 0 to 1 through a greedy strategy. Subsequently, the state of the decision variable associated with the flipping process is determined as the optimal solution to the QUBO problem, which produces the lowest Hamiltonian, the minimum energy, and the minimum value of the objective function in the QUBO form.

[0150] However, in the field of machine learning, out-of-distribution generalization is a prominent challenge. To address this challenge, an embodiment of the present invention proposes a concise and efficient fine-tuning method.

[0151] First, perform data augmentation on the tuning dataset, such as introducing noise in the QUBO graph, dividing the dataset into training / validation / test subsets, and sampling the QUBO graph. Then, load the pre-trained model (previously trained model) and freeze its encoder part. Fine-tune only the task head of the downstream task according to the downstream task data, that is, freeze the parameters of the encoder and only update certain parameters to save fine-tuning time. Finally, test the adjusted model to reflect the process of the online application.

[0152] (IV) Solution Output

[0153] 1. Result expression

[0154] In fact, the solution obtained directly from the model is the solution to the QUBO problem, which represents the value of the decision variable that minimizes the Hamiltonian and involves a "yes / no" decision output used to determine whether the decision variable is included in the solution set. This solution set is actually the partitioning of the maximum independent set problem.

[0155] 2. Check and repair the solution

[0156] If the agent is not fully trained due to data sparsity or insufficient training time, or the penalty term is set too low, the solution found by the neural network (in a network topology, identifying which nodes do not interfere with each other, that is, identifying the maximum independent set, that is, from the perspective of communication, it means that node communication does not occupy the same frequency, correspondingly, it means that two nodes will not be included by the same edge) may not meet the feasibility conditions of the original problem. In order to alleviate these problems, the legitimacy of the solution must be ensured first. If the solution meets the feasibility conditions of the original problem, it is directly output; if not, it needs to be repaired to meet these conditions. In general, the repair method is simple and cheap, and common greedy or local search methods are usually used to repair infeasible solutions. Solution repair is only required when dealing with real data sets and very large-scale graphs, and the simplest greedy method is used to repair the solution.

[0157] IV. Experiment

[0158] 1. Experimental setup

[0159] 1. Baseline

[0160] A baseline algorithm with semi-definite programming is used to obtain the relaxed solution. In addition, two heuristic methods, namely the greedy algorithm and the mean field annealing algorithm, are combined. The comparison of learning-based methods includes the classic S2V-DQN algorithm and two probabilistic view-based methods, namely the ERDOS algorithm and the ANNEAL algorithm. In addition, the current state-of-the-art method GFlowNet algorithm, which samples from the solution space, is evaluated. The baseline methods include two unsupervised learning methods, the RUN-CSP algorithm and the EGN algorithm, and their enhanced versions, the Meta-EGN algorithm. In addition, the latest method, the VAG-CO algorithm, which improves the mean field annealing algorithm, is compared.

[0161] 2. Dataset

[0162] A cross-domain graph dataset is used to make the solution method generalize well and be able to be evaluated on larger-scale problems.

[0163] 3. Evaluation Metrics

[0164] In order to be consistent with previous works, a comprehensive evaluation metric is adopted, which considers the best ratio (R*) and the average ratio (R) of the results obtained from the proposed method to the results obtained by the Gurobi optimizer within one hour, as well as the corresponding standard errors, to measure the efficiency and solution accuracy of the method.

[0165] 2. Real-world data results

[0166] Experiments were conducted on the following datasets: ENZYMES, PROTEINS, IMDB-BINARY, COLLAB, and MUTAG, and compared with the results of VAG-CO. The experiments were divided into 0.6 / 0.1 / 0.3 ratios, and the first 1000 graphs in the COLLAB dataset were used for experiments. Training was performed on RB graphs of size 100, and fine-tuning was performed on other datasets. The experimental results are shown in Table 1.

[0167] Table 1 Experimental results

[0168]

[0169] It can be found that the method of the embodiment of the present invention reaches or is close to the optimal solution, and the fine-tuning process (ft) only requires a few hundred steps, and fine-tuning on the largest COLLAB dataset only requires no more than 1k steps.

[0170] The beneficial technical effects achieved by the embodiments of the present invention are as follows:

[0171] By modeling the node optimization task in the communication network as the maximum independent set problem in combinatorial optimization, the non-interfering node set in the communication network is optimized by finding the maximum independent set. The maximum independent set problem is then expressed in the form of QUBO, demonstrating the pioneering development of a unified end-to-end deep learning reinforcement learning architecture customized for this problem. Due to the particularity of the QUBO graph structure, the transformer mechanism is introduced in the communication network to effectively capture the long-term dependencies between the nodes in the network. Then, a corresponding decision reduction strategy is designed for this problem, and a simple and effective fine-tuning method is designed for a real communication network dataset that is different from the training set distribution. Unlike all previous work, the framework of the embodiment of the present invention stands out by avoiding task-specific adjustments across various problem instances or fine-tuning scenarios. Experimental results show that the method of the embodiment of the present invention is significantly better than the existing state-of-the-art heuristic algorithms, and can well optimize independent nodes in complex communication networks, achieve network performance improvement and efficient resource utilization.

[0172] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of the present disclosure. The attached method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0173] In the above detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are clearly stated in each claim. On the contrary, as reflected in the appended claims, the invention is in a state of having less than all the features of the disclosed individual embodiments. Therefore, the appended claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

[0174] The disclosed embodiments are described above to enable any person skilled in the art to implement or use the present invention. Various modifications of these embodiments are obvious to those skilled in the art, and the general principles defined herein may also be applied to other embodiments without departing from the spirit and scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0175] The above description includes examples of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but it should be recognized by those skilled in the art that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of protection of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, the word is covered in a manner similar to the term "including", just as "including," is explained as a transitional word in the claims. In addition, any term "or" used in the specification of the claims is intended to mean "non-exclusive or".

[0176] Those skilled in the art may also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention may be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly demonstrate the interchangeability of hardware and software, the various illustrative components, units, and steps described above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present invention.

[0177] The various illustrative logic blocks or units described in the embodiments of the present invention can be implemented or operated by a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of the above. The general-purpose processor can be a microprocessor, and optionally, the general-purpose processor can also be any conventional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0178] The steps of the method or algorithm described in the embodiments of the present invention can be directly embedded in hardware, a software module executed by a processor, or a combination of the two. The software module can be stored in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or other storage media of any form in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and the storage medium can be arranged in an ASIC, and the ASIC can be arranged in a user terminal. Optionally, the processor and the storage medium can also be arranged in different components in the user terminal.

[0179] In one or more exemplary designs, the above functions described in the embodiments of the present invention can be implemented in hardware, software, firmware or any combination of the three. If implemented in software, these functions can be stored on a computer-readable medium, or transmitted in the form of one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one place to another. The storage medium can be any available medium that can be accessed by any general or special computer. For example, such computer-readable media can include but are not limited to RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store program codes in the form of instructions or data structures and other forms that can be read by general or special computers, or general or special processors. In addition, any connection can be appropriately defined as a computer-readable medium, for example, if the software is transmitted from a website site, server or other remote resource through a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wirelessly, such as infrared, wireless and microwave, it is also included in the defined computer-readable medium. The disk and disc include compact disk, laser disk, optical disk, DVD, floppy disk and blue-ray disk. Disks usually copy data magnetically, while discs usually copy data optically with lasers. The above combination can also be included in computer readable media.

[0180] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A node optimization method for a communication network, characterized in that: include: Optimizing the utilization problem of nodes of a communication network as a maximum independent set problem, the nodes including: terminal nodes, and transit nodes for converting and exchanging information; Convert the maximum independent set problem into a QUBO model; Constructing a corresponding QUBO graph for the QUBO model; A graph neural network is used to characterize the structure and characteristics of the QUBO graph, and a deep learning model CONQUER is constructed; The deep learning model CONQUER is trained to explore the optimal solution to the node utilization problem of the communication network, and a trained deep learning model CONQUER is obtained; the trained deep learning model CONQUER is used to optimize the node utilization problem of any communication network to obtain a node optimization result, and the node optimization result includes: the nodes used, the number of nodes, and the connection relationship between the nodes.

2. The node optimization method of a communication network according to claim 1, characterized in that: The node utilization problem of the communication network is optimized as a maximum independent set problem, including: The communication network is represented as G = (V, E), where V refers to the node set, including all nodes in the communication network, and E refers to the edge set, including all edges in the communication network. ij ∈E, edge e ij ∈E indicates that there is a connection relationship between nodes i and j; The problem of utilizing the nodes of the communication network is transformed into a corresponding maximum independent set problem, so as to find a non-interfering node subset through the maximum independent set problem. The number of nodes in the non-interfering node subset is maximized, and the maximum independent set problem is expressed as: Where X = (x1, x2.....x n ) represents the decision vector of whether a node is selected as a non-interference node. A non-interference node refers to a node whose communications do not interfere with each other.

3. The node optimization method of the communication network according to claim 2, characterized in that: The maximum independent set problem is transformed into a QUBO model, including: By adding a penalty term to the maximum independent set problem constraint, the maximum independent set problem is converted into a QUBO model, and the QUBO model is expressed as: Where X = (x1, x2.....x n ) represents the decision vector composed of nodes, x i 、x j are the values ​​of the i-th and j-th nodes, respectively, indicating whether these nodes are selected into the non-interference node subset. The matrix Q is a symmetric matrix or an upper triangular matrix. i =Q ii ,so b ij =Q ij ,,a i represents the node weight in the QUBO model, b ij Represents the edge weight on the QUBO model.

4. The node optimization method of the communication network according to claim 3, characterized in that: The step of constructing a corresponding QUBO graph for the QUBO model comprises: A corresponding QUBO graph is constructed for the QUBO model, and the mathematical representation of the QUBO graph is: Where p represents the penalty function.

5. The node optimization method of the communication network according to claim 4, characterized in that: The graph neural network is used to characterize the structure and characteristics of the QUBO graph, and a deep learning model CONQUER is constructed, including: Constructing an encoder, wherein the encoder construction specifically includes: integrating a centrality encoding layer, a message passing layer, a Transformer layer, and an iterative calculation solution layer; wherein: At the integrated centrality encoding layer: Each node in the QUBO graph is enhanced by the corresponding degree to capture the importance of the node in the QUBO graph, and then encoded. Each node in the QUBO graph is enhanced by the corresponding degree and encoded to obtain the corresponding vector representation: Among them, ReLU refers to the activation function, The i in the equation represents the i-th node, 0 represents the initial feature, and x i is the initial feature of the node, z deg (v i ) encodes the degree of the node, v i It refers to the node with serial number i, W1 and W2 are the parameters of the neural network, and they are learnable parameters; At the messaging layer: The neighborhood aggregation process is adopted: edge feature aggregation and node feature aggregation to merge node features and edge features respectively. The edge feature aggregation and node feature aggregation are expressed as follows: Among them, h ij =ReLU(W3x ij ) is the initial edge embedding vector, which is the intermediate output of the neural network, that is, embedding the initial features of the edge into the high-dimensional space, x ij is the initial feature of the edge, i.e., the input feature of the edge. W3, W4, W5, W6, and W7 are the parameters of the neural network, and they are all learnable parameters. After getting the node features of step After that, by splicing the features of the previous step of the node Residual connection, the residual connection is expressed as follows: Among them, W8 is the parameter of the neural network and is a learnable parameter; In the Transformer layer: Aggregate the output of the message passing layer with the output of the global attention layer in the Transformer layer to obtain updated node features: Among them, MHA represents multi-head attention network, LN represents layer normalization, and FFN represents feed-forward neural network layer. represents the node embedding vector obtained at step -1, represents the node embedding vector after the multi-head attention network; In the iterative calculation solution layer: When solving, the updated node features output from the Transformer layer are used as the input of the integrated centrality encoding layer, and the steps of the integrated centrality encoding layer, the message passing layer, and the Transformer layer are repeatedly executed in a loop to iterate. After the iteration is completed, the final embedding vector corresponding to each node is obtained, and the final embedding vector of each node is expressed as: Sum the final embedding vectors of all nodes to get: s =∑ i∈V z i , the solution of the QUBO model is characterized by summation.

6. The node optimization method of the communication network according to claim 5, characterized in that: The graph neural network is used to characterize the structure and characteristics of the QUBO graph, and a deep learning model CONQUER is constructed, which also includes: Build the decoder, including: Formally, a Markov decision process is formed, which is described by the tuple (S, A, P, R), where S is all possible states s i A collection of actions i ∈A represents the assignment of the binary variable X of the QUBO model for a given state si; In status i Execute action a i Get reward i :r i =a i +∑ (i，j)∈E b ij x j ∈R, reward r i Indicates the change of the objective function before and after the execution of the action Markov decision process; Through the decoder, a pair of state-action pairs (s i , a i ) is converted to a scalar Q(s i , a i ; Θ), to evaluate i Next action a i The quality of the decoder is expressed as follows: Q(h,v;Θ)=W 11 ReLU(W 10 ReLU(W9[z s ,With i ])) Among them, W9, W10, W11 are the parameters of the neural network and can be learned. s , z i ] means to convert the vector z s and vector z i Join by rows.

7. The node optimization method of the communication network according to claim 6, characterized in that: By training the deep learning model CONQUER to explore the optimal solution to the node utilization problem of the communication network, a trained deep learning model CONQUER is obtained, including: Obtaining a sample of nodes of the communication network corresponding to the maximum independent set problem, and generating a trajectory based on a Markov decision process representation based on the QUBO graph; Among them, an episode is the complete process of finding a solution that minimizes the Hamiltonian on the QUBO graph, requiring all decision variable states to flip sequentially from 0 to 1; the trajectory generated in a training round is a state-action-reward sequence: s0, a0, r0, s1, a1, r1, ..., s T , ..., s T s0 is the initial state where all decision variables are 0, s T is the terminal state where all decision variables are 1; completing a training process requires a complete training round on the QUBO graph, and a complete training round includes the generation of trajectories and the collection of experience transformations. t , a t , r t，t+n ,s t+n , r t+n represents the cumulative reward at time t+n, the delayed reward step n is a hyperparameter, and the discount factor γ is a hyperparameter; Using the experience transformations selected from the storage structure, the deep learning model CONQUER is enabled to learn and update the network parameters Θ of the deep learning model CONQUER to minimize the following loss function: Among them, E represents the maximum expectation, r t，t+n represents the immediate reward after n steps after executing the action at time t, s t+n represents the state at time t+n, a t+n Indicates the action to be performed at t+n. Indicates that in state s t+n The target network for execution is The action a with the largest return value t+n The reward estimate returned after is the target network parameter; Q(s t , a t ; Θ) indicates that in state s t Execute action a t According to the reward valuation obtained by the original network Q(Θ), s t represents the state at time t, a t represents the action executed at time t, Θ is the parameter of the original network, which updates each parameter through real-time data; the original network Q(Θ) will periodically copy the parameter Θ to the target network Parameters To achieve network updates; When the loss function value is less than the preset value, the trained deep learning model CONQUER can be obtained.

8. The node optimization method of the communication network according to claim 7, characterized in that: Also includes: In the obtained sample of nodes of the communication network corresponding to the maximum independent set problem, noise is introduced into the sample when constructing the QUBO graph; When the loss function is less than a preset value, it is also necessary to freeze the parameters of the encoder and fine-tune the parameters of the encoder for the corresponding deep learning model CONQUER to obtain a trained deep learning model CONQUER.

9. The node optimization method of a communication network according to claim 8, characterized in that: Also includes: When the trained deep learning model CONQUER is used to solve the node utilization problem of the communication network to be solved, when the obtained solution does not meet the feasibility conditions of the communication network to be solved, a greedy search method or a local search method is used to repair the solution.

10. A method for optimizing nodes in a communication network, characterized in that: Also includes: Inputting the utilization problem of the nodes of the communication network to be solved into the trained deep learning model CONQUER described in any one of claims 1 to 9; Output node optimization results, which include: adopted nodes, number of nodes, and connection relationships between nodes.

Images