A node optimization method for communication network

Abstract

An embodiment of the present invention provides a communication network node optimization method, comprising: optimizing the communication network node utilization problem as a maximum independent set problem; converting the maximum independent set problem into a QUBO model; constructing a corresponding QUBO graph for the QUBO model; using a graph neural network to characterize the structure and characteristics of the QUBO graph to construct a deep learning model CONQUER; and training the deep learning model CONQUER to explore the optimal solution to the communication network node utilization problem, thereby obtaining a trained deep learning model CONQUER. This method can quickly and accurately optimize independent nodes or groups in the network, thereby improving the connectivity of the communication network and the efficiency of information transmission.

Description

Technical Field

[0001] The present invention relates to the field of node optimization of communication networks, and in particular to a node optimization method of communication networks. Background Art

[0002] As communication technology matures and people's reliance on wireless connections continues to increase, the configuration of independent nodes in communication networks may affect the flow of information and the maintenance of network quality. For example, in wireless network design, identifying and optimizing potential independent nodes can help engineers more accurately configure the network, improve data transmission efficiency, and reduce signal interference. Furthermore, in network structure analysis, by deeply studying the connectivity and distribution of nodes, it is possible to optimize the configuration of independent nodes and understand their impact on network performance, providing important insights for optimizing and managing communication networks.

[0003] In this context, optimizing independent nodes in communication networks has become a critical task. It directly impacts the operational efficiency of the communication network and the reliability of information transmission, aiming to maximize connectivity and information transmission efficiency. By analyzing the network structure, optimizing the configuration of independent nodes, and implementing corresponding measures, we can improve network stability and reliability, providing users with a better connection experience. Therefore, optimizing and analyzing independent nodes in communication networks has important practical significance and promising applications. Summary of the Invention

[0004] The embodiment of the present invention provides a node optimization method for a communication network, which can solve the problem of optimizing the network configuration of independent nodes in a communication network in the prior art.

[0005] To achieve the above objectives, on the one hand, an embodiment of the present invention provides a node optimization method for a communication network, comprising:

[0006] Optimizing the utilization problem of nodes of a communication network as a maximum independent set problem, wherein the nodes include: terminal nodes and transit nodes for converting and exchanging information;

[0007] Convert the maximum independent set problem into a QUBO model;

[0008] Constructing a corresponding QUBO diagram for the QUBO model;

[0009] A graph neural network is used to characterize the structure and characteristics of the QUBO graph, and a deep learning model CONQUER is constructed.

[0010] By training the deep learning model CONQUER to explore the optimal solution to the node utilization problem of the communication network, a trained deep learning model CONQUER is obtained; the trained deep learning model CONQUER is used to optimize the node utilization problem of any communication network to obtain a node optimization result, which includes: the nodes used, the number of nodes, and the connection relationship between the nodes.

[0011] On the other hand, an embodiment of the present invention provides a node optimization method for a communication network, further comprising:

[0012] Input the node utilization problem of the communication network to be solved into the previously trained deep learning model CONQUER;

[0013] Output node optimization results, including: adopted nodes, number of nodes, and connection relationships between nodes.

[0014] The above technical solution has the following beneficial effects: The QUBO model, a framework widely used in quantum annealing and other optimization algorithms, shows great potential for solving the maximum independent set problem. While the QUBO model effectively handles static optimization problems, the dynamic nature of communication networks requires optimization algorithms to adapt to environmental changes and make real-time adjustments. The key advantage of CONQUER is that it does not require the deep learning architecture to be tuned for the specific scenario-specific problem of identifying critical communication lines, thereby reducing the cost of problem customization. Experimental results show that, despite being trained only on small-scale synthetic graphs, CONQUER demonstrates excellent generalization and can easily handle the challenges of large-scale graphs. By employing a simple fine-tuning strategy, CONQUER can be effectively adapted to real-world datasets. In practical applications, CONQUER has demonstrated outstanding performance, particularly in optimizing individual nodes in communication networks, enabling rapid and accurate optimization of both individual nodes and groups within the network. This improves the connectivity of communication networks and the efficiency of information transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 This is an embodiment of a node optimization method for a communication network according to an embodiment of the present invention;

[0017] Figure 2It is a solution framework of the QUBO model of the embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0019] like Figure 1 As shown, in combination with an embodiment of the present invention, a node optimization method for a communication network is provided, comprising:

[0020] S101: Optimizing the utilization problem of nodes in a communication network as a maximum independent set problem, wherein the nodes include: terminal nodes and transit nodes for converting and exchanging information;

[0021] S102: Converting the maximum independent set problem into a QUBO model;

[0022] S103: Constructing a corresponding QUBO graph for the QUBO model;

[0023] S104: Using a graph neural network to characterize the structure and characteristics of the QUBO graph, and constructing a deep learning model CONQUER;

[0024] S105: The deep learning model CONQUER is trained to explore the optimal solution to the node utilization problem of the communication network, and a trained deep learning model CONQUER is obtained; the trained deep learning model CONQUER is used to optimize the node utilization problem of any communication network to obtain a node optimization result, and the node optimization result includes: the nodes used, the number of nodes, and the connection relationship between the nodes.

[0025] Preferably, the utilization problem of nodes of the communication network is optimized as a maximum independent set problem, comprising:

[0026] The communication network is represented as G = (V, E), where V is the node set, including all nodes in the communication network, and E is the edge set, including all edges in the communication network. ij ∈E, edge e ij ∈E indicates that there is a connection relationship between nodes i and j;

[0027] The problem of utilizing the nodes of the communication network is transformed into a corresponding maximum independent set problem, so as to find a non-interfering node subset through the maximum independent set problem. To maximize the number of nodes in the non-interfering node subset, the maximum independent set problem is expressed as:

[0028]

[0029] Where, X=(x1,x2.....x n ) represents the decision vector of whether the node is selected as a non-interference node. A non-interference node refers to a node whose communications do not interfere with each other.

[0030] Preferably, converting the maximum independent set problem into a QUBO model includes:

[0031] By adding a penalty term to the maximum independent set problem constraint, the maximum independent set problem is converted into a QUBO model, and the QUBO model is expressed as:

[0032]

[0033] Where, X=(x1,x2……x n ) represents the decision vector composed of nodes, x i 、x j Respectively represent the values ​​of the i-th and j-th nodes, indicating whether these nodes are selected into the non-interference node subset. The matrix Q is a symmetric matrix or an upper triangular matrix. i =Q ii ,so b ij =Q ij , a i represents the node weight in the QUBO model, b ij Represents the edge weight on the QUBO model.

[0034] Preferably, constructing a corresponding QUBO diagram for the QUBO model comprises:

[0035] A corresponding QUBO diagram is constructed for the QUBO model. The mathematical representation of the QUBO diagram is:

[0036]

[0037] Where p represents the penalty function.

[0038] Preferably, a graph neural network is used to characterize the structure and characteristics of the QUBO graph, and a deep learning model CONQUER is constructed, including:

[0039] Constructing an encoder, wherein the encoder construction specifically includes: integrating a centrality encoding layer, a message passing layer, a Transformer layer, and an iterative calculation solution layer; wherein:

[0040] At the integrated centrality encoding layer:

[0041] Each node in the QUBO graph is enhanced by the corresponding degree to capture the importance of the node in the QUBO graph and encoded. Each node in the QUBO graph is enhanced by the corresponding degree and encoded to obtain the corresponding vector representation:

[0042]

[0043] Among them, ReLU refers to the activation function, The i in the code represents the i-th node, 0 represents the initial feature, and x i is the initial feature of the node, z deg (v i ) encodes the degree of the node, v i It refers to the node with serial number i, W1 and W2 are the parameters of the neural network, and they are learnable parameters;

[0044] At the messaging layer:

[0045] The neighborhood aggregation process is adopted: edge feature aggregation and node feature aggregation to merge node features and edge features respectively. The edge feature aggregation and node feature aggregation are expressed as follows:

[0046]

[0047]

[0048] Among them, h ij =ReLU(W3x ij ) is the initial edge embedding vector, which is the intermediate output of the neural network, that is, the initial features of the edge are embedded into the high-dimensional space, x ij is the initial feature of the edge, i.e., the input feature of the edge. W3, W4, W5, W6, and W7 are the parameters of the neural network, and they are all learnable parameters.

[0049] After obtaining the node features in step l Afterwards, by splicing the features of the previous step of the node Residual connection, the residual connection is expressed as follows:

[0050]

[0051] Among them, W8 is the parameter of the neural network and is a learnable parameter;

[0052] In the Transformer layer:

[0053] Aggregate the output of the message passing layer with the output of the global attention layer within the Transformer layer to obtain updated node features:

[0054]

[0055]

[0056] Among them, MHA represents multi-head attention network, LN represents layer normalization, and FFN represents feed-forward neural network layer. represents the node embedding vector obtained at step l-1, represents the node embedding vector after the multi-head attention network;

[0057] In the iterative calculation solution layer:

[0058] When solving, the updated node features output from the Transformer layer are used as the input of the integrated centrality encoding layer. The steps of the integrated centrality encoding layer, the message passing layer, and the Transformer layer are repeatedly executed in a loop to iterate. After the iteration is completed, the final embedding vector corresponding to each node is obtained. The final embedding vector of each node is expressed as:

[0059] Sum the final embedding vectors of all nodes to get: s =∑ i∈V z i , the solution of the QUBO model is characterized by summation.

[0060] Preferably, a graph neural network is used to characterize the structure and characteristics of the QUBO graph to construct a deep learning model CONQUER, further comprising:

[0061] Build the decoder, including:

[0062] Formally, a Markov decision process is formed, which is described by the tuple (S, A, P, R), where S is the number of possible states s. i A collection of actions i ∈A represents the assignment of the binary variable X of the QUBO model for a given state si;

[0063] In state s i Execute action a i Get reward i :r i =a i +∑ (i,j)∈E b ij x j ∈R,, represents the change of the objective function before and after the execution of the action Markov decision process;

[0064] Through the decoder, a pair of state-action pairs (s i , a i ) is converted to a scalar Q(s i ,a i ; Θ), to evaluate the i Next action a i The quality of the decoder is expressed as follows:

[0065] Q(h,v;Θ)=W 11 ReLU(W 10 ReLU(W9[z s ,z i ]))

[0066] Among them, W9, W10, and W11 are parameters of the neural network and can be learned. s ,z i ] means to convert vector z s and vector z i Splice by row.

[0067] Preferably, the deep learning model CONQUER is trained to explore the optimal solution to the node utilization problem of the communication network, thereby obtaining the trained deep learning model CONQUER, including:

[0068] Obtain a sample of nodes in the communication network corresponding to the maximum independent set problem, and generate a trajectory based on the Markov decision process representation based on the QUBO graph; wherein a training round is the complete process of finding a solution that minimizes the Hamiltonian on the QUBO graph, requiring that the states of all decision variables are flipped sequentially from 0 to 1; the trajectory generated in an episode is a state-action-reward sequence: s0,a0,r0,s1,a1,r1,...,s T , s0 is the initial state where all decision variables are 0, s T is the terminal state where all decision variables are 1; completing one round requires completing a full training round on the QUBO graph, and a full training round includes the generation of trajectories and the collection of experience transformations s t ,a t ,r t,t+n ,s t+n , r t+n represents the cumulative reward at time t+n, the delayed reward step n is a hyperparameter, and the discount factor γ is a hyperparameter;

[0069] Using the experience transformations selected from the storage structure, the deep learning model CONQUER is enabled to learn and update the network parameters Θ of the deep learning model CONQUER to minimize the following loss function:

[0070]

[0071] Among them, E represents the maximum expectation, r t,t+n represents the immediate reward after n steps after executing the action at time t, s t+n represents the state at time t+n, a t+n Indicates the action to be performed at t+n. In state s t+n Target network for execution The action a with the largest return value t+n The reward valuation returned after is the target network parameter; Q(s t ,a t ; Θ) represents the state s t Execute action a t The reward estimate obtained from the original network Q(Θ), s t represents the state at time t, a t represents the action executed at time t, Θ is the parameter of the original network, which updates each parameter through real-time data; the original network Q(Θ) regularly copies the parameter Θ to the target network Parameters To achieve network updates;

[0072] When the loss function value is less than the preset value, the trained deep learning model CONQUER can be obtained.

[0073] In particular, the node optimization method of the communication network further includes:

[0074] In the obtained sample of nodes of the communication network corresponding to the maximum independent set problem, noise is introduced into the sample when constructing the QUBO graph;

[0075] When the loss function is less than a preset value, it is also necessary to freeze the parameters of the encoder and fine-tune the parameters of the encoder for the corresponding deep learning model CONQUER to obtain a trained deep learning model CONQUER.

[0076] Preferably, the communication network node optimization method further includes:

[0077] When the trained deep learning model CONQUER is used to solve the node utilization problem of the communication network to be solved, if the obtained solution does not meet the feasibility conditions of the communication network to be solved, a greedy search method or a local search method is used to repair the solution.

[0078] In conjunction with an embodiment of the present invention, a method for optimizing nodes in a communication network is provided, further comprising:

[0079] Inputting the node utilization problem of the communication network to be solved into any of the trained deep learning models described above;

[0080] Output node optimization results, including: adopted nodes, number of nodes, and connection relationships between nodes.

[0081] The above technical solutions of the embodiments of the present invention are described in detail below with reference to specific application examples. For technical details not introduced during the implementation process, please refer to the relevant description above.

[0082] An embodiment of the present invention provides a node optimization algorithm for a communication network based on QUBO and deep reinforcement learning. The efficiency and performance of a communication network depend on the optimization of its topological structure. The maximum independent set problem is used to find a maximum set of vertices in a graph so that no two vertices in the maximum set of vertices are adjacent. In a communication network, the maximum independent set can be used to determine the maximum set of non-interfering nodes in a wireless communication network, thereby optimizing the spectrum utilization of each node in the communication network and reducing conflicts in the utilization of each node. Although traditional algorithms such as greedy algorithms and local searches can find better solutions in some scenarios, they often perform poorly in large-scale complex networks. In recent years, the development of quantum computing and machine learning has provided new solutions to this problem.

[0083] The optimization of independent nodes in a communication network is often modeled as the maximum independent set problem, which aims to optimize independent nodes without compromising network connectivity. The maximum independent set problem is a classic NP-hard problem. Traditional operations research optimization methods often fall into a so-called trade-off dilemma when solving the maximum independent set problem: algorithms with high solution quality often suffer from slow solution speed, while algorithms with fast solution speed often suffer from low solution quality.

[0084] In recent years, machine learning methods designed based on the unique structure of problems have demonstrated remarkable advantages in solution speed and quality, while also possessing strong generalization capabilities, and have gradually become the mainstream algorithms for solving the maximum independent set problem. However, in actual modeling, the maximum independent set problem in different contexts exhibits different mathematical forms using different modeling methods, making it difficult for machine learning methods to comprehensively address the maximum independent set problem in all its mathematical forms.

[0085] To solve this problem, the maximum independent set problem in various scenarios is uniformly described as a QUBO model (quadratic unconstrained binary optimization), and the corresponding QUBO graph is constructed, thereby converting the solution of the maximum independent set problem into an energy minimization problem on the QUBO graph. Reinforcement learning provides a powerful framework that continuously improves decision-making strategies by learning from interactions with the environment to explore the optimal solution to the problem. Combining reinforcement learning with the QUBO model can construct an optimization algorithm that dynamically adjusts strategies to respond to changes in network status in real time (network status refers to the ability of various parameters and conditions in the network to change over time. These changes may include the number of nodes and the connection relationships between nodes). Therefore, the deep reinforcement learning framework of CONQUER is used to solve the maximum independent set problem, which mainly includes the following four key steps:

[0086] (1) Problem transformation: The maximum independent set problem in various situations is uniformly described as a QUBO model, and the corresponding QUBO diagram is constructed.

[0087] (2) Graph representation learning: Design a deep learning-based graph transformer model to represent the structure and characteristics of the QUBO graph.

[0088] (3) Solution space exploration based on reinforcement learning: DQN is used to explore the solution space and minimize the energy on the QUBO graph.

[0089] (4) Checking and repairing the solution: Check and repair the found solution to ensure that it is the solution to the maximum independent set problem.

[0090] Compared to traditional methods, the QUBO model, a framework widely used in quantum annealing and other optimization algorithms, shows great potential for solving the maximum independent set problem. Although the QUBO model can effectively handle static optimization problems, the dynamic nature of communication networks requires optimization algorithms to adapt to environmental changes and make real-time adjustments.

[0091] A key advantage of CONQUER is that it does not require tailoring the deep learning architecture to the specific problem of identifying critical communication lines, thus reducing the cost of problem customization. Experimental results demonstrate that, despite being trained on small-scale synthetic graphs, CONQUER demonstrates excellent generalization and can easily handle the challenges of large-scale graphs. By employing a simple fine-tuning strategy, CONQUER can be effectively adapted to real-world datasets. In practical applications, CONQUER demonstrates outstanding performance, particularly in optimizing independent nodes in communication networks, enabling fast and accurate optimization of individual nodes or groups within the network. Identifying independent nodes in communication networks is a crucial issue, directly impacting network connectivity and information transmission efficiency. This problem can be abstractly modeled as the maximum independent set problem: finding the largest set of nodes in a graph such that no two nodes are adjacent. Due to the NP-hard nature of the maximum independent set problem, traditional heuristic methods and integer programming algorithms rely on handcrafted heuristics to make decisions, which are computationally expensive and impractical.

[0092] 1. Maximum Independent Set Problem

[0093] Generally, the problem of identifying independent nodes in a communication network is often modeled as the maximum independent set problem. The goal is to find the largest set of isolated nodes in the network while ensuring the connectivity of the communication network. Formally, the communication network G = (V, E) is usually composed of a set of nodes V representing various types of communication units and a set of edges E indicating the existence of connections between these nodes (that is, there are edges when there is communication between nodes). Each edge e ij ∈E indicates that there is a connection between nodes i and j. The goal is to find a subset of isolated nodes (non-interference nodes) Maximize the number of nodes. Therefore, the mathematical form of the problem is as follows:

[0094]

[0095] Where, X=(x1,x2……x n ) represents the decision vector of whether a node is selected as an isolated node.

[0096] The connection relationship between nodes refers to whether there is an edge between any two nodes. The edge represents the connection between nodes. The maximum independent set means that no two nodes are covered by the same edge.

[0097] In wireless networks, non-interfering nodes typically refer to those that do not transmit signals within a specific frequency band, thereby avoiding interference with other nodes transmitting or receiving signals. In wireless communications, spectrum resources are precious. To avoid mutual interference and improve overall network performance and spectrum efficiency, certain nodes in the network do not transmit data at specific times or on specific frequency bands. This reduces co-channel interference and improves network capacity and coverage. In wireless local area networks (WLANs), non-interfering nodes may refer to access points or client devices that automatically avoid transmitting data when they detect other nodes transmitting, thereby reducing collisions and retransmissions. In cellular networks, non-interfering nodes may refer to base stations or mobile devices that do not transmit data on specific frequencies. In this context, "non-interfering nodes" are nodes whose communications do not interfere with each other. Nodes include relay nodes and terminal nodes that convert and exchange information. Deploying independent nodes in a network ensures that nodes throughout the network communicate without interfering with each other.

[0098] 2. QUBO Form of the Maximum Independent Set Problem

[0099] QUBO, or quadratic unconstrained extreme optimization problem, contains linear and quadratic terms. By adding penalty terms to the constraints, the mathematical form of the combinatorial optimization problem can be converted to QUBO. Specifically, the QUBO problem is formalized as follows:

[0100]

[0101] Where, X=(x1,x2……x n ) represents the decision vector composed of nodes, x i 、x j Respectively represent the values ​​of the i-th and j-th nodes, indicating whether these nodes are selected into the non-interference node subset. The matrix Q is usually a symmetric or upper triangular matrix and a i =Q ii ,so b ij =Q ij , a i represents the node weight in the QUBO graph, b ij represents the edge weight on the QUBO graph, Q ij is the value in the matrix Q, and the objective function value of QUBO (that is, the maximum number of isolated nodes (non-interference nodes) of the line cutting problem) is recorded as H QUBO Since the maximum independent set problem has no additional constraints, no additional transformation of the maximum independent set problem is required.

[0102] 3. Deep Reinforcement Learning Framework CONQUER

[0103] The overall framework is as follows Figure 2shown.

[0104] (1) Problem Transformation

[0105] The maximum independent set problem can be easily converted into a QUBO model using the mainstream software PyQUBO. Its mathematical representation is as follows:

[0106]

[0107] Where p represents the penalty function.

[0108] (2) Qubormer-based graph representation

[0109] From the construction of the QUBO graph, we can know that the QUBO graph contains the objective function and constraint information of the problem. Therefore, in an ideal situation, we hope to find an embedding vector that not only effectively captures the properties of a single node (the parameters attached to the node), but also fully represents the properties of nodes with long-range correlations, thereby accurately mapping the objective function and constraints of the QUBO problem. Considering the unique structure of the QUBO graph, the deep learning model used in the embodiment of the present invention is a new graph transformer structure called the Qubormer framework, which mainly includes two components: an encoder and a decoder.

[0110] 1. Encoder

[0111] In recent years, numerous studies have highlighted that traditional message passing networks (MPNs) encounter bottlenecks such as smoothing, making them less effective for tasks with long-range dependencies. To overcome these shortcomings of traditional neural networks, the idea of ​​integrating the transformer mechanism into MPNs has been proposed. Generally speaking, an encoder consists of three key components.

[0112] (1) Centrality coding

[0113] By incorporating centrality encoding, we can capture the importance of nodes in the QUBO graph. Each node is enhanced with its degree, providing a measure of its relative importance in the network topology.

[0114]

[0115] Among them, ReLU refers to the activation function, The i in the code represents the i-th node, 0 represents the initial feature, and x i is the initial feature of the node, z deg (v i ) encodes the degree of the node, v iRefers to the node with serial number i, v is the abbreviation of the English word vertex, W1 and W2 are the parameters of the neural network, and are learnable parameters. Degree refers to the degree of the node (degree is a professional term in complex networks, which refers to the node v i The number of adjacent nodes. Specifically for communication network issues, it refers to the adjacent entities that the current network entity can directly communicate with, that is, the node v i The number of directly connected entities), encoding refers to converting nodes into vectors.

[0116] (2) Message Passing Layer

[0117] Specifically, transforming various problems into the QUBO model fundamentally integrates the target information and constraint information into the nodes (i.e., nodes in network communication) and edge weights (edge ​​weights) of the QUBO graph, making edge information the primary information of the QUBO graph. Therefore, a two-step neighborhood aggregation process is adopted, including edge feature aggregation and node feature aggregation, to merge node features (aggregate node features onto edges) and edge features (aggregate edge features back onto node features) as follows:

[0118]

[0119]

[0120] Among them, h ij =ReLU(W3x ij ) is the initial edge embedding vector, which is the intermediate output of the neural network, that is, the initial features of the edge are embedded into the high-dimensional space, x ij is the initial feature of the edge, i.e. the input feature of the edge. W3, W4....W7 are all learnable parameters. After obtaining the node feature in step 1 Afterwards, you can concatenate the features of the previous step of the node Implementing residual connections can alleviate the smoothing problem of the over-model. As follows:

[0121]

[0122] Among them, W8 is a learnable parameter.

[0123] (3) Transformer layer

[0124] In previous studies, such as the GraphTrans model, researchers typically stacked Transformer layers on top of several layers of a message passing network. While this approach enhanced the capabilities of message passing networks by introducing positional or structural encoding, it still needed to fundamentally address core issues such as oversmoothing.

[0125] Therefore, the embodiment of the present invention proposes a new point of view: in each layer, the output of the message passing network is aggregated with the output of the global attention layer (which is part of the Transformer layer) to collaboratively update the node features.

[0126]

[0127]

[0128] Among them, MHA represents multi-head attention network, LN represents layer normalization, and FFN represents feed-forward neural network layer. represents the node embedding vector obtained at step l-1, Represents the node embedding vector after the multi-head attention network.

[0129] (4) Iterative calculation to solve

[0130] Iterate the above calculations for T layers to obtain the final embedding vector of each node (that is, the final learned node feature representation). After T layers, the final embedding vector of each node vi is represented as At the same time, the embedding information z of all nodes is aggregated by summing s =∑ i∈V z i , to represent the information related to the entire QUBO problem instance.

[0131] 2. Decoder

[0132] Before discussing the details of the decoder, we need to understand the Markov decision process (MDP) for solving the QUBO problem. Formally, consider an MDP described by a tuple (S, A, P, R), where S is the set of all possible states s i A collection of actions. i ∈A means that for a given state s i The QUBO problem instance, the assignment of binary variables X, a i Represents an action i of selection, that is, selecting a node.

[0133] The obtained r i =a i +∑ (i,j)∈E b ij x j ∈R is in state s i Execute action a i Rewards received i , which indicates the change of the objective function before and after the action is executed.

[0134] The decoder essentially transforms a state-action pair (s i , a i), converted to a scalar Q(s i ,a i ; Θ), used to evaluate the i Next action a i The quality of (i.e., the value, the greater the value, the greater the possibility of the action being executed), Θ is a learnable parameter, and the Q function is expressed as follows:

[0135] Q(h,v;Θ)=W 11 ReLU(W 10 ReLU(W9[z s ,z i ]))

[0136] Among them, W9, W10, W11 are learnable parameters, [z s ,z i ] means to convert vector z s and vector z i Splice by row.

[0137] (3) Minimizing the energy on the DQN-based QUBO graph, i.e. maximizing the number of nodes

[0138] 1. Training

[0139] After the network structure of the CONQUER deep learning model was constructed, the widely used DQN was used for model training. This process primarily involves the following steps: First, instances of the original problem are generated and converted into a Qubo graph. A series of trajectories are then generated based on the Qubo graph and stored in an experience replay area. The agent then selects batches of trajectories from the experience replay area for learning, and updates the neural network parameters accordingly, forming the parameters of the learning model through training instructions.

[0140] (1) Data preparation

[0141] In fact, it is necessary to generate instances of the maximum independent set problem. These instances are determined by the characteristics of the maximum independent set problem itself, and it is not necessarily required that the original problem instance must be a graph.

[0142] (2) Trajectory generation

[0143] As mentioned above, we map the optimization problem of the traffic network into a maximum clique problem on an abstract graph, where the nodes in the graph represent intersections, key points, etc. in the traffic network, while the edges can represent roads, lines, etc. between intersections, and the weights on the edges represent the traffic information of the route. At the same time, the maximum clique problem can be converted into a Hamiltonian minimization problem on the QUBO graph. We use a reinforcement learning algorithm to solve it, that is, we regard the process as a Markov decision process, in which an episode (a complete training round) is the complete process of finding a solution that minimizes the Hamiltonian on the QUBO graph, requiring all decision variable states to flip sequentially from 0 to 1. The trajectory (training trajectory data or training process data) generated in an episode is a state-action-reward sequence s0, a0, r0, s1, a1, r1, ..., s T , where s0 is the initial state where all decision variables are 0, s T is the terminal state where all decision variables are 1. Running a training process game requires completing a complete episode on the QUBO graph, and the model will collect training process data, i.e., s t ,a t ,r t,t+n ,s t+n , where s t Represents the state set of all variables at time t, a t represents the action taken at time t, represents the cumulative reward at time t+n, s t+n The model represents the state set of all variables at time t+n and stores it in the replay buffer (which is similar to a cached data result). Here, the delayed reward step size n and the discount factor γ are both hyperparameters.

[0144] (3) Update Qubormer parameters

[0145] Using the experience transitions selected from the replay buffer, CONQUER is able to learn and update the network parameters Θ to minimize the following loss function.

[0146]

[0147] Among them, E represents the maximum expectation, r t,t+n represents the immediate reward after n steps after executing the action at time t, s t+n represents the state at time t+n, a t+n represents the action to be performed at time t+n, γ is the discount factor (hyperparameter, usually 0.99), In state s t+n Target network for execution The action a with the largest return value t+n The reward estimate returned after is the target network parameter, Q(s t ,a t ; Θ) represents the state s t Execute action a t The reward estimate obtained based on the original network Q(Θ), where s t represents the state at time t, a t represents the action executed at time t, Θ is the parameter of the original network, which is updated by real-time data. In addition, the original network Q(Θ) periodically copies its parameters Θ to the target network Parameters To update the network.

[0148] 2. Online application and fine-tuning

[0149] In the application, a greedy strategy is used to iteratively flip the state of each decision variable in the QUBO graph from 0 to 1. Subsequently, the state of the decision variable associated with the flipping process is determined as the optimal solution to the QUBO problem, which produces the lowest Hamiltonian, the minimum energy, and the minimum value of the objective function in the QUBO form.

[0150] However, in the field of machine learning, out-of-distribution generalization is a prominent challenge. To address this challenge, the embodiments of the present invention propose a concise and efficient fine-tuning method.

[0151] First, data augmentation is performed on the training dataset, such as introducing noise into the Qubo graph, partitioning the dataset into training / validation / test subsets, and sampling the Qubo graph. Next, a pretrained model (previously trained) is loaded and its encoder portion is frozen. Only the task head of the downstream task is fine-tuned based on the downstream task data. This freezes the encoder parameters and updates only certain parameters to save fine-tuning time. Finally, the tuned model is tested to reflect the process of an online application.

[0152] (IV) Solution Output

[0153] 1. Result expression

[0154] In fact, the solution obtained directly from the model is the solution to the QUBO problem, which represents the value of the decision variable that minimizes the Hamiltonian. It involves a "yes / no" decision output, which is used to determine whether the decision variable is included in the solution set. This solution set is actually the partitioning of the maximum independent set problem.

[0155] 2. Check and repair the solution

[0156] If the agent is not fully trained due to data sparsity or insufficient training time, or if the penalty is set too low, the solution found by the neural network (in a network topology, identifying which nodes do not interfere with each other, that is, identifying the maximum independent set, which, from a communication perspective, means that nodes do not communicate on the same frequency, and correspondingly, that no two nodes are included by the same edge) may not meet the feasibility conditions of the original problem. To alleviate these problems, the legitimacy of the solution must first be ensured. If the solution meets the feasibility conditions of the original problem, it is directly output; if not, it needs to be repaired to meet these conditions. Generally speaking, repair methods are simple and inexpensive, and are usually used to repair infeasible solutions using common greedy or local search methods. Solution repair is only necessary when dealing with real-world datasets and very large graphs, and the simplest greedy method is used to repair the solution.

[0157] IV. Experiment

[0158] (1) Experimental setup

[0159] 1. Baseline

[0160] A baseline algorithm using semidefinite programming is used to obtain the relaxed solution. In addition, two heuristic methods, the greedy algorithm and the mean field annealing algorithm, are combined. The comparison of learning-based methods includes the classic S2V-DQN algorithm and two methods based on probabilistic views, the ERDOS algorithm and the ANNEAL algorithm. In addition, the current state-of-the-art method GFlowNet algorithm, which samples from the solution space, is evaluated. The baseline methods include two unsupervised learning methods, the RUN-CSP algorithm and the EGN algorithm, and their enhanced versions, the Meta-EGN algorithm. In addition, the latest method, the VAG-CO algorithm, which improves the mean field annealing algorithm, is also compared.

[0161] 2. Dataset

[0162] A cross-domain graph dataset is used to make the solution method generalize well and be able to be evaluated on larger-scale problems.

[0163] 3. Evaluation Metrics

[0164] To be consistent with previous work, a comprehensive evaluation metric is adopted, which considers the best ratio (R*) and average ratio (R) of the results obtained from the proposed method to the results obtained by the Gurobi optimizer within one hour, as well as the corresponding standard errors, to measure the efficiency and solution accuracy of the method.

[0165] (2) Real-world data results

[0166] Experiments were conducted on the following datasets: ENZYMES, PROTEINS, IMDB-BINARY, COLLAB, and MUTAG, and compared with the results from VAG-CO. The experiments were performed using a 0.6 / 0.1 / 0.3 split ratio and on the first 1000 graphs in the COLLAB dataset. Training was performed on RB graphs of size 100, and fine-tuning was performed on the other datasets. The experimental results are shown in Table 1.

[0167] Table 1 Experimental results

[0168]

[0169] It can be found that the method of the embodiment of the present invention reaches or is close to the optimal solution, and the fine-tuning process (ft) only requires a few hundred steps. Fine-tuning on the largest COLLAB dataset only requires no more than 1k steps.

[0170] The beneficial technical effects achieved by the embodiments of the present invention are as follows:

[0171] By modeling the node optimization task in a communication network as a maximum independent set problem in combinatorial optimization, the set of non-interfering nodes in the communication network is optimized by finding the maximum independent set. The maximum independent set problem is then formulated as a QUBO, demonstrating the groundbreaking development of a unified end-to-end deep learning reinforcement learning architecture tailored for this problem. Due to the particularity of the QUBO graph structure, a transformer mechanism is introduced into the communication network to effectively capture the long-term dependencies between nodes in the network. Then, a corresponding decision reduction strategy is designed for this problem, and a simple and effective fine-tuning method is designed for real communication network datasets that have a distribution different from the training set. Unlike all previous work, the framework of the embodiment of the present invention stands out by avoiding task-specific adjustments across various problem instances or fine-tuning scenarios. Experimental results show that the method of the embodiment of the present invention significantly outperforms existing state-of-the-art heuristic algorithms and can effectively optimize independent nodes in complex communication networks, achieving improved network performance and efficient resource utilization.

[0172] It should be understood that the specific order or hierarchy of steps in the disclosed processes is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of the present disclosure. The accompanying method claims present elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0173] In the foregoing detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that embodiments of the claimed subject matter require more features than are expressly recited in each claim. On the contrary, as reflected in the appended claims, the invention comprises less than all the features of any individual disclosed embodiment. The appended claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

[0174] The above description of the disclosed embodiments is intended to enable any person skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the spirit and scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments presented herein but is intended to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0175] The foregoing description includes examples of one or more embodiments. Of course, it is not possible to describe all possible combinations of components or methods for the purposes of describing the above embodiments, but one of ordinary skill in the art will recognize that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to encompass all such changes, modifications and variations that fall within the scope of the appended claims. Furthermore, to the extent the term "comprising" is used in the specification or claims, the term is intended to be encompassed in a manner similar to the term "including," as explained in terms of "including," used as a transitional word in the claims. Furthermore, any use of the term "or" in the specification of the claims is intended to mean a "non-exclusive or."

[0176] Those skilled in the art will also appreciate that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention can be implemented by electronic hardware, computer software, or a combination of the two. To clearly demonstrate the interchangeability of hardware and software, the various illustrative components, units, and steps described above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present invention.

[0177] The various illustrative logic blocks or units described in the embodiments of the present invention can be implemented or operated by a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor, and optionally, the general-purpose processor can also be any conventional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0178] The steps of the methods or algorithms described in the embodiments of the present invention may be directly embedded in hardware, a software module executed by a processor, or a combination of the two. The software module may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. For example, the storage medium may be connected to the processor so that the processor can read information from the storage medium and write information to the storage medium. Alternatively, the storage medium may also be integrated into the processor. The processor and storage medium may be provided in an ASIC, which may be provided in a user terminal. Alternatively, the processor and storage medium may also be provided in different components in the user terminal.

[0179] In one or more exemplary designs, the above-mentioned functions described in the embodiments of the present invention can be implemented in hardware, software, firmware, or any combination of the three. If implemented in software, these functions can be stored on a computer-readable medium or transmitted in the form of one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one location to another. Storage media can be any available medium that can be accessed by a general or special computer. For example, such computer-readable media can include but are not limited to RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store program code in the form of instructions or data structures and other forms that can be read by a general or special computer, or a general or special processor. In addition, any connection can be appropriately defined as a computer-readable medium. For example, if the software is transmitted from a website, server or other remote resource via a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless methods such as infrared, wireless, and microwave, it is also included in the definition of computer-readable media. The disks and discs mentioned above include compact disks, laser disks, optical disks, DVDs, floppy disks, and Blu-ray discs. Disks typically reproduce data magnetically, while discs typically reproduce data optically with lasers. Combinations of the above may also be included in computer-readable media.

[0180] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A node optimization method for a communication network, characterized in that: include: Optimizing the utilization problem of nodes of a communication network as a maximum independent set problem, wherein the nodes include: terminal nodes and transit nodes for converting and exchanging information; Convert the maximum independent set problem into a QUBO model; Constructing a corresponding QUBO diagram for the QUBO model; A graph neural network is used to characterize the structure and characteristics of the QUBO graph, and a deep learning model CONQUER is constructed. The deep learning model CONQUER is trained to explore the optimal solution to the node utilization problem of the communication network, thereby obtaining a trained deep learning model CONQUER; the trained deep learning model CONQUER is used to optimize the node utilization problem of any communication network, thereby obtaining a node optimization result, wherein the node optimization result includes: the nodes used, the number of nodes, and the connection relationship between the nodes; The node utilization problem of the communication network is optimized as a maximum independent set problem, including: The communication network is represented as G = (V, E), where V is the node set, including all nodes in the communication network, and E is the edge set, including all edges in the communication network. ij ∈E, edge e ij ∈E indicates that there is a connection relationship between nodes i and j; The node utilization problem of the communication network is transformed into a corresponding maximum independent set problem, so as to find a non-interfering node subset through the maximum independent set problem. To maximize the number of nodes in the non-interfering node subset, the maximum independent set problem is expressed as: Where X=(x1,x2.....x n ) represents the decision vector of whether the node is selected as a non-interference node. A non-interference node refers to a node whose communications do not interfere with each other.

2. The node optimization method of a communication network according to claim 1, characterized in that: The maximum independent set problem is transformed into a QUBO model, including: By adding a penalty term to the maximum independent set problem constraint, the maximum independent set problem is converted into a QUBO model, and the QUBO model is expressed as: Where, X=(x1,x2.....x n ) represents the decision vector composed of nodes, x i 、x j Respectively represent the values ​​of the i-th and j-th nodes, indicating whether these nodes are selected into the non-interference node subset. The matrix Q is a symmetric matrix or an upper triangular matrix. i =Q ii ,so b ij =Q ij , a i represents the node weight in the QUBO model, b ij Represents the edge weight on the QUBO model.

3. The node optimization method of a communication network according to claim 2, characterized in that: The step of constructing a corresponding QUBO graph for the QUBO model includes: A corresponding QUBO diagram is constructed for the QUBO model. The mathematical representation of the QUBO diagram is: Where p represents the penalty function.

4. The node optimization method of a communication network according to claim 3, characterized in that: We use graph neural networks to characterize the structure and characteristics of the QUBO graph and construct a deep learning model, CONQUER, which includes: Constructing an encoder, wherein the encoder construction specifically includes: integrating a centrality encoding layer, a message passing layer, a Transformer layer, and an iterative calculation solution layer; wherein: At the integrated centrality encoding layer: Each node in the QUBO graph is enhanced by the corresponding degree to capture the importance of the node in the QUBO graph and encoded. Each node in the QUBO graph is enhanced by the corresponding degree and encoded to obtain the corresponding vector representation: Among them, ReLU refers to the activation function, The i in the code represents the i-th node, 0 represents the initial feature, and x i is the initial feature of the node, z deg (v i ) encodes the degree of the node, v i It refers to the node with serial number i, W1 and W2 are the parameters of the neural network, and they are learnable parameters; At the messaging layer: The neighborhood aggregation process is adopted: edge feature aggregation and node feature aggregation to merge node features and edge features respectively. The edge feature aggregation and node feature aggregation are expressed as follows: Among them, h ij =ReLU(W3x ij ) is the initial edge embedding vector, which is the intermediate output of the neural network, that is, the initial features of the edge are embedded into the high-dimensional space, x ij is the initial feature of the edge, i.e., the input feature of the edge. W3, W4, W5, W6, and W7 are the parameters of the neural network, and they are all learnable parameters. After obtaining the node features in step l Afterwards, by splicing the features of the previous step of the node Residual connection, the residual connection is expressed as follows: Among them, W8 is the parameter of the neural network and is a learnable parameter; In the Transformer layer: Aggregate the output of the message passing layer with the output of the global attention layer within the Transformer layer to obtain updated node features: Among them, MHA represents multi-head attention network, LN represents layer normalization, FFN represents feedforward neural network layer, represents the node embedding vector obtained at step l-1, represents the node embedding vector after the multi-head attention network; In the iterative calculation solution layer: When solving, the updated node features output from the Transformer layer are used as the input of the integrated centrality encoding layer. The steps of the integrated centrality encoding layer, the message passing layer, and the Transformer layer are repeatedly executed in a loop to iterate. After the iteration is completed, the final embedding vector corresponding to each node is obtained. The final embedding vector of each node is expressed as: Sum the final embedding vectors of all nodes to get: s =∑ i∈V z i , the solution of the QUBO model is characterized by summation.

5. The node optimization method of a communication network according to claim 4, characterized in that: A graph neural network is used to characterize the structure and characteristics of the QUBO graph, and a deep learning model CONQUER is constructed, which also includes: Build the decoder, including: Formally, a Markov decision process is formed, which is described by the tuple (S, A, P, R), where S is the number of possible states s. i A collection of actions i ∈A represents the assignment of the binary variable X of the QUBO model for a given state si; In state s i Execute action a i Get reward i :r i =a i +∑ (i,j)∈E b ij x j ∈R, reward r i Indicates the change of the objective function before and after the execution of the action Markov decision process; Through the decoder, a pair of state-action pairs (s i , a i ) is converted to a scalar Q(s i ,a i ; Θ), to evaluate the i Next action a i The quality of the decoder is expressed as follows: Q(h,v;Θ)=W 11 ReLU(W 10 ReLU(W9[z s ,With i ])) Among them, W9, W10, and W11 are parameters of the neural network and can be learned. s ,z i ] means to convert vector z s and vector z i Splice by row.

6. The node optimization method of a communication network according to claim 5, characterized in that: By training the deep learning model CONQUER to explore the optimal solution to the node utilization problem of the communication network, the trained deep learning model CONQUER is obtained, including: Obtaining a sample of nodes of a communication network corresponding to the maximum independent set problem, and generating a trajectory based on a Markov decision process representation based on the QUBO graph; Among them, an episode is the complete process of finding the solution that minimizes the Hamiltonian on the QUBO graph, requiring all decision variable states to flip sequentially from 0 to 1; the trajectory generated in a training round is a state-action-reward sequence: s0,a0,r0,s1,a1,r1,...,s T ,...,s T s0 is the initial state where all decision variables are 0, s T is the terminal state where all decision variables are 1; completing a training process requires a complete training round on the QUBO graph, and a complete training round includes the generation of trajectories and the collection of experience transformations s t ,a t ,r t,t+n ,s t+n , r t+n represents the cumulative reward at time t+n, the delayed reward step n is a hyperparameter, and the discount factor γ is a hyperparameter; Using the experience transformations selected from the storage structure, the deep learning model CONQUER is enabled to learn and update the network parameters Θ of the deep learning model CONQUER to minimize the following loss function: Among them, E represents the maximum expectation, r t,t+n represents the immediate reward after n steps after executing the action at time t, s t+n represents the state at time t+n, a t+n Indicates the action to be performed at t+n. In state s t+n Target network for execution The action a with the largest return value t+n The reward valuation returned after is the target network parameter; Q(s t ,a t ; Θ) represents the state s t Execute action a t The reward estimate obtained from the original network Q(Θ), s t represents the state at time t, a t represents the action executed at time t, Θ is the parameter of the original network, which updates each parameter through real-time data; the original network Q(Θ) regularly copies the parameter Θ to the target network Parameters To achieve network updates; When the loss function value is less than the preset value, the trained deep learning model CONQUER can be obtained.

7. The node optimization method of a communication network according to claim 6, characterized in that: Also includes: In the obtained sample of nodes of the communication network corresponding to the maximum independent set problem, noise is introduced into the sample when constructing the QUBO graph; When the loss function is less than a preset value, it is also necessary to freeze the parameters of the encoder and fine-tune the parameters of the encoder for the corresponding deep learning model CONQUER to obtain a trained deep learning model CONQUER.

8. The node optimization method of a communication network according to claim 7, characterized in that: Also includes: When the trained deep learning model CONQUER is used to solve the node utilization problem of the communication network to be solved, if the obtained solution does not meet the feasibility conditions of the communication network to be solved, a greedy search method or a local search method is used to repair the solution.

9. A node optimization method for a communication network, characterized in that: Also includes: Inputting the node utilization problem of the communication network to be solved into the trained deep learning model CONQUER according to any one of claims 1 to 8; Output node optimization results, including: adopted nodes, number of nodes, and connection relationships between nodes.

Images