Variable length set decision multi-objective optimization method for key node selection of communication network
By combining reinforcement learning graph neural networks and multi-objective evolutionary algorithms, the problem of multi-objective optimization in the selection of key nodes in communication networks is solved, achieving efficient and stable solution set generation and diverse Pareto solution set outputs, which are suitable for the selection of key nodes in communication networks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGZHOU RES INST OF XIAN UNIV OF ELECTRONIC SCI & TECH
- Filing Date
- 2026-03-02
- Publication Date
- 2026-06-09
AI Technical Summary
Existing technologies struggle to achieve multi-objective coordinated optimization of throughput and latency under multiple engineering constraints in the selection of key nodes in communication networks, and are also unable to output a well-distributed Pareto solution set, resulting in low solution generation efficiency, slow convergence speed, and poor stability.
A fusion approach based on reinforcement learning graph neural network (RL-GNN) and multi-objective evolutionary algorithm SPEA2 is adopted. By constructing a Markov decision process, high-quality and diverse feasible seed solutions are generated. Combined with variable-length integer encoding, constraint priority selection and comprehensive repair operators, multi-objective optimization is achieved.
Under multiple constraints, it significantly improves the efficiency of feasible solution generation and search convergence speed, outputs a diverse set of Pareto solutions, and enhances the stability and engineering feasibility of the algorithm.
Smart Images
Figure CN122179325A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of communication technology, and in particular to a multi-objective optimization method for decision-making on variable-length sets for selecting key nodes in communication networks. Background Technology
[0002] Communication networks (such as backbone networks, metropolitan area networks, data center networks, and wireless access / backhaul networks) are widely used for information transmission and service delivery. Common requirements for the operation, maintenance, and performance assurance of communication networks include the selection of critical nodes for key network equipment (such as routers, switches, base stations, and gateways), network vulnerability assessment, robustness enhancement, and prioritization decisions for equipment upgrades / hardening and maintenance under resource-constrained conditions. These problems typically require selecting a set of devices from candidate devices, constrained by both network topology and device attributes, so that the network can still meet service quality requirements after performing corresponding operations, and that several network performance indicators (such as end-to-end throughput, average latency, critical service latency, or their proxies) reach optimal or near-optimal levels.
[0003] Since equipment set selection is a typical discrete combinatorial optimization problem, often accompanied by multiple conflicting optimization objectives and various engineering constraints, its solution difficulty increases exponentially with network size. Existing technologies for solving the problem of equipment upgrade site selection in communication networks are mainly divided into four categories, but all have limitations: The first category is based on heuristic sorting and greedy strategies (such as sorting by centrality or successive optimization). These methods are simple to implement and fast, but they rely on a single topology metric and struggle to handle multi-objective conflicts such as throughput and latency, as well as multiple constraints such as maintenance window, budget, equipment availability, and outage risk. When facing network heterogeneity, strict constraints, or nonlinear performance evaluation, they are prone to unstable solutions, few feasible solutions, and poor generalization ability.
[0004] The second category is mathematical programming methods (such as integer programming and Lagrange relaxation), which are used to seek global optimality or stronger theoretical guarantees. However, communication network optimization is usually large-scale, non-convex, and has many constraints. The objective needs to be evaluated through routing calculations or simulations, which limits the complexity and engineering feasibility of accurate solutions. At the same time, the model has high requirements for linearization and is difficult to cover various practical constraints such as operation and maintenance budgets and outage risks, making modeling and solving difficult.
[0005] The third category is evolutionary computation and multi-objective evolutionary algorithms (such as NSGA-II and SPEA2). Although they have global search capabilities, they face challenges under multiple constraints: device set decisions are naturally variable-length or set-type structures, and commonly used binary fixed-length encoding will lead to dimensional expansion and decreased search efficiency; multiple coupled constraints cause random initialization and crossover mutation to generate a large number of infeasible solutions, reducing convergence efficiency; if the penalty function or repair strategy is not designed properly, it can easily destroy the diversity and stability of the solutions.
[0006] The fourth category is reinforcement learning methods. Reinforcement learning methods (especially policy gradient methods) have shown potential in sequential decision-making and combinatorial optimization. They can model the process of "gradually selecting network devices to form a set" as a Markov decision process and achieve adaptive point selection through policy learning. Graph neural networks can learn representations of communication network topology and device attributes, enhancing the policy's generalization ability to communication networks of different sizes and structures. However, relying solely on reinforcement learning also has shortcomings: the training process is sensitive to sample efficiency and reward design; when faced with multiple objectives such as throughput and latency and strict engineering constraints, the policy may get stuck in local optima or produce solutions with insufficient feasibility; in addition, reinforcement learning usually outputs a single or a small number of solutions, making it difficult to directly provide a Pareto solution set for engineering trade-offs, and often requires additional scalarization design for multi-objective trade-offs, which can easily introduce bias.
[0007] Therefore, there is an urgent need to provide a variable-length set decision-making multi-objective optimization method for the selection of key nodes in communication networks, which can simultaneously achieve: improving the efficiency of generating feasible solutions in a large-scale discrete combinatorial space; outputting a well-distributed Pareto solution set under multi-objective conflict conditions; and effectively utilizing prior information obtained from network structure representation and learning to improve the search convergence speed and solution quality stability, thereby meeting the engineering requirements of communication network operation and maintenance and performance assurance. Summary of the Invention
[0008] To address the problems existing in the prior art, this invention provides a variable-length set decision-making multi-objective optimization method for selecting key nodes in communication networks. Under multiple engineering constraints such as maintenance window, operation and maintenance budget, equipment availability and service interruption risk, existing network equipment upgrade site selection methods are difficult to implement. This invention can efficiently and stably achieve multi-objective collaborative optimization of throughput and latency, and output diverse feasible solutions for engineering trade-offs.
[0009] The technical solution of this invention is implemented as follows: A multi-objective optimization method for decision-making using a variable-length set for selecting key nodes in a communication network includes the following steps: Step 1: Obtain the data of the original communication network and abstract it into a graph structure; for each node in the original communication network, construct an initial feature vector, define node attributes, and set a node state mask; Specifically, the graph structure is represented as , This represents the set of nodes in the graph. This represents the set of edges between nodes; the graph structure uses an adjacency matrix / adjacency list. This is indicated. Furthermore, an edge attribute matrix is constructed for each edge. The edge attribute is one or a combination of link capacity, latency, or weight. It can also be other attributes / features.
[0010] Based on this, The first in Each node is represented as , The feature vector is represented as The node state mask is It is represented using binary encoding or one-hot encoding.
[0011] Binary encoding is either "0" or "1". One-hot encoding is a method to convert categorical data (such as color, city, type) into a numerical format that is easier for machine learning algorithms to understand. For example, it creates a new binary vector (containing only 0s and 1s) with a length equal to the total number of categories, where only one position is "1" and the rest are "0".
[0012] Set a multi-constraint threshold for the node attributes; initialize the evolutionary algorithm parameters and an empty solution set, wherein the node state mask indicates whether the corresponding node is selected into the solution set; the multi-constraint threshold is used to constrain the solution set; Evolutionary algorithm parameters include population size Maximum number of iterations and external file capacity The initial solution set is represented as .
[0013] Step 2: Based on at least two evaluation metrics for the solution set, construct a multi-objective evaluation function; define constraint feasibility determination rules, and based on the node attributes and the multi-constraint threshold, construct a constraint violation function to measure the degree of constraint violation by the solution set; Step 3: Construct a policy model based on RL-GNN according to the graph structure; the policy model includes states, actions, and state transitions; RL stands for Reinforcement Learning, and GNN stands for Graph Neural Network. The above process models the construction of the solution set as a Markov Decision Process (MDP).
[0014] The state includes the graph structure, the embedded representation of the nodes, and the cumulative constraint state of the solution set; the action is to select a node or terminate the action; the state transition is to update the solution set and the cumulative constraint state according to the selected node; the cumulative constraint state is calculated based on the node attributes of all nodes in the solution set. The policy model is trained using the policy gradient method to obtain the final policy; The policy model is trained using the policy gradient method, which maximizes the reward function that balances the marginal increment of the objective with the feasibility of constraints during the interaction process, thus obtaining the final policy after training. Specifically, a graph-level representation is obtained through global pooling. and cumulative constraint state Fusion forms state representation ,Right now: The policy head then outputs the action probability for each node, where the node... The probability is: This means prioritizing nodes with higher marginal contributions and greater feasibility within the current constraint margin. The function is a normalization function that ensures the sum of the selection probabilities of all nodes is 1. This is the strategy function.
[0015] "Target marginal increment" refers to the current solution set Add candidate nodes based on Afterwards, the objective function value is changed from... Become The resulting gain.
[0016] "Taking into account the feasibility of constraints" means that while pursuing a larger marginal increment, it is necessary to ensure that the solution set after adding nodes meets the preset constraints. When a candidate node causes a constraint violation, feasibility filtering, penalty term suppression, or repair operators can be used to adjust the solution into a feasible solution, thereby achieving a unified optimization of target gain and constraint satisfaction.
[0017] To guide the strategy in a way that balances goal optimization and feasibility, this invention designs the immediate reward as a "goal increment". Forms of "restraint and punishment": ; in, It can be constructed by scalarization approximation of weighted marginal increments or vector increments of multiple objectives.
[0018] This invention can use policy gradient methods such as Proximal Policy Optimization (PPO) for training, and improve training stability through pruning mechanisms and entropy regularization terms, avoiding performance fluctuations caused by excessive policy updates in a large action space.
[0019] Step 4: Based on the current state, select a node as a candidate node from all the nodes according to the final strategy; obtain several candidate nodes and add them to the empty solution set, and update the cumulative constraint state and the node state mask; a solution set is a candidate solution of variable length; generate multiple candidate solutions and construct the initial population; Step 5: Establish an external elite profile; using the multi-objective evaluation function and the constraint violation function, calculate the multi-objective value and violation value of each individual in the initial population, and add non-dominated individuals to the external elite profile; use constraint-aware SPEA2 to calculate the fitness of the individuals; Step 6: Select individuals based on their fitness and perform crossover and mutation operations to generate new offspring individuals to update and optimize the external elite profile; Step 7: Obtain and output the non-dominated solution set from the external elite archive. The non-dominated solution set is also known as the Pareto front solution set. One solution corresponds to one solution.
[0020] As a further optimization of the above solution, the node attributes include device deployment cost. Window maintenance time and fault recovery success rate ; The multiple constraint thresholds include a cost threshold. Time threshold Success rate threshold ; The constraint feasibility determination rule is expressed as follows: Cost constraints: ; Time constraints: ; Success rate constraint: ; Denotes the solution set, Indicates the first one 1 node , , They are respectively The node attributes.
[0021] As a further optimization of the above scheme, the evaluation metrics include maximizing residual throughput and minimizing average end-to-end latency.
[0022] This invention constructs a target communication network based on the original communication network. The evaluation index is the performance index of the target communication network, which can be calculated through simulation or a performance prediction model. get.
[0023] The two evaluation metrics above conflict with each other. For a solution set, the corresponding two evaluation metrics are expressed as follows: and Then the multi-objective evaluation function is expressed as: .
[0024] As a further optimization of the above scheme, multiple rounds of message passing are performed based on the graph structure and the feature vectors of the nodes to obtain the embedding matrix of the nodes; wherein, the nodes The The embedding of layers is represented as ,Right now: ; in, It is a non-linear activation function; ; For nodes The set of neighboring nodes; Let the edge feature weight function be... Show node and nodes The edge properties of the edges between them; Used to This is mapped to a weight scalar or vector, which determines the neighboring nodes. Information for the current node The degree of contribution reflects the "constraint attribute of the fusion device node".
[0025] and Both are trainable weight matrices, used for feature transformation of the current node and feature transformation of neighboring nodes, respectively.
[0026] The above process represents the update of the node's state vector. Unlike general GNNs that only encode the topology, this invention integrates device node constraint attributes and dynamic masks in information aggregation, so that the embedding explicitly reflects "remaining candidate space + constrained feasible region".
[0027] Through the above feature transformations, the embedding matrix composed of all nodes can be obtained. At the same time, obtain the action probability distribution , For nodes The final embedding representation, Indicates the number of nodes.
[0028] This invention compresses the network structure and node attributes into node embeddings that can be used for decision-making, and outputs the probability distribution of point selection under the current constraints, thereby generating high-quality candidate solutions.
[0029] As a further optimization of the above scheme, the process of generating / updating the cumulative constraint state is as follows: .
[0030] As a further optimization of the above scheme, the constraint violation function is expressed as follows: ,Right now: .
[0031] As a further optimization of the above scheme, the specific process of step 4 is as follows: Step 4-1: For an empty solution set, based on the current state... The action probability output by the final strategy is used to select a node, which is denoted as a candidate node. Step 4-2: Update the solution set using the candidate nodes, and simultaneously update the cumulative constraint state and the node state mask of the candidate nodes; wherein, the modified node state mask of a node that has been added to the solution set is indicated as not being selectable again, so that there are no duplicate nodes in a solution set.
[0032] The solution set is represented as Candidate nodes are represented as The update process of the solution set is then expressed as: .
[0033] Step 4-3: Repeat steps 4-1 and 4-2 until the update termination condition of a solution set is met, and a candidate solution of variable length is obtained.
[0034] Candidate solutions are denoted as , This indicates the length of the candidate solution, which is the number of nodes it contains.
[0035] Furthermore, the termination condition can be updated to include reaching the size cap, insufficient remaining constraints to continue selecting points, or triggering a termination action. Therefore, "variable length" means that the length of a candidate solution may not meet the size cap.
[0036] As a further optimization of the above solution, the specific process of step 5 is as follows: Step 5-1: Initialize the external elite profile and constraint-aware SPEA2; SPEA2 stands for StrengthPareto Evolutionary Algorithm 2, which is an improved strength Pareto evolutionary algorithm.
[0037] In practical problems, it is necessary to optimize multiple objectives simultaneously, but these objectives often conflict with each other. For example, when selecting nodes, one may want to achieve both "lowest cost" and "maximum coverage." Typically, there is no single optimal solution, but rather a set of Pareto optimal solutions (i.e., solutions that cannot improve any other objective without compromising the others). The purpose of SPEA2 is to find this set of optimal balanced solutions.
[0038] Step 5-2: Use variable-length integer encoding to represent the ID sequence or ID set of an individual; use the multi-objective evaluation function and the constraint violation function to calculate the multi-objective value and violation value of each individual in the initial population, respectively. Specifically, an individual is represented as a sequence / set of node IDs. ,length Variable; this encoding avoids the dimensionality explosion caused by fixed-length binary vectors and is more in line with the essential structure of node set decision-making.
[0039] Step 5-3: Add non-dominated individuals to the external elite profile; use constraint-aware SPEA2 to calculate the fitness of the individuals based on density estimation and individual strength.
[0040] Specifically, the fitness of the i-th individual is represented as: ,Right now ;in This is a density estimate, obtained by calculating the reciprocal of the distance to the k-th nearest neighbor; The initial fitness is calculated based on the individual's strength value, i.e.: . This refers to the strength value of the i-th individual. The quality of each solution depends not only on its own quality but also on how many other solutions it dominates (i.e., how many other solutions it is better than). The more others it dominates, the higher its strength value, indicating a better solution. This indicates that the i-th solution dominates the j-th solution, meaning that solution i is no worse than solution j in all objectives, and has at least one objective that is strictly better than solution j.
[0041] By combining strength dominance and density truncation, the non-dominated solution set is guaranteed to be both convergent and uniformly distributed.
[0042] As a further optimization of the above solution, the specific process of step 6 is as follows: Step 6-1: Execute constraint-priority selection: Prioritize selecting feasible individuals for reproduction; when there are insufficient feasible individuals, supplement them according to the violation degree value from smallest to largest, thereby stabilizing the evolutionary process and improving the effective evaluation ratio. The feasible individual is the individual that satisfies all constraints, and is determined according to the constraint feasibility determination rules. Step 6-2: Use the selected individuals as parent individuals to perform crossover operations; further, perform single-point or double-point segment swaps on the two parent individual sequences to generate child individuals.
[0043] Step 6-3: Perform mutation operations on the offspring individuals; the number of mutations can be determined according to a preset probability or Poisson distribution sampling; mutation types include node replacement, node addition, or node deletion; Furthermore, the number of mutations is set to follow a Poisson distribution, and each mutation performs one of the following: "randomly replace nodes, randomly add nodes, or randomly delete nodes" to achieve a balance between strong perturbation and local refinement in the discrete space.
[0044] Step 6-4: Perform the comprehensive repair operator on the offspring individuals; Specifically, the individual representations are first validated by removing duplicate nodes, eliminating invalid indexes, and normalizing variable-length integer encoding. Then, constraint violation degrees are calculated based on a predefined set of constraints (budget, time, reliability / success rate, risk, etc.). The node set is adjusted according to the principle of "prioritizing the elimination of nodes with the highest violation degrees or contributions, and replacing or supplementing them with nodes of better attributes within the feasible region." This ensures that individuals satisfy the constraint thresholds while maintaining their original multi-objective performance advantages and solution set diversity as much as possible. Through this comprehensive repair operator, the proportion of effective feasible individuals in the evolutionary process is significantly increased, thereby improving the convergence speed and stability of multi-objective search and reducing sensitivity to the selection of penalty function parameters.
[0045] Step 6-5: Store all offspring individuals in the external elite archive and recalculate the strength value, density estimate, and fitness of each individual in the external elite archive. When the capacity of the external elite archive exceeds the limit, density truncation is performed to maintain the distribution of the Pareto front solution set. Specifically, density truncation involves finding the individuals with the highest density (most crowded) based on the density estimates of individuals in the external elite archive, deleting them, and then updating the density information of the remaining individuals. The deletion operation is repeated until the archive size reaches the limit.
[0046] Repeat steps 6-1 to 6-5 until the maximum number of iterations is reached or the preset convergence condition is met.
[0047] As a further optimization of the above scheme, for each solution in the non-dominated solution set, the solution set, multi-objective values, and constraint satisfaction information for each node are output to support communication network engineering decision-making under different preferences. The constraint satisfaction information can be the attribute that satisfies the constraint or the violation value.
[0048] This invention addresses the discrete combinatorial optimization problem of "selecting a set of key nodes" in communication network scenarios, proposing a multi-constraint, multi-objective optimization method based on the fusion of reinforcement learning graph neural networks (RL-GNN) and the multi-objective evolutionary algorithm SPEA2. This method models the node selection process as a Markov decision process, utilizes graph neural networks to extract network structure and node attribute representations, and generates high-quality and diverse feasible seed solutions through reinforcement learning strategies. The seed solutions (individuals) then guide SPEA2 in a global multi-objective search, combining variable-length integer encoding, constraint-priority selection, and comprehensive repair operators. Under the premise of satisfying multiple constraints such as budget, time, reliability / success rate, and risk, it outputs a well-distributed Pareto non-dominated solution set, providing multiple options for engineering decision-making.
[0049] Compared with existing technologies, the variable-length set decision-making multi-objective optimization method for selecting key nodes in communication networks provided by this invention has the following beneficial effects: (1) Before multi-objective evolutionary search, this invention uses reinforcement learning graph neural network to generate high-quality and diverse seed solution sets, so that the initial population is closer to the Pareto front region, reducing the blind exploration of the evolutionary algorithm in a large-scale discrete space, thereby accelerating convergence and improving the quality of the final solution.
[0050] (2) To address the “feasible solution sparsity” problem caused by multiple constraints such as operation and maintenance budget, maintenance window, and equipment availability, this invention introduces masking and feasible domain control in the strategy reasoning stage and adopts constraint priority selection and comprehensive repair mechanism in the evolution stage, which significantly improves the proportion of effective feasible individuals in the evolution process and enhances the stability and repeatability of the algorithm.
[0051] (3) Existing fixed-length binary encoding leads to dimensional expansion and decreased search efficiency in large-scale networks. This invention uses variable-length integer encoding to directly represent the node set, which is more in line with the nature of the problem and facilitates crossover, mutation and repair operations, thereby reducing computational complexity and improving search efficiency.
[0052] (4) Compared with the method that only relies on the penalty function to handle constraints, the present invention restores the feasibility of infeasible individuals by integrating the repair operator, avoiding search bias or training instability caused by improper selection of penalty factors, while preserving the multi-objective advantages and solution set diversity of individuals as much as possible, thereby improving the overall robustness.
[0053] (5) Furthermore, the technical solution of this invention has good portability and can be applied to other similar technical fields with network topology characteristics. For example, in the Internet of Things, vehicle-to-everything (V2X), power communication networks, social networks, or transportation networks, there is a common need to adjust the state of network nodes or links to optimize overall performance. By mapping the network topology, node / link attributes, engineering constraints, and optimization objectives in specific application scenarios to the decision framework constructed by this invention, the technical solution can be migrated and applied.
[0054] In summary, this invention combines the policy prior of reinforcement learning graph neural networks with constraint-aware multi-objective evolutionary search, achieving an integrated solution mechanism of "high-quality initialization—stable feasible evolution—Pareto solution set output" in the multi-constraint, multi-objective key node selection task of communication networks. It significantly improves the efficiency of feasible solution generation and search convergence speed under multiple coupled constraints such as operation and maintenance budget, maintenance window, and equipment availability threshold, while maintaining the diversity and uniform distribution of the solution set. The final output is a set of non-dominated solutions with clear trade-offs, thereby improving the stability, robustness, and engineering feasibility of the algorithm in large-scale discrete combinatorial spaces. Attached Figure Description
[0055] Figure 1 This is a flowchart illustrating a multi-objective optimization method for selecting key nodes in a communication network using a variable-length set decision, provided by an embodiment of the present invention. Detailed Implementation
[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions in the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0057] like Figure 1 As shown, this embodiment provides a multi-objective optimization method for variable-length set decision-making in the selection of key nodes in a communication network, including the following steps: Step 1: Acquire and preprocess the data of the original communication network; preprocessing includes abstracting it into a graph structure; for each node in the original communication network, construct an initial feature vector, define node attributes, and set a node state mask; In this embodiment, the graph structure is represented as follows: , This represents the set of nodes in the graph. This represents the set of edges between nodes; the graph structure uses an adjacency matrix / adjacency list. This is indicated. Furthermore, an edge attribute matrix is constructed for each edge. The edge attribute is one or a combination of link capacity, latency, or weight. It can also be other attributes / features.
[0058] Specifically, node attributes include device deployment cost, window maintenance time, and fault recovery success rate. In other embodiments, node attributes can also be other optional attributes such as business / function weights, like device type, business weight, historical statistics, etc.
[0059] Set multiple constraint thresholds for node attributes; initialize evolutionary algorithm parameters and an empty solution set; node state mask indicates whether the corresponding node is selected into the solution set; multiple constraint thresholds are used to constrain the solution set. Based on this, The first in Each node is represented as , The feature vector is represented as Equipment deployment cost is expressed as The window maintenance time is expressed as The fault recovery success rate is expressed as The node state mask is It is represented using binary encoding or one-hot encoding.
[0060] Binary encoding is either "0" or "1". One-hot encoding is a method to convert categorical data (such as color, city, type) into a numerical format that is easier for machine learning algorithms to understand. For example, it creates a new binary vector (containing only 0s and 1s) with a length equal to the total number of categories, where only one position is "1" and the rest are "0".
[0061] This embodiment uses binary encoding.
[0062] Multiple constraint thresholds include cost thresholds Time threshold Success rate threshold , respectively corresponding , , In other embodiments, constraints may also include upper limits on set size, node type quotas, and area coverage constraints.
[0063] Evolutionary algorithm parameters include population size Maximum number of iterations and external file capacity The initial solution set is represented as .
[0064] Step 2: Construct multi-objective evaluation and constraint violation calculation rules: Based on at least two evaluation indicators for the solution set, construct a multi-objective evaluation function; define constraint feasibility judgment rules, and construct a constraint violation function based on node attributes and multiple constraint thresholds to measure the degree of constraint violation by the solution set; This invention constructs a target communication network based on the original communication network. The evaluation index is the performance index of the target communication network, which can be calculated through simulation or a performance prediction model. Therefore, the evaluation metrics include maximizing residual throughput and minimizing average end-to-end latency, which are conflicting. For a given solution set, the corresponding two evaluation metrics are expressed as follows: and Then the multi-objective evaluation function is expressed as: .
[0065] When more evaluation indicators exist, it is represented as In this embodiment, the constraint feasibility determination rule is expressed as follows: Cost constraints: ; Time constraints: ; Success rate constraint: .
[0066] Step 3: Construct a reinforcement learning environment and train an RL-GNN policy model: Construct an RL-GNN-based policy model based on the graph structure; the policy model includes states, actions, and state transitions; The state includes the graph structure, the embedded representation of nodes, and the cumulative constraint state of the solution set; the action is to select a node or terminate the action; the state transition is to update the solution set and the cumulative constraint state based on the selected node; the cumulative constraint state is calculated based on the node attributes of all nodes in the solution set. The policy model is trained using the policy gradient method to obtain the final policy; RL stands for Reinforcement Learning, and GNN stands for Graph Neural Network. The above process models the construction of the solution set as a Markov Decision Process (MDP).
[0067] In this embodiment, multiple rounds of message passing are performed based on the graph structure and the feature vectors of the nodes to obtain the node embedding matrix. The The embedding of layers is represented as ,Right now: ; in, It is a non-linear activation function; ; For nodes The set of neighboring nodes; Let the edge feature weight function be... Show node and nodes The edge properties of the edges between them. Used to This is mapped to a weight scalar or vector, which determines the neighboring nodes. Information for the current node The degree of contribution reflects the "constraint attribute of the fusion device node".
[0068] and Both are trainable weight matrices, used for feature transformation of the current node and feature transformation of neighboring nodes, respectively.
[0069] The above process represents the update of the node's state vector. Unlike general GNNs that only encode the topology, this invention integrates device node constraint attributes and dynamic masks in information aggregation, so that the embedding explicitly reflects "remaining candidate space + constrained feasible region".
[0070] Through the above feature transformations, the embedding matrix composed of all nodes can be obtained. At the same time, obtain the action probability distribution , For nodes The final embedding representation, Indicates the number of nodes.
[0071] This invention compresses the network structure and node attributes into node embeddings that can be used for decision-making, and outputs the probability distribution of point selection under the current constraints, thereby generating high-quality candidate solutions.
[0072] Specifically, cumulative constraint state The generation / update process is as follows: ; The policy model is trained using the policy gradient method, which maximizes the reward function that balances the marginal increment of the objective with the feasibility of constraints during the interaction process, thus obtaining the final policy after training. Specifically, a graph-level representation is obtained through global pooling. and cumulative constraint state Fusion forms state representation ,Right now: The policy head then outputs the action probability for each node, where the node... The probability is: This means prioritizing nodes with higher marginal contributions and greater feasibility within the current constraint margin. The function is a normalization function that ensures the sum of the selection probabilities of all nodes is 1. This is the strategy function.
[0073] "Target marginal increment" refers to the current solution set Add candidate nodes based on Afterwards, the objective function value is changed from... Become The resulting gain.
[0074] "Taking into account the feasibility of constraints" means that while pursuing a larger marginal increment, it is necessary to ensure that the solution set after adding nodes meets the preset constraints. When a candidate node causes a constraint violation, feasibility filtering, penalty term suppression, or repair operators can be used to adjust the solution into a feasible solution, thereby achieving a unified optimization of target gain and constraint satisfaction.
[0075] To guide the strategy in a way that balances goal optimization and feasibility, this invention designs the immediate reward as a "goal increment". Forms of "restraint and punishment": ; in, It can be constructed by scalarization approximation of weighted marginal increments or vector increments of multiple objectives. The constraint violation function is as follows: The constraint violation function is expressed as ,Right now: .
[0076] This invention can use policy gradient methods such as Proximal Policy Optimization (PPO) for training, and improve training stability through pruning mechanisms and entropy regularization terms, avoiding performance fluctuations caused by excessive policy updates in a large action space.
[0077] Step 4: Based on the current state and the final policy, select one node from all nodes as a candidate node; obtain several candidate nodes and add them to an empty solution set, and update the cumulative constraint state and node state mask; a solution set is a candidate solution of variable length; generate multiple candidate solutions and construct the initial population; the specific steps are as follows: Step 4-1: For an empty solution set According to the current state Utilizing the final strategy The output action probability selects a node, denoted as a candidate node. ; The initial state is determined by the network structure and node characteristics. The initial mask is all 0 and the cumulative constraint state is initialized to 0. When constructing the initial population, the action probabilities of the policy output can be randomly sampled when generating candidate solutions, so as to obtain different variable-length candidate solutions.
[0078] Step 4-2: Utilize candidate nodes Update the solution set, that is: When updating the solution set, the cumulative constraint state is updated synchronously, as well as the node state mask of the candidate nodes. When updating the solution set, it is necessary to determine the feasibility of candidate nodes, that is, to execute according to the constraint feasibility determination rules.
[0079] In this context, the modified state mask of a node already added to the solution set indicates that it cannot be selected again, ensuring that no two nodes in the solution set are the same. For example, a node state mask of 0 indicates that it is selectable; a state mask of 1 indicates that it cannot be selected twice by the same solution set.
[0080] Step 4-3: Repeat steps 4-1 and 4-2 until the termination condition for updating a solution set is met, resulting in a candidate solution of variable length, denoted as... , This indicates the length of the candidate solution, which is also the number of nodes it contains; Specifically, the termination condition can be reaching the size limit, insufficient remaining constraints to continue selecting points, or triggering a termination action. Therefore, "variable length" means that the length of a candidate solution may not meet the size limit.
[0081] Step 5: Establish an external elite profile; using the multi-objective evaluation function and constraint violation function, calculate the multi-objective value and violation value of each individual in the initial population, and add non-dominated individuals to the external elite profile; use constraint-aware SPEA2 to calculate the fitness of individuals; the specific steps are as follows: Step 5-1: Initialize the external elite profile and constraint-aware SPEA2; SPEA2 stands for StrengthPareto Evolutionary Algorithm 2, which is an improved strength Pareto evolutionary algorithm.
[0082] In practical problems, it is necessary to optimize multiple objectives simultaneously, but these objectives often conflict with each other. For example, when selecting nodes, one may want to achieve both "lowest cost" and "maximum coverage." Typically, there is no single optimal solution, but rather a set of Pareto optimal solutions (i.e., solutions that cannot improve any other objective without compromising the others). The purpose of SPEA2 is to find this set of optimal balanced solutions.
[0083] Step 5-2: Use variable-length integer encoding to represent the ID sequence or set of individuals; calculate the multi-objective value for each individual. and violation value ; Specifically, an individual is represented as a sequence / set of node IDs. ,length Variable; this encoding avoids the dimensionality explosion caused by fixed-length binary vectors and is more in line with the essential structure of node set decision-making.
[0084] Step 5-3: Add non-dominated individuals to the external elite profile; calculate the fitness of individuals using constraint-aware SPEA2; where the fitness of the i-th individual is represented as... ,Right now ;in This is a density estimate, obtained by calculating the reciprocal of the distance to the k-th nearest neighbor; The initial fitness is calculated based on the individual's strength value, i.e.: . This refers to the strength value of the i-th individual. The quality of each solution depends not only on its own quality but also on how many other solutions it dominates (i.e., how many other solutions it is better than). The more others it dominates, the higher its strength value, indicating a better solution. This indicates that the i-th solution dominates the j-th solution, meaning that solution i is no worse than solution j in all objectives, and has at least one objective that is strictly better than solution j.
[0085] By combining strength dominance and density truncation, the non-dominated solution set is guaranteed to be both convergent and uniformly distributed.
[0086] Step 6: Select individuals based on fitness for crossover and mutation operations to generate new offspring individuals to update and optimize the external elite profile; the specific steps are as follows: Step 6-1, Execute constraint priority selection: Prioritize the selection of feasible individuals to participate in reproduction; when there are not enough feasible individuals, supplement them in order of increasing violation value, thereby stabilizing the evolutionary process and improving the effective evaluation ratio.
[0087] Specifically, a feasible individual is an individual that satisfies all constraints, and is determined according to the constraint feasibility determination rules.
[0088] Step 6-2: Use the selected individual as the parent individual to perform the crossover operation.
[0089] Specifically, single-point or double-point segment swaps are performed on two parent individual sequences to generate offspring individuals.
[0090] Step 6-3: Perform mutation operations on the offspring individuals; the number of mutations can be determined according to a preset probability or Poisson distribution sampling; mutation types include node replacement, node addition, or node deletion; Specifically, the number of mutations is set to follow a Poisson distribution, and each mutation performs one of the following: "randomly replace nodes, randomly add nodes, or randomly delete nodes" to achieve a balance between strong perturbation and local refinement in the discrete space.
[0091] Step 6-4: Perform the comprehensive repair operator on the offspring individuals; Specifically, the individual representations are first validated by removing duplicate nodes, eliminating invalid indexes, and normalizing variable-length integer encoding. Then, constraint violation degrees are calculated based on a predefined set of constraints (budget, time, reliability / success rate, risk, etc.). The node set is adjusted according to the principle of "prioritizing the elimination of nodes with the highest violation degrees or contributions, and replacing or supplementing them with nodes of better attributes within the feasible region." This ensures that individuals satisfy the constraint thresholds while maintaining their original multi-objective performance advantages and solution set diversity as much as possible. Through this comprehensive repair operator, the proportion of effective feasible individuals in the evolutionary process is significantly increased, thereby improving the convergence speed and stability of multi-objective search and reducing sensitivity to the selection of penalty function parameters.
[0092] Step 6-5: Store all offspring individuals in the external elite archive and recalculate the strength value, density estimate, and fitness of each individual in the external elite archive. When updating the external elite database, diversity control is required. This means that before adding newly generated candidate solutions to the population for updating, it's necessary to avoid excessive similarity and concentration in the same category of solutions. This prevents premature convergence of the algorithm and ensures a more comprehensive coverage of the output multi-objective trade-off solution set. Specifically, after the algorithm completes feasibility assessment and non-dominated sorting, when the external elite database capacity exceeds the limit, density truncation is performed to maintain the distribution of the Pareto front solution set. Density truncation involves identifying and deleting the most dense (most crowded) individuals based on their density estimates within the external elite database, then updating the density information of the remaining individuals; this deletion operation is repeated until the database size reaches the target.
[0093] Repeat steps 6-1 to 6-5 until the maximum number of iterations is reached. Or it may satisfy the convergence condition.
[0094] Step 7: Obtain and output the non-dominated solution set from the external elite archives. The non-dominated solution set is also known as the Pareto front solution set. One solution corresponds to one solution.
[0095] Specifically, for each solution in the non-dominated solution set, the system outputs the solution set, multi-objective values, and constraint satisfaction information for each node to support communication network engineering decision-making under different preferences. Constraint satisfaction information can be attributes that satisfy constraints or violation values.
[0096] Without departing from the core technical ideas of this invention, although those skilled in the art can make equivalent substitutions for specific implementations (e.g., replacing PPO with other policy gradient or value function methods, replacing GNN structures with GCN / GAT / GraphSAGE, etc., replacing SPEA2 with NSGA-II / MOEA / D, etc.), these substitutions are generally only equivalent variations at the implementation level and must still simultaneously satisfy the key technical combination relationship established by this invention: using graph representation learning-supported reinforcement learning policy priors to generate high-quality and diverse feasible seed solutions for communication network device nodes, and using these seed solutions to guide multi-objective evolutionary search with constraint processing mechanisms, thereby stably obtaining a Pareto non-dominated solution set that satisfies multiple constraints. If any of these key elements are missing (e.g., using only a single reinforcement learning without providing a Pareto solution set, or using only an evolutionary algorithm but lacking policy priors resulting in a low proportion of feasible solutions and slow convergence, or lacking constraint priority and repair mechanisms leading to evolutionary instability under multi-constraint coupling), it is difficult to simultaneously achieve the comprehensive effects of this invention in terms of feasible solution generation efficiency, convergence speed, solution set diversity, and engineering feasibility. Therefore, the above-mentioned substitutions can only be carried out within the core closed-loop framework of this invention, and it is difficult to bypass this invention to obtain the same technical effect.
[0097] Based on the disclosure and teachings of the foregoing specification, those skilled in the art can make changes and modifications to the above embodiments. Therefore, the present invention is not limited to the specific embodiments disclosed and described above, and some modifications and changes to the present invention should also fall within the protection scope of the claims of the present invention. Furthermore, although some specific terms are used in this specification, these terms are only for convenience of explanation and do not constitute any limitation on the present invention.
Claims
1. A multi-objective optimization method for decision-making using a variable-length set for selecting key nodes in a communication network, characterized in that, Includes the following steps: Step 1: Obtain the data of the original communication network and abstract it into a graph structure; for each node in the original communication network, construct an initial feature vector, define node attributes, and set a node state mask; Set a multi-constraint threshold for the node attributes; initialize the evolutionary algorithm parameters and an empty solution set, wherein the node state mask indicates whether the corresponding node is selected into the solution set; the multi-constraint threshold is used to constrain the solution set; Step 2: Construct a multi-objective evaluation function based on at least two evaluation metrics for the solution set; Define constraint feasibility determination rules, and construct a constraint violation function based on the node attributes and the multi-constraint threshold to measure the degree of constraint violation by the solution set; Step 3: Construct a policy model based on RL-GNN according to the graph structure; the policy model includes states, actions, and state transitions; The state includes the graph structure, the embedded representation of the nodes, and the cumulative constraint state of the solution set; the action is to select a node or terminate the action; the state transition is to update the solution set and the cumulative constraint state according to the selected node; the cumulative constraint state is calculated based on the node attributes of all nodes in the solution set. The policy model is trained using the policy gradient method to obtain the final policy; Step 4: Based on the current state, select a node as a candidate node from all the nodes according to the final strategy; obtain several candidate nodes and add them to the empty solution set, and update the cumulative constraint state and the node state mask; A solution set is a candidate solution of variable length; Multiple candidate solutions are generated and constructed as the initial population; Step 5: Establish an external elite profile; using the multi-objective evaluation function and the constraint violation function, calculate the multi-objective value and violation value of each individual in the initial population, and add non-dominated individuals to the external elite profile; use constraint-aware SPEA2 to calculate the fitness of the individuals; Step 6: Select individuals based on their fitness and perform crossover and mutation operations to generate new offspring individuals to update and optimize the external elite profile; Step 7: Obtain and output the non-dominated solution set from the external elite files.
2. The variable-length set decision-making multi-objective optimization method for selecting key nodes in a communication network according to claim 1, characterized in that, The node attributes include device deployment costs. Window maintenance time and fault recovery success rate ; The multiple constraint thresholds include a cost threshold. Time threshold Success rate threshold ; The constraint feasibility determination rule is expressed as follows: Cost constraints: ; Time constraints: ; Success rate constraint: ; Denotes the solution set, Indicates the first one 1 node , , They are respectively The node attributes.
3. The variable-length set decision-making multi-objective optimization method for selecting key nodes in a communication network according to claim 1, characterized in that, The evaluation metrics include maximizing residual throughput and minimizing average end-to-end latency.
4. The variable-length set decision-making multi-objective optimization method for selecting key nodes in a communication network according to claim 1, characterized in that, Based on the graph structure and the feature vectors of the nodes, multiple rounds of message passing are performed to obtain the embedding matrix of the nodes; wherein, the nodes The The embedding of layers is represented as ,Right now: ； in, It is a non-linear activation function; ; For nodes The set of neighboring nodes; Let the edge feature weight function be... Show node and nodes The edge properties of the edges between them; and Both are trainable weight matrices, used for feature transformation of the current node and feature transformation of neighboring nodes, respectively.
5. A multi-objective optimization method for variable-length set decision-making in the selection of key nodes in a communication network according to claim 2, characterized in that, The process of generating / updating the cumulative constraint state is as follows: 。 6. A multi-objective optimization method for variable-length set decision-making in the selection of key nodes in a communication network according to claim 2, characterized in that, The constraint violation function is expressed as: ,Right now: 。 7. A multi-objective optimization method for variable-length set decision-making for selecting key nodes in a communication network according to claim 1, characterized in that, The specific process of step 4 is as follows: Step 4-1: For an empty solution set, based on the current state... The action probability output by the final strategy is used to select a node, which is denoted as a candidate node. Step 4-2: Update the solution set using the candidate nodes, and simultaneously update the cumulative constraint state and the node state mask of the candidate nodes; Step 4-3: Repeat steps 4-1 and 4-2 until the update termination condition of a solution set is met, and a candidate solution of variable length is obtained.
8. A multi-objective optimization method for variable-length set decision-making for selecting key nodes in a communication network according to claim 1, characterized in that, The specific process of step 5 is as follows: Step 5-1: Initialize the external elite profile and constraint-aware SPEA2; Step 5-2: Use variable-length integer encoding to represent the ID sequence or ID set of an individual; use the multi-objective evaluation function and the constraint violation function to calculate the multi-objective value and violation value of each individual in the initial population, respectively. Step 5-3: Add non-dominated individuals to the external elite profile; use constraint-aware SPEA2 to calculate the fitness of the individuals based on density estimation and individual strength.
9. A multi-objective optimization method for variable-length set decision-making in the selection of key nodes in a communication network according to claim 8, characterized in that, The specific process of step 6 is as follows: Step 6-1, Execute constraint priority selection: Prioritize the selection of feasible individuals for reproduction; when there are insufficient feasible individuals, supplement them according to the violation degree value from small to large; The feasible individual is the individual that satisfies all constraints, and is determined according to the constraint feasibility determination rules. Step 6-2: Perform crossover operation using the selected individual as the parent individual; Step 6-3: Perform mutation operations on the offspring individuals; Step 6-4: Perform the comprehensive repair operator on the offspring individuals; Step 6-5: Store all offspring individuals in the external elite archive and recalculate the strength value, density estimate, and fitness of each individual in the external elite archive. When the external elite archive capacity exceeds the limit, density truncation is performed to maintain the distribution of the Pareto front solution set; Repeat steps 6-1 to 6-5 until the maximum number of iterations is reached or the preset convergence condition is met.
10. A multi-objective optimization method for variable-length set decision-making in the selection of key nodes in a communication network according to claim 1, characterized in that, For each solution in the non-dominated solution set, output the solution set, multi-objective values, and constraint satisfaction information for the nodes.