Traffic key infrastructure deployment method considering cascade failure and reinforcement learning

Abstract

The invention provides a traffic key infrastructure deployment method considering cascade failure and reinforcement learning. The method comprises the following steps: S1, constructing a road network structure and obtaining node features; s2, establishing a cascade failure model according to the road network structure and the node characteristics; and S3, based on reinforcement learning, identifying key infrastructures in the cascade failure model. The problems that an existing cascade failure model lacks deep analysis on local nodes and edges and is poor in adaptability are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of complex transportation networks, and in particular relates to a method for deploying key transportation infrastructure taking into account cascading failures and reinforcement learning. Background Art

[0002] Modern society is increasingly dependent on the transportation system. As a key infrastructure for urban development, the transportation system has a significant impact on the economy, society and the environment. However, with the acceleration of urbanization and the continuous growth of transportation demand, the transportation system faces many challenges. Factors such as traffic congestion, frequent accidents, natural disasters and man-made damage may lead to partial or complete paralysis of the transportation network. These problems not only affect people's daily lives, but also cause serious losses to the city's economic activities. Therefore, how to improve the robustness and reliability of the transportation system has become an important issue that needs to be solved urgently.

[0003] Complex transportation network theory provides an effective tool for studying the structure and dynamic behavior of transportation systems. Transportation systems can be viewed as complex networks consisting of nodes (such as intersections and stations) and edges (such as roads and tracks). By analyzing the topological structure of transportation networks and the importance of nodes, key nodes and vulnerable points in the system can be revealed, which is of great significance for optimizing the design of transportation systems and improving their risk resistance.

[0004] In view of the deployment of key infrastructure in the transportation system, complex network analysis is used to identify key nodes that play a vital role in traffic flow and network connectivity. Once these nodes fail, they may trigger a cascade effect, leading to large-scale traffic paralysis. Based on the cascading failure model, it is possible to simulate and evaluate the impact of different node failures on the entire transportation network, so as to design a transportation system with better performance. However, when applying the existing cascading failure model, it pays more attention to the global network topology structure and lacks in-depth analysis of local node and edge characteristics; and when faced with emergencies and unexpected situations, it lacks the ability to quickly adjust and adapt, and cannot optimize the response strategy in a timely manner.

[0005] Therefore, there is an urgent need for an effective method for deploying critical transportation infrastructure so that the transportation system can maintain high performance and stability in the face of different types of failures. Summary of the invention

[0006] The purpose of the embodiments of the present invention is to provide a method for deploying critical transportation infrastructure considering cascading failures and reinforcement learning, so as to solve the problem that the existing cascading failure model lacks in-depth analysis of local nodes and edges and has poor adaptability.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is a method for deploying critical traffic infrastructure considering cascading failure and reinforcement learning, comprising the following steps:

[0008] S1: Construct road network structure and obtain node features;

[0009] S2: Establish a cascading failure model based on the road network structure and node characteristics;

[0010] S3: Identifying critical infrastructure in cascading failure models based on reinforcement learning.

[0011] Furthermore, the S1 includes:

[0012] S11: Select a road network, where nodes represent intersections in the road network and edges represent road sections connecting intersections; and construct a training graph;

[0013] S12: Calculate the characteristic information of all nodes, namely, the node degree, closeness centrality, betweenness centrality, eigenvector centrality, and actual traffic flow data passing through each node.

[0014] Further, the S2 includes:

[0015] In the cascading failure model,

[0016] S21: Set the initial state of all nodes;

[0017] S22: Determine the initial load of all nodes; the calculation formula of the initial load of the node is as follows:

[0018]

[0019] Among them, L i is the initial load of node i; φ, is the adjustment parameter; i represents the set of neighbor nodes of node i; m represents the neighbor nodes of node i, k i represents the degree of node i, k m Represents the degree of neighbor nodes;

[0020] S23: Determine the load capacity of all nodes; the calculation formula of the load capacity of the node is as follows:

[0021] C i =(1+β)L i (2)

[0022] Among them, C i represents the load capacity of node i, i.e., the maximum carrying capacity; β is the tolerance coefficient, which represents the additional load that node i can bear, β≥0;

[0023] S24: Determine the load redistribution strategy; after a node fails, the traffic of the failed node is proportionally distributed according to the load level of the neighboring nodes of the failed node. The calculation formula is as follows:

[0024]

[0025] ΔC j =C j -L j (4)

[0026] ΔC m =C m -L m (5)

[0027] Among them, the receiving node j represents a node in the neighbor node set of the failed node i, L' j is the load of receiving node j after load redistribution; L j is the current load of receiving node j, that is, the load before load redistribution; L i represents the load of failed node i; ΔC j Indicates the load level that node j can currently bear, which is the node load capacity C j and the current load L of receiving node j j The difference between the two, m represents any node in the neighbor node set of the failed node i, ΔC m Indicates the load level that node i’s neighbor node m can currently bear, which is the node load capacity C m and the current load L of node m m The difference.

[0028] Further, the S3 includes:

[0029] S31: Construct a reinforcement learning model, initialize and define the reinforcement learning model based on the cascading failure model, and determine the objective function;

[0030] S32: In the training graph, a graph weighting operation is performed to assign a weight to each node;

[0031] S33: In the training graph, a graph embedding operation is performed to encode the state, action, node position, failure probability, node structure information, and traffic flow information into an embedding vector;

[0032] S34: Use the reinforcement learning model built in S31 to perform reinforcement learning, that is, calculate the Q value after removing the failed nodes in a given state in the iterative process to obtain the number and sequence of critical infrastructure.

[0033] Further, the S31 includes:

[0034] In the reinforcement learning model, determine the environment, state S, action A, reward, and objective function;

[0035] Environment: training graph in S1;

[0036] State S: the residual network after the current cascade failure;

[0037] Action A: In state S, remove important nodes;

[0038] Reward: The reward is determined by the change in average network efficiency before and after the failure;

[0039] For the training graph in S1, the ratio of the number of failed nodes to the total number of nodes is θ, 0<θ<1, and R represents the set of failed nodes R={v1,v2,...,v θ*N};

[0040] Among them, v θ*N represents the last node in the failed node set, θ*N represents the number of failed nodes,

[0041] The objective function calculation formula is as follows:

[0042]

[0043] Among them, maxΔE represents the maximum degree of decline in road network efficiency after all nodes in the node set R fail, ΔE represents the degree of decline in road network efficiency after the node sequence fails; E0 represents the full network traffic efficiency of the initial network structure; E V\R represents the traffic efficiency of the remaining network structure after all nodes in the node set R fail; p and q represent any two different nodes in the road network, p*q represents the node connection between p and q in the road network, o pq represents the length of the shortest path between nodes p and q, represents the inverse of the shortest path length between nodes p and q, that is, the traffic efficiency between nodes p and q, N0 represents the total number of nodes in the road network before the node fails, N0(N0-1) represents the number of node pairs before the node fails, N V\R N represents the number of nodes in the remaining network structure after all nodes in the node set R fail; V\R (N V\R -1) indicates the number of remaining node pairs after the node fails.

[0044] Further, the S32 includes:

[0045] The initial degree, closeness centrality, betweenness centrality, and eigenvector centrality of each node are converted into weight values ​​through weighted summation and assigned to each node as the node weight.

[0046] Further, the S33 includes:

[0047] S331: Encode the state S, i.e. the current failed network, into an adjacency matrix z s , z s Each element in represents the connection status between node i and node j; action A, i.e., removing the identified node, is encoded as vector z a , z a Each dimension corresponds to a node in the network. When a node is removed, the value of the corresponding dimension is set to 1, and the rest are set to 0. The position of each node is determined by the failure network obtained after each iteration, and the position of each node is encoded. The failure probability of each node is obtained through the historical failure data and current load conditions of each node, and is encoded.

[0048] S332: Encode the structural information and traffic flow information of each node; the structural information includes the degree, closeness centrality, betweenness centrality, and eigenvector centrality of the node, and the traffic flow information refers to the actual traffic flow data passing through this node;

[0049] S333: The network efficiency contribution of the node Q G (i) Encoding; Q G The calculation formula of (i) is as follows:

[0050]

[0051] Q G (i) = 1 - E' G (i) / E G (8)

[0052] In formula (7), E G represents the average efficiency of the road network; N is the number of nodes in the road network, i and j are any two different nodes in the road network, and d ij is the shortest path length from node i to node j; U represents the node set of the road network; Q G (i) represents the network efficiency contribution of node i, E' G (i) is the average network efficiency after node i fails.

[0053] Further, the S34 includes:

[0054] S341: Initialize the Q value, and preset the number of iterations, learning rate α, discount factor γ, and exploration rate ε;

[0055] S342: According to the calculated degree, closeness centrality, betweenness centrality, eigenvector centrality, and traffic flow data of each node in S12, the nodes are sorted according to the numerical value under each feature information; and important nodes are screened out from each feature information according to the proportion;

[0056] The state at time t is s t , update action a t The embedding vector of t It means that the selected important nodes will be removed at time t, and the number of failed nodes and node labels will be recorded;

[0057] The state s in the training graph t Update to new status t+1 And update its embedding vector, and update the node position, failure probability, structural information, traffic flow information, and the embedding vector of the node's network efficiency contribution of each node.

[0058] Furthermore, the S34 further includes:

[0059] S343: Get instant rewards. The calculation formula of instant rewards is as follows:

[0060] r t (s t ,a t )=E G -E G (i),i∈{v1,v2,…,v n} (9)

[0061] Among them, r t (s t ,a t ) means taking action a t After that, you will get instant rewards, E G represents the average network efficiency before node i fails, E G (i) represents the average network efficiency after node i fails, i represents the failed node, {v1,v2,…,v n} represents the invalid node set;

[0062] S344: Calculate the current Q value and update the Q function; in each iteration, according to the current reward r t (s t ,a t ) and the new state s t+ 1 to update the current Q value. The Q function calculation formula is as follows:

[0063]

[0064] Among them, W4 and W5 are weight parameters, za 、z s Represent the embedding vectors of state and action respectively, z a ,z s ∈R 1*P , Q(s t ,a t ) means in state s t Take action a t The current Q value of represents the transpose of W5, R 1*P Indicates that the embedding vector is a vector with 1 row and p columns. Represents the state embedding vector z a The transpose of

[0065] The update formula of the Q function is as follows:

[0066] Q(s t ,a t )←Q(s t ,a t )+α[r t +γmaxQ(s t+1 ,a')-Q(s t ,a t )] (11)

[0067] Among them, s t is the current state; a t is in state s t The action taken next; t Is to take action a t The immediate reward after t+1 is the new state obtained after taking action a; a' is the new state s t+1 All possible actions under; α is the learning rate; maxQ(s t+1 ,a′) is the next state s t+1 is the maximum Q-value of all possible actions, and γ is the discount factor.

[0068] Furthermore, the S34 further includes:

[0069] S345: Use greedy selection of the next action ε, that is, there is a probability of ε to randomly select a new action, and there is a probability of 1-ε to select the action with the largest Q value in the current state;

[0070] S346: Repeat S342-S345 until the preset number of iterations is reached; during the iteration process, W4 and W5 are updated to obtain the Q function with the best effect under the preset number of iterations, that is, the Q function when the degree of network efficiency reduction is maximized; based on the Q function when the degree of network efficiency reduction is maximized, the number and sequence of important nodes, that is, key infrastructure, are obtained.

[0071] The beneficial effects of the present invention are:

[0072] 1. Using reinforcement learning algorithms, infrastructure deployment strategies can be optimized in dynamic and uncertain environments. Through continuous interaction and learning, strategies can be gradually improved so that the transportation system can maintain high performance and stability in the face of different types of failures.

[0073] 2. The present invention utilizes a method combining reinforcement learning and cascading failure model to effectively identify and reinforce key nodes in the transportation network. It not only considers the direct impact of a single node failure on the network, but also considers the chain reaction caused by cascading failures, thereby improving the accuracy of node importance assessment and enhancing the robustness and risk resistance of the overall system.

[0074] 3. The present invention can extract the node’s degree, closeness centrality, betweenness centrality, and eigenvector centrality in each state based on real-time traffic data and events, and encode these features into embedded vectors as state inputs. As the environment changes, the feature input and state representation will also be dynamically adjusted, so the node importance assessment and infrastructure deployment strategy can be dynamically adjusted to ensure that the model can reflect the current network status in real time, improve the real-time and accuracy of decision-making, and have high adaptability and response speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0076] Figure 1 It is the real road network map Sioux Falls.

[0077] Figure 2 It is the real road network map of Anaheim.

[0078] Figure 3 It is a bar graph of the degree of decrease in network efficiency of the method of the present invention in the SiouxFalls network and the Anaheim network as the number of iterations increases, wherein a is a bar graph of the degree of decrease in network efficiency of the method of the present invention in the SiouxFalls network as the number of iterations increases, and b is a bar graph of the degree of decrease in network efficiency of the method of the present invention in the Anaheim network as the number of iterations increases.

[0079] Figure 41 is a graph showing the degree of decrease in network efficiency of the method of the present invention and the traditional node attack method in the SiouxFalls network and the Anaheim network as the number of failed nodes increases, wherein a is a graph showing the degree of decrease in network efficiency of the method of the present invention and the traditional node attack method in the SiouxFalls network as the number of failed nodes increases, and b is a graph showing the degree of decrease in network efficiency of the method of the present invention and the traditional node attack method in the Anaheim network as the number of failed nodes increases. DETAILED DESCRIPTION

[0080] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0081] This embodiment discloses a method for deploying critical traffic infrastructure considering cascading failures and reinforcement learning. The complete steps of this method are as follows:

[0082] S1: Construct the road network structure. The specific steps are as follows:

[0083] S11: See Figure 1 ,Select the real road network SiouxFalls network, where the nodes in the road network represent intersections, and the edges represent the road sections connecting the intersections, and construct a training graph.

[0084] S12: Obtain node feature information of the road network. The method of the present invention comprehensively considers the global and local attributes of the nodes, and calculates the degree, closeness centrality, betweenness centrality, and eigenvector centrality of all nodes through Gephi visualization software; the degree is the number of neighbors of the node, the closeness centrality is the reciprocal of the average shortest distance from the node to all other nodes in the network, the betweenness centrality is the frequency of the node as the intermediary of the shortest path between other nodes in the network, and the eigenvector centrality is the importance of the neighbors of the node. Four centrality indicators are used to mine node feature information. And obtain the actual traffic flow data passing through each node.

[0085] S2, based on the network structure and node characteristics, establish a cascading failure model that reflects the impact of node failure on the entire network. The specific steps are as follows:

[0086] S21: Set the initial state of the node.

[0087] S22: Set the initial load of the node. Taking into account the degree values ​​of the node and its neighboring nodes, the initial load model of the node is shown in formula (1):

[0088]

[0089] Among them, L i is the initial load of node i; φ, is the adjustment parameter; i represents the set of neighbor nodes of node i; m represents the neighbor nodes of node i, k i represents the degree of node i, k m Represents the degree of neighbor nodes.

[0090] S23: Set the node capacity. The capacity of a node is related to its initial load. The capacity model of a node is shown in formula (2):

[0091] C i =(1+β)L i (2)

[0092] Among them, C i represents the load capacity of node i, that is, the maximum carrying capacity; β is the tolerance coefficient, which represents the additional load that node i can bear, β≥0.

[0093] S24: Set the load redistribution strategy. Use the proportional redistribution strategy to distribute the traffic of the failed node in proportion to the load level of the neighboring nodes of the failed node. The calculation formula is as follows:

[0094]

[0095] ΔC j =C j -L j (4)

[0096] ΔC m =C m -L m (5)

[0097] Among them, the receiving node j represents any node in the neighbor node set of the failed node i, L' j is the load of receiving node j after load redistribution; L j is the load of receiving node j before load redistribution; L i represents the load of failed node i; ΔC j Indicates the current load level of node j, which is the node load capacity C j and the node's current load L j The difference, ΔC m Indicates the current load level of node i’s neighbor node m, which is the node load capacity C m and the node's current load L m The difference.

[0098] S3: Identify the deployment locations of critical infrastructure based on reinforcement learning.

[0099] Based on the impact analysis of cascading failure factors in complex networks, the present invention applies a reinforcement learning model to conduct multiple trainings on the topological structure and cascading failure sequence of complex networks, with the goal of maximizing the reduction of network traffic capacity, and further identifies the optimal set of attack nodes and attack sequence, providing a theoretical basis for the deployment of critical infrastructure.

[0100] S31: The present invention constructs a reinforcement learning model based on the research environment of complex transportation networks and initializes and defines the reinforcement learning model. The specific steps are described as follows:

[0101] Environment: Road network structure in S1;

[0102] State (S): residual network after experiencing cascading failure;

[0103] Action (A): In state (S), remove the node that causes network failure;

[0104] Reward: The reward size is quantified by the change in network efficiency before and after the failure.

[0105] Objective function: For the urban road network structure in S1, the proportion of failed nodes is θ, which is the number of failed nodes / total number of nodes, 0<θ<1, and R represents the set of failed nodes R={v1,v2,...,v θ*N}. θ*N represents the last node in the failed node set, θ*N represents the number of failed nodes, and this step aims to set a node removal sequence (v1, v2, ..., v k ) to maximize the impact of reduced capacity across the road network.

[0106] The objective function is shown in the following formula (6):

[0107]

[0108] Among them, maxΔE represents the maximum degree of decline in road network efficiency after all nodes in the node set R fail, ΔE represents the degree of decline in road network efficiency after the node sequence fails; E0 represents the full network traffic efficiency of the initial network structure; E V\R It represents the traffic efficiency of the remaining network structure after all nodes in the specified node set R fail; p*q represents the node connection combination between any two nodes p and q in the network, p and q represent any two different nodes in the network, o pq represents the length of the shortest path between nodes p and q, represents the inverse of the shortest path length between nodes p and q, that is, the travel efficiency between nodes p and q, N0 represents the total number of nodes in the network before the node fails, N0(N0-1) represents the number of node pairs, N V\R N represents the number of nodes in the remaining network structure after all nodes in the node set R fail; V\R (N V\R -1) indicates the number of remaining node pairs.

[0109] S32: In the training graph, a graph weighting operation is performed to assign a weight to each road network node. In order to better integrate the characteristics of the complex road network structure information and functional information, the training graph needs to be preprocessed. The degree, closeness centrality, betweenness centrality, and eigenvector centrality of each road network node are converted into a weight value through weighted summation and assigned to each node as the node weight.

[0110] S33: In the training graph, use graph embedding technology to perform graph embedding operations to encode the characteristics of state, action, node location and failure probability, node structure information and traffic flow information. The specific steps are as follows:

[0111] S331: Encode the state, action, node position and failure probability. Encode the state (S), i.e. the failure network, into an adjacency matrix z s , z s Each element in represents the connection state between node i and node j, 0 element represents disconnection, and non-zero value element represents connection strength, which is used to record and process the state of the environment during reinforcement learning, such as the overall topological structure of the remaining network, the number of remaining nodes, etc.; the action (A) is encoded as z a , that is, remove the identified node and represent the action as a vector, where each dimension corresponds to a node in the network. When a node is removed, the value of the corresponding dimension can be set to 1, and the rest to 0. The action vector is then mapped to a lower-dimensional embedding vector z through a simple linear layer or a neural network. a ; Using these embedding vectors, we can calculate Q(s,a), the maximum expected reward for performing an action (A) in a given state (S).

[0112] The position of each node on the training graph can be determined by the adjacency matrix of the failed network obtained after each iterative learning, and the connection relationship between the surviving nodes. Through these matrices, the relative position of each node in the network and the strength of its connection with other nodes can be obtained; the position of each node is encoded as an embedded vector;

[0113] The failure probability of a node is predicted by the node's historical failure data and current load conditions, and is encoded as a node attribute. The state and action encoding provides the necessary environment and decision information for the reinforcement learning algorithm, while the node position and failure probability encoding provides the algorithm with specific node topology and vulnerability data. The two work together to help optimize the deployment of critical transportation infrastructure.

[0114] S332: Encode the structural information and traffic flow information of the node to obtain an embedded vector. The structural information includes the degree, closeness centrality, betweenness centrality, and eigenvector centrality of the node, and the traffic flow information refers to the actual traffic flow data passing through this node.

[0115] S333: The network efficiency contribution of the node Q G (i) Encoded into an embedding vector.

[0116] The network average efficiency is used to measure the convenience of connection between node pairs. The network average efficiency reflects the function of network connectivity and connectivity performance. The network average efficiency is shown in the following formula (7):

[0117]

[0118] Among them, E G represents the average efficiency of the road network; N is the number of nodes in the road network, i and j are different nodes in the road network, and d ij is the shortest path length from node i to node j; U represents the node set of the road network.

[0119] The node contribution is calculated by subtracting the network efficiency after the failure of node i from the network efficiency before the failure of node i. It is used to indicate the change in network efficiency before and after the failure of the node, so as to measure the importance of the node to the network. If the difference in network efficiency before and after the failure is large and the network efficiency drops significantly after the failure, then the node has an important connectivity function and is therefore considered an important node in the network. In order to make the node contribution proportional to the degree of change in network efficiency, the network efficiency contribution of node i is calculated as follows:

[0120] Q G (i) = 1 - E' G (i) / E G (8)

[0121] Among them, Q G (i) represents the network efficiency contribution of node i, E G represents the network efficiency before node i fails, E' G (i) is the network efficiency after node i fails.

[0122] S34: In the reinforcement learning model, Q function learning is performed, that is, the Q value after removing the nodes that cause network failure in a given state is calculated in the iterative process. The specific steps are described as follows:

[0123] S341: Parameter initialization. Initialize the Q value, usually with zero or a random value, and set parameters such as the learning rate α, discount factor γ, and exploration rate ε.

[0124] S342: According to the node feature information in S33, such as node degree, closeness centrality, betweenness centrality, eigenvector centrality, traffic flow and other features, perform index sorting to identify important nodes under different evaluation indicators, that is, nodes that need to be removed. When sorting indicators, the present invention sorts nodes from large to small according to the importance of the nodes, because these indicators are all benefit-type indicators, that is, the larger the value of the indicator, the higher the importance of the node. Therefore, according to the calculated degree, closeness centrality, betweenness centrality, eigenvector centrality, and traffic flow data of each node in S12, these values ​​are sorted from large to small. Then, according to a certain ratio, select important nodes from each evaluation indicator. The nodes ranked in the top 10% of each indicator can be selected as important nodes.

[0125] The state at time t is s t , update action a t The embedding vector of t It means that at time t, the selected important nodes are removed, and the number of invalid nodes and node labels are recorded; the important nodes are removed, that is, a t =v i , where v i ∈V; a t represents the node removal operation performed at time t; v i represents the node removed at time t, where V represents the node set in the road network topology structure,

[0126] The state s in the training graph t Update to new status t+1 And update its embedding vector. And update the embedding vector of each node's node position, failure probability, structural information, traffic flow information, and node's network efficiency contribution.

[0127] S343: Obtain an immediate reward. The reward is the quantitative value of the change in the road network traffic efficiency caused by the removed node. The formula is as follows:

[0128] r t (s t ,a t )=E G -E G (i),i∈{v1,v2,…,v n} (9)

[0129] Among them, r t (s t ,a t ) means taking action a t After that, you will get instant rewards, E G represents the average network efficiency before node i fails, E G (i) represents the average network efficiency after node i fails, i represents the failed node, {v1,v2,…,v n} represents a set of invalid nodes.

[0130] S344: Calculate the current Q value and update the Q function. In each iteration, according to the current reward r t (s t ,a t ) and the new state s t+1 The maximum Q value under the condition is used to update the Q value of the current state-action pair (s, a). The specific steps are as follows:

[0131] The Q function calculation formula is as follows:

[0132]

[0133] Among them, W4 and W5 are parameters that need to be updated during the model training phase, a 、z s Represent the embedding vectors of state and action respectively, z a ,z s ∈R 1*P , Q(s t ,a t ) means in state s t Take action a t The current Q value, W5 T represents the transpose of W5, R 1*P Indicates that the embedding vector is a vector with 1 row and p columns. Represents the state embedding vector z a The transpose of .

[0134] The update formula of the Q function is as follows:

[0135] Q(s t ,a t )←Q(s t ,a t )+α[r t +γmaxQ(s t+1 ,a')-Q(s t ,a t )] (11)

[0136] Among them, st is the current state; a t is in state s t The action taken next; t Is to take action a t The immediate reward after t+1 is the new state obtained after taking action a; a' is the new state s t+1 All possible actions under the condition; α is the learning rate, which controls the response to new information, i.e., r, in each iteration. t 、s t+1 In this embodiment, it is experimentally verified that the best effect is achieved when α is 0.1; maxQ(s t+1 ,a′) is the next state s t+1 The maximum Q value of all possible actions under the condition of γ is the discount factor, which is used to balance the immediate reward and future reward. Experimental results show that the best effect is achieved when γ = 0.9 in this embodiment.

[0137] S345: Select the next action. Use ε-greedy selection. Specifically, there is a probability of ε to randomly select an action to explore new behaviors; there is a probability of 1-ε to select the action with the largest Q value in the current state.

[0138] S346: Repeat S342-S345 until the preset number of iterations is reached. In each iteration, the state, node features, and actions need to be embedded. By continuously updating the embedding vector, the model can gradually optimize the node importance assessment and infrastructure deployment strategy. If the network efficiency decreases more after a certain training than the previous training, this indicates that the model has better identified important nodes. As the training progresses, the model gradually learns to identify which nodes’ removal will lead to the greatest decrease in network efficiency, thereby more accurately assessing the importance of nodes. During the iteration process, W4 and W5 are updated to obtain the best Q function under these iteration numbers and predetermined parameters, that is, the Q function when the network efficiency decreases to the maximum. According to the Q function when the network efficiency decreases to the maximum, the number and sequence of important nodes are obtained.

[0139] This embodiment uses the SiouxFalls network and Figure 2 The Anaheim network is used to verify the feasibility and effectiveness of the proposed method. Figure 3 Figures a and b show the training simulation process of the present invention, and statistically analyze the changes in road network efficiency in the SiouxFalls network and the Anaheim network in 10 simulation trainings. It can be seen that the SiouxFalls network obtained the highest degree of network efficiency decline in the 6th training simulation, while the Anaheim network obtained the highest degree of efficiency decline in the 4th training simulation.

[0140] Figure 4Figures a and b show the comparison of the road network efficiency decline of the method of the present invention (LC-RL) and the traditional method, i.e., the node attack strategy based on the evaluation index degree, as the number of failed nodes increases during the training process. The traditional method is to calculate the degree of each node, select the node with the highest degree as the attack target, and remove these nodes in turn. This embodiment uses the node degree as the evaluation index to identify the important nodes. Figure 4 As shown in a and b, the method of the present invention distinguishes the network influence of each node in detail and avoids invalid attacks. Compared with the traditional node attack strategy, the method of the present invention achieves the maximum network efficiency reduction by attacking 70.8% and 72.8% of the nodes in the SiouxFalls network (a total of 24 nodes) and the Anaheim network (a total of 416 nodes), respectively. Figure 4 As can be seen from a, although the traditional method (degree) reaches the threshold faster, due to the existence of too many invalid attacks, the curve quickly becomes a horizontal line, and any attack will not cause the network efficiency to change, so the importance of each node cannot be distinguished; Figure 4 As can be seen in Figure b, the Anaheim network is larger in scale, LC-RL reaches the falling threshold faster, and the curve changes significantly, making it easier to distinguish the influence of each node on network efficiency.

[0141] It can be seen that the nodes identified by the method of the present invention are indeed in key positions in the road network. The method of the present invention can distinguish the importance of nodes, and the important node sequence derived therefrom also provides a reference for the deployment order of critical transportation infrastructure.

[0142] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0143] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A deployment method for critical transportation infrastructure considering cascading failures and reinforcement learning, It is characterized in that The following steps are involved: S1: Construct road network structure and obtain node features; S2: Establish a cascading failure model based on the road network structure and node characteristics; S3: Identifying critical infrastructure in cascading failure models based on reinforcement learning.

2. The method for deploying critical transportation infrastructure considering cascading failure and reinforcement learning according to claim 1, It is characterized in that The S1 includes: S11: Select a road network, where nodes represent intersections in the road network and edges represent road sections connecting intersections; and construct a training graph; S12: Calculate the characteristic information of all nodes, namely, the node degree, closeness centrality, betweenness centrality, eigenvector centrality, and actual traffic flow data passing through each node.

3. The method for deploying critical transportation infrastructure considering cascading failure and reinforcement learning according to claim 1, It is characterized in that The S2 includes: In the cascading failure model, S21: Set the initial state of all nodes; S22: Determine the initial load of all nodes; the calculation formula of the initial load of the node is as follows: Among them, L i is the initial load of node i; φ, is the adjustment parameter; i represents the set of neighbor nodes of node i; m represents the neighbor nodes of node i, k i represents the degree of node i, k m Represents the degree of neighbor nodes; S23: Determine the load capacity of all nodes; the calculation formula of the load capacity of the node is as follows: C i =(1+β)L i (2) Among them, C i represents the load capacity of node i, i.e., the maximum carrying capacity; β is the tolerance coefficient, which represents the additional load that node i can bear, β≥0; S24: Determine the load redistribution strategy; after a node fails, the traffic of the failed node is proportionally distributed according to the load level of the neighboring nodes of the failed node. The calculation formula is as follows: ΔC j =C j -L j (4) ΔC m =C m -L m (5) Among them, the receiving node j represents a node in the neighbor node set of the failed node i, L' j is the load of receiving node j after load redistribution; L j is the current load of receiving node j, that is, the load before load redistribution; L i represents the load of failed node i; ΔC j Indicates the load level that node j can currently bear, which is the node load capacity C j and the current load L of receiving node j j The difference between the two, m represents any node in the neighbor node set of the failed node i, ΔC m Indicates the load level that node i’s neighbor node m can currently bear, which is the node load capacity C m and the current load L of node m m The difference.

4. The method for deploying critical transportation infrastructure considering cascading failure and reinforcement learning according to claim 2, It is characterized in that The S3 includes: S31: Construct a reinforcement learning model, initialize and define the reinforcement learning model based on the cascading failure model, and determine the objective function; S32: In the training graph, a graph weighting operation is performed to assign a weight to each node; S33: In the training graph, a graph embedding operation is performed to encode the state, action, node position, failure probability, node structure information, and traffic flow information into an embedding vector; S34: Use the reinforcement learning model built in S31 to perform reinforcement learning, that is, calculate the Q value after removing the failed nodes in a given state in the iterative process to obtain the number and sequence of critical infrastructure.

5. The method for deploying critical transportation infrastructure considering cascading failure and reinforcement learning according to claim 2 or 4, It is characterized in that The S31 includes: In the reinforcement learning model, determine the environment, state S, action A, reward, and objective function; Environment: training graph in S1; State S: the residual network after the current cascade failure; Action A: In state S, remove important nodes; Reward: The reward is determined by the change in average network efficiency before and after the failure; For the training graph in S1, the ratio of the number of failed nodes to the total number of nodes is θ, 0<θ<1, and R represents the set of failed nodes R={v 1 ,v 2 ,...,v θ*N }; Among them, v θ*N represents the last node in the failed node set, θ*N represents the number of failed nodes, The objective function calculation formula is as follows: Among them, maxΔE represents the maximum degree of decline in road network efficiency after all nodes in the node set R fail, ΔE represents the degree of decline in road network efficiency after the node sequence fails; E 0 represents the full network efficiency of the initial network structure; E V\R represents the traffic efficiency of the remaining network structure after all nodes in the node set R fail; p and q represent any two different nodes in the road network, p*q represents the node connection between p and q in the road network, o pq represents the length of the shortest path between nodes p and q, represents the inverse of the shortest path length between nodes p and q, that is, the travel efficiency between nodes p and q, N 0 represents the total number of nodes in the road network before the node fails, N 0 (N 0 -1) represents the number of node pairs before the node fails, N V\R N represents the number of nodes in the remaining network structure after all nodes in the node set R fail; V\R (N V\R -1) indicates the number of remaining node pairs after the node fails.

6. The method for deploying critical transportation infrastructure considering cascading failure and reinforcement learning according to claim 5, It is characterized in that The S32 includes: The initial degree, closeness centrality, betweenness centrality, and eigenvector centrality of each node are converted into weight values ​​through weighted summation and assigned to each node as the node weight.

7. The method for deploying critical transportation infrastructure considering cascading failure and reinforcement learning according to claim 5, It is characterized in that The S33 includes: S331: Encode the state S, i.e. the current failed network, into an adjacency matrix z s , z s Each element in represents the connection state between node i and node j; action A, i.e., removing the identified node, is encoded as vector z a , z a Each dimension corresponds to a node in the network. When a node is removed, the value of the corresponding dimension is set to 1, and the rest are set to 0. The position of each node is determined by the failure network obtained after each iteration, and the position of each node is encoded. The failure probability of each node is obtained through the historical failure data and current load conditions of each node, and is encoded. S332: Encode the structural information and traffic flow information of each node; the structural information includes the degree, closeness centrality, betweenness centrality, and eigenvector centrality of the node, and the traffic flow information refers to the actual traffic flow data passing through this node; S333: The network efficiency contribution of the node Q G (i) Encoding; Q G The calculation formula of (i) is as follows: Q G (i)=1-E' G (i) / E G (8) In formula (7), E G represents the average efficiency of the road network; N is the number of nodes in the road network, i and j are any two different nodes in the road network, and d ij is the shortest path length from node i to node j; U represents the node set of the road network; Q G (i) represents the network efficiency contribution of node i, E' G (i) is the average network efficiency after node i fails.

8. The method for deploying critical transportation infrastructure considering cascading failure and reinforcement learning according to claim 7, It is characterized in that The S34 includes: S341: Initialize the Q value, and preset the number of iterations, learning rate α, discount factor γ, and exploration rate ε; S342: According to the calculated degree, closeness centrality, betweenness centrality, eigenvector centrality, and traffic flow data of each node in S12, the nodes are sorted according to the numerical value under each feature information; and important nodes are screened out from each feature information according to the proportion; The state at time t is s t , update action a t The embedding vector of t It means that the selected important nodes will be removed at time t, and the number of failed nodes and node labels will be recorded; The state s in the training graph t Update to new status t+1 And update its embedding vector, and update the node position, failure probability, structural information, traffic flow information, and the embedding vector of the node's network efficiency contribution of each node.

9. The method for deploying critical transportation infrastructure considering cascading failure and reinforcement learning according to claim 8, It is characterized in that The S34 further includes: S343: Get instant rewards. The calculation formula of instant rewards is as follows: r t (s t ,a t )=E G -E G (i),i∈{v 1 ,v 2 ,…,v n } (9) Among them, r t (s t ,a t ) means taking action a t After that, you will get instant rewards, E G represents the average network efficiency before node i fails, E G (i) represents the average network efficiency after node i fails, i represents the failed node, {v 1 ,v 2 ,…,v n } represents the invalid node set; S344: Calculate the current Q value and update the Q function; in each iteration, according to the current reward r t (s t ,a t ) and the new state s t+1 The maximum Q value under is used to update the current Q value. The Q function calculation formula is as follows: Among them, W 4 , W 5 is the weight parameter, z a 、z s Represent the embedding vectors of state and action respectively, z a ,z s ∈R 1*P , Q(s t ,a t ) means in state s t Take action a t The current Q value of W 5 The transpose of R 1*P Indicates that the embedding vector is a vector with 1 row and p columns. Represents the state embedding vector z a The transpose of The update formula of the Q function is as follows: Q(s t ,a t )←Q(s t ,a t )+α[r t +γmaxQ(s t+1 ,a')-Q(s t ,a t )] (11) Among them, s t is the current state; a t is in state s t The action taken next; t Is to take action a t The immediate reward after t+1 is the new state obtained after taking action a; a' is the new state s t+1 All possible actions under; α is the learning rate; maxQ(s t+1 ,a′) is the next state s t+1 is the maximum Q-value of all possible actions, and γ is the discount factor.

10. The method for deploying critical transportation infrastructure considering cascading failure and reinforcement learning according to claim 9, It is characterized in that The S34 further includes: S345: Use greedy selection of the next action ε, that is, there is a probability of ε to randomly select a new action, and there is a probability of 1-ε to select the action with the largest Q value in the current state; S346: Repeat S342-S345 until the preset number of iterations is reached; update W during the iteration process 4 and W 5 , the best Q function under the preset number of iterations is obtained, that is, the Q function when the degree of network efficiency reduction is maximized; according to the Q function when the degree of network efficiency reduction is maximized, the number and sequence of important nodes, that is, key infrastructure, are obtained.

Images