Urban flood emergency response method based on multi-dimensional infrastructure relationship network
By building a multi-dimensional infrastructure heterofunction network and partially observable Markov process, multi-intelligent deep reinforcement learning model is trained, which solves the emergency response problems caused by the interdependence of disasters on infrastructure systems, and realizes the urban flood disasters The best emergency response plan improves the efficiency and effectiveness of emergency response.
Patent Information
- Application Number
- CN202411252132.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-09
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-09-09
AI Technical Summary
Under the influence of extreme natural disasters or man-made disasters, critical infrastructure systems may fail, resulting in serious interruption in the provision of basic services such as medical care, electricity and transportation. It is difficult for existing technologies to effectively deal with the infrastructure systems caused by such disasters. Interdependence problem.
By building a multi-dimensional infrastructure heterogeneous network and building part of the observable Markov process based on this, the multi-intelligent deep reinforcement learning model is trained to obtain the urban flood emergency response model. Determine the list of damaged physical components based on urban rainfall data, simulate the city's disaster situation, and generate the best emergency response plan through the emergency response model.
The optimal emergency response plan for a variety of infrastructure systems under urban flood disasters has been achieved, the efficiency and effectiveness of disaster response has been improved, and the stability and service continuity of infrastructure systems have been ensured.
Smart Images

Figure CN118761889B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of disaster emergency response, and in particular to an urban flood emergency response method based on a multi-dimensional infrastructure relationship network. Background Art
[0002] Critical infrastructure systems (CIS), including power systems, water systems, and transportation systems, are large-scale engineering systems used to provide basic goods and services that sustain modern society and economy. When critical infrastructure is affected by extreme natural or man-made disasters, its physical components may fail. Due to the interdependencies between various infrastructure systems, these failures may escalate, causing serious disruptions in the provision of basic services such as medical care, electricity, and transportation, posing a threat to the normal functioning of society. Summary of the invention
[0003] In view of this, the present application provides an urban flood emergency response method based on a multi-dimensional infrastructure relationship network, aiming to determine the best disaster emergency response plan for the infrastructure system based on the disaster scenario.
[0004] In a first aspect of an embodiment of the present application, a method for emergency handling of urban floods based on a multi-dimensional infrastructure relationship network is provided, the method comprising:
[0005] Based on various infrastructure systems, build an interdependent multi-dimensional infrastructure heterogeneous network;
[0006] According to the multi-dimensional infrastructure hetero-functional network, construct a partially observable Markov process corresponding to the multi-dimensional infrastructure hetero-functional network;
[0007] The multi-intelligence deep reinforcement learning model is trained through the partially observable Markov process to obtain a qualified urban flood emergency response model;
[0008] Based on the city’s rainfall data, a list of damaged physical components was determined;
[0009] Simulating a disaster situation in a city based on the multi-dimensional infrastructure heterogeneous functional network and the list of damaged physical components;
[0010] The best emergency response plan is obtained by inputting the urban disaster situation into the urban flood emergency response model for processing.
[0011] Optionally, based on various infrastructure systems, build an interdependent multi-dimensional infrastructure heterogeneous network, including:
[0012] Identify the functions of the physical components in various infrastructure systems as functional nodes;
[0013] According to the mapping relationship between each physical component and each functional node, a corresponding knowledge base is constructed;
[0014] Based on the topological relationships between physical components in various infrastructure systems and the knowledge base, an interdependent multi-dimensional infrastructure hetero-functional network is determined.
[0015] Optionally, according to the multi-dimensional infrastructure hetero-functional network, constructing a partially observable Markov process corresponding to the multi-dimensional infrastructure hetero-functional network includes:
[0016] The managers of various infrastructure systems in the multi-dimensional infrastructure heterogeneous network are identified as a set of intelligent agents of partially observable Markov processes;
[0017] Determine the node attributes of all functional nodes in the multi-dimensional infrastructure hetero-functional network as a set of environmental states of partially observable Markov processes;
[0018] Determining a partially observable Markov process and a set of local observed environmental states corresponding to the agent according to the set of environmental states and the type of infrastructure system corresponding to the agent;
[0019] Determine the set of actions of the partially observable Markov process based on the physical components of the infrastructure system type corresponding to the agent;
[0020] Determine the reward function and state transition function of partially observable Markov processes based on multi-dimensional infrastructure heterogeneous networks;
[0021] According to the discount factor and the determined set of intelligent agents, the set of environmental states, the set of locally observed environmental states, the set of actions, the reward function and the state transfer function, a partially observable Markov process corresponding to the multi-dimensional infrastructure heterofunctional network is constructed.
[0022] Optionally, based on the multi-dimensional infrastructure heterogeneous network, a reward function of a partially observable Markov process is determined, including:
[0023] According to the multi-dimensional infrastructure heterogeneous network, the reward function of the partially observable Markov process is determined as: ;
[0024] in, Represents the performance value of a multi-dimensional infrastructure heterogeneous network, represents the comprehensive satisfaction of the i-th demand function node at time t, is the number of function nodes that can provide services to the i-th demand function node at time t, represents the total number of function nodes that can provide services to the i-th demand function node when no disaster occurs, represents the environmental state at time t, represents the environmental state at time t+1, represents the action performed at time t, Represents the time interval between time t and time t+1.
[0025] Optionally, managers of various infrastructure systems in the multi-dimensional infrastructure heterogeneous network are determined as a set of agents of partially observable Markov processes, including:
[0026] Assign a corresponding single manager to each type of infrastructure system in the multi-dimensional infrastructure heterogeneous network;
[0027] Identify a single manager as a single agent;
[0028] All agents corresponding to all managers are determined as the set of agents of partially observable Markov processes.
[0029] Optionally, the multi-intelligence deep reinforcement learning model is trained through the partially observable Markov process to obtain a qualified urban flood emergency response model, including:
[0030] Initializing the action set and the environment state set of the partially observable Markov process to obtain an initialization result;
[0031] According to the set of local observed environmental states of the partially observable Markov process, determining each local observed environmental state corresponding to each intelligent agent in the environmental state of the initialization result;
[0032] Input the local observed environment state corresponding to the agent into its corresponding single-agent behavior network to obtain the target value of the agent under each action;
[0033] Determining the target action of the agent according to the target values under the various actions;
[0034] According to the target action of each agent, the set of environmental states of the partially observable Markov process is updated, and the reward value under the target action is calculated by the reward function;
[0035] The reward value, the target action of each agent, all local observed environmental states input into the single agent behavior network, and the updated environmental states are stored in the experience pool to form a corresponding piece of historical data;
[0036] When the amount of historical data in the experience pool reaches the set value, the predicted joint value is determined through the hybrid network;
[0037] Calculating a loss function using the predicted joint value and the actual joint value;
[0038] When the loss function is stable, the training is terminated to obtain a qualified urban flood emergency response model;
[0039] When the loss function is not stable, the parameters of the multi-agent reinforcement learning model are updated according to the loss function to perform a new round of training.
[0040] Optionally, the loss function is: ;
[0041] in, , b is the number of samples sampled from the experience pool, is the target network, used as the actual joint value Calculation of is the update rate, r is the reward value at the current moment, θ and are the parameters of the hybrid network that calculates the predicted joint value and the parameters of the target network that calculates the actual joint value, respectively.
[0042] Optionally, based on the city’s rainfall data, determine a list of damaged physical components, including:
[0043] Based on the city’s rainfall data, determine the flood depth of each area at each moment;
[0044] According to the locations of the physical components in various infrastructure systems and the damaged water depths of the physical components, a damaged physical component list is determined, wherein the damaged physical component list records the time when each physical component is damaged.
[0045] Optionally, the node attributes of the functional node include: the node type of the functional node, the functional status of the functional node, the component status of the physical component corresponding to the functional node, and the repair time required after the physical component corresponding to the functional node is damaged.
[0046] Optionally, when the loss function is not stable, before updating the parameters of the multi-agent reinforcement learning model according to the loss function for a new round of training, the method further includes:
[0047] If the loss function is not stable, determining whether a preset number of training times has been reached;
[0048] When the preset number of training times is reached, the training is terminated to obtain a qualified urban flood emergency response model.
[0049] Compared with the prior art, this application has the following advantages:
[0050] The embodiment of the present application provides a method for emergency disposal of urban floods based on a multidimensional infrastructure relationship network. First, according to various infrastructure systems, a multidimensional infrastructure heterofunctional network that depends on each other is constructed; according to the multidimensional infrastructure heterofunctional network, a partially observable Markov process corresponding to the multidimensional infrastructure heterofunctional network is constructed; the multi-intelligent deep reinforcement learning model is trained through the partially observable Markov process to obtain a qualified urban flood emergency disposal model; according to the urban rainfall data, a list of damaged physical components is determined; according to the multidimensional infrastructure heterofunctional network and the list of damaged physical components, the urban disaster situation is simulated; by inputting the urban disaster situation into the urban flood emergency disposal model for processing, the best emergency disposal plan is obtained. Therefore, the present application obtains an urban flood emergency disposal model by training the multi-intelligent deep reinforcement learning model through a partially observable Markov process constructed based on the multidimensional infrastructure heterofunctional network, and uses it to process the urban disaster situation, so as to determine the best emergency disposal plan for various infrastructure systems under the urban disaster situation according to the urban disaster situation.
[0051] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.
[0053] Figure 1 A flowchart of an urban flood emergency response method based on a multi-dimensional infrastructure relationship network provided in an embodiment of the present application;
[0054] Figure 2 A schematic diagram of creating a multi-dimensional infrastructure heterogeneous network based on topological relationships of physical components in an urban flood emergency response method based on a multi-dimensional infrastructure relationship network provided in an embodiment of the present application;
[0055] Figure 3 A structural diagram of a multi-intelligence deep reinforcement learning model in an urban flood emergency response method based on a multi-dimensional infrastructure relationship network provided in an embodiment of the present application. DETAILED DESCRIPTION
[0056] Exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings.
[0057] Figure 1A flowchart of a method for emergency handling of urban floods based on a multi-dimensional infrastructure relationship network is provided in an embodiment of the present application, such as Figure 1 As shown, the method includes:
[0058] Step S1: Based on various infrastructure systems, construct an interdependent multi-dimensional infrastructure heterogeneous network.
[0059] In this embodiment, the infrastructure systems used to provide services to residents include various types, such as power systems, water supply systems, gas supply systems, etc. This application constructs an interdependent multi-dimensional infrastructure hetero-functional network based on the various functions of all physical components included in the various types of infrastructure systems. The nodes in the multi-dimensional infrastructure hetero-functional network are functional nodes, each functional node corresponds to a function of a physical component in the various types of infrastructure systems, and any physical component in the various types of infrastructure systems has at least one function, and some physical components have multiple functions. For ease of description, the various types of infrastructure systems that provide services to residents are collectively referred to as integrated infrastructure systems, that is, the integrated infrastructure system refers to a set of infrastructure systems including various types of infrastructure systems that provide services to residents.
[0060] Step S2: Based on the multi-dimensional infrastructure hetero-functional network, construct a partially observable Markov process corresponding to the multi-dimensional infrastructure hetero-functional network.
[0061] In this embodiment, based on the multi-dimensional infrastructure hetero-functional network constructed in step S1, a partially observable Markov process corresponding to the multi-dimensional infrastructure hetero-functional network is constructed.
[0062] The agent set of the partially observable Markov process constructed is composed of managers of various infrastructure systems in the multi-dimensional infrastructure hetero-functional network, and one manager manages one type of infrastructure system corresponding to itself; the environmental state set of the partially observable Markov process constructed is composed of the node attributes of each functional node in the multi-dimensional infrastructure hetero-functional network; the local observed environmental state set of the partially observable Markov process constructed includes multiple, one local observed environmental state set will correspond to one type of infrastructure system, and the one local observed environmental state set is composed of the node attributes of all functional nodes in the one type of infrastructure system; the action set of the partially observable Markov process constructed is composed of the actions performed by each agent on the damaged physical components of the infrastructure system managed by itself. At the same time, the partially observable Markov process constructed also includes a reward function, a state transition function and a discount factor.
[0063] Step S3: The multi-intelligence deep reinforcement learning model is trained through the partially observable Markov process to obtain a qualified urban flood emergency response model.
[0064] In this embodiment, the multi-intelligence deep reinforcement learning model is trained through the partially observable Markov process constructed in step S2. When the multi-intelligence deep reinforcement learning model is trained successfully, an urban flood emergency response model is obtained, wherein the multi-intelligence deep reinforcement learning model is preferably Qmix (Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning).
[0065] Step S4: Determine a list of damaged physical components based on urban rainfall data.
[0066] In this embodiment, based on the collected urban rainfall data, it is determined which physical components in the integrated infrastructure system were damaged and at which times, so as to obtain a list of damaged physical components, which records the damaged physical components and the damage time of the damaged physical components.
[0067] Step S5: Simulate the disaster situation of the city based on the multi-dimensional infrastructure heterogeneous network and the damaged physical component list.
[0068] In this embodiment, based on the multi-dimensional infrastructure heterofunctional network and the list of damaged physical components, the disaster situation of the city is simulated to obtain the simulation result of the city disaster situation. The simulation result of the city disaster situation refers to the specific values of the node attributes of each functional node in the multi-dimensional infrastructure heterofunctional network under the disaster situation recorded in the list of damaged physical components.
[0069] Step S6: The disaster situation of the city is input into the urban flood emergency response model for processing to obtain the best emergency response plan.
[0070] In this embodiment, the simulation results of the urban disaster situation obtained by simulation in step S5 are input into a trained urban flood emergency response model for processing, and the urban flood emergency response model outputs the best emergency response plan for the integrated infrastructure system determined based on the simulation results.
[0071] The embodiment of the present application provides a method for emergency disposal of urban floods based on a multidimensional infrastructure relationship network. First, according to various infrastructure systems, a multidimensional infrastructure heterofunctional network that depends on each other is constructed; according to the multidimensional infrastructure heterofunctional network, a partially observable Markov process corresponding to the multidimensional infrastructure heterofunctional network is constructed; the multi-intelligent deep reinforcement learning model is trained through the partially observable Markov process to obtain a qualified urban flood emergency disposal model; according to the urban rainfall data, a list of damaged physical components is determined; according to the multidimensional infrastructure heterofunctional network and the list of damaged physical components, the urban disaster situation is simulated; by inputting the urban disaster situation into the urban flood emergency disposal model for processing, the best emergency disposal plan is obtained. Therefore, the present application obtains an urban flood emergency disposal model by training the multi-intelligent deep reinforcement learning model through a partially observable Markov process constructed based on the multidimensional infrastructure heterofunctional network, and uses it to process the urban disaster situation, so as to determine the best emergency disposal plan for various infrastructure systems under the urban disaster situation according to the urban disaster situation.
[0072] In combination with the above embodiments, in one implementation, the present application embodiment further provides an urban flood emergency response method based on a multi-dimensional infrastructure relationship network. In the urban flood emergency response method based on a multi-dimensional infrastructure relationship network, step S1 may include steps S11 to S13:
[0073] Step S11: Determine the functions of the physical components in the various infrastructure systems as functional nodes.
[0074] In this embodiment, based on each physical component in the comprehensive infrastructure system including all types of infrastructure systems, the function of each physical component is determined, and then each function is determined as a corresponding function node.
[0075] By way of example, a simple integrated infrastructure system is taken as an example for explanation. The integrated infrastructure system is composed of multiple types of infrastructure systems, and each type of infrastructure system includes corresponding physical components, such as power plant physical component R1, substation physical component R2, communication base station R3, substation physical component R4, gas station physical component R5, transmission line physical component E1 between R1 and R2, transmission line physical component E2 between R2 and R3, transmission line component E3 between R1 and R4, and transmission line component E4 between R4 and R4. The function of the physical component R1 is to generate electricity, so a corresponding functional node φ1 is determined; the function of the physical component E1 is to transmit electricity from R1 to R2, so a corresponding functional node φ2 is determined; the physical component R2 is used to convert high voltage electricity into low voltage electricity, so a corresponding functional node φ3 is determined; the function of the physical component E2 is to transmit electricity from R2 to R3, so a corresponding functional node φ4 is determined; the function of the physical component R3 is to realize voice communication, so a corresponding functional node φ5 is determined; the function of the physical component R3 is to perform wireless signal transmission, so a corresponding functional node φ6 is determined; the function of the physical component E3 is to transmit electricity from R1 to R4, so a corresponding functional node φ7 is determined; the function of the physical component R4 is to convert high voltage electricity into low voltage electricity, so a corresponding functional node φ8 is determined; the function of the physical component E4 is to transmit electricity from R4 to R5, so a corresponding functional node φ9 is determined; the function of the physical component R5 is to distribute gas, so a corresponding functional node φ10 is determined.
[0076] Step S12: construct a corresponding knowledge base according to the mapping relationship between each physical component and each functional node.
[0077] In this embodiment, since each functional node determined by step S12 is determined by the functions of all physical components included in the integrated infrastructure system, a mapping relationship between the functional node and the physical component is created based on the physical component to which the function of the functional node belongs. Thus, a mapping relationship between each functional node and the physical component can be created. Based on all the obtained mapping relationships, a corresponding knowledge base is created, in which the value of the knowledge base element corresponding to the functional node and physical component having a mapping relationship is 1.
[0078] For example, continuing to use the example in the above step S11, the function corresponding to the function node φ1 is the function of the physical component R1, so a mapping relationship between the function node φ1 and the physical component R1 is created; the function corresponding to the function node φ2 is the function of the physical component E1, so a mapping relationship between the function node φ1 and the physical component E1 is created, and through the same implementation method, a mapping relationship between φ3 to φ10 and the physical components corresponding to themselves will be created. According to all the mapping relationships obtained, a corresponding knowledge base is created, as shown in Table 1, which is a knowledge base determined based on the mapping relationships between the function nodes φ1 to φ10 and the physical components R1 to R5 and E1 to E4.
[0079] Table 1
[0080]
[0081] Step S13: Determine an interdependent multi-dimensional infrastructure heterogeneous network based on the topological relationship between the physical components in various infrastructure systems and the knowledge base.
[0082] In this embodiment, all the function nodes determined by step S11 are determined as nodes in the multidimensional infrastructure heterofunctional network, and then based on the knowledge base constructed by step S12, it is determined to which physical component each function node specifically belongs. After determining the physical component to which each function node belongs, the topological relationship between the physical components corresponding to each two function nodes is determined, and the relationship between the two function nodes is created based on the topological relationship, and the corresponding initial multidimensional infrastructure heterofunctional network is obtained based on all the created relationships. For the initial multidimensional infrastructure heterofunctional network, each demand function node is constructed in the initial multidimensional infrastructure heterofunctional network, and the demand function node is a node constructed based on a community in the city, and the community can be a community with streets as units, or a community with communities as units. Then, based on whether each function node in the initial multidimensional infrastructure heterofunctional network provides services for the demand function node, the relationship between the function node and the demand function node is created, so as to obtain the final interdependent multidimensional infrastructure heterofunctional network.
[0083] For example, continue to use the example in step S12 above, such as Figure 2 As shown, Figure 2, a topological relationship between physical components R1 to R5 and E1 to E4 in an integrated infrastructure system including all types of infrastructure systems is shown. Based on the fact that function node φ1 belongs to physical component R1 and function node φ2 belongs to physical component E1, and based on the topological relationship between R1 and E1, a relationship is created from function node φ1 to function node φ2; based on the fact that function node φ2 belongs to physical component E1 and function node φ3 belongs to physical component R2, and based on the topological relationship between E1 and R2, a relationship is created from function node φ2 to function node φ3; based on the fact that function node φ3 belongs to physical component R2 and function node φ4 belongs to physical component E2, and based on the topological relationship between R2 and E2, a relationship is created from function node φ3 to function node φ4; based on the fact that function node φ4 belongs to physical component E2 and function node φ5 and φ6 all belong to the physical component R3, and based on the topological relationship between E2 and R3, a relationship is created from the function node φ4 to the function nodes φ5 and φ6, and so on. Subsequently, the relationship between the function nodes is created in the same way. After the creation of the relationship between all the function nodes is completed, the corresponding initial multi-dimensional infrastructure heterofunctional network is obtained, and then the demand function nodes are constructed in the initial multi-dimensional infrastructure heterofunctional network. For example, the city includes 3 communities, and 3 demand function nodes N1, N2, and N3 are created accordingly. Then, according to whether the function nodes in the initial multi-dimensional infrastructure heterofunctional network will provide services to the demand function nodes, the relationship between the function nodes and the demand function nodes is created. For example, the function node φ5 for realizing voice communication is used to provide voice communication services for the demand function nodes N1 and N2 in actual work, so the relationship between φ5 and the demand function nodes N1 and N2 is created respectively; the function node φ6 for realizing wireless signal transmission is used to provide wireless signal transmission services for the demand function nodes N2 and N3 in actual work; the function node φ10 for realizing gas distribution is used to provide gas services for the demand function nodes N1 to N3 in actual work, so the relationship between the function node φ10 and the demand nodes N1 to N3 is created respectively. After completing the relationship construction between all the function nodes and the demand function nodes, the final multi-dimensional infrastructure heterofunctional network is obtained, such as Figure 2 As shown, Figure 2 The final multi-dimensional infrastructure hetero-functional network generated for this example is shown.
[0084] In combination with the above embodiments, in one implementation, the embodiment of the present application further provides an urban flood emergency response method based on a multi-dimensional infrastructure relationship network. In the urban flood emergency response method based on a multi-dimensional infrastructure relationship network, step S2 may include steps S21 to S26:
[0085] Step S21: managers of various infrastructure systems in the multi-dimensional infrastructure heterogeneous network are determined as a set of intelligent agents of a partially observable Markov process.
[0086] In this embodiment, a corresponding manager is assigned to each type of infrastructure system used to construct a multi-dimensional infrastructure heterofunctional network, and then all managers of all types of infrastructure systems are determined as a set of intelligent agents of a partially observable Markov process.
[0087] Step S22: Determine the node attributes of all functional nodes in the multi-dimensional infrastructure hetero-functional network as a set of environmental states of a partially observable Markov process.
[0088] In the present application, the node attributes of a functional node include: the node type of the functional node, the functional status of the functional node, the component status of the physical component corresponding to the functional node, and the repair time required after the physical component corresponding to the functional node is damaged.
[0089] In this embodiment, the functional nodes in the constructed multi-dimensional infrastructure hetero-functional network have corresponding node attributes. This application determines the node attributes of all functional nodes in the constructed multi-dimensional infrastructure hetero-functional network as a set of environmental states of a partially observable Markov process. Among them, the node attributes include , The node type of the functional node includes supply functional node, transfer functional node and demand functional node. The supply functional node is a functional node where the in-situ phase change process occurs (such as a power station, a substation, etc.), and the transfer functional node is a functional node where the transmission process occurs (such as a transmission line that transmits electric energy from a power station to a substation, etc.); It is the function status of the function node, which belongs to the bool type, 0 represents abnormal function, and 1 represents normal function; It is the component status of the physical component corresponding to the function node, which is of bool type, 0 represents abnormality and 1 represents normality; It is the repair time required after the physical component corresponding to the functional node is damaged, which belongs to the float type.
[0090] Step S23: Determine a partially observable Markov process and a set of local observed environmental states corresponding to the agent according to the set of environmental states and the type of infrastructure system corresponding to the agent.
[0091] In this embodiment, each environmental state in the environmental state set is classified according to the infrastructure system type corresponding to each environmental state in the determined environmental state set, and the environmental states belonging to the same infrastructure system type are divided into the same category to form a corresponding environmental state subset. Then, according to the infrastructure system corresponding to the environmental state subset, the intelligent agent that manages the infrastructure system is determined, and the environmental state subset is determined as the local observed environmental state set of the intelligent agent, and the intelligent agent is used to observe the specific state value of each environmental state in the local observed environmental state set. Since the environmental state in the environmental state set is converted from the attributes of the functional node, based on the functional node corresponding to the environmental state, and the functional node is related to the infrastructure system, the infrastructure system type to which each environmental state in the environmental state set belongs can be determined.
[0092] Step S24: Determine a set of actions of a partially observable Markov process according to the physical components in the infrastructure system type corresponding to the agent.
[0093] In this embodiment, a corresponding action subset is created for each agent in the agent set, and the action subset of the agent is the set of actions performed by the agent on the damaged physical components in the infrastructure system managed by the agent. Then, the set of all action subsets of all agents is determined as the action set of the partially observable Markov process. The action set is expressed as ,in Represents the action of the i-th agent in the agent set, represented by a one-dimensional vector. The length of the one-dimensional vector is the number of damaged physical components in the infrastructure system type corresponding to the i-th agent. An element in the one-dimensional vector corresponds to a physical component. An element in the one-dimensional vector is 0, which means that the maintenance team does not choose to repair the physical component corresponding to the element. When the element is other values, it indicates the number of maintenance teams assigned to the physical component corresponding to the element.
[0094] Step S25: Determine the reward function and state transition function of the partially observable Markov process based on the multi-dimensional infrastructure heterogeneous network.
[0095] In this embodiment, based on the constructed multi-dimensional infrastructure heterofunctional network, a reward function of a partially observable Markov process that rewards the multi-dimensional infrastructure heterofunctional network for performing corresponding actions to change the environmental state is determined, and a state transition function of the partially observable Markov process is determined.
[0096] In the present application, according to the multidimensional infrastructure heterofunctional network, determining the reward function of the partially observable Markov process includes: according to the multidimensional infrastructure heterofunctional network, determining the reward function of the partially observable Markov process as: ;in, Represents the performance value of a multi-dimensional infrastructure heterogeneous network, represents the comprehensive satisfaction of the i-th demand function node at time t, is the number of function nodes that can provide services to the i-th demand function node at time t, represents the total number of function nodes that can provide services to the i-th demand function node when no disaster occurs, represents the environmental state at time t, represents the environmental state at time t+1, represents the action performed at time t, Represents the time interval between time t and time t+1.
[0097] In this embodiment, the present application is based on a multi-dimensional infrastructure heterogeneous network and determines the reward function of a partially observable Markov process as .in, , , is the number of function nodes that can provide services to the i-th demand function node at time t, represents the total number of function nodes that can provide services to the i-th demand function node when no disaster occurs, represents the environmental state at time t, represents the environmental state at time t+1, represents the action performed at time t, represents the time interval between time t and time t+1. The state transition function of a partially observable Markov process is , indicating taking specific actions The state of the functional node in the heterogeneous network of the post-multidimensional infrastructure is determined by the current Transform to state The transition probability.
[0098] Step S26: Construct a partially observable Markov process corresponding to the multidimensional infrastructure heterofunctional network based on the discount factor and the determined set of intelligent agents, the set of environmental states, the set of locally observed environmental states, the set of actions, the reward function and the state transfer function.
[0099] In this embodiment, a partially observable Markov process corresponding to a multi-dimensional infrastructure heterogeneous network is constructed through the agent set, environment state set, local observed environment state set, action set, reward function, state transition function and discount factor determined in the above steps S21 to S25. The discount factor has a value between 0 and 1.
[0100] In combination with the above embodiments, in one implementation, the present application embodiment further provides an urban flood emergency response method based on a multi-dimensional infrastructure relationship network. In the urban flood emergency response method based on a multi-dimensional infrastructure relationship network, step S21 may include steps S211 to S213:
[0101] Step S211: assigning a corresponding single manager to each type of infrastructure system in the multi-dimensional infrastructure heterogeneous network.
[0102] In this embodiment, a single manager is assigned to each type of infrastructure system involved in the multi-dimensional infrastructure heterogeneous network, that is, one manager is assigned to one type of infrastructure system, and the manager is used to observe the environmental status of the infrastructure system of the corresponding type.
[0103] Step S212: Determine a single manager as a single agent.
[0104] In this embodiment, a single manager is determined as a single intelligent agent, and there are as many managers and intelligent agents as there are types of infrastructure systems involved in the multi-dimensional infrastructure heterofunctional network.
[0105] Step S213: All agents corresponding to all managers are determined as an agent set of a partially observable Markov process.
[0106] In this embodiment, all agents determined by step S212 are determined as a set of agents of a partially observable Markov process.
[0107] For example, when the types of infrastructure systems used to construct a multi-dimensional infrastructure hetero-functional network include X1, X2, and X3, a corresponding manager x1 is assigned to infrastructure system X1, and manager x1 can only partially observe the status of all functional nodes in infrastructure system X1; a corresponding manager x2 is assigned to infrastructure system X2, and manager x2 can only partially observe the status of all functional nodes in infrastructure system X2; a corresponding manager x3 is assigned to infrastructure system X3, and manager x3 can only partially observe the status of all functional nodes in infrastructure system X3. Then, each manager is determined as an intelligent agent, forming an intelligent agent set consisting of managers x1, x2, and x3.
[0108] In combination with the above embodiments, in one implementation, the embodiment of the present application further provides an urban flood emergency response method based on a multi-dimensional infrastructure relationship network. In the urban flood emergency response method based on a multi-dimensional infrastructure relationship network, step S3 may include steps S301 to S310:
[0109] Step S301: Initialize the action set and the environment state set of the partially observable Markov process to obtain an initialization result.
[0110] In this embodiment, the action set and environment state set of a partially observable Markov process constructed based on a multi-dimensional infrastructure heterofunctional network are initialized. Specifically, the action set of the partially observable Markov process is initialized to no action execution, and each environment state in the environment state set of the partially observable Markov process is randomized. For example, the functional states of some functional nodes in the environment state set are randomized to be abnormal, and the states of physical components corresponding to some functional nodes are randomized to be abnormal, etc., so as to obtain an initialization result for the partially observable Markov process.
[0111] Step S302: According to the set of local observed environmental states of the partially observable Markov process, determine each local observed environmental state corresponding to each intelligent agent in the environmental state of the initialization result.
[0112] In this embodiment, since the set of local observed environmental states corresponding to the agent of the partially observable Markov process is obtained from the set of environmental states of the partially observable Markov process, after the set of environmental states of the partially observable Markov process is initialized in step S301, the values of each environmental state in the set of environmental states are also determined. Therefore, based on the set of local observed environmental states of the partially observable Markov process, the values of each environmental state in the set of local observed environmental states corresponding to the agent in the environmental state of the initialized result can be determined.
[0113] Step S303: Input the local observed environment state corresponding to the agent into the single-agent behavior network corresponding to itself to obtain the target value of the agent under each action.
[0114] In this embodiment, if Figure 3As shown, in the multi-intelligence deep reinforcement learning model, each agent has its own corresponding single-agent behavior network, and the values of each environmental state in the local observed environmental state set corresponding to the agent determined by step S302 are input into the single-agent behavior network corresponding to the agent for processing, and the target value of the agent under various different actions is obtained. Based on the same implementation method, each agent inputs the values of each environmental state in the local observed environmental state set corresponding to itself into the single-agent behavior network corresponding to itself for processing, and will obtain its own target value under various different actions. It should be understood that the action of a single agent refers to the action of the agent for the damaged physical component under the infrastructure system type corresponding to itself, and the action includes an action group that has a corresponding action for each damaged physical component. For example, if there are 3 damaged physical components under the infrastructure system type corresponding to the agent, there is a corresponding action group consisting of 3 actions.
[0115] Step S304: Determine the target action of the agent according to the target values of the actions.
[0116] In this embodiment, for each agent's target value under each action corresponding to itself, an action with the largest target value is selected as its own target action.
[0117] Step S305: According to the target action of each agent, the set of environmental states of the partially observable Markov process is updated, and the reward value under the target action is calculated through the reward function.
[0118] In this embodiment, according to the determined target actions of each intelligent agent, the values of each environmental state in the environmental state set of the partially observable Markov process are updated, and the reward value obtained by each intelligent agent after executing its respective determined target action is calculated through the reward function of the partially observable Markov process.
[0119] Step S306: The reward value, the target action of each agent, all local observed environmental states input into the single-agent behavior network, and the updated environmental states are stored in the experience pool to form a corresponding piece of historical data.
[0120] In this embodiment, the reward value of this round, the target action of each agent, the values of all local observed environmental states of each single-agent behavior network, and the values of each environmental state in the updated environmental state set are input into the experience pool of the multi-intelligent deep reinforcement learning model to form a corresponding historical data.
[0121] Step S307: When the amount of historical data in the experience pool reaches a set value, the predicted joint value is determined through the hybrid network.
[0122] In this embodiment, when the number of historical data in the experience pool does not reach the set value, the determination of the historical data of a new round is performed. Specifically, based on the values of each environmental state in the updated environmental state set recorded in the last historical data in the experience pool, the values of each environmental state in the local observed environmental state set corresponding to the agent are determined. Then, the values of the environmental states in the local observed environmental state set corresponding to each of the determined agents are input into the corresponding single agent behavior network, and the target values under various actions of each agent are obtained. Then, for each agent, an action with the largest target value is selected as its target action. After the target action of all agents is determined, based on the target action of all agents, the values of each environmental state in the environmental state set of the partially observable Markov process are updated, and the reward value obtained after each agent performs the target action determined by each agent is calculated by the reward function of the partially observable Markov process, and the reward value is the reward value of the new round. The reward value of the new round, the target action of each agent in the new round, the values of all local observed environmental states input into each single-agent behavior network in the new round, and the values of each environmental state in the updated environmental state set of the new round are stored in the experience pool of the multi-intelligence deep reinforcement learning model to form a new historical data. Then continue to determine whether the number of historical data in the experience pool has reached the set value. If it has not reached the set value, the determination of the historical data for the new round is performed again. The implementation method is the same as the determination of the historical data for the new round mentioned above, and will not be repeated here until the number of historical data in the experience pool reaches the set value. When the number of historical data in the experience pool reaches the set value, the predicted joint value is determined by the hybrid network in the multi-intelligence deep reinforcement learning model. The number of times can be set according to the actual application scenario and is not specifically limited here, such as 5000. Figure 3 As shown, Figure 3 The schematic diagram of the hybrid network determining the predicted joint value in the multi-intelligence deep reinforcement learning model of the present application is shown. It represents the value of each environment state in the environment state set at time t, and the input of the single agent behavior network N Represents the environmental state of the Nth agent at time t in the set of local observed environmental states corresponding to itself and the target action to be performed , the input hybrid network in the figure Represents the temporal action-observation history at time t under the set of local observation environment states and target action The predicted value of the Nth agent under the condition of Represents the observed value of the environmental state sequence under the environmental state set at time t and the target actions of all agents The predicted joint value under .
[0123] In this embodiment, the predicted joint value The expression is:
[0124] .
[0125] in, The time-series action-observation history at the current moment. The time-series action-observation history records the actions and the values of the observed environmental states at each moment in the time period from the start moment to the end moment. For example, the time-series action-observation history at the current moment records the actions and the values of the observed environmental states in the time period from the start moment to the current moment. is the one-dimensional vector of the Nth agent’s action.
[0126] Step S308: Calculate a loss function using the predicted joint value and the actual joint value.
[0127] In this embodiment, the predicted joint value determined in step S307 Calculate the loss function with the actual joint value.
[0128] In this application, the loss function is: ;in, , b is the number of samples sampled from the experience pool, is the target network, used as the actual joint value Calculation of is the update rate, r is the reward value at the current moment, θ and are the parameters of the hybrid network that calculates the predicted joint value and the parameters of the target network that calculates the actual joint value, respectively.
[0129] In this embodiment, the expression of the loss function is: .in, represents the actual joint value, θ and They are the parameters of the hybrid network for calculating the predicted joint value and the parameters of the target network for calculating the actual joint value. Among them, the environmental state sequence observation value records the values of the environmental state at each moment in the time period from the start time to the end time. For example, the environmental state sequence observation value at the current moment records the values of the environmental state at each moment in the time period from the start time to the current moment, and the environmental state sequence observation value at the next moment of the current moment records the values of the environmental state at each moment in the time period from the start time to the next moment of the current moment.
[0130] Step S309: When the loss function is stable, the training is terminated to obtain a qualified urban flood emergency response model.
[0131] In this embodiment, when the calculated loss function reaches stability, the training is terminated to obtain a qualified urban flood emergency response model that can be used to determine the best emergency response plan.
[0132] Step S310: When the loss function is not stable, update the parameters of the multi-agent reinforcement learning model according to the loss function to perform a new round of training.
[0133] In this embodiment, when the loss function is not stable, the parameters of the multi-agent reinforcement learning model are updated based on the loss function, and a new round of training is performed after the update.
[0134] In combination with the above embodiments, in one implementation, the embodiment of the present application further provides an urban flood emergency response method based on a multi-dimensional infrastructure relationship network. In the urban flood emergency response method based on a multi-dimensional infrastructure relationship network, before a new round of training is performed in step S310, the method further includes steps S01 to S02:
[0135] Step S01: when the loss function is not stable, determining whether a preset number of training times has been reached.
[0136] In this embodiment, when it is determined that the loss function is not stable, it is determined whether the current multi-agent reinforcement learning model has reached a preset number of training times.
[0137] Step S02: When the preset number of training times is reached, the training is terminated to obtain a qualified urban flood emergency response model.
[0138] In this embodiment, if the current multi-agent reinforcement learning model has not reached the preset number of training times, the parameters of the multi-agent reinforcement learning model are updated based on the loss function, and a new round of training is performed after the update. If the current multi-agent reinforcement learning model has reached the preset number of training times, the training is terminated to obtain a qualified urban flood emergency response model that can be used to determine the best emergency response plan. Among them, the preset number of training times can be set according to the actual scenario, and is not limited here, such as training 50,000 times.
[0139] In combination with the above embodiments, in one implementation, the present application embodiment further provides an urban flood emergency response method based on a multi-dimensional infrastructure relationship network. In the urban flood emergency response method based on a multi-dimensional infrastructure relationship network, step S4 may include steps S41 to S42:
[0140] Step S41: Determine the flood depth of each area at each time based on the urban rainfall data.
[0141] In this embodiment, urban rainfall data is collected in real time, and the flooding depth of each area in the city at each different time is determined by analyzing the collected urban rainfall data.
[0142] Step S42: Determine a list of damaged physical components according to the locations of the physical components in various infrastructure systems and the damaged water depths of the physical components. The damaged physical component list records the time when each physical component was damaged.
[0143] In this embodiment, the application pre-constructs a mapping relationship between physical components and damage depths in various infrastructure systems. The mapping relationship may be that the same type of physical components correspond to one damage depth. In addition, for the same type of physical components, due to different construction locations, the damage depths that cause the physical components to be damaged may also be different. Therefore, the mapping relationship may also create a corresponding damage depth for each physical component built at different locations. For example, the damage depth corresponding to physical component 1 built at location a in city 1 is 50 cm, while the damage depth corresponding to physical component 2 of the same component type as physical component 1 built at location b in city 1 is 80 cm. Among them, the damage depth corresponding to the physical component refers to the physical component being damaged when the submerged water depth at the location of the physical component reaches the damage depth. Then, after determining the flooding depths of various areas in the city at different times through step S41, the flooding depths of the physical components in various infrastructure systems in the city at various times are compared with the damaged water depths corresponding to the physical components in the mapping relationship, and the moment when the flooding depth at the location of the physical component is greater than or equal to the damaged water depth corresponding to the physical component is determined as the damaged moment of the physical component, which is then recorded in the damaged physical component list. The damaged physical component list records the damaged moments of each physical component.
[0144] It should be noted that, for the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.
[0145] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0146] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the embodiments of the present application may adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0147] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.
[0148] The above is a detailed introduction to the urban flood emergency response method based on a multi-dimensional infrastructure relationship network provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. A method for emergency handling of urban floods based on a multi-dimensional infrastructure relationship network, characterized in that: The method comprises: According to various infrastructure systems, a multi-dimensional infrastructure hetero-functional network that is interdependent is constructed, wherein the nodes in the multi-dimensional infrastructure hetero-functional network are functional nodes, each functional node corresponds to a function of a physical component in various types of infrastructure systems, any physical component in various types of infrastructure systems has at least one function, and some physical components have multiple functions; According to the multi-dimensional infrastructure hetero-functional network, construct a partially observable Markov process corresponding to the multi-dimensional infrastructure hetero-functional network; The multi-intelligence deep reinforcement learning model is trained through the partially observable Markov process to obtain a qualified urban flood emergency response model; Based on the city’s rainfall data, a list of damaged physical components was determined; Simulating a disaster situation in a city based on the multi-dimensional infrastructure heterogeneous functional network and the list of damaged physical components; The best emergency response plan is obtained by inputting the disaster situation of the city into the urban flood emergency response model for processing; According to the multi-dimensional infrastructure hetero-functional network, a partially observable Markov process corresponding to the multi-dimensional infrastructure hetero-functional network is constructed, including: assigning a corresponding single manager to each type of infrastructure system in the multi-dimensional infrastructure hetero-functional network; determining a single manager as a single agent; determining all agents corresponding to all managers as an agent set of the partially observable Markov process; determining the node attributes of all functional nodes in the multi-dimensional infrastructure hetero-functional network as an environmental state set of the partially observable Markov process; determining the partially observable Markov process and the local observed environmental state set corresponding to the agent according to the environmental state set and the infrastructure system type corresponding to the agent; determining the action set of the partially observable Markov process according to the physical components in the infrastructure system type corresponding to the agent; determining the reward function and state transfer function of the partially observable Markov process according to the multi-dimensional infrastructure hetero-functional network; constructing the partially observable Markov process corresponding to the multi-dimensional infrastructure hetero-functional network according to the discount factor and the determined agent set, the environmental state set, the local observed environmental state set, the action set, the reward function and the state transfer function; Wherein, determining the reward function of the partially observable Markov process according to the multidimensional infrastructure heterofunctional network includes: determining the reward function of the partially observable Markov process according to the multidimensional infrastructure heterofunctional network as: ;in, Represents the performance value of a multi-dimensional infrastructure heterogeneous network, represents the comprehensive satisfaction of the i-th demand function node at time t, is the number of function nodes that can provide services to the i-th demand function node at time t, represents the total number of function nodes that can provide services to the i-th demand function node when no disaster occurs, represents the environmental state at time t, represents the environmental state at time t+1, represents the action performed at time t, Represents the time interval between time t and time t+1.
2. The urban flood emergency response method based on a multi-dimensional infrastructure relationship network according to claim 1 is characterized in that: Based on various infrastructure systems, we build an interdependent multi-dimensional infrastructure heterogeneous network, including: Identify the functions of the physical components in various infrastructure systems as functional nodes; According to the mapping relationship between each physical component and each functional node, a corresponding knowledge base is constructed; Based on the topological relationships between physical components in various infrastructure systems and the knowledge base, an interdependent multi-dimensional infrastructure hetero-functional network is determined.
3. The urban flood emergency response method based on a multi-dimensional infrastructure relationship network according to claim 1 is characterized in that: The multi-intelligent deep reinforcement learning model is trained through the partially observable Markov process to obtain a qualified urban flood emergency response model, including: Initializing the action set and the environment state set of the partially observable Markov process to obtain an initialization result; According to the set of local observed environment states of the partially observable Markov process, determining each local observed environment state corresponding to each intelligent agent in the environment state of the initialization result; Input the local observed environment state corresponding to the agent into its corresponding single-agent behavior network to obtain the target value of the agent under each action; Determining the target action of the agent according to the target values under the various actions; According to the target action of each agent, the set of environmental states of the partially observable Markov process is updated, and the reward value under the target action is calculated through the reward function; The reward value, the target action of each agent, all local observed environmental states input into the single agent behavior network, and the updated environmental states are stored in the experience pool to form a corresponding piece of historical data; When the amount of historical data in the experience pool reaches the set value, the predicted joint value is determined through the hybrid network; Calculating a loss function using the predicted joint value and the actual joint value; When the loss function is stable, the training is terminated to obtain a qualified urban flood emergency response model; When the loss function is not stable, the parameters of the multi-agent reinforcement learning model are updated according to the loss function to perform a new round of training.
4. The urban flood emergency response method based on a multi-dimensional infrastructure relationship network according to claim 3 is characterized in that: The loss function is: ; in, , b is the number of samples sampled from the experience pool, is the target network, used as the actual joint value Calculation of is the update rate, r is the reward value at the current moment, θ and are the parameters of the hybrid network that calculates the predicted joint value and the parameters of the target network that calculates the actual joint value, respectively.
5. The urban flood emergency response method based on a multi-dimensional infrastructure relationship network according to claim 1 is characterized in that: Based on the city’s rainfall data, a list of damaged physical components was determined, including: Based on the city’s rainfall data, determine the flood depth of each area at each moment; According to the locations of the physical components in various infrastructure systems and the damaged water depths of the physical components, a damaged physical component list is determined, wherein the damaged physical component list records the time when each physical component is damaged.
6. The urban flood emergency response method based on a multi-dimensional infrastructure relationship network according to claim 1 is characterized in that: The node attributes of the function node include: the node type of the function node, the function status of the function node, the component status of the physical component corresponding to the function node, and the repair time required after the physical component corresponding to the function node is damaged.
7. The urban flood emergency response method based on a multi-dimensional infrastructure relationship network according to claim 3 is characterized in that: In the case where the loss function is not stable, before updating the parameters of the multi-agent reinforcement learning model according to the loss function for a new round of training, the method further includes: If the loss function is not stable, determining whether a preset number of training times has been reached; When the preset number of training times is reached, the training is terminated to obtain a qualified urban flood emergency response model.
