Electric power information physical system elasticity recovery method based on knowledge graph and multi-agent reinforcement learning

Abstract

The invention discloses an electric power information physical system elastic recovery method based on a knowledge graph and multi-agent reinforcement learning, and the method comprises the steps: firstly, describing a power grid topology, a fault causal chain and an evolution path through constructing an electric power knowledge graph comprising entities of equipment, states, actions, rules and the like; then, multiple agents utilize a global semantic state provided by the atlas to make a collaborative decision through a deep reinforcement learning network of a multi-head attention mechanism, and federal asynchronous training is adopted to realize agent strategy sharing updating and privacy protection; and finally, knowledge graph reasoning and multi-agent decision-making form closed-loop interaction, so that the autonomy and interpretability of a fault recovery decision-making are improved, and multi-level collaborative rapid elastic recovery of the electric power information physical system is realized. The elastic recovery method of the force information physical system has good applicability.

Description

Technical Field

[0001] This invention belongs to the field of computer network security technology, and more specifically, relates to a method for elastic recovery of power cyber-physical systems based on knowledge graphs and multi-agent reinforcement learning. Background Technology

[0002] Power cyber-physical systems are a crucial component of modern power grids, and their reliability and resilience directly impact the normal functioning of society. In the event of power grid failures caused by natural disasters, malicious attacks, or sudden accidents, how to quickly restore power supply and minimize the scope and duration of outages is a core issue in power system resilience research.

[0003] Currently, power grid fault recovery methods mainly fall into the following categories: The first is a recovery method based on contingency plans and rules, which generates switching operations and load transfer sequences by combining pre-prepared accident contingency plans, protection settings, and dispatching procedures with topology analysis; the second is a recovery method based on mathematical optimization, which models network reconfiguration, load recovery, and distributed power / energy storage dispatching decisions as constrained optimization problems and obtains recovery schemes by solving integer programming, mixed integer programming, or optimal power flow models; the third is a recovery method based on search and simulation evaluation, which filters and ranks feasible recovery paths in the candidate operation sequence space by combining power flow calculation, state estimation, and heuristic search; and the fourth is a data-driven intelligent recovery method, which uses machine learning / deep learning or (deep) reinforcement learning to evaluate the system state and learn recovery strategies to achieve assisted or automated decision-making.

[0004] However, all of the above methods have some drawbacks that cannot be ignored: First, existing recovery methods based on contingency plans and rules mostly use numerical measurements or local rules as inputs, lacking unified semantic modeling of the dependencies and temporal causal chains between power grid topology, equipment status, fault events, control actions and operating rules, making it difficult to identify the root cause of the fault and form a reusable reasoning basis. Second, existing recovery methods based on mathematical optimization solutions face problems such as high state dimension, explosion of decision variable combinations, and complex constraint coupling in large-scale power grid scenarios, resulting in large computational overhead and making it difficult to meet the needs of online rapid decision-making. Third, existing recovery methods based on search and simulation evaluation usually adopt a centralized solution or centralized scheduling decision mode, which requires cross-regional aggregation of operational data and global information to complete unified calculation. Therefore, there are high data aggregation costs, strong communication dependence and potential privacy leakage risks. Fourth, existing data-driven intelligent recovery methods are prone to problems such as low sample efficiency, slow training convergence, and insufficient generalization ability when directly using high-dimensional original states, which limits the stability and reliability of the recovery strategy. Summary of the Invention

[0005] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a resilient recovery method for power cyber-physical systems based on knowledge graphs and multi-agent reinforcement learning. Its purpose is to solve the following problems: Existing recovery methods based on pre-plans and rules often rely on numerical measurements or local rules as input, lacking unified semantic modeling of the dependencies and temporal causal chains between power grid topology, equipment states, fault events, control actions, and operating rules. This makes it difficult to identify the root causes of faults and form reusable reasoning bases. Furthermore, existing recovery methods based on mathematical optimization solutions face challenges in large-scale power grid scenarios, including high state dimensionality, an explosion of decision variable combinations, and complex constraint coupling, resulting in high computational costs and difficulty in meeting the needs of rapid online decision-making. Existing recovery methods based on search and simulation evaluation typically employ centralized solution or centralized scheduling decision-making modes, requiring cross-regional aggregation of operational data and global information for unified computation, leading to high data aggregation costs, strong communication dependencies, and potential privacy risks. Finally, existing data-driven intelligent recovery methods, when directly using high-dimensional original states, are prone to low sample efficiency, slow training convergence, and insufficient generalization ability, limiting the stability and reliability of the recovery strategy.

[0006] To achieve the above objectives, according to one aspect of the present invention, a method for resilient recovery of a power cyber-physical system based on knowledge graphs and multi-agent reinforcement learning is provided, comprising the following steps: (1) Obtain the entity set V and relation set R from the power information physical system, and the two together form a graph structure semantic model; (2) Input the graph structure semantic model obtained in step (1) into the pre-established knowledge graph model to obtain the power knowledge graph model G; (3) Use a graph neural network to perform online incremental update processing on the power knowledge graph G obtained in step (2) to obtain the set of state vectors of the agents in the next step of multi-agent reinforcement learning. And i∈[1,N], where N represents the number of agents; (4) Take the set of state vectors corresponding to the i-th agent obtained in step (3). Input a pre-trained multi-agent reinforcement learning model to obtain the first global parameters corresponding to the i-th agent. Second global parameter ; (5) Use the first global parameters corresponding to the i-th agent obtained in step (4) Second global parameter Generate the optimal action and use the optimal action to achieve the elastic recovery of the power cyber-physical system; Specifically, this step involves first using the first global parameters corresponding to the i-th agent obtained in step (4). Generate the local recovery control action of the i-th agent at time t. Then, based on the second global parameter and local recovery control actions Get the optimal action And based on the optimal action Achieving elastic recovery of the power cyber-physical system; among which Representing state First global parameter The corresponding strategy distribution Indicates selection Biggest movement The optimal action is a*.

[0007] Preferably, the entity set V includes the device entity set. , , This represents the total number of equipment entities, which include generators, transformers, circuit breakers, and line / busbar nodes. State entity set ={ , , This represents the total number of state entities, which include equipment operating conditions, load levels, and voltage frequencies. Action entity set ={ , This represents the total number of action entities, which include switch opening and closing, load transfer, and microgrid switching. Fault Entity Set Faulty entities include line faults, transformer overloads, communication link congestion, and abnormal measurement and control data; rule entity set The rule entities include protection configurations, operating procedures, and security constraints; Any two entities in the entity set are connected by relations, and all relations constitute the relation set R, including the topological connections between device entities. The relationship between state entities and device entities Causal relationships between faulty entities and state entities Mitigation relationship between action entities and faulty / state entities Constraint relationships between action entities and rule entities And the chronological relationship that depicts the sequence of events. .

[0008] Preferably, the knowledge graph model is constructed through the following steps: (2-1) Obtain a raw dataset D consisting of multiple heterogeneous data from the power information physical system. Perform field cleaning, noise reduction, and format unification processing on the raw dataset D to obtain a structured dataset D. s ; (2-2) The structured dataset D obtained in step (2-1) s Entity extraction and entity normalization are performed sequentially to generate a candidate entity set D. c Each entity in the candidate entity set is assigned a unique identifier and attribute dictionary to obtain a processed candidate entity set, which is then combined and categorized into a device entity set. State entity set Action entity set Fault Entity Set and rule entity set ; (2-3) For the entity candidate set D obtained in step (2-2) c Relation extraction and normalization are performed sequentially to generate a set of triples. h and t represent the head entity and tail entity respectively, and r represents the set of triples. The relationships between any two entities are defined, and all connections constitute a candidate set of relationships. This candidate set is then normalized (i.e., the name, direction, and constraint fields of the relationships are uniformly encoded), and the normalized relationships are categorized as topological connections between device entities. The relationship between state entities and device entities Causal relationships between faulty entities and state entities Mitigation relationship between action entities and faulty / state entities Constraint relationships between action entities and rule entities And the chronological relationship that depicts the sequence of events. ; (2-4) The set of triples obtained in step (2-3) Structural problem detection and correction were performed sequentially to obtain a corrected set of candidate triples. and the revised set of candidate triples Perform consistency verification and persistence processing to obtain a set of triples with enhanced consistency. ; (2-5) The set of triples obtained in step (2-4) Each triplet in Perform instantiation to obtain the instantiated triples; (2-6) Input the nodes and edges in all the instantiated triples obtained in step (2-5) into the pre-established graph database to obtain the initial graph G0, and construct the index structure I based on the initial graph G0; (2-7) Create an electric power knowledge graph model G based on the index structure I obtained in step (2-6).

[0009] Preferably, the heterogeneous data obtained in step (2-1) includes power grid topology and equipment ledger data, protection setting and operation procedure data, historical fault / alarm records, typical dispatch operation records, and power flow / state estimation sample data; Entity extraction in step (2-2) refers to the extraction of entities from the structured dataset D. s The system identifies entity records such as devices, status, actions, faults, and rules. Entity normalization refers to performing naming unification, synonym merging, and disambiguation alignment on the extracted entities, and assigning a unique identifier and attribute dictionary to each entity. The structural problem detection process in steps (2-4) includes detecting erroneous links, redundant links, incorrect relationship directions, broken time-series links, and missing attribute fields. The consistency verification process includes entity deduplication and alignment, elimination of conflicting relationships, completion of missing attributes, and validity checks of rule constraints. The solidification process includes encoding the operating procedures, security constraints, and protection rules into machine-readable rules and associating these machine-readable rules with the corresponding entities or relationships.

[0010] Preferably, step (2-5) involves dividing each triplet... The head entity h and tail entity t are instantiated as nodes in the graph storage, and the relation r is instantiated as a relation edge e in the graph storage. A unique identifier id, entity type tyep, and attribute dictionary attr are created for each node, and a relation type pred, weight w, and timestamp are created for each edge. and constraint field cond; The process of building the index structure in step (2-6) specifically involves first constructing entity type indexes based on all the instantiated triples obtained in step (2-5) and using an inverted index method. and region Then, based on the node IDs in all instantiated triples, an edge adjacency index is constructed using the adjacency index construction method. Adjacency index with inbound edge Finally, based on the timestamps contained in all instantiated triples... The relation edge e is used to construct a time-series index using the B+ tree ordered indexing method. All entity type indexes obtained ,area Outbound Adjacency Index Inbound Adjacency Index and time-series index This constitutes index structure I; Step (2-7) specifically involves first binding the initial graph G0 generated in step (2-6) with the index structure I to obtain the graph-index union object G. J Then, the consistency verification method is used to verify the obtained G. J Perform graph-index integrity verification to obtain the consistent graph-index union object G. I Subsequently, the resulting graph index union object G is... I Perform metadata generation processing to obtain the model encapsulation object G. E Including the map version number Ver G Attribute dictionary version number Ver A Rule set version number Ver R The model is generated based on the time T and the data source summary Src. Finally, the resulting model is encapsulated into an object G. E Persistent encapsulation is performed, that is, it is written as a callable knowledge graph instance into a pre-established knowledge graph model library to generate an electric power knowledge graph model G.

[0011] Preferably, the graph neural network used is a graph neural network based on graph attention network; Step (3) includes the following sub-steps: (3-1) Obtain real-time measurement data and alarm information within the online decision-making cycle at time t. Utilize this real-time measurement data and alarm information, and employ a knowledge graph streaming incremental update method to perform online incremental update processing on the obtained power knowledge graph G, so as to obtain the online updated knowledge graph at time t. ; (3-2) Obtain the recovery task from the power information physical system, and use the k-hop neighborhood subgraph extraction method to process the knowledge graph obtained in step (3-1). Subgraph extraction is performed to obtain the corresponding k-order neighborhood subgraph. ,in This represents the set of nodes related to the recovery task at time t. This represents the set of edges related to the recovery task at time t; the node set is constructed using a node feature construction method. Each node in Constructing initial features This includes entity type, running values, failure probability, and action success rate, and the node set. The initial features of all nodes are stacked to form a feature matrix X.t Based on the edge set The adjacency matrix M is constructed using the adjacency matrix generation method. t k-order neighborhood subgraph , characteristic matrix X t and the adjacency matrix M t Together they constitute the input of the graph neural network. X t M t >; (3-3) The graph attention network embedding computation method is used to process the input obtained in step (3-2). X t M t Process the data to obtain a set of node representation vectors. ; (3-4) Use graph-level pooling to process the set of node representation vectors obtained in step (3-3). Perform graph-level aggregation to obtain the global embedding vector. For the i-th agent, a feature concatenation and fusion method is used to embed the global embedding vector. With the agent's local observation vector Concatenate the vectors along the feature dimension to obtain the set of state vectors corresponding to the i-th agent. ,in This represents the graph-level convergence operator.

[0012] Preferably, the multi-agent reinforcement learning model adopts a federated multi-agent Actor-Critic reinforcement learning model based on multi-head attention, which is trained through the following steps: (4-1) Set the counter k=1, i=1; (4-2) Initialize the experience playback buffer D to be empty; (4-3) Initialize the first global parameter for the k-th iteration Second global parameter They are respectively equal to and And use it as the first local initial parameter of the i-th agent in the k-th iteration. and the second local initial parameters ,in This represents the initial parameter set of the Actor network in a multi-agent reinforcement learning model. This represents the initial parameter set of the Critic network in a multi-agent reinforcement learning model; (4-4) Determine whether the counter k is greater than or equal to the preset maximum aggregation round. If yes, proceed to step (4-19); otherwise, proceed to step (4-5). (4-5) Determine if there is and If the condition is met, proceed to step (4-19); otherwise, proceed to step (4-6). Indicates a preset threshold; (4-6) Determine if i is greater than N. If yes, proceed to step (4-14); otherwise, proceed to step (4-7). (4-7) Obtain the instantaneous reward for the i-th agent at time t. ,in The current load recovery ratio, its value ranges from 0 to 1, preferably 0.6. The penalty is a time or step count penalty, ranging from 0 to 1, preferably 0.2. The number of safety constraint violations at time t, with a value ranging from 1 to 50, preferably 20. These are the weight parameters, and the sum of the three is 1; (4-8) Set the counter j=1; (4-9) Determine whether j is greater than or equal to the preset maximum number of local updates. If yes, proceed to step (4-13); otherwise, proceed to step (4-10). (4-10) The instantaneous reward of the i-th agent at time t obtained according to step (4-7). Obtain the Critic loss function for the i-th agent. The gradient descent method was used to evaluate the loss function of the Critic. Processing is performed to obtain the Critic network parameter set for the i-th agent in the k-th iteration. ; (4-11) The set of Critic network parameters for the i-th agent in the k-th iteration obtained from step (4-10). Obtain the Actor objective function for the i-th agent. The gradient descent method is used to evaluate the parameters of the Actor objective function. Processing is performed to obtain the Actor network parameter set for the i-th agent. ; (4-12) Set counter j = j + 1 and return to step (4-9); (4-13) The number of local initial parameters for the i-th agent in the k-th iteration. Second local initial parameters Upload to the federated server, set the counter i=i+1, and return to step (4-6). (4-14) The number of all first local initial parameters uploaded to the federated server and all second local initial parameters The data is then aggregated to obtain the agent parameter set. And determine the set of parameters of the intelligent agent. If the size m is greater than or equal to the preset threshold, proceed to step (4-16); otherwise, proceed to step (4-15). (4-15) Obtain the current system clock T of the federated server now According to the current system clock T now Get the duration of the aggregation window T=T now -T start It then determines whether the aggregation window duration T is greater than or equal to the preset maximum aggregation time. If so, it proceeds to step (4-16); otherwise, it returns to step (4-14), where T... start This indicates the number of the first local initial parameters in step (4-14). and all second local initial parameters The time for summarizing; (4-16) Use a weighted average method to calculate the number of first local initial parameters obtained in step (4-14). and all second local initial parameters Perform asynchronous aggregation to obtain the first global parameters corresponding to the i-th agent. Second global parameter ; (4-17) The first global parameters corresponding to the i-th agent obtained in step (4-16) Second global parameter Broadcast to the i-th agent, and set the first global parameter of the i-th agent in the k-th iteration. Set the second global parameter of the i-th agent in the k-th iteration. ; (4-18) Set the counter k=k+1 and return to step (4-4). (4-19) Obtain the first global parameter corresponding to the i-th agent. Second global parameter .

[0013] Preferably, the Critic loss function of the i-th agent in step (4-10) is... equal: ; in Let represent the state vector of the i-th agent at time t. Indicates immediate earnings The corresponding joint action vector, This represents the global state vector of the i-th agent at time t+1. This represents the global state vector of the i-th agent at time t. Execute the joint action vector Expected return value, Let be the target reward of the i-th agent at time t. Let be the learning rate of the i-th agent. The Critic network parameters for the i-th agent gradient operator, This indicates random sampling according to the sample distribution in the experience playback buffer D. , , , The corresponding expected value.

[0014] Preferably, the Actor objective function of the i-th agent in step (4-11) is... equal: ; in Indicates the state Actions are performed in the time-experience replay buffer D. Expected value; The first global parameter corresponding to the i-th agent in step (4-16) equal: ; The second global parameter corresponding to the i-th agent equal: ; The weights corresponding to the i-th agent .

[0015] Preferably, step (5) specifically involves first using the first global parameters corresponding to the i-th agent obtained in step (4). Generate the local recovery control action of the i-th agent at time t. Then, based on the second global parameter and local recovery control actions Get the optimal action And based on the optimal action Achieving elastic recovery of the power cyber-physical system; among which Representing state First global parameter The corresponding strategy distribution Indicates selection Biggest movement The optimal action is a*.

[0016] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: (1) Since the present invention adopts steps (1) to (2), it performs standardized semantic modeling of the equipment, status, fault, action and rules of the power information physical system and constructs a power knowledge graph, realizing the structured expression and solidified management of topological dependency relationship, temporal causal relationship and operation constraint rules. Therefore, it can solve the technical problems of existing recovery methods based on contingency plans and rules lacking unified semantic modeling of power grid topology, fault causal chain and temporal evolution, and having difficulty in identifying the root cause of the fault and forming a reusable reasoning basis. (2) Since the present invention adopts steps (3) to (5), it performs subgraph extraction and graph attention embedding calculation based on the online updated knowledge graph to form a low-dimensional and semantically consistent state representation. It also uses the trained policy network to directly generate recovery actions through forward reasoning in the online stage, thereby transforming the complex constraint solution process from "online optimization solution" to "online fast reasoning", significantly reducing online computation overhead and improving response speed. Therefore, it can solve the technical problem that the existing recovery method based on mathematical optimization solution has a large computation overhead and is difficult to meet the online fast decision-making needs in large-scale power grid scenarios due to high state dimension, variable combination explosion and complex constraint coupling. (3) Since the present invention adopts steps (4-13) to (4-17), it uses a federated asynchronous aggregation mechanism to aggregate and update the model parameters of the agents in each region. The federated server only receives and aggregates the parameters without aggregating the original running data. Furthermore, the asynchronous aggregation does not rely on full synchronous upload, thereby reducing the communication dependence and data centralization risk of cross-regional centralized data aggregation. Therefore, it can solve the technical problems of existing recovery methods based on search and simulation evaluation, which usually adopt centralized solution or centralized scheduling decision mode, require cross-regional aggregation of running data and global information, resulting in high data aggregation cost, strong communication dependence, and potential privacy leakage risk. (4) This invention, by employing steps (3) and (4), uses a knowledge graph subgraph as a carrier, employs a graph attention network to compress high-dimensional original measurements and structural relationships into low-dimensional semantic state vectors, and combines centralized value evaluation signals for policy optimization within a multi-agent Actor-Critic framework based on multi-head attention. This improves the sample utilization efficiency of policy learning, accelerates training convergence, and enhances generalization stability. Therefore, it can solve the technical problems of existing data-driven intelligent recovery methods, which suffer from low sample efficiency, slow training convergence, and insufficient generalization ability when directly using high-dimensional original states, resulting in limited stability and reliability of the recovery strategy. Attached Figure Description

[0017] Figure 1This is a flowchart of the elastic recovery method for power cyber-physical systems based on knowledge graphs and multi-agent reinforcement learning, as proposed in this invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0019] The basic idea of ​​this invention is to provide a resilient recovery method for power cyber-physical systems based on knowledge graphs and multi-agent reinforcement learning. First, it constructs a power knowledge graph containing entities such as devices, states, actions, and rules to characterize the power grid topology, fault causal chains, and evolution paths. Then, multiple agents utilize the global semantic state provided by the graph to make collaborative decisions through a deep reinforcement learning network with a multi-head attention mechanism, employing federated asynchronous training to achieve shared policy updates and privacy protection. Finally, knowledge graph reasoning and multi-agent decision-making form a closed-loop interaction, improving the autonomy and interpretability of fault recovery decisions and achieving multi-level collaborative rapid resilient recovery of power cyber-physical systems. This invention's resilient recovery method for power cyber-physical systems has good applicability.

[0020] like Figure 1 As shown, this invention provides a method for the resilient recovery of a power cyber-physical system based on knowledge graphs and multi-agent reinforcement learning, comprising the following steps: (1) Obtain the entity set V and relation set R from the power information physical system, and the two together form a graph structure semantic model; Specifically, the entity set V includes the device entity set. , , The total number of equipment entities, including generators, transformers, circuit breakers, and line / busbar nodes, etc.; the set of state entities. ={ , , The total number of state entities, including equipment operating conditions, load levels, and voltage frequencies; the set of action entities. ={ , The total number of action entities, including switch opening and closing, load transfer, and microgrid switching; the set of fault entities. Fault entities include line faults, transformer overloads, communication link congestion, and abnormal measurement and control data; rule entity set The rule entities include protection configurations, operating procedures, and security constraints; any two entities in the entity set are connected by relations, and all relations constitute the relation set R, including the topological connection relations between device entities. The relationship between state entities and device entities Causal relationships between faulty entities and state entities Mitigation relationship between action entities and faulty / state entities Constraint relationships between action entities and rule entities And the chronological relationship that depicts the sequence of events. ; (2) Input the graph structure semantic model obtained in step (1) into the pre-established knowledge graph model to obtain the power knowledge graph model G; Specifically, the knowledge graph model of the present invention is constructed through the following steps: (2-1) Obtain a raw dataset D consisting of multiple heterogeneous data from the power information physical system. Perform field cleaning, noise reduction, and format unification processing on the raw dataset D to obtain a structured dataset D. s ; Specifically, the heterogeneous data acquired in this step includes power grid topology and equipment ledger data, protection settings and operation procedure data, historical fault / alarm records, typical dispatch operation records, power flow / state estimation sample data, etc.

[0021] (2-2) The structured dataset D obtained in step (2-1) s Entity extraction and entity normalization are performed sequentially to generate a candidate entity set D. c Each entity in the candidate entity set is assigned a unique identifier and attribute dictionary to obtain a processed candidate entity set, which is then combined and categorized into a device entity set. State entity set Action entity set Fault Entity Set and rule entity set ; Specifically, entity extraction in this step refers to extracting data from the structured dataset D. s The process involves identifying entity records such as devices, status, actions, faults, and rules. Entity normalization in this step refers to performing naming unification, synonym merging, disambiguation alignment, and assigning a unique identifier and attribute dictionary to each entity.

[0022] (2-3) For the entity candidate set D obtained in step (2-2) c Relation extraction and normalization are performed sequentially to generate a set of triples. h and t represent the head entity and tail entity respectively, and r represents the set of triples. The relationships between any two entities are defined, and all connections constitute a candidate set of relationships. This candidate set is then normalized (i.e., the name, direction, and constraint fields of the relationships are uniformly encoded), and the normalized relationships are categorized as topological connections between device entities. The relationship between state entities and device entities Causal relationships between faulty entities and state entities Mitigation relationship between action entities and faulty / state entities Constraint relationships between action entities and rule entities And the chronological relationship that depicts the sequence of events. ; Specifically, relation extraction in this step refers to identifying and classifying semantic relationships between entities, and relation normalization in this step refers to mapping the extracted relation candidates to a predefined set of relation types, uniformly encoding relation names, directions and constraint fields, and merging synonymous relations. (2-4) The set of triples obtained in step (2-3) Structural problem detection and correction were performed sequentially to obtain a corrected set of candidate triples. and the revised set of candidate triples Perform consistency verification and persistence processing to obtain a set of triples with enhanced consistency. ; Specifically, the structural problem detection process in this step includes detecting erroneous links, redundant links, incorrect relationship directions, broken time-series links, and missing attribute fields; the consistency verification process in this step includes entity deduplication and alignment, elimination of conflicting relationships, completion of missing attributes, and validity checks of rule constraints; the solidification process includes encoding the operating procedures, security constraints, and protection rules into machine-readable rules, and associating these machine-readable rules with the corresponding entities or relationships. (2-5) The set of triples obtained in step (2-4) Each triplet in Perform instantiation to obtain the instantiated triples; Specifically, this step involves processing each triplet. The head entity h and tail entity t are instantiated as nodes in the graph storage, and the relation r is instantiated as a relation edge e in the graph storage. A unique identifier id, entity type tyep, and attribute dictionary attr are created for each node, and a relation type pred, weight w, and timestamp are created for each edge. and constraint field cond; (2-6) Input the nodes and edges in all the instantiated triples obtained in step (2-5) into the pre-established graph database to obtain the initial graph G0, and construct the index structure I based on the initial graph G0; The specific steps of constructing the index structure in this step are as follows: First, based on all the instantiated triples obtained in steps (2-5), and using the inverted index method, construct the entity type indexes respectively. and region Then, based on the node IDs in all instantiated triples, an edge adjacency index is constructed using the adjacency index construction method. Adjacency index with inbound edge Finally, based on the timestamps contained in all instantiated triples... The relation edge e is used to construct a time-series index using the B+ tree ordered indexing method. All entity type indexes obtained ,area Outbound Adjacency Index Inbound Adjacency Index and time-series index This constitutes index structure I; (2-7) Create an electricity knowledge graph model G based on the index structure I obtained in step (2-6); Specifically, this step involves first binding the initial graph G0 generated in step (2-6) with the index structure I to obtain the graph-index union object G. J Then, the consistency verification method is used to verify the obtained G. J Perform graph-index integrity verification to obtain the consistent graph-index union object G. I Subsequently, the resulting graph index union object G is... I Perform metadata generation processing to obtain the model encapsulation object G. E (It includes the atlas version number Ver) G Attribute dictionary version number Ver A Rule set version number Ver R The model is encapsulated into an object G, which includes the generation time T and the data source summary Src. E Persistent encapsulation is performed, that is, it is written as a callable knowledge graph instance into a pre-established knowledge graph model library to generate an electric power knowledge graph model G.

[0023] The advantages of steps (1) and (2) above are that the standardized modeling, normalized construction and baseline solidification of the semantic model and initial graph of the power knowledge graph are completed offline in one go, forming a power knowledge graph system with reusability, scalability and consistency, thereby providing a stable, reliable and traceable knowledge foundation for real-time incremental updates, state embedding calculation and collaborative recovery decision-making in the subsequent online stage.

[0024] (3) Use a graph neural network to perform online incremental update processing on the power knowledge graph G obtained in step (2) to obtain the set of state vectors of the agents in the next step of multi-agent reinforcement learning. And i∈[1,N], where N represents the number of agents; Specifically, the graph neural network in this invention is a graph neural network based on graph attention network; This step includes the following sub-steps: (3-1) Obtain real-time measurement data and alarm information within the online decision-making cycle at time t. Utilize this real-time measurement data and alarm information, and employ a knowledge graph streaming incremental update method to perform online incremental update processing on the obtained power knowledge graph G, so as to obtain the online updated knowledge graph at time t. ; (3-2) Obtain the recovery task from the power information physical system, and use the k-hop neighborhood subgraph extraction method to process the knowledge graph obtained in step (3-1). Subgraph extraction is performed to obtain the corresponding k-order neighborhood subgraph. ,in This represents the set of nodes related to the recovery task at time t. This represents the set of edges related to the recovery task at time t; the node set is constructed using a node feature construction method. Each node in Constructing initial features (Including entity type, running values, failure probability, action success rate, etc.), the node set The initial features of all nodes are stacked to form a feature matrix X. t Based on the edge set The adjacency matrix M is constructed using the adjacency matrix generation method. t k-order neighborhood subgraph , characteristic matrix X t and the adjacency matrix M t Together they constitute the input of the graph neural network. X t M t >; (3-3) The graph attention network embedding computation method is used to process the input obtained in step (3-2). X t M t Process the data to obtain a set of node representation vectors. ; (3-4) Use graph-level pooling to process the set of node representation vectors obtained in step (3-3). Perform graph-level aggregation to obtain the global embedding vector. For the i-th agent, a feature concatenation and fusion method is used to embed the global embedding vector. With the agent's local observation vector Concatenate the vectors along the feature dimension to obtain the set of state vectors corresponding to the i-th agent. (It is used as input to the subsequent multi-agent reinforcement learning policy network), where Represents a graph-level convergence operator; The advantage of steps (3-2) to (3-4) above is that they perform k-hop neighborhood subgraph extraction guided by the recovery task, and perform graph attention embedding and graph-level pooling on the subgraph to obtain global semantic state representation, thereby compressing the structural information, causal information and constraint information related to fault recovery into low-dimensional state vectors, improving the ability to identify key fault elements and the quality of decision input.

[0025] (4) Take the set of state vectors corresponding to the i-th agent obtained in step (3). Input a pre-trained multi-agent reinforcement learning model to obtain the first global parameters corresponding to the i-th agent. Second global parameter ; Specifically, the multi-agent reinforcement learning model in this invention adopts a federated multi-agent Actor-Critic reinforcement learning model based on multi-head attention, which is trained through the following steps: (4-1) Set the counter k=1, i=1; (4-2) Initialize the experience playback buffer D to be empty; (4-3) Initialize the first global parameter for the k-th iteration Second global parameter They are respectively equal to and And use it as the first local initial parameter of the i-th agent in the k-th iteration. and the second local initial parameters ,in This represents the initial parameter set of the Actor network in a multi-agent reinforcement learning model. This represents the initial parameter set of the Critic network in a multi-agent reinforcement learning model; (4-4) Determine whether the counter k is greater than or equal to the preset maximum aggregation round. If yes, proceed to step (4-19); otherwise, proceed to step (4-5). Specifically, the preset maximum number of aggregation rounds ranges from 500 to 5000, with 2000 being the preferred value. (4-5) Determine if there is and If the condition is met, proceed to step (4-19); otherwise, proceed to step (4-6). This represents a preset threshold value, which ranges from 10. -6 Up to 10 -2 Preferably 10 -4 ; (4-6) Determine if i is greater than N. If yes, proceed to step (4-14); otherwise, proceed to step (4-7). (4-7) Obtain the instantaneous reward for the i-th agent at time t. ,in The current load recovery ratio, its value ranges from 0 to 1, preferably 0.6. The penalty is a time or step count penalty, ranging from 0 to 1, preferably 0.2. The number of safety constraint violations at time t, with a value ranging from 1 to 50, preferably 20. These are the weight parameters, and the sum of the three is 1; (4-8) Set the counter j=1; (4-9) Determine whether j is greater than or equal to the preset maximum number of local updates. If yes, proceed to step (4-13); otherwise, proceed to step (4-10). Specifically, the preset maximum number of local updates ranges from 1 to 200, with 50 being the preferred value. (4-10) The instantaneous reward of the i-th agent at time t obtained according to step (4-7). Obtain the Critic loss function for the i-th agent. The gradient descent method was used to evaluate the loss function of the Critic. Processing is performed to obtain the Critic network parameter set for the i-th agent in the k-th iteration. ; Specifically, the Critic loss function for the i-th agent equal: ; in Let represent the state vector of the i-th agent at time t. Indicates immediate earnings The corresponding joint action vector, This represents the global state vector of the i-th agent at time t+1. This represents the global state vector of the i-th agent at time t. Execute the joint action vector Expected return value, Let be the target reward of the i-th agent at time t. Let be the learning rate of the i-th agent, and its value ranges from 10. -5 Up to 10 -3 Preferably 10 -4 , The Critic network parameters for the i-th agent gradient operator, This indicates random sampling according to the sample distribution in the experience playback buffer D. , , , The corresponding expected value; (4-11) The set of Critic network parameters for the i-th agent in the k-th iteration obtained from step (4-10). Obtain the Actor objective function for the i-th agent. The gradient descent method is used to evaluate the parameters of the Actor objective function. Processing is performed to obtain the Actor network parameter set for the i-th agent. ; Specifically, the Actor objective function of the i-th agent equal: ; in Indicates the state Actions are performed in the time-experience replay buffer D. Expected value; The advantage of steps (4-10) to (4-11) above is that it constructs an Actor network with multi-head attention and combines it with a centralized Critic for value evaluation and joint training, enabling each agent to learn cooperative behavior under the constraint of sharing global value signals. This solves the problems of non-stationarity and action conflict in multi-agent reinforcement learning, and improves the policy convergence speed and the reliability of cross-regional cooperative recovery.

[0026] (4-12) Set counter j = j + 1 and return to step (4-9); (4-13) The number of local initial parameters for the i-th agent in the k-th iteration. Second local initial parameters Upload to the federated server, set the counter i=i+1, and return to step (4-6). (4-14) The number of all first local initial parameters uploaded to the federated server and all second local initial parameters The data is then aggregated to obtain the agent parameter set. And determine the set of parameters of the intelligent agent. If the size m is greater than or equal to the preset threshold, proceed to step (4-16); otherwise, proceed to step (4-15). Specifically, the preset threshold value ranges from 2 to N, preferably 0.5N; (4-15) Obtain the current system clock T of the federated server now According to the current system clock T now Get the duration of the aggregation window T=T now -T start It then determines whether the aggregation window duration T is greater than or equal to the preset maximum aggregation time. If so, it proceeds to step (4-16); otherwise, it returns to step (4-14), where T... start This indicates the number of the first local initial parameters in step (4-14). and all second local initial parameters The time for summarizing; Specifically, the preset maximum aggregation time ranges from 1 second to 300 seconds, with 60 seconds being the preferred value; (4-16) Use a weighted average method to calculate the number of first local initial parameters obtained in step (4-14). and all second local initial parameters Perform asynchronous aggregation to obtain the first global parameters corresponding to the i-th agent. Second global parameter ; Specifically, the first global parameter corresponding to the i-th agent equal: ; The second global parameter corresponding to the i-th agent equal: ; The weights corresponding to the i-th agent ; (4-17) The first global parameters corresponding to the i-th agent obtained in step (4-16) Second global parameter Broadcast to the i-th agent, and set the first global parameter of the i-th agent in the k-th iteration. Set the second global parameter of the i-th agent in the k-th iteration. ; The advantages of steps (4-13) to (4-17) above are that the federated server achieves efficient sharing and updating of cross-regional policy parameters without aggregating the original running data, and avoids waiting for all agents to report synchronously through the dual mechanism of preset threshold and maximum aggregation time, thereby improving the timeliness of aggregation. At the same time, the global parameter broadcast after aggregation reduces the risk of inconsistent actions among agents, thereby improving the convergence efficiency of the collaborative recovery strategy.

[0027] (4-18) Set the counter k=k+1 and return to step (4-4). (4-19) Obtain the first global parameter corresponding to the i-th agent. Second global parameter ; (5) Use the first global parameters corresponding to the i-th agent obtained in step (4) Second global parameter Generate the optimal action and use it to achieve the elastic recovery of the power cyber-physical system.

[0028] Specifically, this step involves first using the first global parameters corresponding to the i-th agent obtained in step (4). Generate the local recovery control action of the i-th agent at time t. Then, based on the second global parameter and local recovery control actions Get the optimal action And based on the optimal action Achieving elastic recovery of the power cyber-physical system; among which Representing state First global parameter The corresponding strategy distribution Indicates selection Biggest movement The optimal action is a*.

[0029] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for elastic recovery of a power cyber-physical system based on knowledge graphs and multi-agent reinforcement learning, characterized in that, Includes the following steps: (1) Obtain the entity set V and relation set R from the power information physical system, and the two together form a graph structure semantic model; (2) Input the graph structure semantic model obtained in step (1) into the pre-established knowledge graph model to obtain the power knowledge graph model G; (3) Use a graph neural network to perform online incremental update processing on the power knowledge graph G obtained in step (2) to obtain the set of state vectors of the agents in the next step of multi-agent reinforcement learning. And i∈[1,N], where N represents the number of agents; (4) Take the set of state vectors corresponding to the i-th agent obtained in step (3). Input a pre-trained multi-agent reinforcement learning model to obtain the first global parameters corresponding to the i-th agent. Second global parameter ; (5) Use the first global parameters corresponding to the i-th agent obtained in step (4) Second global parameter Generate the optimal action and use the optimal action to achieve the elastic recovery of the power cyber-physical system; Specifically, this step involves first using the first global parameters corresponding to the i-th agent obtained in step (4). Generate the local recovery control action of the i-th agent at time t. Then, based on the second global parameter and local recovery control actions Get the optimal action And based on the optimal action Achieving elastic recovery of the power cyber-physical system; among which Representing state First global parameter The corresponding strategy distribution Indicates selection Biggest movement The optimal action is a*.

2. The method for elastic recovery of power cyber-physical systems based on knowledge graphs and multi-agent reinforcement learning according to claim 1, characterized in that, The entity set V includes the device entity set. , , This represents the total number of equipment entities, which include generators, transformers, circuit breakers, and line / busbar nodes. State entity set ={ , , This represents the total number of state entities, which include equipment operating conditions, load levels, and voltage frequencies. Action entity set ={ , This represents the total number of action entities, which include switch opening and closing, load transfer, and microgrid switching. Fault Entity Set Faulty entities include line faults, transformer overloads, communication link congestion, and abnormal measurement and control data; rule entity set The rule entities include protection configurations, operating procedures, and security constraints; Any two entities in the entity set are connected by relations, and all relations constitute the relation set R, including the topological connections between device entities. The relationship between state entities and device entities Causal relationships between faulty entities and state entities Mitigation relationship between action entities and faulty / state entities Constraint relationships between action entities and rule entities And the chronological relationship that depicts the sequence of events. .

3. The method for elastic recovery of power cyber-physical systems based on knowledge graphs and multi-agent reinforcement learning according to claim 1 or 2, characterized in that, A knowledge graph model is constructed through the following steps: (2-1) Obtain a raw dataset D consisting of multiple heterogeneous data from the power information physical system. Perform field cleaning, noise reduction, and format unification processing on the raw dataset D to obtain a structured dataset D. s ; (2-2) The structured dataset D obtained in step (2-1) s Entity extraction and entity normalization are performed sequentially to generate a candidate entity set D. c Each entity in the candidate entity set is assigned a unique identifier and attribute dictionary to obtain a processed candidate entity set, which is then combined and categorized into a device entity set. State entity set Action entity set Fault Entity Set and rule entity set ; (2-3) For the entity candidate set D obtained in step (2-2) c Relation extraction and normalization are performed sequentially to generate a set of triples. h and t represent the head entity and tail entity respectively, and r represents the set of triples. The relationships between any two entities are defined, and all connections constitute a candidate set of relationships. This candidate set is then normalized (i.e., the name, direction, and constraint fields of the relationships are uniformly encoded), and the normalized relationships are categorized as topological connections between device entities. The relationship between state entities and device entities Causal relationships between faulty entities and state entities Mitigation relationship between action entities and faulty / state entities Constraint relationships between action entities and rule entities And the chronological relationship that depicts the sequence of events. ; (2-4) The set of triples obtained in step (2-3) Structural problem detection and correction were performed sequentially to obtain a corrected set of candidate triples. and the revised set of candidate triples Perform consistency verification and persistence processing to obtain a set of triples with enhanced consistency. ; (2-5) The set of triples obtained in step (2-4) Each triplet in Perform instantiation to obtain the instantiated triples; (2-6) Input the nodes and edges in all the instantiated triples obtained in step (2-5) into the pre-established graph database to obtain the initial graph G0, and construct the index structure I based on the initial graph G0; (2-7) Create an electric power knowledge graph model G based on the index structure I obtained in step (2-6).

4. The method for elastic recovery of power cyber-physical systems based on knowledge graphs and multi-agent reinforcement learning according to any one of claims 1 to 3, characterized in that, The heterogeneous data obtained in step (2-1) includes power grid topology and equipment ledger data, protection settings and operation procedure data, historical fault / alarm records, typical dispatch operation records, and power flow / state estimation sample data; Entity extraction in step (2-2) refers to the extraction of entities from the structured dataset D. s The system identifies entity records such as devices, status, actions, faults, and rules. Entity normalization refers to performing naming unification, synonym merging, and disambiguation alignment on the extracted entities, and assigning a unique identifier and attribute dictionary to each entity. The structural problem detection process in steps (2-4) includes detecting erroneous links, redundant links, incorrect relationship directions, broken time-series links, and missing attribute fields. The consistency verification process includes entity deduplication and alignment, elimination of conflicting relationships, completion of missing attributes, and validity checks of rule constraints. The solidification process includes encoding the operating procedures, security constraints, and protection rules into machine-readable rules and associating these machine-readable rules with the corresponding entities or relationships.

5. The method for elastic recovery of power cyber-physical systems based on knowledge graphs and multi-agent reinforcement learning according to claim 4, characterized in that, Steps (2-5) involve processing each triplet. The head entity h and tail entity t are instantiated as nodes in the graph storage, and the relation r is instantiated as a relation edge e in the graph storage. A unique identifier id, entity type tyep, and attribute dictionary attr are created for each node, and a relation type pred, weight w, and timestamp are created for each edge. and constraint field cond; The process of building the index structure in step (2-6) specifically involves first constructing entity type indexes based on all the instantiated triples obtained in step (2-5) and using an inverted index method. and region Then, based on the node IDs in all instantiated triples, an edge adjacency index is constructed using the adjacency index construction method. Adjacency index with inbound edge Finally, based on the timestamps contained in all instantiated triples... The relation edge e is used to construct a time-series index using the B+ tree ordered indexing method. All entity type indexes obtained ,area Outbound Adjacency Index Inbound Adjacency Index and time-series index This constitutes index structure I; Step (2-7) specifically involves first binding the initial graph G0 generated in step (2-6) with the index structure I to obtain the graph-index union object G. J Then, the consistency verification method is used to verify the obtained G. J Perform graph-index integrity verification to obtain the consistent graph-index union object G. I Subsequently, the resulting graph index union object G is... I Perform metadata generation processing to obtain the model encapsulation object G. E Including the map version number Ver G Attribute dictionary version number Ver A Rule set version number Ver R The model is generated based on the time T and the data source summary Src. Finally, the resulting model is encapsulated into an object G. E Persistent encapsulation is performed, that is, it is written as a callable knowledge graph instance into a pre-established knowledge graph model library to generate an electric power knowledge graph model G.

6. The method for elastic recovery of power cyber-physical systems based on knowledge graphs and multi-agent reinforcement learning according to claim 5, characterized in that, The graph neural network uses a graph attention network-based graph neural network. Step (3) includes the following sub-steps: (3-1) Obtain real-time measurement data and alarm information within the online decision-making cycle at time t. Utilize this real-time measurement data and alarm information, and employ a knowledge graph streaming incremental update method to perform online incremental update processing on the obtained power knowledge graph G, so as to obtain the online updated knowledge graph at time t. ; (3-2) Obtain the recovery task from the power information physical system, and use the k-hop neighborhood subgraph extraction method to process the knowledge graph obtained in step (3-1). Subgraph extraction is performed to obtain the corresponding k-order neighborhood subgraph. ,in This represents the set of nodes related to the recovery task at time t. This represents the set of edges related to the recovery task at time t; the node set is constructed using a node feature construction method. Each node in Constructing initial features This includes entity type, running values, failure probability, and action success rate, and the node set. The initial features of all nodes are stacked to form a feature matrix X. t Based on the edge set The adjacency matrix M is constructed using the adjacency matrix generation method. t k-order neighborhood subgraph , characteristic matrix X t and the adjacency matrix M t Together they constitute the input of the graph neural network. X t M t >; (3-3) The graph attention network embedding computation method is used to process the input obtained in step (3-2). X t M t Process the data to obtain a set of node representation vectors. ; (3-4) Use graph-level pooling to process the set of node representation vectors obtained in step (3-3). Perform graph-level aggregation to obtain the global embedding vector. For the i-th agent, a feature concatenation and fusion method is used to embed the global embedding vector. With the agent's local observation vector Concatenate the vectors along the feature dimension to obtain the set of state vectors corresponding to the i-th agent. ,in This represents the graph-level convergence operator.

7. The method for elastic recovery of power cyber-physical systems based on knowledge graphs and multi-agent reinforcement learning according to claim 6, characterized in that, The multi-agent reinforcement learning model employs a federated multi-agent Actor-Critic reinforcement learning model based on multi-head attention. This model is trained through the following steps: (4-1) Set the counter k=1, i=1; (4-2) Initialize the experience playback buffer D to be empty; (4-3) Initialize the first global parameter for the k-th iteration Second global parameter They are respectively equal to and And use it as the first local initial parameter of the i-th agent in the k-th iteration. and the second local initial parameters ,in This represents the initial parameter set of the Actor network in a multi-agent reinforcement learning model. This represents the initial parameter set of the Critic network in a multi-agent reinforcement learning model; (4-4) Determine whether the counter k is greater than or equal to the preset maximum aggregation round. If yes, proceed to step (4-19); otherwise, proceed to step (4-5). (4-5) Determine if there is and If the condition is met, proceed to step (4-19); otherwise, proceed to step (4-6). Indicates a preset threshold; (4-6) Determine if i is greater than N. If yes, proceed to step (4-14); otherwise, proceed to step (4-7). (4-7) Obtain the instantaneous reward for the i-th agent at time t. ,in The current load recovery ratio, its value ranges from 0 to 1, preferably 0.

6. The penalty is a time or step count penalty, ranging from 0 to 1, preferably 0.

2. The number of safety constraint violations at time t, with a value ranging from 1 to 50, preferably 20. These are the weight parameters, and the sum of the three is 1; (4-8) Set the counter j=1; (4-9) Determine whether j is greater than or equal to the preset maximum number of local updates. If yes, proceed to step (4-13); otherwise, proceed to step (4-10). (4-10) The instantaneous reward of the i-th agent at time t obtained according to step (4-7). Obtain the Critic loss function for the i-th agent. The gradient descent method was used to evaluate the loss function of the Critic. Processing is performed to obtain the Critic network parameter set for the i-th agent in the k-th iteration. ; (4-11) The set of Critic network parameters for the i-th agent in the k-th iteration obtained from step (4-10). Obtain the Actor objective function for the i-th agent. The gradient descent method is used to evaluate the parameters of the Actor objective function. Processing is performed to obtain the Actor network parameter set for the i-th agent. ; (4-12) Set counter j = j + 1 and return to step (4-9); (4-13) The number of local initial parameters for the i-th agent in the k-th iteration. Second local initial parameters Upload to the federated server, set the counter i=i+1, and return to step (4-6). (4-14) The number of all first local initial parameters uploaded to the federated server and all second local initial parameters The data is then aggregated to obtain the agent parameter set. And determine the set of parameters of the intelligent agent. If the size m is greater than or equal to the preset threshold, proceed to step (4-16); otherwise, proceed to step (4-15). (4-15) Obtain the current system clock T of the federated server now According to the current system clock T now Get the duration of the aggregation window T=T now -T start It then determines whether the aggregation window duration T is greater than or equal to the preset maximum aggregation time. If so, it proceeds to step (4-16); otherwise, it returns to step (4-14), where T... start This indicates the number of the first local initial parameters in step (4-14). and all second local initial parameters The time for summarizing; (4-16) Use a weighted average method to calculate the number of first local initial parameters obtained in step (4-14). and all second local initial parameters Perform asynchronous aggregation to obtain the first global parameters corresponding to the i-th agent. Second global parameter ; (4-17) The first global parameters corresponding to the i-th agent obtained in step (4-16) Second global parameter Broadcast to the i-th agent, and set the first global parameter of the i-th agent in the k-th iteration. Set the second global parameter of the i-th agent in the k-th iteration. ; (4-18) Set the counter k=k+1 and return to step (4-4). (4-19) Obtain the first global parameter corresponding to the i-th agent. Second global parameter .

8. The method for elastic recovery of power cyber-physical systems based on knowledge graphs and multi-agent reinforcement learning according to claim 7, characterized in that, The Critic loss function for the i-th agent in step (4-10) equal: ； in Let represent the state vector of the i-th agent at time t. Indicates immediate earnings The corresponding joint action vector, This represents the global state vector of the i-th agent at time t+1. This represents the global state vector of the i-th agent at time t. Execute the joint action vector Expected return value, Let be the target reward of the i-th agent at time t. Let be the learning rate of the i-th agent. The Critic network parameters for the i-th agent gradient operator, This indicates random sampling according to the sample distribution in the experience playback buffer D. , , , The corresponding expected value.

9. The method for elastic recovery of power cyber-physical systems based on knowledge graphs and multi-agent reinforcement learning according to claim 8, characterized in that, The objective function of the i-th agent in step (4-11) is the Actor objective function. equal: ； in Indicates the state Actions are performed in the time-experience replay buffer D. Expected value; The first global parameter corresponding to the i-th agent in step (4-16) equal: ； The second global parameter corresponding to the i-th agent equal: ； The weights corresponding to the i-th agent .

10. The method for elastic recovery of power cyber-physical systems based on knowledge graphs and multi-agent reinforcement learning according to claim 9, characterized in that, Step (5) specifically involves first using the first global parameters corresponding to the i-th agent obtained in step (4). Generate the local recovery control action of the i-th agent at time t. Then, based on the second global parameter and local recovery control actions Get the optimal action And based on the optimal action Achieving elastic recovery of the power cyber-physical system; among which Representing state First global parameter The corresponding strategy distribution Indicates selection Biggest movement The optimal action is a*.

Images