Pre-trained message passing neural network elements of a scalable neural network for processing data on a power grid

Abstract

Message passing neural networks (MPNNs) do not scale well for large graphs, such as large power networks. Accordingly, embodiments of an MPNN element are disclosed. The MPNN element can compress a node dimension into a latent dimension before performing message passing, and then decompress the latent dimension back into the node dimension after message passing has been performed. This enables global message passing neural networks to scale for large graphs. Additionally, domain information, such as contingency information, and / or multi-head attention mechanisms can be incorporated into the MPNN element. The message passing neural network can be trained for power flow analysis, contingency analysis, or any other type of analysis in the context of prediction and decision tasks for power systems.

Description

Technical Field

[0001] The embodiments described herein generally relate to machine learning, and more specifically to a pre-trained message-passing neural network element that enables a message-passing neural network to be scaled up for large networks such as power grids. Background Technology

[0002] Machine learning (ML) is gaining popularity in data processing for power systems, such as power grids. In particular, deep learning (DL) models are increasingly capable of performing prediction and decision-making tasks for power grids. The architecture of deep learning models is a key factor in this success.

[0003] In addition, certain artificial neural networks (called graph neural networks (GNNs)) have been developed to process data defined in graphs. An example of a graph neural network is described in F. Scarselli et al., “The Graph Neural Network Model”, IEEE Transactions on Neural Networks, 20 (1): 61-80, doi:10.1109 / TNN.2008.2005605, ISSN 1941-0093, PMID19068426. S2CID 206756462, which is hereby incorporated herein by reference as if fully described herein.

[0004] Message-passing neural networks (MPNNs) are a special type of graph neural network. An example of a message-passing neural network is described in J. Gilmer et al.'s "Neural Message Passing for Quantum Chemistry," arXiv:1704.01212 [cs.LG], mid-April 2017, which is hereby incorporated herein by reference as if fully described herein. In message-passing neural networks, information propagates between different parts of the graph neural network according to an aggregation function.

[0005] Forecasting and decision-making tasks for power grids are typically defined on a power grid induced graph. A power grid includes buses (e.g., substations in a transmission network) and branches (e.g., transmission lines in a transmission network). Buses can be represented as nodes in the graph, and branches can be represented as edges. Data processed for the power grid is typically associated with buses that are nodes in the graph (e.g., node voltage, node power balance, etc.) or with branches that are edges in the graph (e.g., power flow, overload, etc.). Summary of the Invention

[0006] Systems, methods, and nontransitory computer-readable media for pre-trained message-passing neural network elements are disclosed, which enable message-passing neural networks to be scaled up for large networks such as power grids.

[0007] In an embodiment, a method includes using at least one hardware processor to: acquire a pre-trained dataset comprising one or more batches, each batch comprising a plurality of node tensors, wherein each of the plurality of node tensors has a node dimension and a feature dimension, the node dimension representing a plurality of nodes in a graph, the feature dimension representing a plurality of features of each of the plurality of nodes, and each of the plurality of node tensors is labeled with one or more targets of each of the plurality of nodes; and pre-train message-passing neural network elements by minimizing a loss function representing the difference between the one or more targets and the output of the message-passing neural network elements, while for each of the plurality of node tensors, for each of the plurality of features, applying an encoder in an encoder-decoder sequence to compress the node tensor from the node dimension to a latent dimension smaller than the node dimension to generate a latent tensor, processing the latent tensor through one or more neural network layers, and applying a decoder in the encoder-decoder sequence to decompress the processed latent tensor from the latent dimension back to the node dimension.

[0008] The method may further include using the at least one hardware processor to perform the following operations for each of the plurality of node tensors: transposing the node tensor from a first shape to a second shape before applying the encoder; and transposing the node tensor from the second shape to the first shape after applying the decoder.

[0009] Obtaining the pre-trained dataset may include generating the pre-trained dataset by using internal iterative data generated during iterative solution of the system of equations to determine one or more objectives for each of the plurality of node tensors. The method may further include using the at least one hardware processor to perform the following operations: obtaining a training dataset generated by using a power flow solver without using internal iterative data to determine one or more objectives for each of the additional plurality of node tensors; and after pre-training the message-passing neural network elements, incorporating at least one instance of the message-passing neural network elements into a message-passing neural network, and training the message-passing neural network having at least one instance of the incorporated message-passing neural network using the training dataset.

[0010] The method may further include using the at least one hardware processor to append an incident tensor to the latent tensor for each of the plurality of node tensors and for each of the plurality of features during the pre-training process, wherein the incident tensor indicates the presence of each of one or more incidents. The incident tensor may be an indicator vector, wherein the one or more incidents are encoded in the indicator vector. The method may further include using the at least one hardware processor to perform the following operations: obtaining incident information indicating which of the one or more incidents applies to the plurality of node tensors; applying a neural network to the incident information to generate the incident tensor for each of the plurality of node tensors, wherein the dimension of the incident tensor is lower than the dimension of the incident information.

[0011] The message passing neural network elements may use a multi-head attention mechanism, which performs the following operations for each of the plurality of node tensors: obtaining a query from the node tensor; obtaining a value and a key from the output of the encoder; and generating an attention tensor based on the query, the value, and the key, wherein the attention tensor is appended to the output of the decoder.

[0012] The message-passing neural network elements may use skip connections, which append a node tensor, including the plurality of features in the feature dimension, to the output of the decoder for each of the plurality of node tensors.

[0013] The graph can represent a power system. The plurality of features in the first feature dimension can include active power and reactive power, wherein both the one or more objectives and the output of the message-passing neural network include one or more electrical parameters. The method may further include using the at least one hardware processor to perform the following operations: after pre-training the message-passing neural network elements, training a message-passing neural network including at least one instance of the message-passing neural network elements using a training dataset; deploying the message-passing neural network to output the one or more electrical parameters for each of the plurality of nodes in the power system; determining the power flow in the electrical lines within the power system based on the one or more electrical parameters of the plurality of nodes; and adjusting the configuration of the power system based on the determined power flow. The method may further include using the at least one hardware processor to use a power flow solver to determine the one or more objectives for each of a plurality of additional node tensors, thereby generating the training dataset. Adjusting the configuration may include controlling the amount of power injected by at least one generator in the power system. The loss function used to train the message-passing neural network can utilize a physics-based algorithm to compute the plurality of features based on the output of the message-passing neural network, and for each node tensor in the training dataset, a loss is obtained based on the difference between the plurality of features computed for the node tensor using the physics-based algorithm and the plurality of features in the node tensor.

[0014] The method may further include using the at least one hardware processor to perform the following operations after the pre-training: copying the message-passing neural network elements to generate a plurality of message-passing neural network elements; incorporating the plurality of message-passing neural network elements into a message-passing neural network; and training the message-passing neural network incorporated with the plurality of message-passing neural network elements using a training dataset. The plurality of message-passing neural network elements may include at least two parallel message-passing neural network elements, wherein the message-passing neural network combines the outputs of the at least two parallel message-passing neural network elements into a single output.

[0015] The loss function can utilize a physics-based algorithm to compute the plurality of features based on the output of the message-passing neural network elements, and for each of the plurality of node tensors, a loss is obtained based on the difference between the plurality of features computed for the node tensor using the physics-based algorithm and the plurality of features in the node tensor.

[0016] It should be understood that any feature in the above methods can be implemented individually or in any combination with any subset of other features. Therefore, even though the appended claims indicate a specific dependency between features, the disclosed embodiments are not limited to those specific dependencies. Rather, any feature described herein can be combined with any other feature described herein, or implemented in any combination of any features without any one or more other features described herein. Furthermore, any methods described above and elsewhere herein can be embodied individually or in any combination in an executable software module of a processor-based system (such as a server) and / or stored in executable instructions on a non-transitory computer-readable medium. Attached Figure Description

[0017] By studying the accompanying drawings, details of both the structure and operation of the invention can be partially gathered. In the drawings, similar reference numerals refer to similar parts, and in the drawings: Figure 1 The illustration shows an example infrastructure in which any of the processes described herein may be implemented according to an embodiment; Figure 2 The illustration shows an example processing system according to an embodiment that can be used to perform any of the processes described herein; Figure 3 The illustration shows an example data flow for managing a target system according to an embodiment; Figure 4 The diagram illustrates a message-passing neural network based on an example; Figure 5 The illustration shows elements of a dense low-dimensional message-passing neural network (MPNN) according to an embodiment; Figure 6 illustrates the training process for a message-passing neural network according to an embodiment; Figure 7 The illustration shows the operation process of a message-passing neural network according to an embodiment; Figure 8 The illustration shows the incorporation of domain information into MPNN elements according to an embodiment; Figure 9 The illustration shows multiple parallel domain-driven MPNN elements according to an embodiment; Figure 10 The illustration depicts multiple parallel domain-driven MPNN elements with a multi-head attention mechanism according to an embodiment; and Figure 11 The illustration depicts a domain-driven message-passing neural network according to an embodiment. Detailed Implementation

[0018] In the embodiments, systems, methods, and non-transitory computer-readable media for pre-trained messaging neural network elements are disclosed, which enable the messaging neural network to scale to large networks such as power grids. After reading this specification, those skilled in the art will understand how the invention can be practiced in various alternative embodiments and applications. However, while various embodiments of the invention will be described herein, it should be understood that these embodiments are presented by way of example and illustration only and not by way of limitation. Therefore, this detailed description of various embodiments should not be construed as limiting the scope or breadth of the invention as set forth in the appended claims.

[0019] 1. Infrastructure Figure 1 The illustration depicts an example infrastructure in which any of the disclosed processes can be implemented according to embodiments. The infrastructure may include a management system 110 (e.g., including one or more servers) that hosts and / or executes one or more of the various processes described herein, and this management system may be implemented in software and / or hardware. Examples of the management system 110 include, but are not limited to, Supervisory Control and Data Acquisition (SCADA) systems, Power Management Systems (PMS), Energy Management Systems (EMS), Distribution Management Systems (DMS), Advanced DMS (ADMS), etc. The management system 110 may include dedicated servers or may instead be implemented in a computing cloud, where the computing resources of one or more servers are dynamically and elastically allocated to multiple tenants based on demand. In either case, the servers may be co-located (e.g., in a single data center) and / or geographically distributed (e.g., across multiple data centers). The management system 110 may also include or communicatively connect to software 112 and / or database 114. Additionally, the management system 110 may be communicatively connected to one or more user systems 130 and / or target systems 140 via one or more networks 120.

[0020] The (multiple) networks 120 may include the Internet, and the management system 110 may communicate with the (multiple) user systems 130 and / or the (multiple) target systems 140 via the Internet using standard transport protocols such as Hypertext Transfer Protocol (HTTP), HTTP Secure Protocol (HTTPS), File Transfer Protocol (FTP), FTP Secure Protocol (FTPS), Secure Shell FTP (SFTP), Extensible Messaging and Presence Protocol (XMPP), Open FieldMessage Bus (OpenFMB), IEEE Smart Energy Specification Application Protocol (IEEE 2030.5), and proprietary protocols. Although the management system 110 is illustrated as being connected to various systems via a single set of networks 120, it should be understood that the management system 110 may be connected to various systems via different sets of one or more networks. For example, the management system 110 may be connected to a subset of user systems 130 and / or target systems 140 via the Internet, but may also be connected to one or more other user systems 130 and / or target systems 140 via an intranet. Furthermore, although only a few user systems 130 and target system 140, one instance of software 112 and one database 114 are illustrated, it should be understood that the infrastructure can include any number of user systems, power systems, software instances and databases.

[0021] The (multiple) user systems 130 may include any type or more of computing devices capable of wired and / or wireless communication, including but not limited to desktop computers, laptop computers, tablet computers, smartphones or other mobile phones, servers, game consoles, televisions, set-top boxes, electronic kiosks, point-of-sale terminals, embedded controllers, programmable logic controllers (PLCs), etc. However, it is generally envisioned that the (multiple) user systems 130 will include personal computers, mobile devices, or workstations through which the operator's agent of the target system 140 can interact with the management system 110. These interactions may include inputting data (e.g., parameters for configuring the processes described herein) and / or receiving data (e.g., outputs of the processes described herein) via a graphical user interface provided by the management system 110 or the system between the management system 110 and the (multiple) user systems 130. A graphical user interface may include screens (e.g., web pages) that include a combination of content and elements such as text, images, videos, animations, references (e.g., hyperlinks), frames, inputs (e.g., text boxes, text areas, check boxes, radio buttons, drop-down menus, buttons, forms, etc.), scripts (e.g., JavaScript), and elements that contain data stored in database 114 or elements derived from that data.

[0022] Target system 140 may include any type of system that monitors, analyzes, and takes action on its data. Typically, target system 140 may include any system that can be represented as a graph. However, in the context of the specific non-limiting examples provided throughout this disclosure, it will be assumed that target system 140 includes or is constituted by an electric system. An electric system may include one or more, typically multiple, electrical resources connected in a network, which may include distribution networks, transmission networks, etc. Electrical resources may include generators, energy storage systems, loads (e.g., rechargeable energy storage systems or other controllable loads, uncontrollable loads, etc.), etc. Generators may include different types of generators, such as thermal power plants (e.g., coal, natural gas, nuclear, geothermal, etc.), hydropower plants, renewable energy power plants (e.g., solar, wind, geothermal, etc.), etc. Each power resource may be subject to complex technical and / or operational constraints, such as minimum uptime, maximum downtime, ramp-up and / or ramp-down rates, modulation or stability (e.g., a generator may not change its production level too often or at too fast a rate), and start-up and / or shutdown ramp-down rates (e.g., when starting up or stopping, a generator or load must follow a specific power curve, which may depend on how long the unit has been offline or online). Furthermore, networks that can span geographical areas can have complex characteristics, such as network topology, equipment parameters, line power flow limitations, and resource response rates.

[0023] The management system 110 can execute software 112, which includes one or more software modules implementing one or more of the disclosed processes. Additionally, the management system 110 may include a database 114, be communicatively coupled to, or otherwise have access to, that database, which stores data input to and / or output from one or more of the disclosed processes. Any suitable database may be used in database 114, including but not limited to MySQL™, Oracle™, IBM™, Microsoft SQL™, Access™, PostgreSQL™, MongoDB™, etc., and includes cloud-based databases, proprietary databases, and unstructured databases.

[0024] 2. Example processing device Figure 2This is a block diagram illustrating an example wired or wireless system 200 that can be used in conjunction with the various embodiments described herein. For example, system 200 can be used as one or more of, or in combination with, the functions, methods, or other processes described herein (e.g., for storing and / or executing software 112), and can represent components of system management system 110, user systems(s) 130, target systems(s) 140, and / or other processing devices described herein. System 200 can be a server or any conventional personal computer, or any other processor-enabled device capable of wired or wireless data communication. Those skilled in the art will appreciate that other computer systems and / or architectures can also be used.

[0025] System 200 preferably includes one or more processors 210. The processors 210 may include a central processing unit (CPU). Additional processors may be provided, such as a graphics processing unit (GPU), an auxiliary processor for managing input / output, an auxiliary processor for performing floating-point mathematical operations, a dedicated microprocessor (e.g., a digital signal processor) with an architecture suitable for rapidly executing signal processing algorithms, a processor subordinate to the main processor (e.g., a back-end processor), an additional microprocessor or controller and / or coprocessor for dual-processor or multi-processor systems. Such auxiliary processors may be discrete processors or integrated with the main processor 210. Examples of processors that may be used with system 200 include, but are not limited to, any processor supplied by Intel Corporation of Santa Clara, California (e.g., Pentium™, Core i7™, Xeon™, etc.), any processor supplied by Advanced Micro Devices, Inc. of Santa Clara, California (AMD), any processor supplied by Apple Inc. of Cupertino (e.g., A-series, M-series, etc.), any processor supplied by Samsung Electronics Ltd. of Seoul, South Korea (e.g., Exynos™), etc.

[0026] Processor 210 is preferably connected to communication bus 205. Communication bus 205 may include a data channel for facilitating information transfer between the storage device of system 200 and other peripheral components. Furthermore, communication bus 205 may provide a set of signals for communicating with processor 210, including a data bus, address bus, and / or control bus (not shown). Communication bus 205 may include any standard or non-standard bus architecture, such as Industry Standard Architecture (ISA), Extended Industry Standard Architecture (EISA), Micro Channel Architecture (MCA), Peripheral Component Interconnect (PCI) local bus, or bus architectures published by the Institute of Electrical and Electronics Engineers (IEEE) including IEEE 488 Universal Interface Bus (GPIB), IEEE 696 / S-100, etc.

[0027] System 200 may include main memory 215. Main memory 215 provides instruction and data storage for programs executed on processor 210, such as one or more processes discussed herein (e.g., embodied in software 112). It should be understood that programs stored in memory and executed by processor 210 can be written and / or compiled in any suitable language, including but not limited to C / C++, Java, JavaScript, Perl, Visual Basic, .NET, etc. Main memory 215 is typically a semiconductor-based memory, such as dynamic random-access memory (DRAM) and / or static random-access memory (SRAM). Other semiconductor-based memory types include, for example, synchronous dynamic random-access memory (SDRAM), RDRAM, ferroelectric random-access memory (FRAM), etc., including read-only memory (ROM).

[0028] System 200 may also include auxiliary storage 220. Auxiliary storage 220 may optionally include internal media 225 and / or removable media 230. Internal media 225 may include, for example, hard disk drives (HDDs), solid-state drives (SSDs), etc. Removable media 230 may include, for example, magnetic tape drives, optical disc drives (CDs), digital versatile optical disc (DVD) drives, flash memory drives, etc. Auxiliary storage 220 is a non-transitory computer-readable medium on which computer-executable code (e.g., software 112) and / or other data is stored. Computer software or data stored on auxiliary storage 220 is read into main memory 215 for execution by processor 210.

[0029] System 200 may include an input / output (I / O) interface 235. I / O interface 235 provides an interface between one or more components of system 200 and one or more input and / or output devices. Example input devices include, but are not limited to, sensors, keyboards, touchscreens or other touch-sensitive devices, cameras, biometric sensing devices, computer mice, trackballs, pen-based pointing devices, etc. Examples of output devices include, but are not limited to, other processing devices, cathode ray tubes (CRTs), plasma displays, light-emitting diode (LED) displays, liquid crystal displays (LCDs), printers, vacuum fluorescent displays (VFDs), surface-conducting electron emission displays (SEDs), field emission displays (FEDs), etc. In some cases, input and output devices may be combined, such as in the case of a touch panel display (e.g., in a smartphone, tablet, or other mobile device).

[0030] System 200 may include a communication interface 240. Communication interface 240 allows software and other data to be transferred between system 200 and external devices, networks, or other external systems 245. For example, data that may include computer software or executable code may be transferred from external system 245 (e.g., a network server) to system 200 and / or from system 200 to external system 245 via communication interface 240. Examples of communication interface 240 include built-in network adapters, network interface cards (NICs), PCMCIA network cards, card bus network adapters, wireless network adapters, Universal Serial Bus (USB) network adapters, modems, wireless data cards, communication ports, infrared interfaces, IEEE 1394 FireWire, and any other device capable of interfaced system 200 with a network (e.g., network(s) 120) or another computing device. The communication interface 240 preferably implements industry-published protocol standards, such as Ethernet IEEE 802 standard, Fibre Channel, Digital Subscriber Line (DSL), Asynchronous Digital Subscriber Line (ADSL), Frame Relay, Asynchronous Transfer Mode (ATM), Integrated Services Digital Network (ISDN), Personal Communication Services (PCS), Transmission Control Protocol / Internet Protocol (TCP / IP), Serial Line Internet Protocol / Point-to-Point Protocol (SLIP / PPP), etc., but it may also implement customized or non-standard interface protocols.

[0031] Data transmitted via communication interface 240 typically takes the form of electrical communication signals 255. These signals 255 can be provided to communication interface 240 via communication channel 250. In embodiments, communication channel 250 can be a wired or wireless network (e.g., network(s) 120), or any other type of communication link. Communication channel 250 carries signals 255 and can be implemented using a variety of wired or wireless communication means, including wires or cables, optical fibers, traditional telephone lines, cellular telephone links, wireless data communication links, radio frequency (“RF”) links, or infrared links, to name just a few.

[0032] Computer-executable code (e.g., a computer program, such as software 112) is stored in main memory 215 and / or secondary memory 220. The computer program may also be received via communication interface 240 and stored in main memory 215 and / or secondary memory 220. When executed, such a computer program enables system 200 to perform one or more of the processes described elsewhere herein.

[0033] In this specification, the term "computer-readable medium" is used to refer to any non-transitory computer-readable storage medium used to provide computer-executable code and / or other data to or within system 200. Examples of such media include main memory 215, secondary memory 220 (including internal memory 225 and / or removable media 230), and any peripheral device communicatively coupled to communication interface 240, such as external system 245 (e.g., a network information server or other network device). These non-transitory computer-readable media are means for providing executable code, programming instructions, software, and / or other data to processor(s) 210.

[0034] System 200 may also include optional wireless communication components that facilitate wireless communication over voice and / or data networks (e.g., in the case of user system 130 as a smartphone or other mobile device). The wireless communication components include antenna system 270, radio system 265, and baseband system 260. In system 200, radio frequency (RF) signals are transmitted and received over the air by antenna system 270 under the control of radio system 265.

[0035] In one embodiment, antenna system 270 may include one or more antennas and one or more multiplexers (not shown) that perform switching functions to provide transmit and receive signal paths to antenna system 270. In the receive path, received RF signals may be coupled from the multiplexer to a low-noise amplifier (not shown), which amplifies the received RF signals and transmits the amplified signals to radio system 265.

[0036] In alternative embodiments, radio system 265 may include one or more radios configured to communicate on various frequencies. In one embodiment, radio system 265 may combine a demodulator (not shown) and a modulator (not shown) in a single integrated circuit (IC). The demodulator and modulator may also be separate components. In the incoming path, the demodulator removes the RF carrier signal, leaving the baseband received signal transmitted from radio system 265 to baseband system 260.

[0037] The baseband system 260 is also communicatively coupled to processor(s) 210, which can access data storage areas 215 and 220. Therefore, data including a computer program can be received from the baseband processor 260 and stored in main memory 210 or secondary memory 220, or executed upon receipt. When executed, such a computer program enables the system 200 to perform one or more of the disclosed processes.

[0038] 3. Example data stream for target system management Figure 3An example data flow between a management system 110, a user system 130, and a target system 140 according to an embodiment is illustrated. The target system 140 may include a monitoring module 310 and a control module 320. The software 112 of the management system 110 may include an analysis and control module 330 and a human-machine interface (HMI) 340. The database 114 of the management system 110 may store a system model 350. It should be understood that communication between the various systems can be performed via networks 120(s). Additionally, communication between a pair of modules can be performed via an application programming interface (API) provided by one of the modules or by other inter-process communication devices.

[0039] Monitoring module 310 can monitor and collect data output by one or more sensors in target system 140 (e.g., sensors in a power system network). Monitoring module 310 can also derive data from the collected data. Monitoring module 310 can transmit or “push” the collected and / or derived data as system telemetry to analysis and control module 330 (e.g., via API of analysis and control module 330). Alternatively, analysis and control module 330 can retrieve or “pull” system telemetry from monitoring module 310 (e.g., via API of monitoring module 310). System telemetry can include measurements at each of one or more nodes (e.g., buses in a power system) or other points within the network of target system 140. System telemetry can be transmitted from monitoring module 310 to analysis and control module 330 in real time, either when data is collected and / or derived or periodically. As used herein, the term “real time” includes events that occur simultaneously, as well as events separated by time by ordinary delays caused by latency in processing, memory access, communication, etc.

[0040] The analysis and control module 330 receives system telemetry from the monitoring module 310 and uses the system telemetry in conjunction with the system model 350 to determine the configuration of the target system 140 (e.g., one or more resources in a power system), and then controls the target system 140 to transition to the determined configuration. Specifically, the analysis and control module 330 can generate control signals that are transmitted to the control module 320 of the target system 140. For example, the control signals can be sent via the API of the control module 320. The control signals can be transmitted from the analysis and control module 330 of the management system 110 to the control module 320 of the target system 140 in real time, periodically (e.g., before a sliding time window), or in response to user operation, upon receiving and analyzing the system telemetry. The analysis and control module 330 can control the target system 140 automatically (e.g., without any user intervention), semi-automatically (e.g., requiring user approval or confirmation), and / or in response to manual user input.

[0041] The analysis and control module 330 can perform one or more types of analysis, such as power flow analysis, load flow analysis, incident analysis and prediction, security analysis and prediction, load capacity analysis and prediction, optimal power flow, Volt-Var optimization and control, dynamic security analysis (DSA) and prediction, transient security analysis (TSA) and prediction, voltage stability analysis and prediction, generation and load reduction, grid reconfiguration, grid capacity planning, outage planning, and reconfiguration of protection system settings. These are examples of grid sensing tasks routinely performed to ensure stable and economical grid operation. Analysis can be performed in response to user actions or automatically, either in real-time or periodically. In some cases, the analysis can be provided to the user via the human-machine interface 340. In other cases, the analysis and control module 330 can initiate control operations automatically (i.e., without user intervention) or semi-automatically (e.g., with user approval or confirmation) based on the analysis. Initiation of control operations may include transmitting control commands to the control module 320 of the target system 140, which can respond by controlling the target system 140 according to the control commands.

[0042] The control module 320 of the target system 140 receives control signals from the analysis and control module 330 and controls one or more components of the target system 140 according to the control signals. In the context of a power system, examples of such control include setting setpoints (e.g., active and / or reactive power of a generator, voltage, etc.), adjusting the power output of a generator, adjusting the charging or discharging of an energy storage system, adjusting the power input of a load, closing or opening switches (e.g., circuit breakers), etc.

[0043] The human-machine interface 340 can generate a graphical user interface (GUI) that is transmitted to the user system 130 and receive input from the GUI via the user system 130. The GUI can provide information about the current state of the target system 140 determined from system telemetry, the predicted state of the target system 140 determined by the analysis and control module 330, the configuration of the target system 140 determined by the analysis and control module 330, control decisions or recommendations for the target system 140 determined by the analysis and control module 330, and a visual representation of the system model 350. Additionally, the GUI can provide inputs that enable the user of the user system 130 to configure the settings of the analysis and control module 330, configure the system model 350, specify or approve controls to be transmitted to the control module 320 of the target system 140, analyze the target system 140, etc.

[0044] System model 350 can be stored in database 114 and accessed by modules such as analysis and control module 330 via any known means (e.g., via database 114's API, direct queries to database 114, etc.). Database 114 can store system models 350 for each target system 140 managed by management system 110. Each system model 350 can model the target system 140 in any suitable manner. For example, system model 350 may include single-line graphs representing network components and their relationships to each other, systems of equations, adjacency matrices, etc. It should be understood that system model 350 can be implemented as a data structure capable of being automatically analyzed by software modules, including analysis and control module 330.

[0045] 4. introduction Traditionally, grid sensing tasks are accomplished using mathematical models of the power grid. For power flow analysis, the mathematical model includes static equations. For transient stability analysis, the mathematical model includes a system of dynamic differential or differential-algebraic equations (DAEs). The mathematical model is then used in the evaluation, simulation, and / or optimization processes that typically represent the core steps in accomplishing the task.

[0046] Machine learning can be used to replace, supplement, or assist traditional processes for performing these grid sensing tasks. The use of machine learning offers potential benefits such as reduced computation time and improved robustness. In particular, machine learning-based or machine learning-assisted methods are generally faster because their inferences are computationally less expensive than equation solving, numerical integration, and optimization, and they can be easily scaled to state-of-the-art parallel hardware such as GPUs. Furthermore, supplementing traditional processes with machine learning can mitigate the numerical instabilities of traditional processes.

[0047] L. Duchesne et al.'s "Recent Developments in Machine Learning for Energy Systems Reliability Management," Proceedings of the IEEE, Vol. 108, No. 9, pp. 1656-1676, September 2020, doi: 10.1109 / JPROC.2020.2988715 provides an overview of applying machine learning to grid sensing tasks, which is hereby incorporated by reference as if fully described herein. Specifically, machine learning can be applied to grid sensing tasks in at least the following use cases: • Simulation and / or optimization: Machine learning models can be trained based on training data generated through offline simulation or optimization to predict the results of the simulation or optimization. Trained machine learning models can be deployed and operated as an alternative to traditional processes used for simulation or optimization.

[0048] • Partial Decision Making: Trained machine learning models can be used to suggest operational decisions or make predictions. These suggestions can be fully implemented, partially implemented, or ignored based on one or more criteria. For example, a suggestion can be used when the confidence level of the machine learning model's suggestion meets (e.g., equal to or greater than) a threshold, and a conventional or other alternative process can be used when the confidence level of the machine learning model's suggestion does not meet (e.g., less than) a threshold.

[0049] • Pre-screening: The trained machine learning model can be used to generate operational decisions or predictions for multiple operational scenarios (e.g., accidents). The output of the machine learning model can be used for zero or more operational scenarios based on one or more criteria. For example, the output of the machine learning model can be used for operational scenarios where the output confidence level meets a threshold, and conventional or other alternative processes can be used for operational scenarios where the output confidence level does not meet the threshold.

[0050] • Warm start: A trained machine learning model can be used to generate a warm start (e.g., an initial solution) for a conventional or alternative process (such as the Newton-Raphson (NR) algorithm). This process begins with the warm start as the initial solution and iteratively processes the system of equations to obtain the final solution. The goal of the machine learning model is to improve the computational efficiency and / or robustness of the combinatorial process relative to the conventional process.

[0051] • Hypothesis Analysis: A trained machine learning model is used to perform hypothesis analysis, which may extend beyond scenarios handled by traditional processes. For example, a machine learning model could be a generative model that proposes additional key scenarios for analysis in an N-1 security assessment.

[0052] • Operator support: The trained machine learning model can propose actions to the human operator, along with an analysis of the impact of the proposed actions and one or more alternative actions. The human operator can then use this information to decide on the action to take.

[0053] The disclosed embodiments can be used to define deep learning architectures suitable for any of the above-described use cases and / or related use cases. The resulting machine learning models can be used for planning and / or operation in any such use case.

[0054] For ease of understanding, the disclosed embodiments will be described primarily within the context of static power flow and contingency analysis. In this context, the objective is to predict a solution to a power flow problem given nodal power balances in terms of active and reactive power, and a list of potential contingencies (e.g., branches exiting operation). The predicted solution may include nodal voltages (e.g., voltage magnitudes and phase angles), from which power flow along branches between nodes can be calculated. However, it should be understood that the disclosed embodiments are not limited to this context. Rather, the disclosed embodiments can be applied to any decision-making or prediction task that can benefit from machine learning.

[0055] Furthermore, while the target system 140 is primarily exemplified herein as an electrical system, such as a power grid (e.g., transmission network, distribution network, microgrid, etc.), it should be understood that the target system 140 can be any system that can be represented as a graph with nodes and edges. For example, the target system 140 may include a network of connecting components. In an electrical system, these components represent resources connected by electrical lines. In a gas pipeline, components may represent equipment (e.g., gas compressors, pumps, storage tanks, etc.) connected via gas pipes. In a computing system, these components may represent computing resources connected by communication paths. In a geographic system, these components may represent locations connected by roads. It should be understood that these are merely a few examples. Any network can be transformed into a graph including components as nodes and connections between components as edges.

[0056] 5. Message passing neural network Figure 4 The illustration shows a typical message-passing neural network (MPNN) 400 according to an example. The input to the message-passing neural network 400 includes node tensors, which have node dimensions and feature dimensions. The node dimension represents a plurality of nodes in the graph, and the feature dimension represents one or more, typically multiple features, of each of the plurality of nodes. In other words, each node in the node dimension can be represented by a vector of one or more features in the feature dimension. During training, a training dataset comprising the multiple node tensors can be fed into the message-passing neural network 400 in one or more batches. In the case of supervised learning, each of the multiple node tensors is labeled with one or more targets for each of the plurality of nodes.

[0057] The input to the message-passing neural network 400 may also include an adjacency matrix, which represents the adjacency relationships of multiple nodes in the graph. The adjacency matrix provides the message-passing neural network 400 with an awareness of the network topology. The adjacency matrix may include a value for each possible pair of the multiple nodes. A value of zero indicates that there is no edge between the corresponding pair of nodes, while values ​​other than zero (e.g., a value of one in an unweighted adjacency matrix) indicate that there is an edge between the corresponding pair of nodes. In embodiments, the adjacency matrix is ​​weighted. For example, in the context of power flow analysis, the adjacency matrix may be weighted such that each edge is represented by the admittance or impedance value of the electrical line between the corresponding pair of nodes.

[0058] Within the Message Passing Neural Network 400, node tensors are processed through one or more sequences of node processing (i.e., node-level) and message passing (i.e., edge-level). Node processes handle the data for each node without sharing data with other nodes. Typically, the same node processing is applied regardless of the node. The default approach to node processing is a fully connected feedforward network along the last dimension of the node tensor. For example, if the input tensor... x have N × n 节点 × n 输入 The shape, in which, N It's the batch size. n 节点 It is the number of nodes (i.e., the node dimension), and n 输入 If the number of input features (i.e., feature dimension) is a given value, then a fully connected network (FCN) will produce a network with the shape of... N × n 节点 × n 输出 Output Tensor y ,in, n 输出 This refers to the number of output features. Since there is no information shared across nodes or batches of samples, the tensor for each node... i and each node j , ,in, θ This is the weight vector of a fully connected network. Fully connected networks typically use an input tensor and weights... θ A parameterized sequence of matrix multiplications is then performed, followed by a nonlinear mapping (such as a rectified linear unit (ReLU) activation function), where the final operation is typically matrix multiplication. Other known elements, such as pooling and normalization operations, can also be used in node processing.

[0059] Information between different parts of the neural network is propagated via message passing by iteratively updating the representation of each node based on an aggregation function that aggregates information from the node and its neighbors. A typical aggregation function could utilize mean aggregation. During message passing, the data of each node in the hidden layer is shared with any of its neighbors. Each node updates its data in the hidden layer based on an aggregation function that can aggregate data shared by neighboring nodes and the node's own data. Although only a message passing sequence is illustrated, it should be understood that the message passing neural network 400 can include any number of message passing sequences.

[0060] The complexity of information propagation in a naive fully connected network expands quadratically with the number of nodes. Furthermore, it is known that artificial neural networks learn poorly when the dimension of the tensor is too high. Therefore, the disclosed embodiments can influence message passing within a message-passing neural network 400 to overcome these challenges, thereby enabling an expansion of the number of nodes.

[0061] The message-passing neural network 400 is particularly well-suited for power flow prediction in electrical networks of power systems. In this context, the input features in the node tensor can include the active and reactive power of each of the plurality of nodes in the graph, and the target and output features of the message-passing neural network 400 can include electrical parameters of each of the plurality of nodes in the graph, such as voltage magnitude and voltage phase angle. For example, the input and target / output features of the message-passing neural network 400 could be: Input features

[0062] Target / Output Features

[0063] In the above input features, the node type can be one of a PV node, a PQ node, or a slack node, and this node type can be encoded in the indication vector. A PV node is a bus with a generator that regulates the terminal voltage to the voltage setpoint. Therefore, a PV node will have a PV reference voltage amplitude equal to the voltage setpoint ( V 幅值 A PQ node is a bus without voltage control equipment (e.g., no generator) and not remotely controlled by voltage control equipment. A slack node is a bus in the power island with a fixed voltage amplitude and a fixed voltage phase angle. Therefore, a slack node will have a slack reference voltage amplitude equal to the fixed voltage amplitude ( V 幅值 ) and equal to the balanced reference voltage phase angle of the fixed voltage phase angle ( V 相位角Although not shown, in embodiments utilizing incident information as described elsewhere herein, the input features may also include the number of adjacent sides affected by the incident, and the total absolute bus admittance lost due to adjacent incidents in the context of power flow or incident analysis.

[0064] In the above output, the output voltage phase angle ( V 相位角 The sine and cosine values ​​of the output voltage, not the phase angle of the output voltage. V 相位角 This makes it possible to capture the voltage phase angle ( ) itself. V 相位角 Quadrant behavior of ).

[0065] It should be understood that the above input and output characteristics are merely examples applicable to power flow or incident analysis. In other contexts, the input and output characteristics will differ. For example, other possible output characteristics in the context of a power system include, but are not limited to, whether the generator phase angle remains stable or loses synchronization after an incident, and key performance indicators (KPIs) (e.g., grid frequency KPIs, such as the rate of change of frequency (ROCOF), frequency minimums, etc.).

[0066] 6. Dense low-dimensional message passing Naive message passing, typically based on a fixed graph and propagating information from any node in the graph to any other node, has quadratic complexity and therefore does not scale well for a large number of nodes (e.g., ten thousand or more). Empirical studies have shown that naive message passing, using sparse message passing in a typical power grid graph, is detrimental to power flow prediction. These empirical observations are supported by theoretical considerations. In particular, the physical characteristics of power flows mean that local changes (e.g., an increase in generation at a bus) have an impact on the entire power grid. However, naive sparse message passing only has local effects and therefore requires multiple iterations to propagate throughout the entire power grid. Therefore, in this embodiment, dense low-dimensional message passing is incorporated into a message passing neural network. This allows for a greater number of propagation iterations compared to naive message passing.

[0067] Figure 5 The illustration shows a dense low-dimensional MPNN element 500 according to an embodiment. It should be understood that the MPNN element 500 can be a component within a larger message-passing neural network. For example, the MPNN element 500 can replace one or more sequences, or possibly all sequences, of node processing and message passing sequences in the message-passing neural network 400.

[0068] The input to MPNN element 500 is an input node tensor 510 with both node and message-passing (MP) dimensions. It is assumed that the message-passing dimension is typically the feature dimension. However, it should be understood that the input node tensor 510 may have passed through one or more layers of the global message-passing neural network before reaching MPNN element 500. Therefore, the message-passing dimension can be a hyperparameter different from the original feature dimension (e.g., in the range of sixteen to sixty-four). Alternatively, the input node tensor 510 can be the input to the global message-passing neural network, in which case the message-passing dimension would be the feature dimension.

[0069] The input node tensor 510 can be reshaped into a reshaped node tensor 520. In an embodiment, this reshaping includes transposition (i.e., x T The input node tensor is 510. This transpose changes the dimension of the node tensor from the node... Dimension × Message Passing Dimension Flip to Message passing dimension × Node dimension .

[0070] The reshaped node tensor 520 is input to encoder 532 in encoder-decoder sequence 530. Encoder 532 compresses the node dimension of node tensor 520 to the latent dimension to produce initial latent tensor 534A. Therefore, latent tensor 534 can have Message passing dimension × Potential dimension The shape of the node. It should be understood that the latent dimension is smaller than the node dimension. Compression from the node dimension to the latent dimension can filter out information irrelevant to message passing.

[0071] The encoder 532 can be a separate neural network from the global message-passing neural network. Essentially, the encoder 532 can be a mapping that maps each node's dimension vector in the message-passing dimension to a latent dimension vector. In this case, the encoder 532 can be a linear orthogonal mapping. Orthogonality can be achieved during training by penalizing non-orthogonality.

[0072] The use of encoder 532 ensures that the number of trainable weights scales with the number of nodes. Specifically, the number of trainable weights in encoder 532 can be approximately [number missing]. Node dimension × Potential dimension .

[0073] The latent tensor 534 can be processed through one or more, typically multiple neural network layers (but in the latent dimension), according to the sequence of node processing and message passing described elsewhere in this document. As described elsewhere in this document, the node processing can be fully connected. In each sequence, the latent tensor 534 is updated such that, through one or more sequences, the initial latent tensor 534A will produce a final latent tensor 534N. In embodiments, each sequence can be identical via weight sharing (e.g., using the same weight vector). The fully connected network within the encoder-decoder sequence 530 can be implemented using sequences of matrix multiplications, nonlinear mappings (e.g., ReLU activation functions), and / or other suitable deep learning elements.

[0074] The final latent tensor 534N is input to decoder 536 in encoder-decoder sequence 530. Decoder 536 decompresses the latent dimensions back into node dimensions. Therefore, the output of decoder 536 will be node tensor 540, which has lose Output dimension × Node dimension The shape of the decoder 536. The decoder 536 can be the inversion or transpose of the encoder 532. Therefore, the decoder 536 can also be a neural network, potentially including linear orthogonal mappings.

[0075] Alternatively or additionally, decoder 536 may transform the input features(s) in node tensor 520 into output features(s) in node tensor 540. In this case, the output dimension may differ from the message passing dimension. However, it should be understood that when the number of input features and output features is the same, the message passing dimension and the output dimension will be the same. In an alternative embodiment, decoder 536 may not perform the transformation on the features, such that node tensor 540 includes the processed values ​​of the input features(s) in the message passing dimension.

[0076] The encoder-decoder sequence 530 can be applied to each of the multiple input features in the reshaped node tensor 520. For example, sequences of node processing and messaging(s) can be applied to individual vectors having a messaging dimension in the reshaped node tensor 520. Each vector can have Message passing dimension × The shape of 1. In other words, message passing for each feature in the node tensor is performed within the encoder-decoder sequence 530.

[0077] The node tensor 540 output by decoder 536 (which is also the output of encoder-decoder sequence 530) can be reshaped into an output node tensor 550. This reshaping can be the inversion of the reshaping performed on the input node tensor 510. For example, the reshaping can include the transpose of node tensor 540. In any case, the output node tensor 550 has Node Dimension × Message Transmission DimensionalThe shape of the output node tensor 550 can be fed to one or more other neural network layers of the global message-passing neural network to produce the final output of the global message-passing neural network. Alternatively, the output node tensor 550 can be the final output of the global message-passing neural network.

[0078] It is worth noting that the latent dimension can be much smaller than the node dimension. Therefore, the computational cost of message passing within the encoder-decoder sequence 530 is significantly lower than that of naive message passing. In particular, by decompressing the node dimension, the MPNN element 500 can scale to any number of nodes. In other words, the dense, low-dimensional MPNN element 500 enables the message-passing neural network to scale to graphs of any size.

[0079] In an embodiment, the input features represented by node tensor 520 and / or the output features represented by node tensor 540 can have values ​​as complex numbers. However, artificial neural networks (including message-passing neural networks) typically operate using only real numbers. Therefore, in an embodiment, to represent both the real and imaginary components of each complex number, the node dimensions in at least node tensors 520 and 540 can be reshaped to twice the number of nodes (i.e., one real value and one imaginary value representing each feature value for each node). In other words, each feature vector in the message-passing dimension and the output dimension can include both the real and imaginary components of each feature value. In this case, the encoder-decoder sequence 530 operates on these feature vectors to compress the node dimension to twice the number of nodes into a latent dimension.

[0080] 7. train Figure 6 illustrates a training process 600 according to an embodiment, which trains a message-passing neural network during a training phase. The training process 600 can be implemented by software 112. Although the training process 600 is illustrated by a certain arrangement and order of subprocesses, it can be implemented with fewer, more, or different subprocesses, and with different arrangements and / or orders of subprocesses. Furthermore, it should be understood that any subprocess that does not depend on the completion of another subprocess can be executed before, after, or in parallel with that other independent subprocess, even if these subprocesses are described or illustrated in a particular order.

[0081] In subprocess 610, the pre-training dataset is obtained, and in subprocess 620, the training dataset is obtained. The pre-training and training datasets can be obtained in the same way. For example, the pre-training dataset can be extracted from the training dataset. Alternatively, the pre-training and training datasets can be obtained in different ways, as discussed in more detail below.

[0082] Each of the pre-training dataset and the training dataset may include multiple records. Each of these multiple records may include one or more input features labeled with one or more targets. Specifically, each of these multiple records may include a node tensor that includes one or more input features of each of the multiple nodes in the graph. Additionally, for each of these multiple records, an adjacency matrix may be generated. The adjacency matrix represents multiple edges between the multiple nodes in the node tensor. In the context of a power system, the graph represents the power system, each of the multiple nodes represents a bus in the power system, and each of the multiple edges represents an electrical connection between a pair of buses. In the context of power flow analysis, the one or more features may include node active power demand, node reactive power demand, node type, voltage amplitude setpoint, voltage phase angle setpoint, etc. The one or more targets may include voltage amplitude, voltage phase angle, etc.

[0083] In an embodiment, a pre-training dataset is obtained in subprocess 610 by generating (e.g., via simulation) and / or collecting (e.g., from historical data) a relevant dataset of the target system 140. The relevant dataset may include multiple data samples, each representing various operating conditions for each of the multiple nodes in the target system 140. For example, in the context of power flow or incident analysis, each data sample may include the power balance (e.g., active and reactive power) at each node and represent any incidents or other changes in operating conditions (e.g., affecting the admittance matrix).

[0084] Once the relevant dataset has been acquired, each data sample can be solved using traditional processes (such as the Newton-Raphson method or any other method that iteratively solves the system of equations). Iterative data can be collected during this solution process. In the context of power flow or contingency analysis, the Newton-Raphson method iteratively solves linear equations. ,in It is in iteration i The step of the node voltage used in the process Y 母线 It is the admittance matrix of the busbar, and It is iteration iThe node power balance residuals. Therefore, for each iteration, the iterative data can include the node power balance residuals of the iteration, the step of the node voltage used in the iteration, etc. Similar iterative data can be collected from alternative iterative methods. For example, similar iterative data can be collected using a dynamic simulation of the target system 140, which also includes a dedicated information passing mechanism between nodes. Similar to the Newton-Raphson method, information passing in dynamic simulations is typically determined by solving the sparse linear system. It is worth noting that iteration represents the propagation of node information in a specific way, and thus can help the pre-trained MPNN elements 500 propagate node information in a specific way. Regardless of how the iterative data is collected, each data sample can be combined with the associated iterative data to form a labeled node tensor in the pre-training dataset.

[0085] In an embodiment, a training dataset is obtained in subprocess 620 by generating and / or collecting a relevant dataset for the target system 140. This relevant dataset may be substantially the same as the relevant dataset used for the pre-training dataset, but includes different data samples. Alternatively, this relevant dataset may include the same data samples as the relevant dataset used for the pre-training dataset.

[0086] Once the relevant dataset has been acquired, each data sample can be solved using conventional processes (such as the Newton-Raphson method or other iterative or non-iterative methods). For example, in the context of power flow analysis, a power flow solver can be used to determine the (multiple) objectives for each data sample, thereby generating a training dataset. Each data sample can be converted into a node tensor labeled with the (multiple) objectives determined for that data sample to generate a training dataset of multiple labeled node tensors. In this embodiment, the training dataset acquired in subprocess 620 does not incorporate iterative data compared to the pre-training dataset acquired in subprocess 610.

[0087] In sub-procedure 630, the MPNN element 500 is pre-trained using the pre-training dataset acquired in sub-procedure 610. As used herein, the term "pre-training" refers to training the MPNN element 500 before training the message-passing neural network as a whole. The MPNN element 500 can be pre-trained by minimizing a loss function that represents the difference between the target(s) and the output of the MPNN element 500. Specifically, in one or more batches over one or more epochs, for each of the plurality of node tensors in the pre-training dataset, and for each of the plurality of features in the node tensor, an encoder 532 is applied to the node tensor to compress the node tensor from the node dimension to the latent dimension, resulting in a latent tensor. The latent tensor is processed through one or more, typically multiple, neural network layers, and a decoder 536 is applied to the final latent tensor 534N to decompress the final latent tensor 534N from the latent dimension back to the node dimension. The loss function can be used to calculate the loss by comparing the direct or post-processed output of the decoder 536 with the target(s) used to label the node tensor. Then, before processing the next node tensor in the pre-training dataset, one or more parameters (e.g., weights) in the MPNN element 500 can be updated based on the computed loss. Notably, by training the MPNN element 500 using a pre-training dataset that may include iterative data from iterative methods (e.g., the Newton-Raphson method, dynamic simulation, etc.), the MPNN element 500 learns to mimic the way information is propagated iteratively through iterative methods.

[0088] In subprocess 640, one or more instances of the pre-trained MPNN element 500 can be incorporated into a global message-passing neural network as a message-passing mechanism within the network. A single instance of the pre-trained MPNN element 500 can be incorporated, or multiple instances of the pre-trained MPNN element 500 can be incorporated. The pre-trained MPNN element 500 can be replicated to produce multiple instances of it. In embodiments described in more detail elsewhere herein, multiple instances of the MPNN element 500 are incorporated into the message-passing neural network in parallel, such that each MPNN element 500 operates in parallel to produce an output (e.g., node tensor 540 or 550). The outputs of the multiple parallel MPNN elements 500 are then combined into a single output according to an aggregation function.

[0089] In subprocess 650, a message-passing neural network comprising at least one instance of pre-trained MPNN elements 500 is trained using the training dataset acquired in subprocess 620. The message-passing neural network can be trained by minimizing a loss function that represents the difference between the target(s) and the output of the message-passing neural network. Specifically, in one or more batches over one or more epochs, the message-passing neural network is applied to each of the multiple node tensors in the training dataset to produce an output. The loss function can be used to calculate a loss by comparing the direct or post-processed output of the message-passing neural network with the target(s) used to label the node tensor. Then, one or more parameters (e.g., weights) in the message-passing neural network can be updated based on the calculated loss before processing the next node tensor in the training dataset.

[0090] Subprocess 660 may also include evaluating the trained message-passing neural network based on one or more metrics. These metrics may include accuracy, efficiency, etc. When the metrics indicate that the trained message-passing neural network is insufficient (e.g., the metrics do not meet one or more thresholds), the message-passing neural network may be retrained. This evaluation and retraining may occur in one or more iterations until the message-passing neural network is considered sufficient, at which point training process 600 proceeds to subprocess 660.

[0091] In subprocess 660, a trained and fully-fledged messaging neural network is deployed. For example, the messaging neural network can be deployed from a training or development environment to an operational or production environment as deployed messaging neural network 670. As an example, messaging neural network 670 can be deployed with an application programming interface (API) accessible by other services (such as analytics and control module 330) on management system 110 (e.g., in a microservices architecture). Alternatively, messaging neural network 670 can be integrated into analytics and control module 330 or stored in database 114 (e.g., as part of or associated with system model 350).

[0092] The loss function used in subprocess 630 to pre-train the MPNN element 500 may be the same as or different from the loss function used in subprocess 650 to train the message-passing neural network. Typical loss functions may utilize mean squared error (MSE), root mean square error (RMSE), mean absolute error (MAE), etc. However, in embodiments, any or both of these loss functions may be provided by or utilized by a physics-based algorithm. For example, a physics-based algorithm may compute input features(s) based on the output of the MPNN element 500 or the message-passing neural network, and for each node tensor in the corresponding pre-training or training dataset, obtain a loss based on the difference between the input features(s) computed using the physics-based algorithm for the node tensor and the input features(s) in the node tensor. In the context of power flow analysis, a physics-based algorithm may solve power flow equations to compute power balance (e.g., active power and reactive power) based on the voltages (e.g., voltage magnitude and voltage phase angle) output by the MPNN element 500 or the message-passing neural network. Based on experience, the physical information loss function improves the trainability of message-passing neural networks.

[0093] 8. operate Figure 7 The illustration depicts an operation process 700 utilizing a deployed messaging neural network 670 during the operation phase according to an embodiment. The operation process 700 can be implemented by software 112. Although the operation process 700 is illustrated by a certain arrangement and order of subprocesses, it can be implemented with fewer, more, or different subprocesses, and with different arrangements and / or orders of subprocesses. Furthermore, it should be understood that any subprocess independent of the completion of another subprocess can be executed before, after, or in parallel with that other independent subprocess, even if these subprocesses are described or illustrated in a specific order.

[0094] It should be understood that the deployed message-passing neural network 670 is the same as the message-passing neural network trained during training process 600. However, during the operational phase, the loss function will no longer be used to train the message-passing neural network.

[0095] In sub-procedure 710, it is determined whether new data has been received. For example, new data can be received from the monitoring module 310 of the target system 140 in system telemetry. This data can be received continuously (e.g., in real time), periodically, or in response to events (e.g., user actions or other triggers). When new data is received (i.e., "yes" in sub-procedure 710), operation process 700 proceeds to sub-procedure 720. Otherwise, when no new data is received (i.e., "no" in sub-procedure 710), operation process 700 proceeds to sub-procedure 750.

[0096] In subprocess 720, one or more, typically multiple, features are extracted from the data. It should be understood that these are operation values ​​of the same input features, which include multiple records in the pre-training dataset and training dataset, obtained respectively in subprocesses 610 and 620 for each of the multiple nodes.

[0097] In subprocess 730, the features(s) extracted in subprocess 720 are input into a message-passing neural network 670, as deployed in subprocess 660 of training process 600. Specifically, the deployed message-passing neural network 670 processes the features(s) to produce an output. In the context of a power system, the output may include one or more electrical parameters for each of multiple nodes in a graph of the power system. In the context of power flow analysis, the electrical parameters may include voltage magnitude and voltage phase angle.

[0098] In subprocess 740, the output of the deployed message-passing neural network 670 can be provided to one or more downstream functions. A downstream function is any function that can use the output of the deployed message-passing neural network 670 for the target system 140. For example, the analysis and control module 330 can use the output to generate one or more control signals, which are transmitted to the control module 320 to control one or more components of the target system 140, for example, to adjust the operational configuration of the target system 140.

[0099] In the context of power flow analysis, the analysis and control module 330 can determine the power flow in the electrical lines (i.e., edges of the graph) of the power system based on the voltage magnitude and voltage phase angle or other electrical parameters of multiple buses (i.e., nodes of the graph). The analysis and control module 330 can then adjust the configuration of the power system based on the determined power flow. Adjusting the power system configuration can include controlling the amount of power injected by at least one generator in the power system (e.g., at a bus), adjusting the amount of power supplied to the loads in the power system, adjusting the voltage setpoints in the power system (e.g., on voltage control devices) (e.g., to perform voltage regulation, reactive power compensation, etc.), adjusting the loads in the power system (e.g., load shedding), dispatching generation to reduce network congestion, etc.

[0100] In sub-procedure 750, it is determined whether to cancel the deployment of the deployed messaging neural network 670. For example, cancellation can be performed in response to a user action via human-machine interface 340. When it is determined that the deployment of the messaging neural network 670 should be canceled (i.e., "yes" in sub-procedure 750), operation procedure 700 can end. Otherwise, when it is determined that the deployment of the messaging neural network 670 should not be canceled (i.e., "no" in sub-procedure 750), operation procedure 700 can return to sub-procedure 710.

[0101] 9. Domain-driven pre-training In an embodiment, the pre-training of MPNN element 500 in subprocess 630 can be domain-driven. Figure 8 The illustration shows the incorporation of domain information 810 during the pre-training of the encoder-decoder sequence 530 of the MPNN element 500 according to an embodiment. In the primary embodiment, the domain information 810 includes or is composed of incident information. Incident information can modify the network topology. Using incident information during pre-training allows the pre-trained MPNN element 500 to be adapted to various incident scenarios. However, it should be understood that the domain information 810 may include or be composed of other types of information related to or constituting the domain for which the deployed messaging neural network 670 will operate.

[0102] In embodiments where domain information 810 includes incident information, the incident information can be represented as an indication vector. For example, suppose the target system 140 has C One possible unforeseen event to consider. The indicator vector can be of length [missing information]. C A binary vector, where a value of 0 represents a normal state (e.g., normal operation) and a value of 1 represents an unexpected event state (e.g., malfunction or other abnormal operation). In other words, the first value in the indicator vector... c A value of 1 in the position indicates the first c The asset is in a state of contingency. In the context of the power system... C A possible accident can be equal to the number of branches in the network (i.e., edges in the graph), and indicates whether each branch has failed.

[0103] In an embodiment, a neural network 820, which may be a fully connected network, can be applied to domain information 810 (e.g., indicator vectors) in the domain dimension to generate a latent domain tensor 830 in the latent domain dimension. The latent domain tensor 820 may also include or be composed of indicator vectors. In other words, the neural network 820 can compress the domain information 810 from the domain dimension to the latent domain dimension. It should be understood that the latent domain dimension is smaller than the domain dimension. Therefore, the domain information 810 may have 1 × Domain Dimension The shape, and the latent domain tensor 830 can have 1 × Latent Domain Dimension The shape of the latent domain tensor. When the latent domain tensor 830 is used to represent accident information, this latent domain tensor can also be referred to as the "accident tensor" in this paper.

[0104] It should be understood that each node tensor in the pre-training dataset can be associated with corresponding domain information 810. Therefore, when pre-training MPNN elements 500 on specific node tensors from the pre-training dataset, the associated domain information 810 can be processed by the neural network 820 to generate a latent domain tensor 830. This latent domain tensor 830 is then appended to or otherwise incorporated into the initial latent tensor 534A generated by the encoder 532 for the associated node tensors, thereby producing a tensor with a domain information equal to... Latent dimension + Latent domain dimension The total potential dimensions of the potential tensor. Therefore, the compressed domain information 810 (e.g., accident information) is incorporated into the node processing and message passing of the encoder-decoder sequence 530.

[0105] Similarly, during operation 700, each node tensor input to the deployed messaging neural network 670 can be associated with domain information 810. This associated domain information 810 is processed by neural network 820 to produce a latent domain tensor 820, which is incorporated into an initial latent tensor 534 generated by encoder 532 for the associated node tensors. The combined latent tensor is processed and decompressed by decoder 536 to produce an output node tensor that takes into account the associated domain information 810 (e.g., incident information).

[0106] 10. parallel Figure 9 The illustration depicts multiple parallel domain-driven MPNN elements 500 according to an embodiment. In the illustrated embodiment, there are three parallel MPNN elements 500. However, it should be understood that a message-passing neural network (e.g., a deployed message-passing neural network 670) can include any number of MPNN elements 500, including one, two, three, four, five, or more. For simplicity, intermediate node tensors 520 and 540 are omitted.

[0107] In sub-process 640 of training process 600, these parallel MPNN elements 500 can be incorporated into the message-passing neural network. Then, in sub-process 650, the message-passing neural network with the parallel MPNN elements 500 is trained, and in sub-process 660, the message-passing neural network is deployed such that the deployed message-passing neural network 670 includes the parallel MPNN elements 500. In other words, during operation process 700, the input of the deployed message-passing neural network 670 is processed through all the parallel MPNN elements 500. It is worth noting that during training in sub-process 650, the parameters of the parallel MPNN elements 500 can diverge. Furthermore, even if the parameters of each MPNN element 500 are the same, different executions of the same encoder 532 can produce different latent tensors 534, and different executions of the same decoder 536 can produce different node tensors 540. Moreover, due to the randomness element in the MPNN elements 500, different executions of the same MPNN element 500 on the same input can produce different outputs.

[0108] In a domain-driven embodiment, the same domain information 810 can be supplied to each MPNN element 500 via a separate instance of neural network 820, each instance of which generates a latent tensor 830. It should be understood that even if the same neural network 820 is applied to the same domain information 810 for each instance of MPNN element 500, the resulting latent tensor 830 may differ between instances of MPNN element 500. In other words, neural network 820 may not always produce the same output from the same input. In an alternative embodiment, the same latent tensor 830 can be used for each MPNN element 500 (e.g., tiled over the message-passing dimension).

[0109] Given the above, each MPNN element 500 will output a different node tensor 550. Therefore, node tensors 550 can be combined into a single node tensor 950. In this embodiment, node tensors 550 are combined by concatenating them together to form node tensor 950. However, it should be understood that node tensors 550 can be combined in any other suitable manner.

[0110] In this embodiment, a shortcut or "skip" connection can be used. A skip connection can combine the input node tensor 510 with the node tensor 950. For example, node tensor 510 can be appended to node tensor 950 or otherwise concatenated with it. Skip connections can make deep learning models, such as the disclosed message-passing neural network, easier to train.

[0111] The node tensor 950 with spliced ​​node tensor 510 can be processed by the final fully connected network to produce the final output 960. Specifically, the final fully connected network can transform this combined node tensor, representing the output and input node tensors 510 of each MPNN element 500, into a single final set of output features for each node. These output features can be used in the loss function during sub-process 650 of training process 600 and provided as output in sub-process 740 of operation process 700. Using multiple parallel MPNN elements 500 in this way can improve the predictive performance (e.g., accuracy) of the global message-passing neural network. In particular, each MPNN element 500 can explore different (e.g., overlapping or non-overlapping) solution spaces, thereby increasing the overall solution space that can be searched by the deployed message-passing neural network 670.

[0112] 11. Bullish attention Figure 10 The illustration shows multiple parallel domain-driven MPNN elements 500 with a multi-head attention mechanism 1020 according to an embodiment. For simplicity, intermediate node tensors 520 and 540 are omitted. Furthermore, components already described herein will not be redundantly described with respect to this embodiment. It is worth noting that, in addition to the multi-head attention mechanism 1020 and supporting components already added, Figure 10 The embodiments and Figure 9 The embodiments are the same. Therefore, regarding Figure 9 Any description of the embodiments in the example also applies to Figure 10 Examples are shown in the text.

[0113] In this embodiment, each of the plurality of parallel MPNN elements 500 has a corresponding multi-head attention mechanism 1020. Therefore, the number of multi-head attention mechanisms 1020 is equal to the number of MPNN elements 500. An example of an implementation of the multi-head attention mechanism 1020 is described in A. Vaswani et al., “Attention is All You Need,” arXiv: 1706.03762 [cs.CL], mid-December 2017, which is hereby incorporated herein by reference as if fully described herein.

[0114] Each multi-head attention mechanism 1020 may include a dot-product attention layer that receives three inputs: query, key, and value. The multi-head attention mechanism 1020 can be described as mapping a query (Q) and a set of key (K)-value (V) pairs to an output, which is computed as a weighted sum of values ​​(V), where the weight assigned to each value (V) is calculated using a compatibility function between the query (Q) and the corresponding key (K). The query, key, value, and output are all vectors. The query (Q) of each multi-head attention mechanism 1020 can be obtained from the input node tensor 510. The key (K) of each multi-head attention mechanism 1020 can be obtained by concatenating the latent tensor 534 and the latent domain tensor 830. Similarly, the value (V) of each multi-head attention mechanism 1020 can be obtained by concatenating the latent tensor 534 and the latent domain tensor 830. In this way, the MPNN element 500 remains scalable as the number of nodes increases.

[0115] Each multi-head attention mechanism 1020 may include a graph attention network (GAT). The graph attention network uses masked self-attention and includes graph attention layers stacked on top of each other. The input to each graph attention layer is a set of node embeddings that pay attention to the embeddings of neighboring nodes. The output of each graph attention layer is a set of transformed embeddings.

[0116] The output of each multi-head attention mechanism 1020 can be an attention tensor (e.g., generated based on queries, values, and keys), which is combined (e.g., concatenated) into an attention tensor 1030 having the same shape as the input node tensor 510. The attention tensor 1030 can be combined with the node tensor 950 and the input node tensor 510. For example, the node tensor 510 can be appended to or otherwise concatenated with the attention tensor 1030, which can also be appended to or otherwise concatenated with the node tensor 950.

[0117] The node tensor 950, with its concatenated attention tensor 1030 and node tensor 510, can be processed by a final fully connected network to produce the final output 960. Specifically, the final fully connected network can transform this combined node tensor, representing the output of each MPNN element 500, the output of each multi-head attention mechanism 1020, and the input node tensor 510, into a single final set of output features for each node. These output features can be used in the loss function during sub-process 650 of training process 600 and provided as output in sub-process 740 of operation process 700. Empirically, the multi-head attention mechanism 1020 can improve the accuracy of the global message-passing neural network.

[0118] 12. Domain-driven message-passing neural networks Figure 11A domain-driven messaging neural network 1100 according to an embodiment is illustrated. The messaging neural network 1100 is an example of a deployed messaging neural network 670 and utilizes components described elsewhere herein. Therefore, it should be understood that any descriptions of these components elsewhere herein also apply to these components with respect to the messaging neural network 1100 and will therefore not be redundantly included herein.

[0119] As illustrated, domain information 810 is passed through multiple neural networks 820, which may be fully connected networks. Specifically, neural network 820B is applied to the domain information 810 (e.g., an indicator vector) in the domain dimension to generate a latent domain tensor 830A in the latent domain dimension. The latent domain tensor 830A can be combined with the feature vector of each node in the input node tensor 1110 (e.g., appended to or otherwise concatenated with that feature vector). In other words, the latent domain tensor 830A is tiled along the node dimension.

[0120] The resulting tensors (including the input node tensors and the tiled latent domain tensor 830A) can be preprocessed by a fully connected network to produce node tensors 510 in the message-passing dimension. The message-passing dimension can be smaller than the feature dimension. In other words, preprocessing can include at least a small amount of compression of the feature dimension.

[0121] The node tensor 510 is then processed through one or more MPNN elements 500, as discussed elsewhere in this document. Although three parallel MPNN elements 500 are illustrated, it should be understood that the message-passing neural network 1100 may include any number of MPNN elements 500.

[0122] For each MPNN element 500, a neural network 820A can be applied to the domain information 810 in the domain dimension to generate a latent domain tensor 830A in the latent domain dimension. As discussed elsewhere in this document, the latent domain tensor 830 can be incorporated into the corresponding MPNN element 500, for example, by appending the latent domain tensor 830 to the latent tensor 534 or otherwise concatenating it. In the case of multiple parallel MPNN elements 500, the latent domain tensor 830 can be tiled across the multiple parallel MPNN elements 500.

[0123] As discussed elsewhere in this paper, the outputs of multiple parallel MPNN elements 500 can be combined, and then post-processed by a fully connected network to produce an output node tensor 1120 in the output dimension. The output dimension can be equal to the number of output features. In other words, the output node tensor 1120 can include one or more output features from each node in the graph. In the context of power flow analysis, these output features can include voltage magnitude and voltage phase angle.

[0124] 13. Example Application As discussed throughout this disclosure, the disclosed embodiments can be applied to a target system 140 including or composed of a power system. In this context, the deployed message-passing neural network 670 can be used to perform power flow analysis on inputs representing generation and load (e.g., the described input characteristics) and network topology (e.g., as an adjacency matrix) to produce a steady-state power flow, which includes voltage magnitude and phase angle (e.g., the described output characteristics). Incident analysis can be performed in a similar manner by incorporating domain information 810 representing incident information into MPNN element 500. In either case, this steady-state power flow can be used as input to any downstream function that typically uses power flow.

[0125] As an example, the output of the deployed messaging neural network 670 (e.g., including voltage magnitude and phase angle) can be used in a downstream function to calculate the power flow on power lines within the power system network. Based on the calculated power flow on the power lines, it can be determined whether any power line is overloaded (e.g., exceeding its rated limit). This information is beneficial for managing network congestion because operators can reschedule generation in the network to eliminate congestion. For example, if a power line is overloaded (e.g., the power flow exceeds a predefined threshold), the analysis and control module 330 (e.g., automatically or in response to operator input) can activate or increase the power supplied by one or more generators in another part of the network to alleviate congestion on the overloaded power line.

[0126] As another example, the output of the deployed messaging neural network 670 for different contingency scenarios can be used to determine whether the network is safe. In this context, a network can be considered unsafe if the failure of a component would cause significant network congestion or cause the network voltage to shift beyond defined limits. Therefore, one or more contingency scenarios can be supplied to the deployed messaging neural network 670 in domain information 810 (e.g., by switching individual nodes to a contingency state) to determine the impact on network congestion or network voltage when one or more components fail. This can assist operators in system planning and / or system operation.

[0127] As another example, the output of the deployed message-passing neural network 670 can be used to manage reactive power compensation. Specifically, the output can be used to determine how much reactive power compensation must be applied to the load bus to improve the voltage distribution at that load bus. Based on the determined reactive power compensation, the analysis and control module 330 can control (e.g., automatically or in response to operator input) reactive power compensation devices on the network within the power system via the control module 320 to adjust the voltage distribution at the load bus.

[0128] As another example, the output of the deployed message-passing neural network 670 can be used for system planning. For instance, when new equipment such as electrical lines or transformers needs to be installed, each of the old and new network topologies (e.g., as an adjacency matrix) can be fed into the deployed message-passing neural network 670 to determine the power flow in each topology. The differences in power flow between the old and new topologies can then be compared to understand the impact of the new equipment on the power system network. This understanding can inform system planning and investment decisions, as well as measures to reduce power losses in the network.

[0129] It should be understood that the disclosed embodiments can be generalized to implement any other type of analysis, including but not limited to load flow analysis, security analysis and forecasting, load capacity analysis and forecasting, optimal power flow, Volt-Var optimization and control, dynamic security analysis (DSA) and forecasting, transient security analysis (TSA) and forecasting, voltage stability analysis and forecasting, generation and load shedding, grid reconfiguration, grid capacity planning, outage planning, and reconfiguration of protection system settings. Furthermore, the types of analysis implemented can be incorporated into any grid sensing task, including but not limited to simulation, optimization, partial decision-making, pre-screening, hot start, what-if analysis, and operator support.

[0130] In the context of using message-passing neural networks for dynamic simulations, these networks must process time-series data. In this case, appropriate input layers can be added to the message-passing neural network to transform the time-series data into node tensors 510 of (multiple) MPNN elements 500. For each node in the graph, these input layers can transform the time-series data of that node into a feature vector for that node. Such input layers can include, for example, one or more recurrent neural network (RNN) layers, convolutional neural network (CNN) layers, function neural network layers, transformer models, etc.

[0131] 14. Example Implementation The disclosed embodiments can provide numerous advantages over existing technologies. A first objective of the disclosed embodiments is to facilitate learning in message-passing neural networks by achieving scalability for large graphs. A second objective of the disclosed embodiments is to incorporate domain information (such as accident information) into message passing within message-passing neural networks. A third objective of the disclosed embodiments is to improve performance using multi-head attention mechanisms during training. A fourth objective of the disclosed embodiments is to facilitate machine learning for power-related prediction and decision-making tasks.

[0132] To address one or more of these objectives, example embodiments include, but are not limited to: Example 1: A method comprising using at least one hardware processor to: acquire a pre-trained dataset comprising one or more batches, each batch comprising a plurality of node tensors, wherein each of the plurality of node tensors has a node dimension and a feature dimension, the node dimension representing a plurality of nodes in a graph, the feature dimension representing a plurality of features of each of the plurality of nodes, and each of the plurality of node tensors being labeled with one or more targets of each of the plurality of nodes; and pre-training message-passing neural network elements by minimizing a loss function representing the difference between the one or more targets and the output of the message-passing neural network elements, while for each of the plurality of node tensors, for each of the plurality of features, applying an encoder in an encoder-decoder sequence to compress the node tensor from the node dimension to a latent dimension smaller than the node dimension, thereby generating a latent tensor; processing the latent tensor through one or more neural network layers; and applying a decoder in the encoder-decoder sequence to decompress the processed latent tensor from the latent dimension back to the node dimension. This embodiment contributes to the first objective.

[0133] Example 2: The method as described in Example 1 further includes, for each of the plurality of node tensors, using the at least one hardware processor to: transpose the node tensor from a first shape to a second shape before applying the encoder; and transpose the node tensor from the second shape to the first shape after applying the decoder. This example contributes to the first objective.

[0134] Example 3: The method as described in any of the preceding embodiments, wherein obtaining the pre-training dataset includes generating the pre-training dataset by using internal iterative data generated during iterative solution of the system of equations to determine one or more targets for each of the plurality of node tensors. This embodiment contributes to the first objective.

[0135] Example 4: The method as described in Example 3 further includes using the at least one hardware processor to: acquire a training dataset generated by using a power flow solver without using internal iterative data to determine one or more objectives for each of a plurality of additional node tensors; and after pre-training the message-passing neural network elements, incorporate at least one instance of the message-passing neural network elements into a message-passing neural network, and train the message-passing neural network having at least one instance of the incorporated message-passing neural network using the training dataset. This example contributes to the first objective.

[0136] Example 5: The method as described in any of the preceding embodiments further includes using the at least one hardware processor to, during the pre-training period, append an incident tensor to the latent tensor for each of the plurality of node tensors and for each of the plurality of features, wherein the incident tensor indicates the presence of each of one or more incidents. This embodiment contributes to the second objective.

[0137] Example 6: The method as described in Example 5, wherein the incident tensor is an indicator vector, and wherein the one or more incidents are encoded in the indicator vector. This example contributes to the second objective.

[0138] Example 7: The method as described in Example 5 or 6 further includes using the at least one hardware processor to: obtain incident information indicating which of the one or more incidents applies to the plurality of node tensors; apply a neural network to the incident information to generate the incident tensor for each of the plurality of node tensors, wherein the dimension of the incident tensor is lower than the dimension of the incident information. This example contributes to the first and second objectives.

[0139] Example 8: The method as described in any of the preceding embodiments, wherein the message-passing neural network elements employ a multi-head attention mechanism, which performs the following operations for each of the plurality of node tensors: obtaining a query from the node tensor; obtaining a value and a key from the output of the encoder; and generating an attention tensor based on the query, the value, and the key, wherein the attention tensor is appended to the output of the decoder. This embodiment contributes to the third objective.

[0140] Example 9: The method as described in any of the preceding embodiments, wherein the message-passing neural network elements use skip connections, the skip connections appending a node tensor including the plurality of features in the feature dimension to the output of the decoder for each of the plurality of node tensors. This embodiment contributes to the first objective.

[0141] Example 10: The method as described in any of the preceding embodiments, wherein the diagram represents a power system. This embodiment contributes to the fourth objective.

[0142] Example 11: The method as described in Example 10, wherein the plurality of features in the first feature dimension include active power and reactive power, and wherein both the one or more targets and the output of the message passing neural network include one or more electrical parameters. This example contributes to the fourth objective.

[0143] Example 12: The method of Example 11 further includes using the at least one hardware processor to perform the following operations: after pre-training the message-passing neural network elements, training a message-passing neural network including at least one instance of the message-passing neural network elements using a training dataset; deploying the message-passing neural network to output the one or more electrical parameters of each of the plurality of nodes in the power system; determining the power flow in the electrical lines within the power system based on the one or more electrical parameters of the plurality of nodes; and adjusting the configuration of the power system based on the determined power flow. This example contributes to the fourth objective.

[0144] Example 13: The method as described in Example 12 further includes using the at least one hardware processor to use a power flow solver to determine the one or more objectives for each of a plurality of additional node tensors, thereby generating the training dataset. This example contributes to the fourth objective.

[0145] Example 14: The method as described in Example 12 or 13, wherein adjusting the configuration includes controlling the amount of power injected by at least one generator in the power system. This example contributes to the fourth objective.

[0146] Example 15: The method as described in any one of Examples 12 to 14, wherein the loss function used to train the message-passing neural network utilizes a physics-based algorithm to compute the plurality of features based on the output of the message-passing neural network, and for each node tensor in the training dataset, a loss is obtained based on the difference between the plurality of features computed for the node tensor using the physics-based algorithm and the plurality of features in the node tensor. This example contributes to the fourth objective.

[0147] Example 16: The method as described in any of the preceding embodiments further includes using the at least one hardware processor to perform the following operations after the pre-training: copying the message-passing neural network elements to generate a plurality of message-passing neural network elements; incorporating the plurality of message-passing neural network elements into a message-passing neural network; and training the message-passing neural network incorporated with the plurality of message-passing neural network elements using a training dataset. This embodiment contributes to the first objective.

[0148] Example 17: The method as described in Example 16, wherein the plurality of message-passing neural network elements includes at least two parallel message-passing neural network elements, and wherein the message-passing neural network combines the outputs of the at least two parallel message-passing neural network elements into a single output. This example contributes to the first objective.

[0149] Example 18: The method as described in any of the preceding embodiments, wherein the loss function utilizes a physics-based algorithm to compute the plurality of features based on the outputs of the message-passing neural network elements, and for each of the plurality of node tensors, a loss is obtained based on the difference between the plurality of features computed for the node tensor using the physics-based algorithm and the plurality of features in the node tensor. This embodiment contributes to the first objective.

[0150] Example 19: A system comprising: at least one hardware processor; and software configured to perform the method as described in any one of Examples 1 to 18 when executed by the at least one hardware processor.

[0151] Example 20: A non-transitory computer-readable medium having instructions stored thereon, wherein, when executed by a processor, the instructions cause the processor to perform the following operations: perform the method as described in any one of Examples 1 to 18.

[0152] The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the invention. Various modifications to these embodiments will become readily apparent to those skilled in the art, and the general principles described herein can be applied to other embodiments without departing from the spirit or scope of the invention. Therefore, it should be understood that the description and drawings presented herein represent currently preferred embodiments of the invention, and thus represent a broad range of subjects considered in the invention. It should be further understood that the scope of the invention fully encompasses other embodiments that may be obvious to those skilled in the art, and therefore the scope of the invention is not limited.

[0153] As used herein, the terms “comprising,” “comprise,” and “comprises” are open-ended. For example, “A comprises B” means that A may include either (i) B alone, or (ii) a combination of B with one or more (and potentially any number) other components. In contrast, the terms “consisting of,” “consist of,” and “consists of” are closed-ended. For example, “A constitutes B” means that A includes only B and excludes no other components in the same context.

[0154] The combinations described herein, such as "at least one of A, B, or C," "one or more of A, B, or C," "at least one of A, B, and C," "one or more of A, B, and C," and "A, B, C, or any combination thereof," include any combination of A, B, and / or C, and may include multiple A, multiple B, or multiple C. Specifically, combinations such as "at least one of A, B, or C," "one or more of A, B, or C," "at least one of A, B, and C," "one or more of A, B, and C," and "A, B, C, or any combination thereof" may be only A, only B, only C, A and B, A and C, B and C, or A and B and C, and any such combination may contain one or more members of its constituent parts A, B, and / or C. For example, a combination of A and B may include one A and multiple B, multiple A and one B, or multiple A and multiple B.

Claims

1. A method comprising using at least one hardware processor to: Obtain a pre-training dataset, which includes one or more batches, each batch containing multiple node tensors, wherein... Each of the plurality of node tensors has a node dimension and a feature dimension. The node dimension represents a plurality of nodes in a graph, where the graph represents a power system. The feature dimension represents a plurality of features of each of the plurality of nodes. Each of the plurality of node tensors is labeled with one or more targets of each of the plurality of nodes. The plurality of features in the first feature dimension include active power and reactive power. The elements of a message-passing neural network are pre-trained by minimizing a loss function that represents the difference between the one or more targets and the output of the message-passing neural network elements, for each of the plurality of node tensors and for each of the plurality of features, wherein both the one or more targets and the output of the message-passing neural network include one or more electrical parameters. The encoder in the encoder-decoder sequence is applied to compress the node tensor from the node dimension to a latent dimension smaller than the node dimension, thereby generating a latent tensor. The latent tensor is processed through one or more neural network layers, and The decoder in the encoder-decoder sequence is applied to decompress the processed latent tensor from the latent dimension to the node dimension.

2. The method of claim 1, further comprising, for each of the plurality of node tensors, using the at least one hardware processor to: Before applying the encoder, the node tensor is transposed from a first shape to a second shape; and After applying the decoder, the node tensor is transposed from the second shape to the first shape.

3. The method as described in claim 1, wherein, Obtaining the pre-trained dataset involves generating the pre-trained dataset by using internal iterative data generated during iterative solution of the system of equations to determine one or more targets for each of the plurality of node tensors.

4. The method of claim 3, further comprising using the at least one hardware processor to: Obtain a training dataset, which is generated by using a power flow solver without using internal iterative data to determine one or more objectives for each of a plurality of additional node tensors; and After pre-training the elements of the message-passing neural network Incorporate at least one instance of the message-passing neural network element into the message-passing neural network, and The training dataset is used to train a message-passing neural network having at least one instance of the incorporated message-passing neural network.

5. The method of claim 1, further comprising using the at least one hardware processor to, during the pre-training period, append an incident tensor to the latent tensor for each of the plurality of node tensors and for each of the plurality of features, wherein, The incident tensor indicates whether each of one or more incidents exists.

6. The method of claim 5, wherein, The incident tensor is an indicator vector, and the one or more incidents are encoded in the indicator vector.

7. The method of claim 5, further comprising using the at least one hardware processor to: Obtain incident information indicating which of the one or more incidents applies to the plurality of node tensors; The neural network is applied to the accident information to generate the accident tensor for each of the plurality of node tensors, wherein... The dimension of the accident tensor is lower than the dimension of the accident information.

8. The method of claim 1, wherein, The message-passing neural network elements use a multi-head attention mechanism, which performs the following operations on each of the plurality of node tensors: The query is obtained from the node tensor; The value and key are obtained from the output of the encoder; as well as An attention tensor is generated based on the query, the value, and the key. The attention tensor is then appended to the output of the decoder.

9. The method of claim 1, wherein, The message-passing neural network elements use skip connections, which append a node tensor, including the plurality of features in the feature dimension, to the output of the decoder for each of the plurality of node tensors.

10. The method of claim 1, further comprising using the at least one hardware processor to: After pre-training the messaging neural network elements, a messaging neural network including at least one instance of the messaging neural network elements is trained using a training dataset; The message-passing neural network is deployed to output one or more electrical parameters for each of the plurality of nodes in the power system; The power flow in the electrical lines within the power system is determined based on one or more electrical parameters of the plurality of nodes; as well as The power system configuration is adjusted based on the determined power flow.

11. The method of claim 10, further comprising using the at least one hardware processor to use a power flow solver to determine the one or more objectives for each of a plurality of additional node tensors, thereby generating the training dataset.

12. The method of claim 10, wherein, Adjusting the configuration includes controlling the amount of power injected by at least one generator in the power system.

13. The method of claim 10, wherein, The loss function used to train the message-passing neural network utilizes a physics-based algorithm to compute the plurality of features based on the output of the message-passing neural network, and for each node tensor in the training dataset, a loss is obtained based on the difference between the plurality of features computed for the node tensor using the physics-based algorithm and the plurality of features in the node tensor.

14. The method of claim 1, further comprising using the at least one hardware processor to: Duplicate the message passing neural network element to generate multiple message passing neural network elements; The plurality of message-passing neural network elements are incorporated into the message-passing neural network; and The training dataset is used to train a message-passing neural network that incorporates the multiple message-passing neural network elements.

15. The method of claim 14, wherein, The plurality of message-passing neural network elements include at least two parallel message-passing neural network elements, wherein the message-passing neural network combines the outputs of the at least two parallel message-passing neural network elements into a single output.

16. The method of claim 1, wherein, The loss function utilizes a physics-based algorithm to compute the plurality of features based on the outputs of the message-passing neural network elements, and for each of the plurality of node tensors, a loss is obtained based on the difference between the plurality of features computed for the node tensor using the physics-based algorithm and the plurality of features in the node tensor.

17. A system comprising: At least one hardware processor; as well as Software, the software being configured to, when executed by the at least one hardware processor, Obtain a pre-training dataset comprising one or more batches, each batch comprising multiple node tensors. Each node tensor has a node dimension and a feature dimension. The node dimension represents multiple nodes in a graph, where the graph represents a power system. The feature dimension represents multiple features of each node. Each node tensor is labeled with one or more targets for each node. The multiple features in the first feature dimension include active power and reactive power. The elements of a message-passing neural network are pre-trained by minimizing a loss function that represents the difference between the one or more targets and the output of the message-passing neural network elements, for each of the plurality of node tensors and for each of the plurality of features, wherein both the one or more targets and the output of the message-passing neural network include one or more electrical parameters. The encoder in the encoder-decoder sequence is applied to compress the node tensor from the node dimension to a latent dimension smaller than the node dimension, thereby generating a latent tensor. The latent tensor is processed through one or more neural network layers, and The decoder in the encoder-decoder sequence is applied to decompress the processed latent tensor from the latent dimension to the node dimension.

18. A non-transitory computer-readable medium storing message-passing neural network elements thereon, the message-passing neural network elements comprising: encoder-decoder sequence, the encoder-decoder sequence The system receives a node tensor as input. This node tensor has a node dimension and a feature dimension. The node dimension represents multiple nodes in a graph, where the graph represents a power system. The feature dimension represents multiple features of each node. The multiple features in the first feature dimension include active power and reactive power. For each of the plurality of features, The node tensor is compressed from the node dimension to a latent dimension smaller than the node dimension, thereby generating a latent tensor. Decompress the latent tensor from the latent dimension to the node dimension; and One or more neural network layers process the latent tensor before decompressing it via the encoder-decoder sequence.

Images