Cloud-based electrical network component verification

Abstract

Methods, systems, and apparatus, including medium-encoded computer program products, for cloud-based electrical network component verification may include obtaining a computer model of an electrical network. The computer model may include asset models for individual assets connected to the power grid. Each asset model may be configured to replicate operations of a corresponding type of physical electrical network asset. The first asset model may include a hardware emulator configured to run firmware specific to a corresponding type of physical electrical network asset. A modified firmware for the first asset model may be received. The computer model may be processed using a simulation engine to obtain simulation results, and the simulation engine may be configured to simulate operations of various assets of the computer model, including operations of a first asset model according to modified firmware associated with the first asset model. Simulation results may be provided.

Description

[0001] Priority declaration

[0002] This application claims priority under 35 USC §119(e) to U.S. patent application serial number 63 / 359,536, filed on July 8, 2022, the entire contents of which are incorporated herein by reference. Background Art

[0003] The electrical system includes a wide range of various types of interconnected components such as inverters (solar, wind, HVDC, FACTS, etc.), relays, power plant controllers (PPC), energy management systems, remedial action systems (RAS), automatic generator controls, alarm systems, etc. Summary of the invention

[0004] The specification relates to cloud-based verification of components of an electrical system. The verification uses a hardware simulator configured to run component firmware. The verification technique may include simulation of assets within the electrical system, wherein the asset simulation uses a model of the asset to simulate the behavior of an electrical power grid element. The asset model may include firmware specific to a type of asset, and the hardware simulator may be configured to run the component firmware. The system may receive changes to the firmware and verify the components of the electrical system by simulating the electrical system using the changed firmware.

[0005] Specific embodiments of the subject matter described in this specification may be implemented to achieve one or more of the following advantages. The techniques described below may be used to simulate the results of changing the firmware of a component of an electrical grid, which may reduce the likelihood that a firmware defect will cause an error in the grid. The techniques also implement simulations without requiring distribution of hardware components, which may reduce costs and the chance that a malicious party will reverse engineer the hardware components. In addition, the techniques allow a user to perform simulations of various combinations of firmware updates applied to multiple components of a system in various combinations. Such robust simulations may further reduce the likelihood that errors associated with firmware updates will be introduced into actual hardware assets installed in the power system.

[0006] One aspect is characterized by obtaining a computer model of an electric grid. The computer model may include asset models for various assets connected to the electric grid. Each asset model may be configured to replicate the operation of a corresponding type of physical electric grid asset. A first asset model in the asset model may include a hardware emulator configured to execute firmware specific to the corresponding type of physical electric grid asset. An asset interface may be provided, which is configured to allow a user to change the first asset model. Modified firmware for the first asset model may be received from the user through the asset interface. The computer model may be processed using a simulation engine to obtain simulation results, and the simulation engine may be configured to simulate the operation of various assets of the computer model, including the operation of the first asset model according to the modified firmware associated with the first asset model. A simulation result indicating how the first asset model functions when using the modified firmware may be provided for display to the user. Other implementations of this aspect include corresponding systems, apparatus, and computer programs configured to perform the actions of the method encoded on a computer storage device.

[0007] One or more of the following features may be included. In response to determining that the simulation results meet the validation criteria, it may be determined that the modified firmware is valid. The modified firmware may include a replacement of the firmware. The modified firmware may include a configuration setting of the firmware. The asset interface may be configured to allow a user to change the second asset model, and may also include receiving a second modified firmware for the second asset model through the asset interface and from the user. The computer model may be processed using a simulation engine to obtain simulation results, the simulation engine being configured to simulate the operation of each asset of the computer model, including (i) the operation of the first asset model according to the modified firmware associated with the first asset model, and (ii) the operation of the second asset model according to the second modified firmware associated with the second asset model. The specification of the modified firmware may be determined to meet the specification of the asset described by the asset model, and the modified firmware may be applied to the asset. The specification of the asset may include a type, brand, or version of the asset. In response to determining that the simulation results meet the acceptance criteria, an operation may be performed. The operation may include updating the computer model of the power grid. The operation may include providing a notification to the user indicating that the simulation was successful. The operation of each asset of the computer model may include running the firmware. Running the firmware may include accepting inputs and performing operations on the inputs using the firmware to produce results that simulate results that would be produced if the firmware were run on physical hardware and running the same inputs. Processing the computer model may include running a simulation engine on the computer model in a sandbox environment.

[0008] The details of one or more embodiments of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the invention will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 An example environment for cloud-based electrical grid component verification is shown.

[0010] Figure 2 is a flow chart of an example process for cloud-based electrical grid component verification.

[0011] Figure 3 is a block diagram of an example computer system. DETAILED DESCRIPTION

[0012] Electrical systems include a wide range of interconnected components of various types, such as inverters (solar, wind, HVDC, FACTS, etc.), relays, power plant controllers (PPCs), energy management systems, remedial action systems (RASs), automatic generator controls, alarm systems, etc. The operation of one component can affect the operation of other components. For example, a PPC regulates and controls networked inverters within a power plant. Therefore, when a component is planned to change, it is important to test the modified version of the component in an environment that includes another connected element to ensure that the change does not have a harmful effect anywhere in the system. To avoid potential catastrophic failures, updates should be deployed to the production environment only after thorough testing is completed.

[0013] The operation of many components is controlled in part by firmware, which is software that provides low-level control for the component's specific hardware. Firmware can be updated to implement new hardware capabilities and to address any defects detected in the firmware version. However, as mentioned above, to reduce the likelihood that defects in the firmware will cause failures anywhere in the system, before deploying a firmware update, the update should be tested in an environment that simulates the entire production environment.

[0014] One approach to this type of testing is called "hardware-in-the-loop" (HIL) simulation. With HIL simulation, the environment is simulated by adding mathematical representations called "plant simulations" for all relevant components of the environment. The components under test are connected to a HIL simulator that provides electrical simulations of sensors and actuators. These electrical simulations act as an interface between the plant simulations and the components under test. The value of each electrical simulation sensor is controlled by the plant simulation and read by the embedded system under test.

[0015] However, there are several disadvantages to HIL systems. First, simulators are specific to individual electrical components, and if a simulator does not exist for a particular component, HIL cannot be used. Second, HIL simulators are loaded with mathematical representations of other components in the environment, and when those components are updated (e.g., component firmware is updated), the HIL simulators may become outdated. Finally, HIL simulators are physically large, which may hinder their use in real-world deployments.

[0016] This specification describes techniques for cloud-based electrical grid component verification, also referred to as "Firmware-in-the-Loop in the Cloud" (FILIC). A verification system that performs the techniques of this specification may include a cloud-based verification engine that includes a grid model that simulates the overall structure of an electrical grid. The grid model may include asset models for each type of electrical component in a relevant environment.

[0017] By providing FILIC, the techniques described in this specification allow firmware developers to test firmware updates in the context of a grid model without requiring the presence of expensive hardware such as a HIL system. In addition, compared to traditional techniques for testing firmware updates against one firmware version for each of the other grid components, the techniques of this specification enable simulation when multiple electrical components are applying firmware updates simultaneously.

[0018] Figure 1 An example environment for cloud-based electrical grid component verification is shown. The environment may include an electrical grid component verification system 110 and one or more client devices 105a, 105n (collectively 105).

[0019] The electrical grid component verification system 110 may include a power grid model acquisition engine 120, an asset interface engine 130, a model adjustment engine 135, a model processing engine 140, and a result provision engine 150. The electrical grid component verification system 110 may reside on one or more computer servers, which may reside at a single location, or may be geographically distributed and connected via a network (such as the Internet or an intranet). For example, the electrical grid component verification system 110 may be run on a cloud platform. The cloud platform may include computing resources such as servers, storage, databases, networks, and may reside in a single data center, or may be dispersed across multiple data centers connected via a network (such as the Internet or an intranet), which may be geographically remote.

[0020] The power grid model acquisition engine 120 can obtain a model of the power grid (for brevity, a "grid model") 180a. The power grid model acquisition engine 120 may include an application programming interface (API) that allows a user of the system 110 to provide a grid model 180a or the location of the model. In some implementations, the API can accept a uniform resource indicator (URI) that specifies the location of the grid model 180a. The power grid model acquisition engine 120 can use the URI to retrieve the model 180a. In some implementations, the power grid model acquisition engine 120 accesses the grid model 180a from a database. For example, a public model of the electrical grid can be used by many users of the system 110. For example, the power grid model acquisition engine 120 can access a public electrical grid model of a particular geographic region (e.g., California) for all users who wish to test firmware of different physical hardware assets connected to the electrical grid in the particular geographic region.

[0021] The grid models 180a, 180b (collectively 180) may include multiple asset models of various assets connected to the grid and a description of the interconnections between the assets. Each asset model may be configured to replicate the operation of a corresponding type of physical electrical grid asset, and each asset model may be used to simulate the operation of an asset or a type of asset. The asset model may include a hardware emulator. The hardware emulator may be configured to run firmware specific to the corresponding type of physical electrical grid asset.

[0022] Running the firmware may include accepting inputs and performing operations specified by the firmware on the inputs to produce results that simulate the results that would be produced when the firmware is run on physical hardware and runs the same inputs. The inputs may be simulated inputs, such as data captured when operating an actual power grid, random data generated according to parameters encountered in an actual power grid (e.g., voltage varying within some known parameters), data generated by a human operator, etc.

[0023] The grid model 180 of the power grid can be represented as a graph. The nodes of the graph can represent assets, and each node can include information about the asset, such as an identifier, the type of asset, the brand and version of the asset, the installation date, information about services performed on the asset, etc. The edges in the graph can represent connections between assets and can include an indication of the direction of power flow.

[0024] The asset interface engine 130 may receive changed firmware 185. The change may be of many types, including replacement of firmware, replacement of one or more components of firmware, addition or removal of components of firmware, changes to one or more firmware configuration settings, providing initial values ​​for configuration settings, other firmware changes, or any combination thereof. The changed firmware may include the type of change, the specific assets to which the change should be applied, a description of the class of assets to which the change should be applied (e.g., all transformers of a particular brand and version), change data (e.g., firmware update, configuration settings, etc.), and other data related to the change. In some implementations, the asset interface engine 130 may include an API that allows a user of the system 110 to provide changed firmware 185 or a location (e.g., a URI) of the changed firmware 185.

[0025] In some implementations, the asset interface engine 130 may provide user interface presentation data that, when rendered by the client device 105, causes the client device 105 to render one or more user interface panels. Once the user has interacted with the user interface presentation data, the user interface presentation data may cause the client device 105 to transmit the modified firmware 185 or the location of the modified firmware 185 to the asset interface engine.

[0026] Model adjustment engine 135 may accept mesh model 180a and modified firmware 185 and produce modified mesh model 180b. In some implementations, modified mesh model 180b is mesh model 180a modified only by modified firmware 185. Model adjustment engine may apply modified firmware 185 to one or more assets in mesh model 180a to produce modified mesh model 180b, as described further below.

[0027] In some implementations, the model adjustment engine 135 can provide a "sandbox" in which the only change to the grid model is the changed firmware 185 submitted by a specific user. The changed firmware can be applied to one or more appropriate components of the electrical grid model. For example, the changed firmware can be applied to a single component, all components of a specific type (e.g., defined by the brand and model of the component), a subset of components of a specific type, etc. The other components of the grid model remain unchanged. Providing a sandbox environment enables the system to simulate specific firmware changes while keeping other components unchanged. In addition, by providing a sandbox, simulations run by one user are not affected by the changed firmware 185 used in simulations by other users. For example, the grid model or an appropriate portion thereof can be copied to a sandbox environment. The changed firmware can be applied to a specific asset model of the copied grid model within the sandbox environment. A simulation can be run on the copied grid model within the sandbox environment to simulate the operation of an electrical grid with updated firmware applied to a specific asset model.

[0028] To enable simultaneous updating of multiple electrical components, in some implementations, the model adaptation engine 135 may apply multiple instances of the firmware update 185 to the multiple electrical components. The system may accept multiple firmware updates 185 and apply the updates to create a modified grid model 180b. For example, the model adaptation engine 135 may compare the specification associated with the asset type with the specification associated with the firmware update 185, as described further below.

[0029] In some implementations, the model adaptation engine 135 may provide the user with a list containing available firmware updates in addition to the firmware update 185. In response to receiving the indication of the selected update, the model adaptation engine 135 may apply the selected update and the firmware update 185 to generate the modified mesh model 180b.

[0030] In some implementations, the model adjustment engine 135 may generate multiple updated mesh models 185b. The model adjustment engine 135 may generate a first updated mesh model 180b that differs from the mesh model 180a only in the modified firmware 185. The model adjustment engine 135 may generate a second updated mesh model 185b that contains the modified firmware 185 and other adjustments selected, for example, by a user. The model adjustment engine 135 may provide the multiple modified mesh models 185b to the model processing engine 140, which may perform multiple simulations, as further described below. Enabling multiple simulations using multiple modified mesh models 180b enables a user to determine whether a firmware update 185 is compatible with other user-proposed firmware updates 185, thereby reducing the likelihood that multiple firmware updates will create mesh defects. The system may generate a modified mesh model 180b for each candidate firmware update 185 that may be deployed during an update cycle.

[0031] The model processing engine 140 may evaluate the modified grid model 180b to produce simulation results 190 indicating how each asset model functions while using the modified firmware 185. The model processing engine 140 may obtain or create simulated electrical inputs used by the simulation, as described above.

[0032] In some implementations, the simulation results 190 may include instantaneous values ​​of electrical parameters (e.g., current, voltage, etc.) and magnitudes of electrical parameters at one or more time periods and at one or more points within the modified grid model 180b. In some implementations, the simulation results 190 may include conditional values. For example, if the voltage at a particular asset meets a configured value, the simulation results 190 may include an indication such as the asset being highlighted in red. The simulation results 190 may also include animations showing simulation results over time, audio indications when a threshold has been met, and the like.

[0033] In different implementations, the simulation results 190 may include results for various assets within the grid. In some implementations, the simulation results 190 may represent electrical parameters of one or more assets in the modified grid model 185b to which the firmware update 185 is applied. In some implementations, the simulation results 190 may represent electrical parameters of all assets in the modified grid model 185b. In some implementations, the simulation results 190 may represent electrical parameters of one or more assets within the identified area of ​​the modified grid model 185b or assets indicated by the user.

[0034] The simulation results 190 may be encoded in any suitable data format. For example, the simulation results 190 may be encoded in XML, where the elements of the XML represent the electrical points within the modified grid model 185b and the data of the electrical points, as described above (e.g., instantaneous values). The simulation results 190 may include elements of the electrical points at one or more time periods.

[0035] The result providing engine 150 may provide, for example, simulation results 190 for display to a user. The simulation results 190 may be provided in a data format provided by the model processing engine 140, for example, an XML format that may be displayed by a web browser. In some implementations, the result providing engine 150 may create user interface presentation data from the simulation results 190 and provide the user interface presentation data to the client device 105 for display.

[0036] Figure 2 2 is a flow chart of an example process for cloud-based electrical grid component verification. For convenience, process 200 will be described as being performed by a system for cloud-based electrical grid component verification (e.g., Figure 1 The operations of process 200 may also be implemented as instructions stored on one or more computer-readable media that may be non-transitory, and execution of the instructions by one or more data processing devices may cause one or more data processing devices to perform the operations of process 200. One or more other components described herein may perform the operations of process 200.

[0037] The system can obtain (205) a power grid model. The system can provide an API configured to accept a URI that specifies the location of the power grid model. Upon receiving the URI, the system can use a conventional web network protocol (e.g., Hypertext Transfer Protocol (HTTP) or HTTP Secure (HTTP-S)) to retrieve the power grid model from the location specified by the URI.

[0038] The system may provide (210) an asset interface configured to allow a user to change an asset model. As described above, in some implementations, the system may provide an asset interface that provides a user interface presentation data that enables a user to change an asset model. In some implementations, the system may provide an asset interface, such as a web interface, that accepts changes to an asset model.

[0039] The system may receive (215) the changed firmware. In some implementations, the system may provide user interface presentation data that, when rendered by a client device (e.g., by a web browser running on a laptop or desktop computer), allows a user to specify a location of updated firmware and indicates that the firmware needs to be updated. The system may then retrieve the updated firmware from the location and load it into the corresponding model, replacing the existing firmware. In some implementations, the system may provide user interface presentation data that, when rendered by a client device, allows a user to provide (e.g., using a hypertext transfer protocol) firmware associated with a particular asset model to the system. In some implementations, the system may provide user interface presentation data that, when rendered by a client device, allows a user to specify firmware settings associated with a particular asset model. In some implementations, the system may provide an application programming interface (API) that may be called by a client (a user or another program) that provides updated firmware for an asset type or a particular asset present in an environment.

[0040] In some implementations, the system can receive multiple instances of modified firmware. The instances of modified firmware can be associated with different asset models, and / or multiple instances of modified firmware can be associated with one asset model. For example, one instance of modified firmware can be associated with a first asset model, and a second instance of modified firmware can be associated with a second, different asset model. In other examples, one instance of modified firmware can include components to be included in firmware for one asset model, and the second instance of modified firmware can include configuration settings for the firmware for the asset model.

[0041] Once received, the system may change (220) the firmware of at least one asset model in the grid model. The system may first determine the assets in the grid model to which the firmware change (or changes) is applied by comparing the specification of the firmware change with the specifications of the assets in the grid model. The system may first identify any assets in the grid model using an identification specified by the firmware change. The system may further identify any assets in the grid model that have specifications that meet the firmware change specifications. For example, if the firmware change specifies an asset type, an asset brand, and an asset version, the system may identify all assets in the grid model that have the specified asset type, brand, and version. In another example, if the firmware change specifies an asset type and an asset brand, the system may identify all assets in the grid model that have the specified asset type and brand. In some implementations, the firmware change may include a more complex specification (e.g., a regular expression) that identifies assets within the grid model. In such an implementation, the system may evaluate the specification to determine related assets. The system may determine assets related to all firmware changes received in operation 215.

[0042] Once the system has identified the asset, the system can apply the modified firmware. As described above, the modified firmware can include an indication of the type of modification (e.g., changing a setting, replacing firmware, etc.). The system can determine the type of modification and perform the appropriate modification. For example, if a firmware configuration setting is set or changed, the system can set or change the setting; if a component of the firmware is to be replaced, the system can remove the original component from the firmware and insert the new component; if the entire firmware is to be replaced, the system can remove the original firmware and insert the new firmware; and so on.

[0043] The system may process (225) the computer model using a simulation engine to obtain simulation results. The simulation engine may be configured to simulate the operation of various assets in the computer grid model using simulated electrical inputs. In some implementations, the system may obtain configured simulated electrical inputs from a storage device, such as by reading a file containing simulated electrical inputs. In some implementations, the system may randomly or pseudo-randomly generate simulated electrical inputs. In some implementations, the system may randomly or pseudo-randomly generate simulated electrical inputs that follow a rule (e.g., a particular load must be between a minimum and a maximum value) or a configured distribution.

[0044] Once the asset model and simulated electrical input have been loaded, the system performs a simulation by processing the asset model using the simulation engine to obtain simulation results. The simulation results can be generated by evaluating one or more asset models against the simulation input data. As described above, the asset model can include a hardware emulator that can run firmware designed for a specific asset or asset type. In response to receiving the simulated electrical input, the hardware emulator can run the firmware to produce a result, which can include changes in the internal state of the simulated asset and an electrical output similar to how a physical asset would function.

[0045] As described above, the system can process a computer model on one or more grid models including updated firmware. In some implementations, the system processes the computer model using a sandbox so that simulations using one grid model are isolated from other simulations. In some implementations, the system can process the computer model using a grid model that includes all firmware updates. In some implementations, the system can process the computer model using a grid model that includes firmware updates applied in any combination. For example, if the system has three firmware updates F1, F2, and F3 available, the system can process the computer model using only F1; F1 and F2; F1 and F3; F2 and F3; and F1, F2, and F3; or any subset of the combination. Each instance of processing a computer result can produce a simulation result.

[0046] The system may provide (230) simulation results indicating how the asset model functions when using the firmware for display to a user. The results may be included in user interface presentation data that, when rendered by a client device, causes the client device to display the simulation results. As described above, the simulation results may be provided by supplying or making available data including the simulation results, the data including the user interface presentation data.

[0047] As further described above, the simulation result may include a condition value. Such a condition value may indicate whether the firmware update can be verified. For example, if the simulation result satisfies the conditions of all condition values, the system may determine that the firmware update can be verified.

[0048] In some implementations, the system may update the grid model. The system may provide simulation results, and in response to determining that the simulation meets acceptance criteria, the system may perform one or more operations. In some implementations, the operations may include updating the grid model. The acceptance criteria may be a set of predicates (e.g., values ​​falling within an acceptable range) that, when met, indicate that the updated firmware operates correctly. Predicates may be combined by Boolean operations (AND, OR, NOT, NOR, etc.) to form complex predicates. In some implementations, in response to receiving an indication from a user that the simulation results are accepted, the system may update the grid model. In some implementations, the system may update the grid model when the acceptance criteria are met and when the user provides an acceptance indicator.

[0049] The system can update the mesh model, for example, by storing the updated mesh model in a database. In some implementations, the system can replace the previous mesh model in the database with the updated mesh model. In some implementations, the system can add the updated mesh model to the database and store an indicator that the updated mesh model is the current mesh model.

[0050] In some implementations, in response to determining that the acceptance criteria are met, the system can provide a notification to the user indicating that the simulation was successful. For example, the system can provide the notification by delivering a message using a conventional network protocol (e.g., Transmission Control Protocol / Internet Protocol) or by providing user interface presentation data that displays a notification of the success of the simulation (e.g., a user interface message) when rendered by a client device.

[0051] Although this specification primarily describes electrical networks, these techniques can be applied to other network types. For example, a computing network may include components such as routers, switches, cellular towers, wireless access points, etc. A failure caused by a firmware update at one network element may result in cascading failures throughout the network. Similarly, an Internet of Things (IoT) network may include many devices, which may include various sensor types (cameras, temperature sensors, barometers, light detection and ranging (Lidar), radar, etc.), actuators (e.g., for locking or unlocking doors, triggering alarms, activating sensors, moving robotic devices, etc.), and interconnected networks. A defect introduced into one component (e.g., a faulty sensor that transmits too much data) may result in failures throughout the network.

[0052] Figure 3300. The system 300 includes a processor 310, a memory 320, a storage device 330, and an input / output device 340. Each of the components 310, 320, 330, and 340 can be interconnected, for example, using a system bus 350. The processor 310 is capable of processing instructions for operation within the system 300. In one implementation, the processor 310 is a single-threaded processor. In another implementation, the processor 310 is a multi-threaded processor. The processor 310 is capable of processing instructions stored in the memory 320 or on the storage device 330.

[0053] Memory 320 stores information within system 300. In one implementation, memory 320 is a computer readable medium. In one implementation, memory 320 is a volatile memory unit. In another implementation, memory 320 is a non-volatile memory unit.

[0054] The storage device 330 can provide mass storage for the system 300. In one implementation, the storage device 330 is a computer-readable medium. In various implementations, the storage device 330 may include, for example, a hard disk device, an optical disk device, a storage device shared by multiple computing devices over a network (e.g., a cloud storage device), or some other mass storage device.

[0055] The input / output device 340 provides input / output operations for the system 300. In one implementation, the input / output device 340 may include one or more of a network interface device (e.g., an Ethernet card), a serial communication device (e.g., an RS-232 port), and / or a wireless interface device (e.g., an 802.11 card). In another implementation, the input / output device may include a driver device configured to receive input data and send output data to other input / output devices (e.g., a keyboard, a printer, and a display device 360). However, other implementations may also be used, such as mobile computing devices, mobile communication devices, set-top TV client devices, etc.

[0056] Although already Figure 3 An example processing system is described in the specification, but the subject matter and implementation of the functional operations described in this specification may be implemented in other types of digital electronic circuits, or in computer software, firmware or hardware (including the structures disclosed in this specification and their structural equivalents), or in a combination of one or more of them.

[0057] The embodiments of the subject matter and functional operations described in this specification can be implemented in digital electronic circuits, or in computer software, firmware or hardware, including the structures disclosed in this specification and their structural equivalents, or a combination of one or more of them. The embodiments of the subject matter described in this specification can be implemented using one or more modules of computer program instructions encoded on a computer-readable medium, which are used to be executed by a data processing device or control the operation of the data processing device. The computer-readable medium can be a manufactured product, such as a hard drive in a computer system or an optical disk sold through a retail channel, or an embedded system. The computer-readable medium can be obtained separately and then encoded with one or more modules of computer program instructions, such as by delivering one or more modules of computer program instructions via a wired or wireless network. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, or a combination of one or more of them.

[0058] The term "data processing apparatus" encompasses all apparatus, devices and machines for processing data, including, for example, a programmable processor, a computer or a plurality of processors or computers. In addition to hardware, the apparatus may also include code that creates an execution environment for the computer program in question, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, a runtime environment, or a combination of one or more of them. Furthermore, the apparatus may employ a variety of different computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures.

[0059] A computer program (also referred to as a program, software, software application, script, or code) may be written in any suitable form of a programming language, including compiled or interpreted languages, declarative or procedural languages, and may be deployed in any suitable form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). A computer program may be deployed to execute on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communications network.

[0060] The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuits, and the apparatus can also be implemented as special purpose logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits).

[0061] Processors suitable for executing computer programs include, for example, dedicated microprocessors. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices (such as magnetic disks, magneto-optical disks, or optical disks) for storing data, or be operably coupled to receive data from one or more mass storage devices or transmit data to them or both. However, a computer need not have such a device. In addition, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), to name a few. Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM (erasable programmable read-only memory), EEPROM (electrically erasable programmable read-only memory), and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CDROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0062] To provide interaction with a user, embodiments of the subject matter described in this specification may be implemented on a computing device capable of providing information to a user. Information may be provided to the user in any form of sensory format, including visual, auditory, tactile, or a combination thereof. The computing device may be coupled to a display device, such as an LCD (liquid crystal display) display device, an OLED (organic light emitting diode) display device, another monitor, a head-mounted display device, or the like, for displaying information to the user. The computing device may be coupled to an input device. The input device may include a touch screen, a keyboard, and an indicating device, such as a mouse or a trackball, through which a user can provide input to the computing device. Other types of devices may also be used to provide interaction with a user; for example, feedback provided to the user may be any suitable form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any suitable form, including sound, voice, or tactile input.

[0063] A computing system may include a client and a server. The client and the server are usually remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by means of a computer program that runs on a corresponding computer and has a client-server relationship with each other. The embodiments of the subject matter described in this specification may be implemented in a computing system that includes a back-end component (e.g., as a data server), or includes a middleware component (e.g., an application server), or includes a front-end component (e.g., a client computer with a graphical user interface or a web browser, through which a user can interact with the implementation of the subject matter described in this specification), or includes any combination of one or more such back-ends, middleware or front-end components. The components of the system may be interconnected by digital data communication (e.g., a communication network) in any suitable form or medium. Examples of communication networks include local area networks ("LANs") and wide area networks ("WANs"), interconnected networks (e.g., the Internet), and peer-to-peer networks (e.g., self-organizing peer-to-peer networks).

[0064] Although this specification contains many implementation details, these should not be interpreted as limitations on the scope of what is claimed or may be claimed, but rather as descriptions of features specific to a particular embodiment of the disclosed subject matter. Certain features described in the context of a separate embodiment in this specification may also be implemented in combination in a single embodiment. On the contrary, the various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination. In addition, although features may be described above as working in certain combinations and even initially claimed as such, one or more features from the claimed combination may be deleted from the combination in some cases, and the claimed combination may be directed to a sub-combination or a variation of a sub-combination. Therefore, unless otherwise expressly stated, or unless otherwise expressly stated by the knowledge of a person of ordinary skill in the art, any feature of the above-described embodiments may be combined with any other feature of the above-described embodiments.

[0065] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that the operations be performed in the particular order shown or in sequence, or that all of the operations shown be performed, to achieve the desired results. In some cases, multitasking and / or parallel processing may be advantageous. In addition, the separation of various system components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0066] Thus, certain embodiments of the invention have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results.

Claims

1. A method for verifying electrical grid assets, comprising: obtaining a computer model of an electrical grid, the computer model comprising a plurality of asset models for respective assets connected to the electrical grid, each asset model being configured to replicate the operation of a corresponding type of physical electrical grid asset, wherein a first of the asset models comprises a hardware emulator configured to run firmware specific to the corresponding type of physical electrical grid asset; providing an asset interface configured to allow a user to modify the first asset model; receiving, via the asset interface and from a user, modified firmware for a first asset model; processing the computer model using a simulation engine configured to simulate operation of respective assets of the computer model, including operation of the first asset model according to the modified firmware associated with the first asset model, to obtain simulation results; and Simulation results indicating how the first asset model functions when using the modified firmware are provided for display to a user.

2. The electrical grid asset verification method according to claim 1, further comprising: In response to determining that the simulation result satisfies a validation criterion, the modified firmware is determined to be valid.

3. A method for verifying electrical network assets according to any one of the preceding claims, wherein: The modified firmware includes a replacement of the firmware.

4. A method for verifying electrical network assets according to any one of the preceding claims, wherein: The modified firmware includes configuration settings for the firmware.

5. A method for verifying electrical network assets according to any one of the preceding claims, wherein: The asset interface is further configured to allow a user to modify the second asset model, and further includes: receiving, via the asset interface and from a user, second modified firmware for a second asset model; and The computer model is processed using the simulation engine to obtain simulation results, wherein the simulation engine is configured to simulate the operation of each asset of the computer model, including (i) the operation of the first asset model according to the changed firmware associated with the first asset model, and (ii) the operation of the second asset model according to the second changed firmware associated with the second asset model.

6. The electrical grid asset verification method according to claim 5, wherein: Processing the computer model includes running the simulation engine on the computer model in a sandbox environment.

7. The electrical grid asset verification method according to any one of the preceding claims, further comprising: determining that a specification of the modified firmware satisfies a specification of the asset described by the asset model; as well as The modified firmware is applied to the asset.

8. The electrical grid asset verification method according to claim 7, wherein: The specification of the asset described by the asset model includes the type, brand or version of the asset.

9. The electrical grid asset verification method according to any one of the preceding claims, further comprising: In response to determining that the simulation results meet acceptance criteria, an operation is performed.

10. The electrical grid asset verification method according to claim 8, wherein: The operations include updating a computer model of the electrical grid.

11. The electrical grid asset verification method according to claim 8, wherein: The operations include providing a notification to a user indicating that the simulation was successful.

12. A method for verifying electrical network assets according to any one of the preceding claims, wherein: The operation of each asset of the simulated computer model includes running firmware.

13. The electrical grid asset verification method according to claim 10, wherein: The running firmware includes: accept input; and Operations are performed on the inputs using the firmware to produce results that simulate results that would be produced if the firmware were run on physical hardware and operated on the same inputs.

14. A system comprising one or more computers and one or more storage devices storing instructions, the instructions, when executed by the one or more computers, causing the one or more computers to perform operations comprising: obtaining a computer model of an electrical grid, the computer model comprising a plurality of asset models for respective assets connected to the electrical grid, each asset model being configured to replicate the operation of a corresponding type of physical electrical grid asset, wherein a first of the asset models comprises a hardware emulator configured to run firmware specific to the corresponding type of physical electrical grid asset; providing an asset interface configured to allow a user to modify the first asset model; receiving, via the asset interface and from a user, modified firmware for a first asset model; processing the computer model using a simulation engine configured to simulate operation of respective assets of the computer model, including operation of the first asset model according to the modified firmware associated with the first asset model, to obtain simulation results; and Simulation results indicating how the first asset model functions when using the modified firmware are provided for display to a user.

15. The system of claim 14, the operations further comprising: In response to determining that the simulation result satisfies a validation criterion, the modified firmware is determined to be valid.

16. A system according to any one of claims 14 or 15, wherein: The modified firmware includes at least one of the following: (i) replacement or (ii) configuration of the firmware.

17. A system according to any one of claims 14 to 16, wherein: The asset interface is further configured to allow a user to modify the second asset model, and the operations further include: receiving, via the asset interface and from a user, second modified firmware for a second asset model; and The computer model is processed using the simulation engine to obtain simulation results, wherein the simulation engine is configured to simulate the operation of each asset of the computer model, including (i) the operation of the first asset model according to the modified firmware associated with the first asset model, and (ii) the operation of the second asset model according to the second modified firmware associated with the second asset model.

18. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising: obtaining a computer model of an electrical grid, the computer model comprising a plurality of asset models for respective assets connected to the electrical grid, each asset model being configured to replicate the operation of a corresponding type of physical electrical grid asset, wherein a first of the asset models comprises a hardware emulator configured to run firmware specific to the corresponding type of physical electrical grid asset; providing an asset interface configured to allow a user to modify the first asset model; receiving, via the asset interface and from a user, modified firmware for a first asset model; processing the computer model using a simulation engine configured to simulate operation of respective assets of the computer model, including operation of the first asset model according to the modified firmware associated with the first asset model, to obtain simulation results; and Simulation results indicating how the first asset model functions when using the modified firmware are provided for display to a user.

19. The non-transitory computer-readable storage medium of claim 18, the operations further comprising: In response to determining that the simulation result satisfies a validation criterion, the modified firmware is determined to be valid.

20. The non-transitory computer-readable storage medium of any one of claims 18 or 19, wherein the asset interface is further configured to allow a user to modify the second asset model, the operations further comprising: receiving, via the asset interface and from a user, second modified firmware for a second asset model; as well as The computer model is processed using the simulation engine to obtain simulation results, and the simulation engine is configured to simulate the operation of each asset of the computer model, including (i) the operation of the first asset model according to the modified firmware associated with the first asset model, and (ii) the operation of the second asset model according to the second modified firmware associated with the second asset model.

Images