Redundant controllers or input-output gateways without dedicated hardware

Abstract

The present invention is entitled "Redundant Controllers or Input-Output Gateways Without Dedicated Hardware". The present disclosure provides a method of fault-tolerant process control, the method comprising providing a network process control system (600) including a plant-level network (170) in an industrial process facility (IPF), the plant-level network coupling a server (180) to computing platforms (621-624), each computing platform including computing hardware (171) and memory (172) for hosting at least one software application for simultaneously supporting at least one process controller and another process controller or input / output (I / O) gateway. The computing platforms are coupled to each other via a dedicated path redundant network (260) to provide a hardware resource pool. At least some of the computing platforms are directly coupled to a plurality of I / O devices (145) via an I / O mesh network (240), the plurality of I / O devices are coupled to field devices (150), and the field devices are coupled to a processing device (160). Upon detecting at least one failed device in the hardware resource pool, a backup for the failed device from another device at another computing platform in the hardware resource pool is put into service via the dedicated path redundant network.

Description

Technical Field

[0001] The disclosed embodiments of the present invention relate to a network control system, and more particularly to a redundant controller or a redundant input / output (I / O) gateway device in a network control system. Background Art

[0002] Failures in industrial control systems can result in expensive downtime. There are expenses involved in restarting the process as well as actual production losses due to such failures. If the process is designed to run without supervisory or service personnel, all components in the process control system typically need to be fault tolerant.

[0003] A fault-tolerant industrial process control system may employ a 1:1 redundant system to synchronize central processing unit (CPU) data in memory, where memory is maintained in both the primary memory associated with the primary process controller and the secondary memory associated with the dedicated secondary process controller in the same manner using an initial memory transfer and then updating tracking changes to the primary memory image. As is known in the art, a memory cache (cache) is a high-speed buffer used with a low-speed random access memory (RAM). A CPU without a cache expresses all changes (data and addresses) to the RAM bus where they are captured. A CPU with a write-through cache works in a similar manner. In contrast, a CPU with a cache commonly used in process control systems to respond quickly to process changes may not support a write-through cache.

[0004] Traditional high availability control system redundancy uses a primary controller with a 1:1 backup controller. Figure 1A conventional industrial control system 100 is shown including a redundant process controller arrangement shown as a redundant controller platform 1 120 and a redundant controller platform 2 130, each redundant controller platform having a pair of process controllers, wherein each process controller includes computing hardware 171 including at least one processor and associated memory 172. In the redundant controller platform 1 120, there is a primary controller 122 and a dedicated backup (redundant) secondary controller 127, and similarly, in the redundant controller platform 130, there is a primary controller 132 and a secondary controller 137. Both the redundant process controllers 1 120 and 2 130 are shown coupled via an input / output (I / O) mesh network 140 to control a processing device 160, which is shown coupled via field I / O devices 145 and field devices 150. Dedicated path networks 125 and 135 dedicated to data transmission to synchronize the controllers in each redundant controller pair are shown. Also shown is a plant-level network 170 (such as based on Ethernet cables) between the controllers 120 , 130 and one or more servers 180 , at least one human machine interface (HMI) 185 .

[0005] like Figure 1 The conventional embedded controller redundancy shown in the conventional industrial control system 100 in FIG. 1 increases the availability of controllers by deploying the controllers in any desired configuration, with one controller acting as a primary controller and another controller acting as a backup controller ready to assume process control in the event of a failure of the primary controller. Thus, Figure 1 The two redundant controller pairs 120 and 130 shown require a total of four controllers, each controller including dedicated computing hardware 171.

[0006] I / O gateway devices such as remote terminal units (RTUs) or edge gateways can also be configured with device redundancy. An RTU is a microprocessor-controlled electronic device that connects devices or equipment in the physical world to a distributed control system or supervisory control and data acquisition (SCADA) system. An edge gateway is a virtual router used to organize a virtual device context (VDC) network that can be configured to provide network services such as Dynamic Host Configuration Protocol (DHCP), firewalls, network address translation (NAT), static routing, virtual private networks (VPNs), and load balancing. Although technically speaking, I / O devices are different from RTUs, RTUs can include I / O devices as part of the assembly and can therefore be used as I / O devices. Compared to RTUs, I / O devices are generally simpler and do not have telemetry capabilities for remote independent data acquisition transmission. Summary of the invention

[0007] This Summary is provided to introduce a brief selection of disclosed concepts in a simplified form, which are further described below in the Detailed Description including the drawings. This Summary is not intended to limit the scope of the claimed subject matter.

[0008] The disclosed embodiments of the present invention recognize that an industrial control system including a redundant process controller arrangement, such as Figure 1 A significant problem with conventional industrial control systems 100 shown in FIG. 1 is that conventional redundant controllers each require a dedicated backup controller, wherein in the case of redundant process controllers, the backup controller is only used when the primary controller fails. Furthermore, if both the primary process controller and the backup process controller fail, the process control task is no longer possible, which is usually referred to as "loss".

[0009] The disclosed computing platform's M:N redundancy scheme allows for greater flexibility because, with respect to the controller, any active process controller in the control system can serve as a backup controller for another controller while continuing its own process control tasks to take over the process control tasks of the failed primary controller, so that explicit dedicated backup controller hardware including computing hardware is not required to implement the disclosed computing platform redundancy scheme. Therefore, with respect to the controller, even if there are multiple controller failures, the primary control tasks can still be maintained by transferring the workload to the available controller hardware. As used herein, also with respect to the controller, M is the number of primary controller instances / environments, and N is the number of additional controller platforms required to provide redundancy for M.

[0010] Although generally described herein with respect to process controllers, the disclosed hardware redundancy scheme may also be applicable to I / O gateways, such as the following Figure 6 Process controllers or I / O gateways can be embedded devices or virtual devices that host applications that perform functions including analog, digital or sequential control functions, I / O gateways, supervisory controls, advanced control applications, and user-written programs.

[0011] One embodiment disclosed herein includes a method of fault-tolerant process control, the method comprising configuring a networked process control system in an industrial process facility (IPF) comprising a plurality of computing platforms coupled to field devices at a field level through a plurality of I / O devices, the field devices being coupled to a processing device. The plurality of I / Os are configured to enable any of the I / Os to be coupled to any of the process controllers, thereby being not subordinately bound as would be conventionally configured in a conventional hierarchical manner.

[0012] A failure of a first process controller among multiple process controllers or a first I / O gateway among I / O gateways is detected, which can use known methods to detect device failures. These known device failure detection techniques include detecting network failures, diagnostic failures, loss of I / O paths, or watchdog timeouts. One of the process controllers or I / O gateways at another computing platform, while in service in its own task and also serving as a redundant backup device for another device, is put into service in a recovery step so as to also serve as a redundant controller or redundant I / O gateway of the first process controller or the first I / O gateway to control or receive data from a field device associated with the first process controller or the first I / O gateway. The redundant controller or redundant I / O gateway continues to work as the primary controller of its associated field device or the primary I / O gateway of its associated field device. The recovery step can be implemented through user interaction or can be fully automatically implemented by using software that organizes / orchestrate the primary and backup relationships and adjusts the relationships as needed.

[0013] One advantageous aspect of the disclosed embodiment is an I / O mesh network. An I / O mesh network is an I / O configuration in which the I / O is not connected to a specific process controller via a physical dedicated link, but is meshed with all controllers and I / O gateways in a group of controllers and I / O gateways at the channel level, so that any process controller or a group of I / O gateways in the group of controllers and I / O gateways can have flexible workload distribution to be able to perform operations on inputs and outputs in the I / O mesh network. The I / O mesh network supports flexible redundancy schemes. In short, the I / O mesh network is a shared I / O network in which multiple controllers and optional I / O gateways and multiple I / O modules and devices reside on the same network, so that I / O resources can be shared between process controllers and I / O gateways. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A conventional industrial control system is shown that includes redundant process controller arrangements that each include a primary controller and a dedicated redundant controller that are both coupled to field I / O and field devices to control a process installation.

[0015] Figure 2 An exemplary disclosed network control system is shown implementing the disclosed redundant controller without any dedicated auxiliary (redundant) hardware coupled to field I / O and field devices to control a processing device.

[0016] Figure 3 Reference Figure 2 The previous state in a controller has failed. The highlighted text describes how the control operation has changed, indicating a loss of controller redundancy, but not a loss of process control.

[0017] Figure 4 Reference Figure 3 , and assuming the availability of existing or newly added spare hardware in the depicted pool represented by the 5th controller on the right, where backups for the (currently non-redundant) controllers 1 and 4 can be automatically created on the spare hardware.

[0018] Figure 5 It is shown that by switching the primary role of controller 1 to new hardware, full and normal redundant operation has been restored with 4 controllers, all running a primary process control task, and all backed up by another platform. Assuming pre-installed availability of spare platforms on the pool, and assuming a controller fails, this scenario shows how full redundancy of M primary controllers can be restored with a total of N platforms, where N=M+1 and no hardware deployment.

[0019] Figure 6 An exemplary network control system disclosed in the present invention is shown, which implements the I / O gateway hardware and redundant controllers disclosed in the present invention shown as 4 computing platforms, which together implement a redundant I / O gateway device and three redundant controllers 1, 2 and 3. DETAILED DESCRIPTION

[0020] The disclosed embodiments are described with reference to the accompanying drawings, wherein the same reference numerals are used throughout the drawings to represent similar or equivalent elements. The drawings are not drawn to scale, and they are provided only to illustrate certain disclosed aspects. Several disclosed aspects are described below with reference to exemplary applications for illustration. It should be understood that many specific details, relationships, and methods are set forth to provide a complete understanding of the disclosed embodiments.

[0021] As used herein, IPF operates industrial processes involving tangible materials to which the disclosed embodiments are applicable. For example, oil and gas, chemical, beverage, pharmaceutical, pulp and paper, petroleum process, electrical, and water. IPF is different from a data processing system that only performs data manipulation.

[0022] The present disclosure eliminates the need for dedicated auxiliary hardware (e.g., RTU or edge gateway hardware) for backup controllers or dedicated backup I / O gateways. The present disclosure also allows for greater backup coverage. The concept disclosed in the present invention is to create at least one group of master process controllers or a group of I / O gateway devices that, in addition to running their main process control or I / O gateway device tasks (e.g., controllers controlling field devices such as actuators and I / O gateway devices reading data from sensing devices), are also configured to simultaneously serve as backups for other master controllers or other I / O gateway devices. Therefore, if one or more other master controllers or I / O gateway hardware nodes fail, there is the ability to run 2 or more simultaneous master controller or master I / O gateway device tasks.

[0023] Figure 2 An exemplary network control system 200 is shown that implements the redundant controllers disclosed in the present invention without any auxiliary hardware. There are only primary controllers, shown as controller 1 221, controller 2 222, controller 3 223, and controller 4 224, because each primary controller 221, 222, 223, 224 can be used as a backup controller for another primary controller when needed, so no auxiliary controller is required. By way of example only, controller 1 221 is used as a backup controller for controller 4 224, controller 2 222 is used as a backup controller for controller 1 221, controller 3 223 is used as a backup controller for controller 2 22, and controller 4 224 is used as a backup controller for controller 3 223. In Figure 2 Therefore, each master node having a master controller 221-224 also serves as a backup controller for another controller.

[0024] Each controller 221, 222, 223 and 224 includes computing hardware 171 and associated memory 172. The memory is used to host at least one software application, here the control software of the controller. In the case of an I / O gateway, the software application will be used for the communication of the I / O gateway.

[0025] There is a dedicated path redundant network 260 shown to provide a pool of hardware resources by connecting all controllers together, so that any of the multiple process controllers can be put into service of the failed controller in the recovery step while in service of its own task so as to also serve as a redundant controller to control the associated field devices. The dedicated path redundant network 260 is a high-speed network for enabling computing platforms to keep in sync with each other. As described above, the controller serving as a redundant controller continues to operate as the primary controller of its associated field devices, and the recovery step can be achieved through user interaction or can be automatically achieved by using a software component that organizes / orchestrate the primary and backup relationships and adjusts the relationship as needed.

[0026] In another arrangement, any one of the process controllers connected in a group through the dedicated path redundant network 260 may support two or more controllers. Figure 2 This arrangement shown or related disclosed arrangements also allow hot standby hardware devices to be allocated in a controller group, wherein the standby controller is not currently running the main control task unless / until the main controller fails, in which case its control workload is transferred (or handed over) to the standby controller. Standby devices are not required, but can be added on a customer basis if the customer requires additional availability. If a controller or I / O gateway node fails, the controller or I / O gateway device that is backing up the failed controller or I / O gateway device node runs its own and other node process control or I / O gateway tasks simultaneously. Then, another node will back up this device for the control or I / O gateway task it just executed, so that its main control or main I / O gateway tasks are backed up by controllers or I / O gateway devices on other nodes.

[0027] There are many possible design approaches for implementation, some examples of which are provided for redundant process controller implementation. For example, four options are listed below:

[0028] 1. Create a software-based abstraction layer (or software container) that encapsulates controller functionality, where the rest of the system (including configuration tools and HMI 185) views each controller as an "application module" (or AM) hosted by but not connected to the controller hardware platform that includes computing hardware 171 or address.

[0029] 2. In conjunction with the design of Option #1, create a low-level software / firmware layer in the controller platform that allows it to host multiple AMs (or software containers) simultaneously and take into account the physical resource profile of each AM (including memory, central processing unit (CPU), etc.).

[0030] 3. Use a dedicated path network that connects all controller platforms into a group (for redundancy and synchronization of data services).

[0031] 4. A design that ensures that the redundant state information of each AM is considered a property of the AM, not the host platform.

[0032] Figure 2 The arrangement shown in provides an availability profile similar to traditional controller redundancy, but eliminates the need for Figure 1 The requirements for dedicated hardware for any dedicated backup controllers are shown. In normal operation, Figure 2Each controller in the deployment acts as both the active controller and the backup controller for the other controller. When a failure is detected in any controller, its partner (backup) controller assumes the role of dual-active controller. In a deployment, the total number of deployed controllers is Figure 1 half of the baseline case shown in . Figure 2 In the deployment, four controllers (with Figure 1 The present invention discloses an enhanced function that enables each controller to act as both a primary controller and a backup controller at the same time, or to act as a dual-primary controller in the event of a failure of another controller. Therefore, for the same number of controller hardware, users can configure four redundant controllers instead of the above Figure 1 Two controllers of a conventional arrangement are shown.

[0033] The concepts disclosed in the present invention generally rely on having an IO mesh network 140 such that the IOs are not conventionally subordinate to any one particular node in a hierarchical manner. As mentioned above, an I / O mesh network is a relatively new concept in which the I / Os are not connected to one specific process controller via a physical dedicated link, but are meshed with all controllers in the I / O group at the channel level so that any one process controller can perform operations on the inputs and outputs in the I / O mesh network. The I / O mesh network supports flexible redundancy schemes.

[0034] This form of controller / node redundancy disclosed in the present invention represents cost savings (no unused hardware), simpler deployment options and higher availability (multiple backup capabilities are available, not just one specific node). This design disclosed in the present invention utilizes today's 1:1 redundancy methods to achieve the core redundant behaviors of initial synchronization, synchronization maintenance and failover processing. Once a viable backup is determined from the master node pool, initial synchronization can be performed, and trickle synchronization disclosed in U.S. Patent 6,170,044 entitled "Systems and Methods for Synchronizing Redundant Controllers with Minimal Control Disruption" can be used to transfer the complete database in an atomic manner while running control tasks, so there is no interruption or delay to the master control. Synchronization maintenance is performed in a standard 1:1 redundancy manner, and the failover method is performed in a manner similar to 1:1, where the node backing up the master controller continuously checks the health of its partner and synchronizes data from the last clear point to initiate failover / I / O processing / control execution.

[0035] The new features disclosed herein are simultaneous execution of primary control, the ability to back up one or more other primary controllers, and the ability to run two simultaneous control tasks that are independent of each other and have appropriate control performance. The embodiments disclosed in the present invention are generally applicable to any process controller. For example, EXPERION process controllers (e.g., C300), ControlEdge PLCs, and I / O gateway devices (such as RTU2020) are all from Honeywell International Inc. The embodiments disclosed in the present invention may also be applicable to the evolution of Honeywell's Enhanced High Performance Process Manager (EHPM), once it adopts an I / O mesh architecture. EHPM is a process connection device on the Enhanced Universal Control Network (EUCN) for providing supervisory control and sequential operations.

[0036] Figure 3 , Figure 4 and Figure 5 An exemplary controller failure scenario is shown, which demonstrates the self-recovery potential of the disclosed M:N redundancy scheme. Figure 2 , where the controllers are again shown as 221 , 222 , 223 and 224 , depicts controller failure scenarios and recovery. Figure 3 Reference Figure 2 , where one of the process controllers, shown as controller 221, has failed. The text indicates how the control operation has changed, indicating a loss of redundancy, but not a loss of control. Figure 4 Reference Figure 3 , and assuming controller availability of existing or newly added standby controller hardware represented by the 5th controller on the right shown as controller 229 on the controller group connected by the dedicated path redundant network 260, backups for (currently non-redundant) controllers 1 221 and 4 224 can be automatically created on the standby controller hardware. Figure 5 It is shown that full and normal redundant operation has been restored with 4 controller platforms, each running a primary control task, and all backed up by another controller platform, by switching the primary role of controller 1 221 to new hardware shown as controller 229 in the event of a failure. Assuming pre-installed availability of spare platforms on the controller group connected by a dedicated path redundant network 260, and assuming a controller fails, this scenario shows how full redundancy of M primary controllers can be restored with a total of N controller platforms, where N=M+1 and no hardware deployment.

[0037] Figure 6An exemplary network control system 600 disclosed in the present invention is shown, which implements the redundant I / O gateway and redundant controller disclosed in the present invention, which are shown as 4 computing platforms 621, 622, 623 and 624, which together implement a redundant I / O gateway device and three redundant controllers 1, 2 and 3. Each computing platform includes computing hardware 171 and memory 172 for simultaneously supporting at least one process controller and another process controller or I / O gateway, wherein the computing platforms are again coupled to each other through a dedicated path redundant network 260 to provide a hardware resource pool. As known in the art, an I / O gateway is a data acquisition device that provides communication between entities on two different networks (usually an I / O network and an in-plant network), thereby providing a communication path for controllers on the plant-level network 170 to communicate with I / O devices on the I / O network.

[0038] The I / O gateway may potentially perform other functions, including control capabilities similar to those provided by a controller, thereby performing the dual roles of an I / O network gateway and a controller. All redundant entities are represented as two entities existing on different physical computing platforms: a primary entity and a backup entity. Computing platform 621 hosts a primary I / O gateway and a backup controller 3. Computing platform 622 hosts a primary controller 1 and a backup I / O gateway. Computing platform 623 includes a primary controller 2 and a backup controller 1. Computing platform 624 includes a primary controller 3 and a backup controller 2. Although computing platforms 623 and 624 are not shown directly coupled to the I / O mesh network 240, they may be coupled to field I / O 145, field devices 150, and processing devices 160 through computing platforms 621 and 622, or coupled through a separate I / O mesh network (not shown).

[0039] Although various disclosed embodiments have been described above, it should be understood that they are presented by way of example only and not limitation. Many changes may be made to the subject matter disclosed herein in accordance with the present disclosure without departing from the spirit or scope of the present disclosure. For example, expansion to other automation settings, including building controllers. In addition, although specific features may be disclosed with respect to only one of several embodiments, such features may be combined with one or more other features of other embodiments, as may be desired or advantageous for any given or particular application.

Claims

1. A method for fault-tolerant process control, comprising: A network process control system (600) is provided in an industrial process facility (IPF) comprising a plant-level network (170), the plant-level network coupling at least one server (180) to a plurality of computing platforms (621-624), the plurality of computing platforms each comprising computing hardware (171) and memory (172) for hosting at least one software application for simultaneously supporting one or more devices, wherein the one or more devices include a process controller and a backup process controller or an input / output (I / O) gateway, the plurality of computing platforms being coupled to each other via a dedicated path redundant network (260) to provide a pool of hardware resources, wherein the backup process controller is a backup of a process controller hosted on another computing platform; wherein at least some of the plurality of computing platforms are directly coupled to a plurality of I / O devices (145) via an input / output (I / O) mesh network (240), the plurality of I / O devices are coupled to field devices (150), and the field devices are coupled to a processing device (160); detecting at least one failed device hosted in one of the plurality of computing platforms, and using the dedicated path redundant network, placing another device at another computing platform of the plurality of computing platforms into service as a backup for the failed device, wherein the another computing platform is different from the computing platform, wherein the another device provides another task as a process controller or as an I / O gateway both before and after the placing into service, and An initial synchronization is performed after the putting into service, and then a synchronization slower than the initial synchronization is performed to transfer the complete database in an atomic manner when running as the backup.

2. The method of claim 1, wherein the servicing is performed automatically.

3. The method of claim 1, wherein the plurality of computing platforms exclusively support the process controller. 4 . The method of claim 1 , wherein the plurality of computing platforms collectively support both the process controller and the I / O gateway.

5. The method of claim 1, wherein a speed of the dedicated path redundant network is greater than a speed of the factory-level network.

6. A computer platform arrangement for a networked process control system (600) in an industrial process facility (IPF), comprising: a plurality of computing platforms (621-624), each of the plurality of computing platforms comprising computing hardware (171) and memory (172) for hosting at least one software application for simultaneously supporting one or more devices, wherein the one or more devices include a process controller and a backup process controller or an input / output (I / O) gateway, wherein the backup process controller is a backup of a process controller hosted on another computing platform; a dedicated path redundant network (260) coupling the plurality of computing platforms together to provide a hardware resource pool; wherein when at least one failed device in the hardware resource pool is detected, another device at another computing platform among the plurality of computing platforms is put into service as a backup of the failed device through the dedicated path redundant network, wherein the another computing platform is different from the computing platform, wherein the another device provides another task as a process controller or as an I / O gateway before and after being put into service, and Wherein an initial synchronization is performed after the putting into service, and then a synchronization slower than the initial synchronization is performed to transfer the complete database in an atomic manner when running as the backup.

7. A fault-tolerant networked process control system (600) in an industrial process facility (IPF), comprising: a plant-level network (170) coupling at least one server (180) to a computer platform arrangement including a plurality of computing platforms (621-624), each of the plurality of computing platforms including computing hardware (171) and memory (172) for hosting at least one software application for simultaneously supporting one or more devices, wherein the one or more devices include a process controller and a backup process controller or an input / output (I / O) gateway, the computing platforms coupled to each other via a dedicated path redundant network (260) to provide a pool of hardware resources, wherein the backup process controller is a backup of a process controller hosted on another computing platform; wherein at least some of the plurality of computing platforms are directly coupled to a plurality of I / O devices (145) via an input / output (I / O) mesh network (240), the plurality of I / O devices are coupled to field devices (150), the field devices are coupled to a processing device (160), wherein the computer platform arrangement is configured to: detecting at least one failed device hosted in one of the plurality of computing platforms, and using the dedicated path redundant network to put another device at another computing platform of the plurality of computing platforms into service as a backup for the failed device, wherein the another computing platform is different from the computing platform, wherein the another device provides another task as a process controller or as an I / O gateway both before and after the putting into service, and An initial synchronization is performed after the putting into service, and then a synchronization slower than the initial synchronization is performed to transfer the complete database in an atomic manner when running as the backup.

8. The control system of claim 7, wherein the servicing is performed automatically.

Images