Method and apparatus for enhanced diagnostic coverage of a slave device in a redundant controller pair

Abstract

A method used by a master device associated with a slave device in a redundant pair, the master device issuing a synchronization request to its control database, causing the master device to send its own trace memory file to the slave device to update the slave device's control database, and periodically sending cached changes made in the master device to the slave device upon request from the master device to update the slave device's control database. The slave device uses the updated control database to identify the I / O modules assigned to the slave device and the communication connections and paths of the peer device, and performs diagnostic tests on the communication connections and paths identified by the query, and sends diagnostic messages when a fault is detected in the identified communication connections and paths.

Description

Technical Field

[0001] This disclosure relates generally to industrial process control and automation systems. More specifically, this disclosure relates to a method and apparatus for enhanced diagnostic coverage of slave devices in a redundant controller pair using database knowledge of the master device. Background Technology

[0002] Industrial process control and automation systems are frequently used to automate large and complex industrial processes. These types of systems typically include various components such as sensors, actuators, and process controllers. Some process controllers receive measurements from sensors and generate control signals for the actuators. Failures in industrial control and automation systems, such as process controller failures, can lead to significant downtime. For example, restarting an industrial process typically incurs costs, as well as actual production losses due to the failure. Therefore, control and automation systems often include redundant control system components, such as redundant process controllers, which operate in a master / slave configuration. Redundant process controllers typically require supporting components and hardware to ensure that one cooperating device can take over control of operations in the event of a failure or otherwise offline status of another cooperating device.

[0003] In a traditional 1:1 redundant controller pair, the master device provides diagnostic assessments of its own hardware components, such as RAM, ROM, and network transceivers. The master device can also assess its full operational functionality through database knowledge of its own behavior, activities, and storage, including its ability to communicate with all I / O signals connected to the industrial process control and automation system, its peer-to-peer communication connections with other controllers, and assessments of CPU and memory availability. Like the master device, the slave device provides diagnostic assessments of its own hardware components; however, the slave device does not provide a comprehensive assessment of its operational functionality in terms of I / O signaling and peer-to-peer connections because it requires access to the master device's database knowledge.

[0004] Therefore, this disclosure teaches a system and method for providing enhanced diagnostic coverage of the operational functions of a slave device by using system knowledge derived from database knowledge of the master device. This enables deeper and more accurate redundancy decisions, allowing the slave device to perform the full functions of the master device once it advances to that state via command switching or failover due to a master device failure. Furthermore, the enhanced diagnostic assessment can inform plant personnel of potential faults in the functional operation of the slave device before it needs to perform the role of the master device. Summary of the Invention

[0005] This disclosure provides a method and apparatus for using database knowledge of a master device to achieve enhanced diagnostic coverage for slave devices in a redundant controller pair.

[0006] In a first embodiment, a method is disclosed for enhanced diagnostic coverage of a slave device associated with a redundant master device connected to multiple I / O modules via communication connections and paths in a communication network. The method includes synchronizing the master device's control database to the slave device by sending a request from the master device to transfer the master device's trace memory file storage to the slave device. This synchronization request causes a complete copy of the trace memory file storage data to be transferred to the slave device to update the slave device's control database. Trace changes made by the trace device driver to the master device's trace memory storage are written to a buffer to transfer trace data changes to the slave device to update the slave device's control database using any changes made to the master device's control database. The method further includes the slave device querying its control database to identify communication connections and paths assigned to the slave device's I / O modules, and performing diagnostic tests on the communication connections and paths identified by the query using communication diagnostics. The method further includes the slave device sending a diagnostic message on the communication network when a fault is detected in the communication connections and paths identified by the diagnostic tests.

[0007] In a second embodiment, an apparatus is disclosed for enhanced diagnostic coverage of slave devices associated with a redundant master device connected to multiple I / O modules via communication connections and communication paths of a communication network. The master device and slave device each include at least one processor and at least one memory, and at least one processor of the master device issues a synchronization request to the control database of at least one memory of the master device to transfer the master device's trace memory file storage to the slave device. The synchronization request causes a complete copy of the master device's trace memory file storage to be transferred to the slave device to update the slave device's control database. A trace device driver tracks changes made to the master device's trace memory storage and writes any data changes to a buffer to transfer trace data changes to the slave device to update the slave device's control database using any changes made to the master device's control database. Query software executed by at least one processor of the slave device queries the slave device's control database to identify communication connections and communication paths assigned to the I / O modules of the slave device. Communication path diagnostic software, executed by at least one processor of the slave device, performs diagnostic tests on the communication connections and paths identified in the query of the slave device's control database. When a fault is detected in the communication connection and path identified through diagnostic tests, the device sends a diagnostic message on the communication network.

[0008] In a third embodiment, a non-transitory computer-readable medium containing instructions that, when executed, cause at least one processor of a master device and at least one processor of an associated slave device connected to a plurality of I / O modules via communication connections and paths of a communication network to synchronize the master device's control database to the slave device by sending a request from the master device to transfer the master device's trace memory file storage to the slave device. The synchronization request causes a complete copy of the trace memory file storage data to be transferred to the slave device to update the slave device's control database. The master device tracks changes made to the master device's trace memory storage via a trace device driver and writes data changes to a buffer to transfer trace data changes to the slave device to update the slave device's control database using any changes made to the master device's control database. The slave device queries its control database to identify communication connections and paths assigned to the slave device's I / O modules, and performs diagnostic tests on the communication connections and paths identified by the query of the slave device's control database using communication diagnostics. When a fault is detected in the communication connections and paths identified by the diagnostic tests, the slave device sends a diagnostic message on the communication network.

[0009] Other technical features will be apparent to those skilled in the art from the following figures and description. Attached Figure Description

[0010] To gain a more complete understanding of this disclosure, reference is now made to the following description in conjunction with the accompanying drawings, in which:

[0011] Figure 1 An exemplary industrial process control and automation system according to this disclosure is shown;

[0012] Figure 2 An exemplary controller device for an industrial process control and automation system according to the present disclosure is shown;

[0013] Figure 3 An exemplary architecture is shown for implementing enhanced diagnostic coverage for slave devices in a redundant controller pair using database knowledge of the master device according to this disclosure; and

[0014] Figure 4 An exemplary process is shown for implementing enhanced diagnostic coverage for slave devices in a redundant controller pair using database knowledge of the master device according to this disclosure. Detailed Implementation

[0015] The following discussion Figures 1 to 4The various embodiments used to describe the principles of the invention in this patent document are merely illustrative and should not be construed as limiting the scope of the invention in any way. Those skilled in the art will understand that the principles of the invention can be implemented in any suitably arranged device or system.

[0016] As mentioned above, industrial process control and automation systems typically include redundant control system components, such as redundant process controllers. Redundant process controllers usually need to support both hardware and software redundancy to ensure that if one device in a process controller pair fails or otherwise goes offline, the other device can take over control operations. When using fault-tolerant control system components that include hardware and software redundancy, process control industry customers expect high reliability. To support high reliability, it is typically necessary to provide process data received or generated by the master device in a process controller pair to the slave devices in the pair. This allows the slave devices to continue providing control of the process in the event of a master device failure or other offline situation.

[0017] This disclosure provides techniques for supporting enhanced diagnostic coverage of slave devices by utilizing system knowledge from the master device's control database. The expanded diagnostic coverage enables deeper and more accurate redundancy decisions, as well as reporting faults to plant personnel before the slave device needs to perform the master device's role. The enhanced slave device diagnostic coverage independently ensures that the slave device can perform the full functionality of the master device when it advances to that state via command switching or a failover due to a master device failure.

[0018] Figure 1 An exemplary industrial process control and automation system 100 according to this disclosure is shown. Figure 1 As shown, system 100 includes various components that facilitate the production or processing of at least one product or other material. For example, system 100 can be used to facilitate the control of components in one or more industrial plants. Each plant represents one or more processing facilities (or one or more portions thereof), such as one or more manufacturing facilities for producing at least one product or other material. Generally, each plant can implement one or more industrial processes and can be individually or collectively referred to as a process system. A process system typically refers to any system or portion thereof configured to process one or more products or other materials in a certain way.

[0019] exist Figure 1In the example shown, system 100 includes one or more sensors 102a and one or more actuators 102b. Sensors 102a and actuators 102b represent components in a process system capable of performing any of a variety of functions. For example, sensor 102a may measure various characteristics of the process system, such as temperature, pressure, or flow rate. Additionally, actuators 102b may alter various characteristics of the process system. Each sensor in sensor 102a includes any suitable structure or field device for measuring one or more characteristics of the process system. Each actuator in actuator 102b includes any suitable structure or field device for operating or influencing one or more conditions in the process system.

[0020] At least one input / output (I / O) module 104 is coupled to sensor 102a and actuator 102b. I / O module 104 facilitates interaction with sensor 102a, actuator 102b, or other field devices (not shown). For example, I / O module 104 can be used to receive one or more analog inputs (AI), digital inputs (DI), digital input event sequences (DISOE), pulse accumulator inputs (PI), or other inputs from one or more field devices. I / O module 104 can also be used to provide one or more analog outputs (AO), digital outputs (DO), or other outputs to one or more field devices. Each I / O module 104 includes any suitable structure for receiving one or more input signals from one or more field devices or providing one or more output signals to one or more field devices.

[0021] System 100 also includes various controllers 106. Controllers 106 can be used in system 100 to perform various functions to control one or more industrial processes. For example, a first set of controllers 106 can use measurements from one or more sensors 102a to control the operation of one or more actuators 102b. These controllers 106 can interact with sensors 102a, actuators 102b, and other field devices via I / O modules 104. A second set of controllers 106 can be used to optimize the control logic or other operations performed by the first set of controllers.

[0022] At least one pair of controllers 106 in system 100 may be used as a redundant pair of process controllers. For these controllers 106, it is common for one controller 106 to operate as a master device in a primary role, where controller 106 is receiving process data (such as from one or more sensors 102a), performing calculations, and generating outputs (such as one or more control signals from one or more actuators 102b). The other controller 106 typically operates as a slave device in a secondary or backup mode, where slave device 106 receives data received or generated by master device 106 and is ready to take over if master controller 106 fails or otherwise goes offline. It should be noted that one, some, or all of the controllers 106 in system 100 may have associated redundant controllers.

[0023] Each controller 106 includes any suitable structure for controlling one or more aspects of an industrial process. For example, at least some of the controllers in controller 106 may represent proportional-integral-derivative (PID) controllers or multivariable controllers, such as Profit controllers, or other types of controllers that implement model predictive control (MPC) or other advanced predictive control. As a specific example, each controller 106 may represent a computing device running a real-time operating system, a Windows operating system, or another operating system.

[0024] At least some of the controllers in controller 106 can interact with field devices via I / O modules 104 by communicating on at least one I / O network 108. I / O network 108 typically represents any suitable network configured to transfer data between field devices and controllers or other control system components. For example, I / O network 108 may represent at least one Ethernet network (such as an Ethernet network supporting the Foundation Fieldbus protocol), an electrical signal network (such as an addressable remote sensor high-speed channel or the "HART" protocol), a pneumatic control signal network, or any other or additional type of network.

[0025] At least some of the controllers in controller 106 can also interact with I / O module 104 by communicating over at least one control network 110. Control network 110 generally refers to any suitable network configured to transfer data between controller 106, I / O module 104, and operator station 112. Additionally, control network 100 can also transfer data between supervisory controllers, history databases, wireless nodes, or other components connected to system 100. Figure 1 (Not shown in the image). For example, control network 110 can represent an Ethernet network, a redundant pair of Ethernet networks (such as a fault-tolerant Ethernet network from HONEYWELL INTERNATIONAL INC.), or any other or additional type of network.

[0026] Operator access to and interaction with the controller 106 and other components of system 100 can be performed via individual operator stations 112. Each operator station 112 can be used to provide and receive information from the operator. For example, each operator station 112 can provide the operator with information identifying the current state of the industrial process, such as the values ​​of various process variables and warnings, alarms, or other states associated with the industrial process. Each operator station 112 can also receive information that affects how the industrial process is controlled, such as by receiving setpoints for process variables controlled by controller 106 or receiving other information that changes or affects how controller 106 controls the industrial process. Each operator station 112 includes any suitable structure for displaying information to the operator and interacting with the operator.

[0027] Multiple operator stations 112 may be grouped together and used in one or more control rooms 114. Each control room 114 may include any number of operator stations 112 arranged in any suitable manner. In some embodiments, such as when each control room 114 contains operator stations 112 for managing separate sections of an industrial plant, multiple control rooms 114 may be used to control the industrial plant.

[0028] Although Figure 1 An example of an industrial process control and automation system 100 is shown, but it is possible to... Figure 1 Various modifications can be made. For example, system 100 may include any number of sensors, actuators, I / O modules, controllers, networks, operator consoles, control rooms, and other components. Additionally, Figure 1 The composition and arrangement of System 100 shown are for illustrative purposes only. Components may be added, omitted, combined, further subdivided, or placed in any other suitable configuration as needed. Furthermore, specific functions have been described as being performed by specific components of System 100. This is for illustrative purposes only. Generally, control systems and automation systems are highly configurable and can be configured in any suitable manner as needed.

[0029] In systems such as system 100, redundant controller 106 requires an efficient way to enable slave devices in a controller pair to assess their full operational capabilities. The master device assesses its full operational capabilities through its own behavior and activities, such as its ability to communicate with all I / O signals, peering connections with other controllers, network connectivity, and CPU and memory availability. During the operation of the master device, its control database is indirectly updated. However, slave devices cannot fully assess their operational capabilities and capabilities using only their own control database because they lack the knowledge acquired by the master device during its operation. As described in more detail below, a system and method for transferring control database knowledge from the master device to the slave device are described, allowing the slave device to assess its operational capabilities using the same data and system knowledge as the master device, and reporting inconsistencies as faults to plant personnel before the slave device needs to perform the master device's role.

[0030] Figure 1 An example of an operating industrial process control and automation system environment is shown, which can be used to implement an arrangement that allows slave devices in a controller pair to assess their full operational capabilities within its operating environment. The disclosed arrangement can be used in any other suitable system, which may or may not involve industrial process control and automation.

[0031] Figure 2 An exemplary device 200 representing controller 106 is shown. Device 200 may, for example, represent... Figure 1 The device 200 may be any of the controller 106 or other control system components used in the redundant configuration. However, the device 200 may represent any other suitable device that supports operation in a redundant manner, whether or not the device 200 is used for process control and automation.

[0032] like Figure 2 As shown, device 200 includes at least one processor 202, at least one storage device 204, at least one communication unit 206, and at least one I / O unit 208. Each processor 202 is executable with instructions, such as those that can be loaded into memory 210. Each processor 202 represents any suitable processing device, such as one or more microprocessors, microcontrollers, digital signal processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or discrete circuits.

[0033] Memory 210 and persistent storage device 212 are examples of storage device 204, which represents any structure capable of storing information (such as data, program code, and / or other suitable temporary or permanent information) and facilitating the retrieval of that information. Memory 210 may represent random access memory or any other suitable volatile or non-volatile storage device. Persistent storage device 212 may include one or more components or devices supporting longer-term storage of data, such as read-only memory, hard disk drive, flash memory, or optical disk.

[0034] Communication unit 206 supports communication with other systems or devices. For example, communication unit 206 may include at least one network interface card or wireless transceiver, thereby facilitating communication via at least one wired or wireless network. As a specific example, communication unit 206 may support communication with one or more sensors 102a or one or more actuators 102b via I / O network 108. As another specific example, communication unit 206 may support communication with higher-level components via control network 110. Communication unit 206 can support communication via any suitable physical or wireless communication link.

[0035] I / O unit 208 allows for data input and output. For example, I / O unit 208 can provide connectivity for user input via a keyboard, mouse, keypad, touchscreen, or other suitable input device. I / O unit 208 can also send output to a display, printer, or other suitable output device. However, it should be noted that I / O unit 208 for local I / O may not be necessary when device 200 is accessible locally or remotely via a network connection.

[0036] As described in more detail below, the processor 202 of device 200 can be used to execute a diagnostic system that tests the operational functions of device 200. The processor 202 of device 200 can also be used to execute algorithms that support the transfer of data from device 200 to redundant devices (e.g., to an associated redundant process controller 106).

[0037] Although Figure 2 An example of a device 200 for implementing this disclosure is shown, but other devices may be used. Figure 2 Make various changes. For example, make them combinable, further subdivided, or omitted. Figure 2 The computing device comprises various components, and additional components can be added as needed. Furthermore, the computing device can be configured in many ways, and... Figure 2 This disclosure is not intended to be limited to any particular configuration of the device.

[0038] Figure 3An exemplary architecture 300 is illustrated for implementing a system according to this disclosure for transferring control knowledge from a master device control database to a slave device control database. For ease of explanation, architecture 300 can be described as a system in which controllers such as... Figure 1 The master and slave devices of controller 106 in system 100 are implemented within the system 100. Using... Figure 2 The device 200 implements the controller 106. However, Figure 3 The architecture 300 shown can be used with any suitable device and in any suitable system.

[0039] In this example, architecture 300 includes a processor 307 that executes one or more control algorithms 308 stored in a control database 306 representing memory locations in the main writable memory of persistent storage device 212 or other storage device 204. Each control algorithm 308 can be used to control one or more aspects of at least one industrial process. Each control algorithm 308 typically includes or has access to an associated tracking memory 310, which may represent a memory location in memory 212 or other storage device 204. One operation of at least one control algorithm 308 is to track and send data to redundant devices, which in this example occurs via communication path 312. Communication path 312 may represent a direct connection to redundant devices or an indirect path such as via a network.

[0040] Redundancy tracking software 314 is used to facilitate data exchange with slave devices. Figure 3 The same architecture 300 shown is used in both the master and slave devices. To support transparent identification of changes to the tracking memory 310, architecture 300 includes a tracking device driver 316. The tracking device driver 316 includes a tracking memory file store 318, which can represent memory locations in memory 210, persistent storage 212, or other storage devices 204. When the control algorithm 308 is executed, algorithm state data is stored in the tracking memory 310 and in the control database 306. Any changes to the tracking memory 310 are updated in the tracking memory file store 318 by the tracking device driver 316. To support data transfer to redundant devices, architecture 300 includes redundant tracking software 314. Periodically or on demand, the redundant tracking software 314 requests the control database 306 to perform synchronization, which sends data from the control database and change buffer 320 to the cooperating device.

[0041] Upon receiving a synchronization command, the control database 306 provides the tracking device driver 316 with a list of pages written to the tracking memory 310 since the last synchronization command. The tracking device driver 316 compares the list of blocks to be written with its tracking memory file storage 318, generates a complete set of changes between the two, and writes the changes to the change buffer 320. While changes are generated and stored in the change buffer 320, the tracking device driver 316 can update its tracking memory file storage 318. The contents of the tracking buffer 320 are sent to the slave device via communication path 312 and decapsulated into the slave device's tracking memory file storage 318. After a specified number of changes have been identified, after a specified amount of time has elapsed, or at any other suitable time, the change buffer 320, or a portion thereof, can be periodically sent to the slave device. In this way, the slave device can obtain data changes made by the master device's control database. Using the data changes transmitted from the master device, the slave device can update its own memory.

[0042] In this example, control algorithm 308 can be used to support various functions. These functions include reading data from files on one or more physical or virtual storage devices 204 and writing data to files. These functions also include mapping data of files stored on storage device 204 to any user process's virtual memory address space. These functions further include providing a trace memory 310 for at least one control algorithm 308 mapped to file system 306. Additionally, these functions include receiving synchronization requests from redundant trace software 314 and committing all changes in trace memory 310 to storage device 204.

[0043] Although Figure 3 An example of an architecture 300 for implementing process control redundancy is shown, but it is possible to modify it further. Figure 3 Various changes can be made. For example, the functions in architecture 300 can be varied as needed or desired, such as based on the specific operating system used in the device. Moreover, algorithm 308 does not need to involve industrial process control and automation functions.

[0044] Once the slave device's control database is fully synchronized with the master device's data, as explained above, the slave device now possesses enhanced knowledge to extend its diagnostic coverage to all devices connected to the master device. This may include, for example, network I / O modules and controlled devices assigned to the master and slave devices in a redundant pair, as well as network I / O modules and controlled devices assigned to any peer-to-peer network connections between the redundant pair and any other controllers or other controller devices in the industrial process control and automation system. During the synchronization and updating of the slave device's control database, the slave device queries its associated control database 306 to find all connections to the I / O modules assigned to the master and slave devices in the redundant pair and / or other nodes containing I / O modules. The processor 307 then executes the communication path diagnostic algorithm 325 to establish non-control communication connections with the I / O module 104 and periodically tests the connections. This testing is performed in a manner that does not affect the primary control relationship or timing between the controller 106 and the associated I / O module 104. The testing determines that a reliable communication path to the I / O module 104 is available. Communication path testing can identify problems in the communication network 110, such as communication bottlenecks, based on current communication bus busy messages, device timeouts, receive (Rx) timeouts, and / or alarm and / or event failures.

[0045] Similarly, using the communication path diagnostics 325, slave devices will also examine peer connections, such as connections to other controllers in the system, and will also test communication paths to peer nodes without affecting control performance. Other diagnostics enabled by this concept include third-party device communication paths to third-party wireless nodes, such as communication networks, database integrity (valuable for software migration cases), and connectivity of supervisory controllers connected to industrial process control and automation systems.

[0046] Any faults and diagnostic data detected by communication path diagnostics 325 will be sent to availability manager 330 for review. Availability manager 330 logs faults and their diagnostic data as diagnostic events and prioritizes detected faults based on critical status. Prioritizing detected faults may involve organizing the collected faults so that the most important (e.g., urgent) faults can be resolved first. For example, the collected diagnostic data may include multiple anomalous parameters. Diagnostic data containing anomalous parameters that significantly exceed (e.g., are excessively below or above) threshold limits may be listed first to determine the order. Detected faults reported as existing may also be verified to ensure their existence. Verifying the diagnostic data of detected faults may include comparing the collected diagnostic data with past data, parameter settings, and the functionality of the network connection to the redundant controller 106.

[0047] Minor events will be logged to an event log and online diagnostic summary, and reported by the slave device to alert users by sending diagnostic messages along the plant network 110. Alerting users may include alerting (e.g., notifying) users via dashboards, mobile devices, user interfaces, or reports. For example, alert messages may be provided (e.g., displayed and / or presented) to a remote operator (e.g., an expert, user technician) at an associated operator station 112 connected to the plant network 110. Once alerted, the technician will repair the fault before the slave device needs to complete the master device's functionality. However, embodiments of this disclosure are not limited to this. For example, diagnostic messages may be provided to any person and / or entity responsible for diagnosing, repairing, and / or resolving anomalies associated with the automation system, and / or any person and / or entity responsible for diagnosing and / or improving the operation of the field automation system.

[0048] In some implementations, diagnostic messages reported by the availability manager of the slave device may include parameters (e.g., field parameters) and diagnostic data associated with the control system network associated with the slave device. A set of field parameters may include information associated with the control system network, such as system configuration. For example, a set of parameters may include polling frequency and value change delay. The collected diagnostic data may include information related to a set of parameters. For example, the collected diagnostic data may include the sampling frequency of points (e.g., signal processing, continuous signal, discrete signal), upload frequency, number of points and / or configuration parameters, polling frequency, value changes, and other data. For certain emergency failures, such as the detection of a missing communication path to I / O module 104, the availability manager 330 will cause the slave device to abandon synchronization with the master device and send an emergency diagnostic message of the failure to the operator to repair the communication path. For other conditions, such as when only a portion of the communication path is faulty, the slave device will maintain synchronization with the master device and events recorded in the event log.

[0049] Figure 4 The disclosed implementation scheme is shown. Figure 3 The disclosed architecture 300 implements an exemplary process 400 in which enhanced diagnostic coverage is performed by the slave device by using system knowledge from the control database of the master device.

[0050] At step 401, during the initial setup, the control algorithm 308 on the master device sends a request to the control database 306 of the master device to open a read / write memory mapping view of the trace memory 310 in the address space of the control algorithm 308. The control algorithm 308 will use this view to read and write status data from its main writable memory.

[0051] At step 405, the redundancy tracking software 314 of the master device requests the control database 306 of the master device to open a read-only memory mapping view of the tracking memory file store 318 in the address space of the redundancy tracking software 314. The redundancy tracking software 314 will use this view to transfer an initial copy of the tracking memory file store 318 to the slave device.

[0052] Next, at step 410, the redundancy tracing software 314 on the slave device requests the slave device's control database 306 to open a read / write memory mapping view of the trace memory file store 318 in the address space of the slave device's redundancy tracing software 314. The slave device's redundancy tracing software 314 will use this view to copy changes provided by the master device to the slave device so as to maintain an exact copy of the master device's trace memory file store 318 in the slave device.

[0053] In order for the tracking device driver 316 to capture writes in the tracking memory file store 318, at step 415, the master device's control database 306 is commanded to perform synchronization to update all writes performed in the mapped tracking memory 310 to the tracking memory file store 318, and the tracking device driver 316 is commanded to begin monitoring changes to the tracking memory file store 318. The synchronization command helps align the tracking memory file store 318 with all updates, so that once the tracking device driver 316 is enabled, it can use the tracking memory file store 318 to detect all future updates. During startup initialization, the tracking device driver 316 ensures that the change buffer 320 has no entries.

[0054] Prior to the initial synchronization operation, the master and slave devices are not a synchronization pair. To become a synchronization pair, the two devices can perform an initial synchronization operation at step 420. The redundant tracking software 314 on the master device sends a synchronization request to the master device's control database 306 to transfer the master device's tracking memory file storage 318 to the slave device. Initial synchronization typically requires transferring a complete copy of the tracking memory file storage 318 from the master device to the slave device.

[0055] At step 425, once the initial synchronization is complete, the two devices represent a fully synchronized pair, and all that is needed is to continue sending changes from the master device's change buffer 320 to the slave device during the "synchronization maintenance" period. During this phase, the master device's redundant tracking software 314 traverses the tracking memory file store 318 (similar to the initial synchronization period described above) and calculates at least one checksum for at least one block of the tracking memory file store 318. At least one checksum is sent to the slave device to verify the slave device's copy of the tracking memory file store 318. The slave device's redundant tracking software 314 also receives a synchronization maintenance checksum on the current maintenance block of the memory, calculates the checksum for the corresponding block in the slave device's tracking memory 310, and compares the checksums to verify the integrity of the slave device's copy. If the checksums do not match, synchronization can be disconnected.

[0056] Once the slave device's control database is fully synchronized with the master device's database, as explained above, the slave device has the knowledge to extend diagnostic coverage to devices connected to controller pair 106. In step 430, the slave controller queries its control database 306 to find all connections to the I / O module 104 assigned to the controller pair and / or other nodes containing I / O modules. The processor 307 then executes communication path diagnostic algorithm 325 to establish non-control communication connections with I / O module 104 and periodically tests the connection in a manner that does not affect the primary control relationships or timing between the controller and its associated I / O modules. The tests determine that reliable communication paths, such as those from the plant control network 110 to I / O module 104, are valid and available.

[0057] In step 435, the device checks its control database 306 for peer-to-peer connectivity. The processor then executes a communication path diagnostic algorithm 325 to test communication paths, such as those from the factory control network 110 to peer nodes, without affecting control performance. Other diagnostics enabled by this concept include third-party device communication paths to third-party wireless nodes, such as communication networks, database integrity (valuable for software migration cases), and connectivity of supervisory controllers in industrial process control and automation systems.

[0058] In step 440, any faults and diagnostic data detected by communication path diagnostics 325 are sent to and reviewed by availability manager 330. Availability manager logs faults and their diagnostic data as diagnostic events and prioritizes detected faults based on critical status. Prioritizing detected faults may involve organizing collected faults so that the most important (e.g., urgent) faults can be resolved first. For example, collected diagnostic data may include multiple anomalous parameters. Diagnostic data containing anomalous parameters that significantly exceed (e.g., are excessively below or above) threshold limits may be listed first to determine the order. Detected faults reported as existing may also be verified to ensure their existence. Verifying diagnostic data for detected faults may include comparing collected diagnostic data with past data, parameter settings, and the functionality of the network associated with the controller.

[0059] The slave controller reports event logs and online diagnostic summaries to alert users by sending diagnostic messages along the plant network 110. User alerts may include warnings (e.g., notifications) to users via dashboards, mobile notifications, user interfaces, or announcements presented to remote operators (e.g., experts, user technicians) at associated operator stations 112 connected to the plant network 110. Once alerted, the technician will repair the fault before the slave controller needs to complete the main controller's functions.

[0060] although Figure 4 An example of a process for implementing a diagnostic assessment of the operational functions of a slave device operating in a redundant pair is shown, but further details can be made regarding... Figure 4 Various process changes can be made. For example, although it is shown as a series of steps, Figure 4 The steps shown can overlap, occur in parallel, occur in different orders, or occur multiple times. Furthermore, some steps can be combined or removed, and additional steps can be added as needed. Additionally, although process 400 is described in relation to architecture 300, the described architecture does not need to involve industrial process control and automation functions.

[0061] In some embodiments, the various functions described in this patent document are implemented or supported by a computer program, which is formed by computer-readable program code and embodied in a computer-readable medium. The phrase "computer-readable program code" includes any type of computer code, including source code, object code, and executable code. The phrase "computer-readable medium" includes any type of medium that can be accessed by a computer, such as read-only memory (ROM), random access memory (RAM), hard disk drive, optical disc (CD), digital video disc (DVD), or any other type of memory. "Non-transitory" computer-readable medium excludes wired, wireless, optical, or other communication links that transmit transient electrical signals or other signals. Non-transitory computer-readable medium includes media that can permanently store data as well as media that can store and subsequently rewrite data, such as rewritable optical discs or erasable memory devices.

[0062] It may be advantageous to define certain words and phrases used throughout this patent document. The terms “application” and “program” mean one or more computer programs, software components, instruction sets, processes, functions, objects, classes, instances, associated data, or portions thereof suitable for implementation in appropriate computer code (including source code, object code, or executable code). The term “communication” and its derivatives encompass both direct and indirect communication. The terms “comprising” and “including” and their derivatives mean, but are not limited to, this. The term “or” is inclusive, meaning and / or. The phrase “associated with” and its derivatives may mean, including, contained within, interconnected with, contained, contained in, connected to or connected with, coupled to or coupled with, able to communicate with, cooperate with, interleave, juxtapose, proximate, combine with or combine with, have, possess the attributes of, have a relationship with, or have a relationship with, etc. When used with a list of items, the phrase “at least one of” means that different combinations of one or more of the listed items may be used, and only one item in the list may be required. For example, "at least one of A, B and C" includes any of the following combinations: A, B, C, A and B, A and C, B and C, and A and B and C.

[0063] While this disclosure has described certain embodiments and generally associated methods, variations and substitutions of these embodiments and methods will be apparent to those skilled in the art. Therefore, the foregoing description of exemplary embodiments does not limit or restrict this disclosure. Other changes, substitutions, and modifications are possible without departing from the spirit and scope of this disclosure.

Claims

1. A method for enhanced diagnostic coverage of slave devices associated with a redundant master device connected to multiple I / O modules via a communication connection and communication path of a communication network, the method comprising: The master device synchronizes its control database to the slave device by sending a synchronization request to transfer the master device's tracking memory file storage to the slave device. The synchronization request causes a complete copy of the tracking memory file storage data to be transferred to the slave device to update the slave device's control database. The tracking device driver tracks data changes made to the tracking memory storage of the master device and writes the data changes to a buffer to transmit the tracking data changes to the slave device to update the slave device control database with any changes made to the master device control database; The slave device queries its control database to identify the communication connections and paths assigned to the I / O modules of the slave device; By using communication diagnostics, the slave device performs diagnostic tests on the communication connections and paths identified by the query; as well as When a fault is detected in the communication connection and path identified by the diagnostic test, the slave device sends a diagnostic message on the communication network.

2. The method according to claim 1, further comprising: The slave device queries its control database to identify peer communication connections and paths assigned to other controllers of the slave device; By using communication diagnostics, the slave device performs diagnostic tests on the peer communication connections and paths identified by the query; as well as When a fault is detected in the peer communication connection and path identified by the diagnostic test, the slave device sends a diagnostic message on the communication network.

3. The method of claim 2, wherein the master device uses redundant tracking software to transmit tracking memory file storage data and data changes to the slave device.

4. The method according to claim 1, wherein the master device is connected to the slave device via a direct communication connection.

5. The method of claim 1, wherein during synchronization maintenance, the tracking device driver tracks changes made to the tracking memory storage of the master device and writes the data changes to the buffer to transmit the tracking data changes to the slave device so as to update the slave device control database with any changes made to the master device control database, thereby maintaining synchronization between the slave device control database and the master device control database.

6. An apparatus for enhanced diagnostic coverage of slave devices associated with a redundant master device connected to multiple I / O modules via a communication connection and communication path of a communication network, the apparatus comprising: The master device and the slave device, each of the master device and the slave device includes at least one processor and at least one memory; as well as The at least one processor of the master device sends a synchronization request to the control database of the at least one memory of the master device to transfer the master device's trace memory file storage to the slave device. The synchronization request causes a complete copy of the master device's trace memory file storage to be transferred to the slave device to update the slave device's control database. A tracking device driver that tracks data changes made to the tracking memory storage of the master device and writes the data changes to a buffer to transmit tracking data changes to the slave device to update the slave device control database with any changes made to the master device control database; Query software executed by at least one processor of the slave device queries the control database of the slave device to identify communication connections and communication paths assigned to the I / O modules of the slave device; Communication path diagnostic software executed by at least one processor of the slave device performs diagnostic tests on the communication connections and paths identified in the query of the control database of the slave device; as well as When a fault is detected in the communication connection and path identified by the diagnostic test, the slave device sends a diagnostic message on the communication network.

7. The apparatus according to claim 6, wherein: The query software executed by the at least one processor of the slave device queries the control database of the slave device to identify peer communication connections and paths assigned to other controllers of the slave device; Diagnostic tests on the peer communication connections and paths identified by the query are performed by communication path diagnostic software executed by at least one processor of the slave device. as well as When a fault is detected in the peer communication connection and path identified by the diagnostic test, the slave device sends a diagnostic message on the communication network.

8. The apparatus of claim 7, wherein redundant tracking software executed by the master device is used to transmit tracking memory file storage data and data changes to the slave device.

9. The apparatus of claim 6, wherein the master device is connected to the slave device via a direct communication connection.

10. The apparatus of claim 6, wherein during synchronization maintenance, the tracking device driver performs tracking of changes made to the tracking memory storage of the master device and writes the data changes to the buffer to transmit the tracking data changes to the slave device so as to update the slave device control database with any changes made to the master device control database, thereby maintaining synchronization between the slave device control database and the master device control database.

Images