System Architecture for Safety Applications

Abstract

To provide a system architecture for safety applications.SOLUTION: Control systems for industrial machinery (e.g., robots) or other devices such as medical devices utilize a safety processor (SP) designed for integration into safety applications and computational components that are not necessarily safety-rated. The SP monitors performance of the non-safety computational components, including latency checks and verification of identical outputs. One or more sensors send data to the non-safety computational components for sophisticated processing and analysis that the SP cannot perform, but results of this processing are sent to the SP, which then generates safety-rated signals to the machinery or device being controlled by the SP. As a result, the system may qualify for a safety rating despite the ability to perform complex operations beyond the scope of safety-rated components.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims priority to and the benefit of U.S. Provisional Patent Application No. 62 / 811,070, filed February 27, 2019, which is incorporated by reference in its entirety herein.

[0002] The field of the invention relates generally to monitoring industrial environments where humans and machinery interact or are in close proximity, and specifically to systems and methods for detecting unsafe conditions within a monitored workspace. [Background technology]

[0003] Industrial machinery is often dangerous to humans. Some machinery is dangerous unless it is completely shut down, while other machinery can have various operating states, some of which are harmful and some of which are not. In many cases, the degree of harmfulness depends on the location of the human relative to the machinery. As a result, many "guarding" approaches have been developed to separate humans and machines and to prevent the machinery from harming humans.

[0004] One very simple and common type of guarding is a cage that surrounds the machinery, and is configured so that opening the cage door causes the electrical circuits to put the machinery into a safe state. The door may be positioned so that a person cannot reach the machine before it has been shut down. Naturally, this prevents all interaction between the person and the machine, or between the person and the workpiece the machine is processing, severely restricting the use of the workspace. Another simple guarding mechanism is a pressure-sensitive mat that is placed at a distance from the hazardous machinery, and stepping on the mat triggers a safe shutdown of the machinery.

[0005] More sophisticated types of protection may involve, for example, optical sensors such as light curtains that determine if any object has intruded into the monitored area, area scanners that optically detect intrusions, and 2D LIDAR sensors that use active optical sensing to detect the minimum distance to an obstacle along a series of light rays emanating from the sensor. More recently, systems have begun to employ 3D depth information, for example, using 3D time-of-flight sensors, 3D LIDAR, 3D radar, and stereo vision cameras. These sensors provide the ability to detect and locate intrusions into the area surrounding industrial machinery in 3D, providing additional flexibility and power in identifying and tracking objects within the work cell.

[0006] In industrial automation, systems intended to ensure the safety of humans working within the surrounding environment of an industrial process are known as "safety systems." These safety systems incorporate closed control loops with electronics and hardware associated with light curtains, switches on cage doors, emergency stop buttons, and other machine control inputs used as safety interlocks. For example, when a light curtain is triggered, it issues a breach signal to a safety controller. The safety controller responds by issuing a signal to the equipment guarded by the light curtain to shut down in a safe manner. The safety controller may include a "safety processor," i.e., a microprocessor, microcontroller, or other processor designed for integration into a safety system.

[0007] A safety system typically includes a set of redundant circuits that are separate from and isolated from the industrial control systems responsible for the associated industrial process. An alternative approach is to use a control device with an integrated safety system, in which the controller performs both standard and safety tasks, thereby making this unsuitable (for reasons discussed below) in particularly hazardous environments where high safety ratings are required. In addition, it is expensive and difficult to develop software for an integrated safety system, where that software controls both standard and safety tasks in a manner that meets stringent safety standards.

[0008] In their simplest implementation, safety systems are hardwired from switches and relays, including specialized safety relays that compare redundant signals and provide an internal check of machine and safety system status. A "safety relay" has two mechanically linked nodes, one normally open and one normally closed, and when the relay is toggled, the equipment is enabled. In safety-rated systems, two such safety relays are typically used per safety output in a duplex configuration, while the feedback input is single channel, providing redundant checking of the safe state of the output. More sophisticated systems employ a "safety controller" that receives signals from safety interlocks (such as light curtains), processes the incoming signals, and issues signals to equipment to slow or stop in a safe manner. Safety control can be implemented in a variety of ways and can be passive or active, depending on the control and safety complexity required.

[0009] Passive safety control is performed using safety relays, and active safety control is performed using a safety controller. The safety controller can be a programmable logic controller ("safety PLC") module or modules that meet the necessary criteria to qualify for a safety rating, or can be implemented as a more complex dedicated industrial computer. Under the direction of a stored safety control program, the safety controller computer examines a set of inputs that reflect the status of the controlled process and generates a set of safety outputs. The safety controller has at its core a safety processor that receives inputs from the safety controller and determines the safety outputs to the machinery being controlled.

[0010] The inputs and outputs of the safety controller can be binary (on or off) or analog within a continuous range. The inputs can be obtained from light curtains or other sensors attached to the industrial process equipment, and the outputs can be signals to power control relays, actuators, or motors on the equipment. The inputs and outputs can also come from safety input / output (I / O) modules that connect directly to the input sensors and output actuators and are used as a common interface to the safety controller. The safety I / O modules also provide monitoring functions and the ability to better troubleshoot the safety control system during installation and operation.

[0011] For an entire safety system to qualify for a safety rating, all of the elements of the safety system control loop (input sensors, output actuators, safety processor, and interconnections) must generally meet rating criteria, and the interconnections of the different elements and their operation must also comply with industrial safety standards. Because human safety is at stake, these standards are stringent and may dictate failure rates for hardware components and rigorous development practices for both hardware and software components.

[0012] A compliant system ensures that unsafe conditions can be detected with a very high probability, that failures of the system itself will be detected, and that the system will respond to the detected failures by transitioning controlled equipment to a safe state.

[0013] For example, the International Organization for Standardization (ISO) has developed the ISO 13849 standard (Safety of machinery: Safety-related parts of control systems). ISO 13849 provides safety requirements and guidance for the design of safety-related components. ISO 13849 provides guidance on the probability of failure of components and circuits as measured by performance levels (PL). Defined as the average probability of dangerous failure of a component per hour, PL ranges from a (lowest) to e (highest).

[0014] Similarly, the International Electrotechnical Commission (IEC) has developed the general IEC 61508 (Functional Safety of Electrical / Electronic / Programmable Electronic Safety-Related Systems) and the machinery-specific IEC 62061 (Safety of Machinery: Functional Safety of Electrical, Electronic, and Programmable Electronic Control Systems) standards. IEC 61508 and IEC 62061 specify four Safety Integrity Levels (SILs) of performance for safety functions. Safety systems with SILs of 2 (SIL2) and 3 (SIL3) generally require hardware redundancy for sensors, computing elements, and power supplies, and a safety overlay that transitions the system to a fail-safe state in the event of a safety failure. This increases expenses, and in addition to being more expensive, safety-rated hardware tends to lag significantly behind similar non-safety systems in functionality and performance.

[0015] An important variable in a safety system is the "safety response time", which is the amount of time that elapses between a safety-related event (e.g., an intrusion breaking through a light curtain) and when the equipment monitored by the safety system reaches a safe state. This safety response time is a function of several variables and depends on the safety system architecture and configuration. For example, in the simple case where a light curtain intrusion triggers a robot stop, this would be the sum of the time it takes for the light curtain to issue a signal to the safety controller following a detected breakthrough + the time it takes for the signal to reach the safety processor (data transmission latency) + the time it takes for the safety processor to issue a command to the robot (safety processor processing time) + the time it takes for the signal to progress to the robot emergency stop circuit + the time it takes for the robot to acknowledge the signal + the time it takes for the robot to stop from its moving state to a safe stopped state.

[0016] Each of these times depends on the interconnections and the logic instructions in the specific safety program. The controller reaction time is the worst case delay from any input change presented to the controller until the processed output is acknowledged and used to activate a safe state by the machinery being controlled. A properly designed safety system will have a timer to limit the safety task reaction time within acceptable limits. Exceeding this time limit will cause the timer to transition the safety system output to a safe state.

[0017] It is desirable to have the safety response time as fast as possible, which allows safety components such as light curtains to be installed close to the machinery. This is particularly useful when space on the factory floor is limited and there is a need to place protective equipment as close to the machinery as possible. Also, by minimizing this safety response time, there is a greater likelihood that the system will respond to a situation where the intrusion itself is fast moving (e.g., a person tripping into a hazardous work cell), and getting the machinery to a safe state as quickly as possible reduces the chance of injury.

[0018] It is also important that this safety reaction time does not vary between operating cycles or drift as the system is operating. A safety system needs to have a repeatable and reliable safety reaction time. This is important because the locations where protective equipment can be placed depend on the safety reaction time, which may be experimentally determined or calculated.

[0019] Once the guarding equipment is in place and the system is operational, variations or drifts in this safety reaction time can lead to violation of the minimum distance between guarding and equipment required by the safety rating standards. As sensing and computing equipment becomes more sophisticated and complex, it becomes increasingly more difficult for such equipment to meet the demanding safety rating standards. For example, 3D sensor systems offer the possibility of improved granularity in guarding systems. However, 3D sensor systems are more complex than 2D sensor systems, let alone simple intrusion detection mechanisms such as light curtains or pressure mats. First, a specific safety zone must be identified taking into account the harm posed by the machinery, the machinery's motion and trajectory, the possible actions of humans in the workspace, the workspace layout, and the location and field of view of each individual sensor. Especially when trying to ensure safety while optimizing floor space and system processing capacity, it can be difficult to calculate the optimal shape of the exclusion zone when one object may present an occlusion to the sensor and when several objects may be out of range or undetectable by the sensor. Also, the computation times required for 3D calculations are longer, and the variation in computation times depending on the potentially changing geometry or complexity of the workspace means that safety response times for systems using 3D sensors and computational elements are both longer and more variable than those of simpler safety systems using, for example, light curtains.

[0020] In extreme cases, the processing time involved in detecting and classifying elements in a scene is not only long, but can also vary from measurement cycle to measurement cycle as the environment changes. For example, in the case of a 3D sensing safety system, the more obstacles or occlusions present in the cell or the more cameras deployed, the longer the calculation time will be, which may correspond to the complexity of the observed scene. Since obstacles are dynamic and may change over time (e.g., as different numbers of people or objects enter and exit the work cell under observation), the calculation time will vary over time. As a result, it is not possible to meet safety standards that require upper limits on reaction times and / or limits on reaction time variance.

[0021] In short, in industrial safety, including applications involving robotics, good practice or formal safety regulations require that all components of a safety system be safety rated. Unfortunately, conventional safety processors cannot perform all the necessary computational tasks involved in safety monitoring of a work cell. Frequently, the safety processor lacks the processing power to apply the necessary algorithms with sufficient processing power to avoid compromising the safety requirements. Thus, there is a need for a safety architecture that can comply with stringent safety rating standards while performing complex operations and adapting to heterogeneous dynamic environments with unpredictable numbers of people and harm. Summary of the Invention [Means for solving the problem]

[0022] An embodiment of the present invention utilizes a safe processor (SP) and a non-safe multi-processor computing module (MPCM) (i.e., a computing module that includes more than one motherboard). A "non-safe" component means one that was not designed (e.g., according to a safety standard) for integration into a safe system. For example, the architecture may be a server-class computing platform that uses off-the-shelf multi-core CPUs and algorithms with indeterminate runtime but that are safe for industrial use. (As used herein, the term "CPU" or "processor" generally refers to a microprocessor, but in various embodiments, this may be a microcontroller, a peripheral integrated circuit element, a CSIC (customer specific integrated circuit), an ASIC (application specific integrated circuit), a logic circuit, a digital signal processor, a programmable logic device such as an FPGA (field programmable gate array), a PLD (programmable logic device), a PLA (programmable logic array), an RFID processor, a graphics processing unit (GPU), a smart chip, or any other device or arrangement of devices capable of implementing the process of the present invention.) The SP monitors the performance of the MPCM, including latency checks and verification of the output of the same. One or more sensors send data to the MPCM for sophisticated processing and analysis that the SP cannot perform, but the results of this processing are sent to the SP, which then generates a safety-rated signal to the machinery controlled by the architecture. As a result, the system may qualify for a safety rating despite its ability to perform complex operations that are beyond the scope of safety-rated components. Also, the use of a system capable of monitoring elapsed time and placing the system in a safe state with sufficient reliability to meet safety requirements allows for the use of algorithms with non-deterministic or variable runtimes that would otherwise be ineligible for inclusion in a safety system. For illustrative purposes, the discussion that follows will focus on a computation module with a two (dual) processor board (DPCM) implementation, although it should be understood that architectures with more than two processor computation modules are within the scope of the present invention.

[0023] In various embodiments, the SP, which may be an off-the-shelf safety processor, is responsible for both safety-rated and non-safety-rated communications with the machinery, monitoring the overall health of the system and detecting processing latencies that compromise the safety assurances of the system. In alternative embodiments, the SP is responsible for safety-rated communications and the MPCM is responsible for non-safety-rated communications.

[0024] Thus, in a first aspect, the present invention relates to a control system comprising: a plurality of sensors configured to generate sensor outputs, in various embodiments; a plurality of non-safety computation modules, each including a processor, synchronously responsive to the sensor outputs and performing a safety analysis, the non-safety computation modules generating a control signal in response to detection of a safe condition by the safety analysis; and a safety processor configured to receive the control signals from the computation modules and generate therefrom a safety-rated signal for controlling industrial machinery.

[0025] In some embodiments, the safety processor is further configured to monitor the performance of the computation module, including latency checks and validation of the output of the same. The safety processor may further be configured to monitor intermediate results or data structures and their error correction codes, and may include execution of algorithms with non-deterministic or variable runtimes. The sensor may be, for example, a 3D time-of-flight camera, and in various embodiments, the computation module is configured to perform image capture, identification, and classification from the time-of-flight camera data in real time with low latency.

[0026] In some embodiments, the calculation module is configured to analyze occupation and occlusion of the monitored space by aligning the sensors with respect to each other so that the images acquired by the sensors collectively represent the workspace, generating a three-dimensional representation of the workspace as a plurality of volumes, and for each sensor pixel having an intensity level above a threshold, pre-marking as unoccupied a volume that is blocked by a line of sight ray path through the pixel and terminates at an estimated distance from the sensor associated with the occlusion, marking the volume corresponding to the end of the ray path as occupied, marking any volume beyond the occlusion along the ray path as unknown, and for each sensor pixel having an intensity level below the threshold, pre-marking as unknown all voxels that are blocked by a line of sight ray path through the pixel and terminates at a boundary of the workspace, and finally marking as unoccupied volumes that were pre-marked at least once as unoccupied.

[0027] The calculation module may be configured to assess safety conditions within the monitored space by aligning the sensors with respect to each other so that images acquired by the sensors collectively represent the workspace, generating a three-dimensional representation of the workspace as a plurality of volumes, and generating a volumetric representation of all points reachable by the movable machinery within the workspace within a defined period of time.

[0028] In some embodiments, the industrial machinery is at least one robot. The computing modules may be, for example, two dual-processor computing modules. The computing modules may be configured to generate a 3D representation of the work cell monitored by the sensor. The safety analysis may comprise speed and separation monitoring according to at least one of ISO / TS 15066 or ISO 10218-2 and / or protective separation distance monitoring according to at least one of ISO / TS 15066 or ISO 10218-2.

[0029] The computation modules may be configured to trigger the operation of the sensors sequentially to prevent crosstalk therebetween and / or receive latency tags issued by the safety processor. The safety processor may be configured to generate and transmit latency tags to the sensors to return sensor outputs to the computation modules, receive and analyze latency tags returned by the computation modules to determine whether a duration associated with processing by the multiple computation modules exceeds a predetermined maximum interval, receive and analyze intermediate values ​​returned by the computation modules to determine whether a processing error by the multiple computation modules has occurred, verify that the multiple computation modules operate substantially simultaneously according to a latency criterion, verify that commands issued substantially simultaneously by the multiple computation modules are consistent with each other, cause execution by the controlled industrial machinery of commands issued substantially simultaneously by the multiple computation modules following verification that (a) the multiple computation modules operate substantially simultaneously according to a latency criterion and (b) the commands issued substantially simultaneously by the multiple computation modules are consistent with each other, and / or verify that the clock and timing signals are consistent with independent and redundant reference signals.

[0030] In some embodiments, the computation modules perform the same computation, while in other embodiments, the computation modules perform different computations. In some embodiments, the computation modules are the same, while in other embodiments, the computation modules differ in terms of at least one of the processor, architecture, or operating system.

[0031] Generally, as used herein, the term "substantially" means ±10%, and in some embodiments, ±5%. Additionally, references throughout this specification to "one example," "an example," "one embodiment," or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present technology. Thus, appearances of the phrases "in one example," "in an example," "one embodiment," or "an embodiment" in various places throughout this specification do not necessarily all refer to the same embodiment. Furthermore, particular features, structures, routines, steps, or characteristics may be combined in any suitable manner in one or more embodiments of the present technology. The headings provided herein are for convenience only and are not intended to limit or interpret the scope or meaning of the claimed technology. The present invention provides, for example, the following items. (Item 1) 1. A control system comprising: a plurality of sensors configured to generate sensor outputs; a plurality of non-safety computation modules, each including a processor, synchronously responsive to the sensor output and performing a safety analysis, the non-safety computation modules generating a control signal in response to detection of a safe condition by the safety analysis; a safety processor configured to receive the control signal from the computation module and generate therefrom a safety-rated signal for controlling a device; A control system comprising: (Item 2) 2. The control system of claim 1, wherein the safety processor is further configured to monitor the performance of the calculation module, including latency checking and verifying outputs of the same. (Item 3) 2. The control system of claim 1, wherein the safety processor is further configured to monitor intermediate results or data structures and their error correction codes. (Item 4) 2. The control system of claim 1, wherein the safety analysis includes execution of an algorithm that is non-deterministic or has a variable runtime. (Item 5) 2. The control system of item 1, wherein the sensor is a 3D time-of-flight camera. (Item 6) 6. The control system of claim 5, wherein the calculation module is configured to perform image capture, identification, and classification from the time-of-flight camera data in real time with low latency. (Item 7) The calculation module includes: aligning the sensors with respect to one another such that the images acquired by the sensors collectively represent a workspace; generating a three-dimensional representation of the workspace as a plurality of volumes; for each sensor pixel having an intensity level above a threshold, pre-marking as unoccupied a volume blocked by a line-of-sight ray path through said pixel and terminating at an estimated distance from said associated sensor of an occlusion, marking said volume corresponding to the end of said ray path as occupied, and marking any volume beyond the occlusion along said ray path as unknown; for each sensor pixel having an intensity level below the threshold, pre-marking as unknown all voxels that are blocked by a line-of-sight ray path through said pixel and that terminate at a boundary of said workspace; Finally, marking as unoccupied a volume that was previously marked as unoccupied at least once; 6. The control system of claim 5, configured to analyze the occupancy and occlusion of the monitored space by: (Item 8) The calculation module includes: aligning the sensors with respect to one another such that the images acquired by the sensors collectively represent a workspace; generating a three-dimensional representation of the workspace as a plurality of volumes; generating a volumetric representation of all points reachable by the mobile machinery in said workspace within a prescribed time period; 6. The control system of claim 5, configured to assess a safety condition in the monitored space by: (Item 9) 2. The control system of item 1, wherein the device is at least one robot. (Item 10) 2. The control system of claim 1, wherein the plurality of computing modules are two dual-processor computing modules. (Item 11) 2. The control system of claim 1, wherein the plurality of calculation modules are configured to generate a 3D representation of a work cell monitored by the sensors. (Item 12) 2. The control system of claim 1, wherein the safety analysis comprises speed and separation monitoring according to at least one of ISO / TS 15066 or ISO 10218-2. (Item 13) 2. The control system of claim 1, wherein the safety analysis comprises protective separation distance monitoring according to at least one of ISO / TS 15066 or ISO 10218-2. (Item 14) 2. The control system of claim 1, wherein the plurality of calculation modules are configured to trigger operation of the sensors sequentially so as to prevent crosstalk therebetween. (Item 15) 2. The control system of claim 1, wherein the plurality of calculation modules are configured to receive a latency tag issued by the safety processor. (Item 16) 2. The control system of claim 1, wherein the safety processor is configured to generate and transmit a latency tag to the sensor to return the sensor output to the calculation module. (Item 17) Item 16. The control system of item 15, wherein the safety processor is configured to receive and analyze latency tags returned by the calculation modules and determine whether a duration associated with processing by the multiple calculation modules exceeds a predetermined maximum interval. (Item 18) 2. The control system of claim 1, wherein the safety processor is configured to receive and analyze intermediate values ​​returned by the calculation modules and determine whether a processing error has occurred by the multiple calculation modules. (Item 19) 2. The control system of claim 1, wherein the safety processor is configured to verify that the multiple calculation modules operate substantially simultaneously according to a latency criterion. (Item 20) 2. The control system of claim 1, wherein the safety processor is configured to verify that commands issued substantially simultaneously by the multiple computation modules are consistent with each other. (Item 21) 21. The control system of claim 20, wherein the calculation module performs the same calculation. (Item 22) 21. The control system of claim 20, wherein the calculation module performs different calculations. (Item 23) 2. The control system of claim 1, wherein the safety processor is configured to: (a) cause the plurality of computation modules to operate substantially simultaneously according to a latency criterion; and (b) cause execution by the controlled device of the commands issued substantially simultaneously by the plurality of computation modules following verification that the commands issued substantially simultaneously by the plurality of computation modules are consistent with each other. (Item 24) 2. The control system of claim 1, wherein the safety processor is configured to verify that clock and timing signals are consistent with an independent, redundant reference signal. (Item 25) 2. The control system of claim 1, wherein the calculation modules are the same. (Item 26) 2. The control system of claim 1, wherein the calculation modules differ in terms of at least one of a processor, an architecture, or an operating system. [Brief description of the drawings]

[0032] In the drawings, like reference characters generally refer to the same parts throughout the different views. Also, the drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention. In the following description, various embodiments of the invention are described with reference to the following drawings, in which:

[0033] [Figure 1] FIG. 1 diagrammatically illustrates a secure system implementing a dual processor architecture in accordance with one embodiment of the present invention.

[0034] [Diagram 2] FIG. 2 diagrammatically illustrates a secure system implementing a dual processor architecture in accordance with another embodiment of the present invention.

[0035] [Diagram 3]FIG. 3 diagrammatically illustrates a safety system implementing a dual processor architecture in accordance with one embodiment of the present invention for performing speed and separation monitoring in an industrial work cell.

[0036] [Figure 4] FIG. 4 diagrammatically illustrates the functions performed by the DPCM and its communication with the SP in accordance with one embodiment of the present invention for performing speed and separation monitoring in an industrial work cell.

[0037] [Diagram 5] FIG. 5 diagrammatically illustrates the latency monitoring function and DPCM output comparison performed by an SP, according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0038] The system according to the present disclosure combines an SP and one or more MCUs in an architecture that performs computations beyond the capabilities of conventional processors designed for use in safety applications and that can meet regulatory requirements for use in safety applications. PC M. An application of the architecture according to the present specification is a 3D vision safety system used to monitor an industrial work cell to enable safe collaboration between humans and industrial machinery including robots. However, it should be understood that this application represents just one example of a use for which the present invention is suitable, and more broadly, the approach of the present invention is generally applicable to deployments requiring high performance, low latency computation engines to process signals (e.g., sensor signals) in safety systems.

[0039] A representative system is illustrated in FIG. 1. The system 100 includes two main subsystems: one or more sensors 105, e.g., a 3D Time-of-Flight (ToF) sensor, and a control system 110, described in more detail below and sometimes referred to as the "safety architecture." The safety architecture 110 includes a primary computation module 1121 and a secondary computation module 1122, both of which may be DPCMs capable of performing computationally intensive algorithms and data processing with low latency (e.g., using a GPU), and a single safety processor (SP) 115 designed for use in real-time safety applications. The computation modules 112 may each be, for example, a server-class motherboard including one or more Xeon processors, e.g., one of the Server Board S2600BP family of motherboards marketed by Intel Corp. The computation modules 112 may have different processors and / or architectures, the same or different, i.e., from different manufacturers and possibly with different operating systems.

[0040] The architecture provides the benefits of high performance processing while maintaining the fault tolerance and guaranteed latency required by many safety systems.

[0041] The sensors 105 transmit sensor data (e.g., images) by wired or wireless means to the computation module 112, which processes them and determines the appropriate control outputs. In addition, the sensors 105 send latency tags, described in more detail below in connection with FIG. 5, to the computation module 112, which forwards these to the SP 115.

[0042] Each calculation module 112 performs a safety analysis and sends a calculated control output signal to the SP 115. "Safety analysis" means an analysis of the sensor output to detect or predict conditions that may alter or lead to the operation of the controlled system being controlled. In addition, the calculation module 112 may send an error detection code, such as a cyclic redundancy check (CRC) of intermediate data or intermediate data structures to be compared with the control output. This is useful because the calculations performed by the calculation module 112 may involve complex algorithms applied to large data structures, some of which maintain a state that builds over time, but the control output may be as simple as a percentage speed override or even a Boolean value indicating whether the machinery may be allowed to continue to operate. Because a wide variety of calculations may produce the same output, comparing intermediate values ​​can provide a higher degree of confidence that no faults have occurred and that the current system state is correct.

[0043] SP 115 provides control output signals to a controller 120 of the system being controlled (e.g., industrial machinery, robots, medical equipment, etc.) and receives system state information from the controller 120. Thus, the control output signals originate directly from SP 115, rather than from computation module 112, even though the computation module performs the most intensive operations, and as a result, system 100 may meet regulatory requirements for use in safety applications with computational capabilities beyond those provided by conventional safety processors.

[0044] 2, in some embodiments 200, the sensors 105 may provide their own latency guarantees via an alternative mechanism and not pass the latency tag to the computation module 112. In this case, the SP 115 may pass the latency tag directly to the computation module 112 and the safety architecture 210 monitors only the latency of the computation performed on the computation module, not the complete end-to-end latency of the system (including any processing latency within the sensor itself). The SP 115 transmits the latency tag to the computation module 112, not to the sensor 105.

[0045] As shown in FIG. 3, the architecture can be used to implement a 3D vision safety system 300 that will slow or stop any hazards when a person approaches the controlled machinery (in the illustrated embodiment, a robot) and automatically restart the machinery once the person leaves the area. This type of safety protection is known as speed and separation monitoring (SSM) and is defined in ISO 10218-2 and ISO / TS 15066. In this case, the sensor 305 may be a 3D time-of-flight (ToF) sensor. In a typical deployment, there may be multiple (e.g., up to eight) ToF sensors 305. Each sensor 305 is responsible for providing a depth image from its unique viewpoint, and the computation module 112 processes all of the depth images to generate and / or analyze a 3D volume. A suitable ToF sensor is described in U.S. Ser. No. 16 / 553,724, filed Aug. 28, 2019, which is incorporated herein by reference. At 30 frames per second, the SP115 is a 3D sensor 305 continuously (typically in a sequential manner to prevent crosstalk between sensors). The sensor data is transmitted to the calculation module 112 for processing and generation of velocity override commands, if necessary, that are passed to the SP 115 based on the analyzed sensor data. The SP 115 also communicates with the robot controller 320 and provides the robot state data tagged with a latency tag to the calculation module 112. In this embodiment, as shown in FIG. 4, the calculation module 112 respectively triggers all the sensors 305 into a coherent 3D representation of the monitored work cell. In the illustrated embodiment, these modules include an occupancy and occlusion analysis module 410 that analyzes the 3D volume of voxels representing the monitored work cell and identifies objects and occluded regions (e.g., according to U.S. Pat. No. 10,099,372, the entire disclosure of which is incorporated herein by reference). The calculation module 112 also includes a robot future state prediction module 420 that predicts future robot states based on the current state (position and velocity) and kinematics of the robot, as similarly described in the '372 patent, and a module 430 for calculating a protective separation distance (PSD) (as defined in ISO / TS 15066) between objects in the work cell and the future robot states predicted by module 420. As explained in U.S. Ser. No. 62 / 890,718, filed Aug. 23, 2019, the entire contents of which are incorporated herein by reference, the future robot state may be a voxel (or other) representation of all points reachable by the robot within a prescribed time period. In this embodiment, the control output generated by SP115 is a safety signal that overrides task performance commands issued by a conventionally programmed robot controller 320 during the course of normal robot operation. SP115 signals this information to the robot controller 320, for example, via relays and output signal switching devices (OSSDs), which issue a "stop" command to prevent harmful movements. OSSDs are conventional components that stop or interrupt machinery when entry or proximity is harmful. The control output may also include an alternate trajectory designed to avoid collisions. SP115 may also communicate with the robot controller 320, for example, via an Ethernet interface, to obtain robot state information, which it passes to the computation module 112 for use in the PSD calculation. Thus, in various embodiments, SP115 is a "single source of truth" system He is the sole arbiter of the 100.

[0046] An important function performed by SP115 is to check the latency in the system and ensure that it remains within acceptable limits; excessive latency may indicate an unsafe condition. As shown in FIG. 5, SP115 may include a latency check module 510 that first transmits a latency tag, e.g., a random 32-bit number, to the sensors 105, triggering them to capture a frame of data (step 515). SP115 may also passively or actively collect machine status information and maintain this in module 512 or a memory partition. This information may accompany the latency tag as it is transmitted to the computation module 112 to enable the computation module to correlate the machine status with the sensor data in time.

[0047] The calculation modules 112 then perform in parallel the calculations required to determine the control outputs to be transmitted to the controlled machinery. As each calculation module 112 performs these calculations, the latency tag is conveyed alongside the intermediate results derived from its data. Both calculation modules 112 transmit the results of their calculations to the SP 115 via an appropriate protocol (e.g., the Black Channel protocol) together with the latency tags that accompany the data used in their calculations. Meanwhile, in step 517, the latency check module 510 records the generated latency tag in a lookup table 520 that stores each unique tag with its timestamp, i.e. the time it was transmitted to the sensor 105. The latency check module 510 monitors the time that has elapsed since the latency tag was transmitted to the sensor 105 and determines whether too much time has passed between transmission and reception of the latency tag (step 523). In normal operation, the SP 115 will receive the latency tag, along with the control output signal and any intermediate data, from the computation module 112 within a fixed timeout period consistent with normal operation. If the interval between transmission and receipt, or the absolute time elapsed since transmission in the case where the latency tag is never returned to the SP 115, exceeds the timeout period, a protection stop is asserted to the machine controller 120 (step 525).

[0048] However, if both calculation modules 112 provide a control output and a latency tag within the timeout period, the latency check module 510 compares both the latency tags from the two calculation modules 112 and the outputs themselves. If the latency tags are the same (step 530) and issued by the SP 115 within the timeout period (steps 532, 535), and the control output and intermediate data match (step 538), the latency check module 510 forwards the control output to the machine controller 120. However, if one or both latency tags are incorrect or issued outside the timeout period, or if the control output or intermediate data do not match, the latency check module 510 acts independently to put the machinery in a safe state (step 525).

[0049] By monitoring the latency tags, the SP 115 therefore ensures that variability in the performance of unbounded operation for the computation modules 112 cannot result in an unsafe state of the system. Similarly, by ensuring that the control outputs and intermediate data generated by the computation modules 112 are consistent, a single hardware failure or accidental software failure for one of the computation modules cannot result in an unsafe state of the system. Note that multiple latency tags may be in flight at once, for example, if the total processing time of the computation modules 112 exceeds the time between sensor frames. In this case, the computations on the modules 112 may be pipelined, with some cores performing earlier computations on later frames, while other cores perform later computations on earlier frames.

[0050] Further, in some embodiments, the calculation module 112 performs the same calculation whose final and intermediate outputs can be directly compared, while in other embodiments, the calculation module 112 performs a slightly modified calculation that incorporates the same safety features, such as by offsetting the voxel grid by ½ voxel, or by processing each separate sensor channel, and in such implementations, only the decisions are compared in step 538. This latter approach offers increased versatility but reduced diagnostic coverage.

[0051] SP115 may also perform periodic health self-checks, including checking the consistency of the clock and timing signals that drive triggers and data collection, and monitoring the overall health of the system, including temperature, humidity, and power levels (e.g., via conventional sensors not shown), using independent and redundant external clock signals to ensure correct clock functionality of SP115. For example, for a safety relay, there is an external device monitoring (EDM) signal that monitors the safe and proper functioning of the safety relay. This EDM signal is connected to SP112, and in the event that the safety relay is not operating properly, SP112 can generate a signal to a machinery or robot controller to shut it down in a safe manner.

[0052] SP115 includes hardware elements such as core under-voltage and over-voltage monitoring, and if SP115 is unable to function in a safe state, additional health monitoring may be performed to ensure that SP115 and the safety rated outputs can be placed in a safe state. This includes, but is not limited to, accidental failure of SP115 or any of its computational elements, power shortage, or failure of an internal check. If SP115 or the health monitoring hardware generates a fault signal, the SP115 or the health monitoring hardware may be configured to monitor the safety architecture. 110、210 An additional signal can be generated to place the whole in a safe state.

[0053] All of the modules in the computation module 112 and the SP 115 may be programmed in any suitable programming language, including, but not limited to, high-level languages ​​such as C, C++, C#, Ada, Basic, Cobra, Fortran, Java, Lisp, Perl, Python, Ruby, or low-level assembly language. 110、210 allows each of the computation modules 112 to be replaced with faster hardware or updated software without modifying the SP 115 or other parts of the safety system. 110、210 The SP 115 allows the SP 115 to be "fail-safe", i.e., if the latency check module 510 of the SP 115 detects a calculation failure, the SP 115 puts the system into a safe state. If any calculation module 112 suffers from a random fault resulting in a different output, the safety function provided by the system can no longer be guaranteed (i.e., they do not agree on what the output should be, and therefore safety cannot be guaranteed). An alternative embodiment includes three calculation modules 112 that allow for a single random failure in one of them. If the remaining two calculation modules 112 agree on the result, the SP 115 will not issue a signal to shut down the system ("fail-operational" mode).

[0054] Certain embodiments of the present invention are described above. However, it should be expressly noted that the present invention is not limited to those embodiments, but rather, additions and modifications to those specifically described herein are also included within the scope of the present invention.

Claims

1. A control system comprising: a sensor configured to generate a sensor output; a non-safety computation module including a processor responsive to the sensor output and performing a safety analysis, the non-safety computation module generating a control signal in response to detection of a safety condition by the safety analysis; a safety processor configured to receive the control signal from the non-safety computation module and generate a safety-rated signal from the control signal for controlling a device; A control system comprising:

2. The control system of claim 1, wherein the safety processor is further configured to monitor performance of the non-safety computation module including latency checks and verification of same outputs.

3. The control system of claim 1, wherein the safety processor is further configured to monitor intermediate results or data structures and their error correction code.

4. The control system of claim 1, wherein the safety analysis includes execution of an algorithm that is non-deterministic or has a variable runtime.

5. The control system of claim 1, wherein the sensor is a 3D time-of-flight camera.

6. The control system of claim 5, wherein the non-safety computation module is configured to perform image capture, identification, and classification from data from the 3D time-of-flight camera in real time with low latency.

7. The unsafe calculation module, generating a three-dimensional representation of a monitored space as a plurality of volumes; for each sensor pixel having an intensity level above a threshold, pre-marking as unoccupied a volume blocked by a line-of-sight ray path through said pixel and terminating at an estimated distance from said sensor of an occlusion, marking said volume corresponding to the end of said ray path as occupied, and marking any volume beyond the occlusion along said ray path as unknown; for each sensor pixel having an intensity level below said threshold, pre-marking as unknown all voxels that are blocked by a line-of-sight ray path through said pixel and that terminate at a boundary of said monitored volume; Finally, marking as unoccupied a volume that was previously marked as unoccupied at least once; The control system of claim 5 , configured to analyze the occupancy and occlusion of the monitored space by:

8. The unsafe calculation module, generating a three-dimensional representation of a monitored space as a plurality of volumes; generating a volumetric representation of all points reachable by the mobile machinery in said monitored space within a prescribed time period; 6. The control system of claim 5, configured to assess a safety condition within the monitored space by:

9. The control system of claim 1, wherein the device is at least one robot.

10. The control system of claim 1, wherein the non-safety computation module is two dual processor computation modules.

11. The control system of claim 1, wherein the non-safety calculation module is configured to generate a 3D representation of a work cell monitored by the sensor.

12. The control system of claim 1, wherein the safety analysis comprises speed and separation monitoring in accordance with at least one of ISO / TS 15066 or ISO 10218-2.

13. The control system of claim 1, wherein the safety analysis comprises protective separation distance monitoring in accordance with at least one of ISO / TS 15066 or ISO 10218-2.

14. The control system of claim 1, wherein the non-safety calculation module is configured to receive a latency tag issued by the safety processor.

15. The control system of claim 1, wherein the safety processor is configured to generate and transmit a latency tag to the sensor to return the sensor output to the non-safety calculation module.

16. The control system of claim 15, wherein the safety processor is configured to receive and analyze the latency tag returned by the calculation module and determine whether the duration associated with processing by the calculation module exceeds a predetermined maximum interval.

17. The control system of claim 1, wherein the safety processor is configured to receive and analyze intermediate values ​​returned by the non-safety calculation module and determine whether a processing error has occurred by the non-safety calculation module.

Images