A method of, and system for, anomaly detection in an industrial plant

Abstract

A computer-implemented method of monitoring an operational area of an industrial plant, the method utilizing a monitoring system comprising a plurality of sensors located in the operational area and connected to a computer system by a network, the method being executed by at least one hardware processor and comprising the steps of: obtaining sensor measurement data representative of measured values of one or more physical parameters, the sensor measurement data being synchronously captured from the plurality of sensors at each discrete time point; processing the sensor measurement data for each discrete time point to generate one or more feature maps representative of a spatial location and / or spatial scalar field distribution of the values of at least one physical parameter within the operational area at each discrete time point; utilizing the one or more feature maps to identify operational anomalies within the operational area.

Description

TITLEA METHOD OF, AND SYSTEM FOR, ANOMALY DETECTION IN AN INDUSTRIAL PLANTBACKGROUND OF THE INVENTION

[0001] The present invention relates to a computer-implemented method of, and system for, anomaly monitoring and detection in an industrial plant.

[0002] It is known to provide industrial monitoring systems to monitor processes, equipment and activities in one or more industrial plants to ensure the plant operations and processes are operating normally and safely and continue to meet regulatory requirements. Some monitoring systems may be configured to generate indications to plant operators of anomalies, error conditions or equipment failures. These indications may take the form of particular alarm signals and provide an indication of the error type and time it occurred. The plant operators may then take corrective action in response to particular alarm signals and, for example, shut down particular processes or carry out repairs or maintenance as necessary. However, known industrial plant monitoring systems suffer from a number of technical problems.

[0003] First, monitoring of processes and equipment is often practically sub-optimal. For example, monitoring for the emergence of anomalies inside a space such a plant floor area or in relation to specific pieces of equipment located on the plant floor area, it is necessary to sample data as a function of time at or very near the space of interest.

[0004] However, in practice, plant floor areas are not suitable for fitment of data capture capabilities at a precise desired point of interest. This may be a result of an absence of mounting facilities at suitable locations, a lack of power connections, or due to other technical or safety concerns (e.g. trip hazards, operating temperatures and conditions which are unsuitable for monitoring equipment), among many others. In addition, a plant floor area may simply be too large to render specific monitoring uneconomic or impractical.

[0005] Secondly, in complex industrial plants which comprise significant numbers of processes and systems that require monitoring, a large number of sensors may be required. This can present a plant operator with an overload of information, reducing the ability of the plant operator to make informed decisions and take appropriate action.

[0006] Therefore, there exists a need in the art to provide more effective anomaly detection methods and systems to address these issues.SUMMARY OF THE INVENTION

[0007] The following introduces a selection of concepts in a simplified form in order to provide a foundational understanding of some aspects of the present disclosure. The following is not an extensive overview of the disclosure and is not intended to identify key or critical elements of the disclosure or to delineate the scope of the disclosure. The following merely summarizes some of the concepts of the disclosure as a prelude to the more detailed description provided thereafter.

[0008] Several preferred aspects of the methods and systems according to the present invention are outlined below.

[0009] Aspect 1 : A computer-implemented method of monitoring an operational area of an industrial plant comprising one or more elements of process equipment, the method utilizing a monitoring system comprising a plurality of sensors located in the operational area and connected to a computer system by a network, the method being executed by at least one hardware processor and comprising the steps of: a) obtaining, by the computer system and for a plurality of discrete time points, sensor measurement data representative of measured values of one or more physical parameters, the sensor measurement data being synchronously captured from the plurality of sensors at each discrete time point; b) processing, using the computer system, the sensor measurement data for each discrete time point to generate one or more feature maps representative of a spatial location and / or spatial scalar field distribution of the values of at least one physical parameter within the operational area at each discrete time point; and c) utilizing, using the computer system, the one or more feature maps to identify operational anomalies within the operational area.

[0010] Aspect 2: The computer-implemented method according to Aspect 1 , wherein step a) further comprises obtaining sensor measurement data representative of measured values of a plurality of different physical parameters and step c) further comprises generating one or more combined feature maps representative of a spatial location and / or spatial scalar field distribution of the values of the plurality of different physical parameters within the operational area at each discrete time point.

[0011] Aspect 3: The computer-implemented method according to Aspect 1 or 2, wherein the method further comprises the step of: d) utilizing, using the computer system, the one or more feature maps to provide a visual indication to a user of the computer system of one or more alerts relating to identified operational anomalies within the operational area.

[0012] Aspect 4: The computer-implemented method according to Aspect 3, wherein the visual indication comprises a 2D or 3D spatial representation of the operational area showing values of at least one physical parameter as a function of spatial location.

[0013] Aspect 5: The computer-implemented method according to Aspect 4, wherein the visual indication comprises a 2D or 3D spatial representation of the operational area showing values of a plurality of physical parameters as a function of spatial location.

[0014] Aspect 6: The computer-implemented method according to Aspect 5, wherein the values of a plurality of physical parameters as a function of spatial location are overlaid on the same spatial representation.

[0015] Aspect 7: The computer-implemented method according to Aspect 5 or 6, wherein the plurality of physical parameters are selected from the groups of: acoustic parameters, visible light parameters, infrared emission parameters and / or motion parameters.

[0016] Aspect 8: The computer-implemented method according to any one of Aspects 3 to 7, wherein the visual indication comprises a 2D or 3D spatial representation of the operational area showing the spatial location of one or more alerts relating to identified operational anomalies within the operational area.

[0017] Aspect 9: The computer-implemented method according to any one of the preceding Aspects, further comprising, prior to step c), the steps of: e) extracting, using the computer system, one or more data features from the sensor measurement data for each discrete time point; f) performing, using the computer system, statistical analysis on the data features as a function of time point to generate statistical indications representative of deviations from expected behavior of the one or more elements of process equipment in the operational area; and g) utilizing, using the computer system, the statistical indications in step c) to identify operational anomalies within the operational area.

[0018] Aspect 10: The computer-implemented method according to any one of the preceding Aspects, wherein step a) comprises: h) initiating a data capture event at a first time; i) capturing sensor measurement data from each of the sensors synchronously at the first time; and j) transmitting the sensor measurement data from sensor measurement data capture event to a server device.

[0019] Aspect 11 : The computer-implemented method of Aspect 10, wherein step h) comprises sending an event trigger across a network.

[0020] Aspect 12: The computer-implemented method according to any one of the preceding Aspects, further comprising: k) obtaining time-series process equipment data from one or more sensors associated with one or more operational parameters of the elements of process equipment; and I) utilizing the time-series process equipment data in combination with the sensor measurement data in step b) to generate the one or more feature maps.

[0021] Aspect 13: The computer-implemented method according to Aspect 12, wherein step k) further comprises obtaining time-series process equipment data from one or more distributed control systems (DCS), programmable logic controllers (PLC) and / or human machine interfaces (HMI) of the industrial plant.

[0022] Aspect 14: The computer-implemented method according to any one of the preceding Aspects, wherein the one or more physical parameters are selected from the groups of: acoustic parameters, visible light parameters, infrared emission parameters and / or motion parameters.

[0023] Aspect 15: The computer-implemented method according to Aspect 14, wherein the acoustic parameters are selected from one or more of: acoustic amplitude, sound pressure, acoustic source location, acoustic frequency and acoustic frequency distribution.

[0024] Aspect 16: The computer-implemented method according to Aspect 14 or 15, wherein the infrared emission parameters are selected from one or more of: thermal emission, temperature(s), spatial content and thermal distribution.

[0025] Aspect 17: The computer-implemented method according to Aspect 14, 15 or 16, wherein the motion parameters are selected from one or more of: vibrational amplitude, vibrational frequency, vibration frequency distribution, movement in one or more dimensions, acceleration in one or more dimensions, velocity in one or more directions.

[0026] Aspect 18: The computer-implemented method according to any one of Aspects 14 to 17, wherein one or more physical parameters comprises acoustic parameters and step b) further comprises: m) processing acoustic sensor data from a plurality of acoustic sensors to generate an estimated 3D sound field and / or utilizing one or more beamforming algorithms to generate one or more 2D or 3D acoustic images to form one or more of the feature maps.

[0027] Aspect 19: A system for monitoring an operational area of an industrial plant comprising one or more elements of process equipment, the system comprising a plurality of sensors located in the operational area and connected to a computer system by a network, the computer system comprising at least one hardware processor configured to: obtain, for aplurality of discrete time points, sensor measurement data representative of measured values of one or more physical parameters, the sensor measurement data being synchronously captured from the plurality of sensors at each discrete time point; process the sensor measurement data for each discrete time point to generate one or more feature maps representative of a spatial location and / or spatial scalar field distribution of the values of at least one physical parameter within the operational area at each discrete time point; and utilize the one or more feature maps to identify operational anomalies within the operational area.

[0028] Aspect 20: The system according to Aspect 19, wherein the at least one hardware processor is further configured to: obtain sensor measurement data representative of measured values of a plurality of different physical parameters; and generate one or more combined feature maps representative of a spatial location and / or spatial scalar field distribution of the values of the plurality of different physical parameters within the operational area at each discrete time point.

[0029] Aspect 21 : The system according to Aspect 19 or 20, wherein the at least one hardware processor is further configured to: utilize the one or more feature maps to provide a visual indication to a user of the computer system of operational anomalies within the operational area.

[0030] Aspect 22: The system according to Aspect 21 , wherein the visual indication comprises a 2D or 3D spatial representation of the operational area showing values of at least one physical parameter as a function of spatial location.

[0031] Aspect 23: The system according to Aspect 21 or 22, wherein the visual indication comprises a 2D or 3D spatial representation of the operational area showing values of a plurality of physical parameters as a function of spatial location.

[0032] Aspect 24: The system according to Aspect 23, wherein the values of a plurality of physical parameters as a function of spatial location are overlaid on the same spatial representation.

[0033] Aspect 25: The system according to Aspect 23 or 24, wherein the plurality of physical parameters are selected from the groups of: acoustic parameters, visible light parameters, infrared emission parameters and / or motion parameters.

[0034] Aspect 26: The system according to any one of Aspects 21 to 25, wherein the visual indication comprises a 2D or 3D spatial representation of the operational area showing thespatial location of one or more alerts relating to identified operational anomalies within the operational area.

[0035] Aspect 27: The system according to any one of Aspects 19 to 26, wherein the at least one hardware processor is further configured to: extract one or more data features from the sensor measurement data for each discrete time point; perform statistical analysis on the data features as a function of time point to generate statistical indications representative of deviations from expected behavior of the one or more elements of process equipment in the operational area.

[0036] Aspect 28: The system according to any one of Aspects 19 to 27, wherein the at least one hardware processor is further configured to: initiate a data capture event at a first time; capture sensor measurement data from each of the sensors synchronously at the first time; and cause the transmission of the sensor measurement data from sensor measurement data capture event to a server device.

[0037] Aspect 29: The system according to Aspect 28, wherein the at least one hardware processor is further configured to: send an event trigger across a network to initiate the data capture event at the first time.

[0038] Aspect 30: The system according to any one of Aspects 19 to 29, wherein the at least one hardware processor is further configured to: obtain time-series process equipment data from one or more sensors associated with one or more operational parameters of the elements of process equipment; and utilize the time-series process equipment data in combination with the sensor measurement data to generate the one or more feature maps.

[0039] Aspect 31 : The system according to Aspect 30, wherein the at least one hardware processor is further configured to: obtain time-series process equipment data from one or more distributed control systems (DCS), programmable logic controllers (PLC) and / or human machine interfaces (HMI) of the industrial plant.

[0040] Aspect 32: The system according to any one of Aspects 19 to 31 , wherein the one or more physical parameters are selected from the groups of: acoustic parameters, visible light parameters, infrared emission parameters and / or motion parameters.

[0041] Aspect 33: The system according to Aspect 32, wherein the acoustic parameters are selected from one or more of: acoustic amplitude, sound pressure, acoustic source location, acoustic frequency and acoustic frequency distribution.

[0042] Aspect 34: The system according to Aspect 32 or 33, wherein the infrared emission parameters are selected from one or more of: thermal emission, temperature(s), spatial content and thermal distribution.

[0043] Aspect 35: The system according to Aspect 32, 33 or 34, wherein the motion parameters are selected from one or more of: vibrational amplitude, vibrational frequency, vibration frequency distribution, movement in one or more dimensions, acceleration in one or more dimensions, velocity in one or more directions.

[0044] Aspect 36: The system according to Aspect 32, 33, 34 or 35, wherein one or more physical parameters comprises acoustic parameters and the at least one hardware processor is further configured to process acoustic sensor data from a plurality of acoustic sensors to generate an estimated 3D sound field and / or utilizing one or more beamforming algorithms to generate one or more 2D or 3D acoustic images to form one or more of the feature maps.

[0045] Aspect 37: A non-transitory computer readable storage medium storing a program of instructions executable by a machine to perform a method of monitoring an operational area of an industrial plant comprising one or more elements of process equipment, the method utilizing a monitoring system comprising a plurality of sensors located in the operational area and connected to a computer system by a network, being executed by at least one hardware processor and comprising the steps of: obtaining, by the computer system and for a plurality of discrete time points, sensor measurement data representative of measured values of one or more physical parameters, the sensor measurement data being synchronously captured from the plurality of sensors at each discrete time point; processing, using the computer system, the sensor measurement data for each discrete time point to generate one or more feature maps representative of a spatial location and / or spatial scalar field distribution of the values of at least one physical parameter within the operational area at each discrete time point; and utilizing, using the computer system, the one or more feature maps to identify operational anomalies within the operational area.

[0046] Aspect A1 : A computer-implemented method of monitoring an operational area of an industrial plant comprising one or more elements of process equipment, the method utilizing a monitoring system comprising a plurality of sensors located in the operational area and connected to a computer system by a network, being executed by at least one hardware processor and comprising the steps of: a) obtaining, for a plurality of discrete time points, sensor measurement data representative of measured values of a plurality of different physical parameters, the sensor measurement data being synchronously captured from the plurality of sensors at each discrete time point; extracting, using a computer system, one or more datafeatures from the sensor measurement data for each discrete time point; performing, using a computer system, statistical analysis on the data features as a function of time point to generate statistical indications representative of deviations from expected behavior of the one or more elements of process equipment in the operational area; and processing, using a computer system, the sensor measurement data for each discrete time point to generate one or more combined feature maps representative of a spatial location and / or spatial scalar field distribution of the measured values of the plurality of different physical parameters within the operational area at each discrete time point; utilizing the one or more combined feature maps and the generated statistical indications to provide a visual indication of one or more alerts of operational anomalies within the operational area.BRIEF DESCRIPTION OF DRAWINGS

[0047] Embodiments of the present invention will now be described by example only and with reference to the figures in which:

[0048] FIGURE 1 is a schematic diagram of an industrial facility and an operational area thereof;

[0049] FIGURE 2 is a schematic diagram of a sensor device according to an embodiment;

[0050] FIGURE 3 is a schematic diagram of a monitoring system according to an embodiment;

[0051] FIGURE 4 is a schematic diagram of an analyzer forming part of the monitoring system of Figure 3 according to an embodiment;

[0052] FIGURE 5 is a schematic diagram of the components of the analyzer of Figure 4 showing the inputs and outputs from sensor data processing;

[0053] FIGURE 6 shows (lower images) a series of historical IR images of a machine in the form of a compressor (lower eight images) and a cluster map (upper image);

[0054] FIGURE 7 is a flow diagram of a method according to an embodiment;

[0055] FIGURE 8 is a flow diagram of a method according to a further embodiment; and

[0056] FIGURE 9 is a flow diagram of a method according to a further embodiment;

[0057] Embodiments of the present disclosure and their advantages are best understood by referring to the detailed description that follows. It should be appreciated that like reference numbers are used to identify like elements illustrated in one or more of the figures, whereinshowings therein are for purposes of illustrating embodiments of the present disclosure and not for purposes of limiting the same.DETAILED DESCRIPTION OF THE INVENTION

[0058] Various examples and embodiments of the present disclosure will now be described. The following description provides specific details for a thorough understanding and enabling description of these examples. One of ordinary skill in the relevant art will understand, however, that one or more embodiments described herein may be practiced without many of these details. Likewise, one skilled in the relevant art will also understand that one or more embodiments of the present disclosure can include other features and / or functions not described in detail herein. Additionally, some well-known structures or functions may not be shown or described in detail below, so as to avoid unnecessarily obscuring the relevant description.

[0059] The present invention is directed to the technical field of control systems for monitoring of processes and process equipment. The present invention is concerned with monitoring an operational area of an industrial plant comprising one or more elements of process equipment. A plurality of sensors are located in the operational area and connected to a computer system by a network, Sensor measurement data for a plurality of discrete time points is obtained, where the data is representative of measured values of one or more physical parameters. The sensor measurement data is synchronously captured from the plurality of sensors at each discrete time point. The sensor measurement data for each discrete time point is then processed in a computer system to generate one or more feature maps representative of a spatial location and / or spatial scalar field distribution of the values of at least one physical parameter within the operational area at each discrete time point and the one or more feature maps used to identify operational anomalies within the operational area.

[0060] The technology described herein provides technical improvements to the field of industrial plant monitoring. Conventionally, monitoring for the emergence of anomalies inside a space such a plant floor area or in relation to specific pieces of equipment located on the plant floor area requires data to be obtained near to or at the space of interest. However, in practice, plant floor areas are not suitable for fitment of data capture capabilities at a precise desired point of interest.

[0061] However, the technology of the present invention enables a real-time map of the physical properties being monitored across the whole operational area to be generated. Sucha continuous map provides a means to monitor parts of the plant that are not being directly monitored by the devices themselves.

[0062] OVERVIEW OF PRODUCTION FACILITY

[0063] Figure 1 shows a general schematic diagram of an industrial facility 10. The industrial facility 10 comprises a plurality of industrial plants 10-1 , 10-2, 10-3, 10-4. In embodiments, each industrial plant 10-1 , 10-2, 10-3, 10-4 is configured to carry out one or more industrial processes. In embodiments, the industrial plants 10-1 , 10-2, 10-3, 10-4 may comprise gas production facilities, for example hydrogen, nitrogen and / or ammonia and the industrial processes involved may comprise the production, compression liquefaction and / or storage of such gases. However, this is non-limiting and is intended to be merely exemplary.

[0064] Each industrial plant 10-1 , 10-2, 10-3, 10-4 has a plant operational area 12. Only the plant operational area 12 of plant 10-1 is shown in Figure 1 for clarity. By “plant operational area” is meant the region of the respective industrial plant 10-1 , 10-2, 10-3, 10-4 in which process or plant machinery and equipment is located and / or operated, and in which industrial operations are carried out.

[0065] The term “plant operational area” is intended to include, but not be limited to, the plant floor area and any associated regions such as elevated floors, walkways, staircases or any other operational area within the respective industrial plant 10-1 , 10-2, 10-3, 10-4. In other words, “plant operational area” is intended to cover not only the area of the footprint of the operational area but also the three dimensional volume within which process operations are carried out within the process plant 10-1 , 10-2, 10-3, 10-4.

[0066] As shown in Figure 1 , a plurality of elements of process equipment 14-1 , 14-2, 14-3, 14-4 is located in the plant operational area 12 of each industrial plant 10-1 , 10-2, 10-3, 10-4. The process equipment 14-1 , 14-2, 14-3, 14-4 may take any suitable form and each element may comprise one or more pieces of equipment necessary to carry out at least a part of the overall process carried out by the industrial plant.

[0067] In embodiments, each element of process equipment 14-1 , 14-2, 14-3, 14-4 may comprise one or more industrial machines, ancillary elements and / or product storage elements. For example, the process equipment 14-1 , 14-2, 14-3, 14-4 may comprise one or more of the following elements: compressors; pumps; cryogenic pumps; heat exchangers; reactors; furnaces; boilers; distillation columns; refrigeration units; pipework; pressure vessels; storage units; loading / unloading equipment; mixers; adsorption beds; stripping columns; bulk storage; dryers; drum or cylinder storage handlers; robotic handlers; crushers; and grinders.

[0068] Each element of process equipment 14-1 , 14-2, 14-3, 14-4 may be inter-connected in a continuous or cyclic process or may be a separate entity as part of the overall industrial process carried out by the industrial plant.

[0069] In embodiments, it is necessary to monitor the process equipment 14-1 , 14-2, 14-3, 14-4 to determine safe operation and identify any anomalies or faults that may require close monitoring, shutdown, maintenance, or which may pose a risk to plant or worker / operator safety.

[0070] In this regard, a plurality of sensor devices 50 are disposed within the plant operational area 12. In Figure 1 , four sensor devices 50-1 , 50-2, 50-3, 50-4 are shown. However, this is non-limiting and any suitable number of sensor devices 50 may be used as required. Further, the sensor devices 50-1 , 50-2, 50-3, 50-4 are shown schematically and, whilst shown disposed in corners of the plant operational area 12 in Figure 1 for clarity, this is not intended to be limiting.

[0071] In embodiments, the sensor devices 50 may be placed in any suitable location or configuration, and / or at any suitable position or elevation. For example, sensor devices 50 may be mounted on or adjacent one or more elements of process equipment 14, or on walls or other structural elements. In addition, one or more sensor devices 50 may be floor-standing or may be suspended within the volume of the plant operational area 12. In addition, in embodiments, the sensor devices 50 may be handheld and, for example, carried by plant workers or operators.

[0072] A general schematic diagram of a sensor device 50 is shown in Figure 2. Each sensor device 50 comprises one or more sensor elements 52 for acquiring data related to one or more measurable physical parameters, a processor 54 and a transmitter / receiver 56. The sensor device 50 may be a self-contained unit and have a sensor body comprising the or each sensor element 52, processor 54 and transmitter / receiver 56.

[0073] Alternatively, the sensor element 52 of each sensor device 50 may be remote from a sensor body containing the processor 54 and transmitter / receiver 56, e.g. connected by a wire or via a wireless communication network (e.g. Bluetooth, WiFi, LoRa). This may be important in situations where the measured environment is unsuitable for electronic equipment such as a hardware processor due to heat, vibration, moisture, or other environmental and physical factors.

[0074] In embodiments, the measurable physical parameter may comprise one or more acoustic parameters such as environmental ambient and / or local sound level, maximumsound level, sound frequency and / or sound frequency distribution. In such embodiments, the sensor element 52 may comprise a microphone or an array of microphones. The microphone(s) may be directional (such as, for example, a cardioid microphone designed to preferentially detect sound from a particular direction or position and / or omnidirectional.

[0075] In embodiments, the measurable physical parameter may comprise thermal radiation and the data related to thermal radiation may comprise two-dimensional thermal images, average thermal intensity, maximum thermal intensity, maximum local temperature, thermal intensity of a particular point or region, or any other suitable metric. In such embodiments, the sensor element 52 may comprise an infrared camera element, an infrared sensor, an infrared photodetector element or other structure capable of detection of thermal radiation from a region or source. In embodiments, the infrared sensor may comprise an array of pixels.

[0076] In embodiments, the measurable physical parameter may comprise visible radiation and the data related to thermal radiation may comprise two- or three- dimensional visual images, average light intensity, maximum light intensity, maximum local intensity, illumination intensity of a particular point or region, or any other suitable metric. Visual images may be captured as intensity information only (e.g. black and white images) or may be in full colour. Alternatively or additionally, particular wavelength ranges may be selected if appropriate, e.g. images captured in the blue or red regions of the visible spectrum only.

[0077] In such embodiments, the sensor element 52 may comprise a visible radiation camera, a visible light sensor such as a charge coupled device (CCD), a photodetector element or other optical detector structure capable of detection of visible radiation from a region or source. In embodiments, the light sensor may comprise an array of pixels, e,g, a CCD array. The sensor element 52 may comprise a camera or other optical element operable to capture one or more still or moving visible images of a scene; for example, a portion of the plant operational area and / or an element of process equipment 14.

[0078] In embodiments, the measurable physical parameter may comprise physical displacement or motion such as vibration, oscillation and / or movement. In such embodiments, the sensor element 52 may be configured to measure physical displacement, acceleration, or other such motion-based parameters. In embodiments, the sensor element 52 may comprise one or more accelerometers. The person skilled in the art would be readily aware of the type of accelerometers that could be used with disclosed embodiments.

[0079] In embodiments, a sensor device 50 may comprise multiple sensor elements 52, which may be of different types, e.g. an acoustic sensor and a visible image sensor.

[0080] The processor 54 may be operable to control the operation of the sensor device 50 in terms of data capture and processing of obtained data for transmission. In addition, the processor 54 may optionally process the data to apply one or more of calibrations, standardisation or corrections to sound and / or image data received by the various sensor elements 52.

[0081] The transmitter / receiver 56 is configured to receive and / or transmit data signals to / from the sensor device 50. In embodiments, this may be done wirelessly on a local network, internet or other wireless data communication system. Alternatively or additionally, some or all sensor devices 50 may have wired data connections to network systems. This may be appropriate for sensor devices 50 in fixed locations.

[0082] SENSOR LOCATION

[0083] In embodiments, the location of each sensor device 50 is known. This may be through any suitable means. In embodiments, the sensor devices 50-1 , 50-2, 50-3, 50-4 may be located at locations L1 , L2, L3, L4 respectively. The locations L1 , L2, L3, L4 may be defined in any suitable manner. For example, in embodiments, the sensor devices 50-1 , 50-2, 50-3, 50-4 may comprise components operable to transmit or communicate the active location of the respective sensor device 50-1 , 50-2, 50-3, 50-4 through, for example, global positioning system (GPS) alone or in combination with Wi-Fi or another communication network protocol.

[0084] Alternatively or additionally, the locations L1 , L2, L3, L4 may be determined automatically through other means, for example, through cameras or other measurement apparatus located within the plant operational area 12.

[0085] Alternatively or additionally, the locations L1 , L2, L3, L4 may be defined manually and locations entered by a user into a computer system, for example, through a graphical user interface (GUI) as described below.

[0086] In embodiments, plans or layouts of the plant operational area 12 are utilized and the locations L1 , L2, L3, L4 may be entered, stored or recorded on such plans or layouts. For example, in embodiments, the plans or layouts of the plant operational area 12 may take the form of one or more computer aided design (CAD) models of the plant operational area 12 which can be viewed and edited by a suitable operator.

[0087] Specific embodiments of the sensor devices 50 will now be described below.

[0088] Sound acquisition and processing sensor devices 50

[0089] In embodiments, one or more sensor devices 50 may be configured to capture sound sources such as from industrial machines, pipes, or other process equipment 14-1 , 14-2, 14-3, 14-4 using sensor elements 52 comprising a single microphone or array of microphones to capture time-dependent audio and sound data. The processor 54 of each sensor device 50 is operable to receive a suitable signal and enable audio capture at a particular time point or time period.

[0090] Further, the processor 54 of each sensor device 50 is operable to process or pre-process the audio data then obtained. In embodiments, the processor 54 standardizes (i.e., calibrates) the raw sound samples which are then sent (via the transmitter / receiver 56) for further analysis and processing as described below. The audio record may be defined as Sn,r , where S is the audio record from sensor n at time index t. However, this is to be taken as non-limiting and, in embodiments, any processing of the captured audio signals may be carried out remote from the sensor device 50 and by a different entity from processor 54.

[0091] Thermal images acquisition and processing sensor devices 50

[0092] In embodiments, one or more sensor devices 50 may be configured to capture thermal sources and signatures such the temperature profiles from industrial machines, pipes, or other process equipment 14-1 , 14-2, 14-3, 14-4, as well as human personnel, using sensor elements 52 comprising infrared focal plane array (FPA) cameras. The FPA cameras are configured to capture time-dependent infrared image data within the field of view (FoV) of the camera. The processor 54 of each sensor device 50 is operable to receive a suitable signal and enable IR image capture at a particular time point or time period.

[0093] Further, the processor 54 of each sensor device 50 is operable to process or pre-process the thermal data then obtained. In embodiments, the processor 54 standardizes (i.e., calibrates) the raw IR image pixel data samples which are then sent (via the transmitter / receiver 56) for further analysis and processing as described below. The IR pixel image record may be defined as IRn,r , where IR is the thermal image data from sensor n at time index t.

[0094] Visual images acquisition and processing sensor devices 50

[0095] In embodiments, one or more sensor devices 50 may be configured to capture visual images and signatures from industrial machines, pipes, or other process equipment 14-1 , 14-2, 14-3, 14-4, as well as human personnel, using sensor elements 52. In other words, data values or images of bodies generating or illuminated by visible light are captured by one or more sensor devices 50.

[0096] In embodiments, the sensor elements 52 comprise visible light focal plane array (FPA) cameras. The FPA cameras are configured to capture time-dependent visual image data within the FoV of the camera (i.e. the sensor device 50). The processor 54 of each sensor device 50 is operable to receive a suitable signal and enable visual image capture at a particular time point or time period.

[0097] Further, the processor 54 of each sensor device 50 is operable to process or pre-process the visual data then obtained. In embodiments, the processor 54 standardizes (i.e., calibrates) the raw visual image pixel data samples which are then sent (via the transmitter / receiver 56) for further analysis and processing as described below. These images may be defined as Vn,r, where V is the visual image data from sensor n at time index t. The visual image data Vn,r may, in embodiments, be used independently or as a visual or coordinate reference for integration with the acoustic and thermal data.

[0098] Physical displacement and motion sensing sensor devices 50

[0099] In embodiments, one or more sensor devices 50 may be configured to capture data values relating to physical displacement or motion measurements vibration, oscillation and / or movement of plant machines, equipment or pipes. In such embodiments, the sensor element 52 may be configured to measure physical displacement, acceleration or other such motionbased parameters. In embodiments, the sensor element 52 may comprise one or more accelerometers. The person skilled in the art would be readily aware of the type of accelerometers that could be used with disclosed embodiments.

[0100] In embodiments, physical displacement and / or acceleration measurements may be utilized to complement one or more thermal and / or acoustic measurements. For example, if an anomalous sound signature or deviation of the thermal map from an expected distribution is detected, then vibration data can be checked to identify whether a particular machine or piece of plant equipment is operating or not, or whether there has been a change in mode of operation.

[0101] For example, if a particular machine is switched off (no vibration), then there should be no reason for the machine to heat up. However, if an anomalous thermal signature is identified this may indicate another issue, such as a rupture or failure in a hot line inside. Similarly, if a machine changes mode of operation which is captured in the vibration, then the acoustic spectrogram is also expected to change, and this would not be identified as anomalous.

[0102] MONITORING SYSTEM 100

[0103] Figure 3 shows a schematic overview of the components of a monitoring system 100 according to an embodiment of the present invention. The monitoring system 100 comprises a plurality of sensor devices 50, a historian database 120, a processing system 140 and a user system 160. These components are communicatively connected by means of a network 180.

[0104] The network 180 may take any suitable form and may be a local network (for example, a Wi-Fi or Ethernet network), a cloud network (through internet or cellular communication connections) or a mixed network utilizing both technologies. The network 180, therefore, places no specific location constraints on the components of the monitoring system 1- such as the database 120 and processing system 140 which may be located within or remote from the industrial plant facility 10 and / or industrial plant 10-1 , 10-2, 10-3, 10-4.

[0105] The sensor devices 50 correspond to those described and illustrated with respect to Figures 1 and 2. The historian database 120 is operable to store and enable access to the raw time-series data obtained from the sensor devices 50 in use, and to store historical timedependent records relating to the sensor devices 50 over predetermined timescales. Such timescales may be selected as appropriate but may be of the order of months or years of timeseries data.

[0106] The processing system 140 comprises a computer system such as a server device operable to process and analyze data from the historian database 120. The processing system 140 comprises at least one hardware processor 142, a memory 144, and an analyser 146. The functionality of the analyser 146 will be discussed below. The processing system 140 is communicatively connected to the database 120 and sensor devices 50 via the network 180.

[0107] The processing system 140 is also configured to send signals and / or instructions to one or more of the sensor devices 50. For example, as described below, the processing system 140 is operable to send control signals to ensure time-synced capture of data across the sensor devices 50 under the command of the monitoring system 100.

[0108] The user system 160 is configured to be operated by a user such as a plant operator or other plant personnel and comprises at least one hardware processor 162, a memory 164 and a display 166 which may comprise a graphical user interface (GUI) 168. A monitoring application 170 is available to access on the user system 160.

[0109] The monitoring application 170 is operable to enable the user to view the generated monitoring data and to take action or enter commands for control, confirmation and / or monitoring of the monitoring system 100. In embodiments, the monitoring application 170comprises the central point of integration and presentation of all processed data of the monitoring system 100 to the user. The data presented to the user via the GUI 168 may comprise time-series feature data, sound field, multi-modal (acoustic and thermal) view data, and alerts and insights data.

[0110] The GUI 168 may, in embodiments, present the user with the live and historical acoustic and thermal views, which can be contextualized with reference to alerts with the intent of focusing the user to one or more situations requiring immediate attention.

[0111] The monitoring application 170 may be a local application run directly on the user system 160 or the monitoring application 170 may be cloud- or network-based and run remotely with user access via the GUI on the display 166 of the user system 160.

[0112] TIME-SYNCED SYSTEM OPERATION

[0113] The monitoring system 100 is operable to capture data from the sensor devices 50 at specific time points and / or time intervals. Data capture is synchronous over all the sensor devices 50 operational in the monitoring system 100 at any given time. In other words, the sensor devices 50 are coordinated to capture measured data at a synchronised time t such that the data across the sensor devices 50 is obtained substantially simultaneously.

[0114] The time t may be determined by any appropriate measure. In embodiments, the time t may be coordinated by a time-synched command issued across the network 180 from the processing system 140. In embodiments, this may be determined by the user device 160, such as in situations where an on-demand data snapshot is required. In such an example, a user may manually execute a capture command through the user device 160 to obtain data at a specific time point as required. .

[0115] Alternatively, the time t may be determined automatically at predetermined intervals, e.g. a first command at time t, a second command at time t+1 etc.. The time interval between t and t+1 may take any suitable value; for example, between 60 seconds to 5 minutes. Alternatively, for more data-heavy applications, the time interval may be of the order of tens of minutes or one or more hours. Each time interval may be identical or may vary depending on the requirements of each plant facility 10.

[0116] In embodiments, the time interval may be selected based upon the time characteristics of the process being monitored. For example, if a process operates in a steady state and small variations occur on a timescale of hours, then a capture frequency of 5 to 10 minutes may be considered to provide sufficient resolution. However, if a transient event such as a start up orshut down of a machine or process that may have a duration of 1 to 2 minutes, then a capture frequency of 5-10 seconds may be required.

[0117] At each time t, responsive to a data capture command sent across the network 180, each sensor device 50 captures a data set from the respective sensor element 52 of that sensor device 50. This may be pre-processed by the processor 54 of each sensor device 50 if required, before being transmitted across the network 180 by the transmitter / receiver 56 of each sensor device 50. The transmitted data is received and stored by the historian database 120 and received by the processing system 140 for further analysis as described below.

[0118] ANALYZER 146

[0119] The functionality of the analyzer 146 will now be described with reference to Figures 4 and 5. Figure 4 shows an overview of the components of the analyzer 146 including the dataflow path from module to module. Figure 5 shows a selection of the components of the analyzer 146 in more detail together with the inputs and outputs from the sensor data processing.

[0120] In embodiments, the analyzer 146 is operable to process data from the different categories of sensor device 50. An exemplary embodiment is shown in Figures 3 and 4 where the sensor devices 50 are grouped into three of the categories discussed above -acoustic / sound sensors 50S, infrared sensors 50IR, visual sensors 50V and motion (vibration) sensors 50M. Each category of sensors 50S, 50IR, 50V, 50M may comprise any suitable number of sensor devices n. In embodiments, n may take any non-zero integer value.

[0121] The analyzer 146 is operable to process the data according to the relevant category 50S, 50IR, 50V of sensor device 50. Specialist hardware and / or software modules may be employed for each category 50S, 50IR, 50V of sensor device 50.

[0122] In embodiments, the analyzer 146 comprises an acoustic processing module 148. Acoustic processing module 148 comprises three processing elements: a feature extractor 148A, a sensor fusion module 148B and a beamforming module 148C. It is to be understood that each of the above elements may be aspects of a single software and / or hardware module or may be separate software / hardware elements in their own right as required.

[0123] Acoustic processing module

[0124] The acoustic processing module 148 receives the acoustic data from n acoustic sensor devices 50S-1 ... 50S-n. This data may be pre-processed by the processor 54 of each sensor device 50 in order to standardize / calibrate the raw sound samples obtained from the sensorelement 52 However, in embodiments, this may be optional. A plurality of audio records from each sensor device 50S-1 ... 50S-n where each acoustic sensor device 50S-n produces an audio record Sn,rat time index t.

[0125] The acoustic data is input into the three modules 148A, 148B, 148C. The feature extractor 148A comprises any suitable audio processing algorithms or modules, either in hardware and / or software. For example, fast Fourier transforms (FFTs) may be used for feature extraction. In addition, classifiers may be used which may comprise machine learning models.

[0126] In embodiments, the feature extractor module 148A is operable to extract statistical features of the acoustic data such as loudness and / or frequency content as extracted feature data. In embodiments, the feature extractor module 148A may comprise a plurality of feature extraction elements which operate in parallel to extract different and / or specific features from the audio data. In addition, the feature extractor module 148A is operable to generate timeseries tags of the extracted feature data for use in further processing elements downstream. The further processing of the timeseries tag data is described below.

[0127] The feature extractor module 148A may comprise any suitable processing units to achieve the desired functionality. For example, the feature extractor module 148A may utilize fast Fourier transform (FFT) processing as is known in audiovisual signal processing. Alternatively, classifiers utilizing techniques such as thresholding and / or clustering methods can be utilized. Alternatively or additionally, the feature extractor module 148A may comprise principal components analysis (PCA) configured technologies and / or autoencoders.

[0128] If machine learning classifiers are used, then transformer architectures, attentionbased models, convolutional neural networks (CNNs), deep learning networks (DNNs) and / or other forms of artificial neural network (ANN) may be utilized.

[0129] The sensor fusion module 148B is operable to process the acoustic data at time t, from all the acoustic sensors 50S-n to generate an estimated 3D sound field Ltfor the entire plant operational area 12 at time t.

[0130] The beamforming module 148C comprises one or more beamforming algorithms which operate on the acoustic data Sn,rto generate one or more acoustic images, Fn,t , where each image F is for a sensor device 50S-n at time t. Each acoustic image F comprises a spatial picture of the sound sources. In other words, the acoustic image F comprises a 2D or 3D image of sound loudness across different frequency bands across a predefined space. Inembodiments, the predefined space comprises the 2D field-of-view (FOV) of the sensor element 52 of the acoustic sensor device 50S-n.

[0131] Thermal processing module

[0132] In embodiments, the analyzer 146 comprises a thermal processing module 150. Thermal processing module 150 comprises a feature extractor 150A. It is to be understood that this element may be an aspect of a single software and / or hardware module or may comprise separate software / hardware elements in its own right as required.

[0133] The feature extractor 150A is operable to process the thermal image data to extract statistical features relating to the thermal profiles, e.g., max / min / average temperatures, and generates timeseries tags of these feature data and forwards for further downstream processing. Multiple feature extractors (A, B, C...) operate in parallel to extract different features.

[0134] The feature extractor module 150A may comprise any suitable processing units to achieve the desired functionality. For example, the feature extractor module 150A may utilize fast Fourier transform (FFT) processing as is known in audiovisual signal processing. Alternatively, classifiers utilizing techniques such as thresholding, edge detection, regionbased segmentation and / or clustering methods can be utilized. Alternatively or additionally, the feature extractor module 150A may comprise principal components analysis (PCA) configured technologies and / or autoencoders.

[0135] If machine learning classifiers are used, then convolutional neural networks (CNNs) and deep learning networks (DNNs) or other forms of artificial neural network (ANN) may be utilized.

[0136] Visual processing module

[0137] In embodiments, the analyzer 146 comprises a visual processing module 152. It is to be understood that this element may be an aspect of a single software and / or hardware module or may comprise separate software / hardware elements in its own right as required.

[0138] The visual processing module 152 is operable to receive the visual data V„,ffrom the n visual cameras 50V-n. This data may or may not be pre-processed as described above. In specific embodiments, this data is received and utilized by the analyzer 146 in subsequent modules. In embodiments, no specific feature extraction is performed on the visual images. However, this is not intended to be limiting and other processing may be carried out.

[0139] For example, visible images may be input in the visual processing module 152 to one or more convolutional neural networks (CNN) (or other neural network-based architectures) configured to perform image-recognition and image-segmentation to determine one or more operational parameters related to the plant operational area 12. For example, this may include information such as operating conditions, working arrangements and patterns, work situations, unsafe or unauthorized behavior, situations or individuals, or unsafe, unauthorized and / or unapproved machinery installations.

[0140] Overlay module

[0141] The analyzer 146 further comprises an overlay module 154 which is operable to receive images from multiple sensor modalities. In embodiments, the overlay module 154 is operable to receive IR images from the thermal sensor devices 50IR, images acoustic images Fn.tfrom the beamforming module 148C, and visual images from the visual processing module 152.

[0142] In the overlay module 154, the different images from the acoustic, thermal and visual modalities are joined together. In embodiments, this process comprises the application of a homography matrix and panorama stitching algorithms operable to combine the images from different sensor modalities to form multi-modal output images, Mn,t (Figure 4) that shows acoustic, thermal, visual data in a unified view.

[0143] In embodiments, the processing steps involved may comprise homography and / or image-registration of the visual camera 50V image(s) with one or more IR images from a corresponding IR sensor device 50IR on the same device or at the same device location. Alternatively or additionally, the processing steps involved may comprise homography and / or image-registration of the visual camera 50V image(s) with one or more acoustic images from a microphone array of an acoustic device 50S in the same device 50 or in the same location as the visual sensor device 50V.

[0144] In embodiments, the extracted feature maps may be overlaid and / or combined with one or more maps, plans or layouts of the plant operational area 12. For example, in embodiments, the extracted feature maps may be fused or overlaid on one or more computer aided design (CAD) models of the plant operational area 12. Such an overlay or fusion may show various parameters; for example, scalar field distributions such as loudness maps, thermal hotspot maps, average acoustic pitch maps, or more.

[0145] As noted, the acquisition trigger at time t is coordinated across all sensor devices so that any microphones, infrared cameras, and visual cameras used to generate the layout mapscapture data at the same time point. In other words, the data is synchronously captured at each discrete time point across all the sensor devices 50.

[0146] Region of interest (ROI) module

[0147] The analyzer 146 further comprises a region of interest (ROI) module 156. The ROI module 156 is configured to enable a user to focus monitoring and / or anomaly detection on a specific location or region within the operational plant area 12. In embodiments, the ROI module 156 is operable to receive inputs from the overlay module 154, sensor fusion algorithm 148B, IR feature extractor 150A, visual processing module 152 and one or more motion sensors 50M.

[0148] In embodiments, the ROI module 156 may enable a user to focus on or monitor specific machinery or equipment that a user determines may require special attention. In embodiments, a user may select one or more ROIs in the feature image of the plant operational area 12 and generate statistical feature tags for one or more of these ROIs. In embodiments, the “image” may comprise one or more of: a thermal image, a sound field image or a multi-modal image. The operation of this aspect will be described later with regard to the method.

[0149] Statistical and alert modules

[0150] In embodiments, the analyzer 146 further comprises a statistical module 158. The statistical module 158 comprises one or more statistical engines and receives inputs from the feature extractor 148A, IR feature extractor 150A and ROI module 156. In addition, the statistical module 158 has access to the data stored in the historian 120.

[0151] The statistical module 158 is operable to generate statistical models of the received data and compare those models to the current values of measured data at specific time points t to detect significant deviations from normal operation. For example, an overheating machine would generate a higher than normal value for a corresponding ‘temperature’ feature in the IR images of the machine. The statistical module 158 may, in embodiments, utilize historical timeseries data as training data to build one or more internal uni-variate and / or one or more multi-variate data models.

[0152] In embodiments, the analyzer 146 further comprises an alert module 158A. The alert module 158A is operable to receive as an input data from the statistical module 158 and generate, if required, one or more alerts and / or insights.

[0153] In embodiments, the alert module 158A is operable to generate one or more alerts and / or insights in relation to one or more machines, equipment or conditions in response to parametric deviations identified by the statistical module 158. Parametric deviations may include deviations of the values of one or more parameters from a predicted or normal value of said one or more parameters.

[0154] In one embodiment, specific alerts may be generated which are sent to a user terminal as required. The alerts may take any suitable form and may be graded by severity, for example. Insights may not be in the form of alerts which require a response or action but instead may provide a user with information relating to an estimated parameter of likely interest to the user. For example, this may include estimated or predicted machine operation modes from the detected parameters.

[0155] The alert module 158A may, in embodiments, be configured to notify the user(s) by any suitable means. For example, the alert module 158A may be configured to communicate alerts and / or insights via any suitable communications means such as email, instant message, SMS (text message) via any suitable communications network such as the internet, Wi-Fi, 4G / 5G / LTE mobile networks, satellite communication networks or microwave networks.

[0156] The analyzer 146 and modules thereof is operable to communication with the user system 160 to present the data analysis, feature maps and alerts / insights. The user system 160 is configured to be operated by the plant operator or other plant personnel.

[0157] USER SYSTEM 160

[0158] The monitoring application 170 is operable to enable the user to view the generated monitoring data and to take action or enter commands for control, confirmation and / or monitoring of the monitoring system 100. In embodiments, the monitoring application 170 comprises the central point of integration and presentation of all processed data of the monitoring system 100 to the user. The data presented to the user via the GUI 168 may comprise time-series feature data, sound field, multi-modal (acoustic and thermal) view data, and alerts and insights data.

[0159] The GUI 168 may, in embodiments, present the user with the live and historical acoustic and thermal views, which can be contextualized with reference to alerts with the intent of focusing the user to one or more situations requiring immediate attention.

[0160] For example, consider an issue such as a leak in a high-pressure superheated steam line. This would generate an acoustic as well as a thermal signature. The statistical module158 would generate an alert to the operator via the monitoring application 170 and GUI and / or via other communication modes such as SMS, email or other communication methodologies described above. The operator can then view relevant timeseries tags data and spatially identify the issue at hand by also viewing one or more unified (for example, acoustic and thermal maps overlaid on one another) view of the plant operational area 12 or a part thereof.

[0161] The monitoring application 170 is also able to send data input by the operator to the analyzer 146 and components thereof. For example, the operator is able to select regions of interest (ROI) for monitoring through the GUI 168 and this data is sent to the ROI module 156 for analysis, and data generation and reporting.

[0162] In addition, an operator is able to respond to any alerts or insights generated by the alert module 158A. The action taken (or lack of action taken) may be noted by the system and used in a feedback process. In embodiments, the statistical module 158 may utilize the operator action data to improve or modify the statistical analysis performed.

[0163] Consider a situation where an anomaly is identified and an alert generated, and a user cancels the alert without taking any action. In such a scenario, it may be that the anomalous parameter or metric that prompted the alert was a normal variation in the operation of one or more machines, for example, and the user did not need to take any action in response thereto. As a consequence, the statistical module 158 may learn that particular conditions of behavior of specific machines are within normal operating regimes and do not require future alerts to be generated. Instead, the particular parameter variation that triggered the original alert may be flagged as a lower priority notification or an insight for the operator or may be ignored in future.

[0164] HISTORIAN DATABASE 120

[0165] The historian database 120 comprises a data store operable to store and enable access to multiple data sets, correlated by timestamp. In embodiments, the historian 120 is operable to store raw time-series data obtained from the sensor devices 50 at each timestamp. In embodiments, this comprises acoustic data S„,r for each acoustic device 50S-n at each time stamp t, IR data I Rn,ffor each IR sensor device 50IR-n at each time stamp t, visual camera data Vn,r for each visual sensor device 50V-n at each time stamp t, and motion data Mn,r for each vibration sensor device 50M-n at each time stamp t.

[0166] In addition, the historian 120 is operable to store data which has been analyzed and processed in the analyzer 146. For example, for acoustic data, the historian 120 stores data generated by the beamforming module 148C in the form of one or more acoustic images, Fn,tfor timestamp t, together with extracted features A, B, C... from the feature extractor module 148A.

[0167] For the IR data, the historian 120 is operable to store extracted IR feature data A, B, C... from the feature extractor 150A as a function of time t. By utilizing the time series tags for each timestamp t, historical time-dependent records relating to the raw sensor data and processed data can be stored across predetermined timescales. Such timescales may be selected as appropriate but may be of the order of months or years of time-series data. This data is available for subsequent analysis and comparison, and for higher-level analytic processing as described below.

[0168] HIGHER LEVEL ANALYTIC PROCESSING AND PRESENTATION

[0169] In embodiments, the obtained data may be processed by the analyzer 146 to provide higher-level analytics to provide insights. In embodiments, and for clarity of explanation, higher level analytics are described as being performed by the statistical module 158. However, this is non-limiting and other elements of the analyzer 146 may additionally or alternatively perform this analysis.

[0170] Further, in alternative embodiments, the analysis may be performed by an entity other than the analyzer 146 and the processing may be done by a remote networked entity, by computational systems associated with the historian 120, or the client application 170

[0171] The higher level analytics differ from the previously-described processing in that historical data is utilized across different time periods to determine further insights, as opposed to using data having a particular time stamp or for a particular time period as described above. However, this is not intended to be limiting and new or recently-acquired data may also be used in the higher level analytic processing. For example, new data may be used to refine or update models derived from historical data analysis. In an example scenario, if a new operating mode appears, then a clustering algorithm may update the number of clusters or even change the boundaries of the convex hull that define a cluster in response to the new operating mode being detected.

[0172] As noted, the historian 120 is operable to store both data transmitted from the sensor devices 50 as well as certain processed data such as the beamformed images Fn,t as well as extracted feature elements from the acoustic data and IR sensor data. This data, together with the generated multi-modal output images, Mn,t that show acoustic, thermal (and potentially visual) data in a unified view, together with any other elements such as vibration data, can be used for more detailed and historical analysis.

[0173] All of the above data types may serve as inputs into one or more models and / or machine learning models to generate further insights into the plant and / or machine behavior over time. The machine learning models may take any suitable form and may comprise ensemble models with multiple models, where at least one subsequent model takes inputs from a preceding model.

[0174] For example, in embodiments, the outputs of multiple neural networks may be combined. This may be particularly relevant to acoustic data. Examples of models may comprise convolutional neural networks (CNNs), deep autoencoders, convolutional autoencoders and / or Gaussian Mixture Density Neural Network (GMDNN). The latter example may be suitable for acoustic anomaly detection. In addition, CNNs having architectures including but not limited to AlexNet and resNet may be utilized in one or more embodiments for classification and segmentation.

[0175] In addition to the historical and / or real-time data acquired as described above, in specific embodiments, further data sources may also be used. One or more of the elements of process equipment 14-1 , 14-2, 14-3, 14-4 may have one or more sensors (integrated or external) for monitoring the performance thereof. As noted above, the process equipment 14-1 , 14-2, 14-3, 14-4 may comprise one or more industrial machines, ancillary elements and / or product storage elements such as compressors; pumps; cryogenic pumps; heat exchangers; reactors; furnaces; boilers; distillation columns; refrigeration units; pipework; pressure vessels; storage units; loading / unloading equipment; mixers; adsorption beds; stripping columns; bulk storage; dryers; drum or cylinder storage handlers; robotic handlers; crushers; and grinders.

[0176] Any of these elements of process equipment 14 may comprise suitable sensors to measure operational parameters thereof. For example, a pump or compressor may record data representative of measurable parameters such as motor speed, flow rate, pressure, power draw etc.

[0177] In embodiments, this data may be recorded separately as part of the standard plant operational and monitoring systems. In embodiments, the data may be stored as a simple time series data set, with a single numeric value for each recorded timestamp. This may be recorded in one or more plant historians (not shown) and processed by plant computing resources (not shown)

[0178] The plant historians may store time series data from one or more distributed control systems (DCS), programmable logic controllers (PLC) and human machine interfaces (HMI) that the plant operators and engineers may use in real-time to operate the plant efficiently andsafely. This data can also be used as part of the process of the present invention to provide new insights.

[0179] For example, as described above, the database 120 stores all the raw and processed data from the analyzer 146. This, in embodiments, may comprises data from one or more acoustic, visible, IR and vibration sensors as well as timeseries tags as described above. However, in embodiments, the analyzer 146 may be able to access not only this data but also data from the plant historian(s). The combination of these two data sources enables further detailed insights into the operation of the plant systems to be obtained.

[0180] For example, by having access to direct operational parameter data of the process equipment 14-1 , 14-2, 14-3, 14-4 this data can be used in combination with the sensor data to determine cross-correlations and cross-relationships. For example, consider a motor of a pump or compressor. If the direct operational parameter data of the motor indicates an increase in amperage, this may have an effect on other sensor readings simultaneously, such as a measured increase in flow for the pump or compressor and a measurable increase in motor speed. These data values may be captured by direct operational parameter data of the motor direct sensors. In addition, one or more sensors 50 may detect increased sound loudness, a change in frequency, increased vibration in the region of the motor and increased temperature.

[0181] Alternatively, a sudden emergence of thermal and acoustic signature from a point in the plant floor may indicate a new or unauthorized machine running if no new data sources or changes are noticed in the direct operational parameter data.

[0182] These cross-relationships can then be modeled by either mutli-linear variable analysis (MVA) using techniques such as principal component analysis (PCA), partial least squares regression (PLS) or a suitably-configured neural network.

[0183] Examples of higher-level analytics will now be described. In one embodiment, thermal images from individual cameras may be used for unsupervised learning (e.g. clustering) to create multiple clusters denoting multiple modes of operation of a piece of equipment or a machine. This analysis can be used to identify changes in normal operational modes or the appearance of new modes of operation and / or anomalies.

[0184] Figure 6 shows an example of this. Figure 6 shows a series of historical IR images of a machine in the form of a compressor (bottom eight figures). The images were first passed through a principal-component (PC) decomposition engine and K-means clustering wasperformed on the first significant n-numbers (unspecified) of PCs that explain most of the variation in the data.

[0185] As shown in the top diagram, the clustering on this set of training data clearly shows three different clusters within the first two PC-plane that corresponds to normal, rainy-day, or shutdown modes of operation. Additionally, the normal operation cluster appears to comprise two different subclusters (above and below) of operation.

[0186] A mathematical envelope or a convex-hull can be created for each cluster that will denote the boundary of the that cluster. As more data becomes available, each point should generally fall in one of these clusters. If a new data point appears that is outside of these clusters - that may denote a new mode of operation or an anomaly.

[0187] In embodiments, an alert can be generated which would require an operator to look at the data point and discern if it is a real anomaly or a “new normal” / new operating mode and annotate as such. In embodiments, this may create a feedback loop to the algorithm for retraining the data.

[0188] A similar kind of analysis can be done after converting acoustic files from individual microphones into spectrograms and on fusion maps created by analytics explained earlier (fusion I mixing fields etc.). Outputs of all these AI / ML based insight generation are fed to the operator via the GUI 168 and display screen 166 for feedback based learning, alert dissemination and resolution.

[0189] A key element is anomaly detection. By utilizing the method of the present invention, anomalous relationships can be identified even in scenarios where operation of equipment is still within normal or acceptable bounds. Consider, for example, two sound sources from two elements of process equipment 14-1 , 14-2.

[0190] In normal operation, the type of acoustic emission of one element of process equipment 14-1 may bear a correlation with the emission of the other process equipment 14-2. This will result in a particular form of fusion map being generated. Now consider the situation where one element of process equipment 14-1 has a change in emission signature (e.g. value, profile, amplitude or frequency of sound emission) which does not exceed the bounds of historical data for that element. Under most methods, this would not be detected. However, a new fusion map generated using this new emission signature would be different from the previously-generated fusion map due to the breaking of the previously-identified correlation. This will be captured in the neural network and can be used to identify potential issues even when equipment is operating within normal operational parameters or conditions.

[0191] METHOD

[0192] Figure 7 shows a method 200 according to an embodiment. In embodiments, there is provided a computer-implemented method of anomaly detection in an industrial plant facility. The method is implemented on a computer system utilizing at least one hardware processor and having memory and operable to receive inputs from other data modules and processes.

[0193] Steps 202 to 208 describe setting up the sensor devices and this procedure may be optional once the desired configuration is reached.

[0194] Step 202: Provide sensor devices

[0195] At step 202, the sensor devices 50 are provided. The sensor devices 50 may be selected to have a particular form or structure, i.e. comprise a particular type of sensor element 52 (acoustic, I R, visual, motion). Each sensor device 50 may comprise multiple sensor types, e.g. a visual camera and an IR camera which share the same or similar field of view (FoV).

[0196] Step 204: Locate sensor devices

[0197] At step 204, the sensor devices 50 are located in desired locations. They may be fixed in specific locations or may be placed on or against any suitable surface. Referring to the example of Figure 1 , the sensor devices 50-1 , 50-2, 50-3, 50-4 may be located at locations L1 , L2, L3, L4 respectively.

[0198] The locations L1 , L2, L3, L4 are therefore known. The location may be confirmed manually, or though automatic means. For example, in embodiments, the sensor devices 50-1 , 50-2, 50-3, 50-4 may comprise components operable to transmit or communicate the active location of the respective sensor device 50-1 , 50-2, 50-3, 50-4 through, for example, global positioning system (GPS) alone or in combination with Wi-Fi or another communication network protocol.

[0199] Alternatively or additionally, the locations L1 , L2, L3, L4 may be determined automatically through other means, for example, through cameras or other measurement apparatus located within the plant operational area 12.

[0200] Alternatively or additionally, the locations L1 , L2, L3, L4 may be defined manually and locations recorded by operators entered by a user into a computer system, for example, through a graphical user interface (GUI) as described below.

[0201] The locations may optionally contain further metadata or other identifier information. For example, in embodiments, each device 50-1, 50-2, 50-3, 50-4 at respective location L1 , L2, L3, L4 may comprise an identifier identifying the machine or equipment that is being monitored, or what is visible in the field of view. This information tag may assist in locating the device or recording its location if necessary or assist with linking the device 50 to one or more other devices (e.g. vibration monitoring or other data acquisition apparatus).

[0202] The sensor devices 50 are required to be located such that they can be connected to the network 180. As noted above, the network 180 may take any suitable form and may be a local network (for example, a Wi-Fi or Ethernet network), a cloud network (through internet or cellular communication connections) or a mixed network utilizing both technologies.

[0203] Step 206: Configure device monitoring

[0204] In step 206, plans or layouts of the plant operational area 12 are utilized on the monitoring application and analyzer 146 and the locations L1 , L2, L3, L4 may be entered, stored or recorded on such plans or layouts.

[0205] For example, in embodiments, the plans or layouts of the plant operational area 12 may take the form of one or more computer aided design (CAD) models of the plant operational area 12 which can be viewed and edited by a suitable operator and the locations L1 , L2, L3, L4 added manually or automatically, e.g. via the GUI 168 of the display 166.

[0206] Step 208: Set up devices for operation

[0207] At step 208, the devices 50 are connected to the network 180 and primed for operation. The devices 50 are configured to receive a data capture command at time t across the network 180 and capture predetermined data at time t. The predetermined data may be on any suitable timescale.

[0208] For example, at time t, visual and / or IR images may be captured with a relatively short exposure (in embodiments, less than 1 second), whereas audio or vibration may capture instantaneous magnitudes at time t and / or may capture a short burst of data in a predefined time window (e.g. 0.5 s, 1s, 2s) at time t. Alternatively, the devices 50 may have an “always on” operation where data is continuously captured in a rolling format and only saved at time t, in which case any measurement window may be centered on time t.

[0209] The data capture command t may be sent across the network 180, or may be internal or predetermined. If internal or predetermined then it is necessary for the devices 50 to synchronize either across the network 180 or via other means (e.g. GPS) to ensure that datacapture is synchronized across all devices at the desired time t. In embodiments, if the timing is predetermined then the only parameter needed to sync the data capture may be the time zone of the device in question. In embodiments, this may be done by syncing the device 50 time with the server time over the network 180 once the device 50 is connected to the network 180.

[0210] The devices 50 are also network connected such that the captured data at time t can be transmitted across the network 180 by the transmitter / receiver 56 of each sensor device 50. The transmitted data is received and stored by the historian database 120 and received by the processing system 140 for further analysis.

[0211] As noted above, steps 202 to 208 may be optional and not required once a system is set up. The following method 250 comprising steps 252 to 258 relates to the operation of an embodiment of the system 100.

[0212] Figure 8 shows a method 250 according to an embodiment. In embodiments, there is provided a computer-implemented method of anomaly detection in an industrial plant facility. The method is implemented on a computer system utilizing at least one hardware processor and having memory and operable to receive inputs from other data modules and processes.

[0213] The method 250 relates to a computer-implemented method of monitoring the operation of an operational area of an industrial plant utilizing a monitoring system 100. The operational area comprises one or more elements of process equipment.

[0214] The monitoring system comprises a plurality of sensors (for example, sensor devices 50) located in the operational area (for example, the plant operational area 12) and connected to a computer system (for example, processing system 140 described above) by a network (for example, network 180). The method is executed by at least one hardware processor and comprises the following steps.

[0215] Step 252: Obtain sensor data

[0216] At step 252, sensor measurement data for a plurality of discrete time points is obtained from the sensor devices 50. The sensor measurement data is representative of measured values of one or more physical parameters measured at each discrete time point. The sensor measurement data is synchronously captured from the plurality of sensors at each discrete time point.

[0217] In other words, the sensor measurement data is captured from the plurality of sensor devices 50 substantially simultaneously at each discrete time point and / or the data capture bythe sensor devices 50 is coordinated and synchronized to capture sensor measurement data from all the sensor devices 50 at the same discrete time point, and for each time point.

[0218] In embodiments, the sensor measurement data is representative of measured values of a plurality of physical parameters measured at each discrete time point.

[0219] The measurable physical parameter(s) relate to one or more measurable physical properties of the physical environment of the operational area. Non-limiting examples of physical parameters may include one or more of: acoustic parameters (amplitude, sound pressure, source location, frequency and / or frequency distribution); visible light parameters (e.g. color, amplitude, spatial content (e.g. images, forms and visible properties of objects), location); infrared (IR) emission parameters (thermal emission, temperature(s), spatial content (e.g. thermal distribution, thermal properties of objects); motion parameters (e.g. vibrations amplitude, vibration frequency, vibration frequency distribution, movement in one or more dimensions, acceleration in one or more dimensions, velocity in one or more directions). Note that any one or more of the above examples may be selected as physical parameters.

[0220] In embodiments, the plurality of sensor devices 50 may comprise any suitable number provided there at least one sensor type of sensor device 50 within the plurality of sensor devices 50. By this is meant that the plurality of sensor devices 50 is operable to determine measured values of at least one physical parameter. In embodiments, at least two sensor types of sensor device 50 within the plurality of sensor devices 50 may be provided. By this is meant that the plurality of sensor devices 50 is operable to determine measured values of at least two physical parameters. Examples of sensor devices for measuring the parameters defined above are provided in the description above of the components of the system.

[0221] In embodiments, the data capture is synchronous across all the sensor devices 50 operational in the monitoring system 100 at any given time. In other words, the sensor devices 50 are coordinated to capture measured data at a synchronised time point t such that the data across the sensor devices 50 is obtained substantially simultaneously.

[0222] Step 252 may be carried out in any suitable manner. Step 252 may comprise providing the necessary data which has been previously captured at specific time points. Alternatively or additionally, step 252 may comprise initiating a data capture event and acquiring the sensor measurement data as described below.

[0223] For example, a data capture command t may be sent across the network 180 or may be internal or predetermined. If internal or predetermined then it is necessary for the devices50 to synchronize either across the network 180 or via other means (e.g. GPS) to ensure that data capture is synchronized across all devices at the desired time t.

[0224] The time t may be determined by any appropriate measure. In embodiments, the time t may be coordinated by a time-synched command issued across the network 180 from the processing system 140. In embodiments, this may be determined by the user device 160, such as in situations where an on-demand data snapshot is required

[0225] Alternatively, the time t may be determined automatically at predetermined intervals, e.g. a first command at time t, a second command at time t+1 etc.. The time interval between t and t+1 may take any suitable value; for example, between 60 seconds to 5 minutes. Alternatively, for more data-heavy applications, the time interval may be of the order of tens of minutes or one or more hours. Each time interval may be identical or may vary depending on the requirements of each plant facility 10.

[0226] Once the data has been obtained, it is communicated to the necessary systems. In embodiments, this may comprise the processing system 140 (comprising the analyzer 146). This may be across the network 180 or by any other means as required.

[0227] The method proceeds to step 254.

[0228] Step 254: Extract feature data

[0229] At step 254, one or more data features from the sensor measurement data for each discrete time point are extracted. The processing involved is dependent upon the physical parameter to which the sensor measurement data relates.

[0230] In embodiments in which the sensor measurement data comprises acoustic data, the feature extractor module 148A may be configured to extract statistical features of the acoustic data such as loudness and / or frequency content as extracted feature data.

[0231] In embodiments, the feature extractor module 148A may comprise a plurality of feature extraction elements which operate in parallel to extract different and / or specific features from the audio data. In addition, the feature extractor module 148A is operable to generate timeseries tags of the extracted feature data for use in further processing elements downstream.

[0232] The feature extractor module 148A may utilize and suitable extraction methodology and may comprise any suitable processing units to achieve the desired functionality. For example, the feature extractor module 148A may utilize fast Fourier transform (FFT) processing as is known in audiovisual signal processing.

[0233] Alternatively, classifiers utilizing techniques such as thresholding and / or clustering methods can be utilized. Alternatively or additionally, the feature extractor module 148A may comprise principal components analysis (PCA) configured technologies and / or autoencoders.

[0234] In embodiments in which the sensor measurement data comprises thermal (IR) data, the feature extractor 150A may be configured to process the thermal image data to extract statistical features relating to the thermal profiles, e.g., max / min / average temperatures, and generates timeseries tags of these feature data and forwards for further downstream processing. Multiple feature extractors (A, B, C...) may be configured operate in parallel to extract different features.

[0235] The feature extractor module 150A may utilize and suitable extraction methodology and may comprise any suitable processing units to achieve the desired functionality. For example, the feature extractor module 150A may utilize fast Fourier transform (FFT) processing as is known in audiovisual signal processing.

[0236] In embodiments, the frequency content of the image can point to the existence of size of thermal bodies and how sharp the thermal gradients are. E.g. a sudden change in the high frequency content of the image may mean appearance of hot / cold spots, the increase in low frequency content of the image may mean increase of size of thermal blobs etc.

[0237] Alternatively, classifiers utilizing techniques such as thresholding, edge detection, region-based segmentation and / or clustering methods can be utilized. Alternatively or additionally, the feature extractor module 150A may comprise principal components analysis (PCA) configured technologies and / or autoencoders. If machine learning classifiers are used, then convolutional neural networks (CNNs) and deep learning networks (DNNs) or other forms of artificial neural network (ANN) may be utilized.

[0238] In embodiments, IR images may be processed for max I min I average temperature and / or skewness I kurtosis of the captured thermal scene. The images may also be segmented to isolate out specific regions of interest that correspond to equipment pieces or parts (like values, motors) and similar analyses can be performed on those regions.

[0239] Each of these “calculated tags” may have its own operating history and operational bounds can be created using statistical rules. Additionally, the pictures can be inputs to AN Ns I DNNs as described previously. These can again be combined I contextualized by the plant operational data for higher level analytics.

[0240] In embodiments, one or more sensor devices 50 may be configured to capture data values relating to physical displacement or motion measurements vibration, oscillation and / or movement of plant machines, equipment or pipes. In such embodiments, the sensor element 52 may be configured to measure physical displacement, acceleration or other such motionbased parameters. In embodiments, the sensor element 52 may comprise one or more accelerometers.

[0241] In embodiments, for physical displacement / vibration, physical properties such as max I rms acceleration, max I rms displacement (by integrating acceleration profiles) may be calculated or determined.

[0242] In embodiments, these calculated tags can easily correspond to equipment health and operation. For example, rms or max of a vibration scan generally are stable for different operation modes (statistically stationary profiles). Out-of-bounds values or even gradually increasing trends can be anomalies.

[0243] In addition, FFT analytics can be performed on raw waveforms. Peak-detection (generic and proprietary) algorithms can be used to keep track of the number, locations and strength of the peaks for a given mode of operation. Emergence of additional peaks, or disappearance of peaks or changes in the energy content of the spectral domain can indicate anomalies. Additionally, FFT or spectrograms may become inputs for ANN I clustering algorithms as is the case for acoustic data discussed above.

[0244] Finally, in embodiments, visual image data may be utilized as an aid for operators to view IR images in context. However, visual image data may also be used as an input for specific anomaly detection cases. For example, the entry of unauthorized personnel in a specific plant areas or actual incidents that may not even appear on IR cameras such as the emergence of smoke or other phenomena which may not be identified by IR images.

[0245] Step 256A: Perform statistical analysis

[0246] Step 256A may be optional. At step 256A, statistical analysis on the data features as a function of time point may be performed to generate statistical indications representative of deviations from expected behavior of the one or more elements of process equipment in the operational area.

[0247] In embodiments, the statistical module 158 may be utilized to provide statistical insights. The statistical module 158 may take any form suitable for processing the extracted data features comprises one or more statistical engines and receives inputs from the featureextractor 148A, IR feature extractor 150A and ROI module 156. In addition, the statistical module 158 has access to the data stored in the historian 120.

[0248] In step 256A, the statistical module 158 is operable to generate statistical models of the received data and compare those models to the current values of measured data at specific time points to detect significant deviations from normal operation.

[0249] In the case of embodiments utilizing multi-variate statistical models, the statistical module 158 is operable to pull data from the plant historian and build models that predicts one or more of the features calculated. In addition, the statistical module 158 may attempt to match the measured features across the two measurement systems and track the deviations for out of bounds conditions.

[0250] For example, an overheating machine would generate a higher than normal value for a corresponding ‘temperature’ feature in the IR images of the machine. The statistical module 158 may, in embodiments, utilize historical timeseries data as training data to build one or more internal uni-variate and / or one or more multi-variate data models.

[0251] The identified statistical insights and any deviations so identified may be utilized in later steps to generate an alert to a user.

[0252] Step 256B: Utilize process equipment monitoring data

[0253] Step 256B may be optional. In embodiments, at step 256B, further data sources may also be used in combination with the obtained sensor data for data processing and analysis.

[0254] One or more of the elements of process equipment 14-1 , 14-2, 14-3, 14-4 may have one or more sensors (integrated or external) for monitoring the performance thereof. In embodiments, the process equipment 14-1 , 14-2, 14-3, 14-4 may comprise one or more industrial machines, ancillary elements and / or product storage elements such as compressors; pumps; cryogenic pumps; heat exchangers; reactors; furnaces; boilers; distillation columns; refrigeration units; pipework; pressure vessels; storage units; loading / unloading equipment; mixers; adsorption beds; stripping columns; bulk storage; dryers; drum or cylinder storage handlers; robotic handlers; crushers; and grinders.

[0255] Any of these elements of process equipment 14 may comprise suitable sensors to measure operational parameters thereof. For example, a pump or compressor may record data representative of measurable parameters such as motor speed, flow rate, pressure and / or power draw.

[0256] In embodiments, at step 256B this data, which may be recorded separately as part of the standard plant operational and monitoring systems and stored as a simple timeseries data set with a single numeric value for each recorded timestamp, can be processed with the statistical data in step 256A and / or feature data extracted in step 254.

[0257] By utilizing direct operational parameter data of the process equipment 14-1 , 14-2, 14-3, 14-4 in combination with the sensor data, at step 256B cross-correlations and crossrelationships can be determined.

[0258] For example, consider a motor of a pump or compressor. If the direct operational parameter data of the motor indicates an increase in amperage, this may have an effect on other sensor readings simultaneously, such as a measured increase in flow for the pump or compressor and a measurable increase in motor speed. These data values may be captured by direct operational parameter data of the motor direct sensors. In addition, one or more sensors 50 may detect increased sound loudness, a change in frequency, increased vibration in the region of the motor and increased temperature.

[0259] Alternatively, a sudden emergence of thermal and acoustic signature from a point in the plant floor may indicate a new or unauthorized machine running if no new data sources or changes are noticed in the direct operational parameter data.

[0260] At step 256B, this data is processed and / or combined with the statistical data in step 256A (if implemented) and / or feature data extracted in step 254.

[0261] Step 258: Generate feature maps

[0262] At step 258, the sensor measurement data is processed using a computer system (for example, analyzer 146) to generate one or more feature maps representative of a spatial location and / or distribution of the measured values of at least one physical parameter within the operational area at each discrete time point.

[0263] The feature maps may be generated for each sensor modality in accordance with the methods described in the "analyzer 146” section above. For example, in the case of acoustic data, the sensor fusion module 148B may be operable to process the acoustic data at time t, from all the acoustic sensors 50S-n to generate an estimated 3D sound field Ltfor the entire plant operational area 12 at time t.

[0264] Then, the beamforming module 148C comprising one or more beamforming algorithms may operate on the acoustic data Sn,rto generate one or more acoustic images, Fn,t , where each image F is for a sensor device 50S-n at time t. Each acoustic image F comprises aspatial picture of the sound sources. In other words, the acoustic image F comprises a 2D or 3D image of sound loudness across different frequency bands across a predefined space. In embodiments, the predefined space comprises the 2D field-of-view (FOV) of the sensor element 52 of the acoustic sensor device 50S-n.

[0265] In the case of thermal (IR) data, the feature extractor 150 A may be operable to process the thermal image data to extract statistical features relating to the thermal profiles, e.g., max / min / average temperatures, and generates timeseries tags of these feature data and forwards for further downstream processing. Multiple feature extractors (A, B, C...) operate in parallel to extract different features which may then be used on the feature map. In general, IR images are already in a spatial format for further processing.

[0266] By “feature map” is meant a numerical representation of the spatial location and / or spatial distribution of the measured values of at least one physical parameter as a function of the spatial dimensions of the operational area. In particular, the operational area may be defined as a two or three dimensional space and the spatial location and / or distribution of at least one measured physical parameter may be overlaid on a representation of that space.

[0267] The feature map may comprise an array or matrix of spatial co-ordinates with corresponding scalar values. This may, in embodiments, be utilized as a data array for analysis and data processing to identify insights. Optionally, the map may be represented graphically for a user as needed, and as described in specific embodiments below.

[0268] In embodiments, the defined feature map may represent the determined or calculated spatial distribution of the values of a particular parameter. The generated distribution may be substantially continuous in space even though measurements may be performed at discrete locations. This analysis enables data to be generated for regions where it may not, for example, be possible to locate a particular sensor or other monitoring equipment.

[0269] In embodiments, at step 258 the sensor measurement data is processed using a computer system (for example, analyzer 146) to generate one or more combined feature maps representative of a spatial location and / or distribution of the measured values of two or more of the plurality of physical parameters within the operational area at each discrete time point.

[0270] By “combined feature map” is meant a numerical representation of the spatial location and / or spatial distribution of the measured values of two or more of the plurality of physical parameters as a function of the spatial dimensions of the operational area. In particular, the operational area may be defined as a two or three dimensional space and the spatial locationand / or distribution of at least two of the measured physical parameters may be overlaid on a representation of that space.

[0271] The combined feature map may comprise an array or matrix of spatial co-ordinates with corresponding scalar values. This may, in embodiments, be utilized as a data array for analysis. Optionally, the maps may be represented graphically for a user as needed, and as described in specific embodiments below.

[0272] In other words, in embodiments, the feature maps and / or combined feature maps may comprise model-based or data-based inference tools configured to predict values at any point in the operational space. In embodiments, as described below, this may be independent of the visualization of the data itself.

[0273] In embodiments, the feature maps or combined feature maps may utilize “reasonable parametrized distributions” such as basis functions or parametrized manifolds, where the obtained sensor data may be used to estimate the parameters of the manifold. In embodiments, a manifold may be operable to provide a more realistic distribution of scalar fields for parameters such as loudness.

[0274] If, for example, only bi-linear interpolation or distance-weighted averages are used for each point inside the field, then the value inside the field can never exceed the values of the sampled points. However, considering that sound sources will be located in the interior regions, it is reasonable to expect the largest sound amplitude will be somewhere inside the region and not at the point of sampling. Again, this provides a method to estimate the loudness at any point inside the field without the requirement to locate a sensor close to the point of interest.

[0275] In embodiments the sensor measurement data comprises at least acoustic, thermal and visual physical parameter data. In such embodiments, the overlay module 154 is configured to receive IR images from the thermal sensor devices 50IR, images acoustic images Fn,tfrom the beamforming module 148C, and visual images from the visual processing module 152.

[0276] In the overlay module 154, the different images from the acoustic, thermal and visual modalities are joined together. In embodiments, this process comprises the application of a homography matrix and panorama stitching algorithms operable to combine the images from different sensor modalities to form multi-modal output images, Mn,t (Figure 4) that shows acoustic, thermal, visual data in a unified view.

[0277] In embodiments, the processing in step 258 may comprise homography and / or imageregistration of the visual camera 50V image(s) with one or more IR images from a corresponding IR sensor device 501 R on the same device or at the same device location.

[0278] Alternatively or additionally, step 258 may comprise homography and / or imageregistration of the visual camera 50V image(s) with one or more acoustic images from a microphone array of an acoustic device 50S in the same device 50 or in the same location as the visual sensor device 50V.

[0279] By providing such an approach, the captured sensor measurement data can be processed to create “features” using heuristics and / or artificial intelligence models. Features may comprise, but not be limited to, feature parameters such as sound “pressure level”, “loudness”, “shrillness”, heat map, “thermal eccentricity”, “thermal spread”. In step 258, these features can be fitted to a set of mathematical functions (basis functions) that are suitable for the selected feature type and physical process producing such data.

[0280] This processing in step 258 creates a real-time map of the physical properties being monitored across the whole operational area and provides a convenient method of creating continuous and spatially interpolateable maps of all these types of sensor measurement data captured from the sensors 50.

[0281] In addition, such a continuous map provides a means to monitor parts of the plant that are not being directly monitored by the devices themselves. Additionally, one or more motion sensors 50M may be mounted on specific elements of process equipment to capture high frequency vibration scans. The combined feature maps may, in embodiments, be augmented by the data from the vibration to provide a “context” to the maps.

[0282] Once the feature maps or combined feature maps are generated, the method proceeds to step 260.

[0283] Step 260: Utilize feature maps to identify anomalies

[0284] At step 260, the one or more feature maps or combined feature maps may be used to identify operational anomalies within the operational area. In addition, if statistical analysis is performed in step 256A or cross-correlation of the feature map data with process equipment sensor data in step 256B is performed, this additional analysis can be combined as part of anomaly detection.

[0285] At step 260, the computer system is operable to identify operational anomalies based on the data obtained above. This data may be stored for further processing or correlation or may be communicated to the user as appropriate.

[0286] Anomaly detection based on the data sources has numerous advantages. For example, if elevated levels of loudness are identified in a corner of the operational area, vibration data relating to a machine starting up or changing mode provides an explanation. However, a sudden increase in loudness without a change in any other machine signature may indicate an anomaly in a system such as an actual or imminent failure or damage to equipment. The computer system, based on the above information, can usefully identify anomalies to assist with plant operation.

[0287] Step 262: Generate user visualization

[0288] Step 262 may be optional. At step 262 the feature maps or combined feature maps generated in step 258 may be presented to a user. At step 262, the one or more feature maps or combined feature maps may, in embodiments, be utilized to generate user alerts if operational anomalies are identified within the operational area.

[0289] In other words, the one or more (combined) feature maps provide, in embodiments, a visual indication to a user of the computer system of one or more alerts relating to identified operational anomalies within the operational area.

[0290] In embodiments, the visual indication may comprise a 2D or 3D spatial representation of the operational area showing values of at least one physical parameter as a function of spatial location. Spatial representation of the operational area is intended to be non-limiting and does not imply any graphical or structural reconstruction of the precise components in the operational area.

[0291] For example, a 2D spatial representation of a rectangular operational area may simply comprise a displayed rectangle of appropriate dimensions having the values of at least one physical parameter displayed thereon or adjacent. The values may be in the form of a “heat map” with color or shade used to denote different magnitudes or values of the parameter across the spatial representation.

[0292] Alternatively or additionally, numerical or symbolic representations (e.g. arrows, peak values, contour lines) may be used to represent the spatial variation of the values of the physical parameter(s) as a function of spatial location.

[0293] In embodiments, the visual indication may comprise a 2D or 3D spatial representation of the operational area showing the spatial location of one or more alerts relating to identified operational anomalies within the operational area. The alerts could be mapped onto the spatial representation and highlighted or otherwise identified to the user in an appropriate manner.

[0294] Further, if generated, the feature maps or combined feature maps may be integrated with the statistical insights identified in step 256A to generate data to be presented to a user. In embodiments, at step 262, the one or more feature maps or combined feature maps together with the generated statistical indications may be utilized to generate user alerts if operational anomalies are identified within the operational area

[0295] In embodiments, at step 262, the extracted feature maps are overlaid and / or combined with one or more maps, plans or layouts of the plant operational area 12. For example, in embodiments, the extracted feature maps may be fused or overlaid on one or more computer aided design (CAD) models of the plant operational area 12. Such an overlay or fusion may showing various parameters; for example, scalar field distributions such as loudness maps, thermal hotspot maps, average acoustic pitch maps, or more.

[0296] Further, step 262 may also involve generation of alerts and / or insights based on the statistical analysis. In this regard, in embodiments, the analyzer 146 may further comprise an alert module 158A operable to receive as an input data from the statistical module 158 and generate, if required, one or more alerts and / or insights.

[0297] In embodiments, the alert module 158A is operable to generate one or more alerts and / or insights in relation to one or more machines, equipment or conditions in response to parametric deviations identified by the statistical module 158. Parametric deviations may include deviations of the values of one or more parameters from a predicted or normal value of said one or more parameters.

[0298] In one embodiment, specific alerts may be generated which are sent to a user terminal as required. The alerts may take any suitable form and may be graded by severity, for example. Insights may not be in the form of alerts which require a response or action but instead may provide a user with information relating to an estimated parameter of likely interest to the user. For example, this may include estimated or predicted machine operation modes from the detected parameters.

[0299] The alert module 158A may, in embodiments, be configured to notify the user(s) by any suitable means. For example, the alert module 158A may be configured to communicate alerts and / or insights via any suitable communications means such as email, instant message,SMS (text message) via any suitable communications network such as the internet, Wi-Fi, 4G / 5G / LTE mobile networks, satellite communication networks or microwave networks.

[0300] Figure 8 shows a method 300 according to an embodiment. In embodiments, there is provided a computer-implemented method of anomaly detection in an industrial plant facility. The method is implemented on a computer system utilizing at least one hardware processor and having memory and operable to receive inputs from other data modules and processes.

[0301] The method 300 relates to a computer-implemented method of monitoring the operation of an operational area of an industrial plant utilizing a monitoring system 100. The monitoring system comprises a plurality of sensors (for example, sensor devices 50) located in the operational area (for example, the plant operational area 12) and connected to a computer system (for example, processing system 140) by a network. The method is executed by at least one hardware processor and comprises the following steps.

[0302] Step 302: Initiate data capture event

[0303] At step 302, a data capture event is initiated. In embodiments, this may comprise initiating a data capture command t on the network 180. The data capture command t may be sent across the network 180, or may be internal or predetermined. If internal or predetermined then it is necessary for the devices 50 to synchronize either across the network 180 or via other means (e.g. GPS) to ensure that data capture is synchronized across all devices at the desired time t.

[0304] Step 304: Synchronous capture of data at time point

[0305] At step 304, data is captured by the plurality of sensors 50. The sensor measurement data is representative of measured values of a plurality of different physical parameters measured at the discrete time point.

[0306] The sensor measurement data is captured from the plurality of sensor devices 50 substantially simultaneously at each discrete time point. In other words, the data capture by the sensor devices 50 is coordinated and synchronized to capture sensor measurement data from all the sensor devices 50 at the same discrete time point, and for each time point.

[0307] The physical parameters relate to one or more measurable physical properties of the physical environment of the operational area. Non-limiting examples of physical parameters may include one or more of: acoustic parameters (amplitude, pressure, location, frequency and / or frequency distribution); visible light parameters (e.g. color, amplitude, spatial content (e.g. images, forms and visible properties of objects), location); infrared (IR) emissionparameters (thermal emission, temperature(s), spatial content (e.g. thermal distribution, thermal properties of objects); motion parameters (e.g. vibrations amplitude, vibration frequency, vibration frequency distribution, movement in one or more dimensions, acceleration in one or more dimensions, velocity in one or more directions). Note that any one or more of the above examples may be selected as physical parameters.

[0308] In embodiments, the plurality of sensor devices 50 may comprise any suitable number provided there at least one sensor type of sensor device 50 within the plurality of sensor devices 50. By this is meant that the plurality of sensor devices 50 is operable to determine measured values of at least one physical parameter. In embodiments, at least two sensor types of sensor device 50 within the plurality of sensor devices 50 may be provided. By this is meant that the plurality of sensor devices 50 is operable to determine measured values of at least two physical parameters. Examples of sensor devices for measuring the parameters defined above are provided in the description above of the components of the system.

[0309] In embodiments, the data capture is synchronous across all the sensor devices 50 operational in the monitoring system 100 at any given time. In other words, the sensor devices 50 are coordinated to capture measured data at a synchronised time point t such that the data across the sensor devices 50 is obtained substantially simultaneously.

[0310] In each case, the data capture may be different. For example, at time t, visual and / or IR images may be captured with a relatively short exposure (so effectively instantaneously), whereas audio or vibration may capture instantaneous magnitudes at time t and / or may capture a short burst of data in a predefined time window (e.g. 0.5 s, 1s, 2s) at time t. Alternatively, the devices 50 may have an “always on” operation where data is continuously captured in a rolling format and only saved at time t, in which case any measurement window may be centered on time t.

[0311] In embodiments, a sensor device 50 may comprise multiple sensor elements 52, which may be of different types, e.g. an acoustic sensor and a visible image sensor, and so a sensor device 50 may capture measured data relating to a plurality of physical parameters.

[0312] Step 306: Pre-process data

[0313] At step 306, the captured data may optionally be pre-processed to apply one or more of calibrations, standardisation or corrections to sound and / or image data received by the various sensor elements 52. This may, in embodiments, be carried out by the processor 54 of each sensor device 50. Alternatively, this may be carried out at a later stage after the data hasbeen transmitted in step 308. This step is optional and may not be required in all situations of data capture.

[0314] Step 308: Transmit data

[0315] At step 308, the captured sensor measurement data representative of the plurality of measured physical parameters is transmitted across the network 180 to the processing system 140.

[0316] In embodiments, the transmitter / receiver 56 of each sensor device 50 receives and / or transmits data signals to / from the sensor device 50. In embodiments, this may be done wirelessly on the network 180 which may take the form of a local network, internet or other wireless data communication system. Alternatively or additionally, some or all sensor devices 50 may have wired data connections to network systems such as the processing device 140. This may be appropriate for sensor devices 50 in fixed locations.

[0317] Step 310: Extract feature data

[0318] At step 310, one or more data features from the sensor measurement data for each discrete time point are extracted. The processing involved is dependent upon the physical parameter to which the sensor measurement data relates.

[0319] In embodiments in which the sensor measurement data comprises acoustic data, the feature extractor module 148A may be configured to extract features of the acoustic data such as loudness and / or frequency content as extracted feature data. Loudness and / or average pitch may provide a quantitative description of the energy content of the spectrum. Loudness provides an indication of the overall energy content of the spectrum, and average pitch is the frequency centroid of that energy. Statistical data may comprise the standard deviation of the derived FFT distribution

[0320] In embodiments, the feature extractor module 148A may comprise a plurality of feature extraction elements which operate in parallel to extract different and / or specific features from the audio data. In addition, the feature extractor module 148A is operable to generate timeseries tags of the extracted feature data for use in further processing elements downstream.

[0321] The feature extractor module 148A may utilize and suitable extraction methodology and may comprise any suitable processing units to achieve the desired functionality. For example, the feature extractor module 148A may utilize fast Fourier transform (FFT) processing as is known in audiovisual signal processing. Alternatively, classifiers utilizing techniques such as thresholding, edge detection, region-based segmentation and / orclustering methods can be utilized. Alternatively or additionally, the feature extractor module 148A may comprise principal components analysis (PCA) configured technologies and / or autoencoders.

[0322] In embodiments in which the sensor measurement data comprises thermal (IR) data, the feature extractor 150A may be configured to process the thermal image data to extract statistical features relating to the thermal profiles, e.g., max / min / average temperatures, and generates timeseries tags of these feature data and forwards for further downstream processing. Multiple feature extractors (A, B, C...) may be configured operate in parallel to extract different features.

[0323] The feature extractor module 150A may utilize and suitable extraction methodology and may comprise any suitable processing units to achieve the desired functionality. For example, the feature extractor module 150A may utilize fast Fourier transform (FFT) processing as is known in audiovisual signal processing. Alternatively, classifiers utilizing techniques such as thresholding, edge detection, region-based segmentation and / or clustering methods can be utilized.

[0324] Alternatively or additionally, the feature extractor module 150A may comprise principal components analysis (PCA) configured technologies and / or autoencoders. If machine learning classifiers are used, then convolutional neural networks (CNNs) and deep learning networks (DNNs) or other forms of artificial neural network (ANN) may be utilized.

[0325] Step 312A: Perform statistical analysis

[0326] Step 312A may be optional. At step 312A, statistical analysis on the data features as a function of time point may be performed to generate statistical indications representative of deviations from expected behavior of the one or more elements of process equipment in the operational area.

[0327] In embodiments, the statistical module 158 may be utilized to provide statistical insights. The statistical module 158 may take any form suitable for processing the extracted data features comprises one or more statistical engines and receives inputs from the feature extractor 148A, IR feature extractor 150A and ROI module 156. In addition, the statistical module 158 has access to the data stored in the historian 120, together with process equipment data from the plant historian to obtain and analyze data from other plant processes and or / equipment.

[0328] In step 312A, the statistical module 158 is operable to generate statistical models of the received data and compare those models to the current values of measured data at specific time points to detect significant deviations from normal operation.

[0329] For example, an overheating machine would generate a higher than normal value for a corresponding ‘temperature’ feature in the IR images of the machine. The statistical module 158 may, in embodiments, utilize historical timeseries data as training data to build one or more internal uni-variate and / or one or more multi-variate data models.

[0330] The identified statistical insights and any deviations so identified may be utilized in later steps to generate an alert to a user.

[0331] Step 312B: Utilize process equipment monitoring data

[0332] Step 312B may be optional. In embodiments, at step 312B, further data sources may also be used in combination with the obtained sensor data for data processing and analysis.

[0333] One or more of the elements of process equipment 14-1 , 14-2, 14-3, 14-4 may have one or more sensors (integrated or external) for monitoring the performance thereof. In embodiments, the process equipment 14-1 , 14-2, 14-3, 14-4 may comprise one or more industrial machines, ancillary elements and / or product storage elements such as compressors; pumps; cryogenic pumps; heat exchangers; reactors; furnaces; boilers; distillation columns; refrigeration units; pipework; pressure vessels; storage units; loading / unloading equipment; mixers; adsorption beds; stripping columns; bulk storage; dryers; drum or cylinder storage handlers; robotic handlers; crushers; and grinders.

[0334] Any of these elements of process equipment 14 may comprise suitable sensors to measure operational parameters thereof. For example, a pump or compressor may record data representative of measurable parameters such as motor speed, flow rate, pressure and / or power draw.

[0335] In embodiments, at step 312B this data, which may be recorded separately as part of the standard plant operational and monitoring systems and stored as a simple timeseries data set with a single numeric value for each recorded timestamp, can be processed with the statistical data in step 256A and / or feature data extracted in step 254.

[0336] By utilizing direct operational parameter data of the process equipment 14-1 , 14-2, 14-3, 14-4 in combination with the sensor data, at step 312B cross-correlations and crossrelationships can be determined.

[0337] For example, consider a motor of a pump or compressor. If the direct operational parameter data of the motor indicates an increase in amperage, this may have an effect on other sensor readings simultaneously, such as a measured increase in flow for the pump or compressor and a measurable increase in motor speed. These data values may be captured by direct operational parameter data of the motor direct sensors. In addition, one or more sensors 50 may detect increased sound loudness, a change in frequency, increased vibration in the region of the motor and increased temperature.

[0338] Alternatively, a sudden emergence of thermal and acoustic signature from a point in the plant floor may indicate a new or unauthorized machine running if no new data sources or changes are noticed in the direct operational parameter data.

[0339] At step 312B, this data is processed and / or combined with the statistical data in step 312A (if implemented) and / or feature data extracted in step 310.

[0340] Step 314: Generate feature maps

[0341] At step 314, the sensor measurement data is processed using a computer system (for example, analyzer 146) to generate one or more feature maps representative of a spatial location and / or distribution of the measured values of at least one physical parameter within the operational area at each discrete time point.

[0342] The feature maps may be generated for each sensor modality in accordance with the methods described in the "analyzer 146” section above. For example, in the case of acoustic data, the sensor fusion module 148B may be operable to process the acoustic data at time t, from all the acoustic sensors 50S-n to generate an estimated 3D sound field Ltfor the entire plant operational area 12 at time t.

[0343] Then, the beamforming module 148C comprising one or more beamforming algorithms may operate on the acoustic data Sn,rto generate one or more acoustic images, Fn,t , where each image F is for a sensor device 50S-n at time t. Each acoustic image F comprises a spatial picture of the sound sources. In other words, the acoustic image F comprises a 2D or 3D image of sound loudness across different frequency bands across a predefined space. In embodiments, the predefined space comprises the 2D field-of-view (FOV) of the sensor element 52 of the acoustic sensor device 50S-n.

[0344] In the case of thermal (IR) data, the feature extractor 150 A may be operable to process the thermal image data to extract statistical features relating to the thermal profiles, e.g., max / min / average temperatures, and generates timeseries tags of these feature data andforwards for further downstream processing. Multiple feature extractors (A, B, C...) operate in parallel to extract different features which may then be used on the feature map. In general, IR images are already in a spatial format for further processing.

[0345] By “feature map” is meant a numerical representation of the spatial location and / or spatial distribution of the measured values of at least one physical parameter as a function of the spatial dimensions of the operational area. In particular, the operational area may be defined as a two or three dimensional space and the spatial location and / or distribution of at least one measured physical parameter may be overlaid on a representation of that space.

[0346] The feature map may comprise an array or matrix of spatial co-ordinates with corresponding scalar values. This may, in embodiments, be utilized as a data array for analysis and data processing to identify insights. Optionally, the map may be represented graphically for a user as needed, and as described in specific embodiments below.

[0347] In embodiments, the defined feature map may represent the determined or calculated spatial distribution of the values of a particular parameter. The generated distribution may be substantially continuous in space even though measurements may be performed at discrete locations. This analysis enables data to be generated for regions where it may not, for example, be possible to locate a particular sensor or other monitoring equipment.

[0348] In other words, in embodiments, the feature maps and / or combined feature maps may comprise model-based or data-based inference tools configured to predict values at any point in the operational space. In embodiments, as described below, this may be independent of the visualization of the data itself.

[0349] In embodiments, the feature maps or combined feature maps may utilize “reasonable parametrized distributions” such as basis functions or parametrized manifolds, where the obtained sensor data may be used to estimate the parameters of the manifold. In embodiments, a manifold may be operable to provide a more realistic distribution of scalar fields for parameters such as loudness.

[0350] If, for example, only bi-linear interpolation or distance-weighted averages are used for each point inside the field, then the value inside the field can never exceed the values of the sampled points. However, considering that sound sources will be located in the interior regions, it is reasonable to expect the largest sound amplitude will be somewhere inside the region and not at the point of sampling. Again, this provides a method to estimate the loudness at any point inside the field without the requirement to locate a sensor close to the point of interest.

[0351] In embodiments, at step 314 the sensor measurement data is processed using a computer system (for example, analyzer 146) to generate one or more combined feature maps representative of a spatial location and / or distribution of the measured values of two or more of the plurality of physical parameters within the operational area at each discrete time point.

[0352] By “combined feature map” is meant a numerical representation of the spatial location and / or spatial distribution of the measured values of two or more of the plurality of physical parameters as a function of the spatial dimensions of the operational area. In particular, the operational area may be defined as a two or three dimensional space and the spatial location and / or distribution of at least two of the measured physical parameters may be overlaid on a representation of that space.

[0353] The combined feature map may comprise an array or matrix of spatial co-ordinates with corresponding scalar values. This may, in embodiments, be utilized as a data array for analysis. Optionally, the maps may be represented graphically for a user as needed, and as described in specific embodiments below.

[0354] In embodiments the sensor measurement data comprises at least acoustic, thermal and visual physical parameter data. In such embodiments, the overlay module 154 is configured to receive IR images from the thermal sensor devices 50IR, images acoustic images Fn,tfrom the beamforming module 148C, and visual images from the visual processing module 152.

[0355] In the overlay module 154, the different images from the acoustic, thermal and visual modalities are joined together. In embodiments, this process comprises the application of a homography matrix and panorama stitching algorithms operable to combine the images from different sensor modalities to form multi-modal output images, Mn,t (Figure 4) that shows acoustic, thermal, visual data in a unified view.

[0356] In embodiments, the processing in step 314 may comprise homography and / or imageregistration of the visual camera 50V image(s) with one or more IR images from a corresponding IR sensor device 50IR on the same device or at the same device location.

[0357] Alternatively or additionally, step 314 may comprise homography and / or imageregistration of the visual camera 50V image(s) with one or more acoustic images from a microphone array of an acoustic device 50S in the same device 50 or in the same location as the visual sensor device 50V.

[0358] By providing such an approach, the captured sensor measurement data can be processed to create “features” using heuristics and / or artificial intelligence models. Features may comprise, but not be limited to, feature parameters such as sound “pressure level”, “loudness”, “shrillness”, heat map, “thermal eccentricity”, “thermal spread”. In step 314, these features can be fitted to a set of mathematical functions (basis functions) that are suitable for the selected feature type and physical process producing such data.

[0359] This processing in step 314 creates a real-time map of the physical properties being monitored across the whole operational area and provides a convenient method of creating continuous and spatially interpolate-able maps of all these types of sensor measurement data captured from the sensors 50.

[0360] In addition, such a continuous map provides a means to monitor parts of the plant that are not being directly monitored by the devices themselves. Additionally, one or more motion sensors 50M may be mounted on specific elements of process equipment to capture high frequency vibration scans. The combined feature maps may, in embodiments, be augmented by the data from the vibration to provide a “context” to the maps.

[0361] Once the feature maps or combined feature maps are generated, the method proceeds to step 316.

[0362] Step 316: Utilize feature maps to identify anomalies

[0363] At step 316, the one or more feature maps or combined feature maps may be used to identify operational anomalies within the operational area. In addition, if statistical analysis is performed in step 256A or cross-correlation of the feature map data with process equipment sensor data in step 256B is performed, this additional analysis can be combined as part of anomaly detection.

[0364] At step 316, the computer system is operable to identify operational anomalies based on the data obtained above. This data may be stored for further processing or correlation or may be communicated to the user as appropriate.

[0365] Anomaly detection based on the data sources has numerous advantages. For example, if elevated levels of loudness are identified in a corner of the operational area, vibration data relating to a machine starting up or changing mode provides an explanation. However, a sudden increase in loudness without a change in any other machine signature may indicate an anomaly in a system such as an actual or imminent failure or damage toequipment. The computer system, based on the above information, can usefully identify anomalies to assist with plant operation.

[0366] Step 318: Generate user visualization

[0367] Step 318 may be optional. At step 318 the feature maps or combined feature maps generated in step 258 may be presented to a user. At step 318, the one or more feature maps or combined feature maps may, in embodiments, be utilized to generate user alerts if operational anomalies are identified within the operational area.

[0368] In other words, the one or more (combined) feature maps provide, in embodiments, a visual indication to a user of the computer system of one or more alerts relating to identified operational anomalies within the operational area.

[0369] In embodiments, the visual indication may comprise a 2D or 3D spatial representation of the operational area showing values of at least one physical parameter as a function of spatial location. Spatial representation of the operational area is intended to be non-limiting and does not imply any graphical or structural reconstruction of the precise components in the operational area.

[0370] For example, a 2D spatial representation of a rectangular operational area may simply comprise a displayed rectangle of appropriate dimensions having the values of at least one physical parameter displayed thereon or adjacent. The values may be in the form of a “heat map” with color or shade used to denote different magnitudes or values of the parameter across the spatial representation.

[0371] Alternatively or additionally, numerical or symbolic representations (e.g. arrows, peak values, contour lines) may be used to represent the spatial variation of the values of the physical parameter(s) as a function of spatial location.

[0372] In embodiments, the visual indication may comprise a 2D or 3D spatial representation of the operational area showing the spatial location of one or more alerts relating to identified operational anomalies within the operational area. The alerts could be mapped onto the spatial representation and highlighted or otherwise identified to the user in an appropriate manner.

[0373] Further, if generated, the feature maps or combined feature maps may be integrated with the statistical insights identified in step 312A to generate data to be presented to a user. In embodiments, at step 318, the one or more feature maps or combined feature maps together with the generated statistical indications may be utilized to generate user alerts if operational anomalies are identified within the operational area

[0374] In embodiments, at step 318, the extracted feature maps are overlaid and / or combined with one or more maps, plans or layouts of the plant operational area 12. For example, in embodiments, the extracted feature maps may be fused or overlaid on one or more computer aided design (CAD) models of the plant operational area 12. Such an overlay or fusion may show various parameters; for example, scalar field distributions such as loudness maps, thermal hotspot maps, average acoustic pitch maps, or more.

[0375] Further, step 318 may also involve generation of alerts and / or insights based on the statistical analysis. In this regard, in embodiments, the analyzer 146 may further comprise an alert module 158A operable to receive as an input data from the statistical module 158 and generate, if required, one or more alerts and / or insights.

[0376] In embodiments, the alert module 158A is operable to generate one or more alerts and / or insights in relation to one or more machines, equipment or conditions in response to parametric deviations identified by the statistical module 158. Parametric deviations may include deviations of the values of one or more parameters from a predicted or normal value of said one or more parameters.

[0377] In one embodiment, specific alerts may be generated which are sent to a user terminal as required. The alerts may take any suitable form and may be graded by severity, for example. Insights may not be in the form of alerts which require a response or action but instead may provide a user with information relating to an estimated parameter of likely interest to the user. For example, this may include estimated or predicted machine operation modes from the detected parameters.

[0378] The alert module 158A may, in embodiments, be configured to notify the user(s) by any suitable means. For example, the alert module 158A may be configured to communicate alerts and / or insights via any suitable communications means such as email, instant message, SMS (text message) via any suitable communications network such as the internet, Wi-Fi, 4G / 5G / LTE mobile networks, satellite communication networks or microwave networks.

[0379] It will be appreciated by the person of skill in the art that various modifications may be made to the above-described examples without departing from the scope of the invention as defined by the appended claims.

[0380] While the invention has been described with reference to the preferred embodiments depicted in the figures, it will be appreciated that various modifications are possible within the spirit or scope of the invention as defined in the following claims.

[0381] In this specification, unless expressly otherwise indicated, the word "or" is used in the sense of an operator that returns a true value when either or both of the stated conditions are met, as opposed to the operator "exclusive or" which requires only that one of the conditions is met. The word "comprising" is used in the sense of "including" rather than to mean "consisting of".

[0382] Where applicable, various embodiments provided by the present disclosure may be implemented using hardware, software, or combinations of hardware and software. Also, where applicable, the various hardware components and / or software components set forth herein may be combined into composite components comprising software, hardware, and / or both without departing from the spirit of the present disclosure. Where applicable, the various hardware components and / or software components set forth herein may be separated into sub-components comprising software, hardware, or both without departing from the scope of the present disclosure. In addition, where applicable, it is contemplated that software components may be implemented as hardware components and vice-versa.

[0383] Software, in accordance with the present disclosure, such as program code and / or data, may be stored on one or more computer readable mediums. It is also contemplated that software identified herein may be implemented using one or more general purpose or specific purpose computers and / or computer systems, networked and / or otherwise. Where applicable, the ordering of various steps described herein may be changed, combined into composite steps, and / or separated into sub-steps to provide features described herein.

[0384] While various operations have been described herein in terms of “modules”, “units” or “components,” these terms should not limited to single units or functions. In addition, functionality attributed to some of the modules or components described herein may be combined and attributed to fewer modules or components.

[0385] It will be apparent to those of ordinary skill in the art that changes, additions or deletions may be made to the disclosed embodiments without departing from the spirit and scope of the invention. For example, one or more portions of the methods described above may be performed in a different order (or concurrently) and still achieve desirable results.

Claims

1. CLAIMS2.1 . A computer-implemented method of monitoring an operational area of an industrial plant comprising one or more elements of process equipment, the method utilizing a monitoring system comprising a plurality of sensors located in the operational area and connected to a computer system by a network, the method being executed by at least one hardware processor and comprising the steps of:3.a) obtaining, by the computer system and for a plurality of discrete time points, sensor measurement data representative of measured values of one or more physical parameters, the sensor measurement data being synchronously captured from the plurality of sensors at each discrete time point;4.b) processing, using the computer system, the sensor measurement data for each discrete time point to generate one or more feature maps representative of a spatial location and / or spatial scalar field distribution of the values of at least one physical parameter within the operational area at each discrete time point; and c) utilizing, using the computer system, the one or more feature maps to identify operational anomalies within the operational area.

2. The computer-implemented method according to claim 1 , wherein step a) further comprises obtaining sensor measurement data representative of measured values of a plurality of different physical parameters and step c) further comprises generating one or more combined feature maps representative of a spatial location and / or spatial scalar field distribution of the values of the plurality of different physical parameters within the operational area at each discrete time point.

3. The computer-implemented method according to claim 1 , wherein the method further comprises the step of:7.d) utilizing, using the computer system, the one or more feature maps to provide a visual indication to a user of the computer system of one or more alerts relating to identified operational anomalies within the operational area.

4. The computer-implemented method according to claim 3, wherein the visual indication comprises a 2D or 3D spatial representation of the operational area showing values of at least one physical parameter as a function of spatial location.

5. The computer-implemented method according to claim 4, wherein the visual indication comprises a 2D or 3D spatial representation of the operational area showing values of a plurality of physical parameters as a function of spatial location.

6. The computer-implemented method according to claim 5, wherein the values of a plurality of physical parameters as a function of spatial location are overlaid on the same spatial representation.

7. The computer-implemented method according to claim 5, wherein the plurality of physical parameters are selected from the groups of: acoustic parameters, visible light parameters, infrared emission parameters and / or motion parameters.

8. The computer-implemented method according to claim 3, wherein the visual indication comprises a 2D or 3D spatial representation of the operational area showing the spatial location of one or more alerts relating to identified operational anomalies within the operational area.

9. The computer-implemented method according to claim 1 , further comprising, prior to step c), the steps of:13.e) extracting, using the computer system, one or more data features from the sensor measurement data for each discrete time point;14.f) performing, using the computer system, statistical analysis on the data features as a function of time point to generate statistical indications representative of deviations from expected behavior of the one or more elements of process equipment in the operational area; and15.g) utilizing, using the computer system, the statistical indications in step c) to identify operational anomalies within the operational area.

10. The computer-implemented method according to claim 1 , wherein step a) comprises:17.h) initiating a data capture event at a first time;18.i) capturing sensor measurement data from each of the sensors synchronously at the first time; and19.j) transmitting the sensor measurement data from sensor measurement data capture event to a server device.20.1 1 . The computer-implemented method according to claim 10, wherein step h) comprises sending an event trigger across a network.

12. The computer-implemented method according to claim 1 , further comprising:21.k) obtaining time-series process equipment data from one or more sensors associated with one or more operational parameters of the elements of process equipment; and22.l) utilizing the time-series process equipment data in combination with the sensor measurement data in step b) to generate the one or more feature maps.

13. The computer-implemented method according to claim 12, wherein step k) further comprises obtaining time-series process equipment data from one or more distributed control systems (DCS), programmable logic controllers (PLC) and / or human machine interfaces (HMI) of the industrial plant.

14. The computer-implemented method according to claim 1 , wherein the one or more physical parameters are selected from the groups of: acoustic parameters, visible light parameters, infrared emission parameters and / or motion parameters.

15. The computer-implemented method according to claim 14, wherein the acoustic parameters are selected from one or more of: acoustic amplitude, sound pressure, acoustic source location, acoustic frequency and acoustic frequency distribution.

16. The computer-implemented method according to claim 14, wherein the infrared emission parameters are selected from one or more of: thermal emission, temperature(s), spatial content and thermal distribution.

17. The computer-implemented method according to claim 14, wherein the motion parameters are selected from one or more of: vibrational amplitude, vibrational frequency, vibration frequency distribution, movement in one or more dimensions, acceleration in one or more dimensions, velocity in one or more directions.

18. The computer-implemented method according to claim 14, wherein one or more physical parameters comprises acoustic parameters and step b) further comprises: m) processing acoustic sensor data from a plurality of acoustic sensors to generate an estimated 3D sound field and / or utilizing one or more beamforming algorithms to generate one or more 2D or 3D acoustic images to form one or more of the feature maps.

19. A system for monitoring an operational area of an industrial plant comprising one or more elements of process equipment, the system comprising a plurality of sensorslocated in the operational area and connected to a computer system by a network, the computer system comprising at least one hardware processor configured to:29.obtain, for a plurality of discrete time points, sensor measurement data representative of measured values of one or more physical parameters, the sensor measurement data being synchronously captured from the plurality of sensors at each discrete time point;30.process the sensor measurement data for each discrete time point to generate one or more feature maps representative of a spatial location and / or spatial scalar field distribution of the values of at least one physical parameter within the operational area at each discrete time point; and31.utilize the one or more feature maps to identify operational anomalies within the operational area.

20. The system according to claim 19, wherein the at least one hardware processor is further configured to:33.obtain sensor measurement data representative of measured values of a plurality of different physical parameters; and34.generate one or more combined feature maps representative of a spatial location and / or spatial scalar field distribution of the values of the plurality of different physical parameters within the operational area at each discrete time point.

21. The system according to claim 19, wherein the at least one hardware processor is further configured to:36.utilize the one or more feature maps to provide a visual indication to a user of the computer system of operational anomalies within the operational area.

22. The system according to claim 21 , wherein the visual indication comprises a 2D or 3D spatial representation of the operational area showing values of at least one physical parameter as a function of spatial location.

23. The system according to claim 22, wherein the visual indication comprises a 2D or 3D spatial representation of the operational area showing values of a plurality of physical parameters as a function of spatial location.

24. The system according to claim 23, wherein the values of a plurality of physical parameters as a function of spatial location are overlaid on the same spatial representation.

25. The system according to claim 23, wherein the plurality of physical parameters are selected from the groups of: acoustic parameters, visible light parameters, infrared emission parameters and / or motion parameters.

26. The system according to claim 21 , wherein the visual indication comprises a 2D or 3D spatial representation of the operational area showing the spatial location of one or more alerts relating to identified operational anomalies within the operational area.

27. The system according to claim 19, wherein the at least one hardware processor is further configured to:41.extract one or more data features from the sensor measurement data for each discrete time point;42.perform statistical analysis on the data features as a function of time point to generate statistical indications representative of deviations from expected behavior of the one or more elements of process equipment in the operational area.

28. The system according to claim 19, wherein the at least one hardware processor is further configured to: initiate a data capture event at a first time; capture sensor measurement data from each of the sensors synchronously at the first time; and cause the transmission of the sensor measurement data from sensor measurement data capture event to a server device.

29. The system according to claim 28, wherein the at least one hardware processor is further configured to: send an event trigger across a network to initiate the data capture event at the first time.

30. The system according to claim 19, wherein the at least one hardware processor is further configured to: obtain time-series process equipment data from one or more sensors associated with one or more operational parameters of the elements of process equipment; and utilize the time-series process equipment data in combination with the sensor measurement data to generate the one or more feature maps.46.31 . The system according to claim 30, wherein the at least one hardware processor is further configured to: obtain time-series process equipment data from one or more distributed control systems (DCS), programmable logic controllers (PLC) and / or human machine interfaces (HMI) of the industrial plant.

32. The system according to claim 19, wherein the one or more physical parameters are selected from the groups of: acoustic parameters, visible light parameters, infrared emission parameters and / or motion parameters.

33. The system according to claim 32, wherein the acoustic parameters are selected from one or more of: acoustic amplitude, sound pressure, acoustic source location, acoustic frequency and acoustic frequency distribution.

34. The system according to claim 32, wherein the infrared emission parameters are selected from one or more of: thermal emission, temperature(s), spatial content and thermal distribution.

35. The system according to claim 32, wherein the motion parameters are selected from one or more of: vibrational amplitude, vibrational frequency, vibration frequency distribution, movement in one or more dimensions, acceleration in one or more dimensions, velocity in one or more directions.

36. The system according to claim 32, wherein one or more physical parameters comprises acoustic parameters and the at least one hardware processor is further configured to process acoustic sensor data from a plurality of acoustic sensors to generate an estimated 3D sound field and / or utilizing one or more beamforming algorithms to generate one or more 2D or 3D acoustic images to form one or more of the feature maps.

37. A non-transitory computer readable storage medium storing a program of instructions executable by a machine to perform a method of monitoring an operational area of an industrial plant comprising one or more elements of process equipment, the method utilizing a monitoring system comprising a plurality of sensors located in the operational area and connected to a computer system by a network, being executed by at least one hardware processor and comprising the steps of:52.a) obtaining, by the computer system and for a plurality of discrete time points, sensor measurement data representative of measured values of one or more physical parameters, the sensor measurement data being synchronously captured from the plurality of sensors at each discrete time point; b) processing, using the computer system, the sensor measurement data for each discrete time point to generate one or more feature maps representative of a spatial location and / or spatial scalar field distribution of the values of at least one physical parameter within the operational area at each discrete time point; and53.c) utilizing, using the computer system, the one or more feature maps to identify operational anomalies within the operational area.

Images