METHOD AND DEVICE FOR MACHINE MONITORING AND COMPUTER PROGRAM PRODUCT FOR MACHINE MONITORING

Description

[0001] The invention relates to a device for machine monitoring in the sense of a device for recording and evaluating sensor data, in particular image data, for monitoring machines and systems and processes carried out with these machines or systems.

[0002] Furthermore, the invention relates to a method for machine monitoring in the sense of a method for recording and evaluating sensor data, in particular image data, for monitoring machines and systems and processes carried out with these machines or systems.

[0003] Furthermore, the invention relates to a computer program product for machine monitoring, which, when installed on suitable hardware, serves to acquire and evaluate sensor data, in particular image data, for monitoring machines and systems and processes carried out with these machines or systems.

[0004] Today's production and processing machines and systems are generally highly automated. Complex and rapid mechanical processes take place on often large and complex systems. Malfunctions of such machines or systems can occur, particularly during commissioning, after conversions, or due to changing production parameters. These malfunctions can lead to defective products or damage to the machines or systems themselves.

[0005] Malfunctions of machines or systems, or of processes performed by these machines or systems, often involve complex and difficult-to-see machine operations. Furthermore, processes or individual process steps may be too fast or occur too quickly to be perceived by the human eye.

[0006] Therefore, automated devices and procedures are used for machine monitoring, which use technical means to create the necessary conditions for effective monitoring.

[0007] Corresponding devices and methods are already known that incorporate cameras for monitoring machines and systems, or the processes performed with them. Monitoring can be carried out using the image data captured by the cameras. This monitoring serves, for example, to detect defective products manufactured with the machines or systems. The evaluation of the images captured with the cameras, or video sequences consisting of multiple images, allows conclusions to be drawn about, for example, incorrectly set process parameters and other sources of error in the operation of the machines and systems, as well as in the processes performed with these machines or systems.

[0008] The continuous monitoring of machines, systems, and the processes performed by these machines or systems generates a large amount of data that must be stored and evaluated to detect and correct errors. Especially with high-resolution image data captured with fast cameras, large storage facilities are required to store these volumes of data. However, malfunctions often occur only sporadically, so large amounts of data must first be evaluated to even find the relevant time range in the image data.

[0009] Furthermore, the data collected using known machine monitoring devices and methods is usually only available locally, so experts must evaluate the data on-site to identify and correct errors.

[0010] From the document US 2020 / 336659 A1, a surveillance system with cameras connected via a network is known, whereby the failure of a master camera triggers the transmission of image sequences from another camera of the surveillance system.

[0011] Document US 2015 / 213838 A1 discloses a network-based system for monitoring machines using cameras. The image data from the cameras is temporarily stored in a ring buffer. The detection of a machine malfunction generates a trigger signal, which leads to the retrieval and transmission of the video sequences stored in the ring buffer to a data server, thus reducing the amount of data to be evaluated in the event of a fault. Furthermore, the error clock is synchronized with the image data.

[0012] The devices and methods for machine monitoring known from the prior art therefore have various disadvantages which the present invention is intended to remedy.

[0013] An object of the invention is therefore to provide an improved device for machine monitoring.

[0014] A further object of the invention is to provide a device for machine monitoring which automatically reduces the amount of data to be evaluated to at least one relevant time range.

[0015] These objects are achieved according to the invention by a device for machine monitoring according to patent claim 1.

[0016] Another object of the invention is to provide an improved method for machine monitoring.

[0017] This object is achieved according to the invention by a method for machine monitoring according to patent claim 6.

[0018] A further object of the invention is to provide a computer program product for machine monitoring that solves the aforementioned problems.

[0019] This object is achieved according to the invention by a computer program product for machine monitoring according to patent claim 14.

[0020] The features of a machine monitoring device disclosed below are part of the invention both individually and in all possible combinations.

[0021] A device according to the invention for machine monitoring has at least one camera and one control unit as well as one evaluation unit or at least one control and evaluation unit.

[0022] At least one camera is used to capture image data. The camera's captureable image section is adapted to the machine or system being monitored, or to the process being performed by the respective machine or system. This allows for targeted monitoring of a specific machine area.

[0023] Preferably, the at least one camera has an image resolution of at least 1280 x 720 pixels and an image capture rate of approximately 25 fps (frames per second) to at least 240 fps. However, high-speed cameras with image capture rates of approximately 1000 fps can also be used.

[0024] In a preferred embodiment, the at least one camera has an image resolution of at least 1280 x 720 pixels and an image capture rate of at least 30 fps. A higher temporal resolution enables the evaluation of faster processes.

[0025] In a further preferred embodiment of the invention, the at least one camera has an image resolution of at least 1920 x 1080 pixels. A higher resolution enables a more detailed analysis of the captured images, but requires more capacity for image analysis and storage.

[0026] It is advantageous to use an image acquisition rate that is just sufficient to adequately resolve the relevant processes in order to limit the amount of data to be stored and analyzed to what is necessary.

[0027] In an advantageous embodiment, a machine monitoring device according to the invention comprises a plurality of cameras. For example, a machine monitoring device according to the invention comprises 2, 3, or 4 cameras.

[0028] The use of multiple cameras enables the simultaneous monitoring of different machine areas. This allows process steps occurring at different locations on a machine to be simultaneously monitored and investigated for possible causalities.

[0029] In practice, the use of 2, 3 or 4 cameras has proven to be optimal for most applications.

[0030] The at least one camera is connected to the control unit for exchanging data and preferably for controlling the at least one camera.

[0031] This connection is realized using a network module of the machine monitoring device.

[0032] Particularly preferably, the network module is designed to establish a wireless connection to at least one camera.

[0033] In one embodiment of the invention, the at least one camera is connected to the control unit of the machine monitoring device using a WLAN connection.

[0034] Furthermore, the machine monitoring device has at least one storage unit for storing the acquired data. For example, the storage unit is designed as a hard disk.

[0035] In embodiments of the invention, the evaluation unit, the control unit, and the at least one storage unit are integrated into a common housing, while the at least one camera can be positioned independently of these on the machine or system to be monitored. The network module is connected to the control and evaluation units as a separate module.

[0036] In other embodiments of the invention, the evaluation unit, the control unit, the at least one storage unit and the network module are integrated into a common housing, while the at least one camera can be positioned independently of these on the machine or system to be monitored.

[0037] In further embodiments of the invention, the latter comprises a first and a second evaluation unit, wherein at least one of the evaluation units is not integrated into a common housing with the control unit.

[0038] The components integrated into a common housing form the central unit, while at least one camera and, if applicable, additional sensors form the satellites. The central unit comprises at least the evaluation unit, the control unit or the control and evaluation unit, and at least one storage unit.

[0039] The machine monitoring device according to the invention preferably has at least one display via which the acquired data can be output to a user. This allows for immediate on-site data evaluation.

[0040] For example, the display is integrated into the central unit of a machine monitoring device according to the invention.

[0041] If the machine monitoring device does not have an integrated display, in embodiments of the invention at least one external display can be connected to the machine monitoring device or connected to it via a local network.

[0042] However, it is also possible according to the invention not to provide a display on the machine monitoring device itself, so that an evaluation of the image data by experts is only possible "remotely" after the image data has been uploaded to the Internet.

[0043] At least one camera of a machine monitoring device according to the invention is designed as a trigger camera with which the recording of image data can be triggered.

[0044] The use of a trigger camera allows the analysis of the recorded data to be focused on a relevant time period, so that only a part of the total recorded data volume needs to be evaluated.

[0045] The trigger camera is connected to the evaluation unit in such a way that the image data captured using the trigger camera can be continuously evaluated for the occurrence of a predetermined trigger event.

[0046] In embodiments with a first and a second evaluation unit, the trigger camera is connected to the first evaluation unit in such a way that the image data acquired using the trigger camera can be continuously evaluated for the occurrence of a predetermined trigger event.

[0047] The predetermined trigger event is set based on the machine or system to be monitored or the process carried out with the respective machine or system.

[0048] As soon as the trigger camera detects a trigger event that is recognized as such by the evaluation unit, the image data resulting from the continuous recording with at least one camera can be reduced to a relevant time window. The image data from this time window can then be stored for evaluation and analysis using the storage unit.

[0049] This further evaluation and analysis can be carried out in embodiments with two evaluation units using the second evaluation unit.

[0050] In advantageous embodiments of the invention, the time window can be set for all cameras and / or individually for each of the cameras. This is particularly beneficial for limiting the data volumes as effectively as possible, particularly with regard to the arrangement of the cameras on the machine or system depending on the process being performed with the machine or system. For example, if a camera is positioned before the trigger camera in the temporal process sequence, a different time window is relevant than if the camera is positioned after the trigger camera in the temporal process sequence.

[0051] Preferably, the absolute length of the time window and / or the position of the time window can be configured depending on the trigger event.

[0052] In the simplest case, the detection of the trigger event defines a fixed time window around the trigger event for the image data to be saved.

[0053] For example, this is realized by video sequences of predetermined length, whereby those video sequences are discarded which lie completely before the time window surrounding the trigger event, and in addition to the video sequence containing the trigger event and, depending on the embodiment of the invention, video sequences recorded in parallel with other cameras, a defined number of video sequences following the video sequence containing the trigger event are included.

[0054] In embodiments of the invention, the machine monitoring device is designed to detect a trigger event by detecting a movement or detecting a standstill.

[0055] For this purpose, in preferred embodiments of the invention, the evaluation unit is designed to detect differences in successive images. Further details are disclosed in the machine monitoring method according to the invention.

[0056] The use of multiple cameras requires temporal synchronization of the image data for reliable evaluation and analysis of dependencies between the events recorded with the respective cameras. Detection of the trigger event using the trigger camera also requires temporal classification of the trigger event in relation to the image data of the respective camera in order to limit the time frame of the image data recorded with at least one additional camera.

[0057] A delay in the image data transmitted from the various cameras to the evaluation unit results in particular from different data transmission times from the respective camera to the evaluation unit or the transmission of a control command from the control unit to the respective camera.

[0058] In advantageous embodiments of the invention, the machine monitoring device is therefore designed to synchronize the image data captured by the cameras.

[0059] In particular, if a WLAN connection to the at least one camera of the machine monitoring device is realized using the network module, time synchronization is required, since synchronization is not supported by the WLAN protocol itself.

[0060] Details of an advantageous implementation of the synchronization are disclosed in the course of the method according to the invention for machine monitoring.

[0061] In embodiments according to the invention, a machine monitoring device is designed to capture and evaluate data from additional sensors (e.g., temperature, vibration, acceleration sensors, or thermal imaging cameras). This enables the analysis of the influence of measured variables that may not be detectable, or only partially detectable, with conventional cameras on a malfunction of a machine or system.

[0062] In embodiments of the invention, the device for machine monitoring according to the invention has its own sensors, which can be placed in the area of ​​the machine or system to be monitored. In other embodiments, the device for machine monitoring according to the invention has an interface for connecting to the machine or system to be monitored, so that the data from sensors installed in the machine or system itself can also be evaluated using the device for machine monitoring. A combination of the above-mentioned alternatives is also possible, so that a device for machine monitoring according to the invention is designed in embodiments to evaluate the image data acquired using the at least one camera and / or the measurement data from other sensors of the device for machine monitoring and / or the measurement data from sensors installed in the machine or system itself.

[0063] In advantageous embodiments of the invention, the machine monitoring device comprises an internet module for establishing a connection to the internet. This preferably enables remote access to the data acquired using the machine monitoring device, allowing external experts to be involved in evaluating and analyzing the data without requiring them to be on-site.

[0064] Remote access to the acquired data can be achieved either by connecting to the device according to the invention for machine monitoring or by uploading the acquired image data to the Internet, for example to the cloud.

[0065] In embodiments of the invention, the network module and the Internet module are integrated into a common network and Internet module.

[0066] In embodiments with two evaluation units, the second evaluation unit can be arranged spatially independently of the rest of the device and connected to it via an Internet connection.

[0067] In a preferred embodiment of the invention, the at least one camera is equipped with a camera mount by means of which the camera can be fastened to the machine or system or in the area of ​​the machine or system and can be aligned to the desired detection area.

[0068] In a particularly preferred embodiment of the invention, the camera mount has a base that can be releasably connected to a support and to which a flexibly deformable leg is attached. The flexibly deformable leg has sufficient rigidity to maintain a set position. This can be achieved, for example, by a leg formed from several correspondingly rigid ball joints. At the end of the leg, the camera mount has a receiving device for the camera.

[0069] The base of the camera mount is advantageously magnetic, allowing the camera mount to be magnetically connected to a suitable support. This is often the case in the area of ​​machines or systems to be monitored.

[0070] The method steps of a method for machine monitoring according to the invention disclosed below are part of the invention both individually and in all executable combinations.

[0071] A method according to the invention for machine monitoring comprises the camera-based monitoring of a machine or at least one area of ​​a machine with at least one camera, wherein the image data acquired with the aid of the at least one camera are evaluated.

[0072] In preferred embodiments of the method according to the invention, image data from each camera of an area and / or viewing angle of the machine or system to be monitored are simultaneously recorded and evaluated using 2, 3 or 4 cameras.

[0073] In further advantageous embodiments of the invention, additional sensor data is also recorded and evaluated. Such additional sensor data can, for example, be temperature, vibration, or acceleration data recorded with appropriate sensors, or images captured with thermal imaging cameras.

[0074] Since errors in manufacturing processes recorded using the method according to the invention often occur only sporadically and large amounts of data are generated during continuous recording and storage of the image data from at least one camera and / or other sensors, which require a lot of storage space and whose detailed analysis is complex, a camera is configured according to the invention as a trigger camera.

[0075] The trigger camera continuously captures image data within the camera's field of view. This image data is continuously evaluated for the occurrence of a trigger event. As soon as a trigger event is detected, the video captured by at least one camera is limited (or shortened) to a predefined time window. This significantly reduces the amount of data to be stored and evaluated.

[0076] This windowing is also applied in corresponding embodiments of the invention for other possibly acquired sensor data.

[0077] In one embodiment of the invention, video sequences of a predetermined length, such as 50 seconds, are continuously recorded with the at least one camera. These video sequences are then evaluated one after the other for the occurrence of a trigger event, while the next video sequence is simultaneously being recorded with the at least one camera.

[0078] In a preferred embodiment of the method according to the invention, videos are recorded in parallel with 2, 3 or 4 cameras, while one of the cameras is configured as a trigger camera.

[0079] The windowing that occurs upon the occurrence of a trigger event in the video of the trigger camera and the subsequent evaluation of the videos of the other cameras or other sensors requires a temporal synchronization of the videos and, if applicable, the other sensor measurement data with each other, since otherwise the temporal reference of the time-limited videos or sensor measurement data cannot be guaranteed.

[0080] Such temporal synchronization is particularly necessary when connecting at least one camera and at least one other camera and / or another sensor via a network, whereby the transmission delays of the individual data transmission paths (i.e., from a control and evaluation unit to the respective camera or sensor and back) may differ from one another. This is the case, for example, when connecting cameras or sensors via a WLAN connection. Furthermore, current WLAN protocols do not offer their own synchronization mechanisms.

[0081] In an advantageous embodiment of a method according to the invention for machine monitoring, a temporal synchronization of the acquired image data from different cameras and, if applicable, the additionally acquired sensor data is carried out.

[0082] In one embodiment of the invention, the image or sensor measurement data is synchronized by sending a time query to the respective cameras or sensors. Each camera or sensor must therefore have its own local clock (real time clock - RTC). At the time of the time query at the respective camera or sensor, the time of the local clock is read and transmitted via the network to the central control and evaluation unit or to the network module. As soon as the local time of the respective camera or sensor is received by the control and evaluation unit or the network module, it reads the time of its local clock and determines the difference between the times. This difference is then assumed to be the channel delay. The channel delay of a channel is preferably determined by averaging over several measurements.

[0083] By measuring the channel delays of the various channels, in embodiments of the invention the relative delay from channel to channel is determined so that the data collected by means of the cameras or the at least one camera and at least one further sensor can be synchronized in time.

[0084] In a preferred embodiment of the invention, after the detection of a trigger event, the synchronization of the data received via the various channels and the subsequent windowing of the data, the video recordings and, if applicable, the additional sensor data are immediately made available in an analysis mode.

[0085] Depending on the design of the process, the data is presented to a user of the process on-site in a visually prepared form on a display and / or made available for analysis via the Internet.

[0086] In the analysis mode, in a preferred embodiment of the method according to the invention, the video recordings of all cameras and, if applicable, the measurement data of additional sensors are displayed synchronously for parallel evaluation.

[0087] In embodiments of the invention, this representation is realized by a display divided according to the number of cameras and / or sensors, so that the respective data are visible side by side.

[0088] In embodiments of the invention, an evaluation of the image data or the further sensor measurement data frame-by-frame (or measurement value by measurement value) and / or in slow motion and / or with a zoom function is possible in the analysis mode.

[0089] Furthermore, in preferred embodiments of the invention, a color filter can be applied to image data, so that analysis can be facilitated by masking out colors and / or filtering out a single color. Such a color filter can also be used in embodiments of the invention when evaluating the image material with regard to the presence of a trigger event.

[0090] When the color filter is used, the image data captured by at least one camera (3) is processed using a color filter in such a way that the image data set processed using the color filter only has a defined, limited color range. In advantageous embodiments of the invention, videos intended for storage are subjected to video rendering or video compression automatically or following manual input, so that a smaller file size is achieved for exporting these video sequences (e.g. as an mpeg file). The synchronized video sequences of the individual cameras can therefore also be played back and evaluated as a rendered video file on any device with corresponding common playback software and independently of the special software on an inventive device for machine monitoring or the implementation of an inventive method for machine monitoring in software.

[0091] In a particularly preferred embodiment of the method according to the invention, remote access to and / or cloud sharing of the acquired and at least temporarily stored data is implemented. For this purpose, internet-based external access to the data and / or uploading of the data to a web server is implemented, from which the data can be retrieved for analysis and evaluation.

[0092] This enables the involvement of external experts to detect errors in the operation of the machine or system and to correct these errors.

[0093] In preferred embodiments of the invention, an automatic notification of one or more defined users is implemented upon detection of a trigger event, e.g. by e-mail, SMS or push message.

[0094] This allows those responsible for detecting and correcting errors to immediately begin evaluating the collected data without having to wait for unnecessary time.

[0095] In a further advantageous embodiment of the method according to the invention, a visual and / or acoustic detection of errors or a quality control by a master frame comparison is implemented.

[0096] A sequence of the corresponding sensor measurement data, preferably a video sequence, is recorded in a fault-free process or sub-process. It is assumed that a periodic process is occurring in the process or sub-process under consideration.

[0097] This is the case, for example, when the manufactured and ideally identical products in the corresponding area pass through the image area one after the other. A master frame is defined from this sequence. This master frame is either an image in which a specific quality feature or suspected defect is clearly visible, or an acoustic spectrum at a specific point in time.

[0098] The sensor measurement data is recorded according to the set sampling rate at a series of data acquisition times.

[0099] During the ongoing process or sub-process, each period of sensor measurement data is analyzed for the presence of the defined master frame by automatically comparing the sensor measurement data at the individual data acquisition points with the master frame. If a sufficient match between the recorded sensor measurement data and the master frame is determined at at least one data acquisition point in a period, this period is considered error-free. Conversely, a period is considered error-prone if no sufficient match with the master frame could be determined.

[0100] In a preferred embodiment of a method for machine monitoring according to the invention, a device for machine monitoring according to the invention is used.

[0101] A computer program product according to the invention, installed on suitable hardware, serves to acquire and evaluate sensor data for monitoring machines and systems and processes carried out with these machines or systems.

[0102] A computer program product according to the invention for machine monitoring comprises instructions which, when executed by at least one computer, cause the computer to carry out a method according to the invention for machine monitoring.

[0103] In preferred embodiments of the invention, the computer program product comprises at least two software modules, wherein at least a first software module of the computer program product is designed for installation and execution on at least one camera and at least a second software module of the computer program product is designed for installation and execution on a central unit and / or a network module.

[0104] In addition to monitoring machines and systems, the invention also allows for monitoring other objects and processes. For example, it is also intended for use in the field of sports, for example, to detect certain situations.

[0105] Exemplary embodiments of the invention are schematically illustrated in the figures described below. They show: Figure 1: A schematic representation of an inventive device for machine monitoring in the area of ​​a machine, Figure 2: A block diagram of an inventive embodiment of a device for machine monitoring, Figure 3: A schematic flow diagram of the implementation of a trigger camera in an inventive device and in an inventive method for machine monitoring, Figure 4A: A schematic representation of a single recorded image from an area of ​​a monitored machine, Figure 4B: A schematic representation of the image evaluation for detecting the presence of a trigger event, Figure 5: A diagram for image evaluation for detecting the presence of a trigger event, Figure 6: Another diagram for image evaluation for detecting the presence of a trigger event, Figure 7: A diagram for image evaluation without a color filter, Figure 8: A diagram for image evaluation with a color filter,Figure 9: A diagram for image evaluation for a master frame comparison and Figure 10: A schematic flow diagram for synchronizing the various channels of a device or method according to the invention.

[0106] In Figure 1 A device according to the invention for machine monitoring (1) in the area of ​​a machine (100) is schematically shown. The device for machine monitoring (1) has a central unit (2) and four cameras (3) that are attached to the machine (100) to be monitored using camera mounts (4). By aligning the cameras (4), their detection areas are directed at specific areas of the machine (100), so that these areas can be monitored with one camera (3) each.

[0107] Figure 2shows a schematic block diagram of an embodiment of a machine monitoring device (1) according to the invention. This device has four cameras (3), each of which is directed at a specific area (I, II, III, IV) of the machine (100). The central unit (2) of the machine monitoring device (1) has a control unit (5), an evaluation unit (6), a storage unit (7), a network module (8), a display (9), and an input device (10).

[0108] The input device (10) serves to record user inputs on the central unit (2) of the machine monitoring device (1) and the display (9) serves to output image and / or measurement data to at least one user of the device (1).

[0109] The central unit (2) is connected to the cameras (3) by means of the network module (8), which is designed as a network and internet module. These connections are preferably implemented as WLAN connections. Furthermore, the image data captured by the cameras (3) and evaluated and processed in the central unit (2) can be uploaded from the central unit (2) to a web server in the cloud (11) using the network module (8) via an internet connection and / or remote access to the image data on the central unit (2) of the machine monitoring device (1) is enabled for at least one remote access station (12) via an internet connection.

[0110] In Figure 3The configuration of a camera (3) as a trigger camera (3a) in an embodiment of a machine monitoring device (1) and a machine monitoring method according to the invention is schematically shown. The video sequences are each of the same length, for example, 50 seconds.

[0111] With the help of the central unit (2) of the machine monitoring device (1), a command to start recording is sent to all cameras (3) (from the control unit (5) via the network module (8)). The cameras (3) then each record a first video sequence (Video 1). After the video sequences have been recorded, this is sent by the trigger camera (3a) to the central unit (2). The video recorded with the trigger camera (3a) is evaluated for the presence of a trigger event. Since no trigger event is detected in the present example, the remaining videos of the first video sequences are discarded or not retrieved by the central unit (2). After the first video sequences (Video 1) have been recorded, the cameras (3) immediately begin recording another video sequence (Video 2), so that the evaluation of Video 1 from the trigger camera (3a) takes place simultaneously in the central unit (2).After completing the recording of the second video sequence (Video 2), the trigger camera (3) sends the video sequence (Video 2) to the central unit (2), which evaluates the video for the presence of a trigger event. After completing the recording of the second video sequence, the cameras (3) immediately start recording a third video sequence (Video 3). Since a trigger event has now been detected in Video 2 of the trigger camera (3a), the third video sequences (Video 3) from the cameras (3) are relevant for evaluation and analysis.

[0112] In the example shown, a longer time window than the length of a video sequence is to be evaluated after the trigger occurs. Therefore, a post-trigger time is defined, which here corresponds to a predetermined time that must elapse before the video sequences from cameras (3) are retrieved. While the central unit (2) waits for the post-trigger time to expire, the cameras (3) record the next video sequence (video 4).

[0113] After the post-trigger time has elapsed, the remaining video sequences relevant for analysis are retrieved from the cameras (3). Since the trigger camera (3a) transmitted each video sequence to the central unit (2) immediately after the recording was completed, only the video sequences from the other cameras (3) need to be retrieved. Upon sending a corresponding command, the video sequences Video 3 and Video 4 are transmitted from the remaining cameras (3) to the central unit (2).

[0114] In the central unit (2), the video sequences from the cameras (3) are merged and combined into a data packet.

[0115] The data package contains the individual video sequences of the cameras as individual files and / or the video rendered from the individual synchronized video sequences of the cameras.

[0116] The data package is then uploaded to the cloud via an internet connection so that it can be evaluated by experts regardless of location.

[0117] Alternatively and / or additionally, the output of the video sequences directly at the central device (2) and / or remote access to the data packet from at least one remote access station is also possible.

[0118] In other embodiments of the invention, the trigger is realized by the occurrence of another event detected by the trigger camera and / or a change in state detected by another sensor.

[0119] In embodiments of the invention, the trigger event is detected by detecting a movement or a standstill. For example, the standstill of a machine part or of products conveyed by the machine (100) can imply a malfunction. Movement, for example, due to a product identified as defective by other mechanisms and ejected from the process, can also imply such a malfunction. Furthermore, other events, such as the illumination of a (warning) light or the change of a number or other display on a machine operating or monitoring module, are also suitable trigger events within the meaning of the present invention.

[0120] In embodiments of the invention, the detection of movement or standstill is carried out by calculating the differences between successive images (frames) of a video sequence.

[0121] Figure 4A shows such a frame of a recorded video sequence. Figure 4B shows a graphical representation of the calculation of the difference between individual pixels in two consecutive frames. A white pixel represents no change, and a black pixel represents a 100% change. The differences between them are assigned to corresponding grayscale levels.

[0122] In one embodiment of the invention, the frames of a video sequence are extracted for the difference calculation and converted into grayscale images. Each pixel has a value between 0 (black) and 255 (white). Each frame of the video sequence is compared with the temporally subsequent frame, whereby a structural similarity index of the two frames is calculated. This similarity index has a value between 0.0 and 1.0, where 1.0 means that the images are identical and 0.0 means that the images are completely dissimilar.

[0123] In a preferred embodiment of the invention, the method described in "Wang, Z., Bovik, AC, Sheikh, HR, & Simoncelli, EP (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13, 600-612" is used.

[0124] In one embodiment of the invention, the activity level is determined from the inverse of the structural similarity index, which lies between 0% and 100%. The activity level is 0% for identical images and 100% for completely different images. Figure 4B The grayscale image shown shows an activity level of 9.03%.

[0125] If the stopping of a monitored process is to serve as a trigger event, a threshold is defined depending on the process to be monitored and the activity levels expected in the running process. If the activity level falls below this threshold and remains below this threshold for a predefined period of time, the occurrence of a process stop is recognized as a trigger event.

[0126] In one embodiment of the invention, the threshold for detecting a process stop is defined at an activity level of 7%.

[0127] Figure 5shows a graphical representation of the evaluation of the activity level of a video sequence. At a time t of approximately 13 seconds, the activity level falls below 7%, so that a trigger event is detected. After the trigger event, the activity level remains at 0% for longer than a predetermined time (e.g., 5 seconds), meaning the monitored process has stopped (inactive phase).

[0128] Figure 6 shows a graphical representation of the evaluation of the activity level of a video sequence, but here the start of a movement is to be detected as a trigger event. For this purpose, a corresponding threshold value for the activity level is defined, and when exceeded, the trigger event is detected.

[0129] In the example shown, the threshold is set at an activity level of 30%. At time t, approximately 20 seconds, the trigger event is detected.

[0130] The threshold value for detecting movement as a trigger event is also preferably adapted to the activity levels expected in the process to be monitored.

[0131] Based on the Figures 7 and 8 The color filter of the present invention is explained in more detail. In corresponding embodiments of the invention, the color filter functionality allows focusing on a specific color, particularly for evaluating the video sequences of the trigger camera with regard to the presence of a trigger event.

[0132] Each color is assigned an RGB color value (for example, red value: 0x89, green value: 0x17, and blue value: 0x1f for a specific shade of red). A tolerance range is then defined for the color filter around this RGB color value, within which a detected color still corresponds to the defined color of the color filter. In one embodiment of the invention, the tolerance range lies at a distance of 30 around the RGB value of the defined color. In this example, the red value can range from 0x64 to 0xa0; accordingly, the value ranges of the green value and the blue value depend on the other color values ​​within the tolerance range.

[0133] The color filter is particularly used to detect more complex events by focusing the image analysis on a relevant color range.

[0134] Figure 7shows the activity level of a video sequence over time. A constantly varying and usually high activity level is evident throughout the entire video sequence. Figure 8 shows the activity level of the same video sequence with the color filter activated. Regarding the color defined in the color filter, only a low level of activity is evident throughout almost the entire duration of the video sequence. Only in the area just over 40 seconds is a strong increase in activity above the threshold of 6% defined here noticeable.

[0135] The color filter also makes it easy to detect color changes in (warning) lights on a machine or system being monitored.

[0136] In Figure 9 A similarity diagram of a video sequence is plotted over time. This is the functionality of the master frame comparison, in which each frame of a video sequence is compared with a predefined master frame.

[0137] The master frame comparison is suitable in cyclic processes to detect the successful execution of a process or errors in these processes.

[0138] The example shown is based on a video sequence of a bottle filling machine, where a label is applied to one bottle at a time in the monitored camera area. The master frame is defined as an image of a correctly labeled bottle. By comparing the master frames, it is possible to determine in each cycle whether the bottle was labeled successfully or whether there is an error in the process.

[0139] Figure 9shows a video sequence with four complete process cycles. High similarity values ​​of the evaluated frames with the master frame are detected between 3 and 4 seconds, 6 and 7 seconds, 9 and 10 seconds, and approximately at 13 seconds. The similarity values ​​are above the similarity threshold of 70% defined here, thus indicating a successful completion of the cycle in each case.

[0140] Figure 10 shows a schematic sequence of synchronization in a device according to the invention for machine monitoring (1) or in a method according to the invention for machine monitoring.

[0141] The central unit (2) sends a command to query the local time via the transmission channel (13) to a camera (3). The camera (3) reads the time from the local clock and sends it to the central unit (2) via the transmission channel (13). The transmission of the read time is delayed by the transmission delay. As soon as the central unit (2) has received the time from the clock of the camera (3), the central unit (2) reads its own local clock and determines the difference between its own time and the time read from the camera (2). This difference is stored as the delay of the transmission channel (13). This channel delay includes the delay of the transmission channel (13) in the network itself and, more importantly in this application, the time difference between the local clocks of the camera (3) and the central unit (2).

[0142] Since these clocks, especially the local clocks of different cameras (3), are not synchronized with each other, the clocks can (and will) output different values ​​when read at the same time. These local times are transmitted as timestamps with the recorded video sequences, so that the "same" times in the video sequences of different cameras (3) were not actually recorded at the same time.

[0143] However, by comparing the channel delay of the different channels to each other, the data sent from the respective cameras or sensors to the central unit (2) can be synchronized in time.

[0144] One assumption that is made is that the transmission delay of the different channels is approximately the same or that they differ only slightly from each other.

[0145] By repeating the time measurements and calculating the differences, a temporal change in the transmission delay of the channels themselves and of each other can be taken into account.

[0146] In a preferred embodiment of the invention, several time measurements (e.g., 10) are taken in quick succession, and the differences are calculated. The standard deviation is determined from the series of difference values ​​of a sequence, and a check is made to determine whether it is below a certain threshold (e.g., 20 ms). If this condition is met, the mean value of the differences in the sequence is stored as the delay.

Claims

1. Device for machine monitoring (1) comprising at least one camera (3) for capturing image data in the area of a machine (100), a control unit (5) for controlling the components of the device (1), an evaluation unit (6) for evaluating the image data captured with the aid of the at least one camera (3), a storage unit (7) for storing and / or temporarily storing the image data captured with the aid of the at least one camera (3) and a network module (8) for connecting the at least one camera (3) at least to the control unit (5) and the evaluation unit (6), wherein at least one camera (3) is designed as a trigger camera (3a), so that the storage and / or evaluation of the image data captured by the at least one camera (3) can be limited to a relevant time range in the sense of a time window defined as a function of the time of the trigger event, wherein the at least one camera (3) is designed to continuously record video sequences of a predetermined length, wherein the video sequences can be transmitted to the control unit (5) and the evaluation unit (6) by means of the network module (8), characterized in that the evaluation unit (6) is designed to evaluate the video sequences of the trigger camera (3a) for the presence of a trigger event in the sense of a malfunction of the monitored machine (100), wherein the control unit (2) of the device for machine monitoring (1) is designed to retrieve a predefined number of video sequences from the at least one camera (3) via the network module (8) only when a trigger event is present in a video sequence of the trigger camera (3a) and to make the at least one retrieved video sequence available to the evaluation unit (6) for evaluation and / or for storage of the at least one video sequence in the storage unit (7).

2. Device for machine monitoring (1) according to claim 1, characterized in that the control unit (5), the evaluation unit (6) and the storage unit (7) are arranged in a central unit (2) which is connected to the at least one camera (3) by means of the network module (8).

3. Device for machine monitoring (1) according to one of the claims 1 to 2, characterized in that the evaluation unit (6) has a color filter which can be applied to the image data captured with the aid of the at least one camera (3), so that a processed set of image data which has only a defined limited color range can be generated from the captured image data for evaluation.

4. Device for machine monitoring (1) according to one of the claims 1 to 3, characterized in that it has at least two cameras (3), wherein the image data captured by the cameras (3) can be synchronized with each other in time.

5. Device for machine monitoring (1) according to one of the claims 1 to 4, characterized in that it has an internet module with which the device (1) can be connected to the internet in such a way that the captured image data can be uploaded to a web server in the cloud (11) and / or remote access to the image data from a remote access station (12) is enabled.

6. Method for machine monitoring, wherein a machine (100) or facility is monitored with at least one camera (3), in that image data of an area of the machine (100) or facility is recorded with the at least one camera (3), wherein at least one camera is configured as a trigger camera (3a), wherein the trigger camera (3a) continuously records image data, this image data is evaluated for the presence of a defined trigger event in the sense of a malfunction of the monitored machine (100), and the image data of the at least one camera (3) is stored for evaluation in a predetermined manner for a limited time only when a trigger event is detected and / or is made available immediately, characterized in that the at least one camera (3) continuously records video sequences of a predetermined length, in that the video sequences recorded with the trigger camera (3a) are continuously evaluated for the presence of a trigger event, and in that the temporal limitation of the image data recorded with the aid of the at least one camera (3) in the presence of a trigger event in a video sequence recorded with the aid of the trigger camera (3a) is realized by retrieving and storing only a limited predefined number of video sequences recorded with the aid of the at least one camera (3).

7. Method for machine monitoring according to claim 6, characterized in that the image data captured with the aid of at least one camera (3) are processed with the aid of a color filter in such a way that the image data set processed with the aid of the color filter has only a defined limited color range.

8. Method for machine monitoring according to one of the claims 6 and 7, characterized in that image data are continuously captured with the aid of at least two cameras (3), wherein the image data captured with the aid of the cameras (3) are synchronized in time with one another.

9. Method for machine monitoring according to claim 8, wherein the at least two cameras (3) and the central unit (2) or the network module (8) each have a local clock and wherein image data captured with the at least two cameras (3) are provided with a time stamp of the respective local clock, characterized in that the time synchronization of the image data of different cameras (3) is carried out by retrieving the local times of the cameras (3), the difference between the local times of the cameras (3) and the local time of a central unit (2) or a network module (8) is determined and the difference between the local times of the cameras (3) is determined from the differences between the local times of a camera (3) and the central unit (2) or the network module (8), and the image data captured with the aid of the cameras (3) is shifted in time relative to one another in conjunction with the respective time stamp in accordance with the difference between the local times.

10. Method for machine monitoring according to one of the claims 6 to 9, characterized in that a visual and / or an acoustic detection of faults or a quality control in a cyclic sub-process or process monitored by the method is carried out by a master-frame comparison, wherein a sequence of the sensor measurement data corresponding to a cycle of the cyclic sub-process or process is recorded, a master frame is defined from this sequence and the sensor measurement data recorded in each cycle of the sub-process or process is then compared with the master frame at each data acquisition point in time, and a cycle is assumed to be free of errors if a sufficient match with the master frame is established in at least one data acquisition point in time and the cycle is otherwise assumed to be subject to errors.

11. Method for machine monitoring according to claim 10, characterized in that the sufficient correspondence of the sensor measured values with the master frame at a data acquisition time is carried out by determining the similarity value of the sensor measured values with the master frame and comparing the similarity value with a predefined threshold value, wherein a sufficient similarity exists if the similarity value is above the threshold value.

12. Method for machine monitoring according to one of the claims 6 to 11, characterized in that the image data captured and selected during the detection of a trigger event using the at least one camera (3) are uploaded as individual video sequences or rendered into a single video in a data package to a web server in the cloud (11) and / or internet-based remote access to these data is provided.

13. Method for machine monitoring according to one of the claims 6 to 12, characterized in that a device for machine monitoring (1) according to one of the claims 1 to 6 is used.

14. A computer program product for machine monitoring comprising program instructions which, when executed on a computer, cause the computer to execute the machine monitoring method according to any one of the claims 6 to 13.

15. Computer program product for machine monitoring according to claim 14, characterized in that it has at least two software modules, at least one first software module of the computer program product being designed for installation and execution on at least one camera (3) and at least one second software module of the computer program product being designed for installation and execution on a central processing unit (2) and / or a network module (8)