Method and device for the automated monitoring of an operational status of a passenger transport system

Abstract

The invention relates to a method for the automated monitoring of a driving operation (F) of a passenger transport system (1) by means of a monitoring system (5). The monitoring system comprises a hazard analysis module (7) and a movement detection module (9). The movement detection module is directed towards the passenger transport system and configured for detecting electronically processible, real movement sequence recordings (15) of situations. The method comprises: receiving real movement sequence recordings in the hazard analysis module (7); receiving data (X, Y, Z) of a digital doppelganger (29) of the passenger transport system in the hazard analysis module, wherein the data includes at least information relating to physical properties of the passenger transport system, which allows for a conclusion to be drawn in terms of a visual appearance (30) of the passenger transport system in a predetermined state of movement; determining information relating to dynamic objects (31) at the passenger transport system using the hazard analysis module, wherein the information is determined on the basis of the received data of the digital doppelganger and the real movement sequence recordings; determining information relating to a current hazard situation (101A, 101B, 101C) at the passenger transport system using the hazard analysis module, based on an analysis of movements of the determined dynamic objects; and outputting a warning alert (26) using the hazard analysis module, based on the determined information relating to the current hazard situation.

Description

[0001] The invention relates to a method for monitoring the operation of one or more passenger transport systems. The invention further relates to a device for carrying out this method, a correspondingly equipped passenger transport system, a computer program, and a computer-readable medium.

[0002] Passenger transport systems such as escalators and moving walkways are used in buildings like department stores and large shopping centers, as well as in train stations, subway stations, and airports. Particularly in the latter three areas, there can be an increased risk of accidents, for example, when users in a hurry crowd other users on the transport systems. This can cause people to fall on the conveyor belt or step belt of the escalator or on the pallet belt of the moving walkway and suffer serious injuries if the conveyor belt or pallet belt is not stopped, for example, by an emergency stop. Others, in turn, recklessly misuse passenger transport systems as sports and play equipment, thereby endangering themselves and other users.

[0003] Conventional video cameras are used to monitor passenger transport systems. Their video sequences are transmitted in real time to a monitoring room and displayed on screens there. From this monitoring room, several passenger transport systems are usually monitored by staff. However, inattention or fatigue on the part of the monitoring staff can lead to critical situations arising at the passenger transport system not being noticed, or not being noticed in time, and consequently, countermeasures or assistance being initiated too late or not at all.

[0004] To solve the aforementioned problem, monitoring systems are set up so that critical situations involving users on the passenger transport system can be automatically detected by processing motion sequence recordings using image recognition. As soon as a critical situation is detected, the corresponding motion sequence recording is displayed, for example, on the screen in the monitoring room, so that monitoring personnel are alerted and can initiate countermeasures or assistance in a timely manner.

[0005] EP 3 276 535 A1 describes such a monitoring system for a passenger transport system. JP 5 917327 B2 discloses a method and another monitoring system according to the prior art.

[0006] When analyzing motion sequence recordings, such as those provided by cameras used to monitor passenger transport systems, image recognition often employs difference images. A reference image is typically subtracted from one or more currently captured images of a motion sequence. The currently captured images represent the actual passenger transport system, including any passengers and / or objects on it. The reference image, on the other hand, depicts the passenger transport system without passengers and / or objects and was, for example, captured before the system was put into operation. The resulting difference images then show only the passengers and / or objects.Such difference images can be evaluated much more easily and automatically than the original motion sequence recordings, since the original motion sequence recordings depict many details of the passenger transport system, which complicate image analysis without contributing to the detection of critical situations on the passenger transport system.

[0007] However, the practical implementation of generating difference images can be complex. In particular, the fact that parts of the passenger transport system, such as its conveyor belt, move relative to other parts during operation can complicate the generation of difference images. For example, easily evaluable difference images can generally only be generated if the current operating state of the passenger transport system—i.e., the current position of its conveyor belt—corresponds as closely as possible to the operating state the passenger transport system was in when the reference image was captured. For instance, the step edges of a monitored escalator in the currently captured image of a motion sequence recording should correspond precisely to the step edges in a reference image.

[0008] An alternative approach to the automated monitoring of passenger transport systems involves the use of specialized cameras capable of providing three-dimensional images of the monitored area. For example, Time-of-Flight (TOF) cameras can be employed, which, in addition to lateral image resolution, also determine the distance between observed objects or captured pixels in the monitored area relative to the camera. The monitored distance range can be selected to exclude moving components of the passenger transport system, such as its steps or pallets. If necessary, the lateral detection range of the TOF camera can also be limited so that balustrades located to the side of the conveyor belt are no longer visible. In this case, the generation of differential images is unnecessary.However, important areas of the passenger transport system, such as the foot areas on the treads of the steps, are not reproduced in the motion sequence recordings of TOF cameras set up in this way, so that they cannot be taken into account when assessing critical situations.

[0009] To solve the problems described above, there is a need to provide a monitoring procedure and a monitoring system that enable reliable and / or relatively easy-to-implement automated monitoring of the operation of a passenger transport system. Furthermore, there is a need for a suitably equipped passenger transport system, a computer program for implementing the monitoring procedure, and a computer-readable medium for storing the computer program.

[0010] This need is met by the subject matter of the independent claims. Advantageous embodiments are specified in the dependent claims and the following description and are visualized in the figures.

[0011] According to a first aspect of the invention, a method for the automated monitoring of the operation of a passenger transport system is described using a monitoring system. The monitoring system comprises a hazard analysis module and at least one motion detection module. The motion detection module is directed towards the passenger transport system and configured to capture electronically processable motion sequence recordings of situations occurring on the associated passenger transport system. The method comprises at least the following steps, possibly, but not necessarily, in the order given: Receiving real motion sequence recordings in the hazard analysis module, wherein the real motion sequence recordings were acquired by the at least one motion detection module; Receiving data from a digital twin of the passenger transport system in the hazard analysis module, wherein the data includes at least information concerning physical properties of the passenger transport system that allows conclusions to be drawn about a visual appearance of the passenger transport system in a predetermined state of motion; Determining information concerning dynamic objects on the passenger transport system by the hazard analysis module, wherein the information is determined based on the received data from the digital twin and the real motion sequence recordings;Determining information regarding a current hazardous situation at the passenger transport system using the hazard analysis module based on an analysis of the movements of the identified dynamic objects; and issuing a warning message using the hazard analysis module based on the determined information regarding the current hazardous situation.

[0012] According to a second aspect of the invention, a device in the form of a monitoring system for monitoring the operation of a passenger transport system is described. The monitoring system includes a hazard analysis module configured to receive data from a motion detection module and a database module. The motion detection module is directed towards the passenger transport system and configured to capture electronically processable motion sequence recordings of situations occurring on the assigned passenger transport system. The database module stores data from a digital twin of the passenger transport system, the data including at least information concerning the physical properties of the passenger transport system that allows conclusions to be drawn about the visual appearance of the passenger transport system in a predetermined state of motion.The hazard analysis module is configured to perform or control a procedure according to an embodiment of the first aspect of the invention.

[0013] According to a third aspect of the invention, a passenger conveying system is described which has a conveyor belt, a drive for driving the conveyor belt, a control for controlling the drive and a monitoring system according to an embodiment of the second aspect of the invention, wherein the motion detection module of the monitoring system is directed at least towards partial areas of the conveyor belt.

[0014] According to a fourth aspect of the invention, a computer program product is described which comprises machine-readable program instructions which, when executed on a programmable device, cause the device to perform or control a method according to an embodiment of the first aspect of the invention.

[0015] According to a fifth aspect of the invention, a computer-readable medium with a computer program product stored thereon is described according to an embodiment of the fourth aspect of the invention.

[0016] In short, and without limiting the scope of the invention, one underlying idea can be seen as monitoring the operation of a passenger transport system by comparing motion sequence recordings supplied by a motion detection module with, for example, a camera, not, as is conventional, with recordings that serve as reference images and which were also taken with a camera, in order to ultimately create difference images from which information about current hazardous situations can then be derived.Instead of such reference images, data from a digital twin of the passenger transport system will be used to gather information about dynamic objects, such as moving passengers, their luggage and belongings, animals, and the like. This will allow for the analysis of these dynamic objects' movements to identify current hazards. The digital twin's data reflects the physical properties of the passenger transport system, such as the spatial dimensions of components and their dynamic movements, providing insights into its appearance. Accordingly, this data can be used to generate virtual motion sequence recordings that depict the area of ​​the passenger transport system monitored by the motion detection module.These virtual motion sequence recordings can then be used to derive information about the dynamic objects moving on the passenger transport system. If necessary, the virtual motion sequence recordings can be used, similar to conventional reference images, to generate difference images by subtracting them from the real motion sequence recordings. Compared to using conventional reference images, however, the virtual motion sequence recordings can be derived from the data of the digital twin with higher precision, fewer image artifacts, and / or lower computing power. Furthermore, the virtual motion sequence recordings can be synchronized with the currently recorded real motion sequence recordings in a relatively simple manner, which significantly simplifies the generation of difference images.

[0017] Possible configurations and advantages of embodiments are described in more detail below.

[0018] A passenger transport system is designed to move passengers and / or goods within a building. This system can be, for example, an escalator or a moving walkway. It has stationary components that are permanently attached to the building, such as a supporting frame, a drive mechanism, balustrades, etc. Furthermore, it has movable components that can be shifted relative to the stationary components. In escalators, for example, several steps coupled together in one direction of travel form a conveyor belt that can be moved around the perimeter by a drive mechanism relative to the stationary components. In moving walkways, several pallets are coupled together to form a pallet conveyor belt that can be moved around the perimeter. Passengers can move along this conveyor belt.Use a pallet conveyor belt to be transported along a travel path.

[0019] In order to monitor the operation of the passenger transport system, it has a monitoring system in which data determined by a motion detection module is evaluated by a hazard analysis module in order to derive information about current hazardous situations and, if necessary, issue warnings.

[0020] The motion detection module is a device configured to capture motion sequences of situations occurring on the passenger transport system. Specifically, the motion detection module is configured to optically monitor situations on the passenger transport system and output the corresponding data as image sequences. These motion sequences can consist of multiple consecutive images, for example, depicting an area of ​​the passenger transport system in two or three dimensions.For this purpose, the motion detection module can, for example, use one or more image capture devices such as cameras, video cameras, thermal imaging cameras, laser scanners, time-of-flight cameras, a set of multiple sensors, and / or the like, whose motion sequence recordings are accordingly captured as image sequences, video sequences, thermal image sequences, etc., in an electronically processable form. When using multiple motion detection modules per passenger transport system, each motion detection module is preferably assigned a specific section or area, so that the entire escalator or moving walkway is not visible in any one of the motion sequence recordings.

[0021] The real-time motion sequence recordings currently captured by the motion detection module are transmitted to and received by the hazard analysis module as a data stream. To process this data, the hazard analysis module is designed as a data processing system and includes, among other things, a processor that processes the data in a predefined manner.

[0022] The processor is preferably programmable using a computer software product. Additionally, the hazard analysis module generally has a data storage unit and data interfaces to enable data exchange with external devices and / or databases.

[0023] The hazard analysis module described herein is specifically configured to receive and process data from a digital twin of the passenger transport system in addition to the real motion sequence recordings.

[0024] The digital twin is sometimes also referred to as a digital representation. A digital twin is generally understood to be a virtual representation of a real-world object, such as the passenger transport system in this case, which replicates the object's physical properties as realistically as possible. The digital twin is usually stored as a data set (sometimes also called a digital twin data set) in a database. Depending on the application and purpose, this data set can contain a wide variety of parameters or properties of the real object. For example, the data set can include information regarding the mechanical properties, geometric properties, optical properties, electrical properties, magnetic properties, material properties, etc., of the real object.The data contained in the digital twin should reproduce the real-world properties of the object as accurately as possible, allowing for highly realistic reproduction of the object's characteristics at a later time without requiring physical access to the original. Furthermore, the data in the digital twin should ideally enable the reproduction or even prediction of the real object's behavior under specific conditions and / or influences. For this purpose, the data from the digital twin can be used, for example, for physical calculations, modeling, and / or simulations.

[0025] For use in the monitoring procedure presented herein, the data from the digital twin should allow for inferences about the visual appearance of the passenger transport system. In particular, such inferences should be possible for a predetermined movement state of the passenger transport system. The visual appearance should be similar to a top-down view of the passenger transport system obtained from, for example, a camera-based motion detection module with its field of view directed at the passenger transport system, thus reproducing an externally recognizable appearance of the passenger transport system. Specifically, the digital twin should be able to represent the external appearance of the step conveyor or pallet conveyor, an entry and / or exit area, a handrail on a balustrade, etc., of the passenger transport system.

[0026] The digital twin is intended to reproduce the appearance of the passenger transport system in at least one predetermined state of motion. A state of motion can be understood as either a stationary or a dynamic state of the passenger transport system. Generally, the state of motion indicates the positions of moving components of the passenger transport system relative to their stationary components and / or to other moving components. Specifically, the state of motion indicates the current position of steps or pallets of the step conveyor or pallet conveyor. Furthermore, the state of motion can specify the speeds at which the moving components are moving relative to each other and / or relative to the stationary components.

[0027] The data in the digital twin should enable the most accurate possible reproduction of the passenger transport system's visual appearance in at least one predetermined state of motion. For this purpose, the digital twin can contain, for example, design data (e.g., CAD data), material data, processing data, assembly data, maintenance data, etc., whereby this data represents information about physical parameters that influence the appearance (outer shell) of the passenger transport system. Optionally, the digital twin can also contain data that allows for the reproduction of the passenger transport system's visual appearance in several different states of motion.Furthermore, the data stored in the digital twin can be used to calculate, simulate, model, extrapolate or otherwise determine statements about the visual appearance of the passenger transport system in other states of movement.

[0028] The hazard analysis module is designed to process the real-time motion sequence recordings received by the motion detection module, using data from the digital twin, to derive information about any current hazards at the passenger transport system. To this end, the hazard analysis module gathers information about dynamic objects at the passenger transport system. Such dynamic objects can be, for example, passengers, objects, animals, etc., that are transported by the passenger transport system and remain passive themselves. Dynamic objects can also be passengers or similar items that move relative to the passenger transport system, for example, if a passenger falls or if a passenger actively moves along the conveyor belt or pallet conveyor of the system.

[0029] The hazard analysis module then analyzes the movements of the detected dynamic objects in order to derive information regarding a current hazardous situation. For example, the hazard analysis module uses analysis algorithms to examine motion sequence recordings for critical situations, taking into account the data from the digital twin. Automated recognition processes and analysis methods known from the technical field of surveillance electronics, such as image analysis methods and corresponding algorithms, motion analysis methods and corresponding algorithms, statistical and heuristic evaluation methods, and the like, can be used to detect atypical or deviating movement patterns of users of the passenger transport system.Depending on the sequence of movements of a deviating movement process, the hazard analysis module can assume a hazardous situation, i.e., for example, an accident situation in which there is an acute danger that a passenger is or will be injured, or a critical situation in which there is at least a significant risk of such injuries.

[0030] If a hazardous situation is detected, a warning can be issued. This warning can be transmitted to another device, such as a remote monitoring center. Issuing the warning can trigger reactions in the receiving device, ultimately initiating measures to counteract the hazardous situation. For example, the warning can cause the operation of the passenger transport system to be stopped or slowed down. Alternatively or additionally, the warning can trigger a warning signal to users of the passenger transport system, alerting them that the system's operating mode is about to change, for example, that the system is being slowed down.Such a warning signal can preferably be issued visually, audibly or in another way in the immediate vicinity of the passenger transport system.

[0031] According to one embodiment, virtual motion sequence recordings are determined by the hazard analysis module based on the data received from the digital twin. Information concerning dynamic objects on the passenger transport system is then determined by the hazard analysis module by comparing the virtual motion sequence recordings with the actual motion sequence recordings.

[0032] In other words, the data from the digital twin can be used to create virtual motion sequence recordings that reproduce the appearance of the passenger transport system in a similar or identical way to the real motion sequence recordings captured by the motion detection module. By then comparing these virtual motion sequence recordings with the current real motion sequence recordings, information about the dynamic objects on the passenger transport system can be obtained, and conclusions can be drawn about potential hazardous situations.

[0033] According to a more detailed embodiment, the comparison of the real motion sequence recordings with the virtual motion sequence recordings is carried out by calculating a difference between the real motion sequence recordings and the virtual motion sequence recordings.

[0034] In such a differentiation process, a virtual motion sequence recording is subtracted from an associated real motion sequence recording, or vice versa.

[0035] Those image components that are identical in both motion sequence recordings are therefore no longer represented in the resulting difference image. This significantly simplifies the evaluation of the motion sequence recordings.

[0036] According to a further specified embodiment, the method also includes synchronizing the real motion sequence recordings with the virtual motion sequence recordings in such a way that the passenger transport system is in the same state of motion in both motion sequence recordings.

[0037] Synchronizing real and virtual motion sequence recordings can ensure that moving components of the passenger transport system are in the same positions and / or move in the same way in both recording types. This synchronization can be performed within predefined tolerances. Therefore, after such synchronization, the real and virtual motion sequence recordings can be compared particularly effectively.

[0038] If the two synchronized motion sequence recordings depict the passenger transport system in the same state of motion, the components of the passenger transport system are generally no longer shown in a difference image. Passengers or objects that are currently on the passenger transport system and thus shown in the real motion sequence recording, but not in the virtual motion sequence recording, are therefore represented as dynamic objects in the difference image and can be analyzed very effectively therein.

[0039] According to a further, more detailed embodiment, synchronization takes place taking into account data received from a control system of the passenger transport system, which contains information about the actual current movement state of the passenger transport system.

[0040] In other words, the monitoring method presented here for synchronizing real and virtual motion sequence recordings can utilize data provided by the control system of the passenger transport system, data that informs about the current movement status of the system. For example, the control system can hold precise information about the current position of the step conveyor or pallet conveyor along its travel path, i.e., where the steps or pallets are currently arranged. Information about the current movement, in particular the current speed and / or direction of movement, of the step conveyor or pallet conveyor can also be stored in the control system. If such information is retrieved by the hazard analysis module, the system can then...The received data can be used to appropriately process the received data of the digital twin in order to synchronize the virtual appearance represented by it, or the virtual motion sequence recordings of the passenger transport system that can be generated from it, with the real motion sequence recordings.

[0041] According to a further specified embodiment, synchronization can additionally or alternatively be carried out taking into account speed information from a frequency converter of the passenger transport system or a signal from a sensor arranged on the passenger transport system, such as an encoder.

[0042] A frequency converter in the passenger transport system can be used to control the drive of the step conveyor or pallet conveyor by appropriately adjusting frequencies within a power supply. The speed information used or determined by the frequency converter can be retrieved or received by the hazard analysis module. This information can then be used to process the data of the digital twin in such a way that it reproduces the virtual appearance of the passenger transport system with its moving components, in particular a moving step conveyor or pallet conveyor, in such a way that their speed corresponds to the speed of the real components specified by the frequency converter.

[0043] Similarly, in a passenger transport system, an encoder can be used to output signals indicating the current speed of moving components. Such an encoder can, for example, have movable parts that are moved by the moving components of the passenger transport system. The movement of these moving parts can then be detected by suitable sensors, allowing for indirect inferences about the movement of the passenger transport system components. In this case, too, the obtained speed information can be used to synchronize the data of the digital twin regarding the speeds of the virtually moving components with the actual moving components of the passenger transport system.

[0044] According to a further specified embodiment, synchronization can additionally or alternatively be carried out taking into account moving position information, whereby the moving position information is determined by observing a marker which is fixed at a point on a moving component of the passenger transport system and is moved along with it.

[0045] In other words, for example, one or more markings can be attached to a conveyor belt or pallet conveyor at one or more predetermined positions. Such a marking can be visually recognizable and thus detected by a camera or similar device. For example, the marking can be a colored marking. Alternatively, the marking can also be formed by intrinsic properties of the moving component itself, such as a clearly visible edge of a step or pallet. As a further alternative, the marking can be detectable in another way, for example, electrically and / or magnetically. Suitable sensors can be designed and / or positioned at appropriate locations on the personnel conveyor system to monitor the marking(s) and provide corresponding moving position information.Based on this dynamic position information, it is possible to deduce the actual movements of the observed component as well as the current position of its marker. The marker is also virtually represented in the digital twin. Accordingly, the data obtained from the digital twin can be appropriately processed to synchronize it with the actual movements and the correct alignment of the marker of the observed component.

[0046] According to one embodiment, to analyze the movements of the identified dynamic objects, a comparison of the movements of the identified dynamic objects with stored movement scenarios, which represent potential hazard situations, is carried out.

[0047] In particular, the hazard analysis module can perform an automated detection and / or assessment process for critical situations, evaluating the movements of the identified dynamic objects extracted using known analysis methods. To carry out the assessment process, the hazard analysis module can store a set of atypical movement scenarios for critical situations. This set can include various movement scenarios that can be compared with the movements of the identified dynamic objects extracted from motion sequence recordings. This set of stored movement scenarios can be generated, for example, through a machine learning process by using a dummy or stunt performer to recreate typical critical situations, such as falls or reckless actions on the personnel transport system, recording the extracted movement sequences, and adding them to the set as stored movement scenarios.It is of course also possible to create such motion scenarios purely virtually with suitable software and sufficient computing power. Naturally, the set can also be supplemented with further stored motion scenarios that led to an accident during the operation of the passenger transport system and that were not detected by the hazard analysis module. If a dynamic object sufficiently matches a stored motion scenario, the hazard analysis module assumes the presence of a hazardous situation and sends out a warning message and / or a warning signal.

[0048] According to one embodiment, the current hazard situation at the passenger transport system is assessed based on a comparison of the identified dynamic objects with various stored movement scenarios. The warning message is then issued depending on the assessment of the hazard situation.

[0049] Preferably, the various atypical movement scenarios of the set have different weightings in the sense of a ranking. Corresponding to these weightings, different actions can be defined to influence the operation of the passenger transport system.

[0050] For example, a control module can only enable the connection between a manually operated emergency stop release device and the control system of the corresponding passenger transport system if a detected fall of a user is assigned a very high severity level and an emergency stop is defined as the action for this high severity level. The emergency stop is initiated immediately when the monitoring personnel activate the emergency stop release device.

[0051] For example, if a user enters the passenger transport system against its direction of travel, this critical situation may have a medium weighting and for this medium weighting it may be provided that the control module only enables the connection between the manually operated speed controller and the control of the corresponding passenger transport system for operation.

[0052] It is also possible that a user can resolve the critical situation themselves, for example, by catching their fall by grabbing the handrail and then standing upright. Based on the detected situation, this motion sequence recording can be displayed immediately, for example, on the screen of an interaction module, so that the monitoring staff is aware of it and can, if necessary, address the causes of the near fall (user distraction, disruptive influences outside the escalator or moving walkway). However, the low weighting means that the control module does not enable any of the aforementioned connections. It should also be noted here that establishing a connection does not automatically mean that it is enabled.

[0053] A passenger conveying system according to an embodiment of the third aspect of the invention comprises a conveyor belt, a drive, a control system, and an embodiment of the monitoring system described herein. The conveyor belt can be configured as a step conveyor or pallet conveyor. The drive is configured to move the conveyor belt, including any passengers or objects standing on it, along a conveying path. The drive is controlled by the control system. Power can be supplied via a frequency converter. An encoder can be provided on the conveyor belt or on another component moving with the conveyor belt to detect movements of the conveyor belt. The monitoring system is designed to detect hazardous situations in the area of ​​the conveyor belt and / or adjacent regions. For this purpose, its motion detection module is directed at least at partial sections of the conveyor belt.Warning messages can be transmitted directly or indirectly to the control system in order to, for example, slow down or stop the movement of the conveyor belt in the event of a dangerous situation.

[0054] Embodiments of the computer program product according to the fourth aspect of the invention can be formulated in any computer language interpretable by a processor. The computer program product can be stored on a computer-readable medium according to an embodiment of the fifth aspect of the invention. Such a computer-readable medium can be portable. In particular, the medium can store data in a non-volatile or volatile manner. For example, the computer-readable medium can be a CD, a DVD, flash memory, a ROM, an EPROM, or the like. Alternatively, the computer program can also be stored on a computer-readable medium in the form of another computer, a server, a data cloud, or the like, from where it can be downloaded via a data network, in particular the internet.

[0055] It should be noted that possible advantages and embodiments of the invention are described herein partly with reference to a monitoring method according to the invention and partly with reference to a monitoring system according to the invention. A person skilled in the art will recognize that the described features can be appropriately transferred, adapted, exchanged, or modified to arrive at further embodiments of the invention.

[0056] The following describes embodiments of the invention with reference to the accompanying drawing, whereby neither the drawing nor the description is to be interpreted as limiting the invention.

[0057] Figure 1 Figure 1 shows a passenger transport system in a three-dimensional view, the operation of which is to be monitored, and a monitoring system for monitoring this passenger transport system according to an embodiment of the present invention.

[0058] The figure is merely schematic and not necessarily to scale. Identical reference symbols denote identical or equivalent features within the figure.

[0059] Figure 1 Figure 1 shows a three-dimensional view of a passenger transport system 1 with several escalators 3', 3", 3', the operation of which F is monitored by means of a monitoring system 5. The monitoring system 5 comprises a hazard analysis module 7 and a motion detection module 9.

[0060] To monitor the multiple escalators 3', 3", 3‴, the motion detection module 9 has several cameras 11', 11", 11‴, 13', 13", 13‴. In the present embodiment, two escalators 3', 3" are shown in detail. The balustrade sketched with a dashed line symbolizes a possible multitude of further escalators 3‴. Each of these escalators 3', 3", 3‴ is assigned two cameras 11', 13', 11", 13", 11‴, 13‴ of the motion detection module 9, which record actual motion sequence recordings 15 of the operation F of an escalator assigned to it.

[0061] The hazard analysis module 7 and the motion detection module 9 are preferably interconnected via a data network 17. Furthermore, the hazard analysis module 7 is also connected to a database module 19, from which data X, Y, Z of a digital twin 29 of the passenger transport system 1 can be provided. The data network 17 can be established in a variety of ways, such as via a local wired and / or wireless data network, via network connections to a data cloud 47, via CAN bus systems, Bluetooth connections, and the like. The individual modules 7, 9, 19 can exchange data with each other, whereby this data exchange between the individual modules can be unidirectional or bidirectional, as well as continuous, sequential, or temporary, depending on requirements.

[0062] By means of the monitoring system 5, embodiments of the inventive method for monitoring the operation F of the passenger transport system 1 can be carried out.

[0063] Each of the cameras 11', 13', 11", 13", 11‴, 13‴ of the motion detection module 9 is directed towards an assigned passenger transport system 3', 3", 3‴ and captures electronically processable real motion sequence recordings 15 of situations 101A, 101B, 101C occurring on the passenger transport system 1. This may also include the immediate vicinity of the passenger transport system 1. This immediate vicinity includes, for example, the areas upstream of the access areas 21, 23 of a building (not shown) in which the passenger transport system 1 is installed. The upstream areas of the building are also referred to as anterooms and may be monitored by proximity sensors (not shown) of the passenger transport system 1. The proximity sensors transmit their detection data to a "start / stop automatic" system of the passenger transport system 1, which is usually located in the respective controllers 25', 25" of the escalator 3', 3" is implemented.The controls 25', 25" control the operation of a respective drive 45 of the corresponding escalator 3', 3".

[0064] The motion detection module 9 can use video cameras, thermal imaging cameras, laser scanners, time-of-flight (TOF) cameras, a combination of several sensors, and the like, whereby their actual motion sequence recordings 15 are accordingly captured as video sequences, image sequences, thermal image sequences, etc., in electronically processable form. When using several cameras 11, 13, or similar devices in the passenger transport system 1, each of the cameras 11, 13 is preferably assigned a specific section or area. These areas preferably overlap so that there are no gaps in surveillance in which critical situations 101A, 101B, 101C involving users 102A, 102B, 102C could occur unobserved. Since the aim is to achieve the most reliable surveillance possible, the motion detection module 9 preferably continuously records the activity on the passenger transport system 1.This also includes the motion detection module 9 transmitting its real motion sequence recordings 15 to the hazard analysis module 7 in real time.

[0065] The transmitted real motion sequence recordings 15 are received in the hazard analysis module 7. In addition, the hazard analysis module 7 receives data X, Y, Z from the digital twin 29 stored in the database module 19. This data X, Y, Z allows conclusions to be drawn about the visual appearance 30 of the passenger transport system 1 in a predetermined movement state. Accordingly, virtual motion sequence recordings 16 can be determined based on a digital twin 29 and made available to the hazard analysis module 7.

[0066] The hazard analysis module 7 can then determine information about dynamic objects 31, such as moving users 102A, 102B, 102C at the passenger transport system 1, and for this purpose use both the received real motion sequence recordings 15 and the received data X, Y, Z of the digital twin 29. In particular, a comparison of the virtual motion sequence recordings 16 with the real motion sequence recordings 15 can advantageously be carried out, for example by calculating the difference between the two types of motion sequence recordings 15, 16.

[0067] Preferably, the real motion sequence recordings 15 can be synchronized with the virtual motion sequence recordings 16 such that the passenger transport system 1 is in the same motion state in both motion sequence recordings 15 and 16. For this purpose, data can be queried from the controller 25 in the passenger transport system 1, with this data providing information about the current motion state of the real passenger transport system 1. Alternatively or additionally, speed information from a frequency converter 33 of the passenger transport system 1 and / or signals from an encoder 35 on the passenger transport system 1 can be queried and used to synchronize the two motion sequence recordings 15 and 16.As a further alternative or addition, special markings 37 can be provided on movable components 39 of the passenger conveying system 1, such as steps 41 of a conveyor belt 43, which move along with these movable components 39. These markings 37 can, for example, be monitored using sensors to obtain positional information about the movable components 39, which in turn can be used to synchronize the virtual and real motion sequence recordings 15, 16 with positional accuracy.

[0068] The information obtained in this way about the dynamic objects 31 is then examined for hazardous situations 101A, 101B, 101C using analysis algorithms. These analysis algorithms are based, for example, on well-known image processing techniques, which are optimized and applied, for instance, in self-learning processes using artificial intelligence in neural networks. A common image processing technique for generating information from an image is, for example, the calculation of a histogram, which provides information about the statistical brightness distribution in the image. Such a histogram can serve, for example, as a configuration for further image processing steps or as information for a human user of software. Other computable information about an image includes, for example, its entropy or average brightness.Based on this information, vector analyses can be performed to determine how individual key points shift relative to each other, and from this, conclusions can be drawn about movement scenarios 103A, 103B, 103C of users 102A, 102B, 102C.

[0069] Of course, instead of the previously described procedure steps, other or further analysis techniques and methods known from the technical field of video surveillance can be used to extract the movement sequences of users 101A, 101B, 101C from the motion sequence recordings 15, 16.

[0070] Once a user's movement sequence 102A, 102B, 102C has been recognized by the hazard analysis module 7 and, for example, extracted as a skeletal movement sequence, the movement sequence can be compared with a stored set of possible hazardous or critical situations. More precisely, the situations stored in the set represent atypical movement scenarios 103A, 103B, 103C in possible hazardous situations.

[0071] As soon as the hazard analysis module 7 detects a hazardous situation 101A, 101B, 101C, it issues a warning 26. The warning 26 can, for example, be transmitted to a monitoring center 28. From there, in response to the warning 26, the operation of the passenger transport system 1 can be slowed down or stopped by appropriately influencing the control system 25 of the affected escalator 3. Furthermore, a suitable visual or audible warning signal can be issued via signaling devices 27 on the passenger transport system 1 to warn users of the passenger transport system 1.

[0072] On the in Figure 1For a better understanding of the present invention, three hazardous situations 101A to 101C are illustrated by way of example in the passenger conveying system 1 and its immediate vicinity. Corresponding to these hazardous situations 101A, 101B, 101C, a set of possible motion scenarios 103A, 103B, 103C is provided in the hazard analysis module 7, with which dynamic objects 31 extracted from the real motion sequence recordings 15 are compared. The possible location where such a hazardous situation 101A, 101B, 101C could actually occur can also play a role. An atypical motion scenario 103A, which, for example, depicts a fall in the inclined central section of an escalator 3", cannot occur in the same way in the vestibule of the passenger conveying system 1 or in one of the access areas 21, 23.The set of atypical motion scenarios 103A, 103B, 103C can be generated, for example, through a machine learning process by simulating typical hazardous situations 101A, 101B, 101C, such as falls or reckless actions on the passenger transport system 1, using a dummy or stunt performer. The dynamic objects 31 captured and extracted in this process, or their motion sequences, can be included in the set as atypical motion scenarios 103A, 103B, 103C. Naturally, the set can also be supplemented with further atypical motion scenarios that led to an accident during the operation of the passenger transport system 1 and that were not detected by the hazard analysis module 7. If a dynamic object 31 matches a typical motion scenario 103A, 103B, 103C sufficiently, the hazard analysis module 7 can assume or determine the existence of a hazardous situation and issue a warning 26.

[0073] The various atypical movement scenarios 103A, 103B, 103C can have different weightings in terms of a hierarchy. According to this hierarchy, instructions (not shown) can be stored in the monitoring center 28 on how the operation F of the affected escalator 3', 3" should be influenced. Depending on the weighting, for example, an emergency stop must be initiated, the travel speed reduced, an audible and / or visual warning issued, etc. The hazardous situation 101A, recorded by the motion detection module 9, is recognized in the hazard analysis module 7 as a "fall," and the corresponding warning 26 is assigned the highest weighting (= emergency stop), since continuing operation F could lead to serious injuries to the fallen user 102A.

[0074] The critical situation 101B, recorded by motion detection module 9, is identified in hazard analysis module 7 as "entering in the wrong direction," and the corresponding warning 26 is assigned the lowest weighting. By entering in the wrong direction, user 102B does not immediately endanger themselves but rather obstructs oncoming users exiting the passenger transport system 1. In this case, for example, a visual and / or audible warning to the affected user 102B is sufficient.

[0075] The critical situation 101C, recorded by the motion detection module 46, is identified in the hazard analysis module 7 as "entering with a shopping cart," and the corresponding warning 26 is assigned a medium weighting. User 102C is only at risk when she reaches the ascending central section of the escalator 1 with her shopping cart. Based on the medium weighting, the instruction can be to reduce the travel speed F and issue a visual and / or audible warning. Reducing the travel speed allows user 102C to exit the escalator 3' more easily in the opposite direction than at normal speed. This prevents her from entering the ascending section of the escalator too quickly with the shopping cart.

[0076] Finally, it should be noted that terms such as "comprising," "encompassing," etc., do not exclude other elements or steps, and terms such as "a" or "an" do not exclude a plurality. Furthermore, it should be noted that features or steps described with reference to one of the above embodiments may also be used in combination with other features or steps from other embodiments described above. Reference numerals in the claims are not to be considered as limitations.

Claims

1. A method for automated monitoring of the travel operation (F) of a passenger transport system (1) by means of a monitoring system (5), wherein the monitoring system (5) has a hazard analysis module (7) and at least one motion-sensing module (9), wherein the motion-sensing module (9) is directed toward the passenger transport system (1) and is configured to capture electronically processable real motion sequence recordings (15) of situations which occur on the associated passenger transport system (1), wherein the method comprises: receiving real motion sequence recordings (15) in the hazard analysis module (7), wherein the real motion sequence recordings (15) have been captured by the motion-sensing module (9); receiving data (X, Y, Z) of a digital double (29) of the passenger transport system (1) in the hazard analysis module (7), wherein the data (X, Y, Z) comprise at least information relating to physical properties of the passenger transport system (1) which allow a conclusion to be drawn about a visual appearance (30) of the passenger transport system (1) in a predetermined motion state; determining information relating to dynamic objects (31) on the passenger transport system (1) by the hazard analysis module (7), wherein the information is determined based on the received data (X, Y, Z) of the digital double (29) and the real motion sequence recordings (15); determining information relating to a current hazardous situation (101A, 101B, 101C) on the passenger transport system (1) by the hazard analysis module (7) based on an analysis of motions of the determined dynamic objects (31); and outputting a warning (26) by the hazard analysis module (7) based on the determined information relating to the current hazardous situation (101A, 101B, 101C).

2. The method according to claim 1, wherein virtual motion sequence recordings (16) are determined by the hazard analysis module (7) on the basis of the received data (X, Y, Z) of the digital double (29), and wherein the information relating to dynamic objects (31) on the passenger transport system (1) is determined by the hazard analysis module (7) by comparing the virtual motion sequence recordings (16) with the real motion sequence recordings (15).

3. The method according to claim 2, wherein the comparison of the real motion sequence recordings (15) with the virtual motion sequence recordings (16) is carried out by forming the difference between the real motion sequence recordings (15) and the virtual motion sequence recordings (16).

4. The method according to any of claims 2 to 3, wherein the method further comprises synchronizing the real motion sequence recordings (15) with the virtual motion sequence recordings (16) such that movable components (39) of the passenger transport system (1) are in an identical motion state in both motion sequence recordings (15, 16).

5. The method according to claim 4, wherein the synchronization takes place taking into account data which are received from a controller (25) of the passenger transport system (1) and which contain information about an actual current motion state of the passenger transport system (1).

6. The method according to any of claims 4 and 5, wherein the synchronization is carried out taking into account speed information of a frequency converter (33) of the passenger transport system (1) or a signal of an encoder (35) arranged on the passenger transport system (1).

7. The method according to any of claims 4 to 6, wherein the synchronization is carried out taking into account moving position information, wherein the moving position information is determined by observing a marking (37) which is fixed at a location of a movable component (39) of the passenger transport system (1) and is moved along by it.

8. The method according to one of the preceding claims, wherein a comparison of the motions of the determined dynamic objects (31) with stored motion scenarios (103A, 103B, 103C) which represent potential hazardous situations (101A, 101B, 101C) is carried out in order to analyze motions of the determined dynamic objects (31).

9. The method according to one of the preceding claims, wherein the current hazardous situation (101A, 101B, 101C) on the passenger transport system (1) is evaluated based on a comparison of the determined dynamic objects (31) with various stored motion scenarios (103A, 103B, 103C), and wherein the warning is output depending on the evaluation of the hazardous situation (101A, 101B, 101C).

10. A monitoring system (5) for monitoring travel operation (F) of a passenger transport system (1), wherein the monitoring system (5) has a hazard analysis module (7) which is configured to receive data from a motion-sensing module (9) and from a database module (19), wherein the motion-sensing module (9) is directed toward the passenger transport system (1) and is configured to capture electronically processable real motion sequence recordings (15) of situations which occur on the associated passenger transport system (1), and wherein data of a digital double (29) of the passenger transport system (1) are stored in the database module (19), wherein the data comprise at least information relating to physical properties of the passenger transport system (1) which allow a conclusion to be drawn about a visual appearance of the passenger transport system (1) in a predetermined motion state; wherein the hazard analysis module (7) is configured to carry out or control a method according to any of the preceding claims.

11. A passenger transport system (1), having: a transport belt (43); a drive (45) for driving the transport belt (43); a controller (25) for controlling the drive (45); a monitoring system (5) according to claim 10, the motion-sensing module (9) of which is directed at least toward partial regions of the transport belt (43).

12. A computer program product, comprising machine-readable program instructions which, when executed on the hazard analysis module (7) of the monitoring system (5) according to claim 10, cause it to carry out or control a method according to any of claims 1 to 9.

13. A computer-readable medium having a computer program product according to claim 12 stored thereon.

Images