Train monitoring system and door monitoring device

Abstract

The present invention makes it possible to accurately assess whether doors of a train are open even in a situation in which physical sensors for detecting that the doors are open do not work correctly.　A train monitoring system according to one embodiment of the present invention comprises: onboard cameras 12 that capture images, which include door areas of a train, from the interior of the train; a door monitoring device 16 that determines whether doors are open and whether the train is in a traveling state by analyzing the door areas in the images captured by the onboard cameras 12; and notification devices (cabin monitors 13, warning lamps 14, and driver cab monitor 15) that, when it is determined that the doors are open and the train is in a traveling state, notify the passengers or crew of the train of the determination.

Description

Train monitoring system and door monitoring device

[0001] The present invention relates to a train monitoring system that determines whether a train door is in an open state.

[0002] In train operations, it is common for trains to be unable or not to depart due to reasons such as the doors not closing while the train is stopped at a station. However, opening doors while the train is moving is an event that should never occur, and as it could lead to serious accidents as well as the risk of passengers falling, the Ministry of Land, Infrastructure, Transport and Tourism also requires reporting such incidents as serious incidents. However, door opening while the train is moving can occur due to equipment failure, and is frequently reported as a serious incident.

[0003] Prior art in the technical field of the present invention includes the following: For example, Patent Document 1 discloses an invention in which the allocation of camera images to each area obtained by dividing the display region of a monitor is controlled based on information indicating the direction of travel of the train and information indicating the side on which the doors will open.

[0004] Japanese Patent Application Laid-Open No. 2018-113602

[0005] Conventionally, the opening and closing of train doors is detected by a switch-like physical sensor, and a display and notification are sent to the driver's cab or conductor's room via a train control device. However, a malfunction caused by a failure in the switch itself can prevent accurate detection of door opening and closing. Even if the switch itself is normal, a failure in the hinge or arm that activates the switch can prevent the switch from operating correctly. As a result, even if the doors are not completely closed and are in an open state, a display and notification indicating that the doors are closed may be sent to the crew, and the train may continue running with the doors in an open state. For this reason, it is important to detect and notify door opening while taking into account not only failures of the physical sensor itself, but also failures of the equipment and parts that activate the sensor.

[0006] The present invention has been made in consideration of the above-described conventional circumstances, and aims to make it possible to accurately determine whether a train door is open even in a situation where a physical sensor that detects the opening of the door does not function correctly.

[0007] In order to achieve the above object, a train monitoring system according to one aspect of the present invention has the following technical features: That is, the train monitoring system according to one aspect of the present invention includes a camera that captures an image including the door portion of a train from inside the train, and a door monitoring device that determines whether the train door is in an open state by analyzing the door portion in the image captured by the camera.

[0008] Here, in the above train monitoring system, the door monitoring device can operate to determine whether the train door is in an open state based on the position of the door panel in the image or the position of an attachment attached to the door panel.

[0009] In addition, in the above-mentioned train monitoring system, a marker is attached to the inside surface of the door panel near the peripheral edge on the door storage section side that stores the door panel, and the door monitoring device can operate to determine that the train door is in an open state if it cannot detect the marker from the image.

[0010] In addition, the above train monitoring system may further include a mirror positioned within the camera's shooting range, the camera capturing an image including the door portion through the mirror, and the door monitoring device may operate to analyze the mirror portion in the image to determine whether the train door is in an open state.

[0011] In addition, in the above-mentioned train monitoring system, the door monitoring device can operate to not only determine whether the train doors are open or not, but also whether the train is running or not, by analyzing the door portion in the image.

[0012] In addition, in the above-mentioned train monitoring system, the door monitoring device can operate to analyze the door window portion in the image or the gap portion that occurs when the door is in an open state, and determine whether the train is in a running state or not.

[0013] In addition, the above train monitoring system may further include an alarm device for notifying the train crew or passengers when it is determined that the train doors are open and the train is in a running state.

[0014] In the train monitoring system, the notification device may be a display device that displays an image of a door that has been determined to be in an open state.

[0015] A door monitoring device according to another aspect of the present invention has the following technical features: That is, the door monitoring device according to another aspect of the present invention is a device mounted on a train, receives an image captured by a camera that captures an image including the door portion of the train from inside the train, and determines whether the door of the train is in an open state by analyzing the door portion in the image.

[0016] According to the present invention, it becomes possible to accurately determine whether the doors are open even in a situation where a physical sensor for detecting the opening of a train door does not function correctly.

[0017] FIG. 1 is a diagram showing an example of the configuration of a train monitoring system according to one embodiment of the present invention. FIG. 2 is a diagram showing an example of the arrangement of an in-car camera in the train monitoring system of FIG. 1. FIG. 3 is a diagram showing an example of an image captured by an in-car camera under normal circumstances. FIG. 4 is a diagram showing an example of an image captured by an in-car camera under abnormal circumstances. FIG. 5 is a diagram showing an example of a processing flow relating to door open determination using image analysis. FIG. 6 is a diagram showing an example of a display on a driver's cab monitor under abnormal circumstances. FIG. 7 is a diagram showing an example of a display on a passenger compartment monitor near an open door under abnormal circumstances. FIG. 8 is a diagram showing an example of a display on a passenger compartment monitor other than near an open door under abnormal circumstances. FIG. 9 is a diagram showing an example of an image captured under normal circumstances for Modification 1. FIG. 10 is a diagram showing an example of an image captured under normal circumstances for Modification 2. FIG. 11 is a diagram showing an example of an image captured under abnormal circumstances for Modification 2.

[0018] An embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows an example of the configuration of a train monitoring system according to an embodiment of the present invention. FIG. 2 shows an example of the arrangement of in-car cameras in the train monitoring system of FIG. 1. The train in FIG. 1 is a train consisting of N cars 10-1 to 10-N (N is an integer of 2 or greater) connected together. The following description will be given using an example in which car 10-1 is the lead car (i.e., the train is traveling to the left in the drawing). Furthermore, except when a specific car is being described, cars 10-1 to 10-N may be simply referred to as "car 10."

[0019] Each car 10 of the train has an in-car camera 12(A), a passenger compartment monitor 13(A), and a warning light 14(A) near the door 11(A) on the right side (top of the drawing) of the train's direction of travel, and an in-car camera 12(B), a passenger compartment monitor 13(B), and a warning light 14(B) near the door 11(B) on the left side (bottom of the drawing) of the train's direction of travel. The symbols "A" and "B" are used to distinguish between the right and left sides of the train's direction of travel and may be omitted unless a distinction between the right and left sides is necessary. Also, in FIGS. 1 and 2 , each car 10 has two doors 11 and two in-car cameras 12, two passenger compartment monitors 13, and two warning lights 14, but this is merely an example, and the number of doors 11, the in-car cameras 12, the passenger compartment monitors 13, and the warning lights 14 of each car 10 is not particularly limited.

[0020] A cab monitor 15 is installed in the driver's cab of the lead car 10-1 and the last car 10-N. When the train is traveling with car 10-1 at the front, the cab monitor 15 of car 10-1 is used, and when the train is traveling in the opposite direction, the cab monitor 15 of car 10-N is used. The cab monitor 15 has a touch panel function and can accept various instructions by touch operation. Therefore, there is no need to provide a dedicated operating device for manually switching the display on the cab monitor 15. A door monitoring device 16 and a train control device 17 are also installed in the lead car 10-1. The door monitoring device 16 and the train control device 17 may be installed in different cars, or multiple devices may be installed to multiplex the devices.

[0021] The train control device 17 is a device for managing and controlling train operations, and has a function of outputting train operation information. The train operation information includes train speed information, door opening / closing information (e.g., information indicating the door opening / closing state), boarding / alighting information at the station where the train is currently stopped and the next station (e.g., information indicating the side (right and / or left) on which the train doors will open), train traveling direction information, etc. The door opening / closing information is acquired by communication with the physical sensors (such as switches for opening and closing doors) as described above.

[0022] Furthermore, a switching hub 18, which is a network relay device, is installed in each vehicle 10. The switching hub 18 is connected to other devices in the vehicle 10, such as an in-car camera 12, a passenger compartment monitor 13, warning lights 14, a driver's cab monitor 15, a door monitoring device 16, and a train control device 17. The switching hub 18 is also connected to the switching hub 18 of an adjacent vehicle 10, thereby constructing an in-train network.

[0023] Conventionally, the open door state has been detected only by the physical sensors as described above, and notifications have been sent to train crew members via the train control device 17. However, the train monitoring system of this example is also able to detect the open door state by analyzing camera images. In this way, the train monitoring system of this example improves detection accuracy by detecting the open door state using multiple methods, and prevents trains from running with the doors open. The mechanism will be described in detail below.

[0024] As shown in FIG. 2 , in the train monitoring system of this example, the interior camera 12(A) of each vehicle 10 is positioned so that its imaging range 20(A) includes the opposite door area (door 11(B) and its surroundings). Similarly, the interior camera 12(B) of each vehicle 10 is positioned so that its imaging range 20(B) includes the opposite door area (door 11(B) and its surroundings). In this example, it is assumed that the interior camera 12(A) is positioned above the door 11(A) and the interior camera 12(B) is positioned above the door 11(B). However, the interior cameras 12 may be positioned in other locations as long as they are able to capture the opposite door area. Furthermore, the imaging range 20 of each interior camera 12 does not need to include the entire opposite door area, but only needs to include the area necessary for determining whether the door is open and whether the train is running (for example, approximately 30 cm above the door).

[0025] Fig. 3 shows an example of an image captured by the in-car camera 12 under normal circumstances. Fig. 4 shows an example of an image captured by the in-car camera 12 under abnormal circumstances. "Abnormal circumstances" refers to a situation in which some kind of abnormality is occurring, such as when a train is running with the doors not completely closed (i.e., in an open door state). "Normal circumstances" refers to a situation in which no abnormality is occurring, such as when a train is running with the doors completely closed (i.e., in a closed door state) or when the train is stopped at a station with the doors open.

[0026] The vehicle door 11 includes a door panel 31 that slides laterally by a drive source (not shown) and a door panel storage compartment (door pocket) (not shown) into which the door panel 31 is stored when the door is open. When the door is closed, the in-vehicle camera 12 captures an image of the entire door panel 31 (see FIG. 3). On the other hand, when the door is open, i.e., when a portion of the door panel 31 is stored in the door panel storage compartment, the in-vehicle camera 12 captures an image of the portion of the door panel 31 that is not stored in the door panel storage compartment and the gap 35 corresponding to the portion stored in the door panel storage compartment (see FIG. 4). Thus, the image of the door portion changes between the open and closed door states.

[0027] Therefore, by analyzing the door portion in the image captured by the in-vehicle camera 12 using the door monitoring device 16, it is possible to determine whether the door 11 is in an open state. In this example, the positions of the door leaf 31 and the door window 32 in a closed state (e.g., the positions of the edges on the side where the gap 35 is created when the door is open) are stored in memory as reference positions, and the positions of the door leaf 31 and the door window 32 are detected from the image captured by the in-vehicle camera 12 and compared with the reference positions. If the positions detected from the image captured by the in-vehicle camera 12 do not match the reference positions (or if the difference between them is equal to or greater than a predetermined threshold), it is determined that the door 11 is in an open state; otherwise, it is determined that the door 11 is in a closed state.

[0028] The door monitoring device 16 can also determine whether the train is moving by analyzing images captured by the in-car camera 12. Whether the train is moving can be determined, for example, by analyzing the exterior scenery captured through the door window 32 or the exterior scenery captured in the gap 35 that appears when the door 11 is open, among the images captured by the in-car camera 12. Specifically, the image of the door window 32 or the gap 35 (i.e., the exterior scenery) is compared with an image of the same area from one frame earlier (or several frames earlier). Then, if horizontal image movement (specifically, image movement in the opposite direction to the train's direction of travel) is detected from the door window 32 or the gap 35, the train can be determined to be moving. Alternatively, if the amount of image change in the door window 32 or the gap 35 is equal to or greater than a reference value, the train can be determined to be moving.

[0029] Even when the train is stopped at a station, the scenery outside the train that can be seen through the door windows 32 and gaps 35 changes due to passengers walking on the station platform, but the scenery does not change as a whole, and only changes in part. In contrast, when the train is running, the scenery outside the train that can be seen through the door windows 32 and gaps 35 changes as a whole. For this reason, it is possible to distinguish whether the train is running or stopped at a station by checking for image movement in the left-right direction or an amount of image change that exceeds a reference value.

[0030] 5 shows an example of a process flow for determining whether a door is open using image analysis. The door monitoring device 16 periodically determines whether the door is open (step S11) and whether the train is running (step S12). If it determines that the door is open and the train is running, the door monitoring device 16 controls the driver's cab monitor 15 to notify the crew that the door is open (step S13) and the passenger compartment monitor 13 and warning light 14 to alert passengers (steps S14 and S15).

[0031] In determining the door open state (step S11), the door monitoring device 16 makes a determination based on the door open / close information included in the operation information output from the train control device 17 and a determination based on the analysis of the image captured by the in-car camera 12, and outputs the logical sum (OR value) of the two determination results. In other words, if the door monitoring device 16 determines that the door is open based on either the door open / close information or the image analysis, it outputs information indicating the door is open (step S11: Yes), and if not, it outputs information indicating the door is not open (step S11: No).

[0032] In determining the train running state (step S12), the door monitoring device 16 makes a determination based on train speed information included in the operation information output from the train control device 17 and a determination based on analysis of images captured by the in-car camera 12, and outputs the logical sum (OR value) of the two determination results. In other words, if the door monitoring device 16 determines that the train is running based on either the train speed information or the image analysis, it outputs information indicating that the train is running (step S12: Yes), and if not, it outputs information indicating that the train is not running (step S12: No).

[0033] In the door open notification to the crew by the cab monitor 15 (step S13), a display indicating that a door is open while the train is running is output to the cab monitor 15. FIG. 6 shows an example of a display by the cab monitor 15 during an abnormality. The cab monitor 15 in normal operation displays a group of images 41 of the door 11 of each car 10 and its surroundings taken by an exterior camera (not shown) in each of multiple divided areas of its display area. On the other hand, during an abnormality, the cab monitor 15 also displays an image 42 of the door 11 determined to be in an open state and its surroundings taken by the corresponding interior camera 12. At this time, the image 42 may be highlighted by surrounding it with a red frame or the like to make it easier for the crew to notice. In FIG. 6, the image 42 is displayed superimposed on a portion of the group of images 41, but the image 42 may also be displayed over the entire display area of ​​the cab monitor 15.

[0034] In the passenger compartment monitor 13 alerting passengers (step S14), a display indicating that a door is open while the train is running is output to the passenger compartment monitor 13. FIG. 7 shows an example of a display in an abnormal state displayed by the passenger compartment monitor 13 near the open door. As shown, a message warning passengers not to approach because the door is open and therefore dangerous is displayed in a visually emphasized manner on the passenger compartment monitor 13 near the open door. FIG. 8 shows an example of a display in an abnormal state displayed by the passenger compartment monitor 13 other than near the open door. As shown, the passenger compartment monitors 13 other than near the open door display a message warning passengers not to approach because the open door is dangerous, and also display an interior map or the like that allows passengers to identify which door on the train is open. Note that the passenger compartment monitors 13 may display various information for passengers during normal operation, such as advertisements and information about the next stop.

[0035] In the step of alerting passengers using the warning lights 14 (step S15), the warning lights 14 near the open doors are lit or flashed in a warning color such as red to notify passengers of the danger. In addition to the door open notification (step S13) and the alerts (steps S14 and S15) as described above, the train crew and passengers may be notified of the door opening while the train is running by other means such as an automatic voice.

[0036] By performing the above-described processing, the crew can quickly recognize the door opening while the train is moving by checking the display on the driver's cab monitor 15, and can stop the train if necessary. Furthermore, the crew can quickly take measures to prevent passengers from falling through the open door, and to inform and restrict passengers from approaching the open door. While it may take some time for the crew to respond on long trains, passengers are also directly alerted by the passenger compartment monitor 13 and warning lights 14, so passengers can quickly become aware of the door opening while the train is moving.

[0037] As described above, the train monitoring system of this example comprises an in-car camera 12 that takes images including the door portion of the train from inside the car, a door monitoring device 16 that analyzes the door portion in the image taken by the in-car camera 12 to determine whether the door is open or not and whether the train is running or not, and an alarm device (in this example, a passenger compartment monitor 13, a warning light 14, and a driver's cab monitor 15) that notifies the train crew or passengers when it is determined that the door is open and the train is running.

[0038] Therefore, the train monitoring system of this example can accurately determine whether the doors are open even in situations where the physical sensor that detects the open doors of the train does not function properly. Furthermore, if the doors are open and the train is running, the train crew and passengers can be promptly notified and alerted. As a result, it is possible to minimize (or shorten) the number of times the train is running with the doors open, shorten the time from the occurrence of a door open incident to the time of response, and prevent serious accidents from occurring.

[0039] In the above description, the open state of the door is determined based on the positions of the door panel 31 and the door window 32 in the image captured by the in-vehicle camera 12. However, the open state of the door can also be determined by image analysis in other ways. Below, several modified examples of the open state of the door using other methods are given.

[0040] <Modification 1> FIG. 9 shows an example of an image captured under normal conditions according to Modification 1. In Modification 1, whether the door 11 is open is determined based on the position of a marker 51 attached to the door panel 31. The marker 51 is an attachment attached to the door panel 31, similar to the door window 32. The marker 51 is attached to the interior surface of the door panel 31 near the periphery of the door panel 31 on the door storage compartment side where the door panel is stored. In FIG. 9, three circular markers 51 are vertically aligned on the upper left side of the door panel 31. However, this is merely an example, and the shape and number of the markers 51 are not particularly limited. The markers 51 attached near the periphery of the door panel 11 on the door storage compartment side are retracted into the door storage compartment and become invisible when the door 11 is even slightly open. Therefore, the door monitoring device 16 according to Modification 1 performs a marker 51 detection process on the image captured by the in-vehicle camera 12. If the marker 51 cannot be detected from the image, the door monitoring device 16 determines that the door 11 is open. This makes it possible to easily and quickly determine whether the door is open or not.

[0041] <Modification 2> FIG. 10 shows an example of an image captured under normal conditions according to Modification 2. FIG. 11 shows an example of an image captured under abnormal conditions according to Modification 2. In Modification 2, whether the door 11 is open is determined based on an image of the door portion reflected in a mirror 52 provided on the ceiling above the door 11. The mirror 52 may be located elsewhere as long as it is included in the capturing range 20 of the in-vehicle camera 12. As an example, the door monitoring device 16 according to Modification 2 identifies the positions of the door panel 31 and the door window 32 by analyzing the portion of the mirror 52 in the image captured by the in-vehicle camera 12, and compares the positions with a reference position to determine whether the door 11 is open. As another example, the door monitoring device 16 according to Modification 2 performs a detection process for a marker 51 attached to the door panel 31 by analyzing the portion of the mirror 52 in the image captured by the in-vehicle camera 12. If the marker 51 cannot be detected, the door monitoring device 16 determines that the door 11 is open. As a result, even when the area around the door is crowded with passengers and the state of the door 11 cannot be photographed directly as shown in FIG. 11, the state of the door 11 reflected in the mirror 52 can be photographed, so that it is possible to determine whether the door is open or not regardless of the state of congestion of passengers.

[0042] Furthermore, the method described above is just one example, and the determination of the door open state and the train running state through image analysis may be achieved using or in combination with other methods, such as image analysis (AI analysis) using a predetermined AI (Artificial Intelligence) learning model.

[0043] Although the embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take on various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the spirit of the present invention. These embodiments and modifications thereof are included in the scope and spirit of the invention described in this specification, etc., and are included in the invention described in the claims and their equivalents.

[0044] Furthermore, the present invention can be provided not only as devices such as those described above or as systems composed of these devices, but also as methods executed by these devices, programs for realizing the functions of these devices using a processor, and storage media for storing such programs in a computer-readable manner.

[0045] The present invention can be used in a train monitoring system that determines whether the doors of a train are open.

[0046] 10-1 to 10-N: Vehicle, 11(A), 11(B): Door, 12(A), 12(B): In-car camera, 13(A), 13(B): Passenger cabin monitor, 14(A), 14(B): Warning light, 15: Driver's cab monitor, 16: Door monitoring device, 17: Train control device, 20(A), 20(B): Shooting range, 31: Door panel, 32: Door window, 35: Gap, 51: Marker, 52: Mirror

Claims

1. A train monitoring system comprising: a camera that takes images including the door area of ​​a train from inside the car; and a door monitoring device that determines whether the train door is open by analyzing the door area in the image taken by the camera.

2. A train monitoring system as described in claim 1, characterized in that the door monitoring device determines whether the train door is in an open state based on the position of the door panel in the image or the position of an attachment attached to the door panel.

3. A train monitoring system as described in claim 1, wherein a marker is attached to the inner surface of the door panel near the periphery of the door storage section that stores the door panel, and the door monitoring device determines that the train door is open when it cannot detect the marker from the image.

4. A train monitoring system as described in claim 1, further comprising a mirror arranged within the camera's shooting range, wherein the camera takes an image including the door portion through the mirror, and the door monitoring device analyzes the mirror portion in the image to determine whether the train door is in an open state.

5. A train monitoring system as described in claim 1, characterized in that the door monitoring device not only determines whether the train doors are open or not, but also whether the train is running or not, by analyzing the door portion in the image.

6. A train monitoring system as described in claim 5, characterized in that the door monitoring device analyzes the door window portion in the image or the gap portion that occurs when the door is in an open state to determine whether the train is in a running state.

7. A train monitoring system according to any one of claims 1 to 5, further comprising an alarm device for notifying the train crew or passengers when it is determined that the train doors are open and the train is in motion.

8. A train monitoring system according to claim 7, wherein the notification device is a display device that displays an image of a door that has been determined to be in an open state.

9. A door monitoring device mounted on a train, characterized in that it receives an image taken by a camera that takes an image including the door portion of the train from inside the train, and determines whether the door of the train is in an open state by analyzing the door portion in the image.

Images