Train monitoring system

Abstract

To provide a train monitoring system for quickly and securely recognizing occurrence of an abnormal situation.SOLUTION: An image analysis unit being a computer analyzes video obtained from each camera and recognizes presence / absence of detection targets in the video. The camera images a region including a door D. The detection targets are, for example, a passenger (person), a stroller, a wheelchair, and a white cane for visually impaired persons, and are objects that make departure of a train T unfavorable when they are recognized in the vicinity of the door D. The image analysis unit can determine the presence / absence of the detection targets in the video by image analysis using an AI technology. At the standstill time of the train T, the image analysis unit can recognize movement of a passenger when, for example, the passenger is recognized.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a train monitoring system for monitoring conditions related to the safety of train operation. [Background technology]

[0002] Camera-based surveillance systems are used to ensure the safety of train operations. These cameras can be installed, for example, on the platform where the train stops, or on the train itself. In either case, the area that the camera must primarily monitor is the area around the doors where passengers are likely to be getting on and off, and this surveillance must be performed without any blind spots.

[0003] Patent Document 1 describes a train monitoring system that uses a camera fixed to the train. The camera captures images of the area around the train doors, and its operation is controlled in conjunction with the door opening and closing. This ensures reliable monitoring in situations where monitoring is important, while minimizing operations when monitoring is not particularly important. This allows for efficient monitoring, and also allows for a small storage capacity when storing video signals obtained by the camera, for example. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-113602 DISCLOSURE OF THE INVENTION [Problem to be solved by the invention]

[0005] When using a monitoring system such as the one described above, the crew member would view the acquired video on the monitor and would recognize any abnormalities that occurred in the video. An example of an abnormality that should be recognized in this way would be a situation where a passenger is getting on or off the train (door) in close proximity to the train when the train is about to depart. Meanwhile, in the case of train operations, one-man operation is becoming more common for the sake of efficiency, and in this case, this task is actually performed by a single crew member (driver).

[0006] For this reason, a system that can automatically recognize such abnormal situations in video is desired, and in this case, image analysis using AI technology, which has been advancing in recent years, is effective. This makes it possible to automatically recognize, for example, passengers getting on or off near the doors of a train about to depart. However, because this recognition must be performed quickly and reliably, it has been difficult to put such automatic recognition technology into practical use, even when using AI technology.

[0007] The present invention has been made in view of the above circumstances, and has as its object to solve the above problems. [Means for solving the problem]

[0008] The train monitoring system of the present invention is a train monitoring system that manages the safety of trains using images captured by cameras that monitor the periphery of doors installed on trains, and recognizes detection targets in the images. do image analysis a display unit for displaying the image; analysis a control unit that, when the detection object is recognized in the video by the detection unit, displays an alarm on the display unit indicating that the detection object has been recognized; The image analysis unit recognizes the detection object for each frame of the video, and the control unit displays the alarm when the detection object is recognized in a predetermined number of consecutive frames, and when the detection object is recognized in each of the consecutive frames, the image analysis unit recognizes that the detection object is moving if the positional relationship between the detection object in the image of each frame and a fixed object other than the detection object recognized in the image changes. . The fixed object may be the door. A plurality of types of the detection object are set, and the image analysis The unit may recognize the type of the recognized detection object, and the control unit may display the alarm in a different manner depending on the type of the recognized detection object. An event may be defined as a predetermined number of consecutive frames in the video, and the image analysis unit may determine whether the object to be detected has been recognized for each different event consisting of different frames. The image processing device may be provided with a memory unit for storing the image, and the number of consecutive frames used by the image analysis unit to determine the presence or absence of the object to be detected may be determined based on the determination results for past images stored in the memory unit. [Effects of the Invention]

[0009] According to the present invention, the occurrence of an abnormal situation can be quickly and reliably recognized in a train monitoring system. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a configuration of a train monitoring system according to an embodiment of the present invention; [Figure 2] 1 is a diagram showing the configuration of a train in which a train monitoring system according to an embodiment of the present invention is used; [Figure 3] 1 is an example of an image obtained in a train monitoring system according to an embodiment of the present invention. [Figure 4] 10 is an example of a warning display in an image obtained in the train monitoring system according to the embodiment of the present invention. [Figure 5] 10 is an example of an operation of detecting a detection target on an event-by-event basis in the train monitoring system according to the embodiment of the present invention. [Figure 6] 1A to 1C are diagrams showing examples of train running conditions and door opening / closing conditions over time, and corresponding changes in video frame rate and image quality in a train monitoring system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] Next, an embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a simplified diagram illustrating the configuration of a train monitoring system 1 according to an embodiment of the present invention. While only two cameras, cameras 10A and 10B, are shown here, FIG. 2 illustrates the arrangement of cameras 10 used in this train monitoring system 1 within a train T. Here, the train T is assumed to be stopped at a platform P. This arrangement is similar to that described in Patent Document 1. A pair of cameras 10A and 10B are provided corresponding to doors D provided on both sides of each car (T1-T3) in the train's traveling direction, so as to capture images of the area including the doors D from both sides (front and rear) along the traveling direction. Therefore, although FIG. 1 illustrates only two cameras, in reality, as shown in FIG. 2, more cameras are provided in pairs for each door D. Also, while FIG. 2 simplifies the illustration by showing each car having only one pair of doors D on the left and right, in reality, multiple pairs of doors D may be provided along the traveling direction for each car. In FIG. 2, since platform P is located on the right side of train T, only door D on the right side is opened and closed, and accordingly, only cameras 10A and 10B on the right side are used.

[0012] 1, this train monitoring system 1 includes a control unit 20 that controls the entire system, a storage unit 30 such as a hard disk or nonvolatile memory that stores the images (image signals) obtained by each camera, an image analysis unit 40 that uses AI technology to analyze the images (pictures) and recognize abnormalities (presence or absence of a detected object), and a display unit 50 that displays the images. Each of these components is connected to a switching hub 60 via a network (LAN), thereby connecting each component to the other.

[0013] As shown in Fig. 2, cameras 10A, 10B (10) are provided in each vehicle as described above, and display units 50 are provided in front of the lead vehicle T1 and behind the lead vehicle T3 when the train is traveling in the opposite direction, so as to be visible to the crew of the train T, and images of the same content are displayed on these units. Although the control unit 20, storage unit 30, and image analysis unit 40 are not shown here, their installation locations on the vehicle T are set appropriately. As described in Patent Document 1, a switching hub 60 is actually installed in each vehicle corresponding to the camera 10 so that the configuration of Fig. 1 is realized, and the switching hubs 60 are connected so that the configuration of Fig. 1 is realized throughout the train.

[0014] The control unit 20 is a computer that controls each camera, display unit, etc. Here, the frame rate (number of frames per unit time) and image quality (number of effective pixels) of the video signal are controlled by each camera. When the frame rate or image quality is high (number of effective pixels is large), video analysis can be performed with high temporal or spatial resolution, allowing the image analysis unit 40 to operate precisely. However, the video signal capacity (file size) increases, and the analysis process takes time. Conversely, when the frame rate is low or image quality is low (number of effective pixels is small), it is difficult to analyze the video with high temporal or spatial resolution, but the video signal capacity (file size) decreases, and the analysis process takes less time.

[0015] The control unit 20 also causes the image analysis unit 40 to analyze the video signal, or causes the memory unit 30 to store the video signal. As will be described later, the camera is controlled in accordance with the analysis results of the image analysis unit 40. The control unit 20 also receives control signals (train status signals) from the train T to recognize the status of the train T. Such train status signals include an open / close signal to recognize the open / close status of each door D, and a speed signal indicating the speed of the train T (or that it is stopped). As will be described later, the control unit 20 controls the camera 10 in accordance with the train status signal.

[0016] The image analysis unit 40, which is a computer, analyzes the images obtained from each camera and recognizes the presence or absence of a detected object in the image. As described above, the camera captures an image of an area including the door D, and the detected object is, for example, a passenger (human) getting on or off, a stroller, a wheelchair, a white cane for the visually impaired, or other objects that, if detected near the door D, would make it undesirable for the train T to depart. The image analysis unit 40 can determine the presence or absence of these detected objects in the image by image analysis using AI technology. Furthermore, when the train T is stationary, for example, if a passenger getting on or off is detected, the movement of the passenger can be recognized. The control unit 20 and the image analysis unit 40 may be configured as a single computer.

[0017] Furthermore, the control unit 20 causes the display unit 50 to display the image that has been analyzed by the image analysis unit 40 as described above, and if a detection target is recognized at this time, an alarm is issued by displaying a message that the driver can immediately recognize. Such a display may include highlighting the outer frame of the image, as will be described later.

[0018] The operation of the image analysis unit 40 will be described below. Fig. 3(a) is an example of an image (one frame of video) acquired by the camera 10A at a certain point in time when the train T is stopped at the platform and the door D is open. In this image, the platform P is recognized on the left side and the open door P is recognized on the right side, but no object to be detected is present. The image analysis unit 40 can store this image in the storage unit 30 in advance as an image in the default state, and can recognize the object to be detected by comparing the acquired image with this image.

[0019] Fig. 3(b) is an image in the same situation as above, in which a human (passenger H) is also present within the field of view. The image analysis unit 40 compares the image in Fig. 3(b) with the default image (Fig. 3(a)) and recognizes the differences as objects, and further recognizes the morphological characteristics of these objects. If this object is recognized as a human, it can recognize that this is passenger H, who has been identified as the object to be detected.

[0020] When a detection target is recognized in the video, the control unit 20 can issue an alarm by highlighting the image on the display unit 50 so as to distinguish it from when the detection target is not recognized. For example, FIG. 4(a) is an example of a display when the detection target is not recognized (the case of FIG. 3(a)), and FIG. 4(b) is an example of a display when the detection target is recognized (the case of FIG. 3(b)). In this case, when the detection target is recognized (FIG. 4(b)), the outer frame F1 (thick dotted line) of the image is displayed differently from the outer frame F0 (thin black solid line) when the detection target is not recognized (FIG. 4(a)). When a detection target is present, the outer frame F1 may be displayed in red, for example. This allows the crew to immediately recognize that a detection target is present in the video.

[0021] In the examples of FIGS. 3 and 4, the detection object is a passenger H getting on or off, but as described above, multiple types of detection objects, such as a stroller, can be set as the detection object. If information for recognizing multiple types of detection objects set in this way is stored in the storage unit 30, the image analysis unit 40 can recognize the presence or absence of each of these detection objects in the image. In this case, the display can be changed depending on the type of detection object, for example, by turning the outer frame F1 red when a passenger H getting on or off is detected, and by turning the outer frame F1 blue when a stroller is detected. This allows the crew to immediately recognize the type of detection object that has been detected.

[0022] However, as described above, if the presence or absence of a detection target is recognized for each frame and a display (highlighting) according to the detection results is performed for each frame, there is a high possibility that erroneous recognition due to noise, etc., or the movement of the detection target will result in an incorrect recognition and an inappropriate alarm. For this reason, it is preferable to perform the above-described detection for each frame, but to determine whether or not to highlight the target according to the detection results for each consecutive frame over a certain period of time. In this case, it is effective to calculate, for example, the percentage of frames in which the detection target is detected among a certain number of consecutive frames, and to issue an alarm when this percentage exceeds a threshold.

[0023] Such a situation is diagrammatically shown in Figure 5. In Figure 5, images of nine consecutive frames (#1 to #9) are shown in order from the left, and it is assumed that the detection object (passenger H) actually exists in #4 to #7, and the detection object does not exist in the images of the other frames.

[0024] Here, in #2, it is mistakenly recognized that there is a detection target even though there actually is not, and in #6, it is mistakenly recognized that there is no detection target even though there actually is. For this reason, if the frame-by-frame judgment described above is used, a false alarm will occur in #2, and an alarm will not be issued in #6 even though it should have been issued.

[0025] Here, the number of consecutive frames used to determine whether to issue the alert can be set to three, and the alert can be issued if the number of frames in which the detection target is detected is two or more. In this case, the control unit 20 performs the alert determination for each of three consecutive frames (events), and if the detection target is detected in two frames in each event, it recognizes that the detection target has been detected in that event. The upper part of Figure 5 shows the results of this determination for each event based on this criterion. Here, the 1 / 3 and 2 / 3 listed here mean that the detection target was detected in one of three frames and two of three frames, respectively. In this case, despite the existence of the erroneously detected frames, a more accurate determination is made, detecting only the central event and not detecting the events on either side, and the display in Figure 4 can be performed based on this result.

[0026] In other words, by using such criteria, the influence of false detections #2 and #6 can be reduced, and an alarm can be more appropriately issued for events in the period corresponding to actual events #4 to #7. In particular, when the frame rate is high, there is a high probability that frames in which a detection target is present and frames in which a detection target is not present exist consecutively in time, so this type of event-by-event judgment is particularly effective.

[0027] After all actual video is recorded in the storage unit 30, the video can be analyzed in more detail to evaluate the appropriateness of the detection. In this case, the detection rate is (number of successfully detected frames: 3 in the case of FIG. 5) / (number of frames to be detected: 4 in the case of FIG. 5), and the false detection rate is (number of false detection frames: 2 in the case of FIG. 5) / (total number of frames: 9 in the case of FIG. 5). The evaluation of the operation of the image analysis unit 40 can be used for learning and reflected in the operation of the image analysis unit 40. In this case, the frames can be replaced with the above-mentioned events, and a similar evaluation can be performed, which can be reflected in the operation of making judgments on an event-by-event basis. In this case, the number of frames per event (number of consecutive frames used for judgment: 3 in the above example) and the judgment threshold (number of detection frames required to issue an alert: 2 in the above example) can also be optimized.

[0028] As described above, in order to quickly and accurately recognize the detection object, it is preferable that the frame rate of the images obtained from cameras 10A and 10B is high. However, when the frame rate is high, the size of the image file stored in storage unit 30 increases, and it may take a long time to store the image. For this reason, it is preferable to increase the frame rate only during periods when the detection object is likely to be present or when it is particularly important to recognize the detection object, and to decrease the frame rate outside of these periods. As with the frame rate, it is also preferable that the image quality be high (with a large number of effective pixels) only during periods when it is particularly important to recognize the detection object, and to decrease the frame rate (with a small number of effective pixels) outside of these periods.

[0029] As described above, this train monitoring system 1 (cameras 10A and 10B) is mounted on a train. In this case, particularly when the train is traveling between stations, the possibility that a detected object will be present in the video is extremely low, or its detection is not important. On the other hand, when the train is stopped on a station platform, or even when the doors are open while the train is stopped, it is particularly important to detect a detected object for safety purposes when the doors are subsequently closed or when the train departs, and the possibility that a detected object will be present is also high. For this reason, it is preferable to increase the frame rate and image quality of the video when the train is stopped on a station platform, or when the doors are open while the train is stopped, and it is preferable to decrease the frame rate and image quality at other times.

[0030] FIG. 6 is a diagram showing the operation of the control unit 20 from before the train enters the station platform until it departs from the platform and travels to the next stop. Here, the horizontal axis represents the passage of time, the top row represents the status of the train's travel, and the second row from the top represents the status of the train's doors. The bottom three items (frame rate, image quality, whether to issue an alarm) are settings for these items determined according to the operation recognized by the control unit 20. Here, whether to issue an alarm refers to a setting of whether to display the display in FIG. 4 on the display unit 50.

[0031] The running status of the train is recognized by a speed signal output by the train to the control unit 20. Here, the speed signal is simplified and output as two values: H when the speed is 5 km / h or more, and L when the speed is less than 5 km / h. As shown in Figure 6, the train enters the platform from outside and decelerates before stopping. For this reason, the speed signal is L only from the time the train stops until it departs, and is H at all other times. In this case, a speed of less than 5 km is recognized as a stop, so in reality, the train is recognized as a stop just before coming to a complete stop (speed = 0 km / h) and for a short time after departure.

[0032] The door state is output from the train to the control unit 20 as an open / close signal expressed as a binary value, with L representing the closed state and H representing the open state. As shown in Figure 6, the doors open after a certain time has passed since the train stopped, and close a certain time before the train departs.

[0033] By receiving such speed signals and open / close signals as the train status signals in Fig. 1, the control unit 20 can recognize the time when the speed status changes and the time when the open / close status of the doors changes. In Fig. 6, the time (A) when the speed signal changes from H to L can be recognized as the time when the train stops, and the time (D) when the speed signal subsequently changes from L to H can be recognized as the time when the train departs. Similarly, the time (B) when the open / close signal changes from L to H can be recognized as the time when the doors change from a closed state to an open state, and the time (C) when the open / close signal subsequently changes from H to L can be recognized as the time when the doors change from an open state to a closed state.

[0034] The period when passengers are likely to be near the doors when the train is stopped can be considered a period when the detected object should be recognized particularly appropriately. In this case, in the example of Figure 6, this period can be from the time when the doors open (B) to the time when the train departs (D). During this period, the frame rate is set to high (e.g., 8 fps) and the image quality is set to high, and outside this period, the frame rate is set to low (e.g., 1 fps) and the image quality is set to low. In addition, to prevent misidentification by the crew, the display in Figure 4 can be displayed only when the frame rate is high and the image quality is high.

[0035] 6, only during periods when there is a particularly high need to recognize the detected object, images with particularly high frame rates and image quality are used for analysis. In this case, whether or not the detected object has been recognized can be determined on a frame-by-frame or event-by-event basis, as described above.

[0036] In the example of FIG. 6, the period when it is particularly necessary to recognize the detection target object is set from the time the doors are opened (B) to the time the train departs (D), but this period can also be set, for example, from the time the train stops (A) to the time the train departs (D), or the period while the doors are open (the period from B to C). Such settings can be made appropriately depending on the platform configuration, the setting of the speed that serves as the threshold for H / L of the speed signal (5 km / h in the above case), etc. Furthermore, this setting does not need to be the same for all stops, and the setting can be changed depending on the stop station, for example, taking into account the number of regular passengers.

[0037] Also, in Figure 6, the frame rate or image quality is maintained at a high value in a constant state during the period from B to D. However, even during this period, it is possible to perform operations to further increase the frame rate or image quality. Such operations are particularly preferable when the recognized detection object is moving.

[0038] In this operation, as in the above, the image analysis unit 40 recognizes the presence or absence of a detection object in the image. If the detection object is also recognized in the image of a subsequent frame, the image analysis unit 40 compares the position of the detection object in the image between the previous frame and the subsequent frame, and if the position of the detection object has changed, it can recognize that the detection object is moving. In this case, for example, when a train is stopped and an image such as that shown in FIG. 3 is obtained, if the positional relationship between cameras 10A and 10B and door D is fixed, the position of door D does not change in the image. Therefore, by recognizing the position of the detection object based on the position of door D, it becomes easy to recognize whether the detection object is moving or whether its moving speed is fast.

[0039] When the object to be detected is moving or moving at a high speed, the frame rate is increased to reduce the time difference between successive frames, and the fluctuation in the position of the object to be detected between frames is kept small, thereby enabling the object to be recognized with particularly high precision and reliability. In this case, the image quality may also be made high in the same way as the frame rate, but in this case the amount of data to be analyzed per unit time increases, so it is also possible to increase only the frame rate without changing the image quality.

[0040] According to the above operation, the presence of a detected object (such as a passenger H) in the surveillance image can be more reliably recognized, and the crew can quickly recognize this fact. In this case, since the data volume of the image to be handled does not increase unnecessarily, processing and storing the image becomes easier.

[0041] The present invention has been described above based on an embodiment. This embodiment is merely an example, and it will be understood by those skilled in the art that various modifications are possible in the combination of the respective components, and that such modifications are also within the scope of the present invention. [Explanation of symbols]

[0042] 1 Train monitoring system 10, 10A, 10B Camera 20 Control Unit 30 Storage section 40 Image analysis unit 50 Display section 60 Switching Hub D-door H Passengers P Platform T train T1~T3 vehicles

Claims

1. A train monitoring system that manages train safety using images captured by cameras that monitor the area around doors installed on trains, an image analysis unit that recognizes a detection target in the video; a display unit that displays the video; a control unit that, when the image analysis unit recognizes the detection object in the video, displays an alarm on the display unit indicating that the detection object has been recognized; Equipped with The image analysis unit recognizes the detection object for each frame of the video, and the control unit displays the alarm when the detection object is recognized in a predetermined number of consecutive frames, A train monitoring system characterized in that, when the detected object is recognized in each of a plurality of consecutive frames, the image analysis unit recognizes that the detected object is moving if the positional relationship between the detected object in the image of each frame and a fixed object other than the detected object recognized in the image changes.

2. A train monitoring system as described in Claim 1, characterized in that the fixed object is the door.

3. A plurality of types of the detection target object are set, the image analysis unit recognizes the type of the recognized detection object, 3. The train monitoring system according to claim 1, wherein the control unit displays the alarm in different modes depending on the type of the recognized detection object.

4. A train monitoring system as described in claim 1 or 2, characterized in that an event is defined as a predetermined number of consecutive frames in the video, and the image analysis unit determines whether the object to be detected has been recognized for each different event composed of different frames.

5. A storage unit for storing the image, A train monitoring system as described in claim 1 or 2, characterized in that the number of consecutive frames used in the image analysis unit to determine the presence or absence of the object to be detected is set based on the judgment results for past images stored in the memory unit.

Images