Surveillance system and method

Abstract

To enable a driver to easily grasp an object when there is an object moving in the direction of a train while the train is running.SOLUTION: A monitoring system includes a camera 11 mounted on each car of a train, and an analysis device 16 that analyzes a camera video, and the analysis device 16 detects an abnormal action object, which is an object moving in the direction of the train in a camera image taken while the train is running, and outputs abnormal detection information indicating the detection of abnormal action object in response to the detection of the abnormal action object.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a surveillance system using cameras mounted on each car of a train. [Background technology]

[0002] In the operation of railway vehicles, one-man operation by a single driver is adopted for lines with few passengers and short train configurations. As the number of railway workers is decreasing due to demographic trends associated with a decline in the working population, each railway operator is adopting a policy of increasing the number of lines and train configurations that apply one-man operation. However, there is a concern that increasing the number of train configurations increases the burden on a single driver to check safety, making it more likely that accidents will occur due to oversight.

[0003] Conventionally, in order to support one-man operation, a system has been used that uses cameras and monitors or mirrors installed on the platform of each station to check the boarding and disembarking status of passengers. There are also on-board camera systems in which cameras and monitors are installed on the train side, and these have become widespread in recent years because they do not require maintenance of infrastructure equipment. For example, Patent Document 1 discloses a monitoring system that allows the state of a door in which an abnormality has been detected in closing to be confirmed on an in-car monitor. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-92136 A Summary of the Invention [Problem to be solved by the invention]

[0005] In recent years, a surveillance system has been developed that uses cameras attached to each train car to capture images of the train's surroundings (mainly near the doors) and displays the camera images on a display device in the driver's cab, allowing the driver to check the images of passengers getting on and off the train. However, since the driver must visually check the camera images, there is a possibility that a person exhibiting abnormal behavior may be overlooked. For example, it is not easy to find a person who moves toward a car while the train is moving (including when the train starts moving) just by looking at the camera images. In particular, the more cars in a train's formation, the greater the driver's monitoring burden becomes, making it difficult to find such a person.

[0006] The present invention has been made in consideration of the above-mentioned conventional circumstances, and has an object to enable the driver to easily grasp an object moving in the direction of the train while the train is moving. [Means for solving the problem]

[0007] In order to achieve the above object, a monitoring system according to one aspect of the present invention is configured as follows. In other words, the surveillance system of the present invention comprises a camera attached to each car of a train and an analysis device that analyzes the camera's image, and the analysis device is configured to detect abnormal behavior, which is an object moving toward the train in the camera's image taken while the train is moving, and, upon detection of an abnormal behavior, output abnormality detection information indicating the detection of an abnormal behavior.

[0008] Here, in the monitoring system according to the present invention, the analysis device can be configured to detect, as an abnormally behaving object, an object that is moving in the direction of the train and whose distance from the train is less than a threshold value.

[0009] In the monitoring system according to the present invention, the analysis device can be configured to use a plurality of threshold values, the higher the threshold value is set as the train speed increases.

[0010] In addition, the surveillance system according to the present invention may further include a display device that displays the image from the camera, and the display device may be configured to display information indicating the abnormal behavioral entity by superimposing it on the image from the camera when the abnormal behavioral entity is detected by the analysis device.

[0011] In addition, in the surveillance system of the present invention, the display device stops displaying the camera image when the train starts moving, but can be configured to cancel the stopping of the display of the camera image if an abnormal behavioral entity is detected by the analysis device.

[0012] In the surveillance system according to the present invention, the cameras may be attached to the sides of each car of the train, facing in the direction of train travel or in the opposite direction.

[0013] In the monitoring system according to the present invention, the cameras may include a camera that is attached to the front side of the leading car in the train's travel direction and captures images of the area ahead in the train's travel direction.

[0014] Moreover, a monitoring method according to another aspect of the present invention is configured as follows. In other words, the monitoring method of the present invention is a monitoring method that uses cameras attached to each car of a train and an analysis device that analyzes the camera footage, and is configured to detect abnormal behavior objects, which are objects moving toward the train in the camera footage taken while the train is moving, and to output abnormality detection information indicating the detection of an abnormal behavior object in response to the detection of an abnormal behavior object. Effect of the Invention

[0015] According to the present invention, when an object moving in the direction of a traveling train is present, the driver can easily recognize the object. [Brief description of the drawings]

[0016] [Figure 1] 1 is a diagram illustrating an example of the configuration of a monitoring system according to an embodiment of the present invention. [Diagram 2]FIG. 13 is a diagram showing an example of a camera image of a person behaving normally; [Diagram 3] FIG. 13 is a diagram showing another example of a camera image capturing a person behaving normally. [Figure 4] FIG. 13 is a diagram showing an example of a camera image capturing a person behaving abnormally; [Diagram 5] FIG. 11 is a diagram showing an example of a flow relating to detection of an abnormal behavior entity. [Figure 6] 13 is a diagram showing an example of a camera image displayed when an abnormally behaving object is detected; FIG. [Figure 7] FIG. 11 is a diagram illustrating an example of the configuration of a monitoring system according to another embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] An embodiment of the present invention will be described with reference to the drawings. Fig. 1 shows an example of the configuration of a monitoring system according to an embodiment of the present invention. Here, a train consisting of four cars will be described as an example, but the number of cars in a train can be any number. In Fig. 1, each of cars 10-1 to 10-4 of the train has a camera 11(A) facing in the direction of train travel and a camera 11(B) facing in the opposite direction, on both sides of the car body, and also has a relay device (e.g., a switching hub) 22 for building an in-train network. Camera 11(A) is installed on the rear side of the car, and camera 11(B) is installed on the front side of the car.

[0018] In addition, two display devices 13(A) and 13(B) are installed in the driver's cabs of cars 10-1 and 10-4 at both ends of the train. Camera images of each car are displayed on the display devices 13 while the train is stopped at the station. Therefore, the train driver can check the images of passengers getting on and off the train on the spot. In this example, it is assumed that only the display device 13 in the front car in the direction of train travel (for example, car 10-1) is operated, but the display device 13 in the last car 10-4 may also be operated.

[0019] In addition, the driver's cabs in the cars 10-1 and 10-4 at both ends of the train are also equipped with a control device 14 that controls the display on / off of the display device 13 based on car information 15 obtained from the car side or a higher-level system. The car information 15 can be, for example, speed information indicating the running speed of the train. The control device 14 controls the display device 13 so that the display is turned on while the train is stopped at the station and the display is turned off when the train starts running. This allows the driver to concentrate on driving the train, since the camera image is not displayed on the display device 13 while the train is running. In this example, the display device 13 continues to display not only when the train is completely stopped, but also when the train is traveling at a speed that is not considered to be running. Then, when the train's speed reaches a predetermined value (for example, 5 km per hour), the display device 13 stops displaying.

[0020] Furthermore, in the monitoring system of this example, each of the vehicles 10-1 to 10-4 is equipped with an analysis device 16 that analyzes the camera images of the vehicle. The analysis device 16 detects an abnormal behavior object, which is an object moving toward the train in the image captured by the camera 11 while the train is running (including when the train starts running), by analyzing the camera images. An example of an abnormal behavior object is typically a person moving toward the train while it is running, but it may also include objects other than a person (for example, a suitcase). When an abnormal behavior object is detected, the analysis device 16 notifies the driver by outputting abnormality detection information indicating the detection of an abnormal behavior object (details will be described later).

[0021] The detection process of an abnormal behavior object by the analysis device 16 will be described. The camera video is composed of multiple still images (hereinafter referred to as "frames") per second, and the analysis device 16 detects an abnormal behavior object by comparing previous and next frames. The analysis device 16 detects an abnormal behavior object by noting that the vector of the movement of the abnormal behavior object is different from that of a normal person on a station platform. Here, the analysis device 16 compares previous and next frames and determines that an object moving laterally (particularly, in the direction toward the train car) is an abnormal behavior object. A detailed description will be given below with reference to Figs. 2 to 4.

[0022] <Pattern 1 (normal action)> Fig. 2 shows an example of a camera image capturing a person behaving normally. Note that each of the images in the first to third frames shown in Fig. 2 was captured at an arbitrary timing after the train started moving, and the same is true for Figs. 3 and 4 described below. In the image of each frame, reference numeral 21 indicates a train, reference numeral 22 indicates a braille block, reference numerals 23 and 24 indicate a person on the platform, and reference numeral 25 indicates an object (a trash can) installed on the platform.

[0023] Fig. 2 is an example of an image in which a train 21 is traveling upward (in the direction of the white arrow) while people 23, 24 and an installation 25 are stationary. In Fig. 2, the positions of people 23, 24 and installation 25 in the image move downward, opposite to the traveling direction of the train 21, as the frames progress. Such object movement is normal and does not pose any particular problem.

[0024] <Pattern 2 (normal action)> FIG. 3 shows another example of camera footage of a person behaving normally. FIG. 3 shows an example of footage in which train 21 travels upward, person 23 moves upward, person 24 moves downward, and installation 25 is stationary. In FIG. 3, the positions of person 24 and installation 25 in the image move downward, opposite to the direction of travel of train 21, as the frames progress. Also, the position of person 23 in the image is approximately the same in each frame (i.e., the traveling speed of train 21 and the walking speed of person 23 are approximately the same). This case is also normal behavior, and does not pose any particular problem.

[0025] <Pattern 3 (Abnormal Behavior)> FIG. 4 shows an example of a camera image capturing a person behaving abnormally. FIG. 4 shows an example of an image in which a train 21 is traveling upward, a person 23 is moving to the left where the train is, a person 24 is moving downward, and an installation 25 is stationary. In FIG. 4, the positions of the person 24 and the installation 25 in the image are moving downward, opposite to the direction of travel of the train 21, as the frames progress. Meanwhile, the position of the person 23 in the image is moving leftward where the train 21 is as the frames progress. In this case, the movement vector of the person 23 is different from the normal behavior of patterns 1 and 2, and the person is detected as an abnormally behaving person.

[0026] The analysis device 16 detects the above-mentioned abnormal behavior object (person 23 of pattern 3 shown in FIG. 4) by comparing previous and subsequent frames of the camera image. That is, the analysis device 16 detects, for example, the feature amount of the object contained in the image of each frame, tracks the object movement between frames based on the identity or similarity of the feature amount, calculates the amount of movement in the lateral direction (to the left where the train is in this example) between frames, and detects an object whose movement amount is equal to or exceeds a predetermined value as an abnormal behavior object. The position of the train in the image may be set as advance information, or may be recognized from the camera image by deep learning or the like, and the method is not important.

[0027] Here, the analysis device 16 may analyze the image from the camera 11(A) facing the direction of train travel, or may analyze the image from the camera 11(B) facing the opposite direction. However, the camera 11(B) facing the opposite direction to the direction of train travel is less likely to get dirty and has good visibility of the image, so it is more preferable as the analysis object than the camera 11(A) facing the direction of train travel. Note that the images from both the cameras 11(A) and 11(B) may be analyzed.

[0028] Furthermore, the analysis device 16 analyzes the camera image from the side where the doors open at the station (i.e., the platform side) among the cameras on both sides of the vehicle. Which side of the vehicle the platform is on can be easily determined, for example, by setting it as advance information for each station. In addition, the determination may be made by a platform detection device, or by deep learning or the like from the camera image, and the method is not important.

[0029] Here, the analysis device 16 of this example detects an abnormal behavior object not only based on the moving direction of the object in the camera image, but also takes into account the train speed and the distance between the object and the train. The train speed is incorporated into the detection conditions for an abnormal behavior object because the train speed affects safety. In other words, even if an object is moving in the direction of the train, there is little danger when the train is stopped or the train speed is slow, whereas the danger increases as the train speed increases. In this example, the train speed is divided into three stages, "stopped", "low speed", and "medium / high speed", and set as conditions.

[0030] The distance between the object and the train is also included in the detection conditions for abnormal behavior because the distance between the object and the train also affects safety. In other words, if the object and the train are more than a certain distance away, there is no problem even if the train is moving, but if the distance is close, the danger increases accordingly. If the train is "stopped," there is no particular problem even if the object is moving toward the train or if the distance between the object and the train is close. If the train speed is "low," there is little problem even if the distance between the object and the train is somewhat close, so a small value is set as the threshold for the distance between the object and the train. On the other hand, if the train speed is "medium / high," the danger increases, so the threshold for the distance between the object and the train is set to a higher value than for "low."

[0031] 5 shows an example of a process flow for detecting an abnormally behaving object by the analysis device 16. Here, as a threshold value for the distance between an object and a train, Acm is applied when the train is moving at a "low speed," and Bcm (where B>A) is applied when the train is moving at a "medium / high speed." The analysis device 16 determines the speed of the train (step S11). If it is determined in step S11 that the train is "stopped," nothing in particular is done. Here, "stopped" may include not only a state in which the train is completely stopped, but also a state in which the train is traveling at a speed that is not recognized as being in motion (for example, less than 5 km / h).

[0032] If it is determined that the speed is "low" in step S11, the analysis device 16 determines the moving direction of the object included in the video (step S12), and further determines the distance between the object and the train (step S13). If it is determined that the moving direction of the object is vertical in step S12, or if it is determined that the distance is A cm or more in step S13, it determines that the object is not an abnormally acting object. On the other hand, if it is determined that the moving direction of the object is horizontal (approaching the train) in step S12, and if it is determined that the distance is less than A cm in step S13, it determines that the object is an abnormally acting object, and outputs abnormality detection information to notify the driver.

[0033] Even if the train speed is determined to be "medium / high speed" in step S11, the analysis device 16 determines the moving direction of the object included in the video (step S14) and further determines the distance between the object and the train (step S15). If the moving direction of the object is determined to be vertical in step S14, or if the distance is determined to be B cm or more in step S15, the object is determined not to be an abnormally acting object. On the other hand, if the moving direction of the object is determined to be horizontal (approaching the train) in step S14 and the distance is determined to be less than B cm in step S15, the object is determined to be an abnormally acting object and abnormality detection information is output to notify the driver.

[0034] Next, a method of notifying the driver of the detection of an abnormal behavior object will be described. The abnormality detection information output by the analysis device 16 when it detects an abnormal behavior object is transmitted to, for example, the display device 13. This abnormality detection information includes identification information (e.g., camera number) of the camera image in which the abnormal behavior object was detected, and the position and size of the abnormal behavior object in the camera image. Based on the abnormality detection information received from the analysis device 16, the display device 13 displays information indicating the abnormal behavior object by superimposing it on the camera image in which the abnormal behavior object was detected.

[0035] FIG. 6 shows an example of the display of a camera image when an abnormal behavior object is detected. In the example of FIG. 6, two persons 23, 24 are shown in a camera image 31 displayed on the display device 13, and the person 23, who is determined to be an abnormal behavior object, is surrounded by a frame 32. The frame 32 is displayed in an accent color (e.g., red) that is easy for the driver to recognize. Therefore, the driver can confirm the abnormal behavior object moving toward the running train just by glancing at the display of the display device 13. In this example, the frame 32 is displayed as information indicating the abnormal behavior object, but the information indicating the abnormal behavior object may be displayed in other ways, such as by displaying a predetermined mark near the person 23, who is an abnormal behavior object.

[0036] The display device 13 may display only the camera image in which the abnormal behavior object is detected, or may also display the camera image in the opposite shooting direction. In addition, in the case where the display device 13 is configured to display the camera images of each vehicle in a divided manner, the camera image in which the abnormal behavior object is detected may be highlighted by surrounding it with a highlighting color such as a red frame or enlarged more than the other camera images, so that it stands out more than the other camera images. In addition, in the case where two display devices 13(A) and 13(B) are used as in this example, for example, the camera image in which the abnormal behavior object is detected may be displayed on the entire surface of the display device 13(A), and the other display device 13(B) may be used to display the camera images of each vehicle in a divided manner.

[0037] Here, as described above, the display device 13 is controlled so that the display of the camera image stops when the train starts to move. Therefore, if this continues, it is not possible to check the state of the abnormal behavior object on the display device 13. Therefore, in this embodiment, the display device 13 cancels the stop of the display of the camera image when it receives abnormality detection information, that is, when the analysis device 16 detects an abnormal behavior object. As a result, the display of the camera image continues when an abnormal behavior object is detected, so that the abnormal behavior object can be checked promptly. The display of the camera image while the train is moving continues until the driver performs a stop operation.

[0038] The abnormality detection information is not limited to the information instructing the output of a video as described above, and may be information in other forms. As one example, the abnormality detection information may be information instructing a speaker (not shown) installed in the driver's cab to output a predetermined voice message. As another example, the abnormality detection information may be information instructing a warning lamp (not shown) installed in the driver's cab to light or blink. Furthermore, information instructing an emergency stop of the train may be used as the abnormality detection information.

[0039] As described above, the monitoring system of this example includes the camera 11 attached to each car of the train and the analysis device 16 that analyzes the camera image. The analysis device 16 is configured to detect an abnormal behavior object, which is an object moving toward the train in the camera image taken while the train is running, and to output abnormality detection information indicating the detection of the abnormal behavior object in response to the detection of the abnormal behavior object. Therefore, when there is an object moving toward the train while the train is running, the driver can easily recognize the object as an abnormal behavior object. This makes it possible to prevent accidents in which an object comes into contact with a running train.

[0040] Furthermore, the analysis device 16 of this example is configured to detect as an abnormally acting object an object whose distance from the train is less than a threshold, rather than detecting all objects moving in the direction of the train as abnormally acting objects. In other words, since objects moving in the direction of the train at a position away from the train are less dangerous, the output of unnecessary abnormality detection information can be suppressed by not detecting them as abnormally acting objects.

[0041] Furthermore, the analysis device 16 of this example is configured to use threshold A as a threshold for determining the distance from the train when the train is moving at "low speed," and to use threshold B, which is larger than threshold A, when the train is moving at "medium / high speed." In this way, by using multiple thresholds, which are set to values ​​that are larger as the train speed increases, abnormal behavior can be detected while taking into consideration differences in safety distances according to train speed. In the above explanation, the speed of a running train is classified into two stages, "low speed" and "medium / high speed," and thresholds are set, but the speed of a train may be classified into three or more stages and the threshold settings may be subdivided.

[0042] The monitoring system of this embodiment further includes a display device 13 that displays camera images, and the display device 13 is configured to display information indicating an abnormal behavior object by superimposing it on the camera image when the abnormal behavior object is detected by the analysis device 16. Therefore, the driver can recognize the presence of an abnormal behavior object simply by looking at the display on the display device 13.

[0043] Furthermore, the display device 13 of this example is configured so that the display of the camera image is stopped when the train starts moving, but the stopping of the display of the camera image is cancelled if an abnormal behavioural object is detected by the analysis device 16. Therefore, the display of the camera image, which would normally be stopped when the train starts moving, is continued when an abnormal behavioural object is detected, making it possible to quickly confirm the abnormal behavioural object.

[0044] Here, in the above explanation, the analysis device 16 mounted on each car of the train analyzes the camera footage of that car, but it is also possible to mount one analysis device 16 on the train and analyze the camera footage of all the cars with one analysis device 16. Also, in the above explanation, the display device 13 has a function of controlling the display content, but the display content may be controlled by a device separate from the display device 13 (for example, the control device 14).

[0045] Fig. 7 shows an example of the configuration of a monitoring system according to another embodiment of the present invention. The monitoring system shown in Fig. 7 has the same basic configuration as the monitoring system shown in Fig. 1. The difference from the monitoring system shown in Fig. 1 is that the monitoring system shown in Fig. 7 is provided with front cameras 18 on the front sides of cars 10-1 and 10-4 at both ends of the train, which capture images of the area ahead (front) in the direction of train travel.

[0046] In this embodiment, the analysis device 16 mounted on the leading vehicle 10-1 in the train travel direction detects an abnormal behavior object by analyzing the camera image captured by the forward camera 18 when the train enters the station platform. The method of detecting an abnormal behavior object by the analysis device 16 is the same as in the above-mentioned embodiment. When an abnormal behavior object is detected, the analysis device 16 outputs abnormality detection information indicating the detection of the abnormal behavior object. The display device 13 is turned off while the train is running, but is controlled to turn on the camera image display when abnormality detection information is received from the analysis device 16. This allows the driver to quickly recognize the abnormal behavior object even when the train enters the platform, making it possible to prevent accidents in which an object comes into contact with the train entering the platform.

[0047] Here, the entry of a train into a platform can be detected by various methods. For example, the entry of a train into a platform can be detected based on information indicating the position of the train, such as kilometre information or GPS (Global Positioning System) position information. Also, for example, the entry of a train into a platform can be detected by analyzing a camera image captured by the front camera 18. Also, for example, the entry of a train into a platform can be detected when the speed of the train drops below a predetermined speed.

[0048] Although the embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the gist of the present invention. These embodiments and modifications are included in the scope and gist of the invention described in this specification, etc., and are included in the scope of the invention described in the claims and their equivalents.

[0049] Furthermore, the present invention can be provided not only as devices such as those described above or as a system composed of these devices, but also as methods executed by these devices, programs for causing a processor to realize the functions of these devices, and storage media for storing such programs in a computer-readable format. [Industrial Applicability]

[0050] The present invention can be used in a surveillance system that uses cameras mounted on each car of a train. [Explanation of symbols]

[0051] 10-1 to 10-4: vehicle, 11(A), 11(B): camera, 12: relay device, 13(A), 13(B): display device, 14: control device, 15: vehicle information, 16: analysis device, 18: front camera

Claims

1. Cameras installed in each carriage of the train, and an analysis device that analyzes the image of the camera, the analysis device detects features of an object included in each frame of the image captured by the camera while the train is traveling, tracks the movement of the object between frames based on identity or similarity of the features, calculates the amount of movement of the object in the train direction between frames, identifies an object whose amount of movement in the train direction is equal to or greater than a predetermined value, detects the object as an abnormally acting object if the distance between the object and the train is less than a threshold, and, in response to the detection of the abnormally acting object, outputs abnormality detection information indicating the detection of the abnormally acting object; The analysis device has a plurality of thresholds set to larger values ​​as the train speed increases, and the abnormal behavior is detected using the threshold corresponding to the train speed, characterized by the monitoring system.

2. 2. The monitoring system according to claim 1, Further, a display device is provided for displaying an image from the camera. A surveillance system characterized in that, when the abnormal behavior entity is detected by the analysis device, the display device displays information indicating the abnormal behavior entity by superimposing it on the image from the camera.

3. 3. The monitoring system according to claim 2, A surveillance system characterized in that the display device stops displaying the image from the camera when the train starts moving, but if the analysis device detects an abnormal behavioral entity, the stopping of the display of the image from the camera is canceled.

4. 4. The monitoring system according to claim 1, A surveillance system characterized in that the cameras are mounted on the sides of each car of the train, facing in the direction of train travel or in the opposite direction.

5. 5. The monitoring system according to claim 4, The surveillance system is characterized in that the cameras include a camera mounted on the front side of the leading car in the direction of train travel, which captures the area ahead in the direction of train travel.

6. A surveillance method using cameras attached to each carriage of a train and an analysis device that analyzes images from the cameras, comprising: the analysis device detects features of an object included in each frame of the image captured by the camera while the train is traveling, tracks the movement of the object between frames based on identity or similarity of the features, calculates the amount of movement of the object in the train direction between frames, identifies an object whose amount of movement in the train direction is equal to or greater than a predetermined value, detects the object as an abnormally acting object if the distance between the object and the train is less than a threshold, and, in response to the detection of the abnormally acting object, outputs abnormality detection information indicating the detection of the abnormally acting object; The analysis device has a plurality of thresholds set to larger values ​​as the train speed increases, and the abnormal behavior object is detected using the threshold corresponding to the train speed, characterized by this monitoring method.

Images