Video system and analysis device

Abstract

To suppress an erroneous detection.SOLUTION: A video system comprises: a camera attached to each vehicle of a train; and an analysis device configured to analyze a video captured by the camera. The analysis device acquires a reference image captured by the camera at a timing before and after the train enters a platform, compares the acquired reference image with a first latest image captured most recently by the camera, updates the reference image with a first latest image when a similarity between the reference image and the first latest image is higher than a first predetermined threshold value as a result of the comparison, compares the updated reference image with a second latest image most recently captured by the camera while the train is stopped, determines that there is an abnormality in the second latest image when the similarity between the reference image and the second latest image is lower than a second predetermined threshold value as a result of the comparison, and outputs an abnormality signal indicating determination of the abnormality.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a video system for monitoring abnormalities, and more particularly to a technique for setting a reference image to be compared for determining abnormalities. [Background technology]

[0002] A known method for detecting foreign objects (door catches) in train doors is to compare a reference image taken under normal conditions with an image taken after the door is closed.

[0003] For example, technologies for detecting foreign objects in passenger doors in railway vehicles are described in Patent Document 1 (JP 2003-252201 A) and Patent Document 2 (JP 2023-42080 A). Patent Document 1 describes a foreign object detection device that has a door monitoring camera installed at the top outside of the meeting part of the passenger doors for boarding and alighting in a railway vehicle, and that uses this door monitoring camera to acquire images before the doors are opened and images after boarding and alighting, and compares and processes the two images to detect foreign objects near the doors and / or display the images. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2003-252201 Summary of the Invention [Problem to be solved by the invention]

[0005] In the technology described in Patent Document 1, an image taken before the doors are released after detecting the edge of the door-opening switch is used as a reference image, but the railway vehicle is stopped when the door-opening switch is turned on, and passengers and their belongings may be captured in the foreign object detection area, making this an inappropriate reference image. For this reason, it is necessary to capture an image in which passengers and their belongings are not captured in the foreign object detection area and set it as the reference image, but the ambient conditions, such as weather and lighting conditions, differ between the reference image and the image taken during operation, which could result in an inappropriate judgment.

[0006] The present invention aims to reduce false detection by setting the most recent image that does not include passengers or objects as the reference image. [Means for solving the problem]

[0007] A representative example of the invention disclosed in the present application is as follows: That is, a video system includes a camera attached to each car of a train and an analysis device that analyzes video captured by the camera, wherein the analysis device acquires a reference image captured by the camera around the time the train enters a platform, compares the acquired reference image with a first most recent image captured most recently by the camera, and if the comparison shows that the similarity between the reference image and the first most recent image is higher than a first predetermined threshold, updates the reference image with the first most recent image, compares the updated reference image with a second most recent image captured most recently by the camera while the train is stopped, and if the comparison shows that the similarity between the reference image and the second most recent image is lower than a second predetermined threshold, determines that an abnormality exists in the second most recent image and outputs an abnormality signal indicating the abnormality determination.

[0008] In addition, in one example of a video system of the present invention, the analysis device is characterized in that it determines the timing when the train enters the platform based on at least one of the following: when the train's position enters a predetermined range indicating a station; when the train's speed falls below a predetermined threshold; and when it detects that the train is approaching a station platform.

[0009] In addition, in one example of the video system of the present invention, the analysis device is characterized in that it compares the reference image with the first latest image and the reference image with the second latest image in at least one of the following areas of the image captured by the camera: an area showing the side of the vehicle; an area a predetermined distance from the edge of the platform; and an area between the vehicle and the platform.

[0010] In addition, in the video system according to one example of the present invention, the analysis device stops updating the reference image when it detects that the boarding and alighting doors of the train are open.

[0011] In addition, in the video system according to one example of the present invention, the analysis device detects the opening of the boarding and alighting doors of the train based on a door-opening signal output from a control device of the train. [Effects of the Invention]

[0012] According to one aspect of the present invention, it is possible to improve the reliability of anomaly detection. Problems, configurations, and effects other than those described above will become apparent from the following description of the preferred embodiment of the present invention. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram illustrating an example of the configuration of a video system according to an embodiment of the present invention. [Figure 2] 1 is a flowchart of a process executed by the analysis device of the present embodiment. [Figure 3] 4 is a timing chart showing the operation of the video system of the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] An embodiment of the present invention will be described with reference to the drawings.

[0015] FIG. 1 is a diagram showing an example of the configuration of a video system according to an embodiment of the present invention.

[0016] The video system shown in Fig. 1 is mounted on a train consisting of four cars, cars 10-1 to 10-4. The number of cars included in one train is not limited to four. When car 10-1 is the front car, the fourth car 10-4 is the rearmost car. Conversely, when car 10-4 is the front car, the first car 10-1 is the rearmost car.

[0017] The video system of this embodiment includes cameras 111-112, a relay device 12, display devices 131-132, a control device 14, and an analysis device 16. The video system of this embodiment may also include a recording device that records the video captured by the cameras 111-112 in association with the identification information of the camera that captured the video and the time of capture. The cameras 111-112, the display devices 131-132, the control device 14, and the analysis device 16 are connected via the relay device 12. Furthermore, these devices are supplied with power from a power supply unit (not shown) of each vehicle.

[0018] Each of cars 10-1 to 10-4 has a camera 111 facing in the direction of train travel when car 10-1 is at the front, and a camera 112 facing rearward in the direction of train travel, on both sides of the car body. Cameras 111 to 112 are usually attached to the top of each of cars 10-1 to 10-4, facing diagonally downward so that they can capture images of passengers getting on and off through the doors from above. When car 10-1 is at the front, camera 111 is installed from the rear of the car to capture images of the front, and camera 112 is installed from the front of the car to capture images of the rear.

[0019] The cameras 111-112 capture moving images within a predetermined field of view (angle of view) at a predetermined frame rate. The cameras 111-112 convert the captured images (live images) into a format such as H.264, MPEG4, or JPEG, and transmit them to the display devices 131-132 and the analysis device 16 via the relay device 12. The cameras 111-112 may also transmit the captured images to the display devices 131-132 in response to a video transmission request transmitted from the display devices 131-132. Whether the cameras 111-112 transmit images automatically or in response to a request from the display devices 131-132 is set at the start of system operation, but the crew may also be able to switch between these modes by operating the control device 14.

[0020] The cameras installed on the same side may be used differently depending on the direction of travel of the train. For example, when traveling in direction A, where car 10-1 is the lead car, camera 112, which takes pictures from the front to the rear and where there is less chance of foreign matter adhering while traveling, is used. Similarly, when traveling in direction B, where car 10-4 is the lead car, camera 111, which takes pictures from the front to the rear and where there is less chance of foreign matter adhering while traveling, is used. Note that if camera 111 fails, camera 112 is used as a backup. If camera 112 fails, camera 111 is used as a backup.

[0021] Each of the cars 10-1 to 10-4 also has a relay device (e.g., a switching hub) 12 for constructing an in-train network. The relay device 12 of each car is connected to the relay device 12 of an adjacent car by a network cable. In this way, a single network is constructed within the train by connecting the relay devices of adjacent cars. These network cables are connected to a jumper box (not shown), for example, via under the car floor, and between cars, it is preferable to connect the jumper boxes of adjacent cars with jumper wires.

[0022] Two display devices 131-132 are installed in the driver's cabs of cars 10-1 and 10-4 at both ends of the train. The display devices 131-132 display images captured by cameras 111-112 in each car while the train is stopped at a station. In the case of a one-man crew, the crew rides in the car at the front of the train (for example, the first car 10-1) and operates the train (driving, opening and closing doors, etc.) by themselves. The crew can also check images of passengers getting on and off the train using the display devices 131-132. It is preferable to display images of passengers getting on and off on the display device 13 in the front car in the direction of train travel (for example, car 10-1), but it is also possible to display images of passengers getting on and off on the display device 13 in the last car 10-4.

[0023] The display devices 131-132 display live images from all cameras 111-112 or recorded images recorded on the recording devices side by side after performing processing such as trimming and inversion in accordance with instructions from the control device 14. The images recorded on the recording devices are mainly used for analyzing accidents and crimes when they occur.

[0024] The two display devices 131-132 are preferably arranged side by side, one above the other or one to the left and one to the right, so that the crew can see them simultaneously. The display devices 131-132 have a display area that is divided into two sections vertically and three sections horizontally, for a total of six sections. Therefore, one display device can simultaneously display camera images of a three-car train (images from a total of six cameras on one side of the train). Furthermore, by using two display devices together, camera images of a maximum six-car train can be simultaneously displayed.

[0025] Furthermore, cars 10-1 and 10-4 at both ends of the train are provided with a control device 14 that controls the display on display device 13. Note that control device 14 may be provided in both cars 10-1 and 10-4 at both ends of the train, or in one of cars 10-1 and 10-4 at both ends of the train.

[0026] The control device 14 is a device that performs image processing, such as switching between images captured by the cameras 111-112 and displaying them on the display devices 131-132. The control device 14 controls the display of the display device 13 based on vehicle information 15 acquired from the vehicle side or a higher-level system. The vehicle information 15 is, for example, speed information indicating the running speed of the train. The control device 14 controls the display devices 131-132 to display images captured by the cameras 111-112 while the train is stopped at a station, and to stop displaying the images when the train starts moving. As a result, the images captured by the cameras 111-112 are not displayed on the display devices 131-132 while the train is moving, allowing the crew to concentrate on operating the train. In this example, it is preferable that the images captured by the cameras 111-112 are displayed on the display devices 131-132 not only when the train is completely stopped, but also when the train is moving at a low speed before and after stopping. Then, when the speed of the train reaches a predetermined value (for example, 5 km / h), the display devices 131-132 stop displaying the images captured by the cameras 111-112.

[0027] The control device 14 or the relay device 12 may have a DHCP function. This DHCP function allows the cameras 111-112 to automatically acquire IP addresses. The IP addresses assigned to the cameras 111-112 may be fixed IP addresses rather than assigned by the DHCP function. If the relay device 12 provides the DHCP function, the range in which the DHCP function is provided may be limited for each vehicle.

[0028] Furthermore, in the video system of this embodiment, vehicles 10-1 to 10-4 are provided with an analysis device 16 that analyzes the images captured by cameras 111 and 112 of the vehicles. The analysis device 16 generates a reference image by executing the process described later in FIG. 2 and compares the feature values ​​of the generated reference image with the feature values ​​of the most recent image captured to detect abnormalities. In this embodiment, the reference image is, for example, an image captured by cameras 111 and 112 of the side of a train with all doors closed. It is recommended that the reference image and the latest image be compared using at least one of the following areas of the image captured by cameras 111 and 112: an area showing the side of the vehicle, an area a predetermined distance from the edge of the platform (e.g., the area between the edge of the platform and the white line or braille blocks), and an area between the vehicle and the platform. For example, an image processed to include the area used for comparison and with other areas masked may be used. The area to be used for comparison may be determined when the video system is installed. In this way, the similarity can be calculated by excluding areas at the back of the platform, away from the edge, where there is a lot of human movement before the train doors open, and abnormalities such as people or objects getting caught in the boarding and alighting doors can be accurately detected.

[0029] For example, the feature values ​​of an image showing a passenger trapped between train doors are significantly different from the feature values ​​of a reference image, and it can be determined that the image and the reference image have a low similarity. On the other hand, the feature values ​​of an image showing all train doors closed are similar to the feature values ​​of the reference image, and it can be determined that the image and the reference image have a high similarity. The similarity calculated in this way is compared with a predetermined threshold. If the similarity is smaller than the predetermined threshold, it can be determined that a foreign object is captured in the image, and therefore an abnormality has occurred. If the analysis device 16 determines that an abnormality has occurred, it outputs abnormality detection information and notifies the crew.

[0030] The control device 14 and the analysis device 16 are composed of computers having a processor, memory, auxiliary storage device, and communication interface. The processor is a computing device that executes programs stored in memory. The processor executes various programs to realize the functions provided by the control device 14 or the analysis device 16. Note that part of the processing performed by the processor when executing the programs may be executed by another computing device (e.g., hardware such as an ASIC or FPGA). The memory includes ROM, which is a non-volatile storage element, and RAM, which is a volatile storage element. ROM stores unchanging programs (e.g., BIOS). RAM is a high-speed, volatile storage element such as DRAM (Dynamic Random Access Memory) that temporarily stores programs executed by the processor and data used during program execution. The auxiliary storage device is a large-capacity, non-volatile storage device such as flash memory or a magnetic storage device that stores programs executed by the processor and data used during program execution. The communication interface is a network interface device that controls communication with other devices according to a predetermined protocol.

[0031] FIG. 2 is a flowchart of the reference image generation process executed by the analysis device 16 of this embodiment.

[0032] The reference image generation process shown in FIG. 2 is executed when the video system detects a predetermined external trigger. The external trigger, for example, detects when a train enters a platform. Examples of such triggers include when the train enters a predetermined area representing a station, when the train's speed drops below a predetermined value (e.g., 5 km / h), or when a platform detection device mounted on the train detects the train's approach to a station platform. The train's position, speed, and platform detection information can be acquired from the vehicle itself or a higher-level system as vehicle information 15. The reference image generation process shown in FIG. 2 may be executed not only when the train enters the platform, but also before or after the train enters the platform. For example, the trigger may be when the train passes a point a predetermined distance before the platform, or when the train has traveled a predetermined distance or time after detecting its entry into the platform.

[0033] First, analysis device 16 acquires images captured by cameras 111-112 and stores them as reference images (S101). Since the imaging environment differs for each station, the reference image stored for the previous station is updated.

[0034] Thereafter, the analysis device 16 sets a timer for a predetermined time and waits until the timer expires (S102). The timer value can be used to control the interval for acquiring the latest image in step S103.

[0035] When the timer expires, the analysis device 16 acquires the latest image captured by the cameras 111-112 (S103), calculates the similarity between the reference image acquired in step S101 and the latest image acquired in step S103, and compares the reference image with the latest image (S104).

[0036] The similarity between the reference image and the latest image can be measured, for example, using a background subtraction method, which detects areas where the difference in brightness between the reference image and the latest image is equal to or greater than a predetermined threshold, as objects. Specifically, the reference image and the latest image are input to a subtractor. The subtractor calculates the difference in brightness value for each pixel at the same position between the input reference image and the latest image, generates a difference image, and inputs the generated difference image to a binarizer. The binarizer determines the difference in brightness value for each pixel in the input difference image using a predetermined threshold, and calculates a binarized image in which pixels whose brightness value difference is less than the predetermined threshold are set to 0 and pixel values ​​whose brightness value difference is equal to or greater than the predetermined threshold are set to 255. The similarity can then be calculated as the ratio of the number of pixels whose pixel value is 0 (i.e., pixels that match highly between the reference image and the latest image) to the total number of pixels. Alternatively, the similarity can be calculated as the ratio of the area represented by pixels whose pixel value is 0 to the total area of ​​the image.

[0037] In addition to the background subtraction method, the distance in color space may be calculated for each pixel, and the calculated distance may be statistically processed (total, average, etc.) to calculate the similarity between the reference image and the latest image.

[0038] If the comparison shows that the similarity between the reference image and the latest image is equal to or greater than a predetermined threshold, the latest image was taken without any objects in the recorded reference image, so the reference image is updated with the latest image (S105) to replace the reference image with the image taken under the latest shooting conditions (S106), and the process proceeds to step S106. On the other hand, if the similarity between the reference image and the latest image is less than a predetermined threshold, it is assumed that the image contains an object that is inappropriate for use as a reference image, so the reference image is not updated and a door close signal is determined (S106). If a door close signal is not detected, the boarding / exiting door opens and passengers begin boarding and disembarking, so the reference image generation process ends. On the other hand, if a door close signal is detected, the boarding / exiting door is closed, so the process returns to step S102, where, after a predetermined timer period has elapsed, the latest image is acquired and the generation of the reference image is repeated.

[0039] FIG. 3 is a timing chart showing the operation of the video system of this embodiment.

[0040] When an external trigger is detected, the analysis device 16 acquires a reference image (S101). Then, at a predetermined time interval (S102) determined by a timer, the analysis device 16 acquires a latest image (S103). Then, an image determination process is performed to compare the acquired reference image with the latest image (S104). If the analysis device 16 determines that the reference image and the latest image are similar in the image determination process, the analysis device 16 updates the reference image (S105). In FIG. 3, the reference image and the latest image are determined to be similar the first two times, and the reference image is updated. On the other hand, from the third time onwards, the reference image and the latest image are determined to be dissimilar, and the reference image is not updated.

[0041] Thereafter, when the train stops at a station, the door open / close switch is operated, the boarding / exiting door opens, and the output of the door close sensor turns ON indicating the door is open, the analysis device 16 detects the door open signal (S106) and ends the reference image generation process.

[0042] As described above, according to the embodiment of the present invention, an image that does not include passengers or objects is set as the reference image, thereby improving the reliability of anomaly detection. Furthermore, an image captured under similar shooting conditions, such as weather and lighting, is recorded as the reference image, which reduces inappropriate judgments (i.e., false detections) caused by differences in shooting conditions between the reference image and images captured during operation, thereby improving the reliability of foreign object detection.

[0043] The present invention is not limited to the above-described embodiments, but includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to configurations including all of the described configurations. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, part of the configuration of each embodiment may be added, deleted, or replaced with other configurations.

[0044] Furthermore, the aforementioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole in hardware, for example by designing them as integrated circuits, or may be realized in software by having a processor interpret and execute a program that realizes each function.

[0045] Information such as programs, tables, and files that realize each function can be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a recording medium such as an IC card, an SD card, or a DVD.

[0046] In addition, the control lines and information lines shown are those that are considered necessary for explanation, and do not necessarily represent all the control lines and information lines that are necessary for implementation. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0047] 10-1, 10-2, 10-3, 10-4 vehicles 12 Relay device 13 Display device 14 Control device 15. Vehicle Information 16 Analyzer 111, 112 Camera 131, 132 Display device

Claims

1. 1. A video system comprising: Cameras installed in each carriage of the train, an analysis device that analyzes the video captured by the camera, The analysis device A reference image taken by the camera is acquired before and after the train enters the platform, comparing the acquired reference image with a first most recent image captured by the camera; If the comparison result indicates that the similarity between the reference image and the first latest image is higher than a first predetermined threshold, updating the reference image with the first latest image; comparing the updated reference image with a second most recent image captured by the camera while the train is stopped; If the comparison result shows that the similarity between the reference image and the second latest image is lower than a second predetermined threshold, it is determined that the second latest image has an abnormality; a video system that outputs an abnormality signal indicating the abnormality determination;

2. 2. The video system according to claim 1, The analysis device When the train position enters a predetermined range indicating a station, the train's speed falls below a predetermined threshold; and A video system characterized in that the timing when the train enters the platform is determined based on at least one of the cases in which the approach of the train to the platform of the station is detected.

3. 2. The video system according to claim 1, The analysis device compares the reference image with the first latest image and the reference image with the second latest image in at least one of the following areas of the image captured by the camera: an area showing the side of the vehicle; an area a predetermined distance from the edge of the platform; and an area between the vehicle and the platform.

4. 2. The video system according to claim 1, The video system is characterized in that the analysis device stops updating the reference image when it detects that the train's boarding and alighting doors are opening.

5. 5. The video system according to claim 4, The video system is characterized in that the analysis device detects the opening of the boarding and alighting doors of the train based on a door open signal output from the train control device.

6. An analysis device that analyzes images captured by cameras attached to each car of a train, A reference image taken by the camera is acquired before and after the train enters the platform, comparing the acquired reference image with the most recent image captured by the camera; If the comparison result indicates that the similarity between the reference image and the latest image is higher than a first predetermined threshold, updating the reference image with the latest image; While the train is stopped, the updated reference image is compared with the most recent image captured by the camera; If the comparison result shows that the similarity between the reference image and the latest image is lower than a second predetermined threshold, it is determined that the latest image has an abnormality; The analysis device outputs an abnormality signal indicating the abnormality determination.

Images