Surveillance device and surveillance system

Abstract

To prevent an erroneous report by ensuring the accuracy of determination as to whether a vehicle exists in a no-entry zone where vehicles run even in a situation that the appearance of a subject in a camera image changes significantly when a person entering the no-entry zone is detected based on a camera image to issue a report.SOLUTION: A monitoring device performs first processing (railway intruder detection processing) for detecting a person who has entered a no-entry zone, based on camera images of the no-entry zone and a boarding and alighting place for a plurality of camera images, performs second processing (on-rail determination processing) for determining the presence or absence of a vehicle in the no-entry zone based on camera images for a plurality of camera images, determines the necessity of notification based on both a detection result of the first processing and a determination result of the second processing, and further makes an integrated determination as to whether the determination result for each camera in the second processing is correct before the processing of determining the necessity of the notification to correct some determination results which are determined as being erroneous out of the camera-basis determination results.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a surveillance device and surveillance system that detects and issues an alert when a person enters a restricted area based on camera images from multiple cameras that capture the restricted area and boarding and disembarking locations where vehicles are traveling. [Background technology]

[0002] At railway stations, passengers may fall from the platform onto the tracks, and it is desirable to detect such accidents early and quickly implement necessary measures such as operating an emergency stop and rescuing passengers.In recent years, therefore, a monitoring system has been proposed that uses camera images of the platform and tracks to detect people who have fallen from the platform onto the tracks and notify station staff of the occurrence of a fall.

[0003] A conventional monitoring system for detecting such persons who have fallen off a platform has been known to detect persons who have fallen off a platform based on camera images of the platform and the tracks along the platform. In order to avoid mistakenly detecting a train on the tracks along the platform as a person who has fallen, a technology is known that stops the process of detecting persons who have fallen off if a train is present on the tracks along the platform (see Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5386744 Summary of the Invention [Problem to be solved by the invention]

[0005] Environmental changes can significantly change how a subject appears in a camera image. For example, fluctuations in sunlight can change how sunlight enters and reflects, or the shadow conditions of surrounding structures can change, causing significant changes in how the subject appears. In addition, dirt, water droplets, and other foreign objects on a camera lens can significantly change how the subject appears.

[0006] On the other hand, when performing on-track determination based on camera images to determine whether or not a train is present on the tracks along the platform, as in the conventional technology, erroneous determinations may be made due to significant changes in the way the subject appears in the camera image caused by environmental changes. Although it is inevitable that such erroneous determinations will occur to some extent, it is desirable to avoid false reports by correcting the erroneous results of on-track determinations before finally determining whether or not a person has fallen and notifying station staff.

[0007] Therefore, the main object of the present invention is to provide a monitoring device and monitoring system that, when detecting a person entering a restricted area in which a vehicle is traveling based on camera images and issuing an alert, can ensure the accuracy of determining whether a vehicle is present in a restricted area and prevent false alerts, even in situations where the way the subject appears in the camera image changes significantly. [Means for solving the problem]

[0008] The monitoring device of the present invention is a monitoring device equipped with a processor that detects a person entering a prohibited area based on camera images from a plurality of cameras that photograph the prohibited area through which vehicles are traveling and the boarding and disembarking locations, and instructs to issue an alert, wherein the processor performs a first process for each of the plurality of camera images to detect a person who has entered the prohibited area from the boarding and disembarking location based on the camera images, and performs a second process for each of the plurality of camera images to determine the presence or absence of a vehicle in the prohibited area based on the camera images, and determines whether or not to issue an alert based on both the detection result of the first process and the determination result of the second process, and further, prior to the process of determining whether or not to issue an alert, the processor makes an integrated determination of the accuracy of the determination results for each of the cameras in the second process, and corrects some of the determination results for each camera image that are determined to be incorrect.

[0009] In addition, the surveillance system of the present invention is configured to include the surveillance device, a plurality of cameras that photograph the no-entry areas and the boarding and disembarking locations, and an alarm device that performs a predetermined alarm operation in response to instructions from the surveillance device. Effect of the Invention

[0010] According to the present invention, the accuracy of the determination results for each camera regarding the presence or absence of a vehicle in an intrusion-prohibited zone is determined in an integrated manner, and a process for correcting an erroneous determination result (processing for correcting a presence-on-rail determination result) is performed. As a result, even if the determination regarding the presence or absence of a vehicle in an intrusion-prohibited zone fails for some camera images due to a situation in which the appearance of the subject in the camera image changes significantly, the determination result can be corrected in consideration of consistency with the determination results of other camera images, thereby ensuring the accuracy of the determination result regarding the presence or absence of a vehicle in an intrusion-prohibited zone and preventing false reports. [Brief description of the drawings]

[0011] [Figure 1] Overall configuration diagram of a monitoring system according to a first embodiment [Diagram 2] FIG. 1 is an explanatory diagram showing an installation state of a camera according to a first embodiment; [Diagram 3] FIG. 1 is an explanatory diagram showing a detection line and an on-rail detection area set on a camera image according to a first embodiment; [Figure 4] FIG. 1 is an explanatory diagram showing an overview of a fall detection process performed by a monitoring server according to a first embodiment; [Diagram 5] FIG. 1 is an explanatory diagram showing an overview of a process for correcting a result of on-rail determination performed by a monitoring server according to a first embodiment; [Figure 6] FIG. 11 is an explanatory diagram illustrating an example of a process for correcting a result of on-rail determination performed by the monitoring server according to the first embodiment; [Figure 7] FIG. 1 is a block diagram showing a schematic configuration of a monitoring server according to a first embodiment. [Figure 8] FIG. 11 is an explanatory diagram showing a notification screen displayed on a fixed notification terminal according to the first embodiment; [Figure 9] FIG. 1 is a flowchart showing a procedure of a process performed by a monitoring server according to a first embodiment; [Figure 10] FIG. 11 is an explanatory diagram showing an overview of a process performed by a monitoring server according to a second embodiment; [Figure 11] FIG. 13 is an explanatory diagram showing an overview of a process performed by a monitoring server according to a third embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The first invention made to solve the above problem is a surveillance device equipped with a processor that detects a person entering a prohibited area based on camera images from a plurality of cameras that capture the prohibited area through which a vehicle is traveling and the boarding and alighting locations, and instructs an alert, wherein the processor performs a first process for each of the plurality of camera images to detect a person who has entered the prohibited area from the boarding and alighting location based on the camera images, and performs a second process for each of the plurality of camera images to determine the presence or absence of a vehicle in the prohibited area based on the camera images, and determines whether or not to issue the alert based on both the detection result of the first process and the determination result of the second process, and further, prior to the process of determining whether or not to issue the alert, makes an integrated determination of the accuracy of the determination results for each camera in the second process, and corrects some of the determination results for each camera image that are determined to be incorrect.

[0013] According to this, the accuracy of the judgment result for each camera regarding the presence or absence of a vehicle in the no-entry zone is judged comprehensively, and a process for correcting an erroneous judgment result (processing for correcting the presence or absence judgment result) is performed. As a result, even if the judgment regarding the presence or absence of a vehicle in the no-entry zone fails in some camera images due to a situation in which the way the subject is captured in the camera images changes significantly, the judgment result is corrected taking into consideration the consistency with the judgment results of the other camera images, thereby ensuring the accuracy of the judgment result regarding the presence or absence of a vehicle in the no-entry zone and preventing false alarms.

[0014] In addition, the second invention is configured such that the processor determines whether or not there is an error in the on-track determination result for each camera image in the second processing based on the shooting range of the camera image, the expected configuration of the vehicle, and the stopping section of the vehicle according to the configuration.

[0015] This makes it possible to accurately detect errors in the train occupancy determination results for each camera and appropriately correct the train occupancy determination results for each camera.

[0016] In addition, the third invention is configured such that the processor estimates the vehicle's configuration based on the appearance of the vehicle in the camera image, sets a detection line on the boundary between the no-entry area and the boarding and alighting location based on the stopping section corresponding to the estimated configuration, and performs the first processing based on the detection line.

[0017] This makes it possible to accurately detect when a user enters a restricted area in a section of the platform where no trains are parked.

[0018] In addition, a fourth invention is configured such that the processor detects the end of the vehicle in the direction of travel, sets a detection line on the boundary between the no-entry area and the boarding and disembarking location based on the position of the end, and performs the first processing based on the detection line.

[0019] This makes it possible to accurately detect when a user enters a restricted area in a section of the platform where no trains are parked.

[0020] In addition, a fifth invention is a surveillance system comprising the surveillance device, a plurality of cameras that photograph the no-entry area and the boarding and disembarking locations, and an alarm device that performs a predetermined alarm operation in response to instructions from the surveillance device.

[0021] According to this, as in the first invention, when a person entering a restricted area in which a vehicle is traveling is detected and an alert is issued based on camera images, even in situations where the way the subject appears in the camera image changes significantly, the accuracy of determining whether a vehicle is present in the restricted area can be ensured and false alerts can be prevented.

[0022] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0023] (First embodiment) FIG. 1 is a diagram showing the overall configuration of a monitoring system according to a first embodiment.

[0024] The monitoring system detects a person (a fallen person) who falls from the platform onto the tracks at a railway station and notifies station staff of this. This monitoring system includes a camera 1, a recorder 2 (a recording device), a monitoring server 3 (a monitoring device), a fixed notification terminal 4 (announcement device), a warning light 5 (announcement device), and a mobile notification terminal 6 (announcement device). The camera 1, the recorder 2, the monitoring server 3, the fixed notification terminal 4, the warning light 5, and the mobile notification terminal 6 are connected to each other via a network.

[0025] Camera 1 is installed in each predetermined monitoring area within the station. Camera 1 captures the platform and the tracks along the platform as its monitoring area. Camera 1 transmits the camera images of the monitoring area to recorder 2 and monitoring server 3 via the network.

[0026] Recorder 2 acquires camera images from camera 1 and stores them.

[0027] The monitoring server 3 is installed in an equipment room, a data center, or the like within the station. The monitoring server 3 acquires camera images from the camera 1 and performs a process (fall detection process) of detecting a person (a fallen person) who falls from the platform onto the tracks. Furthermore, when the monitoring server 3 detects a person who has fallen, it performs a process of notifying station staff of the fact. Specifically, it instructs the fixed notification terminal 4, the warning light 5, and the mobile notification terminal 6 to perform a predetermined notification operation. The functions of the monitoring server 3 may be realized by a cloud computer.

[0028] The fixed notification terminal 4 is installed in an office or the like. The fixed notification terminal 4 can be realized by installing a fall notification application on a PC. In response to an instruction from the monitoring server 3, the fixed notification terminal 4 displays a notification screen as a notification operation to notify station staff that a person has fallen.

[0029] The warning light 5 is installed in an office or the like. In response to an instruction from the monitoring server 3, the warning light 5 turns on a lamp or outputs an alarm sound as a notification operation to notify station staff that a person has fallen.

[0030] The mobile notification terminal 6 is a mobile terminal such as a smartphone or a tablet terminal. The mobile notification terminal 6 is carried by a station staff member who rushes to the scene and takes necessary measures. In response to an instruction from the monitoring server 3, the mobile notification terminal 6 displays a notification screen, outputs an alarm sound, or outputs a vibration as a notification operation to inform the station staff member that a person has fallen.

[0031] In this embodiment, the device detects an accident in which a user falls from a platform at a railway station onto the tracks and notifies staff, but the device is not limited to railway stations. For example, the device may detect an accident in which a user enters (falls from) the deck where users get on and off an attraction ride into the running area of ​​the ride at an amusement park and notify staff.

[0032] In addition, in this embodiment, the system detects a person falling from the platform onto the tracks at a railway station and notifies station staff of the presence of a person who has fallen, but the target of monitoring is not limited to people, and the system may also detect when an object such as luggage has fallen onto the tracks and notify station staff of the presence of a fallen object.

[0033] Next, the fall detection process performed by the monitoring server 3 according to the first embodiment will be described. Fig. 2 is an explanatory diagram showing the installation state of the camera 1. Fig. 3 is an explanatory diagram showing the detection line and the on-rail determination area set on the camera image. Fig. 4 is an explanatory diagram showing the overview of the fall detection process performed by the monitoring server 3.

[0034] As shown in Fig. 2, camera 1 captures images of the platform (boarding and alighting area) and the tracks along the platform (no entry area) as the monitoring area. Multiple cameras 1 are installed, and the platform and the tracks along the platform are captured without exception by the multiple cameras 1. This makes it possible to eliminate blind spots in the fall detection process and prevent missed detections.

[0035] 3, the monitoring server 3 acquires camera images showing the platform and the tracks along the platform. Based on the camera images, the monitoring server 3 detects a person (a fallen person) falling from the platform onto the tracks (fall detection process) and notifies station staff that a fall accident has occurred.

[0036] Specifically, as shown in FIG. 4, the monitoring server 3 performs a track intrusion person detection process (first process) that detects a person who has intruded from the platform onto the tracks along the platform based on the camera image, and also performs a track presence determination process (second process) that determines the presence or absence of a train (vehicle) on the tracks along the platform based on the camera image, and determines whether or not an alert is needed based on both the detection result of the track intrusion person detection process and the determination result of the track presence determination process.

[0037] Here, if there is a train on the tracks next to the platform, that is, if a train is stopped on the tracks next to the platform or if a train is running on the tracks next to the platform, a person cannot enter (fall off) the tracks from the platform. Also, if a train is stopped on the tracks next to the platform and a person moves from the platform side to the tracks, that person is a passenger who has boarded the train and is not a person who has fallen off.

[0038] Therefore, in this embodiment, when it is determined by the on-track determination process for each camera image that a train is present on the tracks along the platform, the camera image is excluded from the targets of notification. In other words, when it is determined that no train is present on the tracks along the platform in a camera image, and a person who has intruded onto the tracks is detected by the on-track person detection process, a notification is instructed.

[0039] In this embodiment, a detection line is set on the boundary between the platform and the tracks in the camera image as shown in Fig. 3. In the process of detecting a person intruding on the tracks, a person who crosses the detection line and intrudes onto the tracks is detected. Specifically, a person on the platform is detected from the camera image (person detection process), and it is determined whether the detected person has crossed the detection line and moved from the platform side to the tracks side (track intrusion determination process).

[0040] In this embodiment, a rectangular on-rail detection area is set in advance on the camera image for the on-rail detection process, as shown in Fig. 3. When a train is present on the track along the platform in the camera image, this on-rail detection area is set to a portion of the track covered by the body of the train. In the on-rail detection process, an image of the on-rail detection area is cut out from the camera image to obtain an area image, and it is determined whether or not a train is present on the track along the platform based on the area image.

[0041] The on-track detection process is performed using an image recognition model (machine learning model). An image of the on-track detection area (area image) cut out from a camera image is input to the image recognition model, and the on-track detection result is output from the image recognition model. The image recognition model for on-track detection is constructed by performing supervised learning in advance using area images when a train is present on the tracks along the platform and area images when no train is present on the tracks along the platform as learning data.

[0042] Incidentally, in an area image extracted from a camera image of an area where a train is present, the track is captured as the subject when there is no train in the area where a train is present, and the body of the train is captured as the subject when there is a train in the area where a train is present. However, the way in which the subject is captured in the area image changes significantly due to environmental changes. For example, the way in which the subject is captured in the area image changes significantly due to changes in the incidence and reflection of sunlight caused by fluctuations in sunlight, and changes in the state of shadows of surrounding structures. In addition, the way in which the subject is captured in the area image changes significantly due to foreign matter such as dirt or water droplets adhering to the lens of camera 1.

[0043] When the way the subject is captured in the area image changes significantly in this way, the presence-on-rail determination using an image recognition model (machine learning model) may fail. That is, a train may be determined to be present when it is not actually present, or a train may be determined to be absent when it is actually present. It is inevitable that such failures in the presence-on-rail determination will occur in some of the cameras 1.

[0044] Therefore, in this embodiment, as shown in Figure 4, in the monitoring server 3, prior to the process of determining whether or not an alert is necessary (alert necessity determination process), a process is performed in which the accuracy of the determination results for each camera 1 in the on-rail presence determination process is determined, and some of the determination results for each camera 1 that are determined to be incorrect are corrected (on-rail presence determination result correction process).

[0045] Next, a description will be given of the process of correcting the presence-on-railroad determination result performed by the monitoring server 3 according to the first embodiment. Fig. 5 is an explanatory diagram showing an overview of the process of correcting the presence-on-railroad determination result. Fig. 6 is an explanatory diagram showing an example of the process of correcting the presence-on-railroad determination result.

[0046] 5, the on-rail presence determination process is performed for each camera image, and a determination result is obtained for each camera image. Furthermore, the process of detecting a person intruding onto the tracks is also performed for each camera image, and a detection result is obtained for each camera image. Furthermore, in this embodiment, the accuracy of the determination result for each camera image in the on-rail presence determination process is determined, and a process (determination result correction process) is performed to correct some of the determination results for each camera image that are determined to be incorrect.

[0047] In the determination result correction process, the accuracy of the determination results obtained from each of the multiple cameras 1 in the on-track determination process is determined in an integrated manner. Specifically, based on the position of the shooting range of each camera image, the expected train configuration, and the train's stop sections according to the configuration, and taking into consideration consistency with the determination results of other camera images, it is determined whether or not there is an error in the on-track determination result for each camera image, and if there is an error in some of the on-track determination results, those on-track determination results are corrected.

[0048] In the example shown in Fig. 6(A), the presence-on-train determination results for each camera image alone are "not on the train" for camera image #1, "on the train" for camera image #2, "not on the train" for camera image #3, "not on the train" for camera image #4, and "on the train" for camera image #5. In this case, since the presence-on-train determination results for camera images #2 and #5 are "on the train," it is presumed that a train is on the train between the shooting range of camera image #2 and the shooting range of camera image #5. Therefore, the presence-on-train determination result of "not on the train" for camera image #3 is incorrect, and the presence-on-train determination result for camera image #3 is corrected to "on the train."

[0049] In the example shown in Figure 6(B), the presence determination result for only the #3 camera image is "on the line," and the presence determination results for all other camera images are "not on the line." In this case, it is presumed that no trains are stopped on the tracks along the platform. Therefore, the presence determination result of "on the line" for the #3 camera image is incorrect, and the presence determination result for the #3 camera image is corrected to "not on the line."

[0050] In the notification necessity determination process, it is determined whether or not to notify station staff based on the result of the person intrusion detection for each camera image acquired in the person intrusion detection process and the presence on track determination result for each camera image acquired in the presence on track determination process. At this time, if the presence on track determination result is corrected for some of the camera images in the presence on track determination result correction process, it is determined whether or not to notify station staff based on the corrected presence on track determination result.

[0051] In this manner, in this embodiment, a process (processing for correcting the presence on line determination result) is performed in which the accuracy of the determination result for each camera 1 is determined in an integrated manner and an erroneous determination result is corrected. As a result, even if the presence on line determination fails in some camera images due to a situation in which the way the subject is captured in the camera images changes significantly, the presence on line determination result can be corrected in consideration of consistency with the presence on line determination results of the other camera images, thereby ensuring the accuracy of the presence on line determination result and preventing false alarms.

[0052] Next, a schematic configuration of the monitoring server 3 according to the first embodiment will be described.

[0053] The monitoring server 3 includes a communication unit 11, a memory 12, and a processor 13.

[0054] The communication unit 11 communicates with the camera 1, the recorder 2, the fixed notification terminal 4, the warning light 5, and the mobile notification terminal 6 via the network.

[0055] The memory 12 stores programs executed by the processor 13 and the like.

[0056] The processor 13 performs various processes related to fall detection by executing the programs stored in the memory 12. In this embodiment, the processor 13 performs an image acquisition process, a process of determining whether a person is on the tracks (second process), a process of correcting the result of the determination of whether a person is on the tracks (first process), a process of determining whether or not a notification is required, a notification process, and the like.

[0057] In the image acquisition process, the processor 13 acquires camera images received from each camera 1 via the communication unit 11. The camera 1 transmits camera images captured in a monitoring area at a predetermined frame rate (for example, 5 fps).

[0058] In the on-track determination process, processor 13 determines whether or not a train is stopped on the track along the platform, based on each camera image. In this on-track determination process, an image recognition model (machine learning model) for on-track determination is used. An image (area image) of the on-track determination area set on the camera image is cut out from the camera image, and the area image is input to the image recognition model for on-track determination, and a on-track determination result of the image recognition result is output from the image recognition model. This on-track determination process is performed individually for each camera image, and a on-track determination result is obtained for each camera image.

[0059] In addition, the image recognition model may output a reliability score indicating the accuracy of the image recognition result as the on-track determination result, and this reliability score may be compared with a predetermined threshold value to obtain an on-track determination result indicating whether or not a train is stopped on the tracks along the platform.

[0060] In the process of correcting the presence on track determination result, processor 13 determines whether or not there is an error in the presence on track determination result for each camera image acquired in the process of the presence on track determination, based on the position of the shooting range of each camera image, the assumed train configuration, and the train stop section corresponding to the configuration, and corrects the presence on track determination result if there is an error. Specifically, when the position of the shooting range of a camera image is taken into consideration, if the presence on track determination result of that camera image does not match the assumed train configuration and stop section, the presence on track determination result of that camera image is determined to be incorrect, and the presence on track determination result of the camera image with the error is corrected.

[0061] In addition, a reliability score may be obtained as an individual on-rail detection result for each camera image, and the on-rail detection result correction process may be performed based on the reliability score for each camera image.

[0062] In the process of detecting a person intruding on the tracks, processor 13 detects a person moving from the platform side to the track side across the detection line based on each camera image. At this time, processor 13 first detects a person on the platform from the camera image (person detection process). Next, processor 13 determines whether or not the detected person has moved from the platform side to the track side across the detection line based on the detection result of the person detection process (track intrusion determination process). This process of detecting a person intruding on the tracks is performed individually for each camera image, and a result of detecting a person intruding on the tracks is obtained for each camera image.

[0063] In the notification necessity determination process, processor 13 determines whether or not it is necessary to notify a staff member of the occurrence of a fall accident, based on the result of the person intrusion into the tracks detection process for each camera image acquired in the person intrusion into the tracks detection process and the presence on track determination result for each camera image acquired in the presence on track determination process. At this time, if the presence on track determination result has been corrected for some of the camera images in the presence on track determination result correction process, the corrected presence on track determination result is used.

[0064] Here, when a person is detected on a camera image that does not show a presence on the tracks, the system judges the person to be a person who has fallen off the tracks and determines that it is necessary to notify station staff. On the other hand, when a person is detected on the tracks but the presence on the tracks determination result for that camera image shows a person is on the tracks, the system judges the person to be a passenger getting on a train and determines that it is not necessary to notify station staff.

[0065] In the notification process, the processor 13 instructs the station staff to perform notification process according to the result of the notification necessity determination process. Specifically, the fixed notification terminal 4, the warning light 5, and the mobile notification terminal 6 are made to perform notification operations to notify the station staff that a person has fallen. Note that, as a mask function, the notification process is stopped during a time period designated by the manager, such as a time period when no trains are running.

[0066] Next, a notification screen displayed on the fixed notification terminal 4 according to the first embodiment will be described. Fig. 8 is an explanatory diagram showing the notification screen.

[0067] In response to an instruction from the monitoring server 3, the fixed notification terminal 4 displays a notification screen as a notification operation for notifying station staff that a person has fallen.

[0068] An alarm mark 31 is displayed on the notification screen, which indicates that a fall detection notification is being issued.

[0069] The notification screen is provided with a first camera image display section 32 and a second camera image display section 33. The first camera image display section 32 displays a camera image at the time when a person intruding on the tracks is detected. The second camera image display section 33 displays a camera image from a predetermined time before the time when a person intruding on the tracks is detected. The camera images displayed on the first camera image display section 32 and the second camera image display section 33 are recorded in the recorder 2, and are acquired from the recorder 2 and displayed on the screen. The first camera image display section 32 displays the camera image as a still image. On the other hand, the second camera image display section 33 displays the camera image as a video by the station staff operating the operation section 34. This allows the station staff to check the situation immediately after the person falls on the first camera image display section 32. The station staff can also check the situation from just before the person falls to during the person falls on the second camera image display section 33.

[0070] The notification screen is also provided with a log display section 35. The log display section 35 displays a list of fall events detected in the past. Specifically, the date and time of the report for each fall event, the camera name, and the like are displayed. When any of the fall accidents displayed on the log display section 35 is selected, the camera images related to the selected fall accident are displayed on the first camera image display section 32 and the second camera image display section 33. This allows the station staff to select a fall event detected in the past and check the situation of the fall event using the camera images. Also, the station staff may input search conditions from a log search input screen (not shown), select a fall event from the logs hit by the search conditions, and check the situation of the fall event using the camera images.

[0071] When the notification screen is displayed on the fixed notification terminal 4 in this manner, at the same time, the warning light 5 and the mobile notification terminal 6 perform a predetermined notification operation in response to an instruction from the monitoring server 3. The warning light 5 notifies station staff that a person has fallen by turning on a lamp and outputting an alarm sound as notification operations. The mobile notification terminal 6 also notifies station staff that a person has fallen by displaying a notification screen, outputting an alarm sound, or vibrating as notification operations. The notification screen of the mobile notification terminal 6 may display a camera image at the time when a person entering the tracks is detected.

[0072] Next, a procedure of the process performed by the monitoring server 3 according to the first embodiment will be described. FIG 9 is a flow diagram showing a procedure of the process performed by the monitoring server 3.

[0073] In the monitoring server 3, first, the processor 13 acquires the camera image received from the camera 1 via the communication unit 11 (ST101).

[0074] Next, the processor 13 inputs the camera image into an image recognition model for determining whether or not a train is present on the track along the platform (a train presence determination process) (ST102).

[0075] Next, processor 13 determines whether or not there is an error in the on-track determination result for each camera image acquired in the on-track determination process based on the position of the shooting range of each camera image, the expected train configuration, and the train's stopping sections according to the configuration, and if there is an error in the on-track determination result, performs a process to correct the on-track determination result (on-track determination result correction process) (ST103).

[0076] Next, the processor 13 performs a process (a process for detecting a person intruding on the tracks) that detects a person crossing the detection line and intruding on the tracks (a person intruding on the tracks) (ST104).

[0077] Next, as a notification necessity determination process, if the determination result of the on-track determination process is not on the track (No in ST104) and the detection result of the track intrusion person detection process is that a person intrusion on the tracks is present (Yes in ST106), the processor 13 determines that a notification to station staff is necessary and performs notification process (ST107). Specifically, the processor 13 causes the fixed notification terminal 4, the warning light 5, and the mobile notification terminal 6 to perform a notification operation to notify the station staff that a person has fallen.

[0078] Second embodiment Next, a second embodiment will be described. Note that points not specifically mentioned here are the same as those in the above embodiment. Fig. 10 is an explanatory diagram showing an overview of the process performed by the monitoring server 3 according to the second embodiment.

[0079] Accidents in which passengers fall from platforms onto the tracks occur when no trains are stopped on the tracks alongside the platform. On the other hand, when a train is stopped on the tracks alongside the platform, passengers are unlikely to fall from the platform onto the tracks where the train is stopped, i.e., in the train's stopped section, but they may fall from the platform onto the tracks in a location outside the train's stopped section.

[0080] On the other hand, in the process of detecting a person intruding on the tracks, a detection line is set on the boundary between the platform and the tracks, and a person who crosses the detection line and moves from the platform side to the tracks side is detected. Here, places on the platform where there is no possibility of a user falling off, specifically, sections where trains are stopped, can be excluded from the process of detecting a person intruding on the tracks. Therefore, in this embodiment, the detection line is set in sections where there is a possibility of a user falling off, i.e., sections excluding sections where trains are stopped.

[0081] Furthermore, the stopping sections of a train vary depending on the train's configuration, and the stopping sections of the train are determined for each train configuration. That is, the appearance status of the train in the camera images varies for each of the multiple camera images depending on the train's configuration. Therefore, in this embodiment, the train's configuration is estimated based on the appearance status of the train for each of the multiple camera images. Furthermore, the stopping sections of the train are estimated based on the estimated train configuration. Then, a detection line is set based on the estimated stopping sections of the train.

[0082] In the example shown in FIG. 10(A), it is determined that a train is on the track in the camera images of #2, #3, and #5, i.e., that a train is present on the tracks along the platform. The camera images of #2 and #3 show the train in its entirety. On the other hand, the camera image of #5 shows the train in the distance. In this case, it can be estimated that the train stopped at the platform is a three-car train. As a result, a detection line is set based on the stopping section of the three-car train.

[0083] In the example shown in FIG. 10(B), like the example shown in FIG. 10(A), the camera images from #2, #3, and #5 determine that the train is on the track, and furthermore, the camera images from #2 and #3 show the train in its entirety. Meanwhile, the camera image from #5 shows a train nearby. In this case, it can be estimated that the train stopped at the platform is a four-car train. As a result, a detection line is set based on the stopping section of the four-car train.

[0084] In this manner, in this embodiment, when a train is stopped on the tracks alongside the platform, the train's configuration is estimated and the detection line is set based on the train's stopping section corresponding to the train's configuration, thereby making it possible to reliably detect a person who has fallen onto the tracks in a section where the train is not stopped.

[0085] For example, in the example shown in Figure 10 (A), according to the first embodiment, the camera image of #5 is of a train on the line and therefore may be excluded from fall detection, but in this embodiment, by estimating the stopped train to be a three-car train and setting the detection line, it is possible to reliably detect a person who has fallen in the section just before the end (front or rear end) of the train that is shown in the distance in the camera image of #5.

[0086] Third embodiment Next, a third embodiment will be described. Note that points not specifically mentioned here are the same as those in the above-mentioned embodiments. Fig. 11 is an explanatory diagram showing an overview of the process performed by the monitoring server 3 according to the third embodiment.

[0087] In this embodiment, as in the second embodiment, a detection line is set on the boundary between the platform and the tracks excluding the train stop section, but unlike the second embodiment, the train stop section is estimated using object detection technology. Specifically, the front of a train stopped on the tracks along the platform is detected (train front detection process), and based on the detection result, the train stop section is estimated, and a detection line is set based on the train stop section.

[0088] Here, the front face of the train is the end of the train in the direction of travel, i.e., the front or rear end of the train, and the train's stopping section can be estimated based on the detected position of the train's front face. Specifically, the point where an extension line hanging down from the side edge of the detected rectangular front face intersects with the boundary between the platform and the track becomes the end point of the train's stopping section, and the detection line is set based on this end point.

[0089] When performing the train front face detection process, an area on the camera image where the front face of the train is likely to be captured is set in advance as a front face search area. For example, when the train front face detection process detects the periphery of the windshield as the train's front face, the area where the train's windshield is likely to be captured is set as the front face search area.

[0090] In addition, in the train front face detection process, an image (area image) of the front face search area is cut out from the camera image, and the front face of the train is searched for from the area image. This train front face detection process can be performed using an image recognition model (machine learning model). In this case, the image (area image) of the front face search area cut out from the camera image is input to an image recognition model for train front face detection, and the presence or absence of the front face of the train and position information of the front face of the train are output as image recognition results from the image recognition model.

[0091] In this manner, in this embodiment, the front face of the train is detected and the detection line is set based on the detection result, so even if the train does not stop at a predetermined stopping position according to the train configuration, the stopping section of the train can be identified with high accuracy and the detection line can be set appropriately. This makes it possible to reliably detect a person who has fallen onto the tracks in a section where the train is not stopped.

[0092] In this case, in a section where a train is stopped on the tracks alongside the platform, a person who crosses the detection line from the platform to the tracks will be detected as having fallen off, whereas a person boarding a train will not be detected as having fallen off because they do not cross the detection line.

[0093] As described above, the embodiments have been described as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made. In addition, it is also possible to combine the components described in the above embodiments to create new embodiments. [Industrial Applicability]

[0094] The surveillance device and surveillance system of the present invention have the effect of ensuring the accuracy of determining whether or not a vehicle is present in a no-entry area and preventing false alarms, even in situations where the way the subject appears in the camera image changes significantly, when detecting a person entering a no-entry area where a vehicle is traveling and issuing an alert based on camera images, and are useful as a surveillance device and surveillance system that detects and issues an alert when a person enters a no-entry area based on camera images from multiple cameras that photograph the no-entry area where a vehicle is traveling and boarding and disembarking locations. [Explanation of symbols]

[0095] 1 Camera 2 Recorder 3. Monitoring server (monitoring device) 4 Fixed notification terminal (notification device) 5. Warning lights (alarm devices) 6 Mobile notification terminal (notification device) 11 Communications Department 12. Memory 13 Processors 31 Alarm Mark 32 First camera image display unit 33 Second camera image display unit 34 Control section 35 Log display section

Claims

1. A monitoring device including a processor that detects a person entering a restricted area where a vehicle is traveling and a boarding and alighting location based on camera images from a plurality of cameras that photograph the restricted area and a boarding and alighting location, and instructs to issue a notification, The processor, A first process is performed for each of the plurality of camera images to detect a person who has entered the entry-prohibited area from the boarding / alighting location based on the camera images, and performing a second process for determining the presence or absence of the vehicle in the entry-prohibited area based on the camera images for each of the plurality of camera images; determining whether or not the notification is necessary based on both the detection result of the first processing and the determination result of the second processing; Furthermore, the monitoring device is characterized in that, prior to the process of determining whether or not an alert is necessary, the accuracy of the judgment results for each camera in the second process is comprehensively determined, and some of the judgment results for each camera image that are determined to be incorrect are corrected.

2. The processor, The monitoring device according to claim 1, characterized in that it determines whether or not there is an error in the on-track determination result for each camera image in the second processing based on the shooting range of the camera image, the expected configuration of the vehicle, and the stopping section of the vehicle according to the configuration.

3. The processor, The monitoring device described in claim 1, characterized in that the vehicle's configuration is estimated based on the appearance status of the vehicle in the camera image, a detection line is set on the boundary line between the no-entry area and the boarding and alighting location based on the stopping section corresponding to the estimated configuration, and the first processing is performed based on the detection line.

4. The processor, The monitoring device according to claim 1, characterized in that it detects the end of the vehicle in the direction of travel, sets a detection line on the boundary between the no-entry area and the boarding and disembarking location based on the position of the end, and performs the first processing based on the detection line.

5. A monitoring device according to any one of claims 1 to 4, A plurality of cameras for photographing the restricted area and the boarding and disembarking locations; an alarm device that performs a predetermined alarm operation in response to an instruction from the monitoring device; A surveillance system equipped with

Images