Surveillance device and surveillance system

Abstract

To make it possible to accurately perform on-rail determination using an image recognition model without readjusting parameters or relearning when the angle of view of a camera is deviated, but the deviation of the angle of view is within an allowable range.SOLUTION: A monitoring device performs first processing (railway intruder detection processing) for detecting a person who has entered a no-entry zone from a boarding and alighting place based on camera images of the no-entry zone where vehicles run and the boarding and alighting place, performs second processing (on-rail determination processing) for determining the presence or absence of a vehicle in the no-entry zone based on the camera image, determines necessity of notification based on both a detection result of the first processing and a determination result of the second processing, performs image conversion processing for reproducing the camera images of an original angle of view from the camera images of a current angle of view when the angle of view of the camera is deviated, and performs the first processing and the second processing by using the camera images obtained by the image conversion processing.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a monitoring device and a monitoring system that detects and issues an alert when a person enters a restricted area based on camera images of the restricted area and boarding and disembarking locations where vehicles are traveling. [Background technology]

[0002] At railway stations, passengers may fall from the platform onto the tracks, and it is desirable to detect such accidents early and quickly implement necessary measures such as operating an emergency stop and rescuing passengers.In recent years, therefore, a monitoring system has been proposed that uses camera images of the platform and tracks to detect people who have fallen from the platform onto the tracks and notify station staff of the occurrence of a fall.

[0003] A conventional monitoring system for detecting such persons who have fallen off a platform has been known to detect persons who have fallen off a platform based on camera images of the platform and the tracks along the platform. In order to avoid mistakenly detecting a train on the tracks along the platform as a person who has fallen, a technology is known that stops the process of detecting persons who have fallen off if a train is present on the tracks along the platform (see Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5386744 Summary of the Invention [Problem to be solved by the invention]

[0005] The camera's angle of view can shift due to various reasons. For example, a camera with a pan-tilt function may shift its angle of view due to an incorrect operation. The camera's angle of view may also shift due to vibrations caused by a passing train, etc. The camera's angle of view may also shift when a worker touches the camera during cleaning or inspection. When the camera's angle of view shifts in this way, the camera's shooting range changes.

[0006] On the other hand, conventional technology uses a template matching technique to determine whether or not a train is present on the tracks along the platform based on camera images.However, it is also possible to make on-track determination using an image recognition model (machine learning model) constructed using machine learning such as deep learning.

[0007] In such a case of using an image recognition model to detect where a train is present on a line, if the camera's viewing angle shifts and the camera's shooting range changes, the accuracy of the detection drops significantly. In such a case, it would be possible to readjust the parameters of the image recognition model or to take the time to collect learning data after the viewing angle has shifted and re-learn the model, but this would be very time-consuming.

[0008] Therefore, the main object of the present invention is to provide a monitoring device and monitoring system that can accurately determine whether a vehicle is on a line using an image recognition model, even if the camera's angle of view shifts, as long as the angle of view shift is within an acceptable range, without having to readjust or re-learn parameters. [Means for solving the problem]

[0009] The monitoring device of the present invention is a monitoring device including a processor that detects a person entering a restricted area where a vehicle is traveling and a boarding and alighting location based on a camera image acquired from a camera that photographs the restricted area and a boarding and alighting location, and instructs to issue a notification, the processor comprising: Acquired A first process is performed to detect a person who has entered the prohibited area from the boarding / alighting location based on the camera image, and The image was taken with the original angle of view of the camera.A second process is performed to determine the presence or absence of a vehicle in the no-entry area based on the acquired camera image using an image recognition model that has been trained in advance using the camera image when a vehicle is present and the camera image when a vehicle is not present as training data, and the need for the notification is determined based on both the detection result of the first process and the determination result of the second process. Furthermore, when a deviation in the angle of view of the camera occurs, It is determined whether or not the view angle deviation of the camera falls within a predetermined allowable range based on the detection state of the state determination area in the camera image, and when it is determined that the view angle deviation of the camera falls within the allowable range, From the camera image with the current angle of view The original angle of view The camera image obtained by the image conversion process is used to perform the first processing, and the second processing is performed using the image recognition model.

[0010] In addition, the surveillance system of the present invention is configured to include the surveillance device, a camera that photographs the no-entry area and the boarding and disembarking locations, and an alarm device that performs a predetermined alarm operation in response to instructions from the surveillance device. Effect of the Invention

[0011] According to the present invention, If it is determined that the camera's angle of view deviation falls within the allowable range based on the detection state of the state determination area in the camera image, The image conversion process reproduces the camera image with the original angle of view from the camera image with the current angle of view. This allows the camera image to be reproduced without time-consuming parameter readjustment or re-learning, even if the camera angle of view shifts, as long as the shift is within the allowable range for the image conversion process. Image recognition model built when the camera was installed This makes it possible to accurately determine whether the person is on the line using the above method. [Brief description of the drawings]

[0012] [Figure 1] Overall configuration diagram of a monitoring system according to a first embodiment [Diagram 2] FIG. 1 is an explanatory diagram showing a shooting situation of a camera according to a first embodiment; [Diagram 3] FIG. 1 is an explanatory diagram showing a detection line and an on-rail detection area set on a camera image according to a first embodiment; [Figure 4] FIG. 1 is an explanatory diagram showing an overview of a fall detection process performed by a monitoring server according to a first embodiment; [Diagram 5] FIG. 1 is an explanatory diagram showing an overview of an image conversion process performed by a monitoring server according to a first embodiment; [Figure 6] FIG. 1 is a block diagram showing a schematic configuration of a monitoring server according to a first embodiment. [Figure 7] FIG. 11 is an explanatory diagram showing a notification screen displayed on a fixed notification terminal according to the first embodiment; [Figure 8] FIG. 1 is a flowchart showing a procedure of a process performed by a monitoring server according to a first embodiment; [Figure 9] FIG. 11 is an explanatory diagram showing an overview of a state determination process performed by a monitoring server according to a second embodiment; [Figure 10] FIG. 11 is an explanatory diagram showing an example of a case where the view angle deviation of the camera according to the second embodiment falls within the allowable range and a case where the view angle deviation exceeds the allowable range. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] A first invention made to solve the above problems is a monitoring device including a processor that detects a person entering a restricted area where a vehicle is traveling and a boarding and alighting location based on a camera image acquired from a camera that photographs the restricted area and a boarding and alighting location, and issues a notification command, the processor comprising: Acquired A first process is performed to detect a person who has entered the prohibited area from the boarding / alighting location based on the camera image, and The image was taken with the original angle of view of the camera. A second process is performed to determine the presence or absence of a vehicle in the no-entry area based on the acquired camera image using an image recognition model that has been trained in advance using the camera image when a vehicle is present and the camera image when a vehicle is not present as training data, and the need for the notification is determined based on both the detection result of the first process and the determination result of the second process. Furthermore, when a deviation in the angle of view of the camera occurs, It is determined whether or not the view angle deviation of the camera falls within a predetermined allowable range based on the detection state of the state determination area in the camera image, and when it is determined that the view angle deviation of the camera falls within the allowable range, From the camera image with the current angle of view The original angle of view The camera image obtained by the image conversion process is used to perform the first processing, and the second processing is performed using the image recognition model.

[0014] According to this, If it is determined that the camera's angle of view deviation falls within the allowable range based on the detection state of the state determination area in the camera image, The image conversion process reproduces the camera image with the original angle of view from the camera image with the current angle of view. This allows the camera image to be reproduced without time-consuming parameter readjustment or re-learning, even if the camera angle of view shifts, as long as the shift is within the allowable range for the image conversion process. Image recognition model built when the camera was installed This makes it possible to accurately determine whether the person is on the line using the above method.

[0015] In addition, in a second invention, the processor is configured to perform the image conversion processing based on at least three reference points set in correspondence with each other for the camera image based on the current angle of view and the camera image based on the original angle of view.

[0016] This allows the image conversion process to be performed appropriately, so that the camera image with the original angle of view can be reproduced with high accuracy.

[0019] In addition, a third aspect of the present invention is a method for processing a signal from the processor, When it is determined that the angle of view deviation exceeds the allowable range, the first processing The system is configured to perform a process to notify the staff member that the request cannot be continued.

[0020] According to this, if the deviation in the camera's angle of view exceeds the acceptable range, the attendant is notified that the intrusion detection process cannot continue, so that maintenance work to adjust the camera's status is quickly carried out, thereby avoiding a situation where the intrusion detection process is not being carried out properly. In addition, a fourth aspect of the present invention is a method for controlling a computer system, comprising: A state in which the angle of view deviation exceeds the allowable range If this continues for a specified period of time or more, The first process The system is configured to perform a process to notify the staff member that the request cannot be continued. According to this, Angle of view shift If the state where the camera exceeds the tolerance continues for a predetermined period of time or more, a notice is sent to a staff member that the intrusion detection process cannot be continued, so that maintenance work for adjusting the camera state can be prevented from being left unattended even in places with a lot of foot traffic.

[0021] In addition, a fifth invention is a surveillance system comprising the surveillance device, a camera that photographs the no-entry area and the boarding and disembarking locations, and an alarm device that performs a predetermined alarm operation in response to instructions from the surveillance device.

[0022] According to this, as in the first invention, even if the camera's angle of view shifts, as long as the angle of view shift is within an acceptable range, it is possible to accurately determine whether or not a train is on the line using an image recognition model without readjusting or relearning the parameters.

[0023] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0024] (First embodiment) FIG. 1 is a diagram showing the overall configuration of a monitoring system according to a first embodiment.

[0025] The monitoring system detects a person (a fallen person) who falls from the platform onto the tracks at a railway station and notifies station staff of this. This monitoring system includes a camera 1, a recorder 2 (a recording device), a monitoring server 3 (a monitoring device), a fixed notification terminal 4 (announcement device), a warning light 5 (announcement device), and a mobile notification terminal 6 (announcement device). The camera 1, the recorder 2, the monitoring server 3, the fixed notification terminal 4, the warning light 5, and the mobile notification terminal 6 are connected to each other via a network.

[0026] Camera 1 is installed in each predetermined monitoring area within the station. Camera 1 captures the platform and the tracks along the platform as its monitoring area. Camera 1 transmits the camera images of the monitoring area to recorder 2 and monitoring server 3 via the network.

[0027] Recorder 2 acquires camera images from camera 1 and stores them.

[0028] The monitoring server 3 is installed in an equipment room, a data center, or the like within the station. The monitoring server 3 acquires camera images from the camera 1 and performs a process (fall detection process) of detecting a person (a fallen person) who falls from the platform onto the tracks. Furthermore, when the monitoring server 3 detects a person who has fallen, it performs a process of notifying station staff of the fact. Specifically, it instructs the fixed notification terminal 4, the warning light 5, and the mobile notification terminal 6 to perform a predetermined notification operation. The functions of the monitoring server 3 may be realized by a cloud computer.

[0029] The fixed notification terminal 4 is installed in an office or the like. The fixed notification terminal 4 can be realized by installing a fall notification application on a PC. In response to an instruction from the monitoring server 3, the fixed notification terminal 4 displays a notification screen as a notification operation to notify station staff that a person has fallen.

[0030] The warning light 5 is installed in an office or the like. In response to an instruction from the monitoring server 3, the warning light 5 turns on a lamp or outputs an alarm sound as a notification operation to notify station staff that a person has fallen.

[0031] The mobile notification terminal 6 is a mobile terminal such as a smartphone or a tablet terminal. The mobile notification terminal 6 is carried by a station staff member who rushes to the scene and takes necessary measures. In response to an instruction from the monitoring server 3, the mobile notification terminal 6 displays a notification screen, outputs an alarm sound, or outputs a vibration as a notification operation to inform the station staff member that a person has fallen.

[0032] In this embodiment, the device detects an accident in which a user falls from a platform at a railway station onto the tracks and notifies staff, but the device is not limited to railway stations. For example, the device may detect an accident in which a user enters (falls from) the deck where users get on and off an attraction ride into the running area of ​​the ride at an amusement park and notify staff.

[0033] In addition, in this embodiment, the system detects a person falling from the platform onto the tracks at a railway station and notifies station staff of the presence of a person who has fallen, but the target of monitoring is not limited to people, and the system may also detect when an object such as luggage has fallen onto the tracks and notify station staff of the presence of a fallen object.

[0034] Next, the fall detection process performed by the monitoring server 3 according to the first embodiment will be described. Fig. 2 is an explanatory diagram showing the shooting conditions of the camera 1. Fig. 3 is an explanatory diagram showing the detection line and the on-rail determination area set on the camera image. Fig. 4 is an explanatory diagram showing an overview of the fall detection process performed by the monitoring server 3.

[0035] 2, the camera 1 captures images of the platform (boarding and alighting area) and the tracks along the platform (no entry area) as the monitoring area. Note that multiple cameras 1 may be installed and the multiple cameras 1 may capture images of the platform and the tracks along the platform without omission, thereby eliminating blind spots in the fall detection process and preventing missed detections.

[0036] 3, the monitoring server 3 acquires camera images showing the platform and the tracks along the platform. Based on the camera images, the monitoring server 3 detects a person (a fallen person) falling from the platform onto the tracks (fall detection process) and notifies station staff that a fall accident has occurred.

[0037] Specifically, as shown in FIG. 4, the monitoring server 3 performs a track intrusion person detection process (first process) that detects a person who has intruded from the platform onto the tracks along the platform based on the camera image, and also performs a track presence determination process (second process) that determines the presence or absence of a train (vehicle) on the tracks along the platform based on the camera image, and determines whether or not an alert is needed based on both the detection result of the track intrusion person detection process and the determination result of the track presence determination process.

[0038] Here, if there is a train on the tracks next to the platform, that is, if a train is stopped on the tracks next to the platform or if a train is running on the tracks next to the platform, a person cannot enter (fall off) the tracks from the platform. Also, if a train is stopped on the tracks next to the platform and a person moves from the platform side to the tracks, that person is a passenger who has boarded the train and is not a person who has fallen off.

[0039] Therefore, in this embodiment, when it is determined by the on-track determination process for each camera image that a train is present on the tracks along the platform, the camera image is excluded from the targets of notification. In other words, when it is determined that no train is present on the tracks along the platform in a camera image, and a person who has intruded onto the tracks is detected by the on-track person detection process, a notification is instructed.

[0040] In this embodiment, a detection line is set on the boundary between the platform and the tracks in the camera image as shown in Fig. 3. In the process of detecting a person intruding on the tracks, a person who crosses the detection line and intrudes onto the tracks is detected. Specifically, a person on the platform is detected from the camera image (person detection process), and it is determined whether the detected person has crossed the detection line and moved from the platform side to the tracks side (track intrusion determination process).

[0041] In this embodiment, a rectangular on-track determination area is set in advance on the camera image for the on-track determination process, as shown in Fig. 3. When a train is present on the track along the platform in the camera image, this on-track determination area is set to a portion of the track covered by the body of the train. In the on-track determination process, the on-track determination area is cut out from the camera image to obtain an area image, and it is determined whether or not a train is present on the track along the platform based on the area image.

[0042] The on-track detection process is performed using an image recognition model (machine learning model). An image of the on-track detection area (area image) cut out from a camera image is input to the image recognition model, and the on-track detection result is output from the image recognition model. The image recognition model for on-track detection is constructed by performing supervised learning in advance using area images when a train is present on the tracks along the platform and area images when no train is present on the tracks along the platform as learning data.

[0043] As shown in Fig. 2, the angle of view of the camera 1 may shift due to various reasons. For example, the angle of view of the camera 1 may shift due to an erroneous operation of the camera 1 having a pan-tilt function. The angle of view of the camera 1 may also shift due to vibrations caused by a passing train, etc. The angle of view of the camera 1 may also shift when an operator touches the camera 1 during cleaning, inspection, etc.

[0044] In this embodiment, the angle of view of camera 1 represents the range of the subject actually captured by camera 1 and relates to the orientation (posture) of camera 1 as a shooting condition; further, in the case of a camera 1 with a zoom function, it also includes the zoom magnification as a shooting condition, i.e., the width of the shooting range centered on the optical axis of the lens.

[0045] When the angle of view of camera 1 shifts in this way, the shooting range of camera 1 changes, and the subject captured in the on-rail detection area set on the camera image shifts, reducing the accuracy of on-rail detection. Meanwhile, the on-rail detection process is performed using an image recognition model (machine learning model). Therefore, when the angle of view of camera 1 shifts, an engineer must readjust the parameters of the image recognition model, or spend time collecting learning data and redoing the learning. However, such a method is very time-consuming and burdensome.

[0046] Therefore, in this embodiment, as shown in Fig. 4, when a deviation in the angle of view occurs in camera 1, an image conversion process is performed to reproduce a camera image with an original angle of view (the angle of view at the time when the image recognition model was constructed, such as at the time of installation) from a camera image with the current angle of view, and a process for detecting a person intruding onto the tracks (first process) and a process for determining whether a person is on the tracks (second process) are performed using the camera image obtained by this image conversion process. This allows accurate determination of whether a person is on the tracks using the image recognition model, without readjusting or re-learning parameters.

[0047] Next, a description will be given of the image conversion process performed by the monitoring server 3 according to the first embodiment. Fig. 5 is an explanatory diagram showing an overview of the image conversion process.

[0048] When performing the image conversion process, multiple (at least three) reference points are set in correspondence with each other for the camera image with the current angle of view and the camera image with the original angle of view, and affine transformation is performed as the image conversion process based on the reference points. Note that the image conversion process is not limited to affine transformation, and for example, perspective projection transformation may be used.

[0049] In this embodiment, a worker sets multiple reference points by inputting the positions of multiple reference points for both the camera image with the current angle of view and the camera image with the original angle of view. At this time, the worker selects a point with a visually identifiable feature, such as a joint of a structure, and specifies it as the reference point. The worker also visually identifies the same position of the subject in each of the camera images with the current angle of view and the original angle of view, and specifies it as the reference point.

[0050] FIG. 5(A) is a camera image with the original angle of view, and FIG. 5(B) is a camera image with the current angle of view. The camera image with the current angle of view has an angle of view deviation compared to the camera image with the original angle of view.

[0051] Three reference points P1, P2, and P3 are set in correspondence with each other in the camera image with the original angle of view shown in Fig. 5(A) and the camera image with the current angle of view shown in Fig. 5(B). In the image transformation process, affine transformation is performed on the camera image with the current angle of view shown in Fig. 5(B) so that the three reference points P1, P2, and P3 on the camera image with the current angle of view shown in Fig. 5(B) overlap with the three reference points P1, P2, and P3 on the camera image with the original angle of view shown in Fig. 5(A).

[0052] 5, three reference points are set on the camera image, but four or more reference points may be set on the camera image. In this case, for example, three of the set reference points may be used for image conversion processing.

[0053] In addition, in this embodiment, the reference point is set by the operator selecting a point with a characteristic that can be visually identified and designating it as the reference point, but the monitoring server 3 may set the reference point by detecting a characteristic part in the camera image by image recognition.

[0054] Next, a description will be given of a schematic configuration of the monitoring server 3 according to the first embodiment.

[0055] The monitoring server 3 includes a communication unit 11, a memory 12, and a processor 13.

[0056] The communication unit 11 communicates with the camera 1, the recorder 2, the fixed notification terminal 4, the warning light 5, and the mobile notification terminal 6 via the network.

[0057] The memory 12 stores programs executed by the processor 13 and the like.

[0058] The processor 13 performs various processes related to fall detection by executing the programs stored in the memory 12. In this embodiment, the processor 13 performs image acquisition process, image conversion process, on-rail determination process (second process), on-rail person intrusion detection process (first process), notification necessity determination process, notification process, and the like.

[0059] In the image acquisition process, the processor 13 acquires camera images received from each camera 1 via the communication unit 11. The camera 1 transmits camera images captured in a monitoring area at a predetermined frame rate (for example, 5 fps).

[0060] In the image conversion process, the processor 13 reproduces a camera image with an original angle of view (the angle of view at the time when the image recognition model was constructed, such as at the time of installation) from the camera image with the current angle of view, for the camera image with the angle of view shift. Specifically, based on the reference points set for the camera image with the current angle of view and the camera image with the original angle of view, an affine transformation is performed to convert the camera image with the current angle of view into a camera image that approximates the camera image with the original angle of view. Note that the method is not limited to the affine transformation, and for example, a perspective projection transformation may be used.

[0061] In the on-track determination process, the processor 13 determines whether or not a train is stopped on the track along the platform based on each camera image. At this time, if image conversion processing has been performed on some of the camera images, the camera images that have been processed in the image conversion processing are used.

[0062] The on-rail detection process uses an image recognition model (machine learning model) for on-rail detection. An image of the on-rail detection area (area image) set on the camera image is cut out from the camera image, and the area image is input to the image recognition model for on-rail detection, and the on-rail detection result is output from the image recognition model. This on-rail detection process is performed individually for each camera image, and an on-rail detection result is obtained for each camera image.

[0063] In addition, the image recognition model may output a reliability score indicating the accuracy of the image recognition result as the on-track determination result, and this reliability score may be compared with a predetermined threshold value to obtain an on-track determination result indicating whether or not a train is stopped on the tracks along the platform.

[0064] In the process of detecting a person intruding on the tracks, the processor 13 detects a person who crosses the detection line and moves from the platform side to the track side based on each camera image. At this time, if image conversion processing has been performed on some of the camera images, the camera images that have been processed in the image conversion processing are used.

[0065] In the process of detecting a person intruding on the tracks, the processor 13 first detects a person on the platform from the camera image (person detection process). Next, the processor 13 determines whether or not the detected person has crossed the detection line and moved from the platform side to the track side based on the detection result of the person detection process (track intrusion determination process). This process of detecting a person intruding on the tracks is performed individually for each camera image, and a result of detecting a person intruding on the tracks is obtained for each camera image.

[0066] In the process for determining whether or not an alert is required, processor 13 determines whether or not an alert is required to inform staff of the occurrence of a fall accident based on the result of the detection of a person intruding on the tracks for each camera image acquired in the process for detecting a person intruding on the tracks and the result of the presence on the tracks determination for each camera image acquired in the process for determining whether or not a person is on the tracks.

[0067] Here, when a person is detected on a camera image that does not show a presence on the tracks, the system judges the person to be a person who has fallen off the tracks and determines that it is necessary to notify station staff. On the other hand, when a person is detected on the tracks but the presence on the tracks determination result for that camera image shows a person is on the tracks, the system judges the person to be a passenger getting on a train and determines that it is not necessary to notify station staff.

[0068] In the notification process, the processor 13 instructs the station staff to perform notification process according to the result of the notification necessity determination process. Specifically, the fixed notification terminal 4, the warning light 5, and the mobile notification terminal 6 are made to perform notification operations to notify the station staff that a person has fallen. Note that, as a mask function, the notification process is stopped during a time period designated by the manager, such as a time period when no trains are running.

[0069] Next, a notification screen displayed on the fixed notification terminal 4 according to the first embodiment will be described. Fig. 7 is an explanatory diagram showing the notification screen.

[0070] In response to an instruction from the monitoring server 3, the fixed notification terminal 4 displays a notification screen as a notification operation for notifying station staff that a person has fallen.

[0071] An alarm mark 31 is displayed on the notification screen, which indicates that a fall detection notification is being issued.

[0072] The notification screen is provided with a first camera image display section 32 and a second camera image display section 33. The first camera image display section 32 displays a camera image at the time when a person intruding on the tracks is detected. The second camera image display section 33 displays a camera image from a predetermined time before the time when a person intruding on the tracks is detected. The camera images displayed on the first camera image display section 32 and the second camera image display section 33 are recorded in the recorder 2, and are acquired from the recorder 2 and displayed on the screen. The first camera image display section 32 displays the camera image as a still image. On the other hand, the second camera image display section 33 displays the camera image as a video by the station staff operating the operation section 34. This allows the station staff to check the situation immediately after the person falls on the first camera image display section 32. The station staff can also check the situation from just before the person falls to during the person falls on the second camera image display section 33.

[0073] The notification screen is also provided with a log display section 35. The log display section 35 displays a list of fall events detected in the past. Specifically, the date and time of the report for each fall event, the camera name, and the like are displayed. When any of the fall accidents displayed on the log display section 35 is selected, the camera images related to the selected fall accident are displayed on the first camera image display section 32 and the second camera image display section 33. This allows the station staff to select a fall event detected in the past and check the situation of the fall event using the camera images. Also, the station staff may input search conditions from a log search input screen (not shown), select a fall event from the logs hit by the search conditions, and check the situation of the fall event using the camera images.

[0074] When the notification screen is displayed on the fixed notification terminal 4 in this manner, at the same time, the warning light 5 and the mobile notification terminal 6 perform a predetermined notification operation in response to an instruction from the monitoring server 3. The warning light 5 notifies station staff that a person has fallen by turning on a lamp and outputting an alarm sound as notification operations. The mobile notification terminal 6 also notifies station staff that a person has fallen by displaying a notification screen, outputting an alarm sound, or vibrating as notification operations. The notification screen of the mobile notification terminal 6 may display a camera image at the time when a person entering the tracks is detected.

[0075] Next, a procedure of the process performed by the monitoring server 3 according to the first embodiment will be described. FIG 8 is a flow diagram showing a procedure of the process performed by the monitoring server 3.

[0076] In the monitoring server 3, first, the processor 13 acquires the camera image received from the camera 1 via the communication unit 11 (ST101).

[0077] Next, the processor 13 performs a process (image conversion process) for reproducing a camera image with the original angle of view from a camera image with the current angle of view for the camera 1 in which a deviation in angle of view within the allowable range has occurred (ST102).

[0078] Next, the processor 13 inputs the camera image into an image recognition model for on-track determination, and performs a process (on-track determination process) of determining whether or not a train is present on the track along the platform (ST103).

[0079] Next, if the determination result of the on-track determination process is that a person is not on the track (No in ST104), processor 13 then performs a process (track intrusion person detection process) to detect a person (person on the tracks) crossing the detection line and intruding onto the tracks (ST105).

[0080] Next, when the detection result of the process of detecting a person intruding on the tracks indicates that a person intruding on the tracks is present (Yes in ST106), the processor 13 performs a process of notifying station staff (ST107). Specifically, the processor 13 causes the fixed notification terminal 4, the warning light 5, and the mobile notification terminal 6 to perform a notification operation to notify the station staff that a person has fallen off the tracks.

[0081] However, when the angle of view deviation falls within the allowable range (the amount of variation in the reference point is small), the image conversion process can be performed appropriately, and the fall detection process can be continued, but when the angle of view deviation exceeds the allowable range (the amount of variation in the reference point is large), the image conversion process cannot be performed appropriately, and so if the fall detection is continued forcibly, false alarms will occur frequently. For this reason, when the angle of view deviation exceeds the allowable range, the fall detection process must be stopped.

[0082] On the other hand, even if the angle of view deviation exceeds the allowable range, the fall detection process can be resumed when the angle of view deviation falls within the allowable range by manually adjusting the state of the camera 1. In this case, the angle of view does not need to be strictly restored to its initial state, which makes maintenance work easier.

[0083] Second embodiment Next, a second embodiment will be described. Note that the points not specifically mentioned here are the same as those of the above embodiment.

[0084] In this embodiment, the monitoring server 3 judges whether the deviation in the angle of view of the camera 1 exceeds a predetermined allowable range (state judgment process). If the deviation in the angle of view of the camera 1 exceeds the allowable range, the image conversion process cannot be performed properly and the fall detection process cannot be continued, and therefore a station attendant is notified that the fall detection process cannot be continued. On the other hand, if the deviation in the angle of view of the camera 1 is within the allowable range, the image conversion process can be performed properly and the fall detection process continues.

[0085] Furthermore, when a station attendant is notified that the fall detection process cannot be continued, a maintenance operation is performed in which an operator adjusts the state of camera 1. This allows the view angle deviation of camera 1 to be returned to a state within the allowable range. At this time, the monitoring server 3 determines whether the view angle deviation of camera 1 is within the allowable range (state determination process) and determines that the view angle deviation of camera 1 is within the allowable range, and notifies the attendant that the state has been restored to one in which the fall detection process can be resumed.

[0086] Next, the state determination process performed by the monitoring server 3 according to the second embodiment will be described. Fig. 9 is an explanatory diagram showing an overview of the state determination process. Fig. 10 is an explanatory diagram showing an example of a case where the angle of view deviation of the camera 1 falls within the allowable range and a case where it exceeds the allowable range.

[0087] As shown in Fig. 9, in this embodiment, multiple (at least three) reference points that serve as the basis for the image conversion process are set on the camera image based on the original angle of view. Also, in order to determine whether the angle of view deviation exceeds an allowable range, a state determination area including the multiple reference points is set on the camera image based on the original angle of view. In the example shown in Fig. 9, three reference points P1, P2, and P3 are set on the camera image, and a state determination area including these three reference points P1, P2, and P3 is set.

[0088] In the monitoring server 3, when a state where the state determination area cannot be detected from the camera image, that is, when the state where the deviation of the angle of view of the camera 1 exceeds the allowable range, continues for a predetermined time or more, a notification is issued to inform a staff member that the fall detection process cannot be continued. Here, in places with a lot of people, such as train platform, it is common for part of the state determination area to be hidden by people and become invisible, so the predetermined time is set to, for example, one hour.

[0089] The detection of the status determination area in the camera image can be performed by using an image recognition model (machine learning model) for status determination. In this case, an area image obtained by cutting out an image of the status determination area from the camera image with the original angle of view is input to the image recognition model, and a region corresponding to the area image, i.e., a recognition result as to whether or not the status determination area has been detected, is output from the image recognition model.

[0090] 10(A) and (B) show cases where the angle of view of camera 1 has shifted. In the example shown in Fig. 10(A), a state determination area including three reference points P1, P2, and P3 can be detected from the camera image, so it is determined that the angle of view shift does not exceed the allowable range. On the other hand, in the example shown in Fig. 10(B), a state determination area including three reference points P1, P2, and P3 cannot be detected from the camera image, so it is determined that the angle of view shift exceeds the allowable range.

[0091] In this manner, in this embodiment, if the angle of view deviation exceeds the acceptable range, a notice is sent to staff that the intrusion detection process cannot continue, so that maintenance work to adjust the state of camera 1 can be carried out promptly, thereby avoiding a situation in which the intrusion detection process is not being carried out properly.

[0092] In this embodiment, when a status determination area cannot be detected from a camera image, it is determined that the angle of view deviation of camera 1 exceeds the allowable range. However, even when a status determination area can be detected from a camera image, if the amount of change in the position of the status determination area is equal to or greater than a predetermined value, it may be determined that the angle of view deviation of camera 1 exceeds the allowable range, since the image conversion process cannot be performed appropriately.

[0093] Although an example of setting a state determination area has been described with respect to determining whether the view angle deviation of the camera 1 is within the allowable range, various other methods can be adopted. For example, whether the view angle deviation of the camera 1 is within the allowable range may be determined based on whether the amount of change in the position of the center of gravity of the triangle formed by the three reference points P1, P2, and P3 is less than a predetermined value or is equal to or greater than a predetermined value.

[0094] As described above, the embodiments have been described as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made. In addition, it is also possible to combine the components described in the above embodiments to create new embodiments. [Industrial Applicability]

[0095] The monitoring device and monitoring system of the present invention have the advantage that, even if the camera's angle of view shifts, as long as the angle of view shift is within an acceptable range, it can accurately determine whether a person is on the line using an image recognition model without readjusting or relearning parameters, and is useful as a monitoring device and monitoring system that detects and alerts the entry of a person into a no-entry area based on camera images of the no-entry area and boarding and disembarking locations where vehicles are traveling. [Explanation of symbols]

[0096] 1 Camera 2 Recorder 3. Monitoring server (monitoring device) 4 Fixed notification terminal (notification device) 5. Warning lights (alarm devices) 6 Mobile notification terminal (notification device) 11 Communications Department 12. Memory 13 Processors P1,P2,P3 Reference point

Claims

1. A monitoring device including a processor that detects a person entering a restricted area where a vehicle is traveling and a boarding and alighting location based on a camera image acquired from the camera and instructs to issue a notification, The processor, A first process is performed to detect a person who has entered the prohibited area from the boarding / alighting location based on the acquired camera image, and performing a second process of determining whether or not the vehicle is present in the no-entry zone based on the acquired camera images using an image recognition model that has been trained in advance using, as training data, camera images captured with the original angle of view at the time the camera was installed, in which a vehicle is present and camera images captured in which a vehicle is not present; determining whether or not the notification is necessary based on both the detection result of the first processing and the determination result of the second processing; Furthermore, when a deviation in the angle of view of the camera occurs, it is determined whether or not the deviation in the angle of view of the camera falls within a predetermined allowable range based on a detection state of a state determination area in the camera image; A monitoring device characterized in that, when it is determined that the camera's angle of view deviation falls within the acceptable range, an image conversion process is performed to reproduce the camera image with the original angle of view from the camera image with the current angle of view, and the first processing is performed using the camera image obtained by this image conversion process, while the second processing is performed using the image recognition model.

2. The processor, The monitoring device according to claim 1, characterized in that the image conversion processing is performed based on at least three reference points set in correspondence with each other for the camera image based on the current angle of view and the camera image based on the original angle of view.

3. The processor, 2. The monitoring device according to claim 1, wherein, when it is determined that the angle of view deviation exceeds the allowable range, a process is performed to notify a staff member that the first process cannot be continued.

4. The processor, The monitoring device according to claim 3, further comprising a process for notifying an attendant that the first process cannot be continued if the state in which the angle of view deviation exceeds the allowable range continues for a predetermined period of time or more.

5. A monitoring device according to any one of claims 1 to 4, A camera for photographing the restricted area and the boarding and disembarking locations; an alarm device that performs a predetermined alarm operation in response to an instruction from the monitoring device; A surveillance system equipped with

Images