Safety monitoring system and safety monitoring method

Abstract

To efficiently utilize a camera installed on a side surface of a movable body even when the movable body is moved between boarding facilities regardless of the case where the movable body such as a train stops at a boarding facility from the viewpoint of securing the safety of a user.SOLUTION: A safety monitoring system includes a camera 1 provided corresponding to a door through which a person gets on / off a train, an image analyzing server 2 detecting a warning event, which may result in hindrance on ensuring the safety of a user based on a camera image photographed by the camera and generating alarm information, and monitoring terminals 3, 6 acquiring the alarm information and notifying a staff of an occurrence situation of the warning event. The image analyzing server determines whether or not a present position of a train is in a station based on a camera image and detects a predetermined warning event in accordance with the determination result based on the camera image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a safety monitoring system and a safety monitoring method that detects an event that may hinder ensuring the safety of users of a mobile body as a warning event and notifies an attendant of the occurrence of the warning event. [Background technology]

[0002] In railways, it is desirable to detect any events that could hinder passenger safety early and promptly notify train crew members and control center staff of the occurrence of such events.

[0003] In response to such demands, a technology has been known in the past in which a camera is installed on the side of a train car body, and based on the camera image taken by the camera, people on the platform are detected, and tactile paving blocks installed at the edge of the platform are also detected, and if a person on the platform is close to the edge of the platform, a notification is given to the train crew that there is danger (see Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-33601 Summary of the Invention [Problem to be solved by the invention]

[0005] In conventional technology, cameras installed on the side of the train body are used to check safety when the train is stopped at a station platform. This can prevent accidents such as passengers coming into contact with the train body. However, it is also possible that a door may malfunction while the train is traveling between stations, or that a passenger may disembark while the train is stopped between stations. In such cases, cameras installed on the side of the train body could also be used to check safety.

[0006] Therefore, the main object of the present invention is to provide a safety monitoring system and a safety monitoring method that can effectively utilize cameras installed on the sides of a moving body such as a train, not only when the moving body is stopped at a boarding and disembarking facility, but also when the moving body is moving between boarding and disembarking facilities, from the perspective of ensuring the safety of users. [Means for solving the problem]

[0007] The safety monitoring system of the present invention comprises: A vehicle with a door for passengers to get on and off This may be an obstacle to ensuring the safety of users. Occurrence of foreign objects getting caught in the door and warning events Detecting The detection results A safety monitoring system that generates alarm information including: The door and a camera provided corresponding to the camera image captured by the camera. The occurrence of pinching and The system includes a server device that detects the alert event and generates the alert information, and a terminal device that acquires the alert information and notifies an attendant of the occurrence of the alert event, wherein the server device: When the moving body is stopped at a boarding / deboarding facility, the occurrence of the pinching is detected based on the camera image, and further Based on the camera image, the current position of the moving object is Inside or outside the boarding / alighting facility and depending on the result of the judgment, Beforehand, inside and outside the boarding and disembarking facilities The set alert event is detected based on the camera image.

[0008] The safety monitoring method of the present invention also includes: A vehicle with a door for passengers to get on and off This may be an obstacle to ensuring the safety of users. Occurrence of foreign objects getting caught in the door and warning events Detecting The detection results A safety monitoring method that causes an information processing device to perform a process of generating alarm information including: The door Acquire a camera image taken by a camera provided corresponding to the When the moving body is stopped at a boarding / deboarding facility, the occurrence of the pinching is detected based on the camera image, and further Based on the camera image, the current position of the moving object is Inside or outside the boarding / alighting facility and depending on the result of the judgment, Beforehand, inside and outside the boarding and disembarking facilities The set alert event is detected based on the camera image. [Effects of the Invention]

[0009] According to the present invention, When the moving object is stopped at a boarding / alighting facility, the system detects whether a foreign object has been caught in the door, and furthermore,Since the detected warning events change depending on whether the current location of the moving object is within a boarding or alighting facility, it is possible to detect the appropriate warning event according to the situation and issue an appropriate alert. As a result, from the perspective of ensuring the safety of users, the cameras installed on the sides of moving objects can be effectively used not only when the moving object, such as a train, is stopped at a boarding or alighting facility, but also when the moving object is moving between boarding or alighting facilities. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram showing the overall configuration of a safety monitoring system according to the present embodiment; [Figure 2] An explanatory diagram showing an example of a monitoring situation according to the direction of train travel. [Figure 3] FIG. 1 is an explanatory diagram showing a camera image captured by a camera; [Figure 4] An explanatory diagram showing warning events detected by the image analysis server [Figure 5] An explanatory diagram showing a detection area set on a camera image. [Figure 6] Block diagram showing the general configuration of the image analysis server and train management server [Figure 7] An explanatory diagram showing the monitoring screen displayed on the monitoring terminal inside the train [Figure 8] An explanatory diagram showing the monitoring screen displayed on the monitoring terminal inside the train [Figure 9] An explanatory diagram showing the main parts of the monitoring screen [Figure 10] An explanatory diagram showing the settings screen displayed on the monitoring terminal inside the train [Figure 11] An explanatory diagram showing the settings screen displayed on the monitoring terminal inside the train [Figure 12] An explanatory diagram showing the settings screen displayed on the monitoring terminal inside the train [Figure 13] An explanatory diagram showing the settings screen displayed on the monitoring terminal inside the train [Figure 14] Flow diagram showing the processing steps performed by the image analysis server [Figure 15] An explanatory diagram showing a monitoring screen displayed on a monitoring terminal at a monitoring center. [Figure 16]An explanatory diagram showing a monitoring screen displayed on a monitoring terminal at a monitoring center. [Figure 17] An explanatory diagram showing a monitoring screen displayed on a monitoring terminal at a monitoring center. [Figure 18] An explanatory diagram showing a monitoring screen displayed on a monitoring terminal at a monitoring center. DETAILED DESCRIPTION OF THE INVENTION

[0011] The first invention made to solve the above problem is: A vehicle with a door for passengers to get on and off This may be an obstacle to ensuring the safety of users. Occurrence of foreign objects getting caught in the door and warning events Detecting The detection results A safety monitoring system that generates alarm information including: The door and a camera provided corresponding to the camera image captured by the camera. The occurrence of pinching and The system includes a server device that detects the alert event and generates the alert information, and a terminal device that acquires the alert information and notifies an attendant of the occurrence of the alert event, wherein the server device: When the moving body is stopped at a boarding / deboarding facility, the occurrence of the pinching is detected based on the camera image, and further Based on the camera image, the current position of the moving object is Inside or outside the boarding / alighting facility and depending on the result of the judgment, Beforehand, inside and outside the boarding and disembarking facilities The set alert event is detected based on the camera image.

[0012] According to this, When the moving object is stopped at a boarding / alighting facility, the system detects whether a foreign object has been caught in the door, and furthermore, Since the warning events to be detected are changed depending on whether the current location of the mobile object is within a boarding / alighting facility, it is possible to detect a warning event appropriate to the situation and issue an appropriate alert. This makes it possible to effectively use the cameras installed on the sides of the mobile object, not only when the mobile object, such as a train, is stopped at a boarding / alighting facility, but also when the mobile object is moving between boarding / alighting facilities, from the perspective of ensuring the safety of users. Note that the determination of whether the current location of the mobile object is within a boarding / alighting facility can be made based on whether an object specific to the boarding / alighting facility can be recognized from the camera image.

[0013] In addition, in a second invention, when the mobile body is located within the boarding and disembarking facility, the server device is configured to detect, as the alert event, at least one of a person rapidly approaching a train and a person who needs to reserve time to board and disembark safely.

[0014] This allows for appropriate detection of anticipated monitoring events when a mobile body is located within a boarding / disembarking facility.

[0015] In addition, in a third invention, the server device is configured to detect, as the alert event, at least one of an abnormality in an accessory of the mobile body and a person getting off the mobile body when the mobile body is located outside the boarding and disembarking facility.

[0016] This allows for appropriate detection of anticipated monitoring events when the mobile body is located outside the boarding and disembarking facility.

[0017] In addition, in a fourth aspect of the present invention, the server device detects a state of an object related to the alert event, and determines whether or not an alert needs to be issued based on the state of the object.

[0018] According to this, even if an alert event is detected, false alarms can be prevented by determining whether or not an alert needs to be issued based on the state of the object related to the alert event.

[0019] In addition, a fifth invention is configured such that the terminal device displays a monitoring screen including a plurality of the camera images, and the server device changes the display layout of the plurality of the camera images on the monitoring screen depending on the state of the moving body.

[0020] This allows camera images to be displayed on the monitoring screen in an appropriate layout according to the state of the moving object, for example, the direction of travel of the moving object, making it easier for staff to check safety using camera images.

[0021] The sixth invention is: A vehicle with a door for passengers to get on and off This may be an obstacle to ensuring the safety of users. Occurrence of foreign objects getting caught in the door and warning events Detecting The detection results A safety monitoring method that causes an information processing device to perform a process of generating alarm information including: The door Acquire a camera image taken by a camera provided corresponding to the When the moving body is stopped at a boarding / deboarding facility, the occurrence of the pinching is detected based on the camera image, and further Based on the camera image, the current position of the moving object is Inside or outside the boarding / alighting facility and depending on the result of the judgment, Beforehand, inside and outside the boarding and disembarking facilities The set alert event is detected based on the camera image.

[0022] According to this, as with the first invention, from the viewpoint of ensuring the safety of users, cameras installed on the sides of moving bodies such as trains can be effectively utilized not only when the moving body is stopped at a boarding and disembarking facility, but also when the moving body is moving between boarding and disembarking facilities.

[0023] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0024] FIG. 1 is a diagram showing the overall configuration of a safety monitoring system according to this embodiment.

[0025] The safety monitoring system detects events that may hinder the safety of train (mobile) users as warning events, and notifies personnel (train crew members and train control center commanders) of the occurrence of the warning event. The safety monitoring system includes a camera 1, an image analysis server 2 (server device), a monitoring terminal 3 (terminal device), a wireless base station 4, a train management server 5 (server device), and a monitoring terminal 6 (terminal device).

[0026] The camera 1, image analysis server 2, and monitoring terminal 3 are connected via a network within the train. The image analysis server 2 and wireless base station 4 are connected via a mobile communication network. The wireless base station 4, train management server 5, and monitoring terminal 6 are connected via a network such as a closed network or the Internet.

[0027] The camera 1 is installed on the side of the train body and photographs the outside of the doors through which passengers board and disembark from the side.

[0028] The image analysis server 2 is mounted on the train. The image analysis server 2 is composed of an information processing device such as a PC. The image analysis server 2 detects whether a foreign object (such as the body or belongings of a passenger getting on or off) has been caught in the door based on the camera images acquired from each camera 1. The detection results of this entrapment detection process are sent to the monitoring terminal 3 as alert information. The image analysis server 2 also detects an alert event based on the camera images acquired from each camera 1. The detection results of this alert event are sent to the monitoring terminal 3 as alert information. The analysis results and camera images, including the detection results of the alert event, are stored in the server itself, and the camera images and analysis results are sent to the train management server 5 as alert information in real time or at an appropriate timing.

[0029] The monitoring terminal 3 is installed in the train's crew cabin (driver's cab, conductor's cabin, etc.). The monitoring terminal 3 is equipped with a monitor (display device), a speaker, and a lamp. When the image analysis server 2 detects entrapment, the monitoring terminal 3 displays a monitoring screen showing the door position where the entrapment occurred, and alerts the train crew (driver, conductor, etc.) of the door entrapment by outputting an alert sound and turning on a lamp, etc. Furthermore, the monitoring terminal 3 displays the camera images acquired from each camera 1 on the monitoring screen, and when the image analysis server 2 detects an alert event, it highlights the camera image in which the alert event occurred and alerts the train crew (driver, conductor, etc.) of the occurrence of the alert event.

[0030] The train management server 5 is installed in a monitoring center (operation control center) or the like. The train management server 5 collects camera images and analysis results from the image analysis server 2 installed on each train, and performs required processing on the camera images and analysis results to generate various monitoring screens that show staff the occurrence status of warning events, and distributes the monitoring screens to the monitoring terminal 6 as alert information.

[0031] The monitoring terminal 6 is installed in a monitoring center or the like. The monitoring terminal 6 is viewed by a staff member (dispatcher) at the monitoring center or the like. The monitoring terminal 6 displays a monitoring screen distributed from the train management server 5.

[0032] In this embodiment, an example will be described in which the moving body (a vehicle that carries people) that a person boards is a train, but the moving body is not limited to a train. For example, the moving body may be a moving body that uses a cableway, such as a ropeway or a lift. It may also be a moving body that runs on a road, such as a bus.

[0033] Next, a monitoring situation according to the traveling direction of the train will be explained. Fig. 2 is an explanatory diagram showing an example of a monitoring situation according to the traveling direction of the train.

[0034] In the example shown in Figure 2, the train is operated by a single driver. The train is a four-car formation consisting of cars 1, 2, 3, and 4. The train has crew cabins (driver's cabs) in cars 1 and 4. As shown in Figure 2(A), when the train is traveling in the outbound direction, the driver is on duty in car 1, which is the lead car. As shown in Figure 2(B), when the train is traveling in the inbound direction, the driver is on duty in car 4, which is the lead car.

[0035] In the example shown in Figure 2, two cameras 1 are installed on each side of each car. That is, two cameras 1, #1-1 and #1-2, are installed on the left side of car 1, and two cameras 1, #1-3 and #1-4, are installed on the right side of car 1. Cameras 1 are also installed in cars 2, 3, and 4 in the same way.

[0036] The train has monitoring terminals 3 installed in the crew cabins (driver's cabs) of cars 1 and 4. As shown in Figure 2(A), when the train is traveling in the outbound direction, the occurrence of an alert event is notified to the driver by the monitoring terminal 3 installed in car 1. As shown in Figure 2(B), when the train is traveling in the inbound direction, the occurrence of an alert event is notified to the driver by the monitoring terminal 3 installed in car 4.

[0037] In this embodiment, the case where the train is operated by one driver and the driver in the lead car performs safety checks has been described, but the case where the conductor is in the last car may also be described. In this case, the monitoring terminal 3 where the conductor performs safety checks will be the opposite to when the driver performs safety checks, as shown in Figures 2(A) and (B).

[0038] Next, a description will be given of the camera image captured by the camera 1. Fig. 3 is an explanatory diagram showing the camera image. Fig. 4 is an explanatory diagram showing an alert event detected by the image analysis server 2.

[0039] Camera 1 is installed on the side of the train body and captures the outside of the doors where passengers board and disembark from the side. Therefore, when the train is stopped at a station platform, the camera image taken by camera 1 captures the station platform and passengers, as shown in Figure 3(A). On the other hand, when the train is traveling between stations, the camera image captures the tracks and surrounding overhead power lines, etc., as shown in Figure 3(B).

[0040] In this embodiment, when a train is stopped at a platform, the image analysis server 2 detects whether a foreign object (such as the body or belongings of a passenger) has been caught in the door based on camera images, and notifies the train crew (driver or conductor) of the occurrence of a foreign object being caught in the door as alarm information.

[0041] In addition, in this embodiment, based on camera images, warning events that may hinder ensuring the safety of users are detected, and the occurrence of a warning event is notified to train crew members (drivers and conductors) and staff at the monitoring center (operation control center) as alarm information.

[0042] In particular, in this embodiment, as shown in Figure 4, depending on whether the platform can be recognized from the camera image, it is determined whether the train position (current position of the train) is inside the station (position along the platform) or between stations (outside the station) (train position determination process), and the events to be detected as alert events are changed depending on the train position.

[0043] First, when the train position (current position of the train) is within a station (a position along the platform), in order to ensure the safety of passengers (boarding and alighting) on ​​the platform, a person requiring caution, specifically a person rapidly approaching the train, for example, a person rushing to board the train, is detected as a warning event. Also, a person requiring special consideration, specifically, a person who needs time to ensure safe boarding and alighting, for example, a person using a wheelchair, a stroller, or a white cane, is detected as a warning event.

[0044] In detecting an alert event, people are detected from camera images, and object recognition technology is used to detect wheelchairs, strollers, and white canes from camera images, and the detection of these objects is determined to be an alert event.

[0045] On the other hand, when the train is between stations (outside a station), in order to ensure the safety of passengers (people on the train), warning events are detected, such as abnormalities in the train's accessories (accessories and equipment on the car body), such as open doors and broken or fallen windows.Furthermore, warning events are detected when people (passengers) get off the train.

[0046] In this embodiment, when the train is located between stations (outside a station), abnormalities in train accessories are detected as warning events. However, other objects that have been blown in by strong winds and become entangled in the train (such as a plastic sheet) may also be detected as warning events.

[0047] In this embodiment, a person is detected from a camera image and it is determined whether or not the person corresponds to an alert event. In the example shown in Fig. 3(A), person A is a person who uses a white cane and corresponds to an alert event (person requiring consideration). Furthermore, person B is a person who rushes onto a train and corresponds to an alert event (person requiring caution).

[0048] The system also determines the state (orientation and posture) of people who fall under the alert event and decides whether or not to actually issue an alert. In the example shown in Figure 3(A), people A and B have their bodies facing sideways when viewed from camera 1, that is, facing directly toward the train doors. In this case, these people are assumed to be passengers, and therefore will be subject to an alert. On the other hand, person C has his body facing forward when viewed from camera 1, that is, facing directly toward the direction of train travel. In this case, this person is assumed to be a station staff member, and therefore will not be subject to an alert. This makes it possible to exclude station staff from the targets of alert events.

[0049] When the train is located inside a station, this includes not only when the train is stopped on the track along the platform, but also when the train is running on the track along the platform, i.e., when the train is decelerating to stop at the station, or when the train is accelerating after leaving the station. When the train is located between stations (outside a station), this includes not only when the train is running normally on the track between stations, but also when the train is slowing down or stopping based on instructions from traffic lights, etc.

[0050] In the train position determination process, attention is paid to signs on the platform, specifically, tactile paving blocks (blocks to guide the visually impaired) installed along the edge of the platform, lines painted in a specified color (e.g., white, yellow, or orange) along the edge of the platform, and stripes painted in a specified color (e.g., red) on the edge of the platform, and the train position is determined depending on whether these signs can be recognized from the camera image.

[0051] On the other hand, platform doors may be installed on platforms to prevent passengers from falling off the platform or coming into contact with the train. In such cases, the train's location can be determined based on whether the platform doors can be recognized from camera images.

[0052] Next, the detection area set on the camera image will be described. Fig. 5 is an explanatory diagram showing the detection area set on the camera image.

[0053] In this embodiment, an area on the platform where passengers (passengers getting on and off) can move is set in advance on the camera image as a detection area. Then, depending on whether the platform can be recognized from the image of the detection area in the camera image, it is determined whether the current position of the train is inside the station or between stations (outside the station) (train position determination process). Furthermore, an alert event is detected based on the image of the detection area in the camera image (alert event detection process).

[0054] In this embodiment, it is determined whether the track on which each car of the train is located is straight or curved (track shape determination process), and a detection area is set based on the determination result (detection area setting process).The detection area is a strip-shaped area of ​​a predetermined width from the end of the platform.

[0055] Specifically, as shown in FIG. 5(A), when the train's track is straight, the detection area is set to a normal width. On the other hand, as shown in FIG. 5(B), when the train's track is curved, a large gap is created between the front end of the platform and the train's body, so the detection area is set to a wider width than when the track is straight. This allows the detection area to include the same range as when the track is straight, even when the train's track is curved. Note that the width of the detection area when the train's track is curved may be expanded by a predetermined expansion factor (for example, 1.5 times) based on the expected curvature of the curve.

[0056] Next, a description will be given of the general configuration of the image analysis server 2 and the train management server 5. Fig. 6 is a block diagram showing the general configuration of the image analysis server 2 and the train management server 5.

[0057] The image analysis server 2 includes an in-train communication unit 21, an out-train communication unit 22, a storage unit 23, and a processor 24.

[0058] The in-train communication unit 21 communicates with the cameras 1 installed in each carriage (car) of the train and the monitoring terminal 3 installed in the crew cabin.

[0059] The train-external communication unit 22 communicates with the train management server 5 via the wireless base station 4 and the network.

[0060] The storage unit 23 stores programs and the like to be executed by the processor 24. The storage unit 23 also stores camera images and information on the analysis results thereof.

[0061] The processor 24 performs various processes by executing programs stored in the storage unit 23. In this embodiment, the processor 24 performs display processing, track shape determination processing, detection area setting processing, train position determination processing, warning event detection processing, alarm generation determination processing, alarm generation control processing, entrapment detection processing, setting processing, and the like.

[0062] In the display process, the processor 24 acquires real-time camera images from each camera and displays the camera images on the monitoring terminal 3. At this time, the camera images are displayed on the monitoring screen based on setting information related to the display layout. Note that the camera images may not be displayed on the monitoring screen while the train is running. In other words, the camera images may be displayed on the monitoring screen only when the train is stopped at the platform. This allows the crew to use the camera images on the monitoring screen to check safety before departure.

[0063] In the track shape determination process, processor 24 determines whether the track on which each vehicle of the train is located is straight or curved based on the camera images. At this time, the determination is made based on the status of the vehicles before and after captured in the camera images. Specifically, if a vehicle that has left camera 1 is captured in the left camera image but not in the right camera image, the track is determined to be a left curve. On the other hand, if a vehicle that has left camera 1 is not captured in the left camera image but is captured in the right camera image, the track is determined to be a right curve. Furthermore, if a vehicle that has left camera 1 is captured in both the left and right camera images, the track is determined to be straight. Note that the determination of whether the track is straight or curved may also be made based on location information and map information obtained using a satellite positioning system such as GPS.

[0064] In the detection area setting process, processor 24 sets a detection area for the camera image based on the determination result of the track shape determination process. Specifically, if the track on which the vehicle runs is straight, the detection area is set to a normal width. On the other hand, if the track on which the vehicle runs is curved, a large gap will be created between the end of the platform and the body of the train, so the width of the detection area is set to be larger than when the track is straight (see FIG. 5).

[0065] In the train position determination process, the processor 24 determines whether the train position (current position of the train) is inside the station or between stations (outside the station) based on the image of the detection area in the camera image. Specifically, if the platform can be recognized from the image of the detection area, the current position of the train is determined to be inside the station. On the other hand, if the platform cannot be recognized from the image of the detection area, the current position of the train is determined to be between stations. Furthermore, the recognition of the platform in the image of the detection area can be performed based on signs (such as braille blocks or white lines) provided on the platform. The train position may also be determined based on location information and map information acquired using a satellite positioning system such as GPS.

[0066] In the alert event detection process, the processor 24 detects an alert event based on the image of the detection area in the camera image, the determination result of the train position determination process, and the setting information related to the alert event. In this embodiment, the event to be detected as an alert event is changed depending on the determination result of the train position determination process, i.e., whether the train position (current position of the train) is inside the station (a position along the platform) or between stations (outside the station).

[0067] That is, when the train position (current position of the train) is within a station, the system detects, as alert events, persons requiring caution, specifically, persons rapidly approaching the train (for example, a person rushing to board the train), and persons requiring consideration, specifically, persons who need to ensure time to board and disembark safely (for example, persons using wheelchairs, strollers, and white canes). On the other hand, when the train position is between stations (outside a station), the system detects, as alert events, abnormalities in train accessories (for example, open doors, broken or fallen windows), and persons disembarking. Furthermore, the alert event detection process can use an image recognition model (machine learning model) built using machine learning such as deep learning. In this case, camera images are input into the image recognition model, and the alert event detection results output from the image recognition model are obtained.

[0068] In the alert determination process, the processor 24 determines whether or not an alert should be issued for the detected alert event based on the detection result of the alert event detection process and the setting information related to the alert target. That is, whether or not an alert should be issued is determined depending on whether or not the state of the object related to the alert event detected in the alert event detection process is set as an alert target. Specifically, the orientation of the body (object) of a person related to the detected alert event is detected, and whether or not an alert should be issued is determined based on the orientation of the person's body. For example, if the orientation of the person's body does not correspond to the body orientation of a person that is a preset alert target, the person is excluded from the alert target.

[0069] In the alarm issuance control process, when it is determined in the alarm issuance determination process that an alarm should be issued, the processor 24 causes the monitoring terminal 3 to perform an alarm operation. That is, the monitoring terminal 3 notifies the train crew (driver or conductor) of the occurrence of an alert event as alarm information. Specifically, a camera image related to the detected alert event is highlighted on the monitoring screen displayed on the monitor provided in the monitoring terminal 3, and an alert sound is output from the speaker provided in the monitoring terminal 3. In addition, the monitoring terminal 6 in the monitoring center (operation control center) notifies staff of the occurrence of an alert event as alarm information. At this time, the monitoring screen displayed on the monitor provided in the monitoring terminal 6 displays the occurrence status of the alert event.

[0070] In the entrapment detection process, the processor 24 detects whether a foreign object (such as a passenger's body or belongings) has been caught in the door based on the camera image. The entrapment detection process can use an image recognition model (machine learning model) constructed by machine learning such as deep learning. In this case, the camera image is input to the image recognition model, and the entrapment detection result output from the image recognition model is obtained.

[0071] In the setting process, the processor 24 acquires the details of operations performed by the administrator on the setting screen displayed on the monitoring terminal 3 as setting information and stores it in the storage unit 23. Specifically, in accordance with the operations of the administrator, setting information regarding the display layout of camera images on the monitoring screen, setting information regarding alert events to be detected, and setting information regarding the state of objects related to alert events to be alerted is acquired.

[0072] The train management server 5 includes a communication unit 51, a storage unit 52, and a processor 53.

[0073] The communication unit 51 communicates with the image analysis server 2 and the monitoring terminal 6 via the network.

[0074] The storage unit 52 stores programs and the like to be executed by the processor 53. The storage unit 52 also stores camera images acquired from the image analysis server 2 and information on the analysis results thereof.

[0075] The processor 53 performs various processes by executing programs stored in the storage unit 52. In this embodiment, the processor 53 performs analysis processing, display processing, and the like.

[0076] In the analysis process, the processor 53 generates information representing the alert status regarding warning events for each train, each vehicle (car) that makes up the train, and each line, based on the camera images obtained from the image analysis server 2 and the analysis results thereof.

[0077] Specifically, the alarm status for each vehicle can be obtained by integrating the alarm status for each camera image taken by multiple cameras 1 installed in each vehicle (car) of the train. Furthermore, the alarm status for each train can be obtained by integrating the alarm status for each multiple vehicles. Furthermore, the alarm status for each line can be obtained by integrating the alarm status for each multiple trains running on the same line.

[0078] In the display process, the processor 53 generates various monitoring screens for presenting the alert status of an alert event to a staff member, specifically, a route map screen 201 (see FIG. 15), a route detail screen 211 (see FIG. 16), a status display screen by route and station 221 (see FIG. 17), and a status display screen by week and car number 231 (see FIG. 18), based on the camera images and their analysis results acquired from the image analysis server 2 and the information acquired in the analysis process. These monitoring screens are distributed to the monitoring terminal 6.

[0079] Next, the monitoring screen displayed on the monitoring terminal 3 in the train will be described. Figures 7 and 8 are explanatory diagrams showing the monitoring screen displayed on the monitoring terminal 3 in the train. Figure 9 is an explanatory diagram showing the main part of the monitoring screen.

[0080] The monitoring terminal 3 in the train displays monitoring screens 101, 111, 121, and 131 that are viewed by the crew (driver). The monitoring screens 101, 111, 121, and 131 display real-time camera images acquired from each camera 1 based on setting information related to the display layout of the camera images.

[0081] The monitoring screen also has a mode display section 102. The monitoring screen displays a camera image of the side on which the door is to be opened, and the mode display section 102 displays which of the left and right doors is currently open.

[0082] Here, the example shown in FIG. 7 is a case where all camera images on the side where the door opens when the train stops are displayed. Also, the display layout of camera images is set to be reversed for upbound and downbound trains. The example shown in FIG. 7(A) is a case where the driver is on duty in car No. 1 on a downbound train, and the monitoring screen displayed on the monitoring terminal 3 installed in this car No. 1 displays camera images in the order of car No. 1, car No. 2, car No. 3, and car No. 4. On the other hand, the example shown in FIG. 7(B) is a case where the driver is on duty in car No. 4 on an upbound train, and the monitoring screen displayed on the monitoring terminal 3 installed in this car No. 4 displays camera images in the order of car No. 4, car No. 3, car No. 2, and car No. 1.

[0083] The example shown in Figure 8 is a case where, of the camera images on the side where the doors open when the train stops, only the camera image from the car farthest from the crew (driver) is displayed. The example shown in Figure 8(A) is a case where the driver is on duty in car No. 1 on a down train, and the monitoring screen 121 displayed on the monitoring terminal 3 installed in this car No. 1 displays only the camera image from car No. 4, which is the farthest from the driver. On the other hand, the example shown in Figure 8(B) is a case where the driver is on duty in car No. 4 on an up train, and the monitoring screen 131 displayed on the monitoring terminal 3 installed in this car No. 4 displays only the camera image from car No. 1, which is the farthest from the driver.

[0084] Furthermore, on the monitoring screen, when an alert event is detected and the alert event satisfies the conditions for issuing an alert, the camera image in which the alert event occurred is highlighted as a display action to alert (warn) the crew. Specifically, a frame 103 of a predetermined color (e.g., red) is displayed around the camera image. Furthermore, in a message display field 104 below the camera image, text indicating the details of the alert event (e.g., "white cane, wheelchair") is displayed, with the background displayed in a predetermined color (e.g., red). This prevents the alert display from being superimposed on the camera image and from interfering with the staff's visual confirmation of the actual situation using the camera image.

[0085] In the example shown in Figures 7(A) and (B), an alert event occurs in car 4, and the camera image in car 4 is highlighted. In the example shown in Figures 8(A) and (B), an alert event also occurs in car 4. As shown in Figure 8(A), the camera image in car 4 is highlighted on the monitoring screen displayed on monitoring terminal 3 in car 1, where the driver is on duty, for outbound trains. On the other hand, as shown in Figure 8(B), the camera image in car 4, where the driver is on duty, only displays the camera image in car 1, and therefore is not highlighted, for inbound trains.

[0086] Here, the highlighting of the camera image ends when the alert event is resolved. That is, when an alert event that meets the alert condition occurs, the state transitions from the initial state shown in FIG. 9(A) to the state shown in FIG. 9(B), and when the alert event is resolved, the state transitions to the state shown in FIG. 9(C). In this state, the frame 103 displayed around the camera image disappears, and text indicating that the alert event has been resolved is displayed in the message display field 104, with the background displayed in a predetermined color (e.g., blue). In the example shown in FIG. 9, a person using a white cane or wheelchair is detected as an alert event, and the alert event is resolved when the person successfully boards or disembarks, and text indicating that the doors can be safely closed (e.g., "Doors Closed OK") is displayed in the message display field 104.

[0087] Next, a description will be given of the setting screen displayed on the monitoring terminal 3 in the train. Figures 10, 11, 12 and 13 are explanatory diagrams showing the setting screen.

[0088] The monitoring terminal 3 inside the train displays the following setting screens: train display setting screens 151, 161 (see Figures 10 and 11), station display setting screen 171 (see Figure 12), and event detection setting screens 181, 191 (see Figures 13(A) and (B)).

[0089] These setting screens are provided with tabs 141, 142, and 143 for train display settings, station display settings, and event detection settings. When the administrator operates the train display settings tab 141, train display setting screens 151 and 161 (see FIGS. 10 and 11) are displayed. When the administrator operates the station display settings tab 142, station display setting screen 171 (see FIG. 12) is displayed. When the administrator operates the event detection settings tab 143, event detection setting screens 181 and 191 (see FIGS. 13(A) and (B)) are displayed.

[0090] The train display setting screens 151 and 161 shown in FIGS. 10 and 11 are operated by an administrator to set the display layout of the camera images to be displayed on the monitoring screen of the monitoring terminal 3.

[0091] A train display setting screen 151 according to the first example shown in FIG. 10 is provided with a camera image selection section 152 and a display layout setting section 153.

[0092] In the camera image selection unit 152, camera images taken by the cameras 1 installed in each vehicle are displayed in accordance with the position of each camera 1. Specifically, the camera images taken by the cameras 1 installed on the left and right sides of the vehicle are arranged as seen from above the vehicle.

[0093] The display layout setting section 153 is provided with a camera image arrangement section 154, a setting mode designation section 155, and an inverted display designation section 156.

[0094] The camera image layout section 154 corresponds to the display layout of the camera images that are actually displayed on the monitoring screen. The administrator can allocate camera images to the monitoring screen by arranging the camera images selected in the camera image selection section 152 in the camera image layout section 154 using a drag-and-drop operation. To delete an allocated camera image, the administrator simply selects the camera image and then performs a delete operation, for example, right-clicks and selects delete from a pop-up menu (not shown). If there are not enough image frames in the camera image layout section 154, an image frame can be added together with a scroll button by operating an add button (not shown).

[0095] The setting mode designation unit 155 is used by the administrator to switch between a setting mode for configuring settings related to the monitoring screen that is displayed when the left door is open and a setting mode for configuring settings related to the monitoring screen that is displayed when the right door is open. The monitoring screen displays a camera image of the side of the door that is opened, and the display layout of the camera image is set depending on whether the left or right door is opened. The setting mode designation unit 155 is provided with a button 155a for designating the setting mode when the left door is open and a button 155b for designating the setting mode when the right door is open. The administrator can switch between the setting modes by operating either button 155a or 155b.

[0096] The reverse display designation unit 156 allows the administrator to designate whether or not the display layout of camera images should be reversed for uphill and downhill journeys. The vehicles in which the driver is on duty are different for uphill and downhill journeys, and if the display layout of camera images is reversed for uphill and downhill journeys, the camera images will be displayed in order from the vehicle closest to or farthest from the driver (see FIGS. 7(A) and (B)). The reverse display designation unit 156 is provided with a button 156a for designating a setting that leaves the display unchanged for uphill and downhill journeys, and a button 156b for designating a setting that reverses the display layout for uphill and downhill journeys. The administrator can designate whether or not the display layout of camera images should be reversed for uphill and downhill journeys by operating either button 156a or 156b.

[0097] When the setting operation for the display layout of the camera images to be displayed on the monitoring screen is completed on the train display setting screens 151, 161, the administrator operates the setting save button 157. This causes the contents of the operation performed by the administrator to be saved as setting information. Note that the setting information can be managed by linking the positions of the left and right open / close doors at each stop on the train's route with the monitoring screen display layout.

[0098] The train display setting screen 161 according to the second example shown in Fig. 11 is provided with a camera image selection section 162 and a display layout setting section 153. The display layout setting section 153 is similar to the train display setting screen 151 according to the first example (see Fig. 10).

[0099] The camera images of each vehicle are displayed in order in the camera image selection section 162. The administrator can operate the scroll button to display the camera images of all vehicles.

[0100] Here, if all camera images on the side where the doors open when the train stops are to be displayed on the monitoring screen (see Figure 7), the administrator simply assigns the camera images to the monitoring screen on the train display setting screens 151, 161 (see Figures 10 and 11), that is, performs a drag-and-drop operation on all camera images to place the camera images in the camera image selection section 152 in the camera image placement section 154.

[0101] On the other hand, if only the camera image of the car farthest from the driver among the camera images on the side where the door opens when the train stops is to be displayed on the monitoring screen (see FIG. 8), the administrator need only perform an operation on the train display setting screens 151, 161 (see FIGS. 10 and 11) to assign the camera image to the monitoring screen, that is, a drag-and-drop operation to place the camera image in the camera image selection unit 152 in the camera image placement unit 154, for only the relevant camera image. For example, when a train is operating in the outbound direction, for the monitoring terminal 3 of car No. 1, which is the lead car, only the camera image of car No. 4, which is the last car, can be assigned to the monitoring screen.

[0102] Here, when the administrator operates the setting save button 157, a confirmation screen (dialog box) may pop up with a confirmation message such as "Is it okay to only display the camera image from car No. 4?"

[0103] In this embodiment, the administrator performs the operation to assign camera images to the monitoring screen, but the image analysis server 2 may perform the process of assigning camera images to the monitoring screen based on predetermined rules and conditions specified by the administrator (such as the order in which camera images are arranged). For example, a setting mode may be provided in which all camera images are assigned to the monitoring screen, and a setting mode in which only the camera image farthest from the driver is assigned to the monitoring screen, and the administrator may be allowed to select the setting mode, so that the process of assigning camera images to the monitoring screen is performed according to the selected setting mode.

[0104] Similar to the train display setting screens 151 and 161 (see FIGS. 10 and 11), the station display setting screen 171 shown in Fig. 12 is operated by an administrator to set the display layout of camera images to be displayed on the monitoring screen of the monitoring terminal 3. On the other hand, the train display setting screens 151 and 161 (see FIGS. 10 and 11) are used to individually set the display layout of the monitoring screen of the monitoring terminal 3 of each train, but the station display setting screen 171 is set collectively for the monitoring terminals 3 of all trains passing through the station.

[0105] The side on which the doors open when a train stops at a station is determined for each station. Therefore, it is possible to set for each station whether the camera images from the left or right side are to be displayed on the monitoring screen of the monitoring terminal 3. For this reason, the station display setting screen 171 is configured so that the display layout of the camera images to be displayed on the monitoring screen of the monitoring terminal 3 can be set for each station. Specifically, a camera image setting field 172 is provided for each station regarding the camera images of each car (cars 1 to 4), and the administrator can switch between displaying and hiding the camera images by operating the camera image setting field 172.

[0106] The station display setting screen 171 is also provided with a down setting section 173 and an up setting section 174. The down setting section 173 is used to set the display layout of camera images to be displayed on the monitoring screen when a train is heading down. The up setting section 174 is used to set the display layout of camera images to be displayed on the monitoring screen when a train is heading up. This allows the display layout of camera images to be displayed on the monitoring screen to be different for down and up trains.

[0107] In the example shown in Figure 12, at Station A, there are no station staff on the platform to check the safety of all the cars (cars 1 to 4), so the crew (driver) must check the safety, and the camera images are set to be displayed on the monitoring screen.

[0108] Additionally, at Station B, there are station staff on the platform who check the safety of all cars (cars 1 to 4), so there is no need for the crew (driver) to check for safety, and therefore the camera images are set not to be displayed on the monitoring screen.

[0109] Furthermore, at Station C, some cars (car number 1) are parked away from the platform, and the doors of some of these cars are not open, so there is no need for the crew (driver) to check for safety, and so the settings are such that camera images are not displayed on the monitoring screen.On the other hand, for the remaining cars (car numbers 2 to 4), there is no station staff on the platform to check for safety, so the crew (driver) must check for safety, and so the settings are such that camera images are displayed on the monitoring screen.

[0110] When the operation of setting the display layout of the camera images to be displayed on the monitoring screen on the station display setting screen 171 is completed, the administrator operates the setting save button 175. This causes the contents of the operation performed by the administrator to be saved as setting information.

[0111] Here, when the camera image setting field 172 is set to display camera images, the display layout of the camera images to be displayed on the monitoring screen may be set individually. In this case, when the administrator operates the display layout setting field 176, the screen transitions to a setting screen (not shown) similar to the train display setting screens 151, 161 (see FIGS. 10 and 11), and the display layout of the camera images can be set on this setting screen. Note that this setting screen may be provided with a back button in addition to a settings save button, and the administrator may operate the back button to return to the station display setting screen 171.

[0112] 13(A) is operated by the administrator to set events to be subject to alert event detection depending on whether the train position (current position of the train) is within the station (position along the platform) or between stations (outside the station). The event detection setting screen 181 is provided with a target setting field 182, and the administrator can switch whether each event is subject to alert event detection or not by operating the target setting field 182.

[0113] Here, a person requiring caution is a person approaching the train, such as a person rushing to board the train, if the train is located within a station, or a person getting off the train if the train is located between stations. Furthermore, a person requiring special consideration is a person who needs time to board or disembark safely, such as a person using a wheelchair, stroller, or white cane, if the train is located within a station. Furthermore, abnormalities in train accessories include, for example, an open door or a broken or fallen window.

[0114] When the setting of the events to be detected as alert events is completed on the event detection setting screen 181, the administrator operates the setting save button 184. This causes the contents of the operation by the administrator to be saved as setting information.

[0115] Furthermore, on the event detection setting screen 181, when the administrator operates the train position display field 183, which indicates whether the current location of the train is inside the station (platform) or between stations (outside the station), the screen transitions to the event detection setting screen 191 shown in Figure 13 (B).

[0116] The event detection setting screen 191 shown in Fig. 13(B) is operated by the administrator to set whether or not the state of an object (a person, an accessory object to a train, etc.) related to an event that has been set as the target of alert event detection on the event detection setting screen 181 (see Fig. 13(A)) is to be the target of alert generation. The event detection setting screen 191 is provided with a target setting field 192, and the administrator can switch between whether or not each state of an object related to each event is to be the target of alert generation by operating the target setting field 192.

[0117] The example shown in Figure 13(B) is a case where the administrator operates the train position display field 183, which indicates that the current location of the train is within a station, to display the event detection setting screen 191. In this case, since persons requiring special attention and persons requiring consideration are set as targets, it is possible to operate the settings related to the status, but since abnormalities in train accessories are set as excluded from the targets, it is not possible to operate the settings related to the status. On the other hand, when the administrator operates the train position display field 183, which indicates that the current location of the train is within a station, to display the event detection setting screen 191, it is possible to operate the settings related to the status, since abnormalities in train accessories are set as targets.

[0118] When the administrator has completed setting the conditions that will be the targets for alerting on the event detection setting screen 191, the administrator operates the setting save button 193. This causes the contents of the administrator's operations to be saved as setting information. Furthermore, when the administrator operates the back button 194, the screen returns to the event detection setting screen 181 (see FIG. 13(A)).

[0119] Here, on the event detection setting screen 191 (see FIG. 13(B)), settings can be made for each of the following states for suspicious persons: facing sideways, facing forward, and crouching. Of these, the crouching state is set as an exception because it is assumed that the person remains stationary and therefore the risk caused by train movement is low. Furthermore, when a person is facing forward, i.e., facing the front or rear of the train, as in the example shown in FIG. 3(A), they are assumed to be station staff, so the front state is set as an exception.

[0120] Next, we will explain the procedure of the process performed by the image analysis server 2. Fig. 14 is a flow diagram showing the procedure of the process performed by the image analysis server 2. Note that this flow is executed for each camera image. Furthermore, this flow is executed repeatedly at a predetermined cycle.

[0121] As shown in FIG. 14(A), the image analysis server 2 first determines whether the vehicle's road corresponding to the camera image is straight or not based on the camera image (road shape determination process) (ST101).

[0122] If the road is straight (Yes in ST101), the detection area is set to the normal width (ST102). On the other hand, if the road is curved (No in ST101), the detection area is set to an expanded width (detection area setting process) (ST103).

[0123] Next, the image analysis server 2 cuts out an image of the detection area from the camera image, and determines whether the current position of the train is inside a station or not based on the image of the detection area (train position determination process) (ST104).

[0124] Next, the image analysis server 2 detects a preset alert event according to the train position determination result based on the image of the detection area (alert event detection process) (ST105).

[0125] Next, as shown in FIG. 14(B), the image analysis server 2 performs an alarm determination process.

[0126] In this alarm determination process, first, the image analysis server 2 determines whether or not a warning event has been detected (ST111).

[0127] If an alert event has been detected (Yes in ST111), the image analysis server 2 then determines whether the detected alert event satisfies the alert conditions (detailed conditions for alert generation) (ST112). The alert generation condition is that the state of the object related to the detected alert event is set as an alert generation target (see FIG. 13(B)). If the state of the object related to the detected alert event is not set as an alert generation target, the alert generation condition is not met.

[0128] Here, if the detected alert event satisfies the alert condition (Yes in ST112), the image analysis server 2 issues an alert to notify the crew (driver) of the occurrence of the alert event. Specifically, a process is performed to highlight the camera image in which the detected alert event occurred on the monitoring screen of the monitoring terminal 3 (see FIGS. 7 and 8).

[0129] On the other hand, if no alert event has been detected (No in ST111) or if the detected alert event does not meet the alert conditions (No in ST112), the image analysis server 2 does not issue an alert to notify the crew (driver) of the occurrence of an alert event. Also, if an alert to notify the crew (driver) of the occurrence of an alert event was issued in the previous processing and the alert event has been resolved in the current processing, the image analysis server 2 will notify the crew (driver) that it is safe to close the doors.

[0130] Next, a description will be given of the monitoring screen displayed on the monitoring terminal 6 of the monitoring center. Figures 15, 16, 17 and 18 are explanatory diagrams showing the monitoring screen displayed on the monitoring terminal 6 of the monitoring center.

[0131] When the monitoring terminal 6 at the monitoring center accesses the train management server 5, various monitoring screens are displayed, specifically, a route map screen 201 (see Figure 15), a route detail screen 211 (see Figure 16), a status display screen by route and station 221 (see Figure 17), and a status display screen by week and car number 231 (see Figure 18).

[0132] Furthermore, each screen (see FIGS. 16, 17, and 18) except for the route map screen 201 (initial screen) is provided with a back button 219. When an attendant operates the back button 219, the screen returns to the original screen. For example, when an attendant operates the back button 219 on the route details screen 211 (see FIG. 16), the screen returns to the route map screen 201 (see FIG. 15), which is the original screen.

[0133] When the monitoring terminal 6 accesses the train management server 5, a route map screen 201 (see FIG. 15) is first displayed. The route map screen 201 presents the staff with the occurrence status of warning events (already issued) for each route for all routes that are the subject of monitoring.

[0134] A route map 202 is displayed on the route map screen 201. On the route map 202, each route is drawn in a color corresponding to the alert level (warning, caution, good).

[0135] Furthermore, on the route map screen 201, the current date and time is displayed in the date and time display section 203.

[0136] Furthermore, an information display section 204 for each route is provided on the route map screen 201. The information display section 204 displays the number of alert events (alerts issued) by alert level (warning, caution) and the number of unaddressed alert events, i.e., alert events that have not yet been checked by an attendant.

[0137] The route map screen 201 also has a history list display section 206. The history list display section 206 displays a list of information about each alert event that has been issued in the past. Here, the information about the alert event includes the date and time, the route name, the train number, the camera number, and the analysis result. The analysis result displays the content of the alert event and the alert level. This allows staff to check the alert events that have been issued in the past on all routes.

[0138] The information display section 204 of the route map screen 201 is provided with a route button 205. When the attendant operates the route button 205 to select a route, the screen transitions to a route details screen 211 (see FIG. 16).

[0139] The route details screen 211 (see FIG. 16) presents the status of any alert events that have occurred on the selected route to the attendant. The route details screen 211 displays a route details map 212. The route details map 212 displays station marks 213 and train marks 215.

[0140] The train mark 215 is placed at a position corresponding to the actual position of the train on the track and moves as the train moves. The train mark 215 is drawn in a color corresponding to the alert level (warning, caution). In the example shown in Figure 8, an alert event that changes the alert level to warning has occurred on train #113.

[0141] The route details screen 211 also has a history list display section 216. The history list display section 216 displays a list of information about each alert event that has been issued in the past. Here, the information about the alert event includes the date and time, the train number, the camera number, and the analysis result. The analysis result displays the content of the alert event and the alert level. This allows the staff member to check the alert events that have been issued in the past on the selected route.

[0142] The line details screen 211 also has a line / station status display button 218. When an attendant operates the line / station status display button 218, the screen transitions to a line / station status display screen 221 (see FIG. 17).

[0143] The line-by-line / station-by-station status display screen 221 (see FIG. 17) presents to the attendant the status of warning events that have been previously issued on each line and at each station. The line-by-line / station-by-station status display screen 221 is provided with graph display sections 222 and 223.

[0144] The graph display section 222 displays a graph showing the number of alerts issued for each route. The number of alerts is a cumulative total for a predetermined period in the past (for example, the most recent week). This allows staff to understand the status of alerts issued in the past on each route during the predetermined period in the past.

[0145] The graph display section 223 displays a graph showing the number of alerts issued for alert events at each station on the specified line. The number of alerts is a cumulative total for a predetermined period in the past (for example, the most recent week). This allows staff to understand the status of alert events that have been issued in the past at each station during the predetermined period in the past.

[0146] When an attendant operates the train mark 215 to select a train on the route map screen 201 (see FIG. 15), the screen transitions to a weekly / car status display screen 231 (see FIG. 18).

[0147] The weekly / car status display screen 231 (see FIG. 18) presents the status of warning events that have been issued in the past for each week and each car to the staff member. The weekly / car status display screen 231 is provided with graph display sections 232 and 233.

[0148] The graph display section 232 displays a graph showing the number of alerts issued for a specified train per week. The number of alerts is a cumulative total for a specified period in the past (for example, the most recent week). This allows staff to understand the status of alerts issued in the past for each week in the specified period in the past.

[0149] The graph display unit 233 displays a graph showing the number of alerts issued for each car on the specified train. The number of alerts is a cumulative total for a predetermined period in the past (for example, the most recent week). This allows staff to understand the status of alerts issued in each car in the past during the predetermined period in the past.

[0150] As described above, the embodiments have been described as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made. Furthermore, it is also possible to combine the components described in the above embodiments to create new embodiments. [Industrial Applicability]

[0151] From the perspective of ensuring the safety of users, the safety monitoring system and safety monitoring method of the present invention have the effect of being able to effectively utilize cameras installed on the sides of a moving body, such as a train, not only when the moving body is stopped at a boarding and disembarking facility, but also when the moving body is moving between boarding and disembarking facilities, and are useful as a safety monitoring system and safety monitoring method that detects events that may hinder ensuring the safety of users of the moving body as warning events and notifies staff of the occurrence of such warning events. [Explanation of symbols]

[0152] 1 camera 2. Image analysis server (server device) 3 Monitoring terminal (terminal device) 4. Wireless base stations 5 Train management server (server device) 6 Monitoring terminal (terminal device)

Claims

1. A safety monitoring system that detects the occurrence of foreign objects being caught in doors and warning events that may hinder the safety of users of a moving body having doors for boarding and disembarking people, and generates alarm information including the detection results, a camera provided corresponding to the door; a server device that detects the occurrence of the pinching and the alert event based on a camera image taken by the camera and generates the alert information; a terminal device that acquires the alert information and notifies a staff member of the occurrence of the alert event; The server device When the moving body is stopped at a boarding / deboarding facility, the occurrence of the pinching is detected based on the camera image; Furthermore, based on the camera image, it is determined whether the current position of the moving object is inside or outside the boarding / disembarking facility; A safety monitoring system characterized by detecting the alert events that have been set in advance for both inside and outside the boarding and disembarking facility based on the judgment results, based on the camera images.

2. The server device The safety monitoring system described in claim 1, characterized in that when the mobile object is located within the boarding and disembarking facility, the warning event is detected to be at least one of a person rapidly approaching a train and a person who needs to ensure time for safe boarding and disembarking.

3. The server device The safety monitoring system described in claim 1, characterized in that when the mobile body is located outside the boarding and disembarking facility, the warning event is detected as at least one of an abnormality in the mobile body's accessories and a person disembarking from the mobile body.

4. The server device 2. The safety monitoring system according to claim 1, wherein the state of an object related to the alert event is detected, and whether or not an alarm needs to be issued is determined based on the state of the object.

5. The terminal device Displaying a monitoring screen including a plurality of camera images; The server device 2. The safety monitoring system according to claim 1, wherein the display layout of the plurality of camera images on the monitoring screen is changed depending on the state of the moving object.

6. A safety monitoring method for detecting the occurrence of a foreign object being caught in a door of a moving body having a door for passengers to board and disembark, and a warning event that may cause an impediment to ensuring the safety of users of the moving body, and for causing an information processing device to perform processing for generating alarm information including the detection results, acquiring a camera image taken by a camera provided corresponding to the door; When the moving body is stopped at a boarding / deboarding facility, the occurrence of the pinching is detected based on the camera image; Furthermore, based on the camera image, it is determined whether the current position of the moving object is inside or outside the boarding / disembarking facility; A safety monitoring method characterized by detecting the alert events that have been set in advance for the inside and outside of the boarding and disembarking facility based on the judgment result, based on the camera image.

Images