Monitor system, monitor device, monitor method, program

Abstract

To provide a technique which is capable of monitoring situations of people on a platform according to attributes of the people on the station platform.SOLUTION: A monitor system 1 comprises an imaging part 10 which images a platform PF of a station ST, a detection part 20 which detects people on the platform PF on the basis of a picture image which is acquired by the imaging part 10, a classification part 30 which classifies people detected by the detection part 20 into one of a plurality of pre-stipulated types, and a determination part 40 which determines whether safety properties of people detected by the detection part 20 are in a relatively high first state or a relatively low second state on the basis of a prescribed reference. The prescribed reference is regulated so that classification results obtained by the classification part 30 are different according to whether with "station staff" or other types of attributes that are different from "station staff".SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to monitoring systems and the like. [Background technology]

[0002] BACKGROUND ART Conventionally, there is known a technique for using an imaging unit (camera) to acquire an image of a station platform and monitor the situation of people on the platform (see, for example, Patent Documents 1 to 3). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-69047 [Patent Document 2] Japanese Patent Publication No. 2022-95316 [Patent Document 3] Japanese Patent Publication No. 2020-207571 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, it is desirable to monitor the status of people on the station platform in accordance with the attributes of the people present on the platform.

[0005] In view of the above-mentioned problems, an object of the present invention is to provide a technology that can monitor the status of people on a station platform in accordance with the attributes of the people present on the platform. [Means for solving the problem]

[0006] In order to achieve the above object, in one embodiment of the present disclosure, An imaging unit that captures images of the station platform; a detection unit that detects a person present on the platform based on the image acquired by the imaging unit; a classification unit that classifies the person detected by the detection unit into one type of attribute from among a plurality of types of attributes that are defined in advance; a determination unit that determines whether the safety of the person detected by the detection unit is in a first state where the safety is relatively high or a second state where the safety is relatively low, based on a predetermined standard; the predetermined criterion is defined so that a result of classification by the classifier is different between a specific type of attribute among the plurality of types of attributes and a type of attribute different from the specific type of attribute. A monitoring system is provided.

[0007] In another embodiment of the present disclosure, a detection unit that detects a person present on the platform based on an image acquired by an imaging unit that images the platform of the station; a classification unit that classifies the person detected by the detection unit into one type of attribute from among a plurality of types of attributes that are defined in advance; a determination unit that determines whether the safety of the person detected by the detection unit is in a first state where the safety is relatively high or a second state where the safety is relatively low, based on a predetermined standard; the predetermined criterion is defined so that a result of classification by the classifier is different between a specific type of attribute among the plurality of types of attributes and a type of attribute different from the specific type of attribute. A monitoring device is provided.

[0008] In still another embodiment of the present disclosure, a detection step in which the information processing device detects a person present on the platform based on an image acquired by an imaging unit that images the platform of the station; a classification step in which the information processing device classifies the person detected in the detection step into one type of attribute from among a plurality of types of attributes defined in advance; a determination step in which the information processing device determines, based on a predetermined criterion, whether the safety of the person detected in the detection step is in a first state in which the safety of the person is relatively high or in a second state in which the safety of the person is relatively low; the predetermined criterion is defined so that a classification result in the classification step differs between a specific type of attribute among the plurality of types of attributes and a type of attribute different from the specific type of attribute. A monitoring method is provided.

[0009] In still another embodiment of the present disclosure, In the information processing device, a detection step of detecting a person present on the platform based on an image acquired by an imaging unit that images the platform of the station; a classification step of classifying the person detected in the detection step into one type of attribute from among a plurality of types of attributes defined in advance; a determination step of determining whether the safety of the person detected in the detection step is in a first state in which the safety of the person is relatively high or in a second state in which the safety of the person is relatively low, based on a predetermined criterion; the predetermined criterion is defined so that a classification result in the classification step differs between a specific type of attribute among the plurality of types of attributes and a type of attribute different from the specific type of attribute. Programs are offered. [Effects of the Invention]

[0010] According to the above-described embodiment, the status of people on the station platform can be monitored according to the attributes of the people present on the platform. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating a configuration of a first example of a monitoring system. [Figure 2] 10A and 10B are diagrams illustrating an example of a person detection process performed by a detection unit. [Figure 3] 10A and 10B are diagrams illustrating an example of a person classification process performed by a classification unit. [Figure 4] 10A and 10B are diagrams illustrating an example of a safety determination process performed by a determination unit. [Figure 5] FIG. 3 is a diagram showing a first example of the display content of the display unit. [Figure 6] 10 is a flowchart illustrating an example of a process for monitoring the status of people on a platform by a monitoring system. [Figure 7] 10A and 10B are diagrams illustrating another example of the safety determination process performed by the determination unit. [Figure 8] FIG. 10 is a diagram showing a second example of the display content of the display unit. [Figure 9] FIG. 10 is a diagram illustrating the configuration of a third example of a monitoring system. [Figure 10] FIG. 10 is a diagram showing a third example of the display content of the display unit. [Figure 11] 10 is a flowchart illustrating another example of a process for monitoring the status of people on a platform by the monitoring system. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment will be described with reference to the drawings.

[0013] [First example of a monitoring system] A first example of a monitoring system 1 according to this embodiment will be described with reference to FIGS.

[0014] Fig. 1 is a diagram showing the configuration of a first example of a monitoring system 1. Fig. 2 is a diagram explaining an example of processing by a detection unit 20. Fig. 3 is a diagram explaining an example of processing by a classification unit 30. Fig. 4 is a diagram explaining an example of processing by a determination unit. Fig. 5 is a diagram showing a first example of the display content of a display unit 60.

[0015] The monitoring system 1 is installed at a station ST and monitors the status of the platform PF where the railway vehicle TRN enters and stops.

[0016] The surveillance system 1 includes an imaging unit 10, a detection unit 20, a classification unit 30, a determination unit 40, a notification unit 50, and a display unit 60.

[0017] The imaging unit 10 captures an image of the platform PF and acquires an image showing the state of the platform PF. The imaging unit 10 may also be capable of acquiring an image including not only the state of the platform PF but also the state inside the railway vehicle TRN entering the platform PF. The imaging range of the imaging unit 10 is set, for example, so as to include the entire range in the length direction (longitudinal direction) of the platform PF in which entering railway vehicles TRN are located side by side. The length direction (longitudinal direction) of the platform PF is the direction along the entering railway vehicle TRN when the platform PF is viewed from directly above.

[0018] The imaging unit 10 is, for example, a monocular surveillance camera permanently installed in the platform PF and attached to the ceiling of the platform PF. The imaging unit 10 captures an image showing the latest status of the platform PF every 1 / 30 seconds during the business hours of the station ST, for example, and outputs the image data.

[0019] The monitoring system 1 may be capable of capturing the entire monitored area of ​​the home PF using one imaging unit 10, or may be capable of covering the entire monitored area of ​​the home PF using multiple imaging units 10.

[0020] The imaging unit 10 is communicably connected to the detection unit 20 via a one-to-one communication line, a local area network (LAN), or the like, and the output of the imaging unit 10 (ie, image data) is taken into the detection unit 20.

[0021] The detection unit 20 performs image analysis based on the output (image data) of the imaging unit 10 to detect a person on the platform PF. The detection unit 20 may also track the movement of a person within the imaging range of the imaging unit 10 based on time-series person detection results based on images that are sequentially input. In this case, the detection unit 20 may track the movement of a person across the imaging ranges of multiple imaging units 10. If the imaging range of the imaging unit 10 includes the interior of a railway vehicle TRN, the detection unit 20 may detect a person inside the railway vehicle TRN, or may further track the movement of a person within an area extending from the platform PF to the interior of the railway vehicle TRN. Hereinafter, the process of detecting a person performed by the detection unit 20 and the processes associated with this process may be collectively referred to as "person detection process."

[0022] The detection unit 20 detects people in the home PF by using a known image processing technique at each predetermined processing cycle to recognize people from the latest image input from the imaging unit 10. The detection unit 20 may also estimate the position on the home PF of the people recognized from the image.

[0023] The functions of the detection unit 20 may be realized by any hardware or any combination of hardware and software. For example, the detection unit 20 is primarily configured with a computer (information processing device) including a central processing unit (CPU), a memory device, an auxiliary storage device, an interface device, etc. The memory device may be, for example, a static random access memory (SRAM) or a dynamic random access memory (DRAM). The auxiliary storage device may be, for example, a hard disk drive (HDD), a solid state drive (SSD), a flash memory, or an electrically erasable programmable read-only memory (EEPROM). The interface device may include, for example, a communication interface for communicating with other components of the surveillance system 1, such as the imaging unit 10 and the classification unit 30. The interface device may also include, for example, an external interface for connecting to a recording medium. This allows the detection unit 20 to, for example, retrieve various programs from a recording medium via the external interface and install them in the auxiliary storage device. The information processing device may also include a high-speed arithmetic unit operating in conjunction with the CPU. Examples of high-speed computing devices include a graphics processing unit (GPU), a field-programmable gate array (FPGA), and an application-specific integrated circuit (ASIC).

[0024] For example, the detection unit 20 recognizes a person in an image by applying the trained model LM1 based on local features obtained from the image or the image itself.

[0025] The trained model LM1 is obtained, for example, by machine learning (supervised learning) of the base learning model M1 using a training dataset TRD1. For example, each training data included in the training dataset TRD1 is composed of a combination of image features obtained from an image or the image data itself as input data and data specifying a partial image containing a person in the image as output data. As a result, the trained model LM1 can recognize a person in an image using the image features obtained from the image data or the image data itself as input and output data for a range (e.g., a rectangular range) containing the person in the image. For example, the trained model LM1 is mainly composed of a deep neural network (DNN), and the application of backpropagation (error backpropagation) based on the training data improves the efficiency of DNN machine learning. The trained model LM1 may also include a U-Net capable of recognizing people from image data using image data as input.

[0026] Furthermore, the detection unit 20 may recognize people in an image by applying a rule-based algorithm such as pattern matching based on feature points and local feature amounts of image data.

[0027] 2, the latest image 200 acquired by the imaging unit 10 shows a railway vehicle TRN entering the platform PF adjacent to each other, platform doors PD installed at the platform PF, and a passenger PS and station staff (also referred to as "station attendant") SA standing at the platform PF. In this example, the detection unit 20 recognizes the passenger PS and station staff SA as people from the latest image 200 acquired by the imaging unit 10, and outputs data on partial images of rectangular areas AR1 and AR2 that include the passenger PS and station staff SA, respectively.

[0028] In this specification, the term "passengers" is used to include all passengers who plan to board the railway vehicle TRN, passengers currently on board the railway vehicle TRN, and passengers who have boarded the railway vehicle TRN and then disembarked.

[0029] Furthermore, the detection unit 20 may apply a known object tracking method to track the movement path (movement trajectory) of a person detected on an image, thereby enabling the detection unit 20 to output not only the position of the person on the home PF in the current (latest) image, but also the history of the person's past positions on the home PF (i.e., movement trajectory) as a tracking result.

[0030] For example, the detection unit 20 applies MOT (Multi Object Tracking) to track the movement of a person detected on an image. Alternatively, the detection unit 20 may apply an optical flow method to track the movement of a person detected on an image.

[0031] The detection unit 20 is communicatively connected to the classification unit 30 via a one-to-one communication line or a local network, and the output of the detection unit 20, including information regarding the results of the person detection process, is taken into the classification unit 30.

[0032] The information about the result of the person detection processing may include, for example, information indicating whether a person has been detected. The information about the result of the person detection processing may also include information indicating an area (e.g., a rectangular area) containing a person in an image when a person is detected by the detection unit 20. The information about the result of the person detection processing may also include data on a partial image of the area containing the person detected by the detection unit 20. The information about the result of the person detection processing may also include, instead of the partial image data, original data acquired by the imaging unit 10 together with information indicating an area (e.g., a rectangular area) containing a person in the image. The information about the result of the person detection processing may also include information indicating the position of the detected person on the home PF. The output of the detection unit 20 may also include information (e.g., time-series position information, etc.) indicating the tracking result of the person detected within the imaging range of the imaging unit 10.

[0033] Note that, when the detection unit 20 does not detect (recognize) a person in the latest image acquired from the imaging unit 10, it does not have to output information about the result of the person detection processing to the classification unit 30. Furthermore, when the detection unit 20 does not detect (recognize) a person in the latest image acquired from the imaging unit 10, it may output information about the result of the person detection processing, specifically, information indicating that no person was detected, to the determination unit 40 or the notification unit 50.

[0034] The classification unit 30 classifies the person detected by the detection unit 20 based on the output of the detection unit 20. Specifically, the classification unit 30 classifies the person detected by the detection unit 20 into one of a plurality of predefined attribute types based on data of a partial image including the person detected by the detection unit 20. The data of the partial image including the person detected by the detection unit 20 may be input to the classification unit 30 as output data of the detection unit 20, or may be obtained by cutting out the data from image data acquired by the imaging unit 10 based on the output data of the detection unit 20. In the latter case, the latest image data acquired by the imaging unit 10 may be input to the classification unit 30 as output data of the detection unit 20, or may be directly input from the imaging unit 10 to the classification unit 30 when the imaging unit 10 and the classification unit 30 are communicatively connected via a one-to-one communication line, a local network, or the like. Hereinafter, the process of classifying the person detected by the detection unit 20, which is performed by the classification unit 30, may be simply referred to as a "person classification process."

[0035] The classification unit 30 performs a person classification process when the output of the detection unit 20 is input and the output of the detection unit 20 indicates that a person has been detected. On the other hand, the classification unit 30 does not perform a person classification process when the output from the detection unit 20 indicates that a person has not been detected.

[0036] As described above, in the case where the output of the detection unit 20 is input only when a person is detected by the detection unit 20, the classification unit 30 performs the person classification process every time the output of the detection unit 20 is input.

[0037] The functions of the classification unit 30 may be realized by any hardware or a combination of any hardware and software. For example, the classification unit 30 is primarily configured with a computer (information processing device) including a CPU, a memory device, an auxiliary storage device, an interface device, etc. The memory device may be, for example, an SRAM or DRAM. The auxiliary storage device may be, for example, an HDD, an SSD, a flash memory, an EEPROM, etc. The interface device may include, for example, a communication interface for communicating with other components of the monitoring system 1, such as the detection unit 20 and the determination unit 40. The interface device may also include, for example, an external interface for connecting to a recording medium. This allows the classification unit 30 to, for example, retrieve various programs from a recording medium via the external interface and install them in the auxiliary storage device. The information processing device may also include a high-speed arithmetic unit interlocked with the CPU. The high-speed arithmetic unit may include, for example, a GPU, an FPGA, an ASIC, etc.

[0038] For example, the classification unit 30 applies the trained model LM2 to estimate which of multiple types of attributes the person included in the partial image corresponds to, based on local features obtained from data of a partial image corresponding to an area in the image that includes a person and the data of that partial image. The trained model LM2 corresponds to a classifier that classifies the person in the partial image into one of multiple types of attributes, and may be a collection of multiple classifiers that classify whether or not the person corresponds to each of the multiple types of attributes.

[0039] The trained model LM2 is obtained, for example, by performing machine learning (i.e., supervised learning) on ​​the base learning model M2 using the training data set TRD2. The trained model LM2 may be generated by performing machine learning on the learning model M2 in the classification unit 30, or may be generated by performing machine learning on the learning model M2 in an information processing device separate from the classification unit 30. For example, each piece of training data included in the training data set TRD2 is a combination of local features obtained from partial image data containing a person as input data or the partial image data itself, and label data representing the attributes of the person included in the partial image as output data. As a result, the trained model LM2 can estimate the attributes of the person included in the partial image using image features obtained from the partial image data containing a person or the partial image data itself as input, and output label data representing the attributes of the person included in the partial image. For example, the trained model LM2 is mainly configured using a DNN, and the efficiency of DNN machine learning is improved by applying backpropagation based on the training data. The trained model LM2 may also be a support vector machine (SVM).

[0040] Furthermore, the classification unit 30 may apply a rule-based algorithm such as pattern matching based on feature points and local feature amounts of data of a partial image including a person, to classify the attributes of people included in the partial image.

[0041] For example, as shown in FIG. 3, the classification unit 30 receives data of a partial image as input and classifies people included in the partial image into either "station staff" or "people other than station staff." People other than station staff include, for example, passengers on the railway vehicle TRN as well as workers such as cleaners. In this example, the classification unit 30 receives data of the partial image of a rectangular area AR1 of the image 200 as input and classifies people included in the partial image of the rectangular area AR1 (passengers PS) as "people other than station staff." Furthermore, the classification unit 30 receives data of the partial image of a rectangular area AR2 as input and classifies people included in the partial image of the rectangular area AR2 (station staff SA) as "station staff."

[0042] The classification unit 30 is communicatively connected to the determination unit 40 via a one-to-one communication line, a local network, or the like, and the output of the classification unit 30 is taken into the determination unit 40. When person classification processing is performed, the output of the classification unit 30 includes information regarding the result of the person classification processing. When person classification processing is not performed, the output of the classification unit 30 includes information indicating that person classification processing was not performed. The output of the classification unit 30 may also include the output of the detection unit 20. In this case, the determination unit 40 can acquire the output of the detection unit 20 via the classification unit 30.

[0043] The information about the result of the person classification process includes label information that indicates the attributes of the people classified for each region (i.e., partial image) in the image that includes people detected by the detection unit 20. The information about the result of the person classification process may also include information that indicates a region in the image that includes people corresponding to the label information. The information about the result of the person classification process may also include data on the partial image of the region that includes people corresponding to the label information. The output of the classification unit 30 may also include the latest image data acquired by the imaging unit 10 together with information that indicates a region that includes people corresponding to the label information, instead of data on the partial image.

[0044] The determination unit 40 makes a determination (also referred to as an "evaluation") regarding the safety of people in the home PF. Hereinafter, for convenience, the process of determining the safety of people in the home PF performed by the determination unit 40 may be referred to as a "safety determination process."

[0045] It should be noted that the determination unit 40 does not need to perform the safety determination process when the output of the detection unit 20 indicates that no human has been detected.

[0046] The functions of the determination unit 40 may be realized by any hardware or any combination of hardware and software. For example, the determination unit 40 is mainly configured with a computer (information processing device) including a CPU, a memory device, an auxiliary storage device, an interface device, etc. The memory device is, for example, an SRAM or a DRAM. The auxiliary storage device is, for example, an HDD, an SSD, a flash memory, an EEPROM, etc. The interface device includes, for example, a communication interface for communicating with other components of the monitoring system 1, such as the classification unit 30 and the notification unit 50. The interface device also includes, for example, an external interface for connecting to a recording medium. This allows the determination unit 40 to, for example, retrieve various programs from a recording medium via the external interface and install them in the auxiliary storage device.

[0047] For example, when the detection unit 20 detects a person on the current (that is, latest) image of the imaging unit 10, the determination unit 40 makes a determination (evaluation) regarding the safety of the person detected by the detection unit 20.

[0048] Specifically, the determination unit 40 determines whether the safety of each person detected on the latest image by the detection unit 20 is relatively high or low. More specifically, the determination unit 40 determines whether the safety of each person detected on the latest image by the detection unit 20 is relatively high or low, based on predetermined criteria defined for each attribute classified by the classification unit 30.

[0049] For example, as shown in FIG. 4, areas R1 and R2 for determining the safety of people are provided in the home PF.

[0050] Areas R1 and R2 are separated by a boundary line B1 that extends in the longitudinal direction of the platform PF. In the width direction of the platform PF, area R1 is the area on the opposite side of the platform from the boundary line B1, and area R2 is the area on the side of the platform from the boundary line B1. The width direction of the platform PF is perpendicular to the longitudinal direction when viewed from directly above the platform PF.

[0051] Boundary line B1 is set, for example, on the inner side (i.e., the farther side as seen from the railway vehicle TRN) of the braille blocks laid along one end of the platform PF on the railway vehicle TRN side. The area above boundary line B1 may be included in area R1 or area R2.

[0052] If the attribute of the person to be determined is a type different from "station staff" (e.g., "person other than station staff"), the determination unit 40 determines that the person's safety is high when the person is located in area R1, and determines that the person's safety is low when the person is located in area R2. This is because if the person is located in area R1 relatively close to the railway vehicle TRN on the area R1 platform PF, there is a possibility that the person may come into contact with the door or body of the railway vehicle TRN when the doors of the railway vehicle TRN open or close or when the railway vehicle TRN departs. For example, a state in which the person to be determined is located in area R1 means that the person is completely contained within area R1 when viewed from directly above, and a state in which the person to be determined is located in area R2 means that at least a part of the person is contained within area R2 when viewed from directly above.

[0053] On the other hand, when the attribute of the person to be determined is "station staff," the determination unit 40 determines that the safety of the person is high regardless of whether the person is located in area R1 or R2. This is because station staff are personnel who ensure the safety of the platform PF, and may need to perform work in area R1, which is relatively close to the railcar TRN on the platform PF.

[0054] Furthermore, the determination unit 40 may make a determination (evaluation) regarding the overall safety of people when a predetermined range of the home PF is viewed as a whole. Specifically, the determination unit 40 may make a determination (evaluation) regarding the overall safety of people when a predetermined range of the home PF is viewed as a whole, based on a determination result (evaluation result) regarding the safety of each person detected in the current (latest) image by the detection unit 20. The predetermined range is, for example, the entire range of the home PF included in the imaging range of the imaging unit 10 (i.e., the monitoring range of the monitoring system 1). Furthermore, when the monitoring system 1 includes multiple imaging units 10, the predetermined range may be defined for each of multiple imaging ranges corresponding to the multiple imaging units 10, and may be the range of the home PF included in each imaging range.

[0055] For example, when the determination unit 40 determines that the safety of each of all people in a predetermined range detected by the detection unit 20 based on the latest image is relatively high, the determination unit 40 determines that the overall safety of the people in the predetermined range is relatively high. On the other hand, when the determination unit 40 determines that the safety of at least one of all people in a predetermined range detected by the detection unit 20 based on the latest image is relatively low, the determination unit 40 determines that the overall safety of the people in the predetermined range is relatively low. Furthermore, when the output of the detection unit 20 indicates that no people have been detected, the determination unit 40 may determine that the overall safety of people when viewing the predetermined range of the home PF as a whole is relatively high.

[0056] The determination unit 40 is communicably connected to the notification unit 50 via a one-to-one communication line or a local network, and the output of the determination unit 40, including information relating to the result of the safety determination process, is taken into the notification unit 50.

[0057] The information representing the result of the safety determination process includes, for example, information representing the determination result regarding the safety of each person detected by the detection unit 20. The information relating to the result of the safety determination process may also include information representing the determination result regarding the overall safety of people in the entire predetermined range of the home PF. The output of the determination unit 40 may also include the output of the detection unit 20. In this case, the notification unit 50 can acquire the output of the detection unit 20 via the classification unit 30 and the determination unit 40.

[0058] The notification unit 50 notifies the train crew of the train TRN of the monitoring results of the status of people on the platform PF through a display unit 60 provided inside the train TRN. The train crew of the train TRN includes, for example, the driver and conductor of the train TRN. Hereinafter, the process related to notification executed by the notification unit 50 may be referred to as "notification process" for convenience.

[0059] The functions of the notification unit 50 may be realized by any hardware or a combination of any hardware and software. For example, the notification unit 50 is primarily configured with a computer (information processing device) including a CPU, a memory device, an auxiliary storage device, an interface device, etc. The memory device may be, for example, an SRAM or DRAM. The auxiliary storage device may be, for example, an HDD, an SSD, a flash memory, an EEPROM, etc. The interface device may include, for example, a communication interface for communicating with other components of the monitoring system 1, such as the determination unit 40. The interface device may also include, for example, an external interface for connecting to a recording medium. This allows the notification unit 50 to, for example, retrieve various programs from a recording medium via the external interface and install them in the auxiliary storage device. The information processing device may also include a high-speed arithmetic unit interlocked with the CPU. The high-speed arithmetic unit may include, for example, a GPU, an FPGA, an ASIC, etc.

[0060] The notification unit 50 is communicably connected to the display unit 60 via a predetermined wireless communication line (for example, a short-range communication line such as WiFi or Bluetooth (registered trademark)), and the output of the notification unit 50 is taken into the display unit 60.

[0061] The display unit 60 is provided in the driver's cab, the conductor's cab, or the like of the railway vehicle TRN, and displays various notification information, including information input from the notification unit 50, to crew members such as the driver and conductor of the railway vehicle TRN. The display unit 60 is, for example, a liquid crystal display, an organic EL (Electroluminescence) display, or the like. The display unit 60 may also be an indicator lamp, an electronic bulletin board, or the like.

[0062] The display unit 60 is provided, for example, in a fixed manner in the driver's cab or the conductor's cab of the railway vehicle TRN. The display unit 60 may also be a portable terminal device (i.e., a mobile terminal) that is brought into the railway vehicle TRN by a crew member of the railway vehicle TRN. The mobile terminal is, for example, a tablet terminal.

[0063] The notification unit 50 causes the display unit 60 to display information relating to the results of the safety determination process by the determination unit 40, based on the output of the determination unit 40. For example, the notification unit 50 causes the display unit 60 to display the determination result relating to the safety of each person detected by the detection unit 20, based on the output of the determination unit 40. The notification unit 50 may also cause the display unit 60 to display the determination result relating to the overall safety of people for the entire predetermined range of the home PF, based on the output of the determination unit 40. Furthermore, when the output of the detection unit 20 indicates that no person has been detected, the notification unit 50 may cause the display unit 60 to display that no person has been detected within the monitored range of the home PF.

[0064] 5, the notification unit 50 transmits a control command to the display unit 60 to simultaneously display the latest image 500 acquired by the imaging unit 10 and images 510, 520, 530, and 540 corresponding to information relating to the result of the safety determination process performed by the determination unit 40 on the display unit 60. The process of generating the images to be displayed on the display unit 60 may be performed by an information processing device (e.g., a microcomputer) installed in the display unit 60, or may be performed by the notification unit 50. In the former case, the latest image 500 acquired by the imaging unit 10 may be input to the display unit 60 from the notification unit 50, or may be input directly from the imaging unit 10 to the display unit 60 by the imaging unit 10 and the display unit 60 being communicably connected via a predetermined wireless communication line.

[0065] In this example, the latest image 500 acquired by the imaging unit 10 shows a railway vehicle TRN entering the platform PF adjacent to each other, a platform door PD installed at the platform PF, and station staff SA1 and passengers PS1 and PS2 at the platform PF.

[0066] In this example, images 510, 520, 530, and 540 are displayed superimposed on image 500.

[0067] Image 510 is an image showing the above-mentioned boundary line B1 corresponding to the platform PF shown in image 500. This allows the crew of the railway vehicle TRN to understand the criteria for determining the safety of the platform PF on the latest image acquired by the imaging unit 10.

[0068] Image 520 is an image representing a partial image of image 500 that includes passenger PS1 detected by detection unit 20, and is drawn as a rectangular frame surrounding the area of ​​the partial image. Passenger PS1 is classified as a "person other than station staff" by classification unit 30, and is located in the area of ​​image 500 opposite the railway vehicle TRN side (i.e., area R1) based on image 510 corresponding to boundary line B1. Therefore, determination unit 40 determines that passenger PS1 is highly safe based on the criteria applied to attributes other than "station staff," and as a result, image 520 is drawn with a dashed line indicating high safety.

[0069] Image 530 is an image representing a partial image of image 500 that includes passenger PS2 detected by detection unit 20, and is drawn as a rectangular frame surrounding the area of ​​the partial image. Passenger PS2 is classified as a "person other than station staff" by classification unit 30, and in image 500, a portion of passenger PS2 is within the area on the railway vehicle TRN side (i.e., area R2) based on image 510 corresponding to boundary line B1. Therefore, determination unit 40 determines that passenger PS2 is low in safety based on the criteria applied to attributes other than "station staff," and as a result, image 530 is drawn with a solid line indicating low safety.

[0070] Image 540 represents a partial image of image 500 that includes station employee SA1 detected by the detection unit 20, and is drawn as a rectangular frame surrounding the area of ​​the partial image. In this example, station employee SA1 is partially located on the railcar TRN side (i.e., area R2) of image 510, which corresponds to boundary line B1, in image 500. However, because station employee SA1 is classified as a "station employee" by the classification unit 30, the determination unit 40 determines that station employee SA1 has a high level of safety based on the criteria applied to the attribute of "station employee," regardless of station employee SA1's position on the platform PF. As a result, image 520 is drawn with a dashed line, indicating a high level of safety.

[0071] In this way, in this example, the train crew of the railway vehicle TRN can grasp the level of safety of people on the platform PF where the railway vehicle TRN is entering, through images 520 to 540. Therefore, the train crew of the railway vehicle TRN can wait until a high level of safety for passengers PS2 is ensured, for example, through an announcement or a call from station staff SA1, before proceeding with the procedures for opening and closing the doors of the railway vehicle TRN and departing the railway vehicle TRN. Therefore, the monitoring system 1 can support the operation of the railway vehicle TRN based on the determination result regarding the safety of people on the platform PF.

[0072] Furthermore, in this example, even though station staff SA1 is in area R2 on the railroad vehicle TRN side with boundary line B1 as the reference, he is determined to be highly safe based on a determination criterion (evaluation criterion) different from that of passenger PS2. Therefore, the monitoring system 1 can prevent a situation in which the presence of station staff SA1 in area R2 continues to cause a low safety situation for people on platform PF, thereby disrupting the operation of railroad vehicle TRN.

[0073] [Example of monitoring process] Next, an example of a process (monitoring process) for monitoring the status of people on the home PF, which is executed by the monitoring system 1, will be described with reference to Fig. 6. Specifically, an example of the monitoring process executed by the monitoring system 1 of Fig. 1 will be described.

[0074] FIG. 6 is a flowchart showing an example of a process (monitoring process) for monitoring the status of a person on the home PF by the monitoring system 1.

[0075] The flowchart in FIG. 6 is repeatedly executed at predetermined processing intervals within a predetermined time period that includes the time period during which the railcar TRN is stopped at the platform PF, for example.

[0076] As shown in FIG. 6, in step S102, the detection unit 20 acquires the latest image data that has been received from the imaging unit 10 from a reception buffer or the like.

[0077] When the detection unit 20 completes the process of step S102, the process proceeds to step S104.

[0078] In step S104, the detection unit 20 executes a person detection process based on the image data acquired in step S102.

[0079] When the detection unit 20 completes the process of step S104, the process proceeds to step S106.

[0080] In step S106, the detection unit 20 determines whether or not a person has been detected by the person detection processing. If the detection unit 20 detects a person, it transmits output information including information about the result of the person detection processing to the classification unit 30, and upon receiving the output information of the detection unit 20, the classification unit 30 executes step S108. On the other hand, if the detection unit 20 does not detect a person, it transmits output information including information about the result of the person detection processing, i.e., information indicating that no person has been detected, to the notification unit 50, and upon receiving the output information of the detection unit 20, the notification unit 50 executes step S112.

[0081] In step S108, the classification unit 30 performs a person classification process based on the output information of the detection unit 20.

[0082] When the classification unit 30 completes the processing of step S108, it transmits output information including information regarding the results of the person classification processing to the judgment unit 40, and when the judgment unit 40 receives the output information of the judgment unit 40, it executes step S110.

[0083] In step S110 , the determination unit 40 performs safety determination processing based on the output information of the detection unit 20 and the output information of the classification unit 30 .

[0084] When the judgment unit 40 completes the processing of step S110, it transmits output information including information regarding the results of the safety judgment processing to the notification unit 50, and when the notification unit 50 receives the output information of the judgment unit 40, it executes step S112.

[0085] In step S112, the notification unit 50 performs notification processing based on the output information of the detection unit 20 received from the detection unit 20 or the output information of the determination unit 40 received from the determination unit 40.

[0086] Specifically, after the processing of step S106, when the notification unit 50 receives output information (specifically, information indicating that no person has been detected) from the detection unit 20, it causes the display unit 60 to display information that no person has been detected within the range of the home PF that is being monitored. Furthermore, after the processing of step S110, when the notification unit 50 receives output information from the determination unit 40, it causes the display unit 60 to display information regarding the result of the safety determination processing by the determination unit 40.

[0087] When the process of step S112 is completed, the process of this flowchart ends.

[0088] In this way, in this example, the monitoring system 1 can notify the crew of the railway vehicle TRN of the monitoring results regarding the status of people on the platform PF, specifically, information regarding the results of the person detection processing by the detection unit 20 and information regarding the results of the safety judgment processing by the judgment unit 40, via the display unit 60.

[0089] [Second example of a monitoring system] Next, a second example of the monitoring system 1 according to this embodiment will be described with reference to FIGS.

[0090] Hereinafter, in this example, the same symbols are used for configurations that are the same as or correspond to those in the first example described above, and the explanation will focus on the parts that are different from the first example described above, and explanations of the parts that are the same as or correspond to those in the first example described above may be omitted.

[0091] Fig. 7 is a diagram illustrating another example of the safety determination process by the determination unit 40. Fig. 8 is a diagram illustrating a second example of the display content of the display unit 60.

[0092] In this example, the configuration of the monitoring system 1 is shown in Fig. 1, similarly to the first example described above. Therefore, in this example, a diagram showing the configuration of the monitoring system 1 will be omitted, and Fig. 1 will be used instead. Furthermore, the monitoring process of the monitoring system 1 according to this example is the same as that shown in Fig. 7 described above, and therefore a description thereof will be omitted.

[0093] In this example, the monitoring system 1 differs from the first example described above mainly in the content of the safety determination process by the determination unit 40. Specifically, the criteria for determining the safety of a person classified as a "station staff" by the classification unit 30 differ from those in the first example described above.

[0094] For example, as shown in FIG. 7, the home PF is provided with areas R1 and R2 for determining the safety of people, similar to FIG. 4 of the first example described above.

[0095] In this example, area R2 is divided into areas R3 and R4 by a boundary line B2 that extends in the longitudinal direction of the platform PF. In the width direction of the platform PF, area R3 is the area on the opposite side of the platform from railcar TRN, with boundary line B2 as the reference, and area R4 is the area on the side of the platform from railcar TRN, with boundary line B2 as the reference.

[0096] The boundary line B2 is set, for example, at the boundary line on the outside (that is, the closer side when viewed from the railway vehicle TRN) of the braille blocks laid along one end of the platform PF on the railway vehicle TRN side.

[0097] If the attributes of the person being judged are of a type different from "station staff," the judgment unit 40 judges whether the safety of the person detected by the detection unit 20 is high or low using the same criteria as in the first example described above.

[0098] On the other hand, when the attribute of the person to be determined is "station staff," the determination unit 40 determines that the person's safety is high when the person is located in area R1 or area R3, and determines that the person's safety is low when the person is located in area R4. As a result, when the attribute of the person to be determined is "station staff," the determination unit 40 can ensure a wider area on the railroad vehicle TRN side where the person's safety is determined to be higher than when the person is located in an area other than "station staff," while determining that the person's safety is low in area R4, which is very close to the railroad vehicle TRN. Therefore, for example, even if a station staff member needs to perform work in area R2, which is relatively close to the railroad vehicle TRN, if the station staff member approaches area R4, which is very close to the railroad vehicle TRN, ensuring the safety of the station staff member can be prioritized. Therefore, the monitoring system 1 can improve the safety of station staff while suppressing delays in the operation of the railroad vehicle TRN. For example, when the person being judged is located in area R1 or area R3, it means that the person is completely contained within the combined area of ​​areas R1 and R3 when viewed from directly above, and when the person being judged is located in area R4, it means that at least a part of the person is contained within area R4 when viewed from directly above.

[0099] For example, as shown in FIG. 8, the notification unit 50 sends a control command to the display unit 60, causing the display unit 60 to simultaneously display the latest image 800 acquired by the imaging unit 10 and images 810, 820, 830, and 840 corresponding to information regarding the results of the safety assessment process by the assessment unit 40.

[0100] In this example, the latest image 800 of the imaging unit 10 shows a railway vehicle TRN entering the platform PF adjacent to each other, a platform door PD installed on the platform PF, and station staff SA2 and passenger PS3 who are on the platform PF.

[0101] In this example, images 810, 820, 830, and 840 are displayed superimposed on image 800.

[0102] Image 810 is an image showing the above-mentioned boundary line B1 corresponding to the platform PF shown in image 800. Image 820 is an image showing the above-mentioned boundary line B2 corresponding to the platform PF shown in image 800. This allows the crew of the railway vehicle TRN to understand the criteria for determining the safety of the platform PF on the latest image acquired by the imaging unit 10.

[0103] Image 830 is an image representing a partial image of image 800 that includes passenger PS3 detected by the detection unit 20, and is drawn as a rectangular frame surrounding the area of ​​the partial image. Passenger PS3 is classified as a "person other than station staff" by the classification unit 30, and is located in the area of ​​image 800 opposite the railcar TRN side (i.e., area R1) based on image 810, which corresponds to boundary line B1. Therefore, the determination unit 40 determines that passenger PS3 is highly safe based on the criteria applied to attributes other than "station staff," and as a result, image 830 is drawn with a dashed line indicating high safety.

[0104] Image 840 is an image representing a partial image of image 800 that includes station staff SA2 detected by the detection unit 20, and is drawn as a rectangular frame surrounding the area of ​​the partial image. Station staff SA2 is classified as a "station staff" by the classification unit 30, and in image 800, a part of him is within the area on the railway vehicle TRN side (i.e., area R4) based on image 820 corresponding to boundary line B2. Therefore, the determination unit 40 determines that station staff SA2 is low in safety based on the criteria applied to attributes other than "station staff," and as a result, image 840 is drawn with a solid line indicating low safety.

[0105] In this example, the train crew of the railway vehicle TRN can thus grasp the level of safety of people on the platform PF where the railway vehicle TRN is entering, through images 810, 820, 830, and 840. Therefore, the train crew of the railway vehicle TRN can wait until a high level of safety for passengers PS2 is ensured, for example, through an announcement or a call from station staff SA1, before proceeding with the procedures for opening and closing the doors of the railway vehicle TRN and departing the railway vehicle TRN. Therefore, the monitoring system 1 can support the operation of the railway vehicle TRN based on the determination results regarding the safety of people on the platform PF.

[0106] Furthermore, in this example, when station employee SA2 is located in areas R1 and R3 on the opposite side of railway vehicle TRN using boundary line B2 as a reference, the safety of station employee SA2 is determined to be high, and when station employee SA2 is located in area R4 on the railway vehicle TRN side, the safety of station employee SA2 is determined to be low. Therefore, the monitoring system 1 can prioritize the safety of station employee SA2 when station employee SA2 is very close to railway vehicle TRN, while suppressing a situation in which the operation of railway vehicle TRN is disrupted due to the presence of station employee SA2 located in area R2, for example.

[0107] [Third example of a monitoring system] Next, a third example of the monitoring system 1 according to this embodiment will be described with reference to FIGS.

[0108] Hereinafter, in this example, the same symbols are used for configurations that are the same as or correspond to those in the first and second examples described above, and the explanation will focus on the parts that are different from the first and second examples described above, and explanations of the parts that are the same as or correspond to those in the first and second examples described above may be omitted.

[0109] Fig. 9 is a diagram showing the configuration of a third example of the monitoring system 1. Fig. 10 is a diagram showing a third example of the display content of the display unit 60.

[0110] In this example, the monitoring system 1 differs from the first or second example described above mainly in that an estimation unit 70 and a recognition unit 80 are added.

[0111] In this example, the determination unit 40 is communicatively connected to the estimation unit 70 via a one-to-one communication line, a local network, or the like, and the output of the determination unit 40, including information relating to the result of the safety determination process, is taken into the estimation unit 70. Furthermore, similar to the first example described above, the determination unit 40 may be communicatively connected to the notification unit 50 via a one-to-one communication line, a local network, or the like, and the output of the determination unit 40, including information relating to the result of the safety determination process, may be taken into the notification unit 50 directly.

[0112] The estimation unit 70 estimates the posture state of the person detected by the detection unit 20 based on data of a partial image including the person detected by the detection unit 20. Specifically, the estimation unit 70 estimates the posture state of the person detected by the detection unit 20 every time the output of the determination unit 40 is input. For example, the estimation unit 70 estimates the positions of multiple body parts (e.g., joints) in the partial image including the person detected by the detection unit 20. The multiple body parts include, for example, ankles, knees, waist, chest, shoulders, elbows, wrists, neck, and head. The estimation unit 70 may also estimate which of multiple predefined types of posture states the person's posture state corresponds to. The multiple types of posture states are predefined, for example, for multiple types of attributes. Hereinafter, the process related to estimating the posture state of the person executed by the estimation unit 70 may be referred to as a "posture estimation process" for convenience.

[0113] The functions of the estimation unit 70 may be realized by any hardware or a combination of any hardware and software. For example, the estimation unit 70 is mainly configured by a computer (information processing device) including a CPU, a memory device, an auxiliary storage device, an interface device, etc. The memory device is, for example, an SRAM or a DRAM. The auxiliary storage device is, for example, an HDD, an SSD, a flash memory, an EEPROM, etc. The interface device includes, for example, a communication interface for communicating with other components of the monitoring system 1, such as the determination unit 40 and the recognition unit 80. The interface device also includes, for example, an external interface for connecting to a recording medium. This allows the estimation unit 70 to, for example, retrieve various programs from a recording medium via the external interface and install them in the auxiliary storage device. The information processing device may also include a high-speed arithmetic unit interlocked with the CPU. The high-speed arithmetic unit includes, for example, a GPU, an FPGA, an ASIC, etc.

[0114] For example, the estimation unit 70 applies the trained model LM3 based on local features obtained from the data of a partial image containing a person detected by the detection unit 20 and the data of that partial image to estimate the posture state of the person included in the partial image.

[0115] The trained model LM3 is obtained, for example, by performing machine learning (i.e., supervised learning) on ​​the base training model M3 using the training data set TRD3. The trained model LM3 may be generated by performing machine learning on the training model M3 in the estimation unit 70, or may be generated by performing machine learning on the training model M3 in an information processing device separate from the estimation unit 70. For example, each training data included in the training data set TRD3 is a combination of local features obtained from partial image data containing a person as input data or the partial image data itself, and data representing the posture of the person included in the partial image as output data. As a result, the trained model LM3 can estimate the posture of the person included in the partial image using image features obtained from the partial image data containing a person or the partial image data itself as input, and output data representing the person's posture. For example, the trained model LM3 is mainly configured using a DNN, and the application of backpropagation based on the training data makes the DNN machine learning more efficient. Furthermore, the trained model LM3 may be an SVM as a classifier when estimating which of multiple predefined types of posture states the posture state of a person included in a partial image corresponds to.

[0116] Furthermore, the estimation unit 70 may apply a rule-based algorithm based on feature points, local features, etc. of data of a partial image including a person to estimate which of multiple predefined types of posture states the posture state of a person included in the partial image corresponds to.

[0117] The estimation unit 70 is communicably connected to the recognition unit 80 via a one-to-one communication line, a local network, or the like, and the output of the estimation unit 70, including information regarding the result of the posture estimation process, is taken into the recognition unit 80.

[0118] The information about the result of the posture estimation process includes, for example, information representing the posture state of the person detected by the detection unit 20. The information representing the posture state of the person includes information representing the positions of multiple body parts of the person in the partial image. The information about the result of the posture estimation process may also include information representing an area in the image containing the person, which corresponds to the information representing the posture state of the person. The information about the result of the posture estimation process may also include data of a partial image of an area in the image containing the person, which corresponds to the information representing the posture state of the person. The output of the estimation unit 70 may also include the latest image data of the imaging unit 10 together with information representing the area in the image containing the person, instead of data of the partial image. The output of the estimation unit 70 may also include the output of the determination unit 40.

[0119] The recognition unit 80 recognizes the behavior of the person detected by the detection unit 20 based on the output of the estimation unit 70. Specifically, each time the recognition unit 80 receives the output of the estimation unit 70, it recognizes what behavior the person detected by the detection unit 20 is performing. Hereinafter, for convenience, the processing related to recognizing the behavior of people in the home PF, which is executed by the recognition unit 80, may be referred to as "behavior recognition processing."

[0120] The recognition unit 80 recognizes a predetermined specific behavior of the person (hereinafter, for convenience, a "first specific behavior") based on, for example, the posture state of the person estimated by the estimation unit 70 and detected by the detection unit 20. In other words, the recognition unit 80 recognizes whether or not the behavior of the person detected by the detection unit 20 corresponds to the first specific behavior. For example, one or more types of first specific behaviors are predetermined for each of the above-mentioned multiple types of attributes. When multiple types of first specific behaviors are defined, the recognition unit 80 recognizes whether or not the behavior of the person detected by the detection unit 20 is the first specific behavior for each of the multiple types of first specific behaviors. For example, the recognition unit 80 recognizes the first specific behavior of the person by determining whether or not the posture state of the person detected by the detection unit 20 and its changes over time represent the first specific behavior.

[0121] The first specific behavior corresponding to the attribute of "station staff" is, for example, a "monitoring" behavior that indicates that the platform PF is being monitored. The "monitoring" behavior indicates, for example, a behavioral state of standing on the platform PF without moving. The first specific behavior corresponding to the attribute of "station staff" may also be a "signaling" behavior of signaling to the crew of the railway vehicle TRN. The "signaling" behavior indicates, for example, a behavioral state of raising a hand holding a signal light, a hand flag, or the like higher than the head. The first specific behavior corresponding to the attribute of "person other than station staff" is, for example, an "upright" behavior that indicates that a person is standing upright on the platform PF. The first specific behavior corresponding to the attribute of "person other than station staff" may also be a "leaning" behavior of a person leaning on a platform door PD.

[0122] The functions of the recognition unit 80 may be realized by any hardware or a combination of any hardware and software. For example, the recognition unit 80 is primarily configured with a computer (information processing device) including a CPU, a memory device, an auxiliary storage device, an interface device, etc. The memory device may be, for example, an SRAM or a DRAM. The auxiliary storage device may be, for example, an HDD, an SSD, a flash memory, an EEPROM, etc. The interface device may include, for example, a communication interface for communicating with other components of the monitoring system 1, such as the estimation unit 70 and the notification unit 50. The interface device may also include, for example, an external interface for connecting to a recording medium. This allows the recognition unit 80 to, for example, retrieve various programs from a recording medium via the external interface and install them in the auxiliary storage device. The information processing device may also include a high-speed arithmetic unit interlocked with the CPU. The high-speed arithmetic unit may include, for example, a GPU, an FPGA, an ASIC, etc.

[0123] For example, the recognition unit 80 determines whether or not the person detected by the detection unit 20 is engaging in a first specific behavior based on a comparison between data representing the person's posture state estimated by the estimation unit 70 and reference data representing the posture state corresponding to the first specific behavior. The recognition unit 80 may also determine whether or not the person is engaging in a first specific behavior based on a comparison between data representing a time-series change in the person's posture state estimated by the estimation unit 70 and reference data representing a time-series change in the posture state corresponding to the first specific behavior.

[0124] The recognition unit 80 is communicably connected to the notification unit 50 via a one-to-one communication line, a local network, or the like, and the output of the recognition unit 80, including information regarding the results of the behavior recognition processing, is taken into the notification unit 50.

[0125] The information on the result of the behavior recognition processing includes, for example, information indicating whether or not the person has performed a first specific behavior detected by the detection unit 20. When multiple types of first specific behaviors are defined for the attributes of the person detected by the detection unit 20, the information on the result of the behavior recognition processing may include, for each of the multiple types of first specific behaviors, information indicating whether or not the person has performed the first specific behavior detected by the detection unit 20. Furthermore, the output of the recognition unit 80 may include the output of the determination unit 40.

[0126] Similar to the first example described above, the notification unit 50 causes the display unit 60 to display information relating to the result of the safety determination processing by the determination unit 40 based on the output of the determination unit 40. Also, similar to the first example described above, when the output of the detection unit 20 indicates that no person has been detected, the notification unit 50 may cause the display unit 60 to display that no person has been detected within the range of the monitoring target of the home PF. Also, in this example, the notification unit 50 causes the display unit 60 to display information relating to the result of the behavior recognition processing by the recognition unit 80.

[0127] For example, the notification unit 50 displays on the display unit 60 information indicating the first specific behavior of the person detected by the detection unit 20, as recognized by the recognition unit 80. This allows the crew of the railway vehicle TRN to deal with the situation of the person on the platform PF in accordance with the behavioral state of the person detected by the detection unit 20. For example, if the person determined to be unsafe by the determination unit 40 is in an "upright" behavioral state, the crew of the railway vehicle TRN instructs station staff on the platform PF to encourage the person to move by making an announcement at the platform PF, since the urgency of eliminating the unsafe state is not that high. On the other hand, if the person determined to be unsafe by the determination unit 40 is in a "leaning" behavioral state, the crew of the railway vehicle TRN instructs station staff on the platform PF to mobilize to the person's location, since the urgency of eliminating the unsafe state is high. Furthermore, the notification unit 50 may display on the display unit 60 information indicating a countermeasure corresponding to the first specific behavior of the person detected by the detection unit 20, as recognized by the recognition unit 80. For example, if a person determined by the determination unit 40 to be unsafe is in an "upright" behavioral state, information urging an announcement to move away from the railway vehicle TRN is displayed on the display unit 60 as a way of dealing with the person. Also, for example, if a person determined by the determination unit 40 to be unsafe is in a "leaning" behavioral state, information urging a station staff member at the platform PF to be dispatched to the location of the person is displayed on the display unit 60. This allows the crew of the railway vehicle TRN to easily take action in accordance with the first specific behavior of the person by checking the display content displayed on the display unit 60.

[0128] 10 , the notification unit 50 transmits a control command to the display unit 60 to simultaneously display the latest image 1000 acquired by the imaging unit 10 and auxiliary images 1010, 1020, 1030, 1032, 1034, 1040, 1042, 1044, 1050, 1052, and 1054. The images 1010, 1020, 1030, 1040, and 1050 correspond to information about the result of the safety determination process performed by the determination unit 40. The images 1032, 1034, 1042, 1044, 1052, and 1054 correspond to information about the result of the behavior recognition process performed by the recognition unit 80.

[0129] In this example, the latest image 1000 acquired by the imaging unit 10 shows a railway vehicle TRN entering the platform PF adjacent to the platform door PD installed at the platform PF, as well as station employee SA3 and passengers PS4 and PS5 at the platform PF.

[0130] In this example, images 1010, 1020, 1030, 1032, 1034, 1040, 1042, 1044, 1050, 1052, and 1054 are displayed superimposed on image 1000.

[0131] Image 1010 is an image showing the above-mentioned boundary line B1 corresponding to the platform PF shown in image 1000. Image 1020 is an image showing the above-mentioned boundary line B2 corresponding to the platform PF shown in image 1000. This allows the crew of the railway vehicle TRN to understand the criteria for determining the safety of the platform PF on the latest image acquired by the imaging unit 10.

[0132] Image 1030 is an image representing a partial image of image 1000 that includes passenger PS4 detected by the detection unit 20, and is drawn as a rectangular frame surrounding the area of ​​the partial image. Passenger PS4 is classified as a "person other than station staff" by the classification unit 30, and in image 1000, a portion of passenger PS4 is within the area on the railway vehicle TRN side (i.e., area R2) based on image 1010 corresponding to boundary line B1. Therefore, the determination unit 40 determines that passenger PS4 is low in safety based on the criteria applied to attributes other than "station staff," and as a result, image 1030 is drawn with a solid line indicating low safety.

[0133] Image 1032 is an image that represents the posture of the passenger PS4 included in the partial image that corresponds to image 1030. Specifically, image 1032 is an image that represents a skeleton that is a combination of multiple feature points that correspond to the positions of multiple parts of the body of the passenger PS4 included in the partial image that corresponds to image 1030, and straight lines (also called "bones") that connect the feature points. This allows the crew of the railway vehicle TRN to easily understand the posture of the passenger PS4.

[0134] Image 1034 is an image representing a first specific behavior of passenger PS4 included in the partial image corresponding to image 1030, recognized by the recognition unit 80. In this example, passenger PS4 is recognized by the recognition unit 80 as being in the behavioral state of "standing upright," and image 1034 includes text information of "standing upright." This allows the crew of the railway vehicle TRN to take measures appropriate to the behavioral state (i.e., "standing upright") of passenger PS4, which has been determined by the determination unit 40 to be unsafe.

[0135] Image 1040 is an image representing a partial image of image 1000 that includes passenger PS5 detected by detection unit 20, and is drawn as a rectangular frame surrounding the area of ​​the partial image. Passenger PS5 is classified as a "person other than station staff" by classification unit 30, and in image 1000, passenger PS5 is entirely within the area on the railway vehicle TRN side (i.e., area R2) based on image 1010 corresponding to boundary line B1. Therefore, determination unit 40 determines that passenger PS5 is low in safety based on the criteria applied to attributes other than "station staff," and as a result, image 1040 is drawn with a solid line indicating low safety.

[0136] Image 1042 is an image that represents the posture of passenger PS5 included in the partial image that corresponds to image 1040. Specifically, image 1042 is an image that represents a skeleton that is a combination of multiple feature points that correspond to the positions of multiple parts of the body of passenger PS5 included in the partial image that corresponds to image 1040, and straight lines (bones) that connect the feature points. This allows the crew of the railway vehicle TRN to easily understand the posture of passenger PS5.

[0137] Image 1044 is an image representing a first specific behavior of passenger PS5 included in the partial image corresponding to image 1040, recognized by the recognition unit 80. In this example, passenger PS5 is recognized by the recognition unit 80 as being in the behavioral state of "leaning", and image 1044 includes text information of "leaning". This allows the crew of the railway vehicle TRN to take measures appropriate to the behavioral state (i.e., "leaning") of passenger PS5, which has been determined to be unsafe by the determination unit 40.

[0138] Image 1050 is an image representing a partial image of image 1000 that includes station staff SA3 detected by the detection unit 20, and is drawn as a rectangular frame surrounding the area of ​​the partial image. Station staff SA3 is classified as a "station staff" by the classification unit 30, and is located in the area of ​​image 1000 opposite the railway vehicle TRN (i.e., the area combining areas R1 and R3) based on image 1020 corresponding to boundary line B2. Therefore, the determination unit 40 determines that station staff SA3 is highly safe based on the criteria applied to the attribute of "station staff," and as a result, image 1050 is drawn with a dashed line indicating high safety.

[0139] Image 1052 is an image that represents the posture of station employee SA3 included in the partial image that corresponds to image 1050. Specifically, image 1052 is an image that represents a skeleton that is a combination of multiple feature points that correspond to the positions of multiple parts of the body of station employee SA3 included in the partial image that corresponds to image 1050, and straight lines (bones) that connect the feature points. This allows the crew of the railway vehicle TRN to easily understand the posture of station employee SA3.

[0140] Image 1054 is an image representing a first specific behavior of station employee SA3 included in the partial image corresponding to image 1050, recognized by recognition unit 80. In this example, station employee SA3 has been recognized by recognition unit 80 as being in the behavioral state of "monitoring," and image 1054 includes text information of "monitoring." This allows the crew of railway vehicle TRN to understand the behavioral state of station employee SA3 (i.e., "monitoring").

[0141] In this way, in this example, the crew of the railway vehicle TRN can grasp the level of safety of people on the platform PF where the railway vehicle TRN is arriving, through images 1010, 1020, 1030, 1040, and 1050. Therefore, the crew of the railway vehicle TRN can wait until the safety of passengers PS4 and PS5 is ensured before proceeding with the procedures for opening and closing the doors of the railway vehicle TRN and departing the railway vehicle TRN. Therefore, the monitoring system 1 can support the operation of the railway vehicle TRN by taking into consideration the determination results regarding the safety of people on the platform PF.

[0142] Furthermore, in this example, the crew of the railway vehicle TRN can grasp the posture and first specific behavior of people on the platform PF where the railway vehicle TRN is entering, through images 1032, 1034, 1042, 1044, 1052, and 1054. This allows the crew of the railway vehicle TRN to take action in accordance with the posture and first specific behavior of each of the passengers PS4 and PS5 who are in a state of low safety. Therefore, the monitoring system 1 can support the operation of the railway vehicle TRN by taking into account the posture and first specific behavior of people on the platform PF.

[0143] [Other examples of monitoring processes] Next, another example of the process (monitoring process) for monitoring the status of people on the home PF, which is executed by the monitoring system 1, will be described with reference to Fig. 11. Specifically, the monitoring process executed by the monitoring system 1 in Fig. 9 will be described as an example.

[0144] FIG. 11 is a flowchart schematically showing another example of the process (monitoring process) related to monitoring the status of people on the home PF by the monitoring system 1.

[0145] The flowchart in FIG. 11 is repeatedly executed at predetermined processing intervals within a predetermined time period that includes the time period during which the railcar TRN is stopped at the platform PF, for example.

[0146] As shown in FIG. 11, steps S202, S204, and S206 are the same as steps S102, S104, and S106 in FIG. 6 described above, and therefore a description thereof will be omitted.

[0147] If the detection unit 20 detects a person, it transmits output information including information relating to the result of the person detection processing to the classification unit 30, and the classification unit 30 executes step S208 upon receiving the output information of the detection unit 20. On the other hand, if the detection unit 20 does not detect a person, it transmits output information including information relating to the result of the person detection processing, i.e., information indicating that no person was detected, to the notification unit 50, and the notification unit 50 executes step S216 upon receiving the output information of the detection unit 20.

[0148] Steps S208 and S210 are the same as steps S108 and S110 in FIG. 6 described above, and therefore a description thereof will be omitted.

[0149] When the judgment unit 40 completes the processing of step S210, it transmits output information including information regarding the results of the safety judgment processing to the estimation unit 70, and when the estimation unit 70 receives the output information of the judgment unit 40, it executes step S212.

[0150] In step S212, the estimation unit 70 performs a posture estimation process based on information related to the result of the person detection process performed by the detection unit 20.

[0151] Information regarding the result of the human detection processing by the detection unit 20 is input to the estimation unit 70 via the classification unit 30 and the determination unit 40, for example, in a form included in the output information of the determination unit 40. Alternatively, the detection unit 20 and the estimation unit 70 may be configured to be able to communicate with each other via a one-to-one communication line, a local network, or the like, and the information regarding the result of the human detection processing by the detection unit 20 may be input directly from the detection unit 20 to the estimation unit 70.

[0152] When the estimation unit 70 completes the processing of step S212, it transmits output information including information regarding the results of the posture estimation processing to the recognition unit 80, and when the recognition unit 80 receives the output information of the estimation unit 70, it executes step S214.

[0153] In step S214, the recognition unit 80 performs a behavior recognition process based on the output information of the estimation unit 70.

[0154] When the recognition unit 80 completes the processing of step S214, it transmits output information including information regarding the results of the behavior recognition processing to the notification unit 50, and when the notification unit 50 receives the output information from the recognition unit 80, it executes step S216.

[0155] In step S216, the notification unit 50 performs notification processing based on the output information of the detection unit 20 or the output information of the determination unit 40 and the recognition unit 80.

[0156] The output information of the determination unit 40, specifically, information relating to the result of the safety determination process of the determination unit 40, is input to the notification unit 50 via the estimation unit 70 and the recognition unit 80, for example, in a form included in the output information of the recognition unit 80. Alternatively, the determination unit 40 and the notification unit 50 may be configured to be able to communicate with each other via a one-to-one communication line, a local network, or the like, and the information relating to the result of the safety determination process of the determination unit 40 may be input directly from the determination unit 40 to the notification unit 50.

[0157] When the process of step S216 is completed, the process of this flowchart ends.

[0158] In this way, in this example, the monitoring system 1 can notify the crew of the railway vehicle TRN via the display unit 60 of information regarding the results of the safety judgment processing by the judgment unit 40, as well as information regarding the results of the behavior recognition processing by the recognition unit 80.

[0159] [Other embodiments] Next, another embodiment will be described.

[0160] The monitoring system 1 according to the above-described embodiment may be modified or changed as appropriate.

[0161] For example, in the first and second examples of the monitoring system 1 described above, the functions of the detection unit 20, the classification unit 30, the determination unit 40, and the notification unit 50 may be realized by one information processing device. Also, in the first and second examples of the monitoring system 1 described above, the functions of the detection unit 20, the classification unit 30, the determination unit 40, and the notification unit 50 may be realized in a distributed manner by two, three, five or more information processing devices.

[0162] In addition, in the third example of the monitoring system 1 described above, the functions of the detection unit 20, the classification unit 30, the determination unit 40, the notification unit 50, the estimation unit 70, and the recognition unit 80 may be realized by one information processing device. In addition, in the third example of the monitoring system 1 described above, the functions of the detection unit 20, the classification unit 30, the determination unit 40, the notification unit 50, the estimation unit 70, and the recognition unit 80 may be realized in a distributed manner by two, three, four, five, or seven or more information processing devices.

[0163] Furthermore, in the above-described embodiment and examples of variations and modifications thereof, the classification unit 30 may classify the person included in the partial image into one of three or more attributes including "station staff" based on data of the partial image including the person detected by the detection unit 20. For example, the above-described "person other than station staff" may be further classified into two or more attributes (for example, "passenger" and "other person").

[0164] Furthermore, in the first to third examples of the display content of the display unit 60 described above, other types of images may be simultaneously displayed on the display unit 60 along with the image captured by the imaging unit 10. For example, the display unit 60 may display a judgment result regarding the overall safety of people on the platform PF along with the above-described image 500, image 800, or image 1000. Furthermore, the display unit 60 may display information indicating a countermeasure in response to a first specific behavior of the passengers PS4 and PS5 along with the above-described image 1000.

[0165] Furthermore, in the above-described embodiment and examples of variations and modifications thereof, the notification unit 50 may provide the above-described various notifications to other persons involved in the operation of the railway vehicle TRN (for example, station staff at the platform PF, etc.) instead of or in addition to the crew of the railway vehicle TRN. For example, the notification unit 50 may be attached to the ceiling of the platform PF and display the same display content as that of the display unit 60 on a monitor installed in the information section of the platform PF.

[0166] Furthermore, in the above-described embodiment and examples of variations and modifications thereof, the notification unit 50 may notify the crew of the railway vehicle TRN, station staff at the platform PF, etc., using a type of notification means different from visual notification means such as the display unit 60. For example, the notification unit 50 may notify the crew of the railway vehicle TRN, station staff at the platform PF, etc., in an auditory manner via a sound output unit provided inside the railway vehicle TRN.

[0167] Furthermore, in the third example of the monitoring system 1 described above and examples of its variations and modifications, the recognition unit 80 may directly recognize the first specific behavior of a person based on data of a partial image including the person detected by the detection unit 20. In this case, the estimation unit 70 may be omitted. For example, the recognition unit 80 may recognize the specific behavior of a person using a trained model that has undergone supervised learning based on local features obtained from the partial image data or the image data itself, or may recognize the first specific behavior of a person using a rule-based method such as pattern matching.

[0168] In the above-described embodiment and examples of variations and modifications thereof, the determination unit 40 determines the safety of a person on the platform PF based on criteria related to the positional relationship between the person on the platform PF and the railway vehicle TRN, but the determination unit 40 may also determine the safety of a person based on other criteria related to the position of the person on the platform PF. For example, if a person detected by the detection unit 20 is entering a no-entry area set at an end in the lengthwise direction (longitudinal direction) of the platform PF, the determination unit 40 may determine that the person's safety is low.

[0169] Furthermore, in the above-described embodiments and examples of variations and modifications thereof, the judgment unit 40 makes a judgment regarding the safety of people on the home PF based on criteria related to the position of people on the home PF, but instead of or in addition to these criteria, the judgment regarding the safety of people on the home PF may also be made based on other types of criteria.

[0170] For example, the determination unit 40 may determine the safety of a person on the platform PF based on the person's movement direction D and movement speed V. Specifically, the determination unit 40 may determine that the person's safety is low when the person's movement direction D on the platform PF is a direction toward the railway vehicle TRN and the movement speed V is equal to or greater than a predetermined threshold value Vth. This is because the person on the platform PF is moving toward the railway vehicle TRN at a relatively high speed, and there is a possibility that the person will run onto the train.

[0171] Furthermore, the determination unit 40 may determine that the safety of a person is low when the behavior of the person detected by the detection unit 20 corresponds to a specific behavior of low safety (hereinafter, for convenience, referred to as a "second specific behavior"). For example, one or more types of second specific behaviors are predefined for each of the above-mentioned multiple types of attributes. In this case, the recognition unit 80 recognizes the second specific behavior of the person detected by the detection unit 20 in the same manner as in the case of the above-mentioned first specific behavior.

[0172] Furthermore, in the above-described embodiment and examples of modifications and variations thereof, the criteria for determining the safety of a person detected by the detection unit 20 are different between "station staff" and other types of attributes different from "station staff." However, this is not limited to this embodiment. Specifically, the criteria for determining the safety of a person detected by the detection unit 20 may be different between a specific type of attribute different from "station staff" and other types of attributes different from the specific type of attribute. For example, the above-described multiple types of attributes include "person in need of assistance," which represents a person who may require assistance from a station staff member or the like. The criteria for determining the safety of a person detected by the detection unit 20 are different between the case of "person in need of assistance" and the case of other types of attributes different from "person in need of assistance." Examples of persons in need of assistance include people who use white canes and people who use wheelchairs. In this case, the criteria for determining the safety of a person are specified so that when a person detected by the detection unit 20 is a "person in need of assistance," the person's safety is more likely to be determined to be lower than when a person detected by the detection unit 20 has other types of attributes different from "person in need of assistance." This can improve the safety of persons in need of assistance on platform PFs.

[0173] For example, if the person detected by the detection unit 20 is a "person requiring assistance," the boundary line between the low and high safety states is set at a position offset by a predetermined amount on the opposite side of the railway vehicle TRN from the above-mentioned boundary line B1. Furthermore, with regard to the criteria for determining the safety of a person based on the person's movement direction D and movement speed V on the platform PF, if the person detected by the detection unit 20 is a "person requiring assistance," the threshold value Vth of the movement speed V is set to be smaller than in the case of other types of attributes different from "person requiring assistance."

[0174] Furthermore, in the above-described embodiments and examples of variations and modifications thereof, the judgment unit 40 judges the safety of the person detected by the detection unit 20 in two stages, high safety or low safety, but the judgment regarding safety may also be made in three or more stages.

[0175] [Effect] Next, the operations of the monitoring system, monitoring device, monitoring method, and program according to this embodiment will be described.

[0176] In a first aspect of this embodiment, the surveillance system includes an imaging unit, a detection unit, a classification unit, and a determination unit. The surveillance system is, for example, the above-described surveillance system 1. The imaging unit is, for example, the above-described imaging unit 10. The detection unit is, for example, the above-described detection unit 20. The classification unit is, for example, the above-described classification unit 30. The determination unit is, for example, the above-described determination unit 40. Specifically, the imaging unit captures an image of a station platform. The station is, for example, the above-described station ST. The platform is, for example, the above-described platform PF. The detection unit detects a person present on the platform based on the image acquired by the imaging unit. The classification unit classifies the person detected by the detection unit into one type of attribute from among multiple types of attributes that are defined in advance. The determination unit determines, based on predetermined criteria, whether the safety of the person detected by the detection unit is in a first state in which the safety is relatively high or a second state in which the safety is relatively low. The predetermined criteria are defined so that the classification result by the classification unit differs between a specific type of attribute among the plurality of types of attributes and a type of attribute different from the specific type of attribute. The specific type of attribute is, for example, the above-mentioned "station staff" or "person requiring assistance."

[0177] In addition, in the first aspect of this embodiment, the monitoring device may include the detection unit, the classification unit, and the determination unit. The monitoring device is, for example, an information processing device that realizes the functions of the detection unit 20, the classification unit 30, and the determination unit 40 described above.

[0178] In addition, in a first aspect of this embodiment, a monitoring method executed by an information processing device may be realized. The information processing device is, for example, an information processing device that realizes the functions of the above-mentioned detection unit 20, classification unit 30, and determination unit 40. The monitoring method includes a detection step, a classification step, and a determination step. Specifically, in the detection step, a person present on a station platform is detected based on an image acquired by an imaging unit that captures an image of the platform. In addition, in the classification step, the person detected in the detection step is classified into one type of attribute from among multiple types of attributes that are predefined. In addition, in the determination step, it is determined based on a predetermined criterion whether the safety of the person detected in the detection step is in a first state in which the safety of the person is relatively high or in a second state in which the safety of the person is relatively low. The predetermined criterion is defined so that the classification result in the classification step differs between a specific type of attribute from among the multiple types of attributes and a different type of attribute from the specific type of attribute.

[0179] In addition, in the first aspect of this embodiment, a program may be realized that causes an information processing device to execute the above management method, specifically, the detection step, the classification step, and the determination step.

[0180] This allows the monitoring system, monitoring device, and information processing device (hereinafter referred to as "monitoring system, etc.") to monitor the status of people on the platform according to the attributes of the people present on the platform.

[0181] In a second aspect of the present embodiment, based on the first aspect described above, the specific type of attribute may be station staff. The predetermined criterion may be specified such that when the person detected by the detection unit is classified as station staff by the classification unit, the person is less likely to be determined by the determination unit to be in the second state than when the person is classified as another type of attribute different from station staff.

[0182] This allows the monitoring system etc. to monitor the status of people on the platform, including station staff, taking into account the characteristics of the station staff's work on the platform.

[0183] Furthermore, in a third aspect of this embodiment, assuming the second aspect described above, the specified criteria may be criteria relating to the position on the platform of the person detected by the detection unit, and may be specified so that when the detected person is classified as a station employee by the classification unit, the range on the platform corresponding to the first state is wider than when the detected person is classified as another type of attribute different from that of a station employee.

[0184] This allows the monitoring system to monitor the status of people on the platform, including station staff, by allowing them to move around a wide range in response to their work requirements on the platform.

[0185] Furthermore, in a fourth aspect of this embodiment, assuming the third aspect described above, the specified criteria are criteria relating to the positional relationship between the person detected by the detection unit and the railway vehicle entering the platform, and when the person detected by the detection unit is classified as a station employee by the classification unit, the range on the platform corresponding to the first state may be specified to be wider in the direction closer to the railway vehicle than when the person detected by the detection unit is classified as another type of attribute different from station employee.

[0186] This allows the monitoring system, etc., to monitor the status of people on the platform, including station staff, by allowing them to move within a wide range of movement in a direction closer to the railway vehicle in response to the work requirements of station staff on the platform.

[0187] In a fifth aspect of this embodiment, based on the fourth aspect described above, the predetermined criteria may be such that, for an attribute of one of the plurality of types of attributes other than station staff, a first range on the platform corresponding to the first state and a second range on the platform closer to the railway vehicle than the first range corresponding to the second state, and the entire second range for station staff among the plurality of types of attributes corresponds to the first state. The first range and the second range are, for example, the above-described region R1 and region R2, respectively.

[0188] This allows the monitoring system, etc., to monitor the status of people on the platform, including station staff, by allowing them to move within a wide range of movement in a direction closer to the railway vehicle in response to the work requirements of station staff on the platform.

[0189] In a sixth aspect of this embodiment, based on the fourth aspect described above, the predetermined criteria may be such that, for an attribute type other than station staff among the plurality of types of attributes, a first range on the platform corresponding to the first state and a second range on the platform closer to the railcar than the first range corresponding to the second state, and for station staff among the plurality of types of attributes, a portion of the second range that is relatively closer to the railcar corresponds to the second state, and the remaining range corresponds to the first state. The portion of the second range that is relatively closer to the railcar and the remaining range are, for example, the above-described region R4 and region R3, respectively.

[0190] This allows the monitoring system, etc., to improve the safety of station staff while allowing them a wide range of movement in the direction closer to the railway vehicles due to the demands of their work on the platform.

[0191] In a seventh aspect of the present embodiment, assuming any one of the first to sixth aspects described above, the specific type of attribute may include a person requiring assistance, including a person using a white cane or a person using a wheelchair. The predetermined criterion may be defined such that when a person detected by the detection unit is classified as a person requiring assistance by the classification unit, the person is more likely to be determined to be in the second state by the determination unit than when the person is classified as another type different from the person requiring assistance.

[0192] This allows the monitoring system etc. to monitor the status of people in the home, including those requiring care, in accordance with the characteristics of the individuals requiring care.

[0193] Furthermore, in an eighth aspect of this embodiment, assuming any one of the first to seventh aspects described above, a notification unit may be provided that notifies information regarding the determination result of the determination unit to at least one of a crew member of a railway vehicle entering the platform and a station staff member on the platform.

[0194] This allows the monitoring system or the like to notify people on the platform of their level of safety based on their attributes.

[0195] In addition, in a ninth aspect of this embodiment, based on the above-described eighth aspect, the monitoring system or the like may include a recognition unit that recognizes human behavior detected by the detection unit. The recognition unit is, for example, the above-described recognition unit 80. The notification unit may notify at least one of a train crew member of the railway vehicle and a station staff member at the platform of information related to the behavior recognized by the recognition unit.

[0196] This allows the monitoring system, for example, to prompt train crew members or station staff on the platform to take appropriate measures in accordance with the behavior of people on the platform in response to their situation.

[0197] In addition, in a tenth aspect of this embodiment, based on the above-described ninth aspect, the monitoring system or the like may include an estimation unit that estimates a posture state of the person detected by the detection unit based on an image acquired by the imaging unit. The estimation unit is, for example, the above-described estimation unit 70. Then, the recognition unit may recognize the behavior of the person detected by the detection unit based on the estimation result of the estimation unit.

[0198] This allows a monitoring system or the like to recognize a person's behavior based on the person's posture detected on the platform.

[0199] Furthermore, in an eleventh aspect of this embodiment, assuming the above-mentioned tenth aspect, the information regarding the behavior recognized by the recognition unit may be information representing a countermeasure according to the content of the behavior recognized by the recognition unit when the judgment unit judges that the person detected by the detection unit is in the second state.

[0200] This allows the monitoring system, for example, to notify train crew members or station staff on the platform of how to deal with the situation of people on the platform depending on the behavior of the people on the platform.

[0201] Although the embodiments have been described in detail above, the present disclosure is not limited to such specific embodiments, and various modifications and variations are possible within the scope of the gist described in the claims. [Explanation of symbols]

[0202] 1. Surveillance System 10. Imaging unit 20 Detector 30 Classification Department 40 Judgment section 50 Notification Department 60 Display 70 Estimation part 80 Recognition part PF Home ST Station TRN railcars

Claims

1. An imaging unit that captures images of the station platform; a detection unit that detects a person present on the platform based on the image acquired by the imaging unit; a classification unit that classifies the person detected by the detection unit into one type of attribute from among a plurality of types of attributes that are defined in advance; a determination unit that determines whether the safety of the person detected by the detection unit is in a first state in which the safety is relatively high or a second state in which the safety is relatively low, based on a predetermined standard; the predetermined criterion is defined so that a result of classification by the classifier is different between a specific type of attribute among the plurality of types of attributes and a type of attribute different from the specific type of attribute. Surveillance system.

2. the specific type of attribute is station staff; The predetermined criterion is defined so that when the person detected by the detection unit is classified as a station employee by the classification unit, the person is less likely to be determined to be in the second state by the determination unit than when the person is classified as an attribute of another type different from that of a station employee. The monitoring system of claim 1 .

3. the predetermined criterion is a criterion related to the position on the platform of the person detected by the detection unit, and is specified so that when the person detected by the detection unit is classified as a station employee by the classification unit, the range on the platform corresponding to the first state is wider than when the person detected by the detection unit is classified as a station employee by another type of attribute different from that of a station employee; The monitoring system of claim 2 .

4. The predetermined criterion is a criterion regarding a positional relationship between the person detected by the detection unit and a railway vehicle entering the platform, and is specified so that when the person detected by the detection unit is classified as a station employee by the classification unit, the range on the platform corresponding to the first state is wider in a direction closer to the railway vehicle than when the person detected by the detection unit is classified as a station employee by another type of attribute different from that of a station employee. The monitoring system of claim 3 .

5. the predetermined criteria are such that, for a type of attribute other than station staff among the plurality of types of attributes, a first range on the platform corresponding to the first state and a second range on the platform closer to the railway vehicle than the first range corresponding to the second state, and for station staff among the plurality of types of attributes, the entire second range is specified to correspond to the first state; The monitoring system of claim 4.

6. The predetermined criteria are defined such that, for an attribute of a type other than station staff among the plurality of types of attributes, a first range on the platform corresponding to the first state and a second range on the platform closer to the railway vehicle than the first range corresponding to the second state, and for station staff among the plurality of types of attributes, a part of the second range that is relatively closer to the railway vehicle corresponds to the second state, and the remaining range corresponds to the first state. The monitoring system of claim 4.

7. The specific type of attribute includes a person requiring assistance, including a person using a white cane or a person using a wheelchair; The predetermined criterion is defined so that when the person detected by the detection unit is classified as the person requiring care by the classification unit, the person is more likely to be determined to be in the second state by the determination unit than when the person is classified as another type different from the person requiring care. The monitoring system of claim 1 .

8. a notification unit that notifies information about the determination result of the determination unit to at least one of a train crew member of a railway vehicle entering the platform and a station staff member at the platform; A monitoring system according to any one of claims 1 to 7.

9. a recognition unit that recognizes the human behavior detected by the detection unit, The notification unit notifies at least one of a train crew member of the railway vehicle and a station staff member at the platform of information related to the behavior recognized by the recognition unit. The monitoring system of claim 8.

10. an estimation unit that estimates a posture state of the person detected by the detection unit based on the image acquired by the imaging unit, the recognition unit recognizes the behavior of the person detected by the detection unit based on the estimation result of the estimation unit. The monitoring system of claim 9.

11. the information regarding the behavior recognized by the recognition unit is information indicating a countermeasure according to the content of the behavior recognized by the recognition unit when the determination unit determines that the person detected by the detection unit is in the second state; The monitoring system of claim 9.

12. a detection unit that detects a person present on the platform based on an image acquired by an imaging unit that images the platform of the station; a classification unit that classifies the person detected by the detection unit into one type of attribute from among a plurality of types of attributes that are defined in advance; a determination unit that determines whether the safety of the person detected by the detection unit is in a first state in which the safety is relatively high or a second state in which the safety is relatively low, based on a predetermined standard; the predetermined criterion is defined so that a result of classification by the classifier is different between a specific type of attribute among the plurality of types of attributes and a type of attribute different from the specific type of attribute. Monitoring equipment.

13. a detection step in which the information processing device detects a person present on the platform based on an image acquired by an imaging unit that images the platform of the station; a classification step in which the information processing device classifies the person detected in the detection step into one type of attribute from among a plurality of types of attributes defined in advance; a determination step in which the information processing device determines, based on a predetermined criterion, whether the safety of the person detected in the detection step is in a first state in which the safety of the person is relatively high or in a second state in which the safety of the person is relatively low; the predetermined criterion is defined so that a classification result in the classification step differs between a specific type of attribute among the plurality of types of attributes and a type of attribute different from the specific type of attribute. Monitoring method.

14. In the information processing device, a detection step of detecting a person present on the platform based on an image acquired by an imaging unit that images the platform of the station; a classification step of classifying the person detected in the detection step into one type of attribute from among a plurality of types of attributes defined in advance; a determining step of determining whether the safety of the person detected in the detecting step is in a first state in which the safety of the person is relatively high or in a second state in which the safety of the person is relatively low, based on a predetermined criterion; the predetermined criterion is defined so that a classification result in the classification step differs between a specific type of attribute among the plurality of types of attributes and a type of attribute different from the specific type of attribute. program.

Images