Monitoring device, crew member's poor physical condition notification system, and monitoring method

Abstract

To provide a monitoring device capable of improving accuracy of abnormality detection of a crew.SOLUTION: A poor physical condition level determination unit 36 that determines whether or not to determine a poor physical condition level of the crew member 11 using the image information and the operation information, or the operation information, and when determining, determines the poor physical condition level of the crew member 11 based on at least one of the one or more specified face states of the crew member 11 and the operation handling abnormality information, and a notification unit 38 that notifies the crew member 11 of the poor physical condition state.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a monitoring device for monitoring train crew members, a train crew member illness notification system, and a monitoring method. [Background technology]

[0002] In recent years, train operations have been moving toward one-man operation, where only one crew member is on board a train, such as one-man operation and automated operation as specified in GoA (Grade of Automation) 2.5, where the train is operated automatically but the crew member monitors the front. In such a one-man operation, the poor health of a crew member is more likely to affect train operations than in a situation where two or more crew members are on board. However, in a one-man operation, since there is only one crew member on board the train, it is difficult to provide support to the crew member when their health changes.

[0003] For example, one possible method would be to install a camera in the train's driver's cab and constantly monitor the crew remotely from a ground-based control room. However, this would increase the workload of the control room in managing train operations, reducing the benefits of single-operation systems such as one-man operation and automatic operation. Furthermore, constant transmission of video from the train to the ground-based control room would also result in a huge amount of communication traffic. To address these issues, Patent Document 1 discloses technology for automatically detecting whether a train driver is drowsy. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2005-525142 Summary of the Invention [Problem to be solved by the invention]

[0005] The above-mentioned conventional technology detects a driver's drowsiness based on the frequency of the driver's yawning, the state of relaxation of facial muscles, etc., and is an effective method for determining whether a driver is drowsy. However, even if a driver is drowsy, there may be cases where symptoms of drowsiness are not readily apparent due to individual differences. Therefore, there is a problem that drowsiness cannot be detected or is detected late for drivers who do not readily exhibit symptoms.

[0006] The present disclosure has been made in view of the above, and aims to provide a monitoring device that can improve the accuracy of detecting abnormalities in crew members. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the objectives, the monitoring device of the present disclosure is characterized by comprising: a facial state detection unit that detects one or more specified facial states of the crew using the crew's face and specified facial parts contained in the face detected from image information obtained by a camera that photographs the train crew; a driving operation abnormality calculation unit that calculates abnormalities in the crew's driving operation from train operation information obtained from an external device that monitors and controls the train's state and outputs driving operation abnormality information; a health condition level determination unit that uses the image information and operation information, or the operation information, to determine whether or not to determine the crew's health condition level, and when it is determined that the crew's health condition level should be determined, determines the crew's health condition level based on at least one of the one or more specified facial states of the crew detected by the facial state detection unit and the driving operation abnormality information; and a notification unit that notifies the crew of the crew's health condition according to the content and level of the crew's health condition. [Effects of the Invention]

[0008] According to the present disclosure, the monitoring device has the effect of being able to improve the accuracy of detecting abnormalities in crew members. [Brief explanation of the drawings]

[0009] [Figure 1]FIG. 1 is a diagram showing an example of the configuration of a driver's health condition notification system according to a first embodiment. [Figure 2] Flowchart showing the operation of the crew member illness notification system according to the first embodiment [Figure 3] A flowchart showing the operation of determining whether or not to determine the level of poor health of a crew member in the poor health level determination unit of the crew member poor health notification system according to the first embodiment. [Figure 4] FIG. 1 is a diagram showing an example of the configuration of a processing circuit in a case where the processing circuit of the monitoring device according to the first embodiment is realized by a processor and a memory. [Figure 5] FIG. 1 is a diagram showing an example of the configuration of a processing circuit in a case where the processing circuit of the monitoring device according to the first embodiment is realized by dedicated hardware; [Figure 6] FIG. 10 is a diagram showing a configuration example of a driver's health condition notification system according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] A monitoring device, a crew member illness notification system, and a monitoring method according to embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0011] Embodiment 1 FIG. 1 is a diagram illustrating an example of the configuration of a crew member illness notification system 70 according to the first embodiment. The crew member illness notification system 70 includes a monitoring device 30 and a sickness notification unit 61 included in a ground system 60. In the crew member illness notification system 70, when the monitoring device 30 detects that a crew member 11 of a train 10 is in poor health, the monitoring device 30 notifies the crew member 11 of the poor health state and notifies the sickness notification unit 61 of the poor health state of the crew member 11 according to the content and level of the crew member 11's poor health. When the monitoring device 30 notifies the crew member 11 of the poor health of the train 10, the sickness notification unit 61 notifies the train operation command 51 of a ground operation control room 50 included in the ground system 60 that the crew member 11 of the train 10 has become ill. The poor health of the crew member 11 includes cases where the crew member is actually ill due to illness, as well as cases where the crew member is drowsy or distracted, which may cause disruption to the operation of the train 10.

[0012] The train 10 is equipped with a monitoring system 20. The monitoring system 20 is equipped with an external device 21 and a monitoring device 30. The external device 21 monitors and controls the state of the train 10, the state of on-board equipment (not shown) mounted on the train 10, etc. The external device 21 outputs operation information of the train 10 to the monitoring device 30. The operation information of the train 10 includes, for example, the speed information of the train 10, the position information of the train 10, running test information indicating whether the train 10 is undergoing a running test, yard running information indicating whether the train 10 is running within the yard, deadhead information indicating whether the train 10 is being operated, door open information indicating whether the doors of the train 10 are open, station stop mispassing prevention function operation status information regarding the operation status of the station stop mispassing prevention function that warns the crew 11 by voice or display to prevent the train from passing through a station, stop prohibition information indicating sections where the train 10 is prohibited from stopping, such as air section sections or tunnel sections, notification information indicating speed restrictions or operation cancellations of the train 10, slow operation information indicating whether the train 10 is running slowly, and other information may also be included. The external device 21 is, for example, a TCMS (Train Control Management System), but is not limited to this.

[0013] The following describes the configuration and operation of the monitoring device 30. As shown in Fig. 1, the monitoring device 30 includes a camera 31, an image conversion processing unit 32, a face and facial part detection unit 33, a facial state detection unit 34, a communication unit 35, a poor physical condition level determination unit 36, a storage unit 37, a notification unit 38, a driving behavior abnormality calculation unit 41, a vehicle control information generation unit 42, and a ground-to-vehicle communication unit 43.

[0014] The camera 31 photographs the crew 11 of the train 10. The camera 31 outputs image information obtained by photographing the crew 11 of the train 10 to the image conversion processing unit 32. Note that, although the monitoring device 30 is equipped with the camera 31 in the example of FIG. 1, the camera 31 may be located outside the monitoring device 30.

[0015] The image conversion processing unit 32 converts the image information acquired from the camera 31, for example, from a color image to a grayscale image. The image conversion processing unit 32 outputs the converted grayscale image to the face and facial part detection unit 33. Note that the monitoring device 30 can be configured without the image conversion processing unit 32 if the image information of the color image acquired by the camera 31 can be used in the face and facial part detection unit 33 (described later) or the like.

[0016] The face and facial part detection unit 33 detects the face of the crew member 11 and specified facial parts included in the face of the crew member 11 from the grayscale image acquired from the image conversion processing unit 32. The specified facial parts include, for example, the eyes and mouth. The detected face and facial parts can be identified by expressing them as coordinates relative to the original grayscale image. The method for detecting the face and facial parts in the face and facial part detection unit 33 can be a method used in general face authentication, but is not particularly limited. If the monitoring device 30 does not include the image conversion processing unit 32, the face and facial part detection unit 33 will detect the face of the crew member 11 and specified facial parts included in the face of the crew member 11 from the image information acquired by the camera 31. The face and facial part detection unit 33 outputs information on the detected face and facial parts to the face state detection unit 34.

[0017] The facial state detection unit 34 detects one or more prescribed facial states of the crew member 11 by using the face of the crew member 11 and prescribed facial features included in the face detected from image information obtained by the camera 31 that photographs the crew member 11 of the train 10. For example, the facial state detection unit 34 detects the degree to which the crew member 11's eyes are open, the facial direction of the crew member 11, the number of times the crew member 11 blinks, the line of sight of the crew member 11, the degree to which the crew member 11 opens their mouths, the facial muscles of the crew member 11, the brightness of the crew member's skin, etc. by using the face of the crew member 11 and facial features of the crew member 11 obtained from the face and facial feature detection unit 33.

[0018] The degree of eye opening of the crew member 11 indicates the percentage of time during a specified period when the crew member 11 has their eyes open. When the crew member 11 becomes sleepy or unwell, it is considered that the degree of eye opening of the crew member 11 becomes less than usual.

[0019] The facial orientation of the crew member 11 indicates the proportion of facial orientations of the crew member 11 during a specified period. Normally, the crew member 11 spends most of the time facing forward while operating the train 10, but if the crew member 11 becomes sleepy or unwell, he or she may assume a hunched posture with his or her head turned downward.

[0020] The blink count of the crew member 11 indicates the number of blinks of the crew member 11 in a specified period. When the crew member 11 is feeling sleepy but trying to stay awake, it is thought that the crew member 11 will blink more frequently than usual.

[0021] The line of sight of the crew member 11 indicates the movement of the line of sight of the crew member 11. It is considered that the movement of the line of sight of the crew member 11 becomes slower than normal when the crew member 11 becomes sleepy or in poor physical condition.

[0022] The mouth opening degree of the crew member 11 indicates the percentage of time during a specified period when the crew member 11 has their mouth open. If the crew member 11 becomes sleepy or unwell, it is considered that the mouth opening degree of the crew member 11 becomes higher than usual.

[0023] The facial muscles of the crew member 11 indicate the movement of the facial muscles of the crew member 11. When the crew member 11 becomes sleepy or in poor health, it is thought that the crew member 11 will make different movements of the facial muscles than usual, such as furrowing the brow.

[0024] The brightness of the skin of the crew member 11 indicates the brightness of the skin of the crew member 11. When the crew member 11 becomes sleepy or unwell, it is considered that the change in the brightness of the skin of the crew member 11 is different from normal due to trembling of the face, whole body, etc.

[0025] The facial state detection unit 34 outputs the detected eye opening degree of the crew member 11, the facial direction of the crew member 11, the number of blinks of the crew member 11, the line of sight of the crew member 11, the degree of mouth opening of the crew member 11, the facial muscles of the crew member 11, and the brightness of the skin of the crew member 11 to the poor physical condition level determination unit 36. Note that the facial state detection unit 34 may be configured to detect at least one of the eye opening degree of the crew member 11, the facial direction of the crew member 11, the number of blinks of the crew member 11, the line of sight of the crew member 11, the degree of mouth opening of the crew member 11, the facial muscles of the crew member 11, and the brightness of the skin of the crew member 11. When the facial condition detection unit 34 detects at least one of the degree of eye opening of the crew member 11, the facial direction of the crew member 11, the number of blinks of the crew member 11, the line of sight of the crew member 11, the degree of mouth opening of the crew member 11, the facial muscles of the crew member 11, and the brightness of the crew member 11's skin, it only needs to output the detected amount to the physical condition level determination unit 36.

[0026] The communication unit 35 acquires operation information of the train 10 from the external device 21. The communication unit 35 communicates with the external device 21 using a communication method such as Ethernet (registered trademark) or RS (Recommended Standards) 485-HDLC (High-level Data Link Control). The communication unit 35 outputs the acquired operation information of the train 10 to the operation handling abnormality calculation unit 41.

[0027] The driving abnormality calculation unit 41 acquires operation information of the train 10 from an external device 21 that monitors and controls the status of the train 10. The driving abnormality calculation unit 41 calculates driving abnormalities of the crew 11 of the train 10 from the acquired operation information. For example, the driving abnormality calculation unit 41 calculates at least one of the following as driving abnormalities of the crew 11 of the train 10: the frequency of ATC (Automatic Train Control) braking; the period when the notch remains in a deceleration state for a specified period after the train 10 arrives at a station; the period when the doors do not open for a specified period after the train 10 arrives at a station; the frequency at which the speed of the train 10 deviates from the standard value of the run curve; and the degree of deviation from the target stopping position when the train 10 stops. The driving abnormality calculation unit 41 outputs the calculated driving abnormality of the crew 11 of the train 10 to the physical condition level determination unit 36 ​​as driving abnormality information.

[0028] The poor health level determination unit 36 ​​acquires the image information used by the face and facial part detection unit 33, and acquires operation information of the train 10 acquired by the communication unit 35 and used by the operation handling abnormality calculation unit 41 from the operation handling abnormality calculation unit 41. The image information used by the face and facial part detection unit 33 is a grayscale image obtained by the image conversion processing unit 32 if the monitoring device 30 is equipped with the image conversion processing unit 32, or is image information of a color image obtained by the camera 31 if the monitoring device 30 is not equipped with the image conversion processing unit 32. The poor health level determination unit 36 ​​may acquire the image information from the face and facial part detection unit 33 via the facial state detection unit 34, or may acquire the image information directly from the camera 31, the image conversion processing unit 32, or the face and facial part detection unit 33, although connections are not shown in FIG. 1 .

[0029] The poor health level determination unit 36 ​​uses the image information and the operation information of the train 10, or the operation information of the train 10, to determine whether or not to determine the poor health level of the crew member 11. As described above, the operation information of the train 10 acquired from the communication unit 35 via the operation handling abnormality calculation unit 41 includes information such as speed information, position information, running test information, yard running information, deadheading information, door opening information, station stopping train erroneous passing prevention function operation status information, stop prohibition information, notice information, and slow driving information. In addition, the image information used by the face and facial part detection unit 33 can determine whether or not the crew member 11 is driving in a standing position and whether or not an instructor is driving together with the crew member 11.

[0030] Therefore, the health condition level determination unit 36 ​​determines not to determine the health condition level of the crew member 11 when at least one of the following applies: the train 10 is undergoing a running test, the train 10 is running within the yard, the train 10 is in out-of-service operation, the train 10 has a station stopping and mistaken passing prevention function activated, the train 10 is running in a no-stopping section, the train 10 is rescuing another train, the train 10 is slowing down, the train 10 is receiving notification information, the crew member 11 is driving in a standing position, and an instructor is present in the driver's cab of the train 10 with the crew member 11. In other words, the health condition level determination unit 36 ​​determines to determine the health condition level when all of the following conditions are not met: the train 10 is undergoing a running test, the train 10 is running within the yard, the train 10 is in out-of-service operation, the train 10 has a station stopping and mistaken passing prevention function in operation, the train 10 is running in a no-stopping section, the train 10 is rescuing another train, the train 10 is slowing down, the train 10 is receiving notification information, the crew 11 is driving in a standing position, and an instructor is present in the driver's cab of the train 10 together with the crew 11.

[0031] The above conditions under which the poor health level determination unit 36 ​​determines the poor health level are merely examples and are not limiting. The poor health level determination unit 36 ​​can also determine whether or not to determine the poor health level based on conditions other than the above conditions. The poor health level determination unit 36 ​​can also not use some of the above conditions. As a result, the poor health level determination unit 36 ​​can determine whether or not to determine the poor health level of the crew member 11 using operation information of the train 10 acquired by the communication unit 35 and via the operation handling abnormality calculation unit 41, without using the image information used by the face and facial part detection unit 33.

[0032] The reason why the poor health level determination unit 36 ​​makes the above-mentioned determination is that even if it determines that the crew member 11 has become ill through a determination of the poor health level, the crew member 11 will not be notified of the poor health in the above-mentioned situation. Since it is meaningless to determine the poor health level of the crew member 11 in a situation where a poor health notification will not be made, the poor health level determination unit 36 ​​does not determine the poor health level from the start in a situation where a poor health notification will not be made.

[0033] When it is determined that the poor health level of the crew member 11 should be determined, the poor health level determination unit 36 ​​determines the poor health level of the crew member 11 based on at least one of the one or more specified facial states of the crew member 11 detected by the facial state detection unit 34 and the driving abnormality information calculated by the driving abnormality calculation unit 41. For example, when it is determined that the poor health level of the crew member 11 should be determined, the poor health level determination unit 36 ​​determines the poor health level of the crew member 11 based on at least one of the degree of eye opening of the crew member 11, the facial direction of the crew member 11, the number of blinks of the crew member 11, the line of sight of the crew member 11, the degree of mouth opening of the crew member 11, the facial muscles of the crew member 11, the brightness of the skin of the crew member 11, and the driving abnormality information.

[0034] For example, the poor physical condition level determination unit 36 ​​may use the degree of eye opening of the crew member 11 to determine whether the crew member 11 has their eyes open for a specified period of time, use the facial direction of the crew member 11 to determine the ratio of the facial direction of the crew member 11 for a specified period of time, use the number of blinks of the crew member 11 to determine whether the number of blinks of the crew member 11 for a specified period of time is equal to or greater than a threshold, use the line of sight of the crew member 11 to determine whether the movement of the line of sight of the crew member 11 has stopped, use the degree of opening of the mouth of the crew member 11 to determine whether the crew member 11 has yawned a specified number of times or more for a specified period of time, and use the facial muscles of the crew member 11 to determine whether the distance between the eyebrows of the crew member 11 is between the eyebrows of the crew member 11. The poor physical condition level of the crew member 11 is determined based on at least one of the following: whether the condition of the wrinkles on the face is in a specified state; whether the crew member 11 has coughed a specified number of times or more in a specified period using the degree of mouth opening of the crew member 11 and the facial muscles of the crew member 11; whether the change in the brightness of the crew member 11's skin has changed a specified number of times or more in a specified period using the brightness of the crew member 11's skin; whether there has been a delay in opening the doors when the train stopped at a station using the abnormal driving operation information; and whether there has been an increase in the frequency of speeding using the abnormal driving operation information. However, it is considered that the poor physical condition level determination unit 36 ​​can more accurately determine the poor physical condition level of the crew member 11 if it makes a determination using multiple items.

[0035] The poor health level determination unit 36 ​​also stores in the memory unit 37 the determination result that the crew member 11 has become ill in the poor health level determination, and the information used when the crew member 11 has become ill in the poor health level determination. The poor health level determination unit 36 ​​stores a specified number of determination results and information in the memory unit 37, and overwrites determination results and information older than the specified number to update them with the latest determination results and information. The specified number is, for example, 30 times. The poor health level determination unit 36 ​​stores the latest 30 determination results and information in the memory unit 37, and overwrites and updates the oldest determination results and information. Taking memory corruption into consideration in the memory unit 37, the poor health level determination unit 36 ​​calculates a CRC (Cyclic Redundancy Check) or checksum value for the determination results and information to be stored in the memory unit 37 to ensure the validity of the data. The poor physical condition level determination unit 36 ​​causes the determination result and information stored in the memory unit 37 to be transmitted from the ground-to-vehicle communication unit 43 to the operation control room 50 that manages the operation of the train 10.

[0036] Furthermore, the ill-health level determination unit 36 ​​determines, based on the details of the ill health of the crew member 11 and the level of the ill health of the crew member 11, whether or not to output at least one of the details of the ill health of the crew member 11, the level of the ill health of the crew member 11, and image information when the crew member 11 was ill to the operation control room 50 that manages the operation of the train 10. When the ill-health level of the crew member 11 is below a specified level, the ill-health level determination unit 36 ​​causes the notification unit 38 to notify the crew member 11 of their ill health, and when the ill-health level of the crew member 11 is equal to or higher than the specified level, causes the notification unit 38 to notify the crew member 11 of their ill health, and outputs at least one of the details of the ill health of the crew member 11, the level of the ill health of the crew member 11, and image information when the crew member 11 was ill to the operation control room 50.

[0037] The poor health level determination unit 36 ​​stores in advance a specified level to be compared with the poor health level for each item, such as the eye opening degree of the crew member 11. The poor health level determination unit 36 ​​may determine that the crew member 11 has become ill when the poor health level for one item exceeds the specified level, or may determine that the crew member 11 has become ill when no item exceeds the specified level but multiple items detect poor health levels close to the specified levels.

[0038] In the first embodiment, when the ill health level determination unit 36 ​​determines that at least one of the details of the ill health of the crew member 11, the level of the ill health of the crew member 11, and image information when the crew member 11 was ill should be output to the operation control room 50, it outputs at least one of the details of the ill health of the crew member 11, the level of the ill health of the crew member 11, and image information when the crew member 11 was ill to the ground-to-vehicle communication unit 43. The ill health level determination unit 36 ​​causes the ground-to-vehicle communication unit 43 to transmit at least one of the details of the ill health of the crew member 11, the level of the ill health of the crew member 11, and image information when the crew member 11 was ill to the operation control room 50.

[0039] In Patent Document 1 described in the Background Art section, a driver's drowsiness is detected based on the frequency of the driver's yawning, the state of facial muscle relaxation, and the like. However, there is a problem in that drowsiness cannot be detected or is detected late for drivers who are less likely to exhibit such symptoms due to individual differences. Here, when a crew member 11 becomes drowsy or falls into poor physical condition, it is generally considered that their attention level decreases compared to when they are not drowsy and in good physical condition. For example, in the operation of a train 10, the train's running speed in each running section, the stopping position of the train 10 when stopping at a station, and other such conditions are often specified. While these operations are normally possible for the crew member 11 to perform, a crew member 11 who is less attentive than usual due to drowsiness or poor physical condition is more likely to perform operations that deviate from these specifications. Therefore, in embodiment 1, the poor health level determination unit 36 ​​improves the accuracy of detecting abnormalities in the crew member 11, i.e., detecting poor health, by utilizing one or more specified facial states of the crew member 11 detected by the facial state detection unit 34 using image information of the crew member 11, and the abnormal driving operation information calculated by the abnormal driving operation calculation unit 41 using the operating status of the train 10.

[0040] The storage unit 37 stores the result of the determination made by the health condition level determination unit 36 ​​that the crew member 11 has become ill in the determination of the health condition level, and information used when the health condition level determination unit 36 ​​determined that the crew member 11 has become ill in the determination of the health condition level. The storage unit 37 is, for example, a ring buffer. Note that the amount of data of the determination results and information that can be stored in the storage unit 37 is set to 30 determination results and information in the above example, but this can be changed as appropriate depending on the capacity of the storage unit 37, and the storage unit 37 can also store more than 30 determination results and information. Furthermore, although the monitoring device 30 in the example of FIG. 1 includes the storage unit 37, the storage unit 37 may be external to the monitoring device 30.

[0041] The notification unit 38 notifies the crew member 11 of the state of poor health in accordance with the content of the crew member's poor health and the crew member's level of poor health under the control of the poor health level determination unit 36. For example, the notification unit 38 notifies the crew member 11 of the state of poor health by voice in the case of a speaker, or by text in the case of a display device such as a monitor, but is not limited to these. The notification unit 38 may include both a speaker and a display device. The notification unit 38 can change the content of the notification in accordance with the content of the crew member's poor health and the crew member's level of poor health under the control of the poor health level determination unit 36. For example, when the content of the crew member's poor health and the crew member's level of poor health are mild, the notification unit 38 simply notifies the crew member 11 of the state of poor health, and the notification becomes more intense as the content of the crew member's poor health and the crew member's level of poor health become more severe.

[0042] When the ill-health level determination unit 36 ​​determines that at least one of the details of the ill health of the crew member 11, the ill-health level of the crew member 11, and image information when the crew member 11 was ill should be output to the operation control room 50, the ground-to-vehicle communication unit 43 acquires at least one of the details of the ill health of the crew member 11, the ill-health level of the crew member 11, and image information when the crew member 11 was ill from the ill-health level determination unit 36. The ground-to-vehicle communication unit 43 transmits at least one of the acquired details of the ill health of the crew member 11, the ill-health level of the crew member 11, and image information when the crew member 11 was ill to the operation control room 50. The ground-to-vehicle communication unit 43 transmits all of the information acquired from the ill-health level determination unit 36 ​​to the operation control room 50.

[0043] The vehicle control information generation unit 42 generates vehicle control information to stop the train 10 or to instruct the train 10 to continue running, based on the details of the poor health of the crew member 11 and the level of the poor health of the crew member 11 determined by the poor health level determination unit 36, and transmits this information to the external device 21. The details of the poor health of the crew member 11 and the level of the poor health of the crew member 11 that can be determined by the poor health level determination unit 36 ​​of the monitoring device 30 are assumed in advance because the number of determination items is limited. Therefore, the vehicle control information generation unit 42 may previously store corresponding vehicle control information depending on the details of the poor health of the crew member 11 and the level of the poor health of the crew member 11 determined by the poor health level determination unit 36.

[0044] In the operation control room 50 that manages the operation of the train 10, the sickness notification unit 61 included in the ground system 60 receives at least one of the details of the sickness of the crew member 11 of the train 10, the level of the sickness of the crew member 11, and image information of the crew member 11 when he / she is sick, directly from the monitoring device 30. The sickness notification unit 61 notifies the operation command 51 of at least one of the details of the sickness of the crew member 11 of the train 10, the level of the sickness of the crew member 11, and image information of the crew member 11 when he / she is sick. The sickness notification unit 61 may include a display unit that can display the image information and a speaker that can output audio. Similar to the notification unit 38, the sickness notification unit 61 can change the notification content depending on the details of the sickness of the crew member 11 and the level of the sickness of the crew member 11, under the control of the sickness level determination unit 36.

[0045] 2 is a flowchart showing the operation of the crew member sickness notification system 70 according to the first embodiment. In the crew member sickness notification system 70, the image conversion processing unit 32 acquires image information from the camera 31 (step S101). The image conversion processing unit 32 converts the image information acquired from the camera 31 into a grayscale image (step S102). The face and facial part detection unit 33 detects the face of the crew member 11 and specified facial parts included in the face of the crew member 11 from the grayscale image acquired from the image conversion processing unit 32 (step S103). The facial state detection unit 34 detects one or more specified facial states of the crew member 11 using the face and facial parts of the crew member 11 acquired from the face and facial part detection unit 33 (step S104).

[0046] The communication unit 35 acquires operation information of the train 10 from the external device 21 (step S105). The driving abnormality calculation unit 41 acquires the operation information of the train 10 from the external device 21 and calculates an abnormality in the driving of the crew 11 of the train 10 from the acquired operation information (step S106). The driving abnormality calculation unit 41 outputs the acquired operation information of the train 10 and the calculated driving abnormality information indicating an abnormality in the driving of the crew 11 of the train 10 to the physical condition level determination unit 36. The physical condition level determination unit 36 ​​uses the image information used by the face and facial part detection unit 33 and the operation information of the train 10 acquired from the communication unit 35 and the driving abnormality calculation unit 41 to determine whether or not to determine the physical condition level of the crew 11 (step S107).

[0047] The operation of step S107 will now be described in detail. Fig. 3 is a flowchart showing the operation of determining whether or not to determine the level of illness of the crew member 11 in the illness level determination unit 36 ​​of the crew member illness notification system 70 according to the first embodiment. The illness level determination unit 36 ​​determines whether or not to determine the level of illness of the crew member 11 when the train 10 is undergoing a running test (step S201: Yes), when the train 10 is running within the premises (step S202: Yes), when the train 10 is in a deadhead operation (step S203: Yes), when the train 10 is in operation to stop at a station (step S204: Yes), when the train 10 is running in a no-stop section (step S205: Yes), or when the train 10 is in operation to stop at another train If a vehicle is being rescued (step S206: Yes), or if the train 10 is moving slowly (step S207: Yes), or if the train 10 is receiving notification information (step S208: Yes), or if the crew member 11 is driving in a standing position (step S209: Yes), or if an instructor is present in the driver's cab of the train 10 with the crew member 11 (step S210: Yes), it is decided not to judge the level of illness (step S211).

[0048] The physical condition level determination unit 36 ​​determines to determine the physical condition level (step S212) if the train 10 is not undergoing a running test (step S201: No), the train 10 is not running within the premises (step S202: No), the train 10 is not in a deadhead operation (step S203: No), the train 10 is not operating a function to prevent accidental stopping at a station (step S204: No), the train 10 is not running in a no-stop section (step S205: No), the train 10 is not rescuing another train (step S206: No), the train 10 is not moving slowly (step S207: No), the train 10 is not receiving notification information (step S208: No), the crew member 11 is not driving in a standing position (step S209: No), and there is no instructor in the driver's cab of the train 10 (step S210: No).

[0049] Returning to the explanation of FIG. 2, if the crew member illness notification system 70 does not determine the level of illness of the crew member 11 (step S107: No), it returns to step S101 and step S105 and performs the above-mentioned operations. The case of step S107: No corresponds to step S211 in the flowchart shown in FIG. 3. If the crew member illness notification system 70 determines the level of illness of the crew member 11 (step S107: Yes), it determines the level of illness of the crew member 11 (step S108). The case of step S107: Yes corresponds to step S212 in the flowchart shown in FIG. 3. As described above, the illness level determination unit 36 ​​determines the level of illness of the crew member 11 based on at least one of the one or more specified facial states of the crew member 11 detected by the facial state detection unit 34 and the driving abnormality information calculated by the driving abnormality calculation unit 41.

[0050] If it is determined that the crew member 11 has become ill (step S109: Yes), the ill-health level determination unit 36 ​​notifies the crew member 11 of the state of ill health (step S110). Based on the details of the ill health of the crew member 11 and the level of the ill health of the crew member 11, the ill-health level determination unit 36 ​​determines whether or not to output at least one of the details of the ill health of the crew member 11, the level of the ill health of the crew member 11, and image information when the crew member 11 was ill to the operation control room 50 that manages the operation of the train 10 (step S111). If the ill-health level determination unit 36 ​​performs the operation of step S110 of notifying the crew member 11 of their ill-health state multiple times, but the details of the ill-health of the crew member 11 and the level of the ill-health of the crew member 11 do not improve, the unit may determine to output at least one of the details of the ill-health of the crew member 11, the level of the ill-health of the crew member 11, and image information of the crew member 11 when he / she was ill to the operation control room 50 that manages the operation of the train 10.

[0051] When the illness level determination unit 36 ​​outputs at least one of the details of the illness of the crew member 11, the illness level of the crew member 11, and image information when the crew member 11 was unwell to the operation control room 50 that manages the operation of the train 10 (step S111: Yes), it outputs at least one of the details of the illness of the crew member 11, the illness level of the crew member 11, and image information when the crew member 11 was unwell to the operation control room 50 via the land-to-vehicle communication unit 43 (step S112). In the operation control room 50, the illness notification unit 61 of the ground system 60 notifies the operation command 51 of at least one of the details of the illness of the crew member 11 of the train 10, the illness level of the crew member 11, and image information when the crew member 11 was unwell that it received (step S113).

[0052] The poor health level determination unit 36 ​​ends the operation of the flowchart shown in Fig. 2 if it determines that the crew member 11 is not ill (step S109: No), or if it does not output at least one of the details of the crew member 11's ill health, the level of the crew member 11's ill health, and image information when the crew member 11 was ill to the operation control room 50 that manages the operation of the train 10 (step S111: No), or after step S113. The crew member ill health notification system 70 periodically repeats the operation of the flowchart shown in Fig. 2.

[0053] Next, the hardware configuration of the monitoring device 30 included in the driver's poor health notification system 70 according to the first embodiment will be described. In the monitoring device 30, the camera 31 is a camera capable of capturing an image of the driver 11. The communication unit 35 is a communication interface capable of communicating with the external device 21. The ground-to-vehicle communication unit 43 is a communication interface capable of communicating with the operation control room 50 installed on the ground. The storage unit 37 is a memory. The notification unit 38 is a display device, speaker, etc. that notifies the driver 11. The image conversion processing unit 32, the face and facial feature detection unit 33, the facial state detection unit 34, the poor health level determination unit 36, the driving handling abnormality calculation unit 41, and the vehicle control information generation unit 42 are realized by processing circuits. The processing circuit may be a memory that stores a program and a processor that executes the program stored in the memory, or may be dedicated hardware. The processing circuit is also called a control circuit.

[0054] FIG. 4 is a diagram illustrating an example of the configuration of the processing circuit 90 of the monitoring device 30 according to the first embodiment, when the processing circuit is realized by a processor 91 and a memory 92. The processing circuit 90 illustrated in FIG. 4 is a control circuit and includes a processor 91 and a memory 92. When the processing circuit 90 is configured with the processor 91 and the memory 92, each function of the processing circuit 90 is realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 92. In the processing circuit 90, each function is realized by the processor 91 reading and executing the program stored in the memory 92. That is, the processing circuit 90 includes the memory 92 for storing a program that results in the processing of the monitoring device 30 being executed. This program can also be said to be a program that causes the monitoring device 30 to execute each function realized by the processing circuit 90. This program may be provided by a storage medium on which the program is stored, or by other means such as a communication medium.

[0055] The program includes a facial state detection step in which a facial state detection unit 34 detects one or more prescribed facial states of the crew member 11 using the face of the crew member 11 and prescribed facial parts included in the face detected from image information obtained by a camera 31 that photographs the crew member 11 of the train 10; a driving operation abnormality calculation step in which a driving operation abnormality calculation unit 41 calculates an abnormality in the driving operation of the crew member 11 from operation information of the train 10 acquired from an external device 21 that monitors and controls the state of the train 10, and outputs driving operation abnormality information; and a physical condition poor health level determination unit 36 ​​uses the image information and the operation information, or the operation information, to determine whether or not to determine the physical condition poor health level of the crew member 11, and when it is determined that the physical condition poor health level of the crew member 11 should be determined, performs the facial state detection. It can also be said that this is a program that causes the monitoring device 30 to execute the following steps: a health condition level determination step in which the monitoring device 30 determines the health condition level of the crew member 11 based on at least one of the specified one or more facial conditions of the crew member 11 detected by the output unit 34 and the abnormal driving handling information, and determines whether or not to output at least one of the health condition details of the crew member 11, the health condition level of the crew member 11, and image information of the crew member 11 when he / she is ill, to the operation control room 50 that manages the operation of the train 10, based on the health condition details and the health condition level of the crew member 11; and a notification step in which the notification unit 38 notifies the crew member 11 of his / her ill condition according to the health condition details and the health condition level of the crew member 11.

[0056] Here, the processor 91 is, for example, a CPU (Central Processing Unit), a processing device, an arithmetic device, a microprocessor, a microcomputer, or a DSP (Digital Signal Processor), etc. Furthermore, the memory 92 is, for example, a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable ROM), or an EEPROM (registered trademark) (Electrically EPROM), a magnetic disk, a flexible disk, an optical disk, a compact disk, a minidisk, or a DVD (Digital Versatile Disc).

[0057] FIG. 5 is a diagram showing an example of the configuration of the processing circuit 93 of the monitoring device 30 according to the first embodiment when the processing circuit is realized by dedicated hardware. The processing circuit 93 shown in FIG. 5 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The processing circuit 93 may be partially realized by dedicated hardware and partially realized by software or firmware. In this way, the processing circuit 93 can realize each of the above-described functions by dedicated hardware, software, firmware, or a combination thereof.

[0058] As described above, according to this embodiment, in the monitoring device 30, the poor health level determination unit 36 ​​determines whether to determine the poor health level of the crew member 11 of the train 10. When determining the poor health level of the crew member 11 of the train 10, the poor health level of the crew member 11 of the train 10 is determined using one or more specified facial states of the crew member 11 detected by the facial state detection unit 34 using image information of the crew member 11, and driving abnormality information calculated by the driving abnormality calculation unit 41 using the operating status of the train 10. In this way, the poor health level determination unit 36 ​​uses information on the operability of the crew member 11 obtained from the driving abnormality calculation unit 41 in addition to information on the appearance of the crew member 11 obtained from the facial state detection unit 34. This allows the poor health level determination unit 36 ​​to detect the poor health of the crew member 11, i.e., improve the accuracy of detecting abnormalities in the crew member 11, compared to when only one of these is used.

[0059] Furthermore, when the monitoring device 30 detects that the crew member 11 is unwell, it notifies the crew member 11 and also notifies the ground operation control room 50 that manages the operation of the train 10 of the crew member 11's unwell health, depending on the details and level of the crew member 11's unwell health. Specifically, when the details and level of the crew member 11's unwell health are mild, the monitoring device 30 does not notify the ground operation control room 50, but when the details and level of the crew member 11's unwell health are severe, the monitoring device 30 notifies the ground operation control room 50. In this way, the monitoring device 30 changes the details of the notification to the crew member 11 and the operation command 51 of the operation control room 50, depending on the details and level of the crew member 11's unwell health. For example, if the crew member 11 has a minor abnormality, the monitoring device 30 objectively notifies only the crew member 11 of his or her condition, allowing the crew member 11 to decide how to respond. As a result, compared to a case in which the monitoring device 30 constantly notifies the operation control room 50 regardless of the nature of the crew member's ill health and the level of the crew member's ill health, the workload of the operation commander 51 in the operation control room 50 can be reduced, and the transmission capacity from the train 10 to the operation control room 50 on the ground can be reduced. The crew member ill-health notification system 70 can provide appropriate support to the crew member 11 without imposing a burden on the workload of the operation commander 51 in the operation control room 50, thereby realizing good one-man operation, automatic operation, etc.

[0060] Furthermore, the monitoring device 30 transmits vehicle control information from the vehicle control information generation unit 42 to the external device 21 in accordance with the details of the crew member 11's poor health and the level of the crew member 11's poor health. As a result, the monitoring device 30 can automatically stop or run the train 10 even when it becomes difficult for the crew member 11 to operate the train 10.

[0061] In addition, the crew member illness notification system 70 serves as a substitute for a deadman device that can detect the state of the crew member 11 before they die or become unconscious, so the crew member 11 does not need to keep holding the deadman switch, which is a deadman device.

[0062] Embodiment 2 In the first embodiment, the monitoring device 30 includes the ground-to-vehicle communication unit 43. In the second embodiment, a case where the monitoring device 30 does not include the ground-to-vehicle communication unit 43 will be described.

[0063] FIG. 6 is a diagram illustrating an example of the configuration of a driver sickness notification system 70 according to the second embodiment. As described above, the driver sickness notification system 70 according to the second embodiment includes a ground-to-vehicle communication unit 43 outside the monitoring device 30 within the monitoring system 20. In the example of FIG. 6, the ground-to-vehicle communication unit 43 is connected to the external device 21. However, the ground-to-vehicle communication unit 43 may be located inside the external device 21, i.e., the external device 21 may include the ground-to-vehicle communication unit 43. Generally, the external device 21 often has the function of wireless communication with a ground-based operation control room 50. Therefore, when the ground-to-vehicle communication unit 43 is connected to the external device 21 or when the external device 21 includes the ground-to-vehicle communication unit 43, the monitoring device 30 uses the external ground-to-vehicle communication unit 43. As a result, the monitoring device 30 does not need to include the ground-to-vehicle communication unit 43, and therefore the configuration can be simplified compared to the monitoring device 30 according to the first embodiment shown in FIG. 1.

[0064] That is, in the second embodiment, when the poor health level determination unit 36 ​​determines that at least one of the details of the poor health of the crew member 11, the level of poor health of the crew member 11, and image information when the crew member 11 was unwell should be output to the operation control room 50, the poor health level determination unit 36 ​​outputs at least one of the details of the poor health of the crew member 11, the level of poor health of the crew member 11, and image information when the crew member 11 was unwell directly or indirectly to the external device 21. The poor health level determination unit 36 ​​causes the ground-to-vehicle communication unit 43 connected to the external device 21 or the ground-to-vehicle communication unit 43 included in the external device 21 to transmit at least one of the details of the poor health of the crew member 11, the level of poor health of the crew member 11, and image information when the crew member 11 was unwell to the operation control room 50.

[0065] In the operation control room 50 that manages the operation of the train 10, a sickness notification unit 61 provided in the ground system 60 indirectly receives at least one of the details of the sickness of the crew 11 of the train 10, the level of the sickness of the crew 11, and image information of the crew 11 when they are sick from the monitoring device 30 via a ground-to-vehicle communication unit 43 external to the monitoring device 30. The sickness notification unit 61 notifies the operation command 51 of at least one of the received details of the sickness of the crew 11 of the train 10, the level of the sickness of the crew 11, and image information of the crew 11 when they are sick. The operation of the sickness notification unit 61 in the second embodiment is the same as the operation of the sickness notification unit 61 in the first embodiment.

[0066] As described above, according to this embodiment, the monitoring device 30 is configured to use an external ground-to-vehicle communication unit 43 when it is available, and does not itself include a ground-to-vehicle communication unit 43. This allows the monitoring system 20 to simplify its configuration by reducing the number of overlapping functions.

[0067] The configurations shown in the above embodiments are merely examples, and may be combined with other known technologies, or different embodiments may be combined with each other. It is also possible to omit or modify parts of the configurations as long as they do not deviate from the gist of the invention.

[0068] Various aspects of the present disclosure are summarized below as appendices.

[0069] (Appendix 1) a facial state detection unit that detects one or more predetermined facial states of a train crew member using a face of the train crew member and predetermined facial features included in the face detected from image information obtained by a camera that photographs the train crew member; an abnormal driving operation calculation unit that calculates abnormal driving operation of the crew from the train operation information acquired from an external device that monitors and controls the state of the train, and outputs abnormal driving operation information; a physical condition level determination unit that determines whether or not to determine the physical condition level of the driver using the image information and / or the operation information, and, when it is determined that the physical condition level of the driver should be determined, determines the physical condition level of the driver based on at least one of the one or more specified facial conditions of the driver detected by the facial condition detection unit and the driving handling abnormality information; a notification unit that notifies the crew member of a state of poor health according to the content of the crew member's poor health and the level of the crew member's poor health; A monitoring device comprising: (Appendix 2) moreover, a ground-to-vehicle communication unit that acquires from the poor health level determination unit at least one of the details of the poor health of the crew member, the poor health level of the crew member, and the image information when the crew member is in poor health, and transmits the information to an operation control room that manages the operation of the train; Equipped with When the poor health level determination unit determines that at least one of the details of the poor health of the driver, the level of the poor health of the driver, and the image information when the driver was in poor health should be output to the operation control room, the unit causes the land-to-vehicle communication unit to transmit at least one of the details of the poor health of the driver, the level of the poor health of the driver, and the image information when the driver was in poor health to the operation control room. 2. The monitoring device according to claim 1, (Appendix 3) When the poor health level determination unit determines that at least one of the details of the poor health of the crew member, the level of poor health of the crew member, and the image information when the crew member was in poor health should be output to an operation control room that manages the operation of the train, the unit outputs at least one of the details of the poor health of the crew member, the level of poor health of the crew member, and the image information when the crew member was in poor health directly or indirectly to the external device, and causes a ground-to-vehicle communication unit connected to the external device or a ground-to-vehicle communication unit included in the external device to transmit at least one of the details of the poor health of the crew member, the level of poor health of the crew member, and the image information when the crew member was in poor health to the operation control room. 2. The monitoring device according to claim 1, (Appendix 4) The physical condition level determination unit determines not to determine the physical condition level of the crew member when at least one of the following applies: the train is undergoing a running test; the train is running within the yard; the train is in a deadheading operation; the train is operating a station stop erroneous passing prevention function; the train is running in a no-stop section; the train is rescuing another train; the train is slowing down; the train is receiving notification information; the crew member is driving in a standing position; and an instructor is present in the driver's cab of the train together with the crew member. 4. The monitoring device according to any one of claims 1 to 3. (Appendix 5) the face state detection unit detects, as the one or more specified facial states of the crew member, at least one of the degree of eye opening of the crew member, the facial direction of the crew member, the number of blinks of the crew member, the line of sight of the crew member, the degree of mouth opening of the crew member, the facial muscles of the crew member, and the brightness of the crew member's skin, using the face and the facial parts; When the physical condition level determination unit determines that the physical condition level of the crew member should be determined, the physical condition level determination unit determines the physical condition level of the crew member based on at least one of the degree of eye opening of the crew member, the facial direction of the crew member, the number of blinks of the crew member, the line of sight of the crew member, the degree of mouth opening of the crew member, the facial muscles of the crew member, the brightness of the crew member's skin, and the driving handling abnormality information. 5. The monitoring device according to any one of claims 1 to 4. (Appendix 6) The poor physical condition level determination unit determines whether the crew member has their eyes open for a specified period of time using the degree of eye opening of the crew member, whether the crew member's facial direction is the ratio of the crew member's facial direction for the specified period of time using the crew member's facial direction, whether the crew member's blink count is equal to or greater than a threshold value using the crew member's blink count, whether the crew member's gaze is frozen or not, whether the crew member has yawned a specified number of times or more in a specified period of time using the crew member's mouth opening, and whether the crew member has wrinkles between their eyebrows or not using the crew member's facial muscles. The level of the driver's physical condition is determined based on at least one of the following: whether the driver's condition is a specified condition; whether the driver has coughed a specified number of times or more in a specified period using the driver's mouth opening degree and facial muscles; whether the driver's skin brightness has changed a specified number of times or more in a specified period using the driver's skin brightness; whether there has been a delay in opening the doors when the train stopped at a station using the abnormal driving operation information; and whether there has been an increase in the frequency of speeding using the abnormal driving operation information. 6. The monitoring device according to claim 5, (Appendix 7) The driving operation abnormality calculation unit calculates, as the driving operation abnormality of the crew, at least one of the frequency of Automatic Train Control brake application, the period when the notch remains in a deceleration state for a specified period after the train arrives at a station, the period when the doors do not open for a specified period after the train arrives at a station, the frequency of the train speed deviating from the standard value of the run curve, and the degree of deviation from the target stopping position when the train stops. 7. The monitoring device according to any one of claims 1 to 6. (Appendix 8) The poor physical condition level determination unit notifies the crew member of the poor physical condition from the notification unit when the crew member's poor physical condition level is below a specified level, and notifies the crew member of the poor physical condition from the notification unit when the crew member's poor physical condition level is equal to or higher than a specified level, and outputs at least one of the details of the crew member's poor physical condition, the crew member's poor physical condition level, and the image information when the crew member was in poor physical condition to a train operation control room that manages the operation of the train. 8. A monitoring device according to any one of claims 1 to 7. (Appendix 9) moreover, a vehicle control information generation unit that generates vehicle control information for stopping the train or for instructing the train to run based on the details of the crew member's poor health and the crew member's poor health level determined by the poor health level determination unit, and transmits the generated vehicle control information to the external device; 9. The monitoring device according to any one of claims 1 to 8, comprising: (Appendix 10) A monitoring device according to any one of claims 1 to 9; a sickness notification unit in a train operation control room that manages train operations, that receives, directly or indirectly from the monitoring device, at least one of the details of the sickness of the train crew, the level of the sickness of the crew, and image information when the crew is sick, and notifies the train operation control room; Equipped with a health condition level determination unit included in the monitoring device determines whether or not to output at least one of the health condition details of the crew member, the health condition level of the crew member, and the image information when the crew member is in poor health to the operation control room that manages the operation of the train, based on the health condition details of the crew member and the health condition level of the crew member; A system for notifying crew members of poor physical condition. (Appendix 11) a facial state detection step in which a facial state detection unit detects one or more predetermined facial states of the train crew member using the face of the train crew member and predetermined facial features included in the face detected from image information obtained by a camera that photographs the train crew member; a driving abnormality calculation step in which an operation abnormality calculation unit calculates an abnormality in the driving behavior of the crew from the train operation information acquired from an external device that monitors and controls the state of the train, and outputs driving abnormality information; a physical condition level determination step in which a physical condition level determination unit determines whether or not to determine the physical condition level of the driver using the image information and the operation information, or the operation information, and, when it is determined that the physical condition level of the driver is to be determined, determines the physical condition level of the driver based on at least one of one or more specified facial states of the driver detected by the facial state detection unit and the driving handling abnormality information; a notification step in which a notification unit notifies the crew member of a state of poor physical condition according to the content of the crew member's poor physical condition and the level of the crew member's poor physical condition; A monitoring method comprising: (Appendix 12) moreover, a ground-to-vehicle communication step in which a ground-to-vehicle communication unit acquires at least one of the details of the driver's poor health, the driver's poor health level, and the image information when the driver is in poor health from the poor health level determination unit, and transmits the information to a train operation control room that manages the operation of the train; Including, In the step of determining the poor health level, when the poor health level determination unit determines that at least one of the details of the poor health of the crew member, the level of the poor health of the crew member, and the image information when the crew member was in poor health should be output to the operation control room, the unit causes the land-to-vehicle communication unit to transmit at least one of the details of the poor health of the crew member, the level of the poor health of the crew member, and the image information when the crew member was in poor health to the operation control room. 12. The monitoring method according to claim 11, (Appendix 13) In the health condition level determination step, when the health condition level determination unit determines that at least one of the details of the crew member's health condition, the level of the crew member's health condition, and the image information when the crew member was in poor health should be output to an operation control room that manages the operation of the train, the health condition level determination unit outputs at least one of the details of the crew member's health condition, the level of the crew member's health condition, and the image information when the crew member was in poor health directly or indirectly to the external device, and causes a ground-to-vehicle communication unit connected to the external device or a ground-to-vehicle communication unit included in the external device to transmit at least one of the details of the crew member's health condition, the level of the crew member's health condition, and the image information when the crew member was in poor health to the operation control room. 12. The monitoring method according to claim 11, (Appendix 14) In the step of determining the level of physical condition of the train, the physical condition level determination unit determines not to determine the level of physical condition of the train crew when at least one of the following applies: the train is undergoing a running test; the train is running within the yard; the train is in a deadheading operation; the train is operating a station stopping and erroneous passing prevention function; the train is running in a no-stop section; the train is rescuing another train; the train is slowing down; the train is receiving notification information; the train crew is driving in a standing position; and an instructor is present in the driver's cab of the train together with the train crew. 14. The monitoring method according to any one of appendices 11 to 13, (Appendix 15) In the face state detection step, the face state detection unit detects, as the one or more predetermined facial states of the crew member, at least one of an eye opening degree of the crew member, a facial direction of the crew member, a blink frequency of the crew member, a line of sight of the crew member, a mouth opening degree of the crew member, a facial muscle of the crew member, and a skin brightness of the crew member, using the face and the facial parts; In the step of determining the level of poor physical condition, when the poor physical condition level determination unit determines that the level of poor physical condition of the crew member should be determined, the poor physical condition level determination unit determines the level of poor physical condition of the crew member based on at least one of the degree of eye opening of the crew member, the facial direction of the crew member, the number of blinks of the crew member, the line of sight of the crew member, the degree of mouth opening of the crew member, the facial muscles of the crew member, the brightness of the crew member's skin, and the abnormal driving handling information. 15. A monitoring method according to any one of appendices 11 to 14. (Appendix 16) In the step of determining a level of poor physical condition, the physical condition level determination unit determines whether the crew member has their eyes open for a specified period of time using the degree of eye opening of the crew member, whether the crew member's facial direction is a ratio of the crew member's facial direction for the specified period of time using the crew member's facial direction, whether the number of blinks of the crew member for the specified period of time is equal to or greater than a threshold value using the number of blinks of the crew member, whether the movement of the crew member's gaze is stopped using the crew member's line of sight, whether the crew member has yawned a specified number of times or more for the specified period of time using the crew member's mouth opening, and whether the crew member has yawned a specified number of times or more for the specified period of time using the crew member's facial muscles. The level of poor physical condition of the crew member is determined based on at least one of whether the condition of the wrinkles between the eyebrows of the crew member is in a specified state, whether the crew member has coughed a specified number of times or more in a specified period using the degree of mouth opening of the crew member and the crew member's facial muscles, whether the crew member's skin brightness has changed a specified number of times or more in a specified period using the crew member's skin brightness, whether there has been a delay in opening the doors when the train stopped at a station using the driving operation abnormality information, and whether there has been an increase in the frequency of speeding using the driving operation abnormality information. 16. The monitoring method according to claim 15, (Appendix 17) In the operation abnormality calculation step, the operation abnormality calculation unit calculates, as the operation abnormality of the crew, at least one of the frequency of Automatic Train Control brake application, the period when the notch remains in a deceleration state for a specified period after the train arrives at a station, the period when the door does not open for a specified period after the train arrives at a station, the frequency of the train speed deviating from the standard value of the run curve, and the degree of deviation from the target stopping position when the train stops. 17. A monitoring method according to any one of appendices 11 to 16, characterized in that (Appendix 18) In the poor health level determination step, the poor health level determination unit notifies the crew member of their poor health state from the notification unit when the crew member's poor health level is below a specified level, and notifies the crew member of their poor health state from the notification unit when the crew member's poor health level is equal to or higher than the specified level, and outputs at least one of the details of the crew member's poor health, the crew member's poor health level, and the image information when the crew member was in poor health to a train operation control room that manages the operation of the train. 18. A monitoring method according to any one of appendices 11 to 17. (Appendix 19) moreover, a vehicle control information generation step in which a vehicle control information generation unit generates vehicle control information for stopping the train or for instructing the train to run, based on the details of the crew member's poor health and the crew member's poor health level determined by the poor health level determination unit, and transmits the generated vehicle control information to the external device; 19. The monitoring method according to any one of appendices 11 to 18, comprising: [Explanation of symbols]

[0070] 10 Train, 11 Crew, 20 Monitoring system, 21 External device, 30 Monitoring device, 31 Camera, 32 Image conversion processing unit, 33 Face and facial part detection unit, 34 Face state detection unit, 35 Communication unit, 36 Illness level determination unit, 37 Memory unit, 38 Notification unit, 41 Driving handling abnormality calculation unit, 42 Vehicle control information generation unit, 43 Ground-to-vehicle communication unit, 50 Driving control room, 51 Driving command, 60 Ground system, 61 Illness notification unit, 70 Crew illness notification system, 90, 93 Processing circuit, 91 Processor, 92 Memory.

Claims

1. a face state detection unit that detects one or more predetermined facial states of the train crew member using the face of the train crew member and predetermined facial features included in the face detected from image information obtained by a camera that photographs the train crew member; an abnormal driving operation calculation unit that calculates abnormal driving operation of the crew from the train operation information acquired from an external device that monitors and controls the state of the train, and outputs abnormal driving operation information; a physical condition level determination unit that determines whether or not to determine the physical condition level of the driver using the image information and / or the operation information, and, when it is determined that the physical condition level of the driver should be determined, determines the physical condition level of the driver based on at least one of the one or more specified facial conditions of the driver detected by the facial condition detection unit and the driving handling abnormality information; a notification unit that notifies the crew member of a state of poor health according to the content of the crew member's poor health and the level of the crew member's poor health; A monitoring device comprising:

2. moreover, a ground-to-vehicle communication unit that acquires from the poor health level determination unit at least one of the details of the poor health of the crew member, the poor health level of the crew member, and the image information when the crew member is in poor health, and transmits the information to an operation control room that manages the operation of the train; Equipped with when it is determined that at least one of the details of the driver's poor health, the driver's level of poor health, and the image information when the driver was in poor health should be output to the operation control room, the poor health level determination unit causes the land-to-vehicle communication unit to transmit at least one of the details of the driver's poor health, the driver's level of poor health, and the image information when the driver was in poor health to the operation control room; 2. The monitoring device according to claim 1.

3. When the poor health level determination unit determines that at least one of the details of the poor health of the crew member, the level of poor health of the crew member, and the image information when the crew member was in poor health should be output to an operation control room that manages the operation of the train, the unit outputs at least one of the details of the poor health of the crew member, the level of poor health of the crew member, and the image information when the crew member was in poor health directly or indirectly to the external device, and causes a ground-to-vehicle communication unit connected to the external device or a ground-to-vehicle communication unit included in the external device to transmit at least one of the details of the poor health of the crew member, the level of poor health of the crew member, and the image information when the crew member was in poor health to the operation control room.

2. The monitoring device according to claim 1.

4. The physical condition level determination unit determines not to determine the physical condition level of the crew member when at least one of the following applies: the train is undergoing a running test; the train is running within the yard; the train is in a deadheading operation; the train is operating a station stop erroneous passing prevention function; the train is running in a no-stop section; the train is rescuing another train; the train is slowing down; the train is receiving notification information; the crew member is driving in a standing position; and an instructor is present in the driver's cab of the train together with the crew member.

2. The monitoring device according to claim 1.

5. the face state detection unit detects, as the one or more specified facial states of the crew member, at least one of an eye opening degree of the crew member, a facial direction of the crew member, a blink frequency of the crew member, a line of sight of the crew member, a mouth opening degree of the crew member, a facial muscle of the crew member, and a skin brightness of the crew member, using the face and the facial parts; When the physical condition level determination unit determines that the physical condition level of the crew member should be determined, the physical condition level determination unit determines the physical condition level of the crew member based on at least one of the degree of eye opening of the crew member, the facial direction of the crew member, the number of blinks of the crew member, the line of sight of the crew member, the degree of mouth opening of the crew member, the facial muscles of the crew member, the brightness of the crew member's skin, and the driving handling abnormality information.

2. The monitoring device according to claim 1.

6. The poor physical condition level determination unit determines whether the crew member has their eyes open for a specified period of time using the degree of eye opening of the crew member, whether the crew member's facial direction is the ratio of the crew member's facial direction for the specified period of time using the crew member's facial direction, whether the crew member's blink count is equal to or greater than a threshold value using the crew member's blink count, whether the crew member's gaze is frozen or not, whether the crew member has yawned a specified number of times or more in a specified period of time using the crew member's mouth opening, and whether the crew member has wrinkles between their eyebrows or not using the crew member's facial muscles. The level of the driver's physical condition is determined based on at least one of the following: whether the driver's condition is a specified condition; whether the driver has coughed a specified number of times or more in a specified period using the driver's mouth opening degree and facial muscles; whether the driver's skin brightness has changed a specified number of times or more in a specified period using the driver's skin brightness; whether there has been a delay in opening the doors when the train stopped at a station using the abnormal driving operation information; and whether there has been an increase in the frequency of speeding using the abnormal driving operation information.

6. The monitoring device according to claim 5.

7. The driving operation abnormality calculation unit calculates, as the driving operation abnormality of the crew, at least one of the frequency of application of the Automatic Train Control brake, the period when the notch remains in a deceleration state for a specified period after the train arrives at a station, the period when the door does not open for a specified period after the train arrives at a station, the frequency of deviation of the speed of the train from the standard value of the run curve, and the degree of deviation from the target stopping position when the train stops.

2. The monitoring device according to claim 1.

8. The poor physical condition level determination unit notifies the crew member of the poor physical condition from the notification unit when the crew member's poor physical condition level is below a specified level, and notifies the crew member of the poor physical condition from the notification unit when the crew member's poor physical condition level is equal to or higher than the specified level, and outputs at least one of the details of the crew member's poor physical condition, the crew member's poor physical condition level, and the image information when the crew member was in poor physical condition to a train operation control room that manages the operation of the train.

2. The monitoring device according to claim 1.

9. moreover, a vehicle control information generation unit that generates vehicle control information for stopping the train or for instructing the train to run based on the details of the crew member's poor health and the crew member's poor health level determined by the poor health level determination unit, and transmits the generated vehicle control information to the external device; The monitoring device according to claim 1 , further comprising:

10. A monitoring device according to any one of claims 1 to 9; a sickness notification unit in a train operation control room that manages train operations, that receives, directly or indirectly from the monitoring device, at least one of the details of the sickness of the train crew, the level of the sickness of the train crew, and image information when the train crew is sick, and notifies the train operation control room; Equipped with a health condition level determination unit included in the monitoring device determines, based on the health condition details and the health condition level of the crew member, whether or not to output at least one of the health condition details of the crew member, the health condition level of the crew member, and the image information when the crew member is in poor health to the operation control room that manages the operation of the train; A system for notifying crew members of poor physical condition.

11. a facial state detection step in which a facial state detection unit detects one or more predetermined facial states of the train crew member by using the face of the train crew member and predetermined facial features included in the face detected from image information obtained by a camera that photographs the train crew member; a driving abnormality calculation step in which an operation abnormality calculation unit calculates an abnormality in the driving behavior of the crew from the train operation information acquired from an external device that monitors and controls the state of the train, and outputs driving abnormality information; a physical condition level determination step in which a physical condition level determination unit determines whether or not to determine the physical condition level of the driver using the image information and / or the operation information, and when it determines that the physical condition level of the driver should be determined, determines the physical condition level of the driver based on at least one of one or more specified facial states of the driver detected by the facial state detection unit and the abnormal driving handling information; a notification step in which a notification unit notifies the crew member of a state of poor physical condition according to the content of the crew member's poor physical condition and the level of the crew member's poor physical condition; A monitoring method comprising:

12. moreover, a ground-to-vehicle communication step in which a ground-to-vehicle communication unit acquires at least one of the details of the driver's poor health, the driver's poor health level, and the image information when the driver is in poor health from the poor health level determination unit, and transmits the information to a train operation control room that manages the operation of the train; Including, In the poor health level determination step, when the poor health level determination unit determines that at least one of the details of the poor health of the crew member, the level of the poor health of the crew member, and the image information when the crew member was in poor health should be output to the operation control room, the unit causes the land-to-vehicle communication unit to transmit at least one of the details of the poor health of the crew member, the level of the poor health of the crew member, and the image information when the crew member was in poor health to the operation control room. The monitoring method according to claim 11 .

13. In the health condition level determination step, when the health condition level determination unit determines that at least one of the details of the crew member's health condition, the level of the crew member's health condition, and the image information when the crew member was in poor health should be output to an operation control room that manages the operation of the train, the health condition level determination unit outputs at least one of the details of the crew member's health condition, the level of the crew member's health condition, and the image information when the crew member was in poor health condition directly or indirectly to the external device, and causes a ground-to-vehicle communication unit connected to the external device or a ground-to-vehicle communication unit included in the external device to transmit at least one of the details of the crew member's health condition, the level of the crew member's health condition, and the image information when the crew member was in poor health to the operation control room. The monitoring method according to claim 11 .

14. In the step of determining the level of physical condition of the train, the level of physical condition determination unit determines not to determine the level of physical condition of the train crew when at least one of the following applies: the train is undergoing a running test; the train is running within the yard; the train is in a deadheading operation; the train is operating a station stopping and erroneous passing prevention function; the train is running in a no-stop section; the train is rescuing another train; the train is slowing down; the train is receiving notification information; the train crew is driving in a standing position; and an instructor is present in the driver's cab of the train together with the train crew. The monitoring method according to claim 11 .

15. In the face state detection step, the face state detection unit detects, as the one or more specified facial states of the crew member, at least one of an eye opening degree of the crew member, a facial direction of the crew member, a blink frequency of the crew member, a line of sight of the crew member, a mouth opening degree of the crew member, a facial muscle of the crew member, and a skin brightness of the crew member, using the face and the facial parts; In the step of determining the level of poor physical condition, when the poor physical condition determination unit determines that the level of poor physical condition of the crew member should be determined, the poor physical condition determination unit determines the level of poor physical condition of the crew member based on at least one of the degree of eye opening of the crew member, the facial direction of the crew member, the number of blinks of the crew member, the line of sight of the crew member, the degree of mouth opening of the crew member, the facial muscles of the crew member, the brightness of the crew member's skin, and the abnormal driving handling information. The monitoring method according to claim 11 .

16. In the step of determining a level of poor physical condition, the physical condition level determination unit determines whether the crew member has their eyes open for a specified period of time using the degree of eye opening of the crew member, whether the crew member's facial direction is a ratio of the crew member's facial direction for the specified period of time using the crew member's facial direction, whether the number of blinks of the crew member for the specified period of time is equal to or greater than a threshold value using the number of blinks of the crew member, whether the movement of the crew member's gaze is stopped using the crew member's line of sight, whether the crew member has yawned a specified number of times or more for the specified period of time using the crew member's mouth opening, and whether the crew member has yawned a specified number of times or more for the specified period of time using the crew member's facial muscles. The level of poor physical condition of the crew member is determined based on at least one of the following: whether the condition of the frown lines between the crew member's eyebrows is in a specified state; whether the crew member has coughed a specified number of times or more in a specified period using the degree of mouth opening of the crew member and the crew member's facial muscles; whether the crew member's skin brightness has changed a specified number of times or more in a specified period using the crew member's skin brightness; whether there has been a delay in opening the doors when the train stopped at a station using the abnormal driving operation information; and whether there has been an increase in the frequency of speeding using the abnormal driving operation information.

16. The monitoring method of claim 15.

17. In the operation abnormality calculation step, the operation abnormality calculation unit calculates, as the operation abnormality of the crew, at least one of the frequency of application of the Automatic Train Control brake, the period when the notch remains in a deceleration state for a specified period after the train arrives at a station, the period when the door does not open for a specified period after the train arrives at a station, the frequency of deviation of the speed of the train from the standard value of the run curve, and the degree of deviation from the target stopping position when the train stops. The monitoring method according to claim 11 .

18. In the poor health level determination step, the poor health level determination unit notifies the crew member of their poor health state from the notification unit when the crew member's poor health level is below a specified level, and notifies the crew member of their poor health state from the notification unit when the crew member's poor health level is equal to or higher than the specified level, and outputs at least one of the details of the crew member's poor health, the crew member's poor health level, and the image information when the crew member was in poor health to a train operation control room that manages the operation of the train. The monitoring method according to claim 11 .

19. moreover, a vehicle control information generation step in which a vehicle control information generation unit generates vehicle control information for stopping the train or for instructing the train to run, based on the details of the crew member's poor health and the crew member's poor health level determined by the poor health level determination unit, and transmits the generated vehicle control information to the external device; The monitoring method of claim 11, comprising:

Images