Monitoring device, monitoring system, and monitoring method

Abstract

In the present invention, a monitoring device (30) comprises: a face and facial part detection unit (33) that detects, from image information obtained by a camera (31) that captures an image of a driver (11) of a train (10), the face of the driver (11) and a prescribed facial part included in the face; an eye-opening-degree, face-direction, and mouth-opening-degree detection unit (34) that detects at least one of the degree to which the eyes of the driver (11) are open, the direction in which the face of the driver (11) is oriented, and the degree to which the mouth of the driver (11) is open using the face and facial part detected by the face and facial part detection unit (33); a communication unit (35) that acquires information regarding the operation the train (10) from an external device (21) that monitors and controls the state of the train (10); and a drowsiness level determination unit (36) that determines whether or not to determine the drowsiness level of the driver (11) using the operation information alone or together with the image information, and, when a determination has been made to determine the drowsiness level, determines the drowsiness level on the basis of at least one of the eye-opening degree, the face direction, and the mouth-opening degree.

Description

MONITORING DEVICE, MONITORING SYSTEM, AND MONITORING METHOD

[0001] The present disclosure relates to a monitoring device, a monitoring system, and a monitoring method for use in a train.

[0002] Conventionally, a driver's drowsiness level is determined, and when the driver's drowsiness is detected, the driver is awakened. For example, Patent Document 1 discloses a technique for a dialogue device that, when it detects that the driver's level of awakening is decreasing, makes the driver aware of the decrease in the level of awakening through dialogue and encourages the driver to take a break.

[0003] Japanese Patent Application Laid-Open No. 2022-24852

[0004] However, the above-described conventional technology assumes that the target of awakening is a car driver. Similarly, train drivers who operate vehicles can fall asleep, and waking up a driver when drowsiness is detected is useful. However, train operations include designated areas where trains must not be stopped. Therefore, if a device such as that described in Patent Document 1 performs an awakening operation in a designated area, there is a risk that a panicked driver may accidentally stop the train in that area.

[0005] The present disclosure has been made in consideration of the above, and aims to provide a monitoring device that determines the drowsiness level of a train driver at an appropriate timing according to the train's operating status.

[0006] In order to solve the above-mentioned problems and achieve the objectives, the monitoring device of the present disclosure is characterized by comprising: a face and facial part detection unit that detects the face of a train driver and specified facial parts contained in the face from image information obtained by a camera that photographs the train driver; an eye openness, facial direction, and opening degree detection unit that detects at least one of the driver's eye openness, the driver's facial direction, and the driver's opening degree using the face and facial parts detected by the face and facial part detection unit; a communication unit that acquires train operation information from an external device that monitors and controls the train status; and a drowsiness level determination unit that uses the image information and the operation information, or the operation information, to determine whether or not to determine the driver's drowsiness level, and when it is determined that the drowsiness level should be determined, determines the drowsiness level based on at least one of the eye openness, the facial direction, and the opening degree.

[0007] The monitoring device of the present disclosure has the effect of being able to determine the drowsiness level of a train driver at an appropriate timing depending on the train's operating status.

[0008] FIG. 1 is a diagram showing an example of the configuration of a monitoring system according to embodiment 1. FIG. 2 is a flowchart showing the operation of the monitoring device according to embodiment 1. FIG. 3 is a flowchart showing the operation of determining whether or not to determine the driver's drowsiness level in the drowsiness level determination unit of the monitoring device according to embodiment 1. FIG. 4 is a diagram showing an example of the configuration of a processing circuit in the monitoring device according to embodiment 1 when the processing circuit is realized by a processor and a memory. FIG. 5 is a diagram showing an example of the configuration of a processing circuit in the monitoring device according to embodiment 1 when the processing circuit is realized by dedicated hardware.

[0009] A monitoring device, a monitoring system, and a monitoring method according to embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0010] First Embodiment. FIG. 1 is a diagram illustrating an example configuration of a monitoring system 20 according to a first embodiment. The monitoring system 20 is mounted on a train 10 and determines the drowsiness level of the driver 11 of the train 10. When the monitoring system 20 determines that the drowsiness level of the driver 11 of the train 10 is at a level that will affect the operation of the train 10, that is, when the monitoring system 20 determines that the driver 11 of the train 10 is dozing or shows signs of dozing, the monitoring system 20 performs an action to wake up the driver 11 of the train 10. The action to wake up the driver 11 of the train 10 refers to an action to wake up the driver 11 of the train 10. Note that, hereinafter, a specific description will be given of the action of the monitoring system 20 determining the drowsiness level of the driver 11 of the train 10; however, the monitoring system 20 can also determine the drowsiness level of a crew member other than the driver 11, such as a conductor.

[0011] The monitoring system 20 includes an external device 21 and a monitoring device 30. The external device 21 monitors and controls the state of the train 10, the state of on-board equipment (not shown) mounted on the train 10, etc. The external device 21 outputs operation information of the train 10 to the monitoring device 30. The operation information of the train 10 includes, for example, the speed information of the train 10, the position information of the train 10, running test information indicating whether the train 10 is undergoing a running test, yard running information indicating whether the train 10 is running within the yard, deadhead information indicating whether the train 10 is being operated, door open information indicating whether the doors of the train 10 are open, station stop mispass prevention function operation status information regarding the operation status of the station stop mispass prevention function that warns the driver 11 by voice or display to prevent the train from passing through a station, stop prohibition information indicating sections where the train 10 is prohibited from stopping, such as air section sections or tunnel sections, notification information indicating speed restrictions or operation cancellations of the train 10, and slow operation information indicating whether the train 10 is operating at a slow speed. The external device 21 may be, for example, a TCMS (Train Control Management System), but is not limited thereto.

[0012] The following describes the configuration and operation of the monitoring device 30. As shown in Fig. 1 , the monitoring device 30 includes a camera 31, an image conversion processing unit 32, a face and face part detection unit 33, an eye opening degree / face direction opening degree detection unit 34, a communication unit 35, a drowsiness level determination unit 36, a storage unit 37, and a speaker 38.

[0013] The camera 31 photographs the driver 11 of the train 10. The camera 31 outputs image information obtained by photographing the driver 11 of the train 10 to the image conversion processing unit 32. Note that although the monitoring device 30 is equipped with the camera 31 in the example of FIG.

[0014] The image conversion processing unit 32 converts the image information acquired from the camera 31, for example, from a color image to a grayscale image. The image conversion processing unit 32 outputs the converted grayscale image to the face and facial part detection unit 33. Note that the monitoring device 30 can be configured without the image conversion processing unit 32 if the image information of the color image acquired by the camera 31 can be used in the face and facial part detection unit 33 (described later) or the like.

[0015] The face and facial part detection unit 33 detects the face of the driver 11 and specified facial parts included in the face of the driver 11 from the grayscale image acquired from the image conversion processing unit 32. The specified facial parts include, for example, the eyes and mouth. The detected face and facial parts can be identified by expressing them as coordinates relative to the original grayscale image. The method for detecting the face and facial parts in the face and facial part detection unit 33 can be a method used in general face authentication, but is not particularly limited. Note that if the monitoring device 30 does not include the image conversion processing unit 32, the face and facial part detection unit 33 will detect the face of the driver 11 and specified facial parts included in the face of the driver 11 from the image information acquired by the camera 31. The face and facial part detection unit 33 outputs information on the detected face and facial parts to the eye opening degree / face direction opening degree detection unit 34.

[0016] The eye openness / facial direction / mouth openness detection unit 34 detects the eye openness, facial direction, and mouth openness of the driver 11 using the face and facial parts of the driver 11 acquired from the face and facial part detection unit 33. The eye openness of the driver 11 indicates the percentage of time that the driver 11 keeps his / her eyes open during a specified period. When the driver 11 becomes drowsy, it is considered that the degree of eye openness decreases compared to normal. The face direction indicates the percentage of time that the driver 11 keeps his / her face open during a specified period. Normally, the driver 11 spends most of the time facing forward while operating the train 10, but when he / she becomes drowsy, it is considered that he / she turns his / her head downward and assumes a hunched posture. The mouth openness indicates the percentage of time that the driver 11 keeps his / her mouth open during a specified period. When the driver 11 becomes drowsy, it is considered that the degree of mouth openness increases compared to normal. The eye opening degree, face direction, and mouth opening degree detection unit 34 outputs the detected eye opening degree, face direction, and mouth opening degree to the drowsiness level determination unit 36. Note that the eye opening degree, face direction, and mouth opening degree detection unit 34 may be configured to detect at least one of the eye opening degree, face direction, and mouth opening degree. When the eye opening degree, face direction, and mouth opening degree detection unit 34 detects at least one of the eye opening degree, face direction, and mouth opening degree, it is sufficient to output the detected amount to the drowsiness level determination unit 36.

[0017] The communication unit 35 acquires operation information of the train 10 from the external device 21. The communication unit 35 communicates with the external device 21 using a communication method such as Ethernet (registered trademark) or RS (Recommended Standards) 485-HDLC (High-level Data Link Control). The communication unit 35 outputs the acquired operation information of the train 10 to the drowsiness level determination unit 36.

[0018] The drowsiness level determination unit 36 ​​acquires the image information used by the face and facial part detection unit 33, and acquires the operation information of the train 10 acquired by the communication unit 35 from the communication unit 35. The image information used by the face and facial part detection unit 33 is a grayscale image obtained by the image conversion processing unit 32 if the monitoring device 30 is equipped with the image conversion processing unit 32, or is image information of a color image obtained by the camera 31 if the monitoring device 30 is not equipped with the image conversion processing unit 32. The drowsiness level determination unit 36 ​​may acquire the image information from the face and facial part detection unit 33 via the eye opening degree / face direction / opening degree detection unit 34, or may acquire the image information directly from the camera 31, the image conversion processing unit 32, or the face and facial part detection unit 33, although connections are not shown in FIG. 1 .

[0019] The drowsiness level determination unit 36 ​​determines whether or not to determine the drowsiness level of the driver 11, using the image information used by the face and facial part detection unit 33 and the operation information acquired by the communication unit 35. As described above, the operation information acquired from the communication unit 35 includes information such as speed information, position information, running test information, on-premises running information, deadheading information, door opening information, station stopping train erroneous passing prevention function operation status information, stop prohibition information, notice information, and slow driving information. In addition, the image information used by the face and facial part detection unit 33 can determine whether or not the driver 11 is driving in a standing position and whether or not an instructor is driving together with the driver 11. Therefore, the drowsiness level determination unit 36 ​​determines not to determine the drowsiness level when at least one of the following applies: the train 10 is undergoing a running test, the train 10 is running within the yard, the train 10 is in out-of-service operation, the train 10 has a station stopping and erroneous passing prevention function in operation, the train 10 is running in a no-stopping section, the train 10 is rescuing another train, the train 10 is slowing down, the train 10 is receiving notification information, the driver 11 is driving in a standing position, or an instructor is present in the driver's cab of the train 10 with the driver 11. That is, the drowsiness level determination unit 36 ​​determines to determine the drowsiness level when all of the following conditions are not met: the train 10 is undergoing a running test, the train 10 is running within the yard, the train 10 is in a deadheading operation, the train 10 has a station stop erroneous passing prevention function activated, the train 10 is in a no-stop section, the train 10 is rescuing another train, the train 10 is slowing down, the train 10 is receiving notification information, the driver 11 is driving in a standing position, or an instructor is present in the driver's cab of the train 10 together with the driver 11. Note that the above conditions under which the drowsiness level determination unit 36 ​​determines the drowsiness level are merely examples and are not limited to these. The drowsiness level determination unit 36 ​​can also determine whether to determine the drowsiness level based on conditions other than the above. Furthermore, the drowsiness level determination unit 36 ​​can also not use some of the above conditions. As a result, the drowsiness level determination unit 36 ​​can determine whether or not to determine the drowsiness level of the driver 11 using the operation information acquired by the communication unit 35, without using the image information used in the face and facial part detection unit 33.

[0020] The reason why the drowsiness level determination unit 36 ​​makes the above-described determination is that even if the drowsiness level determination determines that the driver 11 is drowsy, no awakening action is performed on the driver 11 in the above-described situation. Since it is meaningless to determine the drowsiness level of the driver 11 in a situation in which no awakening action is performed, the drowsiness level determination unit 36 ​​does not determine the drowsiness level from the start in a situation in which no awakening action is performed.

[0021] When the drowsiness level determination unit 36 ​​determines the drowsiness level of the driver 11 based on the eye opening degree, facial direction, and opening degree acquired from the eye opening degree / facial direction / opening degree detection unit 34, the drowsiness level determination unit 36 ​​may determine the drowsiness level of the driver 11 based on at least one of the eye opening degree, facial direction, and opening degree acquired from the eye opening degree / facial direction / opening degree detection unit 34. However, it is considered that the drowsiness level determination unit 36 ​​can more accurately determine the drowsiness level of the driver 11 by making a determination using multiple items. For example, the drowsiness level determination unit 36 ​​can determine that the driver 11 is drowsy when a state of eye opening below a specified eye opening degree threshold for a normal eye opening degree continues for a period of time equal to or greater than the threshold. Furthermore, the drowsiness level determination unit 36 ​​can determine that the driver 11 is drowsy when, based on the facial direction, the driver 11 looks down for a period of time equal to or greater than the specified facial direction threshold. In addition, the drowsiness level determination unit 36 ​​can determine that the driver 11 is feeling drowsy if the state of opening degree equal to or greater than the opening degree threshold specified for normal opening degree continues for a period of time equal to or greater than the threshold.

[0022] The drowsiness level determination unit 36 ​​also stores in the memory unit 37 the determination result that the driver 11 is drowsy in the drowsiness level determination, and information used when the driver 11 is determined to be drowsy in the drowsiness level determination. The drowsiness level determination unit 36 ​​stores a specified number of determination results and information in the memory unit 37, and overwrites determination results and information older than the specified number to update them with the latest determination results and information. The specified number is, for example, 30 times. The drowsiness level determination unit 36 ​​stores the latest 30 determination results and information in the memory unit 37, and updates them by overwriting the oldest determination results and information. Taking memory corruption in the memory unit 37 into consideration, the drowsiness level determination unit 36 ​​calculates a cyclic redundancy check (CRC) or a checksum value for the determination results and information stored in the memory unit 37 to ensure the validity of the data. The drowsiness level determination unit 36 ​​causes the determination result and information stored in the memory unit 37 to be transmitted from the communication unit 35 to a ground device (not shown) that manages the operation of the train 10 .

[0023] Furthermore, if the drowsiness level determination unit 36 ​​determines that the driver 11 is drowsy in the drowsiness level determination, the drowsiness level determination unit 36 ​​performs an action to awaken the driver 11. For example, the drowsiness level determination unit 36 ​​awakens the driver 11 by outputting a sound from the speaker 38. When outputting a sound from the speaker 38 as an action to awaken the driver 11, if the driver's level of awakening is not improved, the drowsiness level determination unit 36 ​​may increase the driver's level of awakening by switching at least one of the tone, content, and volume of the sound midway. The drowsiness level determination unit 36 ​​may include specific operational information about the train 10 in the content of the sound output from the speaker 38. This allows the driver 11 to understand the current operational status of the train 10, thereby preventing delays of the train 10.

[0024] The storage unit 37 stores the determination result of the drowsiness level determination unit 36 ​​that the driver 11 is drowsy in the drowsiness level determination, and information used when the drowsiness level determination unit 36 ​​determined that the driver 11 is drowsy in the drowsiness level determination. The storage unit 37 is, for example, a ring buffer. Note that the data amount of determination results and information that can be stored in the storage unit 37 is set to 30 determination results and information in the above example, but this can be changed as appropriate depending on the capacity of the storage unit 37, and the storage unit 37 can also store more than 30 determination results and information. Also, although the monitoring device 30 includes the storage unit 37 in the example of FIG. 1 , the storage unit 37 may be external to the monitoring device 30.

[0025] When the drowsiness level determination unit 36 ​​determines that the driver 11 is drowsy in the drowsiness level determination, the speaker 38 outputs a sound to the driver 11 under the control of the drowsiness level determination unit 36 ​​in order to awaken the driver 11. The speaker 38 may be a buzzer or the like. Note that, in the present embodiment, the monitoring device 30 outputs a sound from the speaker 38 as an operation to awaken the driver 11, but this is not limited to this. The monitoring device 30 may awaken the driver 11 using an indicator light (not shown) instead of or together with the speaker 38. In the example of FIG. 1 , the monitoring device 30 includes the speaker 38, but the speaker 38 may be external to the monitoring device 30.

[0026] 2 is a flowchart showing the operation of the monitoring device 30 according to the first embodiment. In the monitoring device 30, the image conversion processing unit 32 acquires image information from the camera 31 (step S101). The image conversion processing unit 32 converts the image information acquired from the camera 31 into a grayscale image (step S102). The face and facial feature detection unit 33 detects the face of the driver 11 and specified facial features included in the face of the driver 11 from the grayscale image acquired from the image conversion processing unit 32 (step S103). The eye opening degree, facial direction, and eye opening degree detection unit 34 detects the eye opening degree, facial direction, and eye opening degree of the driver 11 using the face and facial features of the driver 11 acquired from the face and facial feature detection unit 33 (step S104). The communication unit 35 acquires operation information of the train 10 from the external device 21 (step S105). The drowsiness level determination unit 36 ​​determines whether or not to determine the drowsiness level of the driver 11 using the image information used by the face and facial part detection unit 33 and the operation information acquired by the communication unit 35 (step S106).

[0027] The operation of step S106 will now be described in detail. Fig. 3 is a flowchart showing the operation of determining whether or not to determine the drowsiness level of the driver 11 in the drowsiness level determination unit 36 ​​of the monitoring device 30 according to the first embodiment. The drowsiness level determination unit 36 ​​determines not to determine the drowsiness level (step S211) if the train 10 is undergoing a running test (step S201: Yes), or if the train 10 is running within the premises (step S202: Yes), or if the train 10 is in a deadhead operation (step S203: Yes), or if the station stop prevention function is in operation on the train 10 (step S204: Yes), or if the train 10 is running in a no-stopping section (step S205: Yes), or if the train 10 is rescuing another train (step S206: Yes), or if the train 10 is moving slowly (step S207: Yes), or if the train 10 is receiving notification information (step S208: Yes), or if the driver 11 is driving in a standing position (step S209: Yes), or if an instructor is present in the driver's cab of the train 10 with the driver 11 (step S210: Yes).

[0028] The drowsiness level determination unit 36 ​​determines to perform a drowsiness level determination (step S212) if the train 10 is not undergoing a running test (step S201: No), the train 10 is not running within the premises (step S202: No), the train 10 is not in a deadhead operation (step S203: No), the train 10 is not operating the station stop erroneous passing prevention function (step S204: No), the train 10 is not running in a no-stopping section (step S205: No), the train 10 is not rescuing another train (step S206: No), the train 10 is not moving slowly (step S207: No), the train 10 is not receiving notification information (step S208: No), the driver 11 is not driving in a standing position (step S209: No), or there is no instructor in the driver's cab of the train 10 (step S210: No).

[0029] Returning to the description of FIG. 2 . If the drowsiness level of the driver 11 is not to be determined (step S106: No), the monitoring device 30 returns to steps S101 and S105 and performs the aforementioned operations. The result of step S106: No corresponds to step S211 in the flowchart shown in FIG. 3 . If the drowsiness level of the driver 11 is to be determined (step S106: Yes), the drowsiness level determination unit 36 ​​determines the drowsiness level of the driver 11 (step S107). The result of step S106: Yes corresponds to step S212 in the flowchart shown in FIG. 3 . As described above, the drowsiness level determination unit 36 ​​determines the drowsiness level of the driver 11 by comparing the degree of eye opening, facial direction, and degree of opening with the respective thresholds. If the drowsiness level determination unit 36 ​​determines that the driver 11 is drowsy (step S108: Yes), it performs an operation to wake the driver 11 (step S109). If the drowsiness level determination unit 36 ​​determines that the user is not drowsy (step S108: No), the drowsiness level determination unit 36 ​​ends the operation of the flowchart shown in Fig. 2 without performing the operation of step S109. The monitoring device 30 periodically repeats the operation of the flowchart shown in Fig. 2.

[0030] Next, the hardware configuration of the monitoring device 30 according to the first embodiment will be described. In the monitoring device 30, the camera 31 is a camera capable of capturing an image of the driver 11. The communication unit 35 is a communication interface capable of communicating with the external device 21. The storage unit 37 is a memory. The speaker 38 is a speaker capable of outputting a sound sufficient to wake the driver 11. The image conversion processing unit 32, the face and facial part detection unit 33, the eye opening degree / face direction opening degree detection unit 34, and the drowsiness level determination unit 36 ​​are realized by a processing circuit. The processing circuit may be a memory that stores a program and a processor that executes the program stored in the memory, or may be dedicated hardware. The processing circuit is also called a control circuit.

[0031] FIG. 4 is a diagram illustrating an example of the configuration of the processing circuit 90 of the monitoring device 30 according to the first embodiment, when the processing circuit is realized by a processor 91 and a memory 92. The processing circuit 90 illustrated in FIG. 4 is a control circuit and includes a processor 91 and a memory 92. When the processing circuit 90 is configured with the processor 91 and the memory 92, each function of the processing circuit 90 is realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 92. The processing circuit 90 realizes each function by having the processor 91 read and execute the program stored in the memory 92. That is, the processing circuit 90 includes the memory 92 for storing a program that results in the processing of the monitoring device 30 being executed. This program can also be said to be a program that causes the monitoring device 30 to execute each function realized by the processing circuit 90. This program may be provided by a storage medium on which the program is stored, or by other means such as a communication medium.

[0032] The above program can also be said to be a program that causes the monitoring device 30 to execute the following steps: a first step in which the face and facial part detection unit 33 detects the face of the driver 11 and specified facial parts included in the face from image information obtained by the camera 31 that photographs the driver 11 of the train 10; a second step in which the eye openness, facial direction, and mouth openness detection unit 34 detects at least one of the eye openness degree of the driver 11, the facial direction of the driver 11, and the mouth openness of the driver 11 using the face and facial parts detected by the face and facial part detection unit 33; a third step in which the communication unit 35 acquires operation information of the train 10 from an external device 21 that monitors and controls the state of the train 10; and a fourth step in which the drowsiness level determination unit 36 ​​uses the image information and the operation information, or the operation information, to determine whether or not to determine the drowsiness level of the driver 11, and if it is determined that the drowsiness level should be determined, determines the drowsiness level based on at least one of the eye openness degree, facial direction, and mouth openness.

[0033] Here, the processor 91 is, for example, a CPU (Central Processing Unit), a processing device, an arithmetic unit, a microprocessor, a microcomputer, or a DSP (Digital Signal Processor), etc. The memory 92 is, for example, a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable ROM), or an EEPROM (Electrically EPROM), a magnetic disk, a flexible disk, an optical disk, a compact disk, a minidisk, or a DVD (Digital Versatile Disc).

[0034] FIG. 5 is a diagram illustrating an example of the configuration of the processing circuit 93 of the monitoring device 30 according to the first embodiment when the processing circuit is realized by dedicated hardware. The processing circuit 93 illustrated in FIG. 5 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The processing circuit 93 may be partially realized by dedicated hardware and partially realized by software or firmware. In this way, the processing circuit 93 can realize each of the above-described functions by dedicated hardware, software, firmware, or a combination thereof.

[0035] As described above, according to the present embodiment, in the monitoring device 30, the drowsiness level determination unit 36 ​​does not determine the drowsiness level in a situation where no awakening action is performed on the driver 11 of the train 10, depending on the specific operating status of the train 10, the specific driving status of the driver 11 of the train 10, etc. This enables the monitoring device 30 to prevent unnecessary drowsiness level determinations and prevent delays caused by the driver 11 erroneously stopping the train 10. In other words, the monitoring device 30 can determine the drowsiness level of the driver 11 of the train 10 at an appropriate timing depending on the operating status of the train 10.

[0036] Furthermore, the monitoring device 30 stores, in the memory unit 37, information such as the results of the drowsiness level determination that a driver is drowsy and the operation information used when the drowsiness level was determined, for the most recent specified number of times. This allows a third party to easily verify the drowsiness level determination by using the information stored in the memory unit 37. Furthermore, since the memory unit 37 stores only the data necessary for verification and does not contain unnecessary data, the verification period can be shortened. By using the data stored in the memory unit 37, a third party verifying the drowsiness level determination can perform more accurate verification than by interviewing the driver 11 of the train 10.

[0037] Embodiment 2 In the first embodiment, it has been described that the drowsiness level of the driver 11 of the train 10 may not be determined depending on the operating conditions of the train 10. In the second embodiment, it will be described that even if it is determined not to determine the drowsiness level of the driver 11 of the train 10 in the first embodiment, it is determined to determine the drowsiness level of the driver 11 of the train 10 in a specific situation.

[0038] In the second embodiment, the configurations of the monitoring system 20 and the monitoring device 30 are the same as those of the monitoring system 20 and the monitoring device 30 of the first embodiment shown in Fig. 1. The overall operation of the monitoring device 30 is also the same as that of the flowchart of the first embodiment shown in Fig. 2. In the second embodiment, the content of the operation of step S106 is different.

[0039] Fig. 6 is a flowchart showing the operation of determining whether or not to determine the drowsiness level of the driver 11 in the drowsiness level determination unit 36 ​​of the monitoring device 30 according to the second embodiment. In the flowchart shown in Fig. 6, the operations from step S201 to step S210 are the same as the operations from step S201 to step S210 in the flowchart of the first embodiment shown in Fig. 3. In addition, the operations of step S211 and step S212 in the flowchart of the first embodiment shown in Fig. 3 are also the same as the operations of step S211 and step S212 in the flowchart of the first embodiment shown in Fig. 3. The flowchart shown in Fig. 6 is obtained by adding step S221 before step S211 to the flowchart of the first embodiment shown in Fig. 3.

[0040] The drowsiness level determination unit 36 ​​checks whether emergency information has been received (step S221) when the train 10 is undergoing a running test (step S201: Yes), or when the train 10 is running within the premises (step S202: Yes), or when the train 10 is in a deadhead operation (step S203: Yes), or when the train 10's station stop prevention function is in operation (step S204: Yes), or when the train 10 is running in a no-stopping section (step S205: Yes), or when the train 10 is rescuing another train (step S206: Yes), or when the train 10 is moving slowly (step S207: Yes), or when the train 10 is receiving notification information (step S208: Yes), or when the driver 11 is driving in a standing position (step S209: Yes), or when an instructor is present in the driver's cab of the train 10 with the driver 11 (step S210: Yes).

[0041] Examples of emergency information include, but are not limited to, emergency earthquake information, accident information about an accident that has occurred on the line on which the train 10 runs, and malfunction information about a train that has malfunctioned on the line on which the train 10 runs. If the drowsiness level determination unit 36 ​​has not received emergency information (step S221: No), it determines not to determine the drowsiness level (step S211). If the drowsiness level determination unit 36 ​​has received emergency information (step S221: Yes), it determines to determine the drowsiness level (step S212). In this way, the drowsiness level determination unit 36 ​​determines the drowsiness level when it acquires specified emergency information, even in cases where it would have determined not to determine the drowsiness level in the first embodiment.

[0042] As described above, according to the present embodiment, in the monitoring device 30, the drowsiness level determination unit 36 ​​determines the drowsiness level of the driver 11 of the train 10 in specific situations such as an earthquake, accident, or breakdown, even in cases where it is determined not to determine the drowsiness level of the driver 11 of the train 10 in the first embodiment. This allows the monitoring device 30 to determine the drowsiness level of the driver 11 and perform awakening action as necessary, even in cases where determination of the drowsiness level of the driver 11 or awakening action is not normally required, such as during a running test.

[0043] The configurations shown in the above embodiments are merely examples, and may be combined with other known technologies, or different embodiments may be combined with each other. It is also possible to omit or modify parts of the configurations as long as they do not deviate from the gist of the invention.

[0044] 10 Train, 11 Driver, 20 Monitoring system, 21 External device, 30 Monitoring device, 31 Camera, 32 Image conversion processing unit, 33 Face and facial part detection unit, 34 Eye opening degree, face direction, and opening degree detection unit, 35 Communication unit, 36 Drowsiness level determination unit, 37 Memory unit, 38 Speaker, 90, 93 Processing circuit, 91 Processor, 92 Memory.

Claims

1. A monitoring device comprising: a face and facial part detection unit that detects the face of a train driver and specified facial parts contained in the face from image information obtained by a camera that photographs the driver; an eye opening, facial direction, and opening degree detection unit that detects at least one of the degree of eye opening of the driver, the direction of the driver's face, and the degree of opening of the driver's eyes using the face and the facial parts detected by the face and facial part detection unit; a communication unit that acquires operation information of the train from an external device that monitors and controls the state of the train; and a drowsiness level determination unit that uses the image information and the operation information, or the operation information, to determine whether or not to determine the driver's drowsiness level, and when it is determined that the drowsiness level should be determined, determines the drowsiness level based on at least one of the degree of eye opening, the direction of the face, and the degree of opening of the eyes.

2. The monitoring device according to claim 1, characterized in that the drowsiness level determination unit determines not to determine the drowsiness level when at least one of the following applies: the train is undergoing a running test, the train is running within the yard, the train is in normal operation, the train's station stopping prevention function is in operation, the train is running in a no-stopping section, the train is rescuing another train, the train is slowing down, the train is receiving notification information, the driver is driving in a standing position, and an instructor is present in the driver's cab of the train together with the driver.

3. The monitoring device according to claim 1, characterized in that the drowsiness level determination unit determines to determine the drowsiness level when specified emergency information is acquired in at least one of the following cases: the train is undergoing a running test, the train is running within the yard, the train is in normal operation, the train's station stopping prevention function is in operation, the train is running in a no-stopping section, the train is rescuing another train, the train is slowing down, the train is receiving notification information, the driver is driving in a standing position, and an instructor is present in the driver's cab together with the driver.

4. The monitoring device according to claim 1 or 3, characterized in that it is further provided with a memory unit that stores the result of the drowsiness level determination unit's determination that the driver is drowsy in determining the drowsiness level, and information used when the drowsiness level determination unit determines that the driver is drowsy in determining the drowsiness level.

5. The monitoring device according to claim 4, characterized in that the drowsiness level determination unit causes the determination result and the information stored in the memory unit to be transmitted from the communication unit to a ground device.

6. The monitoring device according to claim 1, 3, 4 or 5, characterized in that the drowsiness level determination unit performs an action to wake up the driver when it determines that the driver is drowsy in the drowsiness level determination.

7. The monitoring device according to claim 6, characterized in that the drowsiness level determination unit outputs a sound from a speaker as an action to wake up the driver, and switches at least one of the tone, content, and volume during the process.

8. A monitoring system comprising: a monitoring device according to any one of claims 1 to 7; and an external device which monitors and controls the state of a train and provides said monitoring device with operation information of said train.

9. A monitoring method comprising: a first step in which a face and facial part detection unit detects the face of a train driver and specified facial parts contained in the face from image information obtained by a camera that photographs the driver; a second step in which an eye opening degree / facial direction / opening degree detection unit detects at least one of the driver's eye opening degree, the driver's facial direction, and the driver's opening degree using the face and the facial parts detected by the face and facial part detection unit; a third step in which a communication unit acquires operation information of the train from an external device that monitors and controls the state of the train; and a fourth step in which a drowsiness level determination unit uses the image information and the operation information, or the operation information, to determine whether or not to determine the driver's drowsiness level, and when it is determined that the drowsiness level should be determined, determines the drowsiness level based on at least one of the eye opening degree, the facial direction, and the opening degree.

10. The monitoring method according to claim 9, characterized in that in the fourth step, the drowsiness level determination unit determines not to determine the drowsiness level when at least one of the following applies: the train is undergoing a running test, the train is running within the yard, the train is in normal operation, the train's station stopping prevention function is in operation, the train is running in a no-stopping section, the train is rescuing another train, the train is slowing down, the train is receiving notification information, the driver is driving in a standing position, and an instructor is present in the driver's cab of the train together with the driver.

11. The monitoring method of claim 9, characterized in that in the fourth step, the drowsiness level determination unit determines to determine the drowsiness level when specified emergency information is acquired while at least one of the following applies: the train is undergoing a running test, the train is running within the yard, the train is in normal operation, the train's station stopping prevention function is in operation, the train is running in a no-stopping section, the train is rescuing another train, the train is slowing down, the train is receiving notification information, the driver is driving in a standing position, and an instructor is present in the driver's cab together with the driver.

12. A monitoring method as described in claim 9 or 11, characterized in that it includes a fifth step in which the memory unit stores the judgment result of the drowsiness level judgment unit that the driver is drowsy in judging the drowsiness level, and information used when the drowsiness level judgment unit judged that the driver is drowsy in judging the drowsiness level.

13. The monitoring method described in claim 12, characterized in that in the fourth step, the drowsiness level determination unit causes the determination result and the information stored in the memory unit to be transmitted from the communication unit to a ground device.

14. The monitoring method described in claim 9, 11, 12 or 13, characterized in that in the fourth step, the drowsiness level determination unit performs an action to wake up the driver if it determines that the driver is drowsy in the drowsiness level determination.

15. The monitoring method described in claim 14, characterized in that in the fourth step, the drowsiness level determination unit outputs a sound from a speaker as an action to wake up the driver, and switches at least one of the tone, content, and volume during the output.

Images