Driver monitoring device

Abstract

To ensure a driver to understand whether or not an action of a driver during automatic driving of a vehicle corresponds to an action other than allowable driving.SOLUTION: A driver monitoring device 12 comprises: a mode switching unit 232 which has a monitoring mode for monitoring an action of a driver of a vehicle 1 based on an image of the driver imaged by a driver monitor camera 7 during automatic driving of the vehicle 1 at a level in which an action other than driving is allowed and a trial mode for confirming whether or not the action of the driver corresponds to an action other than allowable driving based on the image of the driver during automatic driving, and switches between the monitoring mode and the trial mode in response to an operation by the driver; a warning unit 233 which warns the driver when a state in which the action of the driver corresponds to an action other than a non-allowable driving continues for a first threshold value or more at the time of the monitoring mode; and a notification unit 234 which notifies the driver when a state in which the action of the driver corresponds to the action other than the non-allowable driving continues for a second threshold value or more at the time of trial mode, wherein the second threshold value is smaller than the first threshold value.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a driver monitoring device. [Background technology]

[0002] Patent Document 1 describes a technology that performs automatic driving of a vehicle and notifies the driver of the vehicle of various kinds during automatic driving. The technology described in Patent Document 1 detects the driver's non-driving behavior (e.g., looking away, operating a smartphone, operating vehicle equipment (e.g., audio equipment or a navigation system)) based on an image of the driver captured by a driver monitor camera. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-091790 A Summary of the Invention [Problem to be solved by the invention]

[0004] The driver's non-driving behavior is also referred to as "non-driving behavior". In autonomous driving level 3 of the Society of Automotive Engineers (SAE), the driver is permitted to perform non-driving behavior during autonomous driving of the vehicle. In detail, the driver's non-driving behavior includes those that are permitted and those that are not permitted during autonomous driving of the vehicle. Therefore, the driver needs to understand whether his / her non-driving behavior is permitted or not permitted during autonomous driving of the vehicle and perform the non-driving behavior. However, in the technology described in Patent Document 1, the driver cannot understand whether the driver's behavior during autonomous driving of the vehicle corresponds to a permitted non-driving behavior.

[0005] In view of the above, an object of the present invention is to provide a driver monitoring device that enables the driver to easily understand whether the driver's actions during automatic driving of a vehicle correspond to actions other than permissible driving. [Means for solving the problem]

[0006] One aspect of the present invention is a driver monitoring device having a monitoring mode that monitors the actions of a driver of a vehicle based on an image of the driver captured by a driver monitor camera during autonomous driving of the vehicle at a level at which actions other than driving are permitted, and a trial mode that is a mode for confirming whether the actions of the driver correspond to the permissible actions other than driving based on the image of the driver during the autonomous driving, the driver monitoring device comprising: a mode switching unit that switches between the monitoring mode and the trial mode in response to an operation of the driver; a warning unit that issues a warning to the driver when a state in which the driver's action corresponds to the unpermissible actions other than driving continues for a first threshold value or more during the monitoring mode; and a notification unit that issues a notification to the driver when a state in which the driver's action corresponds to the unpermissible actions other than driving continues for a second threshold value or more during the trial mode, the second threshold value being set to a value smaller than the first threshold value. Effect of the Invention

[0007] According to the present invention, it is possible to allow the driver to easily understand whether or not the driver's actions during automatic driving of a vehicle correspond to actions other than permissible driving. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram showing an example of a vehicle to which a driver monitoring device of a first embodiment is applied. [Diagram 2] FIG. 2 is a diagram illustrating an example of processing by a processor of the driver monitoring device of the first embodiment. [Diagram 3] FIG. 11 is a diagram illustrating an example of processing by a processor of a driver monitoring device according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] First Embodiment FIG. 1 is a diagram showing an example of a vehicle 1 to which a driver monitoring device 12 of the first embodiment is applied. In the example shown in Fig. 1, a vehicle 1 includes an exterior camera 2, a radar 3, a LiDAR (Light Detection And Ranging) 4, a GPS (Global Positioning System) unit 5, a map information unit 6, a driver monitor camera 7, an HMI (Human Machine Interface) 8, an automatic driving control device 10, and a driver monitoring device 12. These are connected via an in-vehicle network 13. The vehicle 1 also includes a steering actuator 14, a braking actuator 15, and a drive actuator 16.

[0010] The automatic driving control device 10 is configured by, for example, an automatic driving ECU (Electronic Control Unit). The automatic driving control device 10 can control the vehicle 1 at a driving control level that does not require the driver of the vehicle 1 to operate the steering actuator 14, the braking actuator 15, and the driving actuator 16 and to monitor the surroundings of the vehicle 1, and has a function of executing automatic driving of the vehicle 1. In addition, the automatic driving control device 10 can control the vehicle 1 at a driving control level in which the driver is involved in driving the vehicle 1, and has a function of executing manual driving of the vehicle 1. During the execution of automatic driving of the vehicle 1 by the automatic driving control device 10, actions other than driving of the driver of the vehicle 1 are permitted. In other words, the automatic driving control device 10 can execute automatic driving of the vehicle 1 at a level at which actions other than driving of the driver of the vehicle 1 are permitted. The driver's actions other than driving include those that are permitted during automatic driving of the vehicle 1 and those that are not permitted during automatic driving of the vehicle 1. The driver's actions other than driving that are permitted during automatic driving of the vehicle 1 include, for example, looking away, operating a smartphone, operating vehicle equipment, etc. Actions other than driving by the driver that are not permitted during autonomous driving of vehicle 1 include actions that prevent the driver from smoothly responding to a request to switch from autonomous driving to manual driving (e.g., leaving the driver's seat, falling asleep, etc.).

[0011] The driver monitoring device 12 has a function of detecting an unacceptable behavior of the driver other than driving during the automatic driving of the vehicle 1 and having the HMI 8 execute a warning to the driver. The driver monitoring device 12 has a monitoring mode and a trial mode. The monitoring mode is a mode for monitoring the behavior of the driver based on an image of the driver of the vehicle 1 captured by the driver monitor camera 7 during the automatic driving of the vehicle 1 at a level at which behavior other than driving is permitted. The trial mode is a mode for confirming whether or not the behavior of the driver corresponds to an acceptable behavior other than driving based on an image of the driver of the vehicle 1 captured by the driver monitor camera 7 during the automatic driving of the vehicle 1 at a level at which behavior other than driving is permitted. That is, in the trial mode, the driver of the vehicle 1 actually performs an action, thereby confirming whether or not the behavior of the driver corresponds to an acceptable behavior other than driving. The driver monitoring device 12 is configured by a microcomputer including a communication interface (I / F) 21, a memory 22, and a processor 23. The communication I / F 21, the memory 22, and the processor 23 are connected via a signal line 24. The communication I / F 21 has an interface circuit for connecting the driver monitoring device 12 to the in-vehicle network 13. The memory 22 stores programs and various data used in the processes executed by the processor 23. The processor 23 has a function as an acquisition unit 231, a function as a mode switching unit 232, a function as a warning unit 233, and a function as a notification unit 234.

[0012] The acquisition unit 231 acquires data of an image of the driver of the vehicle 1 captured by the driver monitor camera 7, information indicating a switching operation between a monitoring mode and a trial mode by the driver of the vehicle 1, and the like. The mode switching unit 232 executes switching between the monitoring mode and the trial mode in response to a switching operation by the driver of the vehicle 1. The warning unit 233 causes the HMI 8 to execute a warning to the driver of the vehicle 1 when a state in which the driver's action of the vehicle 1 corresponds to an action other than an unacceptable driving continues for a first threshold value or more during the monitoring mode of the driver monitoring device 12. The warning by the warning unit 233 can alert the driver of the vehicle 1 so that the driver's action of the vehicle 1 does not correspond to an action other than an unacceptable driving. The notification unit 234 causes the HMI 8 to execute a notification to the driver of the vehicle 1 when a state in which the driver's action of the vehicle 1 corresponds to an action other than an unacceptable driving continues for a second threshold value or more during the trial mode of the driver monitoring device 12. The second threshold value is set to a value smaller than the first threshold value. The notification by the notification unit 234 allows the driver of the vehicle 1 to easily understand whether or not the driver's actions during automatic driving of the vehicle 1 correspond to actions other than permissible driving.

[0013] FIG. 2 is a flowchart for explaining an example of processing executed by the processor 23 of the driver monitoring device 12 of the first embodiment during automatic driving of the vehicle 1. 2, in step S10, the acquisition unit 231 acquires image data of the driver of the vehicle 1 captured by the driver monitor camera 7, and for example, the warning unit 233 determines whether or not an unacceptable act other than driving by the driver of the vehicle 1 has been detected based on the image of the driver of the vehicle 1. If YES, the process proceeds to step S11, and if NO, the routine shown in FIG. 2 is terminated. In step S11, for example, the warning unit 233 determines whether or not an unacceptable behavior other than driving by the driver of the vehicle 1 has been detected for the first time since the vehicle 1 started traveling. If YES, the process proceeds to step S12, and if NO, the routine shown in FIG. 2 is terminated. In step S12, for example, the warning unit 233 causes the HMI 8 to display or output a weak alert as a TIPS (hint). In step S13, for example, the notification unit 234 determines whether or not the mode switching unit 232 has switched the mode of the driver monitoring device 12 to the trial mode in response to a switching operation by the driver of the vehicle 1. If YES, the process proceeds to step S14, and if NO, the routine shown in FIG. In step S14, for example, the notification unit 234 sets a second threshold value necessary for notifying the driver of the vehicle 1.

[0014] In steps S15 to S17, the notification unit 234 executes the processes when the driver monitoring device 12 is in the trial mode. In detail, when the driver monitoring device 12 is in trial mode, the driver monitor camera 7 captures an image of the driver of the vehicle 1, and in step S15, the acquisition unit 231 acquires data of the image of the driver of the vehicle 1 captured by the driver monitor camera 7. In step S16, the notification unit 234 determines whether or not the behavior of the driver of the vehicle 1 corresponds to an unacceptable driving behavior based on the image of the driver of the vehicle 1. If YES, the process proceeds to step S17, and if NO, the process returns to step S15. In step S17, the notification unit 234 causes the HMI 8 to execute a notification that the behavior of the driver of the vehicle 1 corresponds to an unacceptable behavior other than driving. In detail, when the state in which the behavior of the driver of the vehicle 1 corresponds to an unacceptable behavior other than driving continues for more than a second threshold, the notification unit 234 causes the HMI 8 to execute a notification to the driver. The notification to the driver is performed, for example, by causing the HMI 8 to output an alert with a soft tone for a short period of time. In step S18, for example, the notification unit 234 judges whether or not to end the trial mode of the driver monitoring device 12. If YES, the routine shown in FIG. 2 is ended, and if NO, the process returns to step S15. For example, the notification unit 234 judges to end the trial mode of the driver monitoring device 12 when the mode switching unit 232 switches the mode of the driver monitoring device 12 to the monitoring mode in response to a switching operation by the driver of the vehicle 1. Also, for example, when the state in which the driver monitoring device 12 is in the trial mode continues for a predetermined time, the notification unit 234 judges to end the trial mode of the driver monitoring device 12 because it is necessary to return the driver monitoring device 12 to the monitoring mode.

[0015] In the example shown in Figures 1 and 2, the driver of vehicle 1 can confirm whether or not the driver's action falls under an action other than permissible driving by actually performing an action during the trial mode of the driver monitoring device 12, and can easily grasp the boundary between permissible non-driving actions and unpermissible non-driving actions.

[0016] That is, in the example shown in FIG. 1 and FIG. 2, when an unacceptable behavior of the driver of the vehicle 1 other than driving is detected, a weak alert is output as a TIPS first. The weak alert as a TIPS functions as a suggestion of a trial mode from the driver monitoring device 12 to the driver of the vehicle 1, so that the driver of the vehicle 1 can be expected to switch the driver of the vehicle 1 from the monitoring mode to the trial mode in response to the weak alert as a TIPS. If an unacceptable behavior of the driver of the vehicle 1 other than driving is detected even after the notification unit 234 executes the process in the trial mode of the driver monitoring device 12, the warning unit 233 outputs a strong alert to the HMI 8 as a warning to the driver of the vehicle 1. If an unacceptable behavior of the driver of the vehicle 1 other than driving is detected even after the strong alert is output, a request for switching driving from automatic driving to manual driving is made to the driver of the vehicle 1.

[0017] <Second embodiment> FIG. 3 is a flowchart for explaining an example of processing executed by the processor 23 of the driver monitoring device 12 of the second embodiment during automatic driving of the vehicle 1. 3, in order to suppress overconfidence of the driver of the vehicle 1 in the driver monitoring device 12, in step S17-2, for example, the notification unit 234 changes the range of permissible acts other than driving. In detail, for example, in step S16, the notification unit 234 changes the range of permissible acts other than driving so that the act of the driver of the vehicle 1 is more likely to be determined to correspond to an unpermissible act other than driving. [Explanation of symbols]

[0018] 1...vehicle, 2...exterior camera, 3...radar, 4...LiDAR, 5...GPS unit, 6...map information unit, 7...driver monitor camera, 8...HMI, 10...automatic driving control device, 12...driver monitoring device, 14...steering actuator, 15...braking actuator, 16...driving actuator, 21...communication interface, 22...memory, 23...processor, 231...acquisition unit, 232...mode switching unit, 233...warning unit, 234...notification unit

Claims

[Claim 1] A monitoring mode for monitoring the actions of a driver of the vehicle based on an image of the driver captured by a driver monitor camera during autonomous driving of the vehicle at a level where actions other than driving are permitted; A driver monitoring device having a trial mode for confirming whether or not an action of the driver corresponds to an action other than the permissible driving based on an image of the driver during the automatic driving, a mode switching unit that switches between the monitoring mode and the trial mode in response to an operation by the driver; a warning unit that issues a warning to the driver when a state in which an action of the driver corresponds to an unacceptable action other than the driving continues for a period equal to or longer than a first threshold value during the monitoring mode; a notification unit that executes a notification to the driver when a state in which the driver's action corresponds to an unacceptable action other than the driving action continues for a period equal to or longer than a second threshold value during the trial mode; A driver monitoring device, wherein the second threshold value is set to a value smaller than the first threshold value.

Images