Driver Monitoring Device

Abstract

To provide a driver monitoring device capable of suppressing lowering of a driver's attention without making the driver feel annoyed by a hands-on request.SOLUTION: A driver monitoring device comprises: a driver information acquisition unit that acquires driver information related to at least one of a line of sight and a face direction of a driver; a surroundings information acquisition unit that acquires surroundings information for a vehicle being driven by the driver; a determination unit that determines an attention level of the driver on the basis of the driver information acquired by the driver information acquisition unit and the surroundings information acquired by the surroundings information acquisition unit; and a notification unit that notifies the driver of the attention level determined by the determination unit.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a driver monitoring device. [Background technology]

[0002] Patent Document 1 discloses a driver monitoring system that requests the driver to keep their hands on the steering wheel of the vehicle every certain period of time in order to prevent a decline in the driver's attention to driving during automatic steering operation in which the driver is allowed to take their hands off the steering wheel of the vehicle to steer the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-12356 Summary of the Invention [Problem to be solved by the invention]

[0004] The vehicle seat described in Patent Document 1 requires the driver to put their hands on the seat every certain time, which can be annoying. To reduce the annoyance caused by the hands-on requirement, it is conceivable to determine whether the driver is able to maintain their attention, determine whether a hands-on requirement is necessary based on the result of this determination, and issue a hands-on requirement if necessary, thereby reducing unnecessary hands-on requirements. However, for drivers who find it difficult to maintain their attention while hands-off, the hands-on requirement may not be reduced, leaving room for improvement.

[0005] In consideration of the above, an object of the present invention is to provide a driver monitoring device that can prevent a decrease in the driver's attention without causing the driver to feel annoyed by the need for hands-on driving. [Means for solving the problem]

[0006] The driver monitoring device according to the present invention as set forth in claim 1 includes a driver information acquisition unit that acquires driver information relating to at least one of the driver's line of sight and face direction, a surrounding information acquisition unit that acquires surrounding information about the vehicle driven by the driver, a determination unit that determines the attention level of the driver based on the driver information acquired by the driver information acquisition unit and the surrounding information acquired by the surrounding information acquisition unit, and a determination unit that calculates the attention level determined by the determination unit. During handoff, the driver takes their hands off the steering wheel of the vehicle. a notification unit that notifies the driver, the notification unit being a robot equipped with a speaker and the monitor display screen of at the same time The driver is notified with motivational words according to the attention level.

[0007] In the driver monitoring device according to the present invention as set forth in claim 1, the determination unit determines the driver's attention level based on the driver information acquired by the driver information acquisition unit and the surrounding information of the vehicle driven by the driver acquired by the surrounding information acquisition unit, and notifies the driver of the level by the notification unit. Therefore, the driver can recognize his / her current attention level, and when notified that the attention level is decreasing, the driver can take action such as paying close attention to the surroundings to increase the attention level.

[0008] Furthermore, if the driver is notified that the attention level is being maintained, the driver will be able to recognize that the attention level is being maintained and can take measures to continue maintaining the attention level. This makes it possible to prevent a decline in attention without feeling the annoyance of a hands-on requirement. 。 [Effects of the Invention]

[0009] As described above, the driver monitoring device according to the present invention has the excellent effect of being able to prevent the driver's attention from decreasing without feeling annoyed by the need for hands-on driving. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a hardware configuration of a driver monitoring system including a driver monitoring device according to an embodiment of the present invention. [Figure 2] 1A and 1B are diagrams illustrating an example of a notification by a driver monitoring device according to an embodiment of the present invention, respectively; [Figure 3] 3 is a flowchart showing a series of processes performed by a driver monitoring device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] A driver monitoring system 10 including a driver monitoring device 20 according to an embodiment of the present invention will be described below with reference to the accompanying drawings. Fig. 1 is a block diagram showing the hardware configuration of a driver monitoring system 10 including a driver monitoring device 20 according to an embodiment of the present invention.

[0012] The driver monitoring system 10 of this embodiment is mounted on a vehicle (not shown) and, as shown in Figure 1, comprises a driver monitoring device 20, a sensor unit 12, a surrounding monitoring camera 14, a line of sight monitoring camera 16, a recording unit 18, and an alarm means 30.

[0013] The sensor unit 12 is a millimeter-wave radar and includes a front radar 12A and a rear radar 12B, each of which is a radar device. The front radar 12A and the rear radar 12B are disposed at the center of the front end and the center of the rear end of the vehicle, respectively, and detect targets in the front area and the rear area of ​​the vehicle, respectively. Specifically, information indicating the target's direction relative to the vehicle, the distance between the vehicle and the target, and the relative speed between the vehicle and the target, etc., is obtained as "target information" every time a predetermined time elapses, and output to the driver monitoring device 20 and the recording unit 18.

[0014] The perimeter monitoring camera 14 is a camera that captures images of the perimeter of the vehicle and is equipped with multiple imaging units such as color CCD (Charge Coupled Device) cameras. In this embodiment, a front camera 14A that captures images in front of the vehicle and a rear camera 14B that captures images behind the vehicle are provided. The front camera 14A and the rear camera 14B are installed at the center of the vehicle width direction on the inside or outside of the front and rear of the vehicle, respectively. The perimeter monitoring camera 14 captures images of the perimeter of the vehicle at a predetermined frame rate and outputs the captured image data to the driver monitoring device 20 and the recording unit 18.

[0015] The gaze monitoring camera 16 is composed of a color CCD camera or the like, and is installed, for example, on the instrument panel near the steering wheel, with its imaging range set in advance to an area that can capture an image of the driver's face. The gaze monitoring camera 16 outputs captured image data to the driver monitoring device 20 and the recording unit 18.

[0016] The recording unit 18 is a non-volatile memory such as a flash memory, and records the captured images taken by the perimeter monitoring camera 14, the target information output from the sensor unit 12, and the captured images taken by the line-of-sight monitoring camera 16.

[0017] The notification means 30 is a means for notifying the driver of various information, and in this embodiment, as an example, is configured as a display screen of a monitor that provides visual information to the driver. Specifically, the notification means 30 is configured as a meter display provided on the driver's seat side of the instrument panel, in front of the driver's seat. The display screen of the monitor serving as the notification means 30 is provided in a position that is within the field of view of the driver when the driver is looking ahead of the vehicle. However, the notification means 30 is not limited to this, and may also be configured as a projection surface projected by a head-up display device (not shown). Specifically, the notification means 30 can be configured so that an image is projected from a head-up display device provided on the vehicle front side of the instrument panel onto a display area of ​​the windshield glass.

[0018] The driver monitoring device 20 includes a CPU (Central Processing Unit) (not shown), a ROM (Read Only Memory) (not shown), a RAM (Random Access Memory) (not shown), a storage (not shown), and an input / output I / F (not shown). The CPU, ROM, RAM, storage, and input / output I / F are connected to each other via a bus so as to be able to communicate with each other.

[0019] The driver monitoring device 20 also has, as its functional components, a driver information acquisition unit 210, a surrounding information acquisition unit 220, a determination unit 230, and a notification unit 240. Each functional component is realized by the CPU reading and executing an execution program recorded in the ROM or storage.

[0020] The driver information acquisition unit 210 acquires driver information relating to at least one of the driver's line of sight and facial direction. In this embodiment, as an example, driver information relating to both the driver's line of sight and facial direction is acquired. Specifically, the driver's line of sight and facial direction are acquired using known technology based on image data captured by the line of sight monitoring camera 16.

[0021] The surrounding information acquisition unit 220 acquires surrounding information of the vehicle driven by the driver, i.e., the vehicle equipped with the driver monitoring system 10. Specifically, based on the target information acquired by the sensor unit 12 and the image data captured by the surrounding monitoring camera 14, the surrounding information acquisition unit 220 acquires the distance from the vehicle to the target, the direction of the target, the size of the target, the position of the target, etc. as surrounding information of the vehicle. Note that this information is acquired using known technology.

[0022] The determination unit 230 determines the driver's attention level based on the driver information acquired by the driver information acquisition unit 210 and the peripheral information acquired by the peripheral information acquisition unit 220. Specifically, as an example, attention levels are set in stages, and the current driver's attention level is determined based on the driver information and the peripheral information. As an example of a determination method, a classifier (not shown) that has been machine-trained in advance using learning information including multiple data sets of pairs of driver information, peripheral information, and driver's attention level can be used. The classifier is trained so that it outputs an attention level when driver information or peripheral information is input.

[0023] The notification unit 240 notifies the driver of the caution level determined by the determination unit 230. In this embodiment, the notification unit 240 notifies the driver of the caution level using the notification means 30. Fig. 2(A) is a diagram showing an example of notification by the driver monitoring device 20 according to one embodiment of the present invention.

[0024] For example, depending on the level of the caution level, the notification unit 240 may, on the display screen as the notification means 30 as shown in Fig. 2(A), emphasize the color of the small medal 310, emphasize the color of each area of ​​the periphery that constitutes the large medal 320, or emphasize the color of the star 330 inside the large medal 320. In this way, it is possible to visually recognize whether the caution level is good or bad.

[0025] Next, a series of processes performed by the driver monitoring device 20 of this embodiment will be described. Fig. 3 is a flowchart showing a series of processes performed by the driver monitoring device according to one embodiment of the present invention.

[0026] 3, in the driver monitoring device 20, the driver information acquisition unit 210 acquires driver information in step S10, and the surrounding information acquisition unit 220 acquires surrounding information in step S12. Next, in step S14, the determination unit 230 determines the driver's attention level based on the driver information and surrounding information acquired in steps S10 and S12. Then, in step S16, the notification unit 240 notifies the driver of the attention level determined by the determination unit 230. In this way, the driver's attention level is notified to the driver, as shown in FIG. 2(A).

[0027] Next, the effects of this embodiment will be described.

[0028] In the driver monitoring device 20 according to this embodiment, the determination unit 230 determines the driver's attention level based on the driver information acquired by the driver information acquisition unit 210 and the surrounding information of the vehicle driven by the driver acquired by the surrounding information acquisition unit 220, and notifies the driver of the level by the notification unit 240. Therefore, the driver can recognize his or her current attention level, and when notified that the attention level is decreasing, the driver can take action such as paying close attention to the surroundings to raise the attention level.

[0029] Furthermore, if the driver is notified that the attention level is being maintained, the driver can recognize that the attention level is being maintained and can take measures to continue maintaining the attention level. This makes it possible to prevent a decline in attention without feeling the annoyance of a hands-on driving request.

[0030] The notification of the driver's attention level by the driver monitoring device 20 in this embodiment can be used when the driver takes their hands off the steering wheel of the vehicle. However, this is not only effective when the driver takes their hands off the steering wheel, but also when the driver is holding the steering wheel of the vehicle and is in a hands-on state, and can also be used to maintain the driver's attention level while manually driving the vehicle.

[0031] In the above embodiment, the notification means 30 is a display screen of a monitor that provides visual information to the driver, but the present invention is not limited to this. Here, a modified example of the notification means 30 will be described. Figure 2(B) is a diagram showing a modified example of notification by the driver monitoring device 20 according to one embodiment of the present invention.

[0032] 2(B), in addition to the display screen of a monitor that provides visual information to the driver, the notification means 30 uses a robot equipped with a speaker that provides auditory information to the driver as the notification means 32. In the modified example, the notification unit 240 causes the robot as the notification means 32 to speak positive, praising words such as "Your attention level is rising. That's good," or "Your attention level is falling. Try a little harder and be more careful," depending on the attention level determined by the determination unit 230.

[0033] This positive praise encourages drivers to adjust their level of attention. maintain It is possible to increase or maintain the motivation of the driver to drive the vehicle. In the modified example, both the monitor display screen as the notification means 30 and the robot as the notification means 32 are used, but the present invention is not limited to this, and only the robot may be used. Furthermore, the robot may be installed anywhere in the vehicle interior.

[0034] Although one embodiment of the present invention has been described above, the present invention is not limited to such an embodiment, and one embodiment may be appropriately combined with various modified examples, and the present invention may of course be embodied in various forms as long as it does not deviate from the gist of the present invention. [Explanation of symbols]

[0035] 20 Driver monitoring device 210 Driver information acquisition unit 220 Surrounding Information Acquisition Department 230 Judgment section 240 Information Department

Claims

[Claim 1] a driver information acquisition unit that acquires driver information relating to at least one of the driver's line of sight and face direction; a surrounding information acquisition unit that acquires surrounding information of the vehicle driven by the driver; a determination unit that determines an attention level of the driver based on the driver information acquired by the driver information acquisition unit and the surrounding information acquired by the surrounding information acquisition unit; a notification unit that notifies the driver of the attention level determined by the determination unit when the driver releases his / her hands from the steering wheel of the vehicle during a handoff; Equipped with The notification unit simultaneously uses both a robot equipped with a speaker and a monitor display screen to notify the driver with motivational messages according to the attention level.

Images