Driver monitoring device

Abstract

To provide a driver monitoring device capable of avoiding dangerous driving due to poor visibility in backlight and ensuring safe driving.SOLUTION: A driver monitoring device 11 comprises: a photographing unit 111 for acquiring a vehicle front image obtained by photographing the front of the vehicle from the vehicle; a parameter acquisition unit 112 for acquiring biological parameters related to the eyes of a driver driving the vehicle; a determination unit 113 for determining whether or not the front of the vehicle is in a backlit state as seen by the driver, based on the comparison between the acquired biological parameters and the biological parameters obtained in advance during normal operation; and a video projection unit 114 for projecting a vehicle front image in a direction different from the direction where the sun exists as seen from the driver when it is determined that the front of the vehicle is in a backlit state.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a driver monitoring device, and more particularly to a driver monitoring device capable of avoiding dangerous driving due to poor visibility in backlit conditions and ensuring safe driving. [Background technology]

[0002] There is a known driving support technology that uses a sensing device to detect risk factors around a vehicle and display a warning to the driver of the vehicle. There is also a known display technology that converts a projection image in the vehicle's travel direction based on information about an object in front of the vehicle to project an support image in front of the vehicle. However, in the case of poor visibility due to backlight, projecting the support image in front of the vehicle does not improve the poor visibility due to backlight, and there is a problem that it is difficult to avoid dangerous driving due to poor visibility in backlight and ensure safe driving.

[0003] Patent Document 1 describes a vehicle display device that includes a display that projects a road shape and an indicator that indicates the vehicle's path onto a combiner mounted on a vehicle based on map information consisting of three-dimensional information stored in a map information storage means, and displays the road shape and the indicator by superimposing them on the outside world, and includes an imaging means that captures the outside world, and a control means that causes the display to project onto the combiner a display image in which a background part corresponding to the background of the road shape among the captured information by the imaging means is added to the road shape and the indicator part. The technology described in Patent Document 1 does not relate to a driver monitoring device that can avoid dangerous driving due to poor visibility in backlighting and ensure safe driving.

[0004] Patent Document 2 describes a head-up display device that displays a virtual image in front of a vehicle, which includes an information acquisition means for acquiring information about an object recognized from image data of the object in front of the vehicle captured by an imaging device, an image generation means for generating a projection image to be projected as a virtual image, an image position determination means for determining a display position of the projection image in the traveling direction of the vehicle with reference to the object based on information about the object acquired by the information acquisition means, an image conversion means for converting the projection image into a support image viewed from a predetermined direction, and a projection means for projecting the support image in front of the vehicle. The technology described in Patent Document 2 does not relate to a driver monitoring device that can avoid dangerous driving due to poor visibility in backlighting and ensure safe driving. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2006-151215 A [Patent Document 2] JP 2019-32243 A Summary of the Invention [Problem to be solved by the invention]

[0006] As described above, there was a problem in that it was difficult to avoid dangerous driving due to poor visibility in backlit conditions and ensure safe driving.

[0007] An object of the present invention is to provide a driver monitoring device that solves the above-mentioned problems. [Means for solving the problem]

[0008] Therefore, the present invention provides An image capturing unit that captures a front image of the vehicle from the vehicle; A parameter acquisition unit that acquires biometric parameters related to the eyes of a driver who drives the vehicle; a determination unit that determines whether or not the front of the vehicle is in a backlight state as seen from the driver based on a comparison between the acquired biometric parameters and the biometric parameters during normal driving that have been acquired in advance; an image projection unit that projects the image in front of the vehicle in a direction different from a direction in which the sun is present as seen from the driver when it is determined that the front of the vehicle is in the backlit state; A driver monitoring device comprising: Effect of the Invention

[0009] According to the present invention, it is possible to provide a driver monitoring device that can avoid dangerous driving caused by poor visibility in backlit conditions and ensure safe driving. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram illustrating a driver monitoring device according to an embodiment; [Diagram 2] 4 is a flowchart illustrating an operation of a driver monitoring device according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] [Embodiment Mode] <Configuration> FIG. 1 is a block diagram illustrating a driver monitoring device according to an embodiment.

[0012] As shown in FIG. 1, a driver monitoring device 11 according to the embodiment includes an image capturing unit 111, a parameter acquiring unit 112, a determining unit 113, and an image projecting unit 114.

[0013] The image capturing unit 111 acquires a vehicle front image captured from the vehicle. The image capturing unit 111 may be, for example, an in-vehicle camera mounted on the vehicle or a front camera mounted on the front of the vehicle. The in-vehicle camera may be an in-vehicle camera. The image capturing unit 111 may capture an image of the front of the vehicle using LiDAR (Light Detection And Ranging).

[0014] The image in front of the vehicle is not limited to a real image such as a photograph, and may be, for example, an image acquired using a LiDAR and including at least the shape of the road and the contours of obstacles.

[0015] Examples of images in front of the vehicle (images that allow the driver to check the situation in front of the vehicle) include, but are not limited to, the following images: -Image captured by a high dynamic range CMOS (Complementary Metal Oxide Semiconductor) image sensor. The dynamic range of a CMOS image sensor refers to the difference between light and dark. - Millimeter wave radar footage (image). -Images taken by combining the CMOS image sensor and millimeter wave radar. An image in which glare has been reduced using a Neutral Density (ND) filter, or an image in which the dynamic range has been expanded using High Dynamic Range (HDR) synthesis technology.

[0016] The parameter acquisition unit 112 acquires biometric parameters related to the eyes of a driver who drives a vehicle. The parameter acquisition unit 112 may acquire biometric parameters during normal driving and / or driving in backlight in advance and store them in a storage unit (not shown).

[0017] The determination unit 113 determines whether the front of the vehicle is backlit as seen from the driver based on a comparison between the acquired biometric parameter and a biometric parameter during normal driving previously acquired. That is, the determination unit 113 determines that the front of the vehicle is backlit as seen from the driver when the difference between the value of the acquired biometric parameter and the value of the biometric parameter during normal driving previously acquired is equal to or greater than a predetermined difference. The determination unit 113 may also determine whether the front of the vehicle is backlit as seen from the driver based on a comparison between the acquired biometric parameter and a biometric parameter during backlit driving previously acquired.

[0018] Here, the biological parameters may include the driver's eye opening abnormality rate and the number of blinks of the driver's eyes in a predetermined time. The driver's eye opening abnormality rate is, for example, the ratio of time during which the driver's eyes are not normally open in a predetermined time. In this case, the determination unit 113 may determine that the vehicle is in a backlit state when at least one of a first requirement that the driver's eye opening abnormality rate is equal to or greater than a predetermined abnormality rate and a second requirement that the number of blinks of the driver's eyes is equal to or greater than a predetermined number is satisfied.

[0019] The driver monitoring device 11 may measure the amount of light emitted from a light source (the sun in this case), and if the amount of light is equal to or greater than a predetermined amount, determine that the front of the vehicle is backlit as seen by the driver. The determination unit 113 may also determine by image analysis whether direct sunlight or a spotlight from the sun is hitting the driver's eyes, and determine whether the vehicle is backlit based on the result.

[0020] When it is determined that the front of the vehicle is backlit, the image projection unit 114 projects the image in front of the vehicle in a direction different from the direction in which the sun is present as seen from the driver. For example, the image projection unit 114 projects the image in front of the vehicle in a direction in which the sun is not visible as seen from the driver. That is, the image projection unit 114 may project the image in front of the vehicle in a direction that is not dazzling (a direction that is not backlit). The direction that is not backlit means, for example, projecting the image onto the windshield in front of the driver's seat and passenger seat, or projecting onto the windshield in a direction that includes an extension of the driver's line of sight (a direction that is not dazzling).

[0021] Instead of the image projection unit 114, the image in front of the vehicle may be displayed on a monitor screen of an in-vehicle device located in front of the driver and between the driver's seat and the passenger seat. Displaying on the monitor screen means displaying at a position that is off the line connecting the driver's eyes and the sun (light source), so that the direction is not dazzling for the driver. The image projection unit 114 may project the image in front of the vehicle captured by the image capture unit 111, the image having a brightness difference equal to or greater than a predetermined brightness difference.

[0022] The image projection unit 114 continues to project the image of the area in front of the vehicle while the backlighting state continues. When the image projection unit 114 determines that the backlighting state is no longer occurring, it stops projecting the image of the area in front of the vehicle. When the image projection unit 114 determines that the backlighting state has been released, it stops the projection and returns to normal operation.

[0023] Furthermore, the driver monitoring device 11 can detect (calculate) the position of the sun by the following methods, but is not limited to these. -Calculated based on the direction of the shadow in the video (image) captured by the imaging unit 111. -Calculations are based on footage (images) from the drive recorder installed in the vehicle. - Calculated based on the vehicle's orientation and GPS (Global Positioning System) location information.

[0024] <Effects> The driver monitoring device according to the embodiment has the feature that "when it is determined that the vehicle is in a backlit state based on a comparison between the acquired bio-parameters (e.g., eye opening abnormality rate, number of blinks) and previously acquired bio-parameters during normal driving, the device projects an image ahead of the vehicle in a direction different from the direction in which the sun is present as seen from the driver." This makes it possible to provide a driver monitoring device that can avoid dangerous driving due to poor visibility in backlit conditions and ensure safe driving.

[0025] Moreover, according to the embodiment, it is possible to reduce the driving load caused by poor visibility in backlight. Also, according to the embodiment, it is possible to provide driving assistance without making the driver move his / her line of sight significantly. Furthermore, when an image in front of the vehicle is projected onto the windshield in front of the passenger seat, not only the driver but also the passenger (the person sitting in the passenger seat) can receive the poor visibility assistance, so that more effective driving assistance can be provided.

[0026] <Operation> FIG. 2 is a flowchart illustrating the operation of the driver monitoring device according to the embodiment.

[0027] 2, the driver monitoring device 11 starts driver monitoring of the driver of the vehicle (step S101). The driver monitoring monitors the amount of change in biological parameters such as the rate of eye opening abnormality of the driver and the number of blinks of the driver's eyes.

[0028] The driver monitoring device 11 checks whether a predetermined monitoring period has elapsed (step S102).

[0029] When the predetermined monitoring period has elapsed (step S102: YES), the driver monitoring device 11 performs an extraction process of the monitored biological parameters (step S103).

[0030] The driver monitoring device 11 checks whether the amount of change in the bioparameter is different from the bioparameter during normal driving (step S104). That is, the driver monitoring device 11 checks whether the difference between the acquired bioparameter value and the bioparameter value during normal driving acquired in advance is equal to or greater than a predetermined difference. This is to detect the backlight state by determining the degree of deviation from the bioparameter during normal driving.

[0031] When the amount of change in the biological parameter deviates from the biological parameter during normal driving (step S104: YES), the driver monitoring device 11 performs a process of starting projection of an image in front of the vehicle captured by the front camera (step S105).

[0032] Examples of images that the driver monitoring device 11 projects in backlit conditions include, but are not limited to, the following images. -High dynamic range CMOS image sensor output image. - Millimeter wave radar output image is less affected by rain, fog, backlight, etc. Output image using multi-fusion sensing function that combines the above images.

[0033] The locations where the driver monitoring device 11 projects an image in backlighting include the following, but are not limited to the following images. - A location on the windshield that is an extension of the driver's line of sight, not a location that requires the driver to move their eyes to view a car navigation device, etc. -Not only the windshield in front of the driver's seat, but also the windshield in front of the passenger seat.

[0034] The driver monitoring device 11 checks whether a predetermined monitoring continuation period has elapsed (step S106).

[0035] When the predetermined monitoring continuation period has elapsed (step S106: YES), the driver monitoring device 11 performs a biological parameter extraction process (step S107).

[0036] The driver monitoring device 11 checks whether or not the value of the biological parameter has returned to the value during normal driving (step S108).

[0037] When the value of the biological parameter has returned to the value during normal driving (step S108: YES), the driver monitoring device 11 confirms that the value of the biological parameter has not changed for a certain period of time (step S109).

[0038] The method in steps S108-S109 is a method for detecting that the backlight state has disappeared, and specific methods thereof include the following, but are not limited to these. The system monitors the rate at which the driver's eyes are open abnormally and / or the amount of change in bio-parameters such as the number of blinks, and determines whether the bio-parameters have returned to normal driving values. If it is then confirmed that the bio-parameters have remained stable (there is little change) for a certain period of time, it is determined (detected) that the driver is no longer in a backlit condition.

[0039] After step S108: YES, if there is no change in the value of the biological parameter for a certain period of time (step S109: YES), the driver monitoring device 11 performs processing to end the projection of the image ahead of the vehicle captured by the front camera (step S110).

[0040] After step S110, the driver monitoring device 11 checks whether or not to end the driver monitoring (step S111).

[0041] If the driver monitoring device 11 determines that the driver monitoring is to be ended (step S111: YES), the driver monitoring device 11 performs a process of ending the driver monitoring (step S112). If the driver monitoring device 11 determines that the driver monitoring is not to be ended (step S111: NO), the driver monitoring device 11 returns to step S102.

[0042] <Features> The features of the driver monitoring device according to the embodiment are as follows. The driver monitoring device's onboard camera detects whether the vehicle is in backlight. After detecting a backlit condition, the image from the vehicle-mounted camera is projected in a direction that is not dazzling to the driver, for example, on the windshield in front of the driver's seat and / or passenger seat. Projection continues as long as the backlighting condition continues. The driver monitoring device's onboard camera is used to detect that the vehicle is not in a backlit condition. If it is determined that there is no backlighting, the projection will be stopped (cancelled) and normal operation will resume. In this way, backlit conditions are not avoided (prevented) but the driving load when backlit is present is reduced.

[0043] It should be noted that a part or all of the above processes may be executed by a computer program. The above-mentioned program may be stored using various types of non-transitory computer readable media and provided to a computer. The non-transitory computer readable medium includes various types of tangible storage media. Examples of the non-transitory computer readable medium include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program may also be provided to a computer by various types of transitory computer readable media. Examples of the transitory computer readable medium include electrical signals, optical signals, and electromagnetic waves. The transitory computer readable medium may provide the program to a computer via a wired communication path such as an electric wire and an optical fiber, or via a wireless communication path.

[0044] The invention has been specifically described above based on the embodiment, but it goes without saying that the present invention is not limited to the above embodiment and can be modified in various ways without departing from the spirit of the invention. [Explanation of symbols]

[0045] 11. Driver monitoring device 111…Camera Department 112...Parameter acquisition unit 113...judgment department 114…Image projection unit

Claims

1. An image capturing unit that captures a front image of the vehicle from the vehicle; A parameter acquisition unit that acquires biometric parameters related to the eyes of a driver who drives the vehicle; a determination unit that determines whether or not the front of the vehicle is in a backlight state as seen from the driver based on a comparison between the acquired biometric parameters and the biometric parameters during normal driving that have been acquired in advance; an image projection unit that projects the image in front of the vehicle in a direction different from a direction in which the sun is present as seen from the driver when it is determined that the front of the vehicle is in the backlit state; A driver monitoring device comprising:

2. The biological parameters include an eye opening abnormality rate of the driver, which is a ratio of a time during which the driver's eyes are not normally open in a predetermined time, and a blink frequency of the driver, which is a blink frequency of the driver's eyes in the predetermined time, The determination unit determines that the vehicle is in a backlit state when at least one of a first requirement that an eye opening abnormality rate of the driver is equal to or greater than a predetermined eye opening abnormality rate and a second requirement that a number of blinks of the driver's eyes is equal to or greater than a predetermined number is satisfied. The driver monitoring device according to claim 1 .

3. The image projection unit projects the image ahead of the vehicle in a direction in which the sun is not visible to the driver. The driver monitoring device according to claim 1 .

4. Instead of the image projection unit, the vehicle front image is displayed on a monitor screen located in front of the driver and between the driver's seat and the passenger seat. The driver monitoring device according to claim 1 .

5. The photographing unit photographs the front of the vehicle using LiDAR (Light Detection And Ranging). The driver monitoring device according to claim 1 .

Images