Driver evaluation information providing device and evaluation information providing method

Abstract

To restrict a driver of a vehicle from feeling discomfort with respect to an output of warning corresponding to a driving involvement degree by providing the driver with information on an evaluation result of the driving involvement degree of the driver recognized by a device.SOLUTION: A driver evaluation information provision device includes: a driver monitoring section for monitoring a state of a driver based on at least a photographed image of the driver of a vehicle photographed by a driver monitor camera of the vehicle; a driving involvement degree evaluation section for evaluating a driving involvement degree of the driver based on the monitoring result of the state of the driver by the driver monitoring section; and an information provision section for providing information on the driving involvement degree evaluated by the driving involvement degree evaluation section to the driver.SELECTED DRAWING: Figure 5

Description

Technical Field

[0004] , , , , ,

[0005] , , , , ,

[0006] , The information provision unit is characterized by providing information to the driver regarding the degree of driving involvement based on the length of the bar-shaped illuminated portion of a display unit located in front of the driver's seat of the vehicle.

[0001] The present invention relates to a driver evaluation information providing apparatus and an evaluation information providing method.

Background Art

[0002] Conventionally, as a technical document related to a driver evaluation information providing apparatus, International Publication No. 2020-188634 is known. This publication shows a driver monitoring apparatus that outputs a warning according to the state of a driver of a vehicle.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, when there is a large difference between the evaluation of the driver's state by the vehicle device and the driver's own evaluation of their state, the driver may feel discomfort with respect to warnings or the like given to the driver by the device.

Means for Solving the Problems

[0005] A driver evaluation information providing apparatus according to an aspect of the present invention includes: a driver monitoring unit that monitors a driver's state based at least on an imaging image of a driver of a vehicle captured by a driver monitoring camera of the vehicle; a driving participation degree evaluation unit that evaluates a driving participation degree of the driver based on a monitoring result of the driver's state by the driver monitoring unit; and an information providing unit that provides information on the driving participation degree evaluated by the driving participation degree evaluation unit to the driver. The information provision unit is characterized by providing information to the driver regarding the degree of driving involvement based on the length of the bar-shaped illuminated portion of a display unit located in front of the driver's seat of the vehicle. .

[0006] According to one aspect of the present invention, the driver evaluation information providing device can provide the vehicle driver with information on the degree of driver involvement being evaluated by the device. Therefore, even if a warning is issued to the driver or autonomous driving or driver assistance is terminated according to the degree of driver involvement, it is possible to suppress the driver from feeling uncomfortable.

[0007] In a driver evaluation information providing device according to one aspect of the present invention, the information providing unit provides information when driving control as autonomous driving or driver assistance is being performed in the vehicle. When the degree of involvement in driving is reduced, The driver may be informed that their level of driving involvement is approaching the termination threshold, which is the threshold at which driving control is terminated when the level of driving involvement falls below a certain level. This driver evaluation information provider informs the driver when their level of driving involvement is approaching the termination threshold. This helps to prevent the driver from feeling uncomfortable when the automated driving or driving assistance system terminates due to a decrease in the driver's level of driving involvement as evaluated by the device.

[0008] In a driver evaluation information providing device according to one aspect of the present invention, the information providing unit provides information when driving control as autonomous driving or driver assistance is being performed in the vehicle. And when the degree of involvement in the operation is increasing, The driver may be informed that their level of driving involvement is moving away from the termination threshold, which is the threshold at which driving control is terminated when the level of driving involvement falls below a certain level. This driver evaluation information provision device informs the driver when the level of driving involvement evaluated by the device is moving away from the termination threshold. This makes it possible to motivate drivers who wish to continue with autonomous driving or driving assistance to increase their level of driving involvement.

[0009] In a driver evaluation information providing device according to one aspect of the present invention, the information providing unit provides information while driving control is being performed. , The length of the illuminated portion of the shape may be used to continuously provide information to the driver regarding the degree of involvement in the driving process. According to this driver evaluation information provision device, while driving control is being performed, the display unit continuously provides information to the driver, so the driver can check the degree of driving involvement evaluated by the device at any time during autonomous driving or driving assistance.

[0010] In a driver evaluation information providing device according to one aspect of the present invention, the information providing unit highlights the ends of the bar-shaped illuminated portion of the display unit whose length changes according to the change in the degree of driver involvement, and may distinguish between highlighting the ends when the length of the illuminated portion shortens due to a decrease in the degree of driver involvement and highlighting the ends when the length of the illuminated portion lengthens due to an increase in the degree of driver involvement. According to this driver evaluation information providing device, the ends of the illuminated bar-shaped section of the display unit whose length changes according to the change in the driver's level of involvement are highlighted, making it easier for the driver to notice changes in their level of involvement. Furthermore, by differently highlighting the ends when the length of the illuminated section shortens compared to when the length of the illuminated section lengthens, the device makes it easier for the driver to understand whether their level of involvement is decreasing or increasing.

[0011] Another aspect of the present invention is a method for providing evaluation information to a driver evaluation information providing device that provides information to a vehicle driver, wherein the driver evaluation information providing device at The system monitors the driver's condition based at least on the image of the vehicle driver captured by the vehicle's driver monitoring camera, and provides driver evaluation information to the driver evaluation information provider. at Based on the monitoring results of the driver's status, the degree of the driver's involvement in driving is evaluated and the driver evaluation information is provided to the driver evaluation information provision device. at , The length of the bar-shaped illuminated section of the display unit located in front of the driver's seat of the vehicle Provide drivers with information on their level of involvement in driving. Characterized by .

[0012] According to the evaluation information providing method of the driver evaluation information providing device according to one aspect of the present invention, it is possible to provide information on the driving involvement degree of the driver evaluated by the device to the driver of the vehicle. Therefore, even if a warning is output to the driver or automatic driving or driving support is terminated according to the driving involvement degree, it is possible to suppress the driver from feeling discomfort.

Effect of the Invention

[0013] According to each aspect of the present invention, by providing the evaluation result of the driving involvement degree of the driver recognized by the device to the driver of the vehicle, it is possible to suppress the driver from feeling discomfort with respect to the output according to the driving involvement degree.

Brief Description of the Drawings

[0014] [Figure 1] It is a block diagram showing a driver evaluation information providing device according to an embodiment. [Figure 2] It is a diagram showing an example of a display unit provided in front of the driver's seat. [Figure 3] It is a graph showing an example of the change in the information providing content according to the driving involvement degree. [Figure 4] It is a table showing an example of the information providing content corresponding to the driving involvement degree and the display mode of the display unit. [Figure 5] It is a flowchart showing an example of the evaluation information providing method of the driver evaluation information providing device. [Figure 6] It is a flowchart showing an example of a service level switching method as an information providing target.

Modes for Carrying Out the Invention

[0015] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0016] FIG. 1 is a block diagram showing a driver evaluation information providing apparatus 100 according to an embodiment. The driver evaluation information providing apparatus 100 shown in FIG. 1 is mounted on a vehicle and provides information on the degree of driving involvement evaluated by the apparatus to the driver of the vehicle. The degree of driving involvement is the degree of involvement or concentration in the driving by the driver.

[0017] The vehicle is, for example, a vehicle capable of executing either automatic driving or driving support. Automatic driving is a driving control for automatically running the vehicle without the driver's driving operation. In the driving control as automatic driving, both the vehicle speed and steering of the vehicle are controlled without the driver's driving operation. Automatic driving corresponds to, for example, control at an automatic driving level of 3 or higher recognized by the Society of Automotive Engineers (SAE) of the United States.

[0018] Driving support is a vehicle control for supporting the driver's driving operation so that the vehicle runs appropriately. The driving support according to the present embodiment includes vehicle driving controls such as ACC [Adaptive Cruise Control], LKA [Lane Keeping Assist], and parking assist. In the driving control as driving support, at least one of the vehicle speed and steering of the vehicle is controlled while the driver is not inputting a driving operation. Driving support corresponds to, for example, automatic driving levels 1 and 2 recognized by the Society of Automotive Engineers (SAE) of the United States. Note that the vehicle does not necessarily have to be capable of executing automatic driving or driving support.

[0019] [Configuration of Driver Evaluation Information Providing Apparatus] Hereinafter, the configuration of the driver evaluation information providing apparatus 100 according to the present embodiment will be described. As shown in FIG. 1, the driver evaluation information providing apparatus 100 includes a driver evaluation ECU 10 [Electronic Control Unit] that comprehensively manages the apparatus.

[0020] The driver evaluation ECU 10 is an electronic control unit having a CPU (Central Processing Unit) and a memory unit such as ROM (Read Only Memory) or RAM (Random Access Memory). The driver evaluation ECU 10 implements various functions, for example, by executing a program stored in the memory unit using the CPU. The driver evaluation ECU 10 may be composed of multiple electronic units.

[0021] The driver evaluation ECU 10 is connected to the driver monitor camera 1, external sensors 2, steering sensor 3, accelerator pedal sensor 4, brake pedal sensor 5, display unit 6, speaker 7, vibration unit 8, and vehicle control unit 20.

[0022] The driver monitor camera 1 is a camera that captures images of the driver. The driver monitor camera 1 is installed, for example, on the cover of the vehicle's steering column, in front of the driver's seat. Multiple driver monitor cameras 1 may be installed to capture images of the driver from multiple directions. The driver monitor camera 1 transmits the captured images of the driver to the driver evaluation ECU 10.

[0023] External sensor 2 is a detection device that detects the surrounding environment of the vehicle. External sensor 2 includes an external camera and a radar sensor. The external camera includes, for example, a front camera that is mounted behind the vehicle's windshield and captures images of the area in front of the vehicle. The external camera may also include cameras mounted on the rear and sides of the vehicle. The external camera transmits the image information of the area outside the vehicle to the driver evaluation ECU 10.

[0024] A radar sensor is a detection device that uses radio waves (e.g., millimeter waves) or light to detect objects around a vehicle. Examples of radar sensors include millimeter-wave radar and lidar (LIDAR: Light Detection and Ranging). The radar sensor detects obstacles by transmitting radio waves or light around the vehicle and receiving the radio waves or light reflected by objects. The radar sensor transmits the detected object information to the driver evaluation ECU 10.

[0025] The steering sensor 3 is a sensor that detects the driver's grip on the vehicle's steering wheel. The steering sensor 3 may include a pressure sensor provided on the steering wheel, or it may include a touch sensor that detects the electric current of the human body. The steering sensor 3 may also be capable of detecting the driver's operation of the steering wheel. The steering sensor 3 transmits the detection result, such as the driver's grip on the steering wheel, to the driver evaluation ECU 10. Note that the steering wheel is not limited to a wheel shape, but can be anything that functions as a steering wheel.

[0026] The accelerator pedal sensor 4 detects when the driver depresses the vehicle's accelerator pedal. The accelerator pedal sensor 4 is configured to detect when the driver's foot is on the accelerator pedal. The accelerator pedal sensor 4 transmits the detection results, such as when the driver depresses the vehicle's accelerator pedal, to the driver evaluation ECU 10.

[0027] The brake pedal sensor 5 detects when the driver depresses the brake pedal. The brake pedal sensor 5 is configured to detect when the driver has their foot on the brake pedal. The brake pedal sensor 5 transmits the detection results, such as when the driver depresses the vehicle's brake pedal, to the driver evaluation ECU 10.

[0028] The display unit 6 is a device used to provide visual information to the driver. The display unit 6 is installed, for example, in front of the driver's seat. Here, Figure 2 shows an example of a display unit installed in front of the driver's seat. Figure 2 shows a first display unit 6a, a second display unit 6b, and a center display 6c. As shown in Figure 2, the display unit 6 is composed of, for example, a first display unit 6a, a second display unit 6b, and a center display 6c.

[0029] The first display unit 6a is a bar-shaped light source installed on the vehicle's instrument panel, extending from left to right across the vehicle. The first display unit 6a is positioned within the driver's field of vision while the driver is monitoring the area in front of the vehicle. The first display unit 6a can be made of, for example, LEDs. As an example, the first display unit 6a is installed from in front of the driver's seat to in front of the passenger seat.

[0030] The first display unit 6a provides the driver with information about their level of involvement in driving, for example, by the length of a bar-shaped illuminated portion L. The end of the illuminated portion L whose length changes according to the change in the level of involvement in driving is indicated by the symbol La. The length of the first display unit 6a is not particularly limited and may be the length from the front of the driver's seat to the center of the vehicle, or it may be a shorter length.

[0031] The second display unit 6b is a light source provided on the upper part of the vehicle's steering wheel 50. The second display unit 6b can be, for example, a steering illumination system made of LEDs. The second display unit 6b can be a rectangular light source extending to the left and right of the vehicle. The shape of the second display unit 6b is not particularly limited. The shape of the second display unit 6b may be a square, a rhombus, or an ellipse. The second display unit 6b may be provided on the cover of the steering column instead of on the upper part of the steering wheel 50.

[0032] The center display 6c is a display located in the center of the vehicle's instrument panel. The center display 6c is used, for example, for route guidance by the vehicle's navigation system. The center display 6c does not necessarily need to be located in the center of the instrument panel. Furthermore, the center display 6c does not necessarily need to be usable as a display for a navigation system.

[0033] The display unit 6 does not necessarily have to include all of the first display unit 6a, the second display unit 6b, and the center display 6c; it may include only one of them. The display unit 6 may not include all of the first display unit 6a, the second display unit 6b, and the center display 6c, but may include other devices that provide visual information to the driver. The display unit 6 may include, for example, a HUD (Head Up Display) that projects image information onto the vehicle's windshield, or a MID (Multi-Information Display) for the instrument cluster. The display unit 6 only needs to include at least one of the display devices described above.

[0034] Speaker 7 is a device used to provide auditory information to the driver. Speaker 7 is configured to output sound and voice to the driver inside the vehicle.

[0035] The vibration unit 8 is a device used to provide information to the driver through vibration. This information provision through vibration is combined with visual or auditory information provision. The vibration unit 8 includes, for example, a steering actuator that vibrates the vehicle's steering wheel.

[0036] The vibration unit 8 may include a belt actuator that retracts the seat belt in the driver's seat of the vehicle. The vibration unit 8 may also provide information to the driver by lightly pulling the driver's seat belt. The vibration unit 8 may also include a seat actuator that vibrates the armrest or seat of the driver's seat.

[0037] The vehicle control unit 20 controls the vehicle's movement by transmitting control signals to the vehicle's drive units (engine actuator, brake actuator, steering actuator, etc.). The vehicle control unit 20 performs autonomous driving or driver assistance as authorized by the driver. A detailed explanation of autonomous driving and driver assistance is omitted. If the vehicle control unit 20 receives a signal from the driver evaluation ECU 10 to terminate the driving control (driving control as autonomous driving or driver assistance) while performing autonomous driving or driver assistance, it terminates the driving control that is currently being performed.

[0038] The vehicle control unit 20 may, when it receives a signal from the driver evaluation ECU 10 to terminate the driving control during the execution of autonomous driving of the vehicle, transition to driver assistance driving control. In other words, the vehicle control unit 20 may, when it receives a signal from the driver evaluation ECU 10 to terminate the driving control, transition from autonomous driving level 3 or higher to autonomous driving level 2.

[0039] Next, the functional configuration of the driver evaluation ECU 10 will be described. As shown in Figure 1, the driver evaluation ECU 10 has a driver monitoring unit 11, a driving involvement evaluation unit 12, and an information provision unit 13.

[0040] The driver monitoring unit 11 monitors the driver's condition based on the image of the vehicle's driver captured by the vehicle's driver monitor camera 1. The driver monitoring unit 11, for example, recognizes the direction of the driver's face. When the driver is paying attention to driving or operating the vehicle, the driver's face is considered to be facing forward. The driver monitoring unit 11 recognizes the direction of the driver's face using well-known image processing techniques.

[0041] The driver monitoring unit 11 may recognize the orientation of the driver's face using a machine learning model. The machine learning model takes, for example, an image of the driver captured by the driver monitor camera 1 as input and outputs the result of recognizing the orientation of the face. The machine learning model is, for example, a neural network such as a convolutional neural network (CNN). The neural network can include multiple layers, including multiple convolutional layers and pooling layers. A deep learning network using deep learning may be used as the neural network. The machine learning models that appear in this embodiment from now on can also have a similar neural network structure.

[0042] The driver monitoring unit 11 may monitor the driver's eye condition. The driver's eye condition includes the driver's gaze. The driver's eye condition may include at least one of the following: eye saccades, blink frequency, eye closing speed, eye closing state (upper eyelid position), etc. The driver monitoring unit 11 recognizes the driver's eye condition, including gaze, using well-known image processing techniques or machine learning models.

[0043] The driver monitoring unit 11 may recognize that the driver is distracted or looking away from the driver based on the direction of the driver's face or the state of their eyes. Distracted driving refers to a state in which the driver is looking at scenery outside the vehicle that is unrelated to driving the vehicle. Looking away refers to a state in which the driver is looking at the inside of the vehicle that is unrelated to driving the vehicle. Looking away includes the state in which the driver is talking to other passengers, the state in which the driver is operating the vehicle's navigation system or other equipment, and the state in which the driver is operating a smartphone or the like.

[0044] The driver monitoring unit 11 may recognize that the driver is in a state of inattention based on the direction of the driver's face or the state of their eyes. Inattention includes situations where the driver is unable to concentrate on driving due to lack of sleep or when the driver is preoccupied with other thoughts. The driver monitoring unit 11 may recognize that the driver is in a state of inattention based on the saccades of the eyeballs, blinking frequency, eye closing speed, and eye closing state (upper eyelid position).

[0045] The driver monitoring unit 11 may monitor the driver's state based on the detection results of the steering sensor 3. If the driver is paying attention to driving or operating the vehicle, it is assumed that the driver is gripping the steering wheel. The driver monitoring unit 11 recognizes from the detection results of the steering sensor 3 that the driver is gripping the steering wheel. The driver monitoring unit 11 may distinguish and recognize whether the driver is gripping the steering wheel with both hands or with one hand. The driver monitoring unit 11 may also recognize the amount of steering input to the steering wheel by the driver.

[0046] The driver monitoring unit 11 may monitor the driver's condition based on the detection results of the accelerator pedal sensor 4 or the brake pedal sensor 5. If the driver's foot is on the accelerator pedal or brake pedal, it can be assumed that the driver is likely paying attention to driving or operating the vehicle.

[0047] The driver monitoring unit 11 recognizes that the driver's foot is on the accelerator pedal or brake pedal based on the detection result of the accelerator pedal sensor 4 or the brake pedal sensor 5. The driver monitoring unit 11 may also recognize the amount the driver has pressed down on the accelerator pedal or brake pedal.

[0048] The driver involvement evaluation unit 12 evaluates the driver's level of involvement based on the driver monitoring unit 11's monitoring of the driver's state. For example, if the driver is distracted, the driver involvement evaluation unit 12 evaluates the level of involvement lower than if the driver is looking straight ahead. However, if the driver's gaze is directed towards the side mirror, the driver involvement level is not evaluated lower.

[0049] The driving involvement evaluation unit 12 may rate the degree of driving involvement lower when the driver is looking away compared to when the driver is looking straight ahead. However, if the driver's gaze is directed towards the rearview mirror, the driving involvement will not be rated lower. The driving involvement evaluation unit 12 may rate the degree of driving involvement lower when the driver is looking away and also operating an information device installed in the vehicle for navigation, music playback, telephone, or other purposes, compared to when the driver is not operating an information device. The driving involvement evaluation unit 12 acquires signals related to the driver's operation from the information device installed in the vehicle.

[0050] The driving involvement evaluation unit 12 may rate the degree of driving involvement lower when the driver is in a state of insufficient concentration compared to when the driver is not in a state of insufficient concentration. The driving involvement evaluation unit 12 may rate the degree of driving involvement lower when the driver is in a state of insufficient concentration due to sleep deprivation compared to when the driver is in a state of insufficient concentration for other reasons. The driving involvement evaluation unit 12 may rate the degree of driving involvement lower the longer the duration of the driver's distracted state, looking away state, or insufficient concentration state.

[0051] The driver involvement evaluation unit 12 may evaluate the driver's involvement lower if the frequency of the driver performing surrounding monitoring actions is below a certain value, compared to when the frequency of the driver performing surrounding monitoring actions is above a certain value. Surrounding monitoring actions can include checking not only the area in front of the vehicle, but also side mirrors, rearview mirrors, adjacent lanes, adjacent sidewalks, and traffic lights in front of the vehicle.

[0052] The driver involvement evaluation unit 12 may evaluate the driver's level of involvement in driving based on the external driving environment detected by the external sensor 2. The driver involvement evaluation unit 12 determines whether the driver is facing the direction of the vehicle's lane based on the direction of the vehicle's lane detected by the external sensor 2 and the direction of the driver's face recognized by the driver monitoring unit 11. When a driver is focused on driving or operating the vehicle, it is thought that the direction of the driver's face will change according to the direction of the curve, such as when the lane is curved. The driver involvement evaluation unit 12 may evaluate the level of involvement lower when the driver is not facing the direction of the vehicle's lane compared to when the driver is facing the direction of the vehicle's lane.

[0053] Furthermore, the driving involvement evaluation unit 12 may rate the degree of driving involvement lower if there are many visual objects detected by the external sensor 2, such as traffic lights, road signs, preceding vehicles, overtaking vehicles, and pedestrians in front of the vehicle, that the driver has never seen. The driving involvement evaluation unit 12 may rate the degree of driving involvement lower if the number of times the driver sees a visual object per certain period of time (the number of times any visual object is seen) is less than a predetermined value, compared to when the number of sightings is equal to or greater than the predetermined value. Well-known techniques can be used to determine the driver's visibility.

[0054] The driver involvement evaluation unit 12 may evaluate the driver's level of involvement based on the vehicle's acceleration or deceleration and the driver's head position. When a driver is focused on driving or moving the vehicle, they anticipate the vehicle's driving conditions (starting, accelerating, decelerating, stopping) and brace themselves for changes in longitudinal acceleration. Therefore, their head movements are likely to be smaller compared to a driver who is not focused on driving. The driver involvement evaluation unit 12 may rate the level of driver involvement lower the greater the driver's head movement during vehicle acceleration or deceleration.

[0055] The driving involvement evaluation unit 12 may rate the degree of driving involvement lower when the driver is not holding the steering wheel compared to when the driver is holding the steering wheel. The driving involvement evaluation unit 12 may rate the degree of driving involvement lower the longer the time elapsed since the driver last held the steering wheel when the driver is not holding it. Furthermore, the driving involvement evaluation unit 12 may rate the degree of driving involvement lower when the driver is holding the steering wheel with one hand compared to when the driver is holding the steering wheel with both hands.

[0056] The driving involvement evaluation unit 12 may rate the degree of driving involvement lower when the driver's foot is not on the accelerator pedal or brake pedal compared to when the driver's foot is on the accelerator pedal or brake pedal. The driving involvement evaluation unit 12 may rate the degree of driving involvement lower the longer the time elapsed since the driver's foot was last placed on the accelerator pedal or brake pedal.

[0057] The driver involvement evaluation unit 12 may evaluate the degree of driver involvement lower if the number of overrides per certain period of time in which the driver intervenes in the vehicle's movement by steering the steering wheel, pressing the accelerator pedal, or pressing the brake pedal is less than a predetermined value, compared to when the number of overrides per certain period of time is equal to or greater than a predetermined value.

[0058] The driver involvement evaluation unit 12 may change the method of evaluating driver involvement depending on whether the vehicle is performing autonomous driving or driver assistance. For example, when autonomous driving is being performed, the driver involvement evaluation unit 12 may reduce the amount of decrease in the evaluation of driver involvement due to the driver being distracted, looking away, or lacking concentration, compared to when driver assistance is being performed. For example, when autonomous driving is being performed, the driver involvement evaluation unit 12 may reduce the amount of decrease in the evaluation of driver involvement due to the driver not holding the steering wheel, compared to when driver assistance is being performed. For example, when autonomous driving is being performed, the driver involvement evaluation unit 12 may reduce the amount of decrease in the evaluation of driver involvement due to the driver's foot not being on the accelerator pedal or brake pedal, compared to when driver assistance is being performed.

[0059] The information provision unit 13 provides the driver with information on the driver's level of involvement as evaluated by the driving involvement evaluation unit 12. The information provision unit 13 provides information to the driver, for example, using the display unit 6. The information provided to the driver by the information provision unit 13 is not limited to visual information provision, but may also be provided by sound or voice, or by vibration of the steering wheel or other methods.

[0060] The information provision unit 13 provides information to the driver on the degree of driving involvement by, for example, shortening the bar-shaped illuminated portion in the first display unit 6a of the display unit 6 as the degree of driving involvement decreases (see Figure 2). The information provision unit 13 may change the length of the illuminated portion extending toward the passenger side based on the driver's side end of the first display unit 6a, or it may change the length of the illuminated portions extending to the left and right based on the central position of the first display unit 6a.

[0061] The information providing unit 13 may highlight the end of the illuminated portion on the side where the length of the illuminated portion changes (for example, end La in Figure 2). When the illuminated portion expands or contracts horizontally, both ends are highlighted. The information providing unit 13 may increase the brightness of the end or change the color of the end as a highlight. The color of the end may be changed to a darker or lighter color compared to the other parts, or to a different color. The information providing unit 13 may shorten the illumination cycle of the end compared to other parts or add animation effects such as fading as a highlight. When displaying a bar-shaped illuminated portion on the HUD, various animation effects can be employed, such as displaying multiple star-shaped images around the end.

[0062] The information providing unit 13 may differentiate the highlighting of the end when the length of the illuminated portion shortens due to a decrease in the degree of driver involvement and the highlighting when the length of the illuminated portion lengthens due to an increase in the degree of driver involvement. For example, the information providing unit 13 changes the color of the end as a highlight when the length of the illuminated portion shortens and when the length of the illuminated portion lengthens. The information providing unit 13 may also change the blinking cycle and animation effect of the end as a highlight. In addition, the information providing unit 13 may change the blinking cycle of the end to a shorter time the faster the speed of change in the length of the illuminated portion (the speed of increase or decrease in the degree of driver involvement) is, and may change the color of the end to a darker color the faster the speed of change in the length of the illuminated portion is.

[0063] The information provision unit 13 may, for example, provide information on the degree of driving involvement to the driver by shortening the flashing cycle of the light source as the degree of driving involvement decreases on the second display unit 6b on the steering wheel 50 in the display unit 6. The information provision unit 13 may also provide information on the degree of driving involvement to the driver by, for example, displaying a numerical value or text corresponding to the degree of driving involvement on the center display 6c in the display unit 6. The information provision unit 13 may use a HUD or MID instead of the first display unit 6a, the second display unit 6b, and the center display 6c.

[0064] The information provision unit 13 continuously provides real-time information on the degree of driving involvement when the driver has turned on the driving involvement information provision function. However, the information provision unit 13 does not necessarily have to provide information continuously; it may provide information at regular intervals. The regular interval is not particularly limited. It may be 1 second, 5 seconds, or 10 seconds.

[0065] The information provision unit 13 may determine whether the degree of driver involvement has fallen below the termination threshold when driving control as autonomous driving or driver assistance is being performed in the vehicle. The termination threshold is the threshold for terminating autonomous driving or driver assistance driving control. The termination threshold for autonomous driving and the termination threshold for driver assistance can be different values. The termination threshold for autonomous driving may be greater than the termination threshold for driver assistance.

[0066] When the information provision unit 13 determines that the degree of driver involvement has fallen below the termination threshold, it requests the driver to start manual driving and transmits a driving control termination signal to the vehicle control unit 20. Upon receiving the driving control termination signal, the vehicle control unit 20 terminates the driving control as either automatic driving or driver assistance. Note that a request to start manual driving when the vehicle is performing automatic driving corresponds to a so-called hands-on request.

[0067] The information provision unit 13 may provide the driver with information on the relationship between the degree of driving involvement and the termination threshold when driving control as autonomous driving or driving assistance is being performed in the vehicle. If the degree of driving involvement is decreasing, the information provision unit 13 will inform the driver that the degree of driving involvement is approaching the termination threshold. If the degree of driving involvement is increasing, the information provision unit 13 will inform the driver that the degree of driving involvement is moving away from the termination threshold. The information provision unit 13 may also provide the driver with information on the relationship between the degree of driving involvement and the termination threshold as the remaining time until the driving control is terminated.

[0068] Specifically, the information provision unit 13 can provide information to the driver in the manner shown in Figures 3 and 4. This explanation assumes that the vehicle is in the process of autonomous driving. Figure 3 is a graph showing an example of how the information provided changes according to the degree of driver involvement.

[0069] In Figure 3, the vertical axis corresponds to the degree of driver involvement, and the horizontal axis corresponds to the service level. "Hands-off" is the level at which the driver is allowed to release the steering wheel. "Hands-on soon" is the level at which the driver will soon be asked to take hands-on action due to a decrease in driver involvement. "Hands-on request" is the level at which the driver is asked to prepare for manual driving. Figure 3 shows the termination warning threshold Th1 and the termination determination threshold Th2. The termination warning threshold Th1 is the threshold used to notify the driver that the end of autonomous driving is approaching.

[0070] Figure 4 is a table showing an example of the information provided and the display mode of the display unit corresponding to the level of driver involvement. In Figure 4, a "high" level of driver involvement corresponds to the case in Figure 3 where the level of driver involvement is equal to or greater than the termination warning threshold Th1. A "medium" level of driver involvement corresponds to the case in Figure 3 where the level of driver involvement is less than the termination warning threshold Th1 but equal to or greater than the termination judgment threshold Th2. A "low" level of driver involvement corresponds to the case in Figure 3 where the level of driver involvement is less than the termination judgment threshold Th2.

[0071] As shown in Figures 3 and 4, the information provision unit 13 provides the driver with "hands off" information when the level of driving involvement is "high". The information provision unit 13 sets the bar-shaped illuminated portion of the first display unit 6a to the longest display. The length of the bar-shaped illuminated portion may change continuously rather than in steps. The information provision unit 13 sets the flashing cycle of the first display unit 6a to either continuous illumination (no flashing) or a 10-second cycle. The information provision unit 13 also sets the color of the illuminated portion of the first display unit 6a to blue.

[0072] The information provision unit 13 provides the driver with the message "Hands-on coming soon" when the level of driver involvement is "medium". The information provision unit 13 sets the bar-shaped illuminated portion of the first display unit 6a to a medium length. The information provision unit 13 sets the flashing cycle of the first display unit 6a to a 1-second cycle. The information provision unit 13 also sets the color of the illuminated portion of the first display unit 6a to yellow. In addition to the display unit 6, the information provision unit 13 may also provide information to the driver (warning that autonomous driving is ending) through voice output from the speaker 7 and vibration of the steering wheel, etc., from the vibration unit 8.

[0073] The information provision unit 13 provides the driver with a "hands-on request" when the driver's level of involvement is "low". The information provision unit 13 sets the bar-shaped illuminated portion of the first display unit 6a to the shortest display period. The information provision unit 13 sets the flashing cycle of the first display unit 6a to 0.5 seconds. The information provision unit 13 also sets the color of the illuminated portion of the first display unit 6a to red. Note that the illumination pattern shown in Figure 4 may also be adopted in the second display unit 6b or in other display devices. In addition to the display unit 6, the information provision unit 13 may also provide information to the driver through audio output from the speaker 7 and vibration of the steering wheel, etc., from the vibration unit 8.

[0074] Furthermore, if a "hands-on request" is issued, an automatic driving (driving control) termination signal has already been sent to the vehicle control unit 20, so even if the driver tries to increase their level of driving involvement, the automatic driving will end. If the information provision unit 13 continuously changes the length of the bar-shaped illuminated portion, it may match the timing when the length of the illuminated portion becomes zero with the timing when the automatic driving ends. In this case, the shorter the illuminated portion, the more time remains until the automatic driving ends.

[0075] The information provision unit 13 may provide information for driver assistance when the system switches to driver assistance due to the termination of autonomous driving (when the system transitions from autonomous driving level 3 to autonomous driving level 2). If the driver's level of involvement in driving continues to decrease even after switching to driver assistance, the information provision unit 13 will provide the driver with information to warn of the termination of driver assistance when the level of involvement falls below the driver assistance termination warning threshold. When the level of involvement in driving falls below the driver assistance termination determination threshold Th2, the information provision unit 13 transmits a driver assistance (driving control) termination signal to the vehicle control unit 20 and provides information regarding the request for manual driving due to the termination of driver assistance.

[0076] The information provision unit 13 may revert the length of the bar-shaped illuminated portion to the longest display when switching from autonomous driving to driver assistance. The information provision unit 13 may also change the color of the illuminated portion when switching to driver assistance compared to autonomous driving. The information provision unit 13 may set the color of the illuminated portion to green when the degree of driver involvement is "high", orange when the degree of driver involvement is "medium", and pink when the degree of driver involvement is "low".

[0077] [Method for providing evaluation information for a driver evaluation information provider] Next, the method for providing evaluation information of the driver evaluation information provision device 100 in this embodiment will be described with reference to Figures 5 and 6. Figure 5 is a flowchart showing an example of the evaluation information provision method of the driver evaluation information provision device. The driver evaluation information provision device 100 executes the evaluation information provision method when the driver has turned on the driving involvement information provision function. The driver evaluation information provision device 100 may also execute the evaluation information provision method when autonomous driving or driving assistance is started.

[0078] As shown in Figure 5, the driver evaluation ECU 10 of the driver evaluation information providing device 100 monitors the status of the vehicle's driver using the driver monitoring unit 11 as S10. The driver monitoring unit 11 monitors the status of the driver, such as the direction of the driver's face, based on the image of the vehicle's driver captured by the vehicle's driver monitor camera 1.

[0079] In S11, the driver evaluation ECU 10 evaluates the driver's level of involvement in driving using the driving involvement evaluation unit 12. The driving involvement evaluation unit 12 evaluates the driver's level of involvement in driving based on the results of monitoring the driver's state by the driver monitoring unit 11.

[0080] In S12, the driver evaluation ECU 10 provides information to the driver via the information provision unit 13. The information provision unit 13 provides information regarding the driver's level of involvement as evaluated by the driving involvement evaluation unit 12. The information provision unit 13 performs the information provision using the first display unit 6a of the display unit 6, etc. The information provision unit 13 may also perform actions such as outputting sound or voice from the speaker 7 and vibrating the steering wheel from the vibration unit 8. After that, the driver evaluation ECU 10 returns to S10 and repeats the process.

[0081] Figure 6 is a flowchart showing an example of a service level switching method for information provision. The service level switching method shown in Figure 6 is performed, for example, as part of the information provision process in S12 shown in Figure 5. Here, the service level switching method is illustrated assuming that the vehicle is performing autonomous driving.

[0082] As shown in Figure 6, the driver evaluation ECU 10 of the driver evaluation information providing device 100 determines in S20 whether the degree of operational involvement is equal to or greater than the termination warning threshold Th1, based on the information providing unit 13. If the driver evaluation ECU 10 determines that the degree of operational involvement is equal to or greater than the termination warning threshold Th1 (S20: YES), it proceeds to S21. If the driver evaluation ECU 10 does not determine that the degree of operational involvement is equal to or greater than the termination warning threshold Th1 (S20: NO), it proceeds to S22.

[0083] In S21, the driver evaluation ECU 10 provides information to the driver corresponding to the "hands-off" service level via the information provision unit 13. The information provision unit 13 provides information on the "hands-off" service level related to the degree of driver involvement via the display unit 6.

[0084] In S22, the driver evaluation ECU 10, as part of S22, determines whether the degree of driver involvement is equal to or greater than the termination threshold Th2, using the information provision unit 13. If the driver evaluation ECU 10 determines that the degree of driver involvement is equal to or greater than the termination threshold Th2 (S22: YES), it proceeds to S23. If the driver evaluation ECU 10 does not determine that the degree of driver involvement is equal to or greater than the termination threshold Th2 (S22: NO), it proceeds to S24.

[0085] In S23, the driver evaluation ECU 10 provides information to the driver corresponding to the service level "Hands-on approaching" via the information provision unit 13. The information provision unit 13 provides information regarding the service level "Hands-on approaching" related to the degree of driver involvement via the display unit 6.

[0086] In S24, the driver evaluation ECU 10 provides information to the driver corresponding to the service level "hands-on request" via the information provision unit 13. The information provision unit 13 provides information on the service level "hands-on request" related to the degree of driver involvement via the display unit 6.

[0087] According to the driver evaluation information providing device 100 of this embodiment described above, the device can provide the vehicle driver with information on the degree of driver involvement in driving as evaluated by the device. Therefore, even if a warning is issued to the driver or autonomous driving or driver assistance is terminated according to the degree of driver involvement, it is possible to suppress the driver from feeling any discomfort.

[0088] Furthermore, the driver evaluation information provider 100 informs the driver that their level of driving involvement is approaching the termination threshold, thus preventing the driver from feeling uncomfortable with the termination of autonomous driving or driver assistance due to a decrease in the driver's level of driving involvement as evaluated by the device. Similarly, the driver evaluation information provider 100 informs the driver that their level of driving involvement as evaluated by the device is moving away from the termination threshold, thus motivating drivers who wish to continue autonomous driving or driver assistance to increase their level of driving involvement.

[0089] Furthermore, the driver evaluation information providing device 100 continuously provides information to the driver in real time via the display unit 6 while driving control is being performed, so the driver can check the degree of driving involvement evaluated by the device at any time during autonomous driving or driving assistance. In addition, the driver evaluation information providing device 100 highlights the ends of the bar-shaped illuminated portion of the display unit 6 whose length changes according to the change in the degree of driving involvement, making it easier for the driver to notice the change in the degree of driving involvement. Moreover, by differently highlighting the ends when the length of the illuminated portion shortens and when the length of the illuminated portion lengthens, the driver can easily understand whether the degree of driving involvement is decreasing or increasing.

[0090] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above. The present invention can be implemented in various forms, starting with the embodiments described above, by making various changes and improvements based on the knowledge of those skilled in the art.

[0091] The driver evaluation information providing device 100 only needs to be equipped with at least a driver monitor camera 1 as equipment for driver monitoring, and the external sensor 2, steering sensor 3, accelerator pedal sensor 4, and brake pedal sensor 5 are not required. Similarly, the driver evaluation information providing device 100 only needs to be equipped with at least a display unit 6 as equipment for providing information, and the speaker 7 and vibration unit 8 are not required.

[0092] The driver monitoring unit 11 does not necessarily need to determine whether the driver is distracted, looking away, or lacking concentration. The driving involvement evaluation unit 12 may directly evaluate the degree of driving involvement based on the monitoring results from the driver monitoring unit 11, such as the direction of the driver's face and the state of their eyelids.

[0093] The driver monitoring unit 11 may acquire the driver's brain waves, heart rate, blood pressure, and other biometric information from a wearable device worn by the driver. The driving involvement evaluation unit 12 may evaluate the driver's level of driving involvement based on the driver's biometric information.

[0094] The driver evaluation ECU 10 may provide information to the driver by communicating with the driver's wearable device. Wearable devices include smartwatches (including wristband-type devices that do not have a clock function), smart glasses, etc. The driver evaluation ECU 10 may provide information to the driver through vibration, sound output, or flashing lights in the case of a smartwatch, or through projection of images into the driver's field of view or sound output in the case of smart glasses. [Explanation of symbols]

[0095] 1...Driver monitor camera, 2...External sensor, 3...Steering sensor, 4...Accelerator pedal sensor, 5...Brake pedal sensor, 6...Display unit, 6a...First display unit, 6b...Second display unit, 6c...Center display, 7...Speaker, 8...Vibration unit, 10...Driver evaluation ECU, 11...Driver monitoring unit, 12...Driving involvement evaluation unit, 13...Information provision unit, 20...Vehicle control unit, 100...Driver evaluation information provision device, L...Illuminated part, La...End, Th1...Termination warning threshold, Th2...Termination judgment threshold.

Claims

1. A driver monitoring unit monitors the status of the driver based at least on an image of the vehicle's driver captured by a vehicle driver monitor camera, A driver involvement evaluation unit evaluates the degree of the driver's involvement in driving based on the driver status monitoring results by the driver monitoring unit, An information providing unit that provides the driver with information on the degree of driving involvement evaluated by the driving involvement evaluation unit, Equipped with, The driver evaluation information providing device is characterized in that the information providing unit provides information to the driver regarding the degree of involvement in driving based on the length of the bar-shaped illuminated portion of a display unit provided in front of the driver's seat of the vehicle.

2. The driver evaluation information providing device according to claim 1, wherein the information providing unit provides information to the driver that when driving control as an automated driving or driving assistance is being performed in the vehicle and the degree of driving involvement is decreasing, the degree of driving involvement is approaching a termination determination threshold that terminates the driving control when the degree of driving involvement falls below a certain level.

3. The driver evaluation information providing device according to claim 1, wherein the information providing unit provides information to the driver that when driving control as an automated driving or driving assistance is being performed in the vehicle and the degree of driving involvement is increasing, the degree of driving involvement is moving away from a termination determination threshold that terminates the driving control when the degree of driving involvement falls below a certain level.

4. The driver evaluation information providing device according to claim 2 or 3, wherein the information providing unit continuously provides the driver with information on the degree of involvement in driving based on the length of the bar-shaped illuminated portion while the driving control is being performed.

5. The information providing unit highlights the end of the illuminated portion of the bar shape whose length changes according to the change in the degree of involvement in the operation. The driver evaluation information providing device according to claim 4, wherein the highlighting of the end portion differs when the length of the illuminated portion becomes shorter due to a decrease in the degree of involvement in the driving, and when the highlighting of the end portion becomes longer due to an increase in the degree of involvement in the driving.

6. A method for providing evaluation information to a driver evaluation information provision device that provides information to the driver of a vehicle, In the driver evaluation information providing device, the driver's state is monitored at least based on the image of the vehicle's driver captured by the vehicle's driver monitor camera. In the driver evaluation information providing device, the degree of the driver's involvement in operation is evaluated based on the monitoring results of the driver's state, A method for providing evaluation information, characterized in that the driver evaluation information providing device provides information to the driver regarding the degree of involvement in driving based on the length of a bar-shaped illuminated portion of a display unit provided in front of the driver's seat of the vehicle.

Images