Driver evaluation information provision device and evaluation information provision method

Abstract

To restrict a driver of a vehicle from feeling discomfort with respect to an output of warning corresponding to a driving involvement degree by providing the driver with information on an evaluation result of the driving involvement degree of the driver recognized by a device.SOLUTION: A driver evaluation information provision device includes: a driver monitoring section for monitoring a state of a driver based on at least a photographed image of the driver of a vehicle photographed by a driver monitor camera of the vehicle; a driving involvement degree evaluation section for evaluating a driving involvement degree of the driver based on the monitoring result of the state of the driver by the driver monitoring section; and an information provision section for providing information on the driving involvement degree evaluated by the driving involvement degree evaluation section to the driver.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a driver evaluation information providing device and an evaluation information providing method. [Background technology]

[0002] Conventionally, as a technical document related to a driver evaluation information providing device, International Publication No. 2020-188634 is known. This publication discloses a driver monitoring device that outputs a warning according to the state of a driver of a vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2020-188634 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when there is a large difference between the evaluation of the driver's state by the vehicle device and the driver's evaluation of his / her own state, the driver may feel uncomfortable with the warning or the like given to the driver by the device. [Means for solving the problem]

[0005] A driver evaluation information providing device according to one embodiment of the present invention comprises a driver monitoring unit that monitors the driver's state based at least on an image of the driver of a vehicle captured by the vehicle's driver monitor camera, a driving involvement evaluation unit that evaluates the driver's degree of driving involvement based on the results of the monitoring of the driver's state by the driver monitoring unit, and an information providing unit that provides the driver with information on the degree of driving involvement evaluated by the driving involvement evaluation unit.

[0006] According to one aspect of the driver evaluation information providing device of the present invention, information can be provided to the vehicle driver regarding the degree of driving involvement of the driver as evaluated by the device, so that even if a warning is output to the driver or automatic driving or driving assistance is terminated depending on the degree of driving involvement, the driver can be prevented from feeling uncomfortable.

[0007] In a driver evaluation information providing device according to one embodiment of the present invention, when driving control is being executed as automatic driving or driving assistance in a vehicle, the information providing unit may provide information to the driver that the degree of driving involvement is approaching a termination determination threshold below which driving control is terminated. This driver evaluation information providing device provides the driver with information that his or her driving involvement level is approaching the termination judgment threshold, thereby preventing the driver from feeling uncomfortable about the termination of automatic driving or driving assistance driving control due to a decrease in the driver's driving involvement level evaluated by the device.

[0008] In a driver evaluation information providing device according to one embodiment of the present invention, when driving control is being executed as automatic driving or driving assistance in a vehicle, the information providing unit may provide information to the driver that the degree of driving involvement is moving away from a termination determination threshold below which driving control is terminated. This driver evaluation information providing device provides the driver with information that the driving involvement level evaluated by the device is moving away from the termination judgment threshold, making it possible to motivate drivers who wish to continue driving control as automatic driving or driving assistance to increase their driving involvement level.

[0009] In a driver evaluation information providing device according to one embodiment of the present invention, the information providing unit may continuously provide the driver with information on the degree of driving involvement based on the length of a bar-shaped lit portion of a display unit provided in front of the driver's seat of the vehicle while driving control is being executed. According to this driver evaluation information providing device, while driving control is being executed, information is continuously provided to the driver via the display unit, so that the driver can check the degree of driving involvement evaluated by the device at any time during automatic driving or driving assistance.

[0010] In a driver evaluation information providing device according to one embodiment of the present invention, the information providing unit may highlight ends of the bar-shaped lit portion of the display unit whose length changes depending on changes in the level of driving involvement, and may differentiate the highlighting of the end when the length of the lit portion shortens due to a decrease in the level of driving involvement from the highlighting of the end when the length of the lit portion increases due to an increase in the level of driving involvement. According to this driver evaluation information providing device, the ends of the bar-shaped lit portion of the display unit whose length changes depending on the change in driving involvement level are highlighted, making it easier for the driver to notice the change in driving involvement level, and by differentiating the highlighting of the ends when the length of the lit portion becomes shorter from the highlighting of the ends when the length of the lit portion becomes longer, the driver can easily understand whether their driving involvement level is decreasing or increasing.

[0011] Another aspect of the present invention is an evaluation information providing method of a driver evaluation information providing device that provides information to a driver of a vehicle, wherein the driver evaluation information providing device monitors the driver's condition based at least on an image of the driver of the vehicle captured by a driver monitor camera of the vehicle, the driver evaluation information providing device evaluates the driver's degree of involvement in driving based on the results of the monitoring of the driver's condition, and the driver evaluation information providing device provides the driver with information on the degree of involvement in driving.

[0012] According to the evaluation information providing method of a driver evaluation information providing device of one embodiment of the present invention, information can be provided to the vehicle driver regarding the degree of driving involvement of the driver evaluated by the device, so that even if a warning is output to the driver or automatic driving or driving assistance is terminated depending on the degree of driving involvement, the driver can be prevented from feeling uncomfortable. Effect of the Invention

[0013] According to each aspect of the present invention, by providing the vehicle driver with information on the evaluation result of the driver's driving involvement recognized by the device, it is possible to prevent the driver from feeling uncomfortable with outputs based on the driving involvement. [Brief description of the drawings]

[0014] [Figure 1] 1 is a block diagram showing a driver evaluation information providing device according to an embodiment; [Diagram 2] 1 is a diagram showing an example of a display unit provided in front of a driver's seat; [Diagram 3] 10 is a graph showing an example of changes in the content of information provided according to the degree of driving involvement. [Figure 4] 11 is a table showing an example of information provision contents and display modes of a display unit corresponding to a driving involvement level. [Diagram 5] 4 is a flowchart showing an example of an evaluation information providing method of the driver evaluation information providing device. [Figure 6] 11 is a flowchart showing an example of a method for switching a service level as an information provision target. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0016] Fig. 1 is a block diagram showing a driver evaluation information providing device 100 according to an embodiment. The driver evaluation information providing device 100 shown in Fig. 1 is mounted on a vehicle and provides information on the degree of involvement in driving evaluated by the device to the driver of the vehicle. The degree of involvement in driving is the degree of the driver's involvement or concentration in driving.

[0017] The vehicle is, for example, a vehicle capable of executing either autonomous driving or driving assistance. Autonomous driving is driving control that allows the vehicle to run automatically without the driver's operation. In autonomous driving driving control, both the vehicle speed and steering are controlled without the driver's operation. Autonomous driving corresponds to, for example, autonomous driving level 3 or higher control certified by the Society of Automotive Engineers (SAE).

[0018] Driving assistance is vehicle control that assists the driver in driving operations so that the vehicle runs appropriately. Driving assistance according to this embodiment includes vehicle driving controls such as ACC [Adaptive Cruise Control], LKA [Lane Keeping Assist], and parking assist. In driving control as driving assistance, at least one of the vehicle speed and steering is controlled while the driver is not inputting driving operations. Driving assistance corresponds to, for example, automated driving levels 1 and 2 certified by the Society of Automotive Engineers (SAE). Note that the vehicle does not necessarily need to be capable of executing automated driving or driving assistance.

[0019] [Configuration of the driver evaluation information providing device] The following describes the configuration of the driver evaluation information providing device 100 according to this embodiment. As shown in Fig. 1, the driver evaluation information providing device 100 includes a driver evaluation ECU 10 (Electronic Control Unit) that performs overall management of the device.

[0020] The driver evaluation ECU 10 is an electronic control unit having a central processing unit (CPU) and a storage unit such as a read only memory (ROM) or a random access memory (RAM). In the driver evaluation ECU 10, various functions are realized by, for example, executing a program stored in the storage unit by the CPU. The driver evaluation ECU 10 may be composed of multiple electronic units.

[0021] The driver evaluation ECU 10 is connected to a driver monitor camera 1 , an external sensor 2 , a steering sensor 3 , an accelerator pedal sensor 4 , a brake pedal sensor 5 , a display unit 6 , a speaker 7 , a vibration unit 8 , and a vehicle control unit 20 .

[0022] The driver monitor camera 1 is a camera that captures an image of the driver. The driver monitor camera 1 is provided, for example, in a position in front of the driver's seat on the cover of the steering column of the vehicle. A plurality of driver monitor cameras 1 may be provided to capture images of the driver from a plurality of directions. The driver monitor camera 1 transmits the captured image of the driver to the driver evaluation ECU 10.

[0023] The external sensor 2 is a detection device that detects the surrounding environment of the vehicle. The external sensor 2 includes an external camera and a radar sensor. The external camera includes, for example, a front camera that is provided behind the windshield of the vehicle and captures an image in front of the vehicle. The external camera may include cameras provided on the back and sides of the vehicle. The external camera transmits image information of the outside of the vehicle to the driver evaluation ECU 10.

[0024] A radar sensor is a detection device that detects objects around a vehicle using radio waves (e.g., millimeter waves) or light. Radar sensors include, for example, millimeter wave radars and LIDAR (Light Detection and Ranging). A radar sensor detects obstacles by transmitting radio waves or light to the surroundings of the vehicle and receiving the radio waves or light reflected by the objects. The radar sensor transmits information about the detected objects to the driver evaluation ECU 10.

[0025] The steering sensor 3 is a sensor that detects the driver's grip on the steering wheel of the vehicle. The steering sensor 3 may be configured to include a pressure sensor provided on the steering wheel, or may be configured to include a touch sensor that detects an electric current in the human body. The steering sensor 3 may be capable of detecting the operation of the steering wheel by the driver. The steering sensor 3 transmits the detection result of the driver's grip on the steering wheel, etc. to the driver evaluation ECU 10. Note that the steering wheel is not limited to a wheel-shaped one, and may be any one that functions as a handle.

[0026] The accelerator pedal sensor 4 detects the depression of the accelerator pedal of the vehicle by the driver. The accelerator pedal sensor 4 is configured to be able to detect a state in which the driver has his / her foot on the accelerator pedal. The accelerator pedal sensor 4 transmits the detection results of the depression of the accelerator pedal of the vehicle by the driver, etc., to the driver evaluation ECU 10.

[0027] The brake pedal sensor 5 detects the depression of the brake pedal by the driver. The brake pedal sensor 5 is configured to be able to detect a state in which the driver has his / her foot on the brake pedal. The brake pedal sensor 5 transmits the detection result of the depression of the brake pedal of the vehicle by the driver to the driver evaluation ECU 10.

[0028] The display unit 6 is a device used to provide visual information to the driver. The display unit 6 is provided, for example, in front of the driver's seat. Here, FIG. 2 is a diagram showing an example of a display unit provided in front of the driver's seat. FIG. 2 shows a first display unit 6a, a second display unit 6b, and a center display 6c. As shown in FIG. 2, the display unit 6 is configured to include, for example, the first display unit 6a, the second display unit 6b, and the center display 6c.

[0029] The first display unit 6a is a bar-shaped light source provided on the instrument panel of the vehicle so as to extend to the left and right of the vehicle. The first display unit 6a is provided at a position within the field of view of the driver while the driver is monitoring the area ahead of the vehicle. The first display unit 6a can be configured, for example, with an LED. As an example, the first display unit 6a is provided from in front of the driver's seat to in front of the passenger seat.

[0030] As an example, the first display unit 6a provides the driver with information on the degree of driving involvement by the length of a bar-shaped lighting portion L. Of the lighting portion L, an end portion whose length changes depending on the change in the degree of driving involvement is indicated by the symbol La. The length of the first display unit 6a is not particularly limited, and may be the length from the front of the driver's seat to the center of the vehicle, or may be shorter than that.

[0031] The second display unit 6b is a light source provided on the upper part of the steering wheel 50 of the vehicle. The second display unit 6b may be, for example, a steering illumination configured with LEDs. The second display unit 6b may be a rectangular light source extending to the left and right of the vehicle. The shape of the second display unit 6b is not particularly limited. The shape of the second display unit 6b may be a square, a rhombus, or an ellipse. The second display unit 6b may be provided on the cover of the steering column instead of on the upper part of the steering wheel 50.

[0032] The center display 6c is a display provided in the center of the vehicle's instrument panel. The center display 6c is used, for example, for route guidance by the vehicle's navigation system. The center display 6c does not necessarily have to be located in the center of the instrument panel. In addition, the center display 6c does not necessarily have to be usable as a display for the navigation system.

[0033] The display unit 6 does not necessarily include all of the first display unit 6a, the second display unit 6b, and the center display 6c, and may include only one of them. The display unit 6 may not include all of the first display unit 6a, the second display unit 6b, and the center display 6c, and may include other devices that provide visual information to the driver. The display unit 6 may include, for example, a HUD [Head Up Display] that projects image information on the windshield of the vehicle, or may include a MID [Multi Information Display] of the meter. The display unit 6 may include at least one of the above-mentioned display devices.

[0034] The speaker 7 is a device used to provide information to the driver by sound, and is configured to be able to output sounds and voices to the driver in the vehicle cabin.

[0035] The vibration unit 8 is a device used to provide information to the driver by vibration. The provision of information by vibration is performed in combination with the provision of visual information or information by sound. The vibration unit 8 includes, for example, a steering actuator that vibrates the steering wheel of the vehicle.

[0036] The vibration unit 8 may include a belt actuator that winds up the seat belt of the driver's seat of the vehicle. The vibration unit 8 may provide information to the driver by gently pulling the driver's seat belt. The vibration unit 8 may include a seat actuator that vibrates the armrest or seat of the driver's seat.

[0037] The vehicle control unit 20 controls the driving of the vehicle by sending control signals to the driving units of the vehicle (engine actuator, brake actuator, steering actuator, etc.). The vehicle control unit 20 executes automatic driving or driving assistance of the vehicle permitted by the driver. A detailed description of automatic driving and driving assistance will be omitted. When the vehicle control unit 20 receives a driving control end signal from the driver evaluation ECU 10 while executing automatic driving or driving assistance of the vehicle, it ends the driving control being executed (driving control as automatic driving or driving assistance).

[0038] The vehicle control unit 20 may be configured to transition to driving control for driving assistance when the vehicle control unit 20 receives a driving control end signal from the driver evaluation ECU 10 and ends the driving control for the autonomous driving during the execution of the autonomous driving of the vehicle. In other words, the vehicle control unit 20 may be configured to transition from autonomous driving level 3 or higher to autonomous driving level 2 when the vehicle control unit 20 receives a driving control end signal from the driver evaluation ECU 10.

[0039] Next, a description will be given of the functional configuration of the driver evaluation ECU 10. As shown in FIG.

[0040] The driver monitoring unit 11 monitors the state of the driver based on an image of the driver of the vehicle captured by the driver monitor camera 1 of the vehicle. The driver monitoring unit 11 recognizes, for example, the direction of the driver's face. When the driver is paying attention to driving or traveling the vehicle, the driver's face is considered to be facing forward of the vehicle. The driver monitoring unit 11 recognizes the direction of the driver's face by a well-known image processing technique.

[0041] The driver monitoring unit 11 may recognize the direction of the driver's face using a machine learning model. The machine learning model, for example, receives an image of the driver captured by the driver monitor camera 1 as an input and outputs a recognition result of the direction of the face. The machine learning model is, for example, a neural network such as a convolutional neural network (CNN). The neural network may include multiple layers including multiple convolution layers and pooling layers. As the neural network, a deep learning network using deep learning may be used. Hereinafter, the machine learning models appearing in this embodiment may also have a similar neural network structure.

[0042] The driver monitoring unit 11 may monitor the eye state of the driver. The eye state of the driver includes the driver's line of sight. The eye state of the driver may include at least one of eye saccade, blink frequency, eye closing speed, eye closing state (upper eyelid position), and the like. The driver monitoring unit 11 recognizes the eye state of the driver including the line of sight by a well-known image processing technique or machine learning model.

[0043] The driver monitoring unit 11 may recognize whether the driver is in an inattentive state or a state of looking away from the driver based on the direction of the driver's face or the state of the driver's eyes. The inattentive state is a state in which the driver is viewing the scenery outside the vehicle that is not related to the driving of the vehicle. The looking away state is a state in which the driver is viewing the inside of the vehicle that is not related to the driving of the vehicle. The looking away state includes a state in which the driver is facing and talking to other passengers, a state in which the driver is operating the navigation system or other equipment of the vehicle, and a state in which the driver is operating a smartphone or the like.

[0044] The driver monitoring unit 11 may recognize that the driver is in a state of insufficient concentration from the direction of the driver's face or the state of the driver's eyes. The state of insufficient concentration includes a state in which the driver cannot concentrate on driving due to lack of sleep, or a state in which the driver cannot concentrate on driving due to thinking about something else. The driver monitoring unit 11 recognizes that the driver is in a state of insufficient concentration from the saccade of the eyeball, the blink frequency, the eye closing speed, the eye closing state (the position of the upper eyelid), etc.

[0045] The driver monitoring unit 11 may monitor the state of the driver based on the detection result of the steering sensor 3. When the driver is paying attention to driving or traveling the vehicle, it is considered that the driver is gripping the steering wheel. The driver monitoring unit 11 recognizes that the driver is gripping the steering wheel from the detection result of the steering sensor 3. The driver monitoring unit 11 may distinguish and recognize whether the driver is gripping the steering wheel with both hands or with one hand. The driver monitoring unit 11 may recognize the steering amount of the steering wheel by the driver.

[0046] The driver monitoring unit 11 may monitor the state of the driver based on the detection result of the accelerator pedal sensor 4 or the brake pedal sensor 5. When the driver's foot is placed on the accelerator pedal or the brake pedal, it can be considered that there is a high possibility that the driver is paying attention to driving or traveling the vehicle.

[0047] The driver monitoring unit 11 recognizes that the driver's foot is placed on the accelerator pedal or the brake pedal based on the detection result of the accelerator pedal sensor 4 or the brake pedal sensor 5. When the driver is depressing the accelerator pedal or the brake pedal, the driver monitoring unit 11 may recognize the amount of pedal depression by the driver.

[0048] The driving involvement evaluation unit 12 evaluates the driver's driving involvement level based on the monitoring result of the driver's state by the driver monitoring unit 11. For example, when the driver is inattentive, the driving involvement evaluation unit 12 evaluates the driving involvement level lower compared to when the driver faces the front of the vehicle. Note that when the driver's line of sight is directed toward the side mirror, the driving involvement level is not evaluated as low.

[0049] When the driver is looking away, the driving involvement evaluation unit 12 may evaluate the driving involvement level lower than when the driver is facing the front of the vehicle. When the driver's line of sight is directed toward the rearview mirror, the driving involvement level is not evaluated as low. When the driver is looking away and is operating an information device installed in the vehicle for vehicle navigation, music playback, telephone, or the like, the driving involvement level evaluation unit 12 may evaluate the driving involvement level lower than when the driver is not operating the information device. The driving involvement evaluation unit 12 acquires a signal related to the driver's operation from the information device installed in the vehicle.

[0050] The driving involvement assessment unit 12 may assess the driving involvement level lower when the driver is in a state of insufficient concentration compared to when the driver is not in a state of insufficient concentration. The driving involvement assessment unit 12 may assess the driver's insufficient concentration in stages. The driving involvement assessment unit 12 may assess the driving involvement level lower when the driver is in a state of insufficient concentration due to lack of sleep compared to when the driver is in a state of insufficient concentration due to other reasons. The driving involvement assessment unit 12 may assess the driving involvement level lower the longer the driver's inattentive state, looking away state, or insufficient concentration state continues.

[0051] When the frequency of the driver's periphery monitoring behavior is less than a certain value, the driving involvement evaluation unit 12 may evaluate the driver's periphery monitoring behavior to be lower than when the frequency of the driver's periphery monitoring behavior is equal to or greater than a certain value. The periphery monitoring behavior may include not only the behavior of checking the area in front of the vehicle, but also the behavior of checking the side mirrors, the rearview mirror, adjacent lanes, adjacent sidewalks, and traffic lights in front of the vehicle.

[0052] The driving involvement evaluation unit 12 may evaluate the driving involvement of the driver in consideration of the driving environment outside the vehicle detected by the external sensor 2. The driving involvement evaluation unit 12 determines whether the driver faces the extending direction of the driving lane based on the extending direction of the driving lane of the vehicle detected by the external sensor 2 and the face direction of the driver recognized by the driver monitoring unit 11. When the driver is paying attention to driving or traveling the vehicle, it is considered that the face direction of the driver changes depending on the direction of the curve, for example, when the driving lane is curved. When the driver is not facing the extending direction of the driving lane, the driving involvement evaluation unit 12 may evaluate the driving involvement lower compared to when the driver faces the extending direction of the driving lane.

[0053] Furthermore, the driving involvement evaluation unit 12 may evaluate the driving involvement level lower as the number of visual targets that the driver has never visually recognized among visual targets detected by the external sensor 2, such as traffic lights, road signs, preceding vehicles, overtaking vehicles, and pedestrians in front of the vehicle, increases. When the number of times the driver visually recognizes visual targets per certain time period (the number of times any visual target is visually recognized) is less than a predetermined value, the driving involvement evaluation unit 12 may evaluate the driving involvement level lower compared to when the number of times the driver visually recognizes the visual targets is equal to or greater than the predetermined value. Well-known techniques can be used to determine the driver's visual recognition.

[0054] The driving involvement evaluation unit 12 may evaluate the driver's driving involvement level based on the acceleration or deceleration of the vehicle and the state of the driver's head. When the driver is focused on driving or running the vehicle, the driver predicts the vehicle's running situation (starting, accelerating, decelerating, stopping) and prepares for changes in forward and backward acceleration, so it is considered that the driver's head shaking is smaller than that of a driver who is not focused on driving. The driving involvement evaluation unit 12 may evaluate the driver's driving involvement level to be lower the greater the shaking of the driver's head when the vehicle accelerates or decelerates.

[0055] The driving involvement assessment unit 12 may assess the driving involvement level lower when the driver is not gripping the steering wheel compared to when the driver is gripping the steering wheel. When the driver is not gripping the steering wheel, the driving involvement assessment unit 12 may assess the driving involvement level lower the longer the elapsed time since the driver last gripped the steering wheel. Furthermore, when the driver is gripping the steering wheel with one hand, the driving involvement assessment unit 12 may assess the driving involvement level lower compared to when the driver is gripping the steering wheel with both hands.

[0056] When the driver's feet are not placed on the accelerator pedal or the brake pedal, the driving involvement assessment unit 12 may assess the driving involvement level lower compared to when the driver's feet are placed on the accelerator pedal or the brake pedal. When the driver's feet are not placed on the accelerator pedal or the brake pedal, the driving involvement assessment unit 12 may assess the driving involvement level lower the longer the elapsed time since the driver's feet were last placed on the accelerator pedal or the brake pedal.

[0057] The driving involvement evaluation unit 12 may evaluate the driving involvement lower when the number of times per certain period of time that the driver intervenes in the vehicle's driving by turning the steering wheel, pressing the accelerator pedal, or pressing the brake pedal is less than a predetermined value while the vehicle is in automatic driving or driving assistance mode, compared to when the number of times per certain period of time that the driver overrides is equal to or greater than the predetermined value.

[0058] The driving involvement evaluation unit 12 may change the method of evaluating the driving involvement between when the vehicle is performing automatic driving and when the vehicle is performing driving assistance. For example, during automatic driving, the driving involvement evaluation unit 12 may reduce the amount of reduction in the evaluation of the driving involvement caused by the driver being inattentive, looking away, or not concentrating, compared to when driving assistance is being performed. For example, during automatic driving, the driving involvement evaluation unit 12 may reduce the amount of reduction in the evaluation of the driving involvement caused by the driver not gripping the steering wheel, compared to when driving assistance is being performed. For example, during automatic driving, the driving involvement evaluation unit 12 may reduce the amount of reduction in the evaluation of the driving involvement caused by the driver's feet not being placed on the accelerator pedal or the brake pedal, compared to when driving assistance is being performed.

[0059] The information providing unit 13 provides the driver with information on the degree of driving involvement of the driver evaluated by the driving involvement evaluation unit 12. The information providing unit 13 provides the driver with information, for example, by using the display unit 6. Note that the information provided to the driver by the information providing unit 13 is not limited to visual information, and may be information provided by sound or voice, or information may be provided by vibration of the steering wheel or other methods.

[0060] The information providing unit 13 provides the driver with information on the degree of driving involvement by, for example, shortening the bar-shaped lit portion in the first display portion 6a of the display unit 6 as the degree of driving involvement decreases (see FIG. 2). The information providing unit 13 may be configured to change the length of the lit portion extending toward the passenger seat side based on the end portion on the driver's seat side of the first display portion 6a, or to change the length of the lit portion extending to the left and right based on the center position of the first display portion 6a.

[0061] The information providing unit 13 may highlight the end of the lighting portion on the side where the length of the lighting portion changes (for example, end La in FIG. 2). When the lighting portion expands and contracts left and right, both ends are highlighted. The information providing unit 13 may increase the brightness of the end as a highlight, or may change the color of the end. The color of the end may be changed to a darker or lighter color than the other parts, or may be changed to a different color. The information providing unit 13 may shorten the lighting cycle of the end as a highlight compared to other parts, or may apply an animation effect such as fading. When a bar-shaped lighting portion is displayed on the HUD, various animation effects can be adopted, such as displaying multiple star-shaped images around the end.

[0062] The information providing unit 13 may differentiate the highlighting of the end when the length of the lit portion is shortened due to a decrease in the driving involvement level from the highlighting when the length of the lit portion is lengthened due to an increase in the driving involvement level. For example, the information providing unit 13 changes the color of the end as a highlighting when the length of the lit portion is shortened and when the length of the lit portion is lengthened. The information providing unit 13 may change the blinking cycle or animation effect of the end as a highlighting. In addition, the information providing unit 13 may change the blinking cycle of the end to a shorter time as the speed of change of the length of the lit portion (the speed of increase / decrease of the driving involvement level) increases, and may change the color of the end to a darker color as the speed of change of the length of the lit portion increases.

[0063] The information providing unit 13 may provide the driver with information on the degree of driving involvement by, for example, shortening the blinking cycle of the light source on the second display unit 6b on the steering wheel 50 of the display unit 6 as the degree of driving involvement becomes lower. The information providing unit 13 may provide the driver with information on the degree of driving involvement by, for example, displaying a numerical value or text corresponding to the degree of driving involvement on the center display 6c of the display unit 6. The information providing unit 13 may use a HUD or a MID instead of the first display unit 6a, the second display unit 6b, and the center display 6c.

[0064] When the driver turns on the information provision function of the degree of involvement in driving, the information provision unit 13 provides information on the degree of involvement in driving continuously in real time. Note that the information provision unit 13 does not necessarily need to provide information continuously, and may provide information at regular intervals. The regular interval is not particularly limited. The regular interval may be 1 second, 5 seconds, or 10 seconds.

[0065] When driving control as automatic driving or driving assistance is being executed in the vehicle, the information providing unit 13 may determine whether the driving involvement degree is less than the end determination threshold. The end determination threshold is a threshold for terminating driving control of automatic driving or driving assistance. The end determination threshold for automatic driving and the end determination threshold for driving assistance may be different values. The end determination threshold for automatic driving may be a value larger than the end determination threshold for driving assistance.

[0066] When the information providing unit 13 determines that the driving involvement level is less than the end determination threshold, it requests the driver to start manual driving and transmits a driving control end signal to the vehicle control unit 20. The vehicle control unit 20, which has received the driving control end signal, ends the driving control as automatic driving or driving assistance. Note that a request to start manual driving when the vehicle is performing automatic driving corresponds to a so-called hands-on request.

[0067] When driving control is being executed as automatic driving or driving assistance in the vehicle, the information providing unit 13 may provide the driver with information on the relationship between the driving involvement level and the end determination threshold. When the driving involvement level is decreasing, the information providing unit 13 provides the driver with information that the driving involvement level is approaching the end determination threshold. When the driving involvement level is increasing, the information providing unit 13 provides the driver with information that the driving involvement level is moving away from the end determination threshold. The information providing unit 13 may provide the driver with information on the relationship between the driving involvement level and the end determination threshold as the remaining time until the end of driving control.

[0068] Specifically, the information providing unit 13 can provide information to the driver in the manner shown in Fig. 3 and Fig. 4. Here, the description is given on the assumption that the vehicle is in an automatic driving mode. Fig. 3 is a graph showing an example of a change in the content of information provided according to the driving involvement degree.

[0069] In Figure 3, the vertical axis corresponds to the driving involvement level, and the horizontal axis corresponds to the service level. "Hands off" is the level at which the driver is allowed to release the steering wheel. "Hands on soon" is the level at which the driver is requested to put their hands on the wheel soon due to a decrease in the driving involvement level. "Hands on required" is the level at which the driver is requested to prepare for manual driving. Figure 3 shows the end warning threshold Th1 and the end determination threshold Th2. End warning threshold Th1 is a threshold used to warn the driver that the end of automated driving is approaching.

[0070] Fig. 4 is a table showing an example of information provision contents corresponding to the degree of driving involvement and the display mode of the display unit. The degree of driving involvement "high" in Fig. 4 corresponds to the case where the degree of driving involvement in Fig. 3 is equal to or greater than the end notice threshold Th1. The degree of driving involvement "medium" corresponds to the case where the degree of driving involvement in Fig. 3 is less than the end notice threshold Th1 and equal to or greater than the end determination threshold Th2. The degree of driving involvement "low" corresponds to the case where the degree of driving involvement in Fig. 3 is less than the end determination threshold Th2.

[0071] As shown in Figs. 3 and 4, when the driving involvement level is "high", the information providing unit 13 provides the driver with "hands off" information. For example, the information providing unit 13 displays the bar-shaped lit portion as the longest in the first display unit 6a. Note that the length of the bar-shaped lit portion may change continuously rather than stepwise. The information providing unit 13 sets the blinking cycle of the first display unit 6a to ON (no blinking) or a 10-second cycle. Furthermore, the information providing unit 13 sets the color of the lit portion of the first display unit 6a to blue.

[0072] When the driving involvement level is "medium", the information providing unit 13 provides the driver with the information "hands on soon". The information providing unit 13 displays the bar-shaped lit portion of the first display unit 6a with a medium length. The information providing unit 13 sets the blinking period of the first display unit 6a to a period of 1 second. The information providing unit 13 also sets the color of the lit portion of the first display unit 6a to yellow. In addition to the display unit 6, the information providing unit 13 may provide information to the driver (a warning that autonomous driving will end) by audio output from the speaker 7 and vibration of the steering wheel or the like from the vibration unit 8.

[0073] When the driving involvement level is "low", the information providing unit 13 provides the driver with information of "hands-on request". The information providing unit 13 displays the bar-shaped lit portion in the first display unit 6a for the shortest period. The information providing unit 13 sets the blinking period of the first display unit 6a to 0.5 seconds. The information providing unit 13 also sets the color of the lit portion of the first display unit 6a to red. The lighting mode shown in FIG. 4 may be adopted in the second display unit 6b, or may be adopted in other display devices. The information providing unit 13 may provide information to the driver by audio output from the speaker 7 and vibration of the steering wheel or the like by the vibration unit 8 in addition to the display unit 6.

[0074] When a "hands-on request" occurs, an end signal for autonomous driving (driving control) has already been sent to the vehicle control unit 20, so autonomous driving will end even if the driver tries to increase the degree of driving involvement. When the information providing unit 13 continuously changes the length of the bar-shaped illuminated portion, it may match the timing when the length of the illuminated portion becomes zero with the timing when autonomous driving ends. In this case, the length of the illuminated portion corresponds to the remaining time until the end of autonomous driving.

[0075] When autonomous driving ends and the system switches to driving assistance (when autonomous driving level 3 is switched to autonomous driving level 2), the information providing unit 13 may provide information for driving assistance. When the driver's driving involvement level continues to decrease even after switching to driving assistance, the information providing unit 13 provides the driver with information notifying the end of driving assistance when the driving involvement level falls below the driving assistance end notice threshold. When the driving involvement level falls below the driving assistance end judgment threshold Th2, the information providing unit 13 transmits an end signal for driving assistance (driving control) to the vehicle control unit 20 and provides information of a manual driving request due to the end of driving assistance.

[0076] When switching from autonomous driving to driving assistance, the information providing unit 13 may restore the length of the bar-shaped illuminated portion to the longest display. When switching to driving assistance, the information providing unit 13 may change the color of the illuminated portion compared to the case of autonomous driving. The information providing unit 13 may display the color of the illuminated portion as green when the driving involvement level is "high" in driving assistance, as orange when the driving involvement level is "medium," and as pink when the driving involvement level is "low."

[0077] [Method of providing evaluation information by the driver evaluation information providing device] Next, an evaluation information providing method of the driver evaluation information providing device 100 in this embodiment will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a flowchart showing an example of the evaluation information providing method of the driver evaluation information providing device. The driver evaluation information providing device 100 executes the evaluation information providing method when the driver turns on the information providing function of the driving involvement degree. The driver evaluation information providing device 100 may be configured to execute the evaluation information providing method at the start of automatic driving or driving assistance.

[0078] 5, the driver evaluation ECU 10 of the driver evaluation information providing device 100 monitors the state of the driver of the vehicle by the driver monitoring unit 11 as S10. The driver monitoring unit 11 monitors the state of the driver, such as the facial direction, based on a captured image of the driver of the vehicle captured by the driver monitor camera 1 of the vehicle.

[0079] In S11, the driver evaluating ECU 10 evaluates the degree of involvement in driving of the driver by the driving involvement evaluation unit 12. The driving involvement evaluation unit 12 evaluates the degree of involvement in driving of the driver based on the monitoring result of the driver state by the driver monitoring unit 11.

[0080] In S12, the driver evaluation ECU 10 provides information to the driver through the information providing unit 13. The information providing unit 13 provides information on the driver's driving involvement evaluated by the driving involvement evaluation unit 12. The information providing unit 13 executes the information provision using the first display unit 6a of the display unit 6, etc. The information providing unit 13 may also execute the information provision by outputting sound or voice from the speaker 7, vibrating the steering wheel by the vibration unit 8, etc. After that, the driver evaluation ECU 10 returns to S10 and repeats the process.

[0081] Fig. 6 is a flowchart showing an example of a method for switching a service level as an information provision target. The method for switching a service level shown in Fig. 6 is executed as a part of the information provision process of S12 shown in Fig. 5, for example. Here, the method for switching a service level is illustrated on the premise that the vehicle is performing autonomous driving.

[0082] 6, the driver evaluation ECU 10 of the driver evaluation information providing device 100 determines, in S20, whether the driving involvement degree is equal to or greater than the end notice threshold Th1 using the information providing unit 13. If the driver evaluation ECU 10 determines that the driving involvement degree is equal to or greater than the end notice threshold Th1 (S20: YES), the process proceeds to S21. If the driver evaluation ECU 10 does not determine that the driving involvement degree is equal to or greater than the end notice threshold Th1 (S20: NO), the process proceeds to S22.

[0083] In S21, the driver evaluation ECU 10 provides information corresponding to the service level “hands off” to the driver through the information providing unit 13. The information providing unit 13 provides information on the service level “hands off” related to the degree of involvement in driving through the display unit 6.

[0084] In S22, the driver evaluation ECU 10 determines whether the driving involvement degree is equal to or greater than the end determination threshold Th2 using the information provision unit 13. If the driver evaluation ECU 10 determines that the driving involvement degree is equal to or greater than the end determination threshold Th2 (S22: YES), the process proceeds to S23. If the driver evaluation ECU 10 does not determine that the driving involvement degree is equal to or greater than the end determination threshold Th2 (S22: NO), the process proceeds to S24.

[0085] In S23, the driver evaluation ECU 10 provides information corresponding to the service level “hands-on soon” to the driver through the information providing unit 13. The information providing unit 13 provides information on the service level “hands-on soon” related to the degree of driving involvement through the display unit 6.

[0086] In S24, the driver evaluation ECU 10 provides information corresponding to the service level “hands-on request” to the driver through the information providing unit 13. The information providing unit 13 provides information on the service level “hands-on request” related to the degree of involvement in driving through the display unit 6.

[0087] According to the driver evaluation information providing device 100 of this embodiment described above, it is possible to provide the vehicle driver with information on the driver's driving involvement level evaluated by the device, so that even if a warning is issued to the driver or automatic driving or driving assistance is terminated depending on the driving involvement level, it is possible to prevent the driver from feeling uncomfortable.

[0088] In addition, the driver evaluation information providing device 100 provides the driver with information that the driving involvement level is approaching the end determination threshold, so that the driver can be prevented from feeling uncomfortable about the end of automatic driving or driving control as driving assistance due to a decrease in the driving involvement level of the driver evaluated by the device. Similarly, the driver evaluation information providing device 100 provides the driver with information that the driving involvement level evaluated by the device is moving away from the end determination threshold, so that it is possible to motivate a driver who wishes to continue automatic driving or driving control as driving assistance to increase his / her driving involvement level.

[0089] Moreover, according to the driver evaluation information providing device 100, while driving control is being executed, information is continuously provided to the driver in real time by the display unit 6, so that the driver can check the driving involvement degree evaluated by the device at any timing during automatic driving or driving assistance. Furthermore, according to the driver evaluation information providing device 100, by highlighting the ends of the bar-shaped lighting part of the display unit 6, whose length changes according to the change in the driving involvement degree, the driver can easily notice the change in the driving involvement degree, and by differentiating the highlighting of the ends when the lighting part length is shortened from the highlighting of the ends when the lighting part length is lengthened, the driver can easily understand whether the driving involvement degree is decreasing or increasing.

[0090] Although the embodiment of the present invention has been described above, the present invention is not limited to the above-mentioned embodiment. The present invention can be embodied in various forms including the above-mentioned embodiment and various modifications and improvements based on the knowledge of those skilled in the art.

[0091] The driver evaluation information providing device 100 needs to be equipped with at least the driver monitor camera 1 as a device for monitoring the driver, and does not necessarily include the external sensor 2, the steering sensor 3, the accelerator pedal sensor 4, and the brake pedal sensor 5. Similarly, the driver evaluation information providing device 100 needs to be equipped with at least the display unit 6 as a device for providing information, and does not necessarily include the speaker 7 and the vibration unit 8.

[0092] The driver monitoring unit 11 does not necessarily need to determine whether the driver is looking aside, away from the road, or not concentrating enough, etc. The driving involvement evaluation unit 12 may directly evaluate the driving involvement level based on the monitoring results of the driver monitoring unit 11 on the direction of the driver's face, the state of the eyelids, etc.

[0093] The driver monitoring unit 11 may acquire the driver's brain waves, heart rate, blood pressure, and other biological information from a wearable device worn by the driver. The driving involvement evaluation unit 12 may evaluate the driving involvement level based on the driver's biological information.

[0094] The driver evaluation ECU 10 may provide information to the driver by communicating with a wearable device of the driver. The wearable device may include a smart watch (including a wristband-type device without a watch function), smart glasses, etc. The driver evaluation ECU 10 may provide information to the driver by vibration, sound output, or flashing light in the smart watch, or may provide information by projecting an image into the driver's field of vision or outputting sound in the smart glasses. [Explanation of symbols]

[0095] 1...driver monitor camera, 2...external sensor, 3...steering sensor, 4...accelerator pedal sensor, 5...brake pedal sensor, 6...display unit, 6a...first display unit, 6b...second display unit, 6c...center display, 7...speaker, 8...vibration unit, 10...driver evaluation ECU, 11...driver monitoring unit, 12...driving involvement evaluation unit, 13...information provision unit, 20...vehicle control unit, 100...driver evaluation information provision device, L...illuminated portion, La...end portion, Th1...end warning threshold, Th2...end judgment threshold.

Claims

1. A driver monitoring unit that monitors a state of the driver based at least on an image of the driver of the vehicle captured by a driver monitor camera of the vehicle; a driving involvement evaluation unit that evaluates a driving involvement level of the driver based on a monitoring result of the state of the driver by the driver monitoring unit; an information providing unit that provides the driver with information on the driving involvement evaluated by the driving involvement evaluation unit; A driver evaluation information providing device comprising:

2. 2. The driver evaluation information providing device according to claim 1, wherein, when driving control as automatic driving or driving assistance is being executed in the vehicle, the information providing unit provides the driver with information that the driving involvement level is approaching a termination determination threshold below which the driving control is terminated.

3. 3. The driver evaluation information providing device according to claim 1, wherein, when driving control as automatic driving or driving assistance is being executed in the vehicle, the information providing unit provides the driver with information that the driving involvement degree is moving away from a termination determination threshold that terminates the driving control when the driving involvement degree falls below the threshold.

4. 4. The driver evaluation information providing device according to claim 2, wherein the information providing unit continuously provides the driver with the degree of driving involvement based on the length of a bar-shaped lit portion of a display unit provided in front of the driver's seat of the vehicle while the driving control is being executed.

5. the information providing unit highlights an end of the bar-shaped illuminated portion of the display unit, the end changing in length in response to a change in the driving involvement level; 5. The driver evaluation information providing device according to claim 4, wherein the highlighting of the end portion when the length of the lit portion is shortened due to a decrease in the degree of driving involvement is different from the highlighting of the end portion when the length of the lit portion is lengthened due to an increase in the degree of driving involvement.

6. 1. A method for providing evaluation information of a driver evaluation information providing device that provides information to a driver of a vehicle, comprising: The driver evaluation information providing device monitors a state of the driver based at least on an image of the driver of the vehicle captured by a driver monitor camera of the vehicle; The driver evaluation information providing device evaluates a degree of involvement in driving of the driver based on a result of monitoring the state of the driver; The driver evaluation information providing device provides the driver with the degree of driving involvement as information.

Images