Driver evaluation information providing device and evaluation information providing method

Abstract

The present invention relates to a driver evaluation information provision device and an evaluation information provision method, the driver evaluation information provision device comprising: a driver monitoring unit configured to monitor a state of a driver on the basis of an image of the driver of a vehicle captured by a driver monitoring camera of the vehicle; a driving participation degree evaluation unit configured to evaluate a driving participation degree of the driver on the basis of a result of monitoring the state of the driver by the driver monitoring unit; and an information provision unit configured to provide the driver with information relating to the driving participation degree evaluated by the driving participation degree evaluation unit.

Description

Technical Field

[0001] The present disclosure relates to a driver evaluation information providing device and a driver evaluation information providing method. Background Art

[0002] As a technical document related to a driver evaluation information providing device, International Publication No. 2020-188634 is known. This publication shows a driver monitoring device that outputs a warning according to the state of the driver of a vehicle.

[0003] It should be noted that, when there is a large difference between the evaluation of the driver's condition by the vehicle device and the driver's evaluation of his own condition, the driver may feel uncomfortable with the warning or the like issued by the device. Summary of the invention

[0004] The driver evaluation information providing device involved in the first embodiment of the present disclosure includes: a driver monitoring unit, configured to monitor the state of the above-mentioned driver of the above-mentioned vehicle based on the image of the driver captured by the vehicle's driver monitoring camera; a driving participation evaluation unit, configured to evaluate the driving participation of the above-mentioned driver based on the monitoring result of the above-mentioned state of the above-mentioned driver by the above-mentioned driver monitoring unit; and an information providing unit, configured to provide the above-mentioned driver with information related to the above-mentioned driving participation evaluated by the above-mentioned driving participation evaluation unit.

[0005] According to the driver evaluation information providing device involved in the first embodiment of the present disclosure, since the driver's driving participation level evaluated by the driver information providing device of the vehicle can be used, even if a warning is output to the driver, automatic driving or driving assistance is terminated based on the driving participation level, the driver's discomfort can be suppressed.

[0006] In the driver evaluation information providing device involved in the first embodiment of the present disclosure, the information providing unit may be configured to provide the driver with information indicating that the driving participation is approaching the end determination threshold when the vehicle is executing driving control as automatic driving or driving assistance. When the driving participation is lower than the end determination threshold, the driving control may be ended. According to the driver evaluation information providing device, since the information indicating that the driving participation is approaching the end determination threshold is provided to the driver, the driver can be restrained from feeling uncomfortable about the end of driving control as automatic driving or driving assistance due to the decrease in the driver's driving participation evaluated by the device.

[0007] In the driver evaluation information providing device involved in the first aspect of the present disclosure, the information providing unit may be configured to provide the driver with information indicating that the driving participation is moving away from the termination determination threshold when the vehicle is executing the driving control. According to the driver evaluation information providing device, since the information that the driving participation evaluated by the device is moving away from the termination determination threshold is provided to the driver, the driver who wishes to continue the driving control as an automatic driving or driving assistance can be motivated to increase the driving participation.

[0008] In the driver evaluation information providing device according to the first aspect of the present disclosure, the information providing unit may be configured to continuously provide the driver with information related to the driving participation by changing the length of the bar-shaped illuminated portion of the display unit provided in front of the driver's seat of the vehicle during the execution of the driving control. According to the driver evaluation information providing device, since the information is continuously provided to the driver through the display unit during the execution of the driving control, the driver can confirm the driving participation evaluated by the device at any time during the automatic driving or driving assistance.

[0009] In the driver evaluation information providing device according to the first aspect of the present disclosure, the information providing unit may be configured to emphasize the end of the bar-shaped lit portion of the display unit. The end may be configured to change in length according to the change in the driving engagement. The information providing unit may be configured to make the first emphasized display different from the second emphasized display. The first emphasized display may be an emphasized display of the end when the length of the bar-shaped lit portion becomes shorter due to the decrease in the driving engagement. The second emphasized display may be an emphasized display of the end when the length of the bar-shaped lit portion becomes longer due to the increase in the driving engagement. According to the driver evaluation information providing device, by emphasizing the end of the bar-shaped lit portion of the display unit whose length changes according to the change in the driving engagement, the driver can easily notice the change in the driving engagement, and by making the emphasized display of the end when the length of the lit portion becomes shorter different from the emphasized display of the end when the length of the lit portion becomes longer, the driver can easily understand whether the driving engagement is decreasing or increasing.

[0010] The second aspect of the present disclosure is an evaluation information providing method of a driver evaluation information providing device configured to provide information to a driver of a vehicle. The evaluation information providing method includes: monitoring the state of the driver based on an image of the driver captured by a driver monitoring camera of the vehicle by the driver evaluation information providing device; evaluating the driving participation of the driver based on the monitoring result of the state of the driver by the driver evaluation information providing device; and providing information related to the driving participation to the driver by the driver evaluation information providing device.

[0011] According to the evaluation information providing method of the driver evaluation information providing device involved in the second embodiment of the present disclosure, since the driving participation of the driver can be evaluated by the driver information providing device of the vehicle, even if a warning is output to the driver, automatic driving or driving assistance is terminated based on the driving participation, the driver's discomfort can be suppressed.

[0012] The third mode of the present disclosure is a driver evaluation information providing device. The driver evaluation information providing device includes an electronic control device. The electronic control device is configured to monitor the state of the driver of the vehicle based on an image of the driver captured by a driver monitoring camera of the vehicle, and is configured to evaluate the driving participation of the driver based on the monitoring result of the state of the driver, and is configured to provide the driver with information related to the driving participation.

[0013] According to each aspect of the present disclosure, the evaluation result of the driver's driving engagement level recognized by the driver information providing device of the vehicle can suppress the driver from feeling uncomfortable with the output or the like according to the driving engagement level. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Hereinafter, features, advantages, technical and industrial significance of exemplary embodiments of the present invention will be described with reference to the accompanying drawings, in which like reference numerals represent like components, wherein:

[0015] Figure 1 This is a block diagram showing a driver evaluation information providing device according to one embodiment.

[0016] Figure 2 This is a diagram showing an example of a display unit provided in front of a driver's seat.

[0017] Figure 3 This is a graph showing an example of changes in the content of information provision according to the degree of driving participation.

[0018] Figure 4This is a table showing an example of the information provision content and the display method of the display unit according to the driving participation level.

[0019] Figure 5 This is a flowchart showing an example of an evaluation information providing method of the driver evaluation information providing device.

[0020] Figure 6 This is a flowchart showing an example of a method for switching the service level to be provided with information. DETAILED DESCRIPTION

[0021] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0022] Figure 1 1 is a block diagram showing a driver evaluation information providing device 100 according to an embodiment. Figure 1 The driver evaluation information providing device 100 shown is mounted on a vehicle and provides information on the driving engagement level evaluated by the device to the driver of the vehicle. The driving engagement level refers to the degree of participation or concentration of the driver in driving.

[0023] The vehicle is, for example, a vehicle capable of performing either automatic driving or driving assistance. Automatic driving is a driving control that allows the vehicle to drive automatically without a driver's driving operation. In the driving control as automatic driving, both the vehicle speed and steering control are controlled without a driver's driving operation. Automatic driving is, for example, equivalent to control above automatic driving level 3 recognized by the Society of Automotive Engineers (SAE) of the United States.

[0024] Driving assistance is vehicle control that assists the driver's driving operation in a manner that the vehicle drives properly. The driving assistance involved in this embodiment includes vehicle driving control such as ACC [Adaptive Cruise Control], LKA [Lane Keeping Assist], and parking assistance. In driving control as driving assistance, during the period when the driver does not input driving operations, at least one of the vehicle's speed and steering operation is controlled. Driving assistance is equivalent to, for example, autonomous driving levels 1 and 2 recognized by the Society of Automotive Engineers (SAE) of the United States. Among them, the vehicle does not necessarily need to be able to perform autonomous driving or driving assistance.

[0025] Configuration of driver evaluation information providing device

[0026] The following describes the structure of the driver evaluation information providing device 100 according to the present embodiment. Figure 1As shown, the driver evaluation information providing device 100 includes a driver evaluation ECU 10 (Electronic Control Unit) that manages the device in a centralized manner.

[0027] The driver evaluation ECU 10 is an electronic control unit having a CPU (Central Processing Unit) and a storage unit such as a ROM (Read Only Memory) or a RAM (Random Access Memory). In the driver evaluation ECU 10, various functions are realized by, for example, the CPU executing a program stored in the storage unit. The driver evaluation ECU 10 may be composed of a plurality of electronic units.

[0028] The driver evaluation ECU 10 is connected to the driver monitoring camera 1 , the exterior sensor 2 , the steering sensor 3 , the accelerator pedal sensor 4 , the brake pedal sensor 5 , the display unit 6 , the speaker 7 , the vibration unit 8 , and the vehicle control unit 20 .

[0029] The driver monitoring camera 1 is a camera for photographing the driver. The driver monitoring camera 1 is installed, for example, on the cover of the steering column of the vehicle in front of the driver's seat. A plurality of driver monitoring cameras 1 may be installed to photograph the driver from multiple directions. The driver monitoring camera 1 sends the image of the driver photographed to the driver evaluation ECU 10.

[0030] The external sensor 2 is a detection device for detecting the surrounding environment of the vehicle. The external sensor 2 includes an external camera and a radar sensor. The external camera includes, for example, a front camera that is arranged on the back side of the front windshield of the vehicle and photographs the front of the vehicle. The external camera may include cameras arranged on the back and sides of the vehicle. The external camera sends the photographing information of the outside of the vehicle to the driver evaluation ECU 10.

[0031] A radar sensor is a detection device that uses radio waves (e.g., millimeter waves) or light to detect objects around the vehicle. Radar sensors include, for example, millimeter wave radars or laser radars [LIDAR]. Radar sensors detect obstacles by sending radio waves or light around the vehicle and receiving radio waves or light reflected by objects. The radar sensor sends the detected object information to the driver evaluation ECU 10.

[0032] The steering sensor 3 is a sensor for detecting the driver's grip on the steering wheel of the vehicle. The steering sensor 3 may be configured to include a pressure-sensitive sensor provided on the steering wheel, or may be configured to include a touch sensor for detecting the current of the human body. The steering sensor 3 may be capable of detecting the driver's operation of the steering wheel. The steering sensor 3 sends the detection results such as the driver's grip on the steering wheel to the driver evaluation ECU 10. In addition, the steering wheel is not limited to a wheel-shaped structure, as long as it can function as a handle.

[0033] The accelerator pedal sensor 4 detects the driver's stepping on the accelerator pedal of the vehicle. The accelerator pedal sensor 4 is configured to detect the state where the driver places his foot on the accelerator pedal. The accelerator pedal sensor 4 transmits the detection result of the driver's stepping on the accelerator pedal of the vehicle to the driver evaluation ECU 10.

[0034] The brake pedal sensor 5 detects the driver's stepping on the brake pedal. The brake pedal sensor 5 is configured to detect the driver's foot resting on the brake pedal. The brake pedal sensor 5 sends the detection result of the driver's stepping on the brake pedal of the vehicle to the driver evaluation ECU 10.

[0035] The display unit 6 is a device used to provide visual information to the driver. The display unit 6 is provided, for example, in front of the driver's seat. Figure 2 This is a diagram showing an example of a display unit provided in front of a driver's seat. Figure 2 The first display unit 6a, the second display unit 6b and the center display unit 6c are shown in FIG. Figure 2 As shown, the display unit 6 is configured to include, for example, a first display unit 6a, a second display unit 6b, and a center display unit 6c.

[0036] The first display unit 6a is a light source provided on the dashboard of the vehicle in the shape of a strip extending to the left and right of the vehicle. The first display unit 6a is provided at a position that enters the driver's field of vision when the driver monitors the front of the vehicle. The first display unit 6a can be composed of, for example, LEDs. As an example, the first display unit 6a is provided from in front of the driver's seat to in front of the co-pilot's seat.

[0037] As an example, the first display unit 6a provides information on driving engagement to the driver through the length of the bar-shaped illuminated portion L. The end of the illuminated portion L whose length changes due to a change in driving engagement is indicated by reference symbol La. In addition, the length of the first display unit 6a is not particularly limited, and may be the length from the front of the driver's seat to the center of the vehicle, or may be shorter than that.

[0038] The second display unit 6b is a light source disposed on the upper portion of the steering wheel 50 of the vehicle. The second display unit 6b can be, for example, a steering illumination lamp composed of an LED. The second display unit 6b can be a rectangular light source extending along the left and right sides of the vehicle. The shape of the second display unit 6b is not particularly limited. The shape of the second display unit 6b can be a square, a rhombus, or an ellipse. The second display unit 6b can be disposed on the cover of the steering column instead of being disposed on the upper portion of the steering wheel 50.

[0039] The center display 6c is a display provided at the center of the dashboard of the vehicle. The center display 6c is used, for example, in route guidance related to the navigation system of the vehicle. The center display 6c does not necessarily need to be located at the center of the dashboard. In addition, the center display 6c does not necessarily need to be used as a display of the navigation system.

[0040] The display unit 6 does not necessarily need to include all of the first display unit 6a, the second display unit 6b, and the center display 6c, and may be a structure that includes only any one of them. The display unit 6 may also include other devices that provide visual information to the driver without including all of the first display unit 6a, the second display unit 6b, and the center display 6c. The display unit 6 may include, for example, a HUD [Head Up Display] that projects image information on the front windshield of the vehicle, or an MID [Multi Information Display] of an instrument. The display unit 6 only needs to include at least one of the above-mentioned display devices.

[0041] The speaker 7 is a device used to provide information to the driver by sound. The speaker 7 is configured to be able to output sound or voice to the driver in the vehicle cabin.

[0042] The vibration unit 8 is a device used to provide information to the driver by vibration. The information provided by vibration is combined with visual information or sound information. The vibration unit 8 includes, for example, a steering actuator that vibrates the steering wheel of the vehicle.

[0043] The vibration unit 8 may include a belt actuator for winding up the seat belt of the driver's seat of the vehicle. The vibration unit 8 may provide information to the driver by gently pulling the driver's seat belt. The vibration unit 8 may include a seat actuator for vibrating the armrest or seat of the driver's seat.

[0044] The vehicle control unit 20 controls the driving of the vehicle by sending control signals to the driving units of the vehicle (engine actuator, brake actuator, steering actuator, etc.). The vehicle control unit 20 performs automatic driving or driving assistance of the vehicle permitted by the driver. Detailed descriptions of automatic driving and driving assistance are omitted. When a driving control end signal is received from the driver evaluation ECU 10 during the execution of automatic driving or driving assistance of the vehicle, the vehicle control unit 20 ends the driving control being executed (driving control as automatic driving or driving assistance).

[0045] The vehicle control unit 20 may also be a mode in which the vehicle shifts to driving assistance driving control when the vehicle receives a driving control termination signal from the driver evaluation ECU 10 during the execution of the vehicle's automatic driving and terminates the automatic driving driving control. That is, the vehicle control unit 20 may be a mode in which the vehicle shifts from automatic driving level 3 or higher to automatic driving level 2 when the vehicle receives a driving control termination signal from the driver evaluation ECU 10.

[0046] Next, the functional structure of the driver evaluation ECU 10 will be described. Figure 1 As shown, the driver evaluation ECU 10 includes a driver monitoring unit 11, a driving participation evaluation unit 12, and an information providing unit 13. An electronic control device is an example of the driver evaluation ECU 10.

[0047] The driver monitoring unit 11 monitors the state of the driver based on the captured image of the driver of the vehicle captured by the driver monitoring camera 1 of the vehicle. The driver monitoring unit 11, for example, recognizes the orientation of the driver's face. When the driver is paying attention to driving or driving the vehicle, it can be considered that the driver's face is facing the front of the vehicle. The driver monitoring unit 11 recognizes the orientation of the driver's face using a well-known image processing technology.

[0048] The driver monitoring unit 11 may also use a machine learning model to identify the orientation of the driver's face. The machine learning model, for example, uses as input an image of the driver captured by the driver monitoring camera 1 to output a recognition result of the orientation of the face. The machine learning model is, for example, a neural network such as a convolutional neural network (CNN). The neural network may include multiple layers including multiple convolutional layers and pooling layers. As the neural network, a deep learning network based on deep learning may also be used. In the future, the machine learning model appearing in this embodiment may also be of the same neural network structure.

[0049] The driver monitoring unit 11 may monitor the state of the driver's eyes. The state of the driver's eyes includes the driver's line of sight. The state of the driver's eyes may include at least one of eyeball saccade, blinking frequency, eye closing speed, eye closing state (position of upper eyelid), etc. The driver monitoring unit 11 identifies the state of the driver's eyes including the line of sight through image processing technology or a machine learning model.

[0050] The driver monitoring unit 11 can identify whether the driver is in a looking state or a squinting state based on the orientation of the driver's face or the state of his eyes. The looking state refers to a state in which the driver is visually checking the scenery outside the vehicle that is not related to the driving of the vehicle. The squinting state refers to a state in which the driver is visually checking the inside of the vehicle that is not related to the driving of the vehicle. The squinting state includes a state in which the driver is talking to other passengers, a state in which the driver is operating a device such as a navigation system of the vehicle, a state in which the driver is operating a smartphone, etc.

[0051] The driver monitoring unit 11 can identify that the driver is in a state of insufficient concentration based on the orientation of the driver's face or the state of the driver's eyes. The state of insufficient concentration includes a state where the driver cannot concentrate on driving due to lack of sleep, or a state where the driver cannot concentrate on driving due to thinking. The driver monitoring unit 11 identifies that the driver is in a state of insufficient concentration based on eye saccades, blinking frequency, eye closing speed, eye closing state (position of upper eyelids), etc.

[0052] The driver monitoring unit 11 can monitor the driver's state based on the detection result of the steering sensor 3. When the driver is paying attention to driving or driving the vehicle, it can be considered that the driver is holding the steering wheel. The driver monitoring unit 11 recognizes that the driver is holding the steering wheel based on the detection result of the steering sensor 3. The driver monitoring unit 11 can distinguish and recognize whether the driver is holding the steering wheel with both hands or with one hand. The driver monitoring unit 11 can recognize the amount of steering control of the steering wheel by the driver.

[0053] The driver monitoring unit 11 can monitor the driver's state based on the detection results of the accelerator pedal sensor 4 or the brake pedal sensor 5. When the driver's foot is placed on the accelerator pedal or the brake pedal, it can be considered that the driver is likely to be paying attention to driving or traveling the vehicle.

[0054] The driver monitoring unit 11 recognizes that the driver's foot is placed on the accelerator pedal or the brake pedal based on the detection result of the accelerator pedal sensor 4 or the brake pedal sensor 5. When the driver steps on the accelerator pedal or the brake pedal, the driver monitoring unit 11 can recognize the driver's pedal stepping amount.

[0055] The driving engagement evaluation unit 12 evaluates the driving engagement of the driver based on the monitoring result of the driver's state by the driver monitoring unit 11. For example, when the driver is in a looking state, the driving engagement evaluation unit 12 evaluates the driving engagement as low compared to when the driver is looking toward the front of the vehicle. In addition, when the driver's line of sight is toward the side mirror, the driving engagement is not evaluated as low.

[0056] When the driver is in a state of looking sideways, the driving participation evaluation unit 12 may evaluate the driving participation as low compared to a case where the driver is looking toward the front of the vehicle. In addition, when the driver's line of sight is toward the rearview mirror inside the vehicle, the driving participation is not evaluated as low. When the driver is in a state of looking sideways and the driver is operating an information device installed in the vehicle for navigation, music playback, telephone, etc., the driving participation evaluation unit 12 may evaluate the driving participation as low compared to a case where the driver is not operating the information device. The driving participation evaluation unit 12 obtains a signal related to the driver's operation from the information device installed in the vehicle.

[0057] When the driver is in a state of insufficient concentration, the driving participation evaluation unit 12 may evaluate the driving participation as low compared to a state in which the driver is not in a state of insufficient concentration. The driving participation evaluation unit 12 may also evaluate the driver's insufficient concentration in stages. When the driver is insufficiently concentrated due to lack of sleep, the driving participation evaluation unit 12 may evaluate the driving participation as low compared to a state in which the driver is insufficiently concentrated due to other reasons. The longer the driver's looking state, squinting state, or insufficient concentration state lasts, the lower the driving participation evaluation unit 12 may evaluate the driving participation.

[0058] When the frequency of the driver's surrounding monitoring action is less than a certain value, the driving engagement evaluation unit 12 may evaluate the driving engagement as low compared to when the frequency of the driver's surrounding monitoring action is greater than a certain value. The surrounding monitoring action may include not only the action of checking the front of the vehicle, but also the action of checking the side mirrors, the interior rearview mirror, the adjacent lanes, the adjacent sidewalks, and the traffic lights in front of the vehicle.

[0059] The driving participation evaluation unit 12 can evaluate the driver's driving participation based on the driving environment outside the vehicle detected by the external sensor 2. The driving participation evaluation unit 12 determines whether the driver is facing the extension direction of the driving lane based on the extension direction of the vehicle's driving lane detected by the external sensor 2 and the orientation of the driver's face recognized by the driver monitoring unit 11. In a case where the driver is paying attention to the driving or driving of the vehicle, when the driving lane is a curve, etc., it can be considered that the orientation of the driver's face changes according to the orientation of the curve. In a case where the driver is not facing the extension direction of the driving lane, the driving participation evaluation unit 12 can evaluate the driving participation as low compared to a case where the driver is facing the extension direction of the driving lane.

[0060] In addition, the more visual confirmation objects detected by the external sensor 2, such as traffic lights, road signs, preceding vehicles, overtaking vehicles, and pedestrians in front of the vehicle, that the driver has not visually confirmed once, the lower the driving participation evaluation unit 12 can evaluate the driving participation. When the number of visual confirmations of the visual confirmation objects per constant time by the driver (the number of visual confirmations of any visual confirmation object) is less than a prescribed value, the driving participation evaluation unit 12 can evaluate the driving participation as lower than when the number of visual confirmations is greater than the prescribed value.

[0061] The driving engagement evaluation unit 12 can evaluate the driver's driving engagement based on the acceleration or deceleration of the vehicle and the state of the driver's head. When the driver is paying attention to the driving or driving of the vehicle, since the driver is prepared for the change in front and rear acceleration by predicting the vehicle's driving conditions (starting, accelerating, decelerating, stopping), it can be considered that the head swing is smaller than that of a driver who is not paying attention to driving. The greater the swing of the driver's head during the acceleration or deceleration of the vehicle, the lower the driving engagement can be evaluated by the driving engagement evaluation unit 12.

[0062] When the driver does not hold the steering wheel, the driving engagement evaluation unit 12 may evaluate the driving engagement as lower than when the driver holds the steering wheel. When the driver does not hold the steering wheel, the longer the time elapsed since the driver last held the steering wheel, the lower the driving engagement evaluation unit 12 may evaluate the driving engagement. In addition, when the driver holds the steering wheel with one hand, the driving engagement evaluation unit 12 may evaluate the driving engagement as lower than when the driver holds the steering wheel with both hands.

[0063] When the driver's foot is not placed on the accelerator pedal or the brake pedal, the driving engagement evaluation unit 12 may evaluate the driving engagement as lower than when the driver's foot is placed on the accelerator pedal or the brake pedal. When the driver's foot is not placed on the accelerator pedal or the brake pedal, the longer the time from when the driver's foot was last placed on the accelerator pedal or the brake pedal, the lower the driving engagement evaluation unit 12 may evaluate the driving engagement as lower.

[0064] During the execution of automatic driving or driving assistance of the vehicle, when the number of times per constant time that the driver overrides the vehicle's driving by steering the steering wheel, stepping on the accelerator pedal, or stepping on the brake pedal is less than a specified value, the driving participation evaluation unit 12 may evaluate the driving participation as low compared to a case where the number of times per constant time that the driver overrides the vehicle's driving is greater than the specified value.

[0065] The driving participation evaluation unit 12 may change the evaluation method of driving participation when the vehicle is executing autonomous driving and when the vehicle is executing driving assistance. For example, in the execution of autonomous driving, the driving participation evaluation unit 12 may reduce the amount of reduction in the evaluation of driving participation caused by the driver being in a looking state, a squinting state, or a state of insufficient concentration, compared to the execution of driving assistance. For example, in the execution of autonomous driving, the driving participation evaluation unit 12 may reduce the amount of reduction in the evaluation of driving participation caused by the driver not holding the steering wheel, compared to the execution of driving assistance. For example, in the execution of autonomous driving, the driving participation evaluation unit 12 may reduce the amount of reduction in the evaluation of driving participation caused by the driver's feet not being placed on the accelerator pedal or the brake pedal, compared to the execution of driving assistance.

[0066] The information providing unit 13 provides the driver with the driving engagement of the driver evaluated by the driving engagement evaluation unit 12. The information providing unit 13 provides information to the driver using, for example, the display unit 6. The information providing unit 13 is not limited to providing information visually, but may also provide information based on sound or voice, or may provide information based on methods such as vibration of the steering wheel.

[0067] For example, in the first display unit 6a of the display unit 6, the information providing unit 13 provides information about the driving participation level to the driver by shortening the lit portion of the bar shape as the driving participation level is lower (refer to Figure 2 The information providing unit 13 may change the length of the illuminated portion extending toward the passenger seat side based on the end of the first display unit 6a on the driver's seat side, or may change the length of the illuminated portion extending to the left and right based on the center position of the first display unit 6a.

[0068] The information providing unit 13 can change the length of the lighted portion at the end of the lighted portion (for example, Figure 2 The ends (La) of the lighted portion are highlighted. When the lighted portion is extended to the left or right, the two ends are highlighted. As an emphasized display, the information providing unit 13 can increase the brightness of the ends, or change the color of the ends. The color of the ends can be changed to a darker or lighter color than other parts, or to different colors. As an emphasized display, the information providing unit 13 can shorten the lighting cycle of the ends compared to other parts, or give animation effects such as fading. In addition, when the strip-shaped lighted portion is displayed on the HUD, various animation effects such as displaying multiple star-shaped images around the ends can be used.

[0069] The information providing unit 13 can make the emphasized display of the end portion when the length of the illuminated portion becomes shorter due to a decrease in driving engagement different from the emphasized display when the length of the illuminated portion becomes longer due to an increase in driving engagement. For example, the information providing unit 13 changes the color of the end portion displayed as emphasized when the length of the illuminated portion becomes shorter and when the length of the illuminated portion becomes longer. The information providing unit 13 can change the flashing cycle and animation effect of the end portion displayed as emphasized. In addition, the information providing unit 13 can change the flashing cycle of the end portion to a shorter time when the speed of change of the length of the illuminated portion (the speed of increase or decrease of driving engagement) is faster, and can also change the color of the end portion to a darker color when the speed of change of the length of the illuminated portion is faster.

[0070] For example, in the second display unit 6b on the steering wheel 50 in the display unit 6, the information providing unit 13 can provide the driver with information on driving engagement by shortening the flashing cycle of the light source as the driving engagement is lower. For example, the information providing unit 13 can provide the driver with information on driving engagement by displaying a numerical value or text corresponding to the driving engagement on the center display 6c in the display unit 6. The information providing unit 13 can also use HUD or MID instead of the first display unit 6a, the second display unit 6b, and the center display 6c.

[0071] When the driver turns on the function of providing information on driving engagement, the information providing unit 13 continuously provides information on driving engagement in real time. In addition, the information providing unit 13 does not necessarily need to provide information continuously, and may also provide information at regular intervals. The regular time is not particularly limited. The regular time may be 1 second, 5 seconds, or 10 seconds.

[0072] When driving control as automatic driving or driving assistance is being performed in the vehicle, the information providing unit 13 may determine whether the driving participation degree becomes less than the termination determination threshold. The termination determination threshold is a threshold for terminating the driving control of automatic driving or driving assistance. The termination determination threshold for automatic driving and the termination determination threshold for driving assistance may be different values. The termination determination threshold for automatic driving may be a value greater than the termination determination threshold for driving assistance.

[0073] When it is determined that the driving participation level has become less than the end determination threshold, the information providing unit 13 requests the driver to start manual driving and sends a driving control end signal to the vehicle control unit 20. The vehicle control unit 20 that receives the driving control end signal ends the driving control as automatic driving or driving assistance. The request to start manual driving when the vehicle is executing automatic driving is equivalent to a so-called hands-on request.

[0074] When driving control as automatic driving or driving assistance is being performed in the vehicle, the information providing unit 13 may provide information to the driver about the relationship between the driving participation and the end determination threshold. When the driving participation decreases, the information providing unit 13 provides information to the driver that the driving participation is approaching the end determination threshold. When the driving participation increases, the information providing unit 13 provides information to the driver that the driving participation is away from the end determination threshold. The information providing unit 13 may provide information to the driver about the relationship between the driving participation and the end determination threshold as the remaining time until the driving control ends.

[0075] Specifically, the information providing unit 13 can Figure 3 as well as Figure 4 Here, the description is based on the premise that the vehicle is in the process of executing the automatic driving. Figure 3 This is a graph showing an example of changes in the content of information provision according to the degree of driving participation.

[0076] exist Figure 3 In the figure, the vertical axis corresponds to driving engagement, and the horizontal axis corresponds to the service level. "Hands-off" is a level that allows the driver to release the steering wheel. "Hands-on as soon as possible" is a level that requests the driver to take action as soon as possible due to the decrease in driving engagement. "Hands-on request" is a level that requests the driver to be prepared for manual driving. Figure 3 2 shows the end notification threshold value Th1 and the end determination threshold value Th2. The end notification threshold value Th1 is a threshold value used to notify the driver that the automatic driving is about to end.

[0077] Figure 4This is a table showing an example of the information provision content and the display method of the display unit according to the driving participation level. Figure 4 The driving engagement level of "high" in Figure 3 The driving participation level of "medium" is equal to the driving participation level of "medium" when it is above the end notice threshold value Th1. Figure 3 The driving participation degree in is less than the termination warning threshold Th1 and is greater than the termination determination threshold Th2. Figure 3 The driving participation level in is less than the end determination threshold Th2.

[0078] like Figure 3 as well as Figure 4 As shown, when the driving participation is "high", the information providing unit 13 provides the driver with "hands-off" information. For example, the information providing unit 13 makes the lit portion of the bar shape in the first display unit 6a the longest display. The length of the lit portion of the bar shape may not be staged but may change continuously. The information providing unit 13 makes the flashing cycle of the first display unit 6a be lit (no flashing) or a 10-second cycle. In addition, the information providing unit 13 makes the color of the lit portion of the first display unit 6a blue.

[0079] When the driving participation level is "medium", the information providing unit 13 provides the driver with information to "take action as soon as possible". The information providing unit 13 displays the lit portion of the bar shape in the first display unit 6a as a medium length. The information providing unit 13 makes the flashing cycle of the first display unit 6a a 1-second cycle. In addition, the information providing unit 13 makes the color of the lit portion of the first display unit 6a yellow. In addition to the display unit 6, the information providing unit 13 can provide the driver with information (warning that the automatic driving is ended) through sound output based on the speaker 7 and vibration of the steering wheel, etc. based on the vibration unit 8.

[0080] When the driving participation is "low", the information providing unit 13 provides the driver with information of "hands-on request". The information providing unit 13 makes the bar-shaped lit part in the first display unit 6a the shortest display. The information providing unit 13 makes the flashing cycle of the first display unit 6a a 0.5 second cycle. In addition, the information providing unit 13 makes the color of the lit part of the first display unit 6a red. Figure 4 The lighting method shown can be adopted in the second display unit 6b or in other display devices. The information providing unit 13 can provide information to the driver through sound output based on the speaker 7 and vibration of the steering wheel or the like based on the vibration unit 8 in addition to the display unit 6.

[0081] In the case of a "hands-on request", since the end signal of the automatic driving (driving control) has been sent to the vehicle control unit 20, the automatic driving ends even if the driver wants to increase the driving participation. When the length of the lit portion in the shape of a bar is continuously changed, the information providing unit 13 can make the timing when the length of the lit portion is zero coincide with the timing of the end of the automatic driving. In this case, the length of the lit portion is equivalent to the remaining time until the end of the automatic driving.

[0082] In the case of switching to driving assistance due to the termination of automatic driving (when moving from automatic driving level 3 to automatic driving level 2), the information providing unit 13 can provide information for driving assistance. When the driver's driving participation continues to decrease after switching to driving assistance, when the driving participation is less than the termination notification threshold for driving assistance, the information providing unit 13 provides the driver with information of the termination of driving assistance. When the driving participation becomes less than the termination determination threshold Th2 for driving assistance, the information providing unit 13 sends a termination signal of driving assistance (driving control) to the vehicle control unit 20, and provides information on the manual driving request related to the termination of driving assistance.

[0083] When switching from automatic driving to driving assistance, the information providing unit 13 can restore the length of the bar-shaped lit portion to the longest display. When switching to driving assistance, the information providing unit 13 can change the color of the lit portion compared to the automatic driving. For the information providing unit 13, when the driving participation degree is "high" in driving assistance, the color of the lit portion can be green, when the driving participation degree is "medium", the color of the lit portion can be orange, and when the driving participation degree is "low", the color of the lit portion can be pink.

[0084] Evaluation information providing method of driver evaluation information providing device

[0085] Next, refer to Figure 5 as well as Figure 6 The evaluation information providing method of the driver evaluation information providing device 100 in this embodiment will be described. Figure 5 The flowchart of an example of an evaluation information providing method of the driver evaluation information providing device is shown. When the driver turns on the driving participation information providing function, the driver evaluation information providing device 100 executes the evaluation information providing method. Alternatively, the driver evaluation information providing device 100 executes the evaluation information providing method at the start of automatic driving or driving assistance.

[0086] like Figure 5As shown, as S10, the driver evaluation ECU 10 of the driver evaluation information providing device 100 monitors the state of the driver of the vehicle through the driver monitoring unit 11. The driver monitoring unit 11 monitors the state of the driver's face direction based on the image of the driver of the vehicle captured by the driver monitoring camera 1 of the vehicle.

[0087] In S11, the driver evaluation ECU 10 evaluates the driving engagement of the driver by the driving engagement evaluation unit 12. The driving engagement evaluation unit 12 evaluates the driving engagement of the driver based on the monitoring result of the driver's state by the driver monitoring unit 11.

[0088] In S12, the driver evaluation ECU 10 provides information to the driver through the information providing unit 13. The information providing unit 13 provides information related to the driving participation of the driver evaluated by the driving participation evaluation unit 12. The information providing unit 13 uses the first display unit 6a of the display unit 6 to perform information provision. The information providing unit 13 may also perform information provision by combining the output of sound or voice based on the speaker 7 and the vibration of the steering wheel based on the vibration unit 8. Then, the driver evaluation ECU 10 returns to S10 and repeats the process.

[0089] Figure 6 is a flowchart showing an example of a method for switching the service level of an information provision object. Figure 5 The information provision process of S12 shown in FIG. Figure 6 Here, the service level switching method is exemplified on the premise that the vehicle is in the process of executing automatic driving.

[0090] like Figure 6 As shown, as S20, the driver evaluation ECU 10 of the driver evaluation information providing device 100 determines whether the driving participation is greater than the end notification threshold Th1 through the information providing unit 13. When it is determined that the driving participation is greater than the end notification threshold Th1 (S20: Yes), the driver evaluation ECU 10 moves to S21. When it is not determined that the driving participation is greater than the end notification threshold Th1 (S20: No), the driver evaluation ECU 10 moves to S22.

[0091] In S21, the driver evaluation ECU 10 provides information corresponding to the service level “hands-off” to the driver through the information providing unit 13. The information providing unit 13 provides information of the service level “hands-off” related to driving participation through the display unit 6.

[0092] In S22, the driver evaluation ECU 10 determines whether the driving participation is greater than or equal to the end determination threshold value Th2 as S22 through the information providing unit 13. When it is determined that the driving participation is greater than or equal to the end determination threshold value Th2 (S22: Yes), the driver evaluation ECU 10 moves to S23. When it is not determined that the driving participation is greater than or equal to the end determination threshold value Th2 (S22: No), the driver evaluation ECU 10 moves to S24.

[0093] In S23 , the driver evaluation ECU 10 provides information corresponding to the service level “get started soon” to the driver through the information providing unit 13 . The information providing unit 13 provides information on the service level “get started soon” related to driving participation through the display unit 6 .

[0094] In S24, the driver evaluation ECU 10 provides information corresponding to the service level "hands-on request" to the driver via the information providing unit 13. The information providing unit 13 provides information of the service level "hands-on request" related to driving participation via the display unit 6.

[0095] According to the driver evaluation information providing device 100 involved in the present embodiment described above, since it is possible to provide information on the driver's driving participation level evaluated by the device to the driver of the vehicle, even if a warning is output to the driver, automatic driving or driving assistance is terminated based on the driving participation level, the driver's discomfort can be suppressed.

[0096] In addition, according to the driver evaluation information providing device 100, since the information that the driving participation is close to the end determination threshold is provided to the driver, the driver can be prevented from feeling uncomfortable about the end of the driving control as automatic driving or driving assistance due to the decrease of the driver's driving participation evaluated by the device. Similarly, according to the driver evaluation information providing device 100, since the information that the driving participation evaluated by the device is far from the end determination threshold is provided to the driver, the driver who wants to continue the driving control as automatic driving or driving assistance can be motivated to increase the driving participation.

[0097] In addition, according to the driver evaluation information providing device 100, since information is continuously provided to the driver in real time through the display unit 6 during the execution of driving control, the driver can confirm the driving participation evaluated by the device at any time during automatic driving or driving assistance. In addition, according to the driver evaluation information providing device 100, by emphasizing the end of the end of the bar-shaped lit portion of the display unit 6 whose length changes according to the change of driving participation, the driver can easily notice the change of driving participation, and by making the emphasized display of the end when the length of the lit portion is shortened different from the emphasized display of the end when the length of the lit portion is lengthened, the driver can easily understand whether the driving participation is decreasing or increasing.

[0098] Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments. The present disclosure can be implemented in various forms obtained by performing various changes and improvements on the basis of the above-described embodiments.

[0099] The driver evaluation information providing device 100 only needs to include at least the driver monitoring camera 1 as a device for driver monitoring, and the external sensor 2, the steering sensor 3, the accelerator pedal sensor 4, and the brake pedal sensor 5 are not essential. Similarly, the driver evaluation information providing device 100 only needs to include at least the display unit 6 as a device for providing information, and the speaker 7 and the vibration unit 8 are not essential.

[0100] The driver monitoring unit 11 does not necessarily need to determine the driver's looking state, sideways state, lack of concentration state, etc. The driving participation evaluation unit 12 can directly evaluate the driving participation based on the monitoring results of the driver monitoring unit 11 on the driver's facial orientation, eyelid state, etc.

[0101] Alternatively, the driver monitoring unit 11 may acquire biological information of the driver, such as brain waves, heart rate, and blood pressure, from a wearable device worn by the driver. The driving engagement evaluation unit 12 may evaluate the driving engagement based on the biological information of the driver.

[0102] The driver evaluation ECU 10 can provide information to the driver by communicating with the driver's wearable device. Wearable devices include smart watches (including wristband devices without clock functions), smart glasses, etc. The driver evaluation ECU 10 can provide information to the driver through vibration, sound output, and light flashing in the smart watch, and can provide information to the driver by projecting images into the driver's field of view and outputting sounds in the smart glasses.

Claims

1. A driver evaluation information providing device, characterized in that: include: a driver monitoring unit configured to monitor a state of the driver of the vehicle based on an image of the driver captured by a driver monitoring camera of the vehicle; a driving participation evaluation unit configured to evaluate the driving participation of the driver based on a monitoring result of the state of the driver by the driver monitoring unit; as well as The information providing unit is configured to provide the driver with information related to the driving engagement level evaluated by the driving engagement level evaluating unit.

2. The driver evaluation information providing device according to claim 1, characterized in that: The information providing unit is configured to provide the driver with information indicating that the driving participation level is approaching an end determination threshold value when the vehicle is executing driving control as automatic driving or driving assistance, When the driving participation degree is lower than the end determination threshold, the travel control ends.

3. The driver evaluation information providing device according to claim 2, characterized in that: The information providing unit is configured to provide the driver with information indicating that the driving participation degree is moving away from the end determination threshold value when the vehicle is executing the travel control.

4. The driver evaluation information providing device according to claim 2 or 3, characterized in that: The information providing unit is configured to continuously provide the information on the driving participation level to the driver by changing the length of a bar-shaped lighting portion of a display unit provided in front of a driving seat of the vehicle during execution of the driving control.

5. The driver evaluation information providing device according to claim 4, characterized in that: The information providing unit is configured to highlight an end portion of the bar-shaped lit portion of the display unit, The end portion is configured such that the length thereof changes according to the change of the driving participation degree. The information providing unit is configured to make the first emphasis display different from the second emphasis display, The first emphasis display is an emphasis display of the end portion when the length of the lit portion in the bar shape becomes shorter due to a decrease in the driving participation level. The second emphasis display is an emphasis display of the end portion when the length of the lit portion configured as the bar shape becomes longer due to an increase in the driving participation level.

6. An evaluation information providing method, comprising: a driver evaluation information providing device configured to provide information to a driver of a vehicle, wherein: include: monitoring the state of the driver based on an image of the driver captured by a driver monitoring camera of the vehicle by the driver evaluation information providing device; evaluating the driving participation of the driver based on the monitoring result of the state of the driver by the driver evaluation information providing device; as well as The driver evaluation information providing device provides the driver with information related to the driving participation level.

7. A driver evaluation information providing device, characterized in that: An electronic control device comprising: monitoring a status of the driver of the vehicle based on an image of the driver captured by a driver monitoring camera of the vehicle; evaluating the driving engagement of the driver based on the monitoring result of the state of the driver; as well as Information related to the driving engagement is provided to the driver.

Images