Vehicle display control device, vehicle display control system, and vehicle display control method

Abstract

To allow a driver to easily recognize whether an automatic driving after a switchover is a hands-on mode or a hands-off mode for the switchover from automatic driving without monitoring duty to automatic driving with monitoring duty.SOLUTION: A vehicle display control device includes: a display control unit 106 that displays images of the surrounding situation of an own vehicle on a display unit 91 used in a cabin of the vehicle; and a mode identification unit 102 that identifies whether the vehicle performs automatic driving in a hands-on mode, which requires grasping a steering wheel, or in a hands-off mode, which does not require grasping the steering wheel, in automatic driving with monitoring duty. When the own vehicle switches from automatic driving without monitoring duty to automatic driving with monitoring duty, the display control unit 106 displays different images of the surrounding situation depending on whether the own vehicle identified its driving as the hands-on mode or the hands-off mode by the mode identification unit 102.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a vehicle display control device, a vehicle display control system, and a vehicle display control method. [Background technology]

[0002] Patent Document 1 discloses a technology for gradually switching a vehicle from a manual driving mode to an autonomous driving mode, and a technology for displaying the automation level when gradually switching from the manual driving mode to the autonomous driving mode using a notification indicator.

[0003] Known automation levels are classified into levels 0 to 5 as defined by the SAE. Level 0 is a level where the driver performs all driving tasks without system intervention. Level 0 corresponds to so-called manual driving. Level 1 is a level where the system assists with either steering or acceleration / deceleration. Level 2 is a level where the system assists with both steering and acceleration / deceleration. Levels 1 and 2 are automated driving where the driver has the responsibility to monitor safe driving (hereinafter simply referred to as the monitoring responsibility). Level 3 is a level where the system can perform all driving tasks in specific locations such as highways, and the driver takes over driving operations in emergencies. Level 4 is a level where the system can perform all driving tasks except under specific circumstances such as incompatible roads or extreme environments. Level 5 is a level where the system can perform all driving tasks in all environments. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-24746 Summary of the Invention [Problem to be solved by the invention]

[0005] In addition to switching from a manual driving mode to an automated driving mode as disclosed in Patent Document 1, switching to a lower level of automated driving within an automated driving mode is also possible. Here, when switching from automated driving at level 3 or higher, which does not require supervision, to automated driving at level 2, which does require supervision, the tasks required of the driver may differ even at the same level of automation. Specifically, because level 2 automated driving does not require the driver to perform any driving operations, it may be in a hands-on mode, in which the driver must hold the steering wheel, or in a hands-off mode, in which the driver must not hold the steering wheel. To address this issue, the configuration that displays the automation level as disclosed in Patent Document 1 does not allow the driver to recognize whether the automated driving mode after the automation level change is a hands-on mode or a hands-off mode.

[0006] One object of this disclosure is to provide a vehicle display control device, a vehicle display control system, and a vehicle display control method that, when switching from autonomous driving without a monitoring obligation to autonomous driving with a monitoring obligation, enable the driver to easily recognize whether the autonomous driving after the switch is in hands-on mode or hands-off mode. [Means for solving the problem]

[0007] The above object is achieved by the combination of features recited in the independent claims, and the subclaims define further advantageous embodiments of the disclosure. The reference numerals in parentheses in the claims correspond to specific means described in the following embodiments as one aspect, and do not limit the technical scope of the present disclosure.

[0008] In order to achieve the above object, 1stThe vehicle display control device includes a display control unit (106, 106a, 106b, 106c) that displays a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in the cabin of the vehicle, and a mode identification unit (102) that identifies whether the vehicle is being performed in a hands-on mode of autonomous driving that requires the driver to hold the steering wheel of the vehicle or a hands-off mode of autonomous driving that does not require the driver to hold the steering wheel, in autonomous driving with monitoring obligation, and the display control unit changes the display of the surrounding situation image depending on whether the mode identification unit has identified the vehicle as being in the hands-on mode of autonomous driving or the hands-off mode of autonomous driving. The surrounding situation image includes an image of a lane, and the display control unit displays the vehicle's own lane, which is the lane the vehicle is traveling in, and surrounding lanes other than the own lane, when the mode specification unit specifies the autonomous driving mode as a hands-on mode, and displays only the own lane among the own lane and surrounding lanes, when the mode specification unit specifies the autonomous driving mode as a hands-off mode. . In order to achieve the above object, a second display control device for a vehicle of the present disclosure includes a display control unit (106, 106a, 106b, 106c) that causes a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, to be displayed on a display (91, 91b) used in a vehicle cabin, and a mode identification unit (102) that identifies whether the vehicle is to be executed in a hands-on mode of autonomous driving that requires the driver to hold the steering wheel of the vehicle or a hands-off mode of autonomous driving that does not require the driver to hold the steering wheel, when the mode identification unit identifies that the vehicle is to be executed in a hands-on mode of autonomous driving that requires the driver to hold the steering wheel. The display of the surrounding situation image differs depending on whether the mode specification unit specifies autonomous driving in hands-on mode or autonomous driving in hands-off mode, and the surrounding situation image is an image of the area around the vehicle viewed from a virtual viewpoint, and when the mode specification unit specifies autonomous driving in hands-on mode, the display control unit displays the surrounding situation image viewed from a virtual viewpoint that is farther away from the display target in the surrounding situation image than when the mode specification unit specifies autonomous driving in hands-off mode, while when the mode specification unit specifies autonomous driving in hands-off mode, the display control unit displays the surrounding situation image viewed from a virtual viewpoint that is closer to the display target than when the mode specification unit specifies autonomous driving in hands-on mode. In order to achieve the above object, a third display control device for a vehicle of the present disclosure includes a display control unit (106, 106a, 106b, 106c) that causes a display (91, 91b) used in a vehicle cabin to display a surrounding situation image, which is an image for showing a surrounding situation of the vehicle, and a mode identification unit (102) that identifies whether the vehicle is to be executed in a hands-on mode autonomous driving that requires the driver to hold the steering wheel of the vehicle or a hands-off mode autonomous driving that does not require the driver to hold the steering wheel, when the mode identification unit determines that the vehicle is in a hands-on mode The display of the surrounding situation image differs depending on whether the mode is identified as autonomous driving in hands-on mode or autonomous driving in hands-off mode, and the surrounding situation image is an image of the area around the vehicle viewed from a virtual viewpoint, and when the mode identification unit identifies autonomous driving in hands-on mode, the display control unit displays the surrounding situation image viewed from a virtual viewpoint looking down from above, more than when the mode identification unit identifies autonomous driving in hands-off mode, while when the mode identification unit identifies autonomous driving in hands-off mode, the display control unit displays the surrounding situation image viewed from a virtual viewpoint looking down from below, more than when the mode identification unit identifies autonomous driving in hands-on mode. In order to achieve the above object, a fourth display control device for a vehicle of the present disclosure includes a display control unit (106, 106a, 106b, 106c) that causes a display (91, 91b) used in a vehicle cabin to display a surrounding situation image, which is an image for showing a surrounding situation of the vehicle, and a mode identification unit (102) that identifies whether the surrounding situation image to be executed in a supervision-obligated autonomous driving mode in which the driver is obligated to supervise the vehicle is a hands-on mode autonomous driving mode in which the driver is obligated to supervise the vehicle, or a hands-off mode autonomous driving mode in which the driver is not obligated to supervise the vehicle, and the display control unit is configured to The display of the surrounding situation image differs depending on whether the mode identification unit has identified autonomous driving in hands-on mode or autonomous driving in hands-off mode, and when the mode identification unit has identified autonomous driving in hands-on mode, the display control unit widens the area around the vehicle to be displayed as the surrounding situation image compared to when the mode identification unit has identified autonomous driving in hands-off mode, while when the mode identification unit has identified autonomous driving in hands-off mode, the display control unit narrows the area around the vehicle to be displayed as the surrounding situation image compared to when the mode identification unit has identified autonomous driving in hands-on mode. In order to achieve the above object, a fifth display control device for a vehicle of the present disclosure includes a display control unit (106, 106a, 106b, 106c) that causes a surrounding situation image, which is an image for showing a surrounding situation of the vehicle, to be displayed on a display (91, 91b) used in a vehicle cabin, and a mode identification unit (102) that identifies whether the autonomous driving to be performed in a hands-on mode that requires the driver to hold the steering wheel of the vehicle or a hands-off mode that does not require the driver to hold the steering wheel, during autonomous driving with monitoring obligation, in which the vehicle is autonomously driven with monitoring obligation, and the display control unit The display of the surrounding situation image differs depending on whether the mode identification unit has identified the vehicle as being in hands-on mode or hands-off mode, and when the vehicle has switched to hands-off mode autonomous driving and the vehicle changes lanes through autonomous driving, or when it is estimated that a vehicle surrounding the vehicle will cut into the lane in which the vehicle is traveling, the display control unit switches to displaying the surrounding situation image when the mode identification unit has identified hands-on mode autonomous driving, even if the hands-off mode autonomous driving continues. In order to achieve the above object, a sixth display control device for a vehicle of the present disclosure includes a display control unit (106, 106a, 106b, 106c) that displays a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in a vehicle cabin, and a mode control unit (106, 106a, 106b, 106c) that specifies whether the autonomous driving with supervision obligation, which is the autonomous driving with the driver's supervision obligation, is to be performed in a hands-on mode that requires the driver to hold the steering wheel of the vehicle or in a hands-off mode that does not require the driver to hold the steering wheel. The display control unit is configured to change the display of the surrounding situation image depending on whether the mode identification unit has identified the vehicle as being in hands-on mode or hands-off mode, and when the vehicle has switched to hands-off mode autonomous driving and the elapsed time since this switch reaches a specified time, the display control unit switches to displaying the surrounding situation image when the mode identification unit has identified hands-on mode autonomous driving, even if the hands-off mode autonomous driving continues. In order to achieve the above object, a seventh display control device for a vehicle of the present disclosure includes a display control unit (106, 106a, 106b, 106c) that causes a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, to be displayed on a display (91, 91b) used in a vehicle cabin, and a mode identification unit (102) that identifies whether the autonomous driving to be performed in a hands-on mode that requires the driver to hold the steering wheel of the vehicle, or a hands-off mode that does not require the driver to hold the steering wheel, is to be performed in an autonomous driving with monitoring obligation, which is an autonomous driving with monitoring obligation of the vehicle, and the display control unit: The display of the surrounding situation image differs depending on whether the mode identification unit identifies the vehicle as being in hands-on mode or hands-off mode, and the vehicle is provided with a grip identification unit (105) that identifies the driver's grip on the steering wheel, and when the grip identification unit identifies the driver's grip on the steering wheel when the vehicle has switched to hands-off mode autonomous driving, the display control unit switches to the display of the surrounding situation image when the mode identification unit identifies hands-on mode autonomous driving, even if hands-off mode autonomous driving continues. In order to achieve the above object, an eighth display control device for a vehicle of the present disclosure includes a display control unit (106, 106a, 106b, 106c) that displays a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in the cabin of the vehicle, and a mode identification unit (102) that identifies whether the vehicle is to be executed in a hands-on mode of autonomous driving that requires the driver to hold the steering wheel of the vehicle or a hands-off mode of autonomous driving that does not require the driver to hold the steering wheel, when the mode identification unit identifies the vehicle as being in the hands-on mode of autonomous driving or the hands-off mode of autonomous driving. The display of the surrounding situation image differs depending on whether the mode identification unit has identified the vehicle as being in automatic driving in hands-off mode, and the display control unit is provided with a grip identification unit (105) that identifies the driver's grip on the steering wheel, and when the grip identification unit identifies the driver's grip on the steering wheel while the vehicle has switched to automatic driving in hands-off mode, the display control unit continues to display the surrounding situation image for when the mode identification unit has identified the vehicle as being in automatic driving in hands-on mode for a predetermined time after the grip identification unit has identified the driver's grip on the steering wheel, and then switches to the display of the surrounding situation image for when the mode identification unit has identified the vehicle as being in automatic driving in hands-on mode, even if automatic driving in hands-off mode continues. In order to achieve the above object, a ninth display control device for a vehicle of the present disclosure includes a display control unit (106, 106a, 106b, 106c) that displays a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in a vehicle cabin, and a mode identification unit (102) that identifies whether the vehicle is being performed in a hands-on mode of autonomous driving that requires the driver to hold the steering wheel of the vehicle or a hands-off mode of autonomous driving that does not require the driver to hold the steering wheel, in autonomous driving with monitoring obligation, and the display control unit changes the display of the surrounding situation image depending on whether the mode identification unit has identified the vehicle as being in the hands-on mode of autonomous driving or the hands-off mode of autonomous driving. The display control unit (106b, 106c) is used in a vehicle that is capable of switching between automated driving without supervision obligation, which is automated driving without the driver's supervision obligation, and automated driving with supervision obligation, as stages of automated driving, and is capable of at least sleep-enabled automated driving, in which the driver is permitted to sleep, and sleep-disabled automated driving, in which the driver is not permitted to sleep, as the automated driving without supervision obligation. The display control unit (106b, 106c) displays driving-related information regarding the driving of the vehicle on the display, and when switching from sleep-enabled automated driving to sleep-disabled automated driving, the display control unit increases the amount of driving-related information displayed on the display when sleep-disabled automated driving is in progress compared to the amount of driving-related information displayed on the display when sleep-enabled automated driving is in progress. In order to achieve the above object, a tenth display control device for a vehicle of the present disclosure includes a display control unit (106, 106a, 106b, 106c) that displays a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in the cabin of the vehicle, and a mode identification unit (102) that identifies whether the vehicle is being executed in a hands-on mode of autonomous driving that requires the driver to hold the steering wheel of the vehicle or a hands-off mode of autonomous driving that does not require the driver to hold the steering wheel, during autonomous driving with monitoring obligation, and the display control unit changes the display of the surrounding situation image depending on whether the mode identification unit has identified the vehicle as being in a hands-on mode of autonomous driving or a hands-off mode of autonomous driving, and the display control unit is configured to display the surrounding situation image differently ... the stage of autonomous driving and the stage of autonomous driving. The present invention is used in a vehicle that is at least capable of switching between automatic driving without monitoring obligation, which is automatic driving without the driver's monitoring obligation, and automatic driving with monitoring obligation, and that is capable of at least sleep-enabled automatic driving, in which the driver is permitted to sleep, and sleep-disallowed automatic driving, in which the driver is not permitted to sleep, as automatic driving without monitoring obligation, and the display control unit (106b, 106c) displays driving-related information regarding the driving of the vehicle on the display, and is equipped with a state identification unit (107) that identifies the state of the driver, and during sleep-enabled automatic driving, the display control unit increases the amount of driving-related information to be displayed on the display when the state identification unit has identified the driver as sleeping, compared to the amount of driving-related information to be displayed on the display when the state identification unit has identified the driver as awake. In order to achieve the above object, an eleventh display control device for a vehicle of the present disclosure includes a display control unit (106, 106a, 106b, 106c) that displays a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in the cabin of the vehicle, and a mode identification unit (102) that identifies whether the vehicle is being performed in a hands-on mode of autonomous driving that requires the driver to hold the steering wheel of the vehicle or a hands-off mode of autonomous driving that does not require the driver to hold the steering wheel, in autonomous driving with monitoring obligation, and the display control unit changes the display of the surrounding situation image depending on whether the mode identification unit has identified the vehicle as being in the hands-on mode of autonomous driving or the hands-off mode of autonomous driving. The display control unit (106b, 106c) displays driving-related information regarding the driving of the vehicle on the display, and when the automation switches from sleep-enabled autonomous driving to driving at a level lower than autonomous driving with monitoring obligation, the display control unit increases the amount of driving-related information displayed on the display after switching to driving at a level lower than autonomous driving with monitoring obligation.

[0009] In order to achieve the above object, 1st The vehicle display control method includes a display control step, executed by at least one processor, of displaying a surrounding situation image, which is an image for showing a surrounding situation of the vehicle, on a display (91, 91b) used in a vehicle cabin; The method includes a mode specification step of specifying whether the vehicle is to be performed in a hands-on mode of autonomous driving that requires the driver to hold the steering wheel of the vehicle or a hands-off mode of autonomous driving that does not require the driver to hold the steering wheel in the supervision-obligated autonomous driving mode, and the display control step of the method varies the display of the surrounding situation image depending on whether the vehicle is to be performed in a hands-on mode of autonomous driving or a hands-off mode of autonomous driving that is specified in the mode specification step. The surrounding situation image includes an image of a lane, and in the display control step, when the mode specification step specifies the autonomous driving in a hands-on mode, the own lane, which is the lane in which the vehicle is traveling, and surrounding lanes other than the own lane are displayed, whereas when the mode specification step specifies the autonomous driving in a hands-off mode, only the own lane among the own lane and surrounding lanes is displayed. . In order to achieve the above object, a second display control method for a vehicle of the present disclosure includes a display control step executed by at least one processor to display a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in a vehicle cabin, and a mode identification step to identify whether the vehicle is performing a hands-on mode of autonomous driving, which requires the driver to hold the steering wheel of the vehicle, or a hands-off mode of autonomous driving, which does not require the driver to hold the steering wheel, in the autonomous driving with monitoring obligation, and in the display control step, the vehicle is performing a hands-on mode of autonomous driving, which is identified as the hands-on mode of autonomous driving in the mode identification step. The display of the surrounding situation image is made different depending on whether it is determined that the mode is autonomous driving in hands-on mode or hands-off mode, and the surrounding situation image is an image of the area around the vehicle viewed from a virtual viewpoint, and in the display control process, if the mode is determined to be autonomous driving in hands-on mode in the mode specification process, the surrounding situation image viewed from a virtual viewpoint that is farther away from the display target in the surrounding situation image is displayed than when the mode is determined to be autonomous driving in hands-off mode in the mode specification process, while if the mode is determined to be autonomous driving in hands-off mode in the mode specification process, the surrounding situation image viewed from a virtual viewpoint that is closer to the display target is displayed than when the mode is determined to be autonomous driving in hands-on mode. In order to achieve the above object, a third display control method for a vehicle of the present disclosure includes a display control step executed by at least one processor to display a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in the cabin of the vehicle, and a mode specification step to specify whether the vehicle will be performing a hands-on mode of autonomous driving in which the driver must hold the steering wheel of the vehicle, or a hands-off mode of autonomous driving in which the driver must not hold the steering wheel, in the mode specification step. The display of the surrounding situation image is different depending on whether it is determined that the vehicle is in automatic driving mode or in automatic driving mode with hands-off operation, and the surrounding situation image is an image of the area around the vehicle viewed from a virtual viewpoint, and in the display control process, if the mode determination process determines that the vehicle is in automatic driving mode with hands-on operation, the surrounding situation image is displayed as viewed from a virtual viewpoint looking down from above, compared to when the mode determination process determines that the vehicle is in automatic driving mode with hands-off operation, whereas if the mode determination process determines that the vehicle is in automatic driving mode with hands-off operation, the surrounding situation image is displayed as viewed from a virtual viewpoint looking down from below, compared to when the mode determination process determines that the vehicle is in automatic driving mode with hands-on operation. In order to achieve the above object, a fourth display control method for a vehicle of the present disclosure includes a display control step executed by at least one processor to display a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in a vehicle cabin, and a mode identification step to identify whether the vehicle is to be executed in a hands-on mode of autonomous driving with a monitoring obligation, which is an autonomous driving with a driver's monitoring obligation, in which the vehicle is to be executed in a hands-off mode of autonomous driving without a driver's monitoring obligation, and in the display control step, the vehicle is The display of the surrounding situation image is made different depending on whether the setting process has identified autonomous driving in hands-on mode or autonomous driving in hands-off mode, and in the display control process, if the mode setting process has identified autonomous driving in hands-on mode, the area around the vehicle to be displayed as the surrounding situation image is made wider than when the mode setting process has identified autonomous driving in hands-off mode, while if the mode setting process has identified autonomous driving in hands-off mode, the area around the vehicle to be displayed as the surrounding situation image is made narrower than when the mode setting process has identified autonomous driving in hands-on mode. In order to achieve the above object, a fifth display control method for a vehicle of the present disclosure includes a display control step executed by at least one processor to display a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in a vehicle cabin, and a mode identification step to identify whether the surrounding situation image to be executed in a supervised autonomous driving with a driver's supervision obligation is a hands-on mode autonomous driving that requires the driver to hold the steering wheel of the vehicle or a hands-off mode autonomous driving that does not require the driver to hold the steering wheel, and in the display control step, Both processes change the display of the surrounding situation image depending on whether the mode identification process identifies autonomous driving in hands-on mode or autonomous driving in hands-off mode, and in the display control process, when the vehicle has switched to autonomous driving in hands-off mode and the vehicle changes lanes through autonomous driving, or when it is estimated that a vehicle surrounding the vehicle will cut into the lane in which the vehicle is traveling, the display is switched to the surrounding situation image that would be displayed if autonomous driving in hands-on mode was identified in the mode identification process, even if autonomous driving in hands-off mode continues. In order to achieve the above object, the sixth display control method for a vehicle of the present disclosure includes a display control step executed by at least one processor to display a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in the vehicle cabin, and a mode identification step to identify whether the vehicle will be operating in a hands-on mode of autonomous driving, which requires the driver to hold the steering wheel of the vehicle, or a hands-off mode of autonomous driving, which does not require the driver to hold the steering wheel, when the vehicle is operating in an autonomous driving with monitoring obligation, in which the driver is required to monitor the vehicle. The display control step changes the display of the surrounding situation image depending on whether the vehicle is operating in a hands-on mode or a hands-off mode of autonomous driving identified in the mode identification step. When the vehicle has switched to autonomous driving in the hands-off mode and the elapsed time since this switch reaches a specified time, the display control step switches to displaying the surrounding situation image for the case in which autonomous driving in the hands-on mode is identified in the mode identification step, even if autonomous driving in the hands-off mode continues. In order to achieve the above object, a seventh display control method for a vehicle of the present disclosure includes a display control step executed by at least one processor, for displaying a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in a vehicle cabin, and a mode identification step for identifying whether the vehicle is to be operated in a hands-on mode of autonomous driving with a monitoring obligation, which is an autonomous driving with a driver's monitoring obligation, in which the driver is required to hold the steering wheel of the vehicle, or in a hands-off mode of autonomous driving that does not require the driver to hold the steering wheel, and in the display control step, The display of the surrounding situation image differs depending on whether the mode identification process identifies the vehicle as being in hands-on mode or hands-off mode, and includes a grip identification process for identifying the driver's grip on the steering wheel.In the display control process, when the vehicle has switched to hands-off mode autonomous driving and the grip identification process identifies the driver's grip on the steering wheel, the display is switched to the surrounding situation image for when the mode identification process identifies hands-on mode autonomous driving, even if hands-off mode autonomous driving continues. In order to achieve the above object, an eighth display control method for a vehicle of the present disclosure includes a display control step executed by at least one processor to display a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in the cabin of the vehicle, and a mode specification step to specify whether the vehicle will be performing a hands-on mode of autonomous driving, which requires the driver to hold the steering wheel of the vehicle, or a hands-off mode of autonomous driving, which does not require the driver to hold the steering wheel, in the monitoring-obligated autonomous driving, and the display control step specifies whether the vehicle will be performing a hands-on mode of autonomous driving, which has been specified in the mode specification step, or a hands-off mode of autonomous driving, which has been specified in the hands-on mode of autonomous driving, or a hands-off mode of autonomous driving, which does not require the driver to hold the steering wheel. The display of the surrounding situation image differs depending on whether it is determined that the vehicle is in automatic driving in hands-off mode, and includes a grip identification process for identifying the driver's grip on the steering wheel. In the display control process, when the driver's grip on the steering wheel is identified in the grip identification process while the vehicle has switched to automatic driving in hands-off mode, the display of the surrounding situation image for when the mode identification process has identified automatic driving in hands-on mode is continued for a predetermined time after the driver's grip on the steering wheel is identified in the grip identification process, and then the display is switched to the surrounding situation image for when the mode identification process has identified automatic driving in hands-on mode, even if automatic driving in hands-off mode continues. In order to achieve the above object, a ninth display control method for a vehicle of the present disclosure includes a display control step executed by at least one processor, for displaying a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in the cabin of the vehicle, and a mode identification step for identifying whether the vehicle is performing a hands-on mode of autonomous driving in which the driver must hold the steering wheel of the vehicle, or a hands-off mode of autonomous driving in which the driver must not hold the steering wheel, in the monitoring-obligated autonomous driving, and the display control step controls the display of the surrounding situation image depending on whether the vehicle is performing the hands-on mode of autonomous driving or the hands-off mode of autonomous driving identified in the mode identification step. The display control process is used in vehicles that are capable of at least switching between automatic driving without monitoring obligation, which is automatic driving where the driver is not required to monitor, and automatic driving with monitoring obligation, and that are capable of at least sleep-enabled automatic driving, in which the driver is permitted to sleep, and sleep-disabled automatic driving, in which the driver is not permitted to sleep, as automatic driving without monitoring obligation.The display control process displays driving-related information regarding the driving of the vehicle on the display, and when switching from sleep-enabled automatic driving to sleep-disabled automatic driving, the display control process increases the amount of driving-related information displayed on the display when sleep-disabled automatic driving is in progress compared to the amount of driving-related information displayed on the display when sleep-enabled automatic driving is in progress. In order to achieve the above object, a tenth display control method for a vehicle of the present disclosure includes a display control step executed by at least one processor, for displaying a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in a vehicle cabin, and a mode identification step for identifying whether the vehicle is performing a hands-on mode of autonomous driving in which the driver must hold the steering wheel of the vehicle, or a hands-off mode of autonomous driving in which the driver must not hold the steering wheel, in the monitoring-obligation autonomous driving, and the display control step varies the display of the surrounding situation image depending on whether the vehicle is performing the hands-on mode of autonomous driving or the hands-off mode of autonomous driving identified in the mode identification step, and This is used in a vehicle that can at least switch between automatic driving without monitoring obligation, which is automatic driving without the driver's monitoring obligation, and automatic driving with monitoring obligation, and is capable of at least sleep-enabled automatic driving, in which the driver is allowed to sleep, and sleep-disallowed automatic driving, in which the driver is not allowed to sleep, as automatic driving without monitoring obligation, and the display control process displays driving-related information regarding the vehicle's driving on a display, and includes a state identification process for identifying the driver's state, and during sleep-enabled automatic driving, the display control process increases the amount of driving-related information displayed on the display when the state identification process identifies the driver as sleeping, compared to the amount of driving-related information displayed on the display when the state identification process identifies the driver as awake. In order to achieve the above object, an eleventh display control method for a vehicle of the present disclosure includes a display control step executed by at least one processor to display a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in the cabin of the vehicle, and a mode identification step to identify whether the vehicle is performing automatic driving with supervision obligation, which is automatic driving with a driver's supervision obligation, in a hands-on mode that requires the driver to hold the steering wheel of the vehicle, or in a hands-off mode that does not require the driver to hold the steering wheel, and in the display control step, the display of the surrounding situation image is controlled depending on whether the vehicle is performing automatic driving with a hands-on mode or a hands-off mode that is identified in the mode identification step. The display control process is used in vehicles that are at least capable of switching between automatic driving without a monitoring obligation, which is automatic driving without the driver's monitoring obligation, and automatic driving with a monitoring obligation, and that are at least capable of sleep-enabled automatic driving, in which the driver is permitted to sleep, and sleep-disabled automatic driving, in which the driver is not permitted to sleep, as automatic driving without a monitoring obligation. The display control process displays driving-related information regarding the driving of the vehicle on the display, and when the automation switches from sleep-enabled automatic driving to driving at a level lower than automatic driving with a monitoring obligation, after the display control process switches to driving at a level lower than automatic driving with a monitoring obligation, the amount of driving-related information displayed on the display is increased compared to when during sleep-enabled automatic driving.

[0010] According to the above configuration, the display of the surrounding situation image displayed on the display device used in the vehicle cabin is changed depending on whether the autonomous driving mode is switched from autonomous driving without a monitoring obligation to autonomous driving in a hands-on mode or a hands-off mode, which are autonomous driving modes with a monitoring obligation. Therefore, the driver of the vehicle can more easily recognize whether the autonomous driving mode is switched to autonomous driving in a hands-on mode or a hands-off mode from the difference in the display of the surrounding situation image. As a result, when the autonomous driving mode is switched from autonomous driving without a monitoring obligation to autonomous driving with a monitoring obligation, the driver can more easily recognize whether the autonomous driving mode after the switch is in a hands-on mode or a hands-off mode.

[0011] In order to achieve the above object, a vehicle display control system according to the present disclosure includes a display (91, 91b) provided in a vehicle with a display surface facing the interior of the vehicle, and the above-described vehicle display control device (10, 10a, 10b, 10c).

[0012] According to this, since it includes the above-mentioned vehicle display control device, when switching from autonomous driving without a monitoring obligation to autonomous driving with a monitoring obligation, it becomes possible for the driver to more easily recognize whether the autonomous driving after the switch is in hands-on mode or hands-off mode. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram illustrating an example of a schematic configuration of a vehicle system 1. FIG. [Figure 2] FIG. 2 is a diagram illustrating an example of a schematic configuration of an HCU 10. [Figure 3] FIG. 2 is a diagram for explaining an example of a surrounding situation image. [Figure 4] 10A and 10B are diagrams for explaining an example of a difference in the display manner of a surrounding situation image between a hands-on mode and a hands-off mode. [Figure 5] 10A and 10B are diagrams for explaining an example of a difference in the display manner of a surrounding situation image between a hands-on mode and a hands-off mode. [Figure 6] 10A and 10B are diagrams for explaining an example of a difference in the display manner of a surrounding situation image between a hands-on mode and a hands-off mode. [Figure 7] 10A and 10B are diagrams for explaining an example of a difference in the display manner of a surrounding situation image between a hands-on mode and a hands-off mode. [Figure 8] 10A and 10B are diagrams for explaining an example of a difference in the display manner of a surrounding situation image between a hands-on mode and a hands-off mode. [Figure 9] 10A and 10B are diagrams for explaining an example of a difference in the display manner of a surrounding situation image between a hands-on mode and a hands-off mode. [Figure 10] 10A and 10B are diagrams for explaining an example of a difference in the display manner of a surrounding situation image between a hands-on mode and a hands-off mode. [Figure 11] 10A and 10B are diagrams for explaining an example of a difference in the display manner of a surrounding situation image between a hands-on mode and a hands-off mode. [Figure 12] 10 is a flowchart showing an example of the flow of a first display control-related process in the HCU 10 according to the first embodiment. [Figure 13] 10 is a flowchart showing an example of the flow of a first display control-related process in an HCU 10 according to the second embodiment. [Figure 14] 10A and 10B are diagrams for explaining an example of a difference in the display manner of a surrounding situation image between a hands-on mode and a hands-off mode. [Figure 15] FIG. 2 is a diagram illustrating an example of a schematic configuration of an HCU 10a. [Figure 16]FIG. 10 is a diagram for explaining the difference in timing of switching the display depending on whether or not a surrounding situation image is displayed during autonomous driving of the vehicle at level 3 or higher. [Figure 17] FIG. 2 is a diagram showing an example of a schematic configuration of a vehicle system 1b. [Figure 18] FIG. 2 is a diagram illustrating an example of a schematic configuration of an HCU 10b. [Figure 19] 13 is a flowchart showing an example of the flow of second display control-related processing in an HCU 10b according to the sixth embodiment. [Figure 20] FIG. 2 is a diagram illustrating an example of a schematic configuration of an HCU 10c. DETAILED DESCRIPTION OF THE INVENTION

[0014] A number of embodiments for the purpose of disclosure will be described with reference to the drawings. For the sake of convenience, parts having the same functions as parts shown in the drawings used in the previous explanations in the number of embodiments will be given the same reference numerals, and their description may be omitted. For parts given the same reference numerals, the explanations in other embodiments may be referred to.

[0015] (Embodiment 1) <General configuration of vehicle system 1> A first embodiment of the present disclosure will be described below with reference to the drawings. A vehicle system 1 shown in FIG. 1 is used in a vehicle capable of autonomous driving (hereinafter referred to as an autonomous driving vehicle). As shown in FIG. 1, the vehicle system 1 includes an HCU (Human Machine Interface Control Unit) 10, a communication module 20, a locator 30, a map database (hereinafter referred to as a map DB) 40, a vehicle state sensor 50, a periphery monitoring sensor 60, a vehicle control ECU 70, an autonomous driving ECU 80, a display 91, a grip sensor 92, and a user input device 93. This vehicle system 1 corresponds to a vehicle display control system. Although a vehicle using the vehicle system 1 is not necessarily limited to an automobile, the following description will be given taking the case of use in an automobile as an example.

[0016] There are multiple levels of autonomous driving for autonomous vehicles (hereafter referred to as "automation levels"), as defined by the SAE, for example. Automation levels are categorized into levels 0 to 5, for example, as follows:

[0017] Level 0 is the level at which the driver performs all driving tasks without system intervention. The driving task may also be referred to as a dynamic driving task. Examples of driving tasks include steering, acceleration / deceleration, and surrounding monitoring. Level 0 corresponds to so-called manual driving. Level 1 is the level at which the system assists with either steering or acceleration / deceleration. Level 1 corresponds to so-called driving assistance. Level 2 is the level at which the system assists with both steering and acceleration / deceleration. Level 2 corresponds to so-called partial driving automation. Levels 1 and 2 are also considered to be part of automated driving.

[0018] For example, automated driving at levels 1 and 2 is an automated driving system in which the driver has a supervisory obligation (hereinafter simply referred to as supervisory obligation) regarding safe driving. The supervisory obligation includes visual monitoring of the surroundings. Autonomous driving at levels 1 and 2 can be rephrased as automated driving in which a second task is not permitted. A second task is an action other than driving that is permitted to the driver, and is a specific action that has been specified in advance. A second task can also be rephrased as a secondary activity, other activity, etc. A second task must not prevent the driver from responding to a request from the automated driving system to take over driving operations. As examples, actions such as watching content such as videos, operating a smartphone, reading, and eating are considered as second tasks.

[0019] Level 3 is a level where the system can perform all driving tasks in specific locations such as highways, and the driver takes over driving operations in emergencies. At level 3, the driver is required to be able to respond quickly when the system requests a handover of driving. This handover of driving can also be described as the transfer of the responsibility of monitoring the surroundings from the vehicle's system to the driver. Level 3 corresponds to so-called conditional driving automation. Level 4 is a level where the system can perform all driving tasks except under certain circumstances such as on roads that it cannot handle or in extreme environments. Level 4 corresponds to so-called high driving automation. Level 5 is a level where the system can perform all driving tasks in all environments. Level 5 corresponds to so-called full driving automation.

[0020] For example, autonomous driving at levels 3 to 5 is defined as autonomous driving in which the driver does not have a monitoring obligation. Autonomous driving at levels 3 to 5 can also be described as autonomous driving in which a second task is permitted. Among autonomous driving levels 3 to 5, autonomous driving at level 4 or higher is defined as autonomous driving in which the driver is permitted to sleep (hereinafter referred to as sleep-enabled autonomous driving). Among autonomous driving levels 3 to 5, autonomous driving at level 3 is defined as autonomous driving in which the driver is not permitted to sleep (hereinafter referred to as sleep-disabled autonomous driving). In this embodiment, the presence or absence of a monitoring obligation is determined when switching between an automation level of level 3 or higher and an automation level of level 2 or lower. Therefore, when switching from an automation level of level 3 or higher to an automation level of level 2 or lower, the driver is required to monitor safe driving. Meanwhile, the transfer of driving control to the driver may be required, for example, when switching from an automation level of level 2 or higher to an automation level of level 1 or lower. In this embodiment, an example in which driving control is transferred to the driver when switching from an automation level of level 2 or higher to an automation level of level 1 or lower will be described.

[0021] The autonomous vehicle of this embodiment is assumed to be capable of switching automation levels. The automation level may be configured to be switchable only among some of levels 0 to 5. In this embodiment, an example will be described in which the autonomous vehicle is capable of switching between automation level 3 autonomous driving, automation level 2 autonomous driving, and automation level 1 autonomous driving or manual driving. In this embodiment, for example, autonomous driving at automation level 3 is permitted only during traffic jams. Note that this embodiment may also be configured in such a way that autonomous driving at automation level 3 is permitted only during traffic jams and when driving on specific road sections such as expressways or motorways. In the following, an example will be described in which autonomous driving at automation level 3 is permitted only during traffic jams and when driving on specific road sections such as expressways or motorways.

[0022] Furthermore, in this embodiment, automated driving at automation level 2 includes a hands-on mode that requires the driver to hold the steering wheel of the vehicle, and a hands-off mode that does not require the driver to hold the steering wheel of the vehicle. As an example, the use of the hands-on mode and the hands-off mode may be differentiated as follows. For example, if the switch from automation level 3 to automation level 2 is planned based on a situation that can be predicted in advance, the system may be configured to switch to automated driving in the hands-off mode. On the other hand, if the switch from automation level 3 to automation level 2 is unplanned (i.e., sudden) based on a situation that cannot be predicted in advance, the system may be configured to switch to automated driving in the hands-on mode. This is because if the switch from automation level 3 to automation level 2 is sudden, there is a high possibility that relatively large vehicle behavior will occur, and it is considered that the driver will need to hold the steering wheel. Note that automated driving at automation level 1 corresponds to automated driving in the hands-on mode.

[0023] In addition, the present invention is not limited to the above example, and a configuration may be adopted in which the hands-on mode and the hands-off mode are switched depending on whether or not high-precision map data is available in the section. For example, the hands-off mode may be selected in the section where high-precision map data is available, while the hands-on mode may be selected in the section where high-precision map data is not available. High-precision map data will be described later. Furthermore, a configuration may be adopted in which the hands-on mode and the hands-off mode are switched depending on whether or not the vehicle is approaching a specific point. For example, the hands-off mode may be selected when the vehicle is not approaching a specific point, while the hands-on mode may be selected when the vehicle is approaching a specific point. Whether or not the vehicle is approaching a specific point may be determined by whether or not the distance to the specific point is equal to or less than a predetermined value. Examples of specific points include a toll booth in the above-mentioned specific road section, an exit in the above-mentioned specific road section, a merging point, an intersection, a two-way traffic section, a point where the number of lanes decreases, and so on. A specific point can also be described as a point where it is estimated that the driver is more likely to need to hold the steering wheel.

[0024] The communication module 20 transmits and receives information to and from other vehicles via wireless communication. That is, it performs vehicle-to-vehicle communication. The communication module 20 may also transmit and receive information to and from roadside devices installed on the roadside via wireless communication. That is, it may perform road-to-vehicle communication. When performing road-to-vehicle communication, the communication module 20 may receive information about surrounding vehicles transmitted from surrounding vehicles of the host vehicle via the roadside device. Furthermore, the communication module 20 may transmit and receive information to and from a center external to the host vehicle via wireless communication. That is, it may perform wide-area communication. When performing wide-area communication, the communication module 20 may receive information about surrounding vehicles transmitted from surrounding vehicles of the host vehicle via the center. Additionally, when performing wide-area communication, the communication module 20 may receive traffic congestion information, weather information, and the like around the host vehicle from the center.

[0025] Locator 30 includes a GNSS (Global Navigation Satellite System) receiver and an inertial sensor. The GNSS receiver receives positioning signals from multiple positioning satellites. The inertial sensor includes, for example, a gyro sensor and an acceleration sensor. Locator 30 sequentially determines the position of the vehicle (hereinafter referred to as the vehicle position) on which locator 30 is mounted by combining the positioning signals received by the GNSS receiver with the measurement results of the inertial sensor. The vehicle position is represented, for example, by latitude and longitude coordinates. Note that the vehicle position may also be determined using a travel distance calculated from signals sequentially output from a vehicle speed sensor mounted on the vehicle.

[0026] The map DB 40 is a non-volatile memory that stores high-precision map data. The high-precision map data is map data with higher precision than the map data used for route guidance in the navigation function. The map DB 40 may also store map data used for route guidance. The high-precision map data includes information usable for automated driving, such as three-dimensional road shape information, information on the number of lanes, and information indicating the permitted travel direction for each lane. The high-precision map data may also include node point information indicating the positions of both ends of road markings such as lane markings. Note that the locator 30 may be configured to use the three-dimensional road shape information without using a GNSS receiver. For example, the locator 30 may be configured to determine the vehicle position using the three-dimensional road shape information and detection results from a perimeter monitoring sensor 60, such as a LIDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging) or a perimeter monitoring camera, that detects a point cloud of characteristic points of the road shape and structures. The three-dimensional road shape information may be generated based on captured images using REM (Road Experience Management).

[0027] The communication module 20 may receive map data distributed from an external server, for example, via wide-area communication, and store the data in the map DB 40. In this case, the map DB 40 may be configured as a volatile memory, and the communication module 20 may successively acquire map data for an area corresponding to the vehicle position.

[0028] The vehicle state sensor 50 is a group of sensors for detecting various states of the vehicle. The vehicle state sensor 50 includes a vehicle speed sensor for detecting the vehicle speed, a steering sensor for detecting the steering angle, etc. The vehicle state sensor 50 outputs the detected sensing information to an in-vehicle LAN. The sensing information detected by the vehicle state sensor 50 may be configured to be output to the in-vehicle LAN via an ECU mounted in the vehicle.

[0029] The perimeter monitoring sensor 60 monitors the environment around the vehicle. As an example, the perimeter monitoring sensor 60 detects obstacles around the vehicle, such as moving objects such as pedestrians and other vehicles, and stationary objects such as fallen objects on the road. The perimeter monitoring sensor 60 also detects road markings around the vehicle, such as lane markings. The perimeter monitoring sensor 60 is, for example, a perimeter monitoring camera that captures an image of a predetermined area around the vehicle, or a sensor such as millimeter-wave radar, sonar, or LIDAR that transmits a search wave to a predetermined area around the vehicle. The perimeter monitoring camera sequentially outputs the captured images as sensing information to the autonomous driving ECU 80. The sensor that transmits the search wave, such as sonar, millimeter-wave radar, or LIDAR, sequentially outputs scanning results based on received signals obtained when receiving waves reflected by obstacles to the autonomous driving ECU 80 as sensing information. The sensing information detected by the perimeter monitoring sensor 60 may be output to an in-vehicle LAN via the autonomous driving ECU 80.

[0030] The vehicle control ECU 70 is an electronic control device that controls the driving of the vehicle. Examples of driving control include acceleration / deceleration control and / or steering control. The vehicle control ECU 70 includes a steering ECU that controls steering, a power unit control ECU that controls acceleration / deceleration, and a brake ECU. The vehicle control ECU 70 controls driving by outputting control signals to each driving control device installed in the vehicle, such as an electronically controlled throttle, a brake actuator, and an EPS (Electric Power Steering) motor.

[0031] The autonomous driving ECU 80 includes, for example, a processor, memory, I / O, and a bus connecting these, and executes control programs stored in the memory to perform processes related to autonomous driving. The memory referred to here is a non-transitory tangible storage medium that non-temporarily stores computer-readable programs and data. The non-transitory tangible storage medium is realized by a semiconductor memory, a magnetic disk, or the like.

[0032] The autonomous driving ECU 80 includes a first autonomous driving ECU 81 and a second autonomous driving ECU 82. The following description will be given assuming that the first autonomous driving ECU 81 and the second autonomous driving ECU 82 each include a processor, memory, I / O, and a bus connecting these. Note that a configuration may also be adopted in which a common processor performs the functions of the first autonomous driving ECU 81 and the second autonomous driving ECU 82 using virtualization technology.

[0033] The first automatic driving ECU81 is responsible for the functions of automatic driving at level 2 or below mentioned above. In other words, the first automatic driving ECU81 enables automatic driving with a monitoring obligation to be performed. For example, the first automatic driving ECU81 is capable of performing at least one of longitudinal control and lateral control of the host vehicle. The longitudinal direction is the direction that coincides with the front-to-rear direction of the host vehicle. The lateral direction is the direction that coincides with the width direction of the host vehicle. The first automatic driving ECU81 performs acceleration / deceleration control of the host vehicle as longitudinal control. The first automatic driving ECU81 performs steering control of the host vehicle as lateral control. The first automatic driving ECU81 has functional blocks such as a first environment recognition unit, an ACC control unit, an LTA control unit, and an LCA control unit.

[0034] The first environment recognition unit recognizes the driving environment around the host vehicle based on sensing information acquired from the periphery monitoring sensor 60. As an example, the first environment recognition unit recognizes the detailed position of the host vehicle in the driving lane from information such as the dividing lines on the left and right of the lane in which the host vehicle is traveling (hereinafter referred to as the host lane). In addition, the first environment recognition unit recognizes the positions and speeds of obstacles such as vehicles around the host vehicle. The first environment recognition unit recognizes the positions and speeds of obstacles such as vehicles in the host lane. Furthermore, the first environment recognition unit recognizes the positions and speeds of obstacles such as vehicles in lanes surrounding the host lane. The surrounding lanes may be, for example, lanes adjacent to the host lane. Alternatively, the surrounding lanes may be lanes other than the host lane in the road section in which the host vehicle is located. The first environment recognition unit may have the same configuration as the second environment recognition unit described below.

[0035] The ACC control unit executes adaptive cruise control (ACC) to enable the host vehicle to travel at a constant speed at a target speed or to follow a preceding vehicle. The ACC control unit executes ACC control using the positions and speeds of vehicles around the host vehicle recognized by the first environment recognition unit. The ACC control unit executes ACC control by causing the vehicle control ECU 70 to perform acceleration / deceleration control.

[0036] The LTA control unit executes LTA (Lane Tracing Assist) control to keep the host vehicle traveling within the lane. The LTA control unit may execute LTA control by using the detailed position of the host vehicle in the host vehicle lane recognized by the first environment recognition unit. The LTA control unit may execute LTA control by causing the vehicle control ECU 70 to perform steering control. Note that ACC control is an example of longitudinal control, and LTA control is an example of lateral control.

[0037] The LCA control unit executes LCA (Lane Change Assist) control to automatically change lanes of the host vehicle from the host vehicle's own lane to an adjacent lane. The LCA control unit may execute LCA control using the positions and speeds of vehicles around the host vehicle recognized by the first environment recognition unit. For example, the LCA control may be executed when the speed of a vehicle ahead of the host vehicle is low, equal to or less than a predetermined value, and there are no surrounding vehicles approaching the host vehicle from the side or rear side. For example, the LCA control unit may execute LCA control by causing the vehicle control ECU 70 to perform acceleration / deceleration control and steering control.

[0038] The first automatic driving ECU 81 realizes Level 2 automatic driving by executing both ACC control and LTA control. LCA control may be executable, for example, when ACC control and LTA control are being executed. The first automatic driving ECU 81 may realize Level 1 automatic driving by executing either ACC control or LTA control.

[0039] Meanwhile, the second automatic driving ECU 82 is responsible for the aforementioned level 3 or higher automatic driving functions. In other words, the second automatic driving ECU 82 enables automatic driving without the obligation of monitoring. The second automatic driving ECU 82 includes functional blocks such as a second environment recognition unit, an action determination unit, and a trajectory generation unit.

[0040] The second environment recognition unit recognizes the driving environment around the vehicle based on sensing information acquired from the surroundings monitoring sensor 60, the vehicle position acquired from the locator 30, map data acquired from the map DB 40, and information on other vehicles acquired by the communication module 20. As an example, the second environment recognition unit uses this information to generate a virtual space that reproduces the actual driving environment.

[0041] The second environment recognition unit distinguishes between manual driving areas (hereinafter referred to as MD areas) in the area where the host vehicle is traveling. The second environment recognition unit distinguishes between autonomous driving areas (hereinafter referred to as AD areas) in the area where the host vehicle is traveling. The second environment recognition unit distinguishes between ST sections in AD areas. The second environment recognition unit distinguishes between non-ST sections in AD areas.

[0042] An MD area is an area where automated driving is prohibited. In other words, an MD area is an area where the driver must perform all of the vehicle's longitudinal control, lateral control, and periphery monitoring. For example, an MD area may be an ordinary road.

[0043] An AD area is an area where automated driving is permitted. In other words, an AD area is an area where the vehicle can take over one or more of longitudinal control, lateral control, and perimeter monitoring. For example, an AD area may be an expressway or a road for automobiles only.

[0044] AD areas are divided into non-ST sections where automated driving at level 2 or below is possible, and ST sections where automated driving at level 3 or above is possible. In this embodiment, there is no separate division between non-ST sections where automated driving at level 1 is permitted and non-ST sections where automated driving at level 2 is permitted. An ST section may be, for example, a driving section where congestion occurs (hereinafter referred to as a congested section). An ST section may be, for example, a driving section for which high-precision map data has been developed. A non-ST section may be a section that does not fall under the category of an ST section.

[0045] The behavior determination unit determines the behavior planned for the vehicle (hereinafter referred to as future behavior) based on the recognition result of the driving environment by the second environment recognition unit, etc. The behavior determination unit determines the future behavior for driving the vehicle by autonomous driving. The behavior determination unit may determine the type of behavior that the vehicle should take to arrive at the destination as the future behavior. Examples of this type include going straight, turning right, turning left, changing lanes, etc.

[0046] Furthermore, when the behavior determination unit determines that a driver changeover is necessary, it generates a changeover request and outputs it to the HCU 10. An example of a case in which a driver changeover is necessary is when the vehicle moves from an ST section in an AD area to a non-ST section. Another example of a case in which a driver changeover is necessary is when the vehicle moves from an ST section in an AD area to an MD area. Other causes of a driver changeover (hereinafter referred to as a changeover cause) include the elimination of traffic congestion and a lack of high-precision map data.

[0047] A shortage of high-precision map data can be predicted. The behavior determination unit can predict a shortage of high-precision map data for the planned route of the host vehicle by using the host vehicle position measured by the locator 30 and the high-precision map data stored in the map DB 40. When a shortage of high-precision map data is predicted, the behavior determination unit can determine that a driver change is necessary and output a driver change request to the HCU 10 before the host vehicle reaches the point where a shortage of high-precision map data is predicted.

[0048] The elimination of congestion may or may not be predictable. Specifically, if communication module 20 is able to receive congestion information and information on surrounding vehicles, the elimination of congestion can be predicted from this information. The behavior determination unit may predict the elimination of congestion on the planned route of the vehicle using the vehicle's own position measured by locator 30 and the congestion information received by communication module 20. Alternatively, the behavior prediction unit may predict the elimination of congestion on the planned route of the vehicle using the number and speed of surrounding vehicles identified from the information on surrounding vehicles received by communication module 20. Then, if the behavior determination unit predicts the elimination of congestion, it may determine that a driver change is necessary.

[0049] On the other hand, if the communication module 20 cannot receive traffic congestion information or information on surrounding vehicles, it is assumed that the elimination of the traffic congestion cannot be predicted. If the elimination of the traffic congestion cannot be predicted, the elimination of the traffic congestion can be determined using the number, speed, etc. of surrounding vehicles recognized by the second environment recognition unit using the surrounding monitoring sensor 60. Then, if the behavior determination unit determines that the traffic congestion has been resolved, it can determine that a driver change is necessary.

[0050] In addition, there are cases where a driver change is necessary for reasons other than the elimination of traffic congestion or a lack of high-precision map data. Examples include changes in road structure, sudden sensor loss, and sudden bad weather. Changes in road structure that require a driver change include the end of a section with a median strip, a reduction in the number of lanes, and entry into a construction zone. These road structure changes may cause a driver change because they may reduce the accuracy of the recognition of the driving environment. Changes in road structure can be predicted. The behavior determination unit may predict changes in road structure, such as the end of a section with a median strip or a reduction in the number of lanes, along the planned path of the vehicle, using the vehicle's position measured by the locator 30 and the high-precision map data stored in the map DB 40. The behavior determination unit may also predict changes in road structure, such as the vehicle's entry into a construction zone, based on the presence of construction signs or other signs recognized by the second environment recognition unit using the perimeter monitoring sensor 60. The behavior determination unit may then determine that a driver change is necessary when such changes in road structure are predicted.

[0051] Sudden sensor loss is caused by a malfunction of the perimeter monitoring sensor 60, a failure to recognize the driving environment using the perimeter monitoring sensor 60, etc. Sudden bad weather is caused by heavy rain, snow, fog, etc. Sudden bad weather can cause a driver change because it may reduce the accuracy of recognition of the driving environment using the perimeter monitoring sensor 60. Sudden bad weather can also cause a driver change because it may cause a problem with communication in the communication module 20. Sudden sensor loss and sudden bad weather cannot be predicted. The behavior determination unit can determine a sudden sensor loss or sudden bad weather from the recognition results of the driving environment by the second environment recognition unit, etc. Furthermore, the behavior determination unit can determine that a driver change is necessary when it determines a sudden sensor loss or sudden bad weather.

[0052] The trajectory generation unit generates a driving trajectory for the vehicle in a section where autonomous driving is possible, based on the results of the driving environment recognition by the second environment recognition unit and the future action determined by the action determination unit. The driving trajectory includes, for example, target positions of the vehicle according to the progress and target speeds at each target position. The trajectory generation unit sequentially provides the generated driving trajectory to the vehicle control ECU 70 as control commands to be followed by the vehicle during autonomous driving.

[0053] An autonomous driving system including the autonomous driving ECU 80 described above enables the host vehicle to perform autonomous driving at level 2 or lower and level 3 or higher. Furthermore, for example, the autonomous driving ECU 80 may be configured to switch the automation level of the host vehicle's autonomous driving as needed. As an example, when the host vehicle moves from an ST section in an AD area to a non-ST section, the autonomous driving may be switched from level 3 autonomous driving to level 2 or lower autonomous driving. Furthermore, when the host vehicle moves from an ST section in an AD area to an MD area, the autonomous driving ECU 80 may be configured to switch from level 3 autonomous driving to manual driving.

[0054] If a reason for switching from level 3 autonomous driving to level 2 autonomous driving occurs and the reason for the switch was predictable, the autonomous driving ECU 80 can switch to hands-off mode autonomous driving within level 2 autonomous driving. On the other hand, if a reason for switching from level 3 autonomous driving to level 2 autonomous driving occurs and the reason for the switch was not predictable, the autonomous driving ECU 80 can switch to hands-on mode autonomous driving within level 2 autonomous driving. Note that when switching from level 3 autonomous driving to level 1 autonomous driving, autonomous driving will switch to hands-on mode autonomous driving. Whether the autonomous driving will switch to hands-on mode or hands-off mode due to the driver switch can be determined, for example, by the behavior determination unit.

[0055] The display 91 is a display device provided in the vehicle. The display 91 is provided so that the display surface faces the interior of the vehicle. For example, the display 91 is provided so that the display surface is located in front of the driver's seat of the vehicle. Various displays such as a liquid crystal display, an organic EL display, and a head-up display (hereinafter referred to as HUD) can be used as the display 91.

[0056] The grip sensor 92 detects whether the driver is gripping the steering wheel of the vehicle. The grip sensor 92 may be configured to be provided on the rim of the steering wheel. The user input device 93 accepts input from the user. The user input device 93 may be an operation device that accepts operation input from the user. The operation device may be a mechanical switch or a touch switch integrated with a display device. Note that the user input device 93 is not limited to an operation device that accepts operation input, as long as it is a device that accepts input from the user. For example, it may be a voice input device that accepts voice command input from the user.

[0057] HCU10 mainly consists of a computer equipped with a processor, volatile memory, non-volatile memory, I / O, and a bus connecting these components, and is connected to display 91 and in-vehicle LAN. By executing the control program stored in the non-volatile memory, HCU10 controls the display on display 91. This HCU10 corresponds to the vehicle display control device. The configuration of HCU10 regarding the control of the display on display 91 will be described in detail below.

[0058] <Schematic Configuration of HCU10> Subsequently, the schematic configuration of HCU10 will be described using FIG. 2. Regarding the control of the display on display 91, as shown in FIG. 2, HCU10 includes a shift request acquisition unit 101, a mode identification unit 102, an interruption estimation unit 103, a lane change identification unit 104, a grip identification unit 105, and a display control unit 106 as functional blocks. Also, the execution of the processing of each functional block of HCU10 by a computer corresponds to the execution of the vehicle display control method. Note that part or all of the functions executed by HCU10 may be hardware-configured by one or more ICs, etc. Also, part or all of the functional blocks included in HCU10 may be realized by a combination of software execution by a processor and hardware components.

[0059] The shift request acquisition unit 101 acquires the shift request output from the automatic driving ECU 80. The shift request acquisition unit 101 acquires this shift request when the shift request is output from the automatic driving ECU 80.

[0060] The mode identification unit 102 identifies whether the autonomous driving mode to be executed by the host vehicle when the autonomous driving is at automation level 2 or lower is a hands-on mode or a hands-off mode. This processing by the mode identification unit 102 corresponds to a mode identification step. Autonomous driving at automation level 2 or lower can be rephrased as autonomous driving with a supervisory obligation. The mode identification unit 102 may perform the above identification based on the determination result by the action determination unit of the autonomous driving ECU 80 as to whether the autonomous driving mode will be switched to a hands-on mode or a hands-off mode when the driver is changed over. The mode identification unit 102 may maintain the above identification result until the automation level of the host vehicle is switched. Note that when autonomous driving is switched from automation level 2 and hands-off mode to automation level 1, the mode identification unit 102 may identify the autonomous driving as a hands-on mode.

[0061] The cut-in estimation unit 103 estimates whether a nearby vehicle will cut in to the lane in which the host vehicle is traveling (i.e., the host vehicle's lane). The cut-in estimation unit 103 may estimate that a nearby vehicle will cut in to the host vehicle's lane, for example, based on the recognition result of the nearby vehicle of the host vehicle in the driving environment recognized by the first environment recognition unit of the autonomous driving ECU 80. For example, the cut-in estimation unit 103 may estimate that a nearby vehicle will cut in to the host vehicle's lane when the acceleration of the nearby vehicle toward the host vehicle's lane is equal to or greater than a threshold. The cut-in estimation unit 103 may also estimate that a nearby vehicle will cut in to the host vehicle's lane based on the illumination of a turn signal lamp of the nearby vehicle toward the host vehicle's lane. The illumination of the turn signal lamp of the nearby vehicle may be configured to be recognized by the first environment recognition unit through image analysis of an image captured by a periphery monitoring camera. Alternatively, if the information about the nearby vehicle received by the communication module 20 includes information indicating that a nearby vehicle will cut in to the host vehicle's lane, the cut-in estimation unit 103 may use this information to estimate that a nearby vehicle will cut in to the host vehicle's lane.

[0062] The lane change identification unit 104 identifies that the host vehicle will change lanes through autonomous driving. Since LCA control is executed by the LCA control unit of the autonomous driving ECU 80, for example, the lane change identification unit 104 may identify that the host vehicle will change lanes through autonomous driving.

[0063] The grip identification unit 105 identifies how the driver is gripping the steering wheel of the vehicle. For example, the grip identification unit 105 may identify how the driver is gripping the steering wheel from the detection result of the grip sensor 92. Note that the grip identification unit 105 may also identify how the driver is gripping the steering wheel from a source other than the detection result of the grip sensor 92. For example, the driver's grip on the steering wheel may be identified by performing image recognition on an image of the driver captured by a DSM (Driver Status Monitor).

[0064] The display control unit 106 controls the display on the display 91. This processing by the display control unit 106 corresponds to a display control step. When the host vehicle is being driven autonomously at level 2 or lower, or when the host vehicle is being driven manually, the display control unit 106 causes the display 91 to display an image showing the surrounding conditions of the host vehicle (hereinafter, a surrounding conditions image). The display control unit 106 may use the positional relationship between the host vehicle and surrounding vehicles in the driving environment recognized by the autonomous driving ECU 80 to cause the display 91 to display the surrounding conditions image as an overhead image showing the positional relationship between the host vehicle and surrounding vehicles as seen from a virtual viewpoint above the host vehicle. This virtual viewpoint may be directly above the host vehicle or may be a position offset from directly above the host vehicle. For example, the surrounding conditions image may be an overhead view seen from a virtual viewpoint above and behind the host vehicle. The surrounding conditions image may be a virtual image showing the surrounding conditions of the host vehicle, or may be a processed image captured by a perimeter monitoring camera of the perimeter monitoring sensor 60.

[0065] An example of a surrounding situation image will now be described with reference to FIG. 3. Sc in FIG. 3 indicates the display screen of the display device 91. PLI in FIG. 3 indicates an image showing lane markings (hereinafter referred to as lane marking image). HVI in FIG. 3 indicates an image showing the host vehicle (hereinafter referred to as host vehicle image). OVI in FIG. 3 indicates an image showing vehicles surrounding the host vehicle (hereinafter referred to as surrounding vehicle image). FIGS. 3 to 11 show examples where the surrounding vehicles are vehicles ahead of the host vehicle. Ve in FIG. 3 indicates an image showing the vehicle speed of the host vehicle (hereinafter referred to as vehicle speed image).

[0066] As shown in Fig. 3, the surrounding situation image includes an image of the vehicle itself, an image of surrounding vehicles, an image of lane lines, and an image of vehicle speed. The image of the vehicle itself, the image of surrounding vehicles, the image of lane lines, and the image of vehicle speed correspond to the image elements of the surrounding situation image. As shown in Fig. 3, the surrounding situation image may include image elements other than the image of the vehicle itself, the image of surrounding vehicles, and the image of lane lines, which are images showing the surrounding situation of the vehicle itself.

[0067] In addition, when an image showing the view in front of the vehicle is used as the surrounding situation image, the surrounding situation image may not include an image of the vehicle. Also, the surrounding situation image may include image elements such as an assistance execution image, a hands-on / off image, and a background image. The assistance execution image is an image showing control related to driving assistance being performed on the vehicle. Examples of control related to driving assistance include the above-mentioned ACC control and LTA control. The hands-on / off image is an image showing whether the vehicle is autonomously driving in hands-on mode or hands-off mode. The background image is an image showing the background of the surrounding situation image.

[0068] On the other hand, when the host vehicle is driving autonomously at level 3 or higher, the display control unit 106 may, for example, not display the surrounding situation image, but instead display an image explaining the action permitted as the second task, an image indicating the host vehicle's speed, etc., on the display 91. Another example of not displaying the surrounding situation image is to display an image of the host vehicle and an image of the lane markings corresponding to the host vehicle's lane, but not images of surrounding vehicles. This means that images of surrounding vehicles are not displayed even if the perimeter monitoring sensor 60 detects surrounding vehicles.

[0069] When the host vehicle switches from level 3 autonomous driving to level 2 or lower autonomous driving, the display control unit 106 changes the display of the surrounding situation image depending on whether the mode identification unit 102 identifies the autonomous driving as hands-on mode or hands-off mode. Note that autonomous driving at automation level 3 can also be rephrased as autonomous driving without monitoring obligation. Hereinafter, an example of the difference in the display mode of the surrounding situation image between hands-on mode and hands-off mode when the host vehicle switches from level 3 autonomous driving to level 2 autonomous driving will be described with reference to FIGS. 4 to 11. HON in FIGS. 4 to 11 indicates the display mode in hands-on mode. Meanwhile, HOFF in FIGS. 4 to 11 indicates the display mode in hands-off mode.

[0070] When the mode specification unit 102 specifies that the autonomous driving mode is a hands-on mode, the display control unit 106 may display the current lane and surrounding lanes. On the other hand, when the mode specification unit 102 specifies that the autonomous driving mode is a hands-off mode, the display control unit 106 may display only the current lane out of the current lane and surrounding lanes. The surrounding lanes may be, for example, lanes adjacent to the current lane. Alternatively, the surrounding lanes may be lanes other than the current lane in the road section in which the current vehicle is located. As a specific example, as shown in FIG. 4, in the hands-on mode, it is sufficient to display lane marking images for both the current lane and surrounding lanes. On the other hand, in the hands-off mode, it is sufficient to display a lane marking image for only the current lane out of the current lane and surrounding lanes.

[0071] In hands-off mode, which is likely to ensure safety more than hands-on mode, it is considered sufficient for the driver to know the situation closer to the vehicle. Conversely, in hands-on mode, it is considered that the driver wants to know the situation further away from the vehicle. In response to this, with the above configuration, when the vehicle is in hands-on mode, the situation of more lanes is displayed than when the vehicle is in hands-off mode. Therefore, it is possible to display a surrounding situation image in a display mode corresponding to whether the vehicle is in hands-on mode or hands-off mode. Furthermore, because the number of lanes displayed in the surrounding situation image differs depending on whether the vehicle is in hands-on mode or hands-off mode, the driver of the vehicle can more easily recognize from this difference whether the system is switching to autonomous driving in hands-on mode or hands-off mode.

[0072] When the mode specification unit 102 specifies the autonomous driving mode as hands-on mode, the display control unit 106 may display a surrounding situation image viewed from a virtual viewpoint that is farther away from the display target in the surrounding situation image than when the mode specification unit 102 specifies the autonomous driving mode as hands-off mode. On the other hand, when the mode specification unit 102 specifies the autonomous driving mode as hands-off mode, the display control unit 106 may display a surrounding situation image viewed from a virtual viewpoint that is closer to the display target than when the mode specification unit 102 specifies the autonomous driving mode as hands-on mode. The display target here refers to objects, lane lines, and the like represented in the surrounding situation image. As a specific example, as shown in FIG. 5, in the hands-on mode, the display control unit 106 may display a surrounding situation image that looks like the surroundings of the vehicle are viewed from a farther distance than in the hands-off mode. On the other hand, in the hands-off mode, the display control unit 106 may display a surrounding situation image that looks like the surroundings of the vehicle are viewed from a closer distance than in the hands-on mode.

[0073] According to the above configuration, when the host vehicle is in hands-on mode, a wider range of the situation is displayed than when the host vehicle is in hands-off mode. Therefore, it is possible to display the surrounding situation image in a display mode that corresponds to whether the host vehicle is in hands-on mode or hands-off mode. Furthermore, because the perspective of the virtual viewpoint of the surrounding situation image differs depending on whether the host vehicle is in hands-on mode or hands-off mode, the driver of the vehicle can more easily recognize from this difference whether the vehicle is switching to autonomous driving in hands-on mode or hands-off mode.

[0074] When the mode specification unit 102 specifies the autonomous driving mode as hands-on mode, the display control unit 106 may display a surrounding situation image viewed from a virtual viewpoint looking down from above, compared to when the mode specification unit 102 specifies the autonomous driving mode as hands-off mode. On the other hand, when the mode specification unit 102 specifies the autonomous driving mode as hands-off mode, the display control unit 106 may display a surrounding situation image viewed from a virtual viewpoint looking down from below, compared to when the mode specification unit 102 specifies the autonomous driving mode as hands-on mode. As a specific example, as shown in FIG. 6, in the hands-on mode, the display control unit 106 may display a surrounding situation image in which the situation of the host vehicle is viewed from a higher viewpoint than in the hands-off mode. On the other hand, in the hands-off mode, the display control unit 106 may display a surrounding situation image in which the situation of the host vehicle is viewed from a lower viewpoint than in the hands-on mode.

[0075] According to the above configuration, when the host vehicle is in hands-on mode, a wider range of the situation is displayed than when the host vehicle is in hands-off mode. Therefore, it is possible to display the surrounding situation image in a display mode that corresponds to whether the host vehicle is in hands-on mode or hands-off mode. Furthermore, because the height of the virtual viewpoint of the surrounding situation image differs depending on whether the host vehicle is in hands-on mode or hands-off mode, the driver of the vehicle can more easily recognize from this difference whether the vehicle is switching to autonomous driving in hands-on mode or hands-off mode.

[0076] When the mode specification unit 102 specifies the autonomous driving mode as hands-on mode, the display control unit 106 may display a wider area around the host vehicle as a surrounding situation image than when the mode specification unit 102 specifies the autonomous driving mode as hands-off mode. On the other hand, when the mode specification unit 102 specifies the autonomous driving mode as hands-off mode, the display control unit 106 may display a narrower area around the host vehicle as a surrounding situation image than when the mode specification unit 102 specifies the autonomous driving mode as hands-on mode. As a specific example, as shown in FIG. 7 , in the hands-on mode, a surrounding situation image with a wider cropped area around the host vehicle than in the hands-off mode may be displayed. On the other hand, in the hands-off mode, a surrounding situation image with a narrower cropped area around the host vehicle than in the hands-on mode may be displayed.

[0077] According to the above configuration, when the host vehicle is in hands-on mode, a wider range of the situation is displayed than when the host vehicle is in hands-off mode. Therefore, it is possible to display the surrounding situation image in a display mode that corresponds to whether the host vehicle is in hands-on mode or hands-off mode. Furthermore, because the range of the surroundings of the host vehicle shown in the surrounding situation image differs depending on whether the host vehicle is in hands-on mode or hands-off mode, the driver of the vehicle can more easily recognize from this difference whether the system is switching to autonomous driving in hands-on mode or hands-off mode.

[0078] The display control unit 106 may simply change the color tone of at least a part of the surrounding situation image depending on whether the mode identification unit 102 has identified the autonomous driving as a hands-on mode or a hands-off mode. As a specific example, as shown in FIG. 8, the color tone of the assistance execution image (see ACC and LTA in FIG. 8) may be changed between the hands-on mode and the hands-off mode. ACC in FIG. 8 shows an assistance execution image indicating that ACC control is being executed. LTA in FIG. 8 shows an assistance execution image indicating that LTA control is being executed. While the example in FIG. 8 shows an example in which the color tone of the assistance execution image is changed between the hands-on mode and the hands-off mode, this is not necessarily limited to this. For example, a configuration may be adopted in which the color tone of image elements of the surrounding situation image other than the assistance execution image is changed.

[0079] According to the above configuration, the color tone of the image elements in the surrounding situation image is changed depending on whether the vehicle is in hands-on mode or hands-off mode, so that the driver of the vehicle can more easily recognize from this difference whether the vehicle is switching to hands-on mode autonomous driving or hands-off mode autonomous driving.

[0080] Furthermore, when the mode identification unit 102 identifies the autonomous driving mode as hands-on mode, it is preferable that the display control unit 106 displays the image elements of the surrounding situation image in a color tone that is more likely to call attention than when the mode identification unit 102 identifies the autonomous driving mode as hands-off mode. For example, when the hands-on mode is identified, the image elements may be displayed in an exciting color tone such as red. On the other hand, when the hands-off mode is identified, the image elements may be displayed in a calming color tone such as blue.

[0081] It is considered that the driver needs to pay more attention to the driving of the vehicle in hands-on mode than in hands-off mode. In response to this, with the above configuration, when the vehicle is in hands-on mode, the image elements of the surrounding situation image are displayed in colors that are more likely to attract attention than when the vehicle is in hands-off mode. Therefore, it is possible to display the surrounding situation image in a display mode that corresponds to whether the vehicle is in hands-on mode or hands-off mode.

[0082] The display control unit 106 may change at least one of the layout and size ratio of image elements in the surrounding situation image depending on whether the mode specification unit 102 specifies autonomous driving in hands-on mode or hands-off mode. As a specific example, the layout of image elements may be changed between hands-on mode and hands-off mode, as shown in Fig. 9. HM in Fig. 9 indicates the hands-on / off image. In the example of Fig. 9, the layout of the image elements in the surrounding situation image that indicate the surrounding situation of the vehicle and the hands-on / off image on the left and right is changed between hands-on mode and hands-off mode.

[0083] According to the above configuration, the arrangement of image elements in the surrounding situation image differs depending on whether the vehicle is in hands-on mode or hands-off mode, making it easier for the vehicle driver to recognize from this difference whether the vehicle is switching to hands-on mode autonomous driving or hands-off mode autonomous driving.

[0084] Furthermore, as shown in FIG. 10, when the mode identification unit 102 identifies the autonomous driving mode as hands-on mode, it is preferable that the display control unit 106 increases the size ratio of the hands-on / off image compared to when the mode identification unit 102 identifies the autonomous driving mode as hands-off mode.

[0085] In the hands-off mode, the driver does not need to grip the steering wheel, but in the hands-on mode, the driver must grip the steering wheel. Therefore, it is preferable that the driver is more likely to notice the hands-on / off image in the hands-on mode than in the hands-off mode. In contrast, with the above configuration, when the host vehicle is in the hands-on mode, the hands-on / off image is displayed larger than when the host vehicle is in the hands-off mode, making it easier for the driver to notice the hands-on / off image. Therefore, it is possible to display the surrounding situation image in a display mode that corresponds to whether the host vehicle is in the hands-on mode or the hands-off mode.

[0086] The display control unit 106 may simply change the background image in the surrounding situation image depending on whether the mode identification unit 102 has identified autonomous driving as hands-on mode or hands-off mode. As a specific example, the background image may be different between hands-on mode and hands-off mode, as shown in FIG. 11. BI in FIG. 11 indicates the background image. As an example, if a pattern is to be displayed as the background image, this pattern may be changed. Alternatively, the background image may be displayed more clearly in hands-on mode than in hands-off mode.

[0087] According to the above configuration, the background image in the surrounding situation image is different depending on whether the vehicle is in hands-on mode or hands-off mode, so the driver of the vehicle can more easily recognize from this difference whether the vehicle is switching to hands-on mode autonomous driving or hands-off mode autonomous driving.

[0088] The display control unit 106 may be configured to switch between various display modes depending on whether the vehicle is in hands-on mode or hands-off mode, as illustrated in Figures 4 to 11, or may be configured to switch between a number of modes in combination. When the vehicle switches from level 3 autonomous driving to level 1 autonomous driving or manual driving, the display control unit 106 may display the surrounding situation image in the hands-off mode display mode.

[0089] When the host vehicle switches to autonomous driving in hands-off mode, and the host vehicle changes lanes by autonomous driving, or when it is estimated that a nearby vehicle will cut into the host vehicle's lane, the display control unit 106 preferably switches the display of the surrounding situation image to that in the case where the mode identification unit 102 identifies the host vehicle as being in autonomous driving in hands-on mode, even if the host vehicle continues to drive autonomously in hands-off mode. In other words, even if the mode identification unit 102 identifies the host vehicle as being in autonomous driving in hands-off mode, it is preferable to switch the display of the surrounding situation image to the same display mode as in hands-on mode. The lane change identification unit 104 may identify that the host vehicle will change lanes by autonomous driving. The cut-in estimation unit 103 may estimate that a nearby vehicle will cut into the host vehicle's lane.

[0090] When the vehicle changes lanes due to autonomous driving or when it is estimated that a nearby vehicle will cut into the vehicle's lane, the possibility of a relatively large vehicle behavior increases even in hands-off mode, and it is considered that the possibility of transitioning to hands-on mode increases. In contrast, with the above configuration, even if autonomous driving in hands-off mode continues, if the possibility of transitioning to hands-on mode increases, the driver can more easily prepare for the transition to hands-on mode.

[0091] When the time elapsed since the host vehicle switched to autonomous driving in hands-off mode reaches a specified time, the display control unit 106 preferably switches to displaying a surrounding situation image for when autonomous driving in hands-on mode is identified by the mode identification unit 102, even if autonomous driving in hands-off mode continues. The specified time here is a time that can be set arbitrarily.

[0092] It is considered that the amount of information that the driver must confirm is greater when the vehicle is in hands-on mode than when the vehicle is in hands-off mode. In response to this, with the above configuration, the display of the surrounding situation image is switched to the same display as in hands-on mode even before switching from hands-off mode to hands-on mode. This reduces the amount of information that is newly added when switching from hands-off mode to hands-on mode, thereby reducing the burden on the driver.

[0093] When the vehicle has switched to automatic driving in hands-off mode and the grip identification unit 105 identifies that the steering wheel is being gripped, it is preferable that the display control unit 106 switch to displaying an image of the surrounding conditions when the mode identification unit 102 identifies automatic driving in hands-on mode, even if automatic driving in hands-off mode continues.

[0094] Even when the vehicle is in hands-off mode, if the driver holds the steering wheel, it is considered preferable to display a surrounding situation image similar to that in hands-on mode, just as if the vehicle were in hands-on mode. In contrast, with the above configuration, even when the vehicle is in hands-off mode, if the driver holds the steering wheel, it is possible to display a surrounding situation image similar to that in hands-on mode.

[0095] In addition, the display control unit 106 may be configured to reverse or customize the displays for the hands-on mode and the hands-off mode according to the driver's preferences. As an example, it may be configured to reverse or customize the displays for the hands-on mode and the hands-off mode according to the input received by the user input device 93.

[0096] <First display control related processing in HCU10> Here, using the flowchart of FIG. 12, an example of the flow of processing related to the control of the display according to whether the hands-on mode or the hands-off mode is selected in HCU10 (hereinafter, the first display control related processing) will be described. The flowchart of FIG. 12 may be configured to start, for example, when a driving change is made after the host vehicle starts level 3 automated driving. HCU10 may determine that a driving change has occurred based on the acquisition of a change request by the change request acquisition unit 101. Further, as described above, during level 3 automated driving, the display control unit 106 may not display the surrounding situation image, and may display, for example, an image explaining an action permitted as a second task on the display 91.

[0097] First, in step S1, the mode identification unit 102 identifies whether the automated driving to be executed by the host vehicle after the driving change is the hands-on mode or the hands-off mode. If it is identified as the hands-on mode (YES in S1), the process proceeds to step S2. On the other hand, if it is identified as the hands-off mode (NO in S1), the process proceeds to step S3.

[0098] In step S2, the display control unit 106 displays the surrounding situation image on the display 91 in the display mode for the hands-on mode described above, and the process proceeds to step S8. On the other hand, in step S3, the display control unit 106 displays the surrounding situation image on the display 91 in the display mode for the hands-off mode described above.

[0099] In step S4, if the lane change identification unit 104 identifies that the vehicle will change lanes by autonomous driving (YES in S4), the process proceeds to S2. On the other hand, if the lane change identification unit 104 does not identify that the vehicle will change lanes by autonomous driving (NO in S4), the process proceeds to step S5.

[0100] In step S5, if the cut-in estimation unit 103 estimates that a nearby vehicle will cut into the own lane (YES in S5), the process proceeds to S2. On the other hand, if the cut-in estimation unit 103 does not estimate that a nearby vehicle will cut into the own lane (NO in S5), the process proceeds to S6.

[0101] In step S6, if the grip identification unit 105 identifies that the steering wheel is being gripped (YES in S6), the process proceeds to S2. On the other hand, if the grip identification unit 105 does not identify that the steering wheel is being gripped (NO in S6), the process proceeds to S7.

[0102] In step S7, if the elapsed time since the driver changeover has reached the specified time (YES in S7), the process proceeds to S2. On the other hand, if the elapsed time since the driver changeover has not reached the specified time (NO in S7), the process proceeds to step S8.

[0103] In step S8, if it is time to end the first display control-related process (YES in S8), the first display control-related process is ended. On the other hand, if it is not time to end the first display control-related process (NO in S8), the process returns to S1 and is repeated. Examples of timing to end the first display control-related process include when the power switch is turned off, when the vehicle switches to autonomous driving at level 3 or higher, etc.

[0104] <Summary of First Embodiment> According to the configuration of the first embodiment, the display of the surrounding situation image displayed on the display 91 used in the cabin of the vehicle is changed depending on whether the autonomous driving mode is switched from autonomous driving without a monitoring obligation to autonomous driving in a hands-on mode or a hands-off mode, which are autonomous driving modes with a monitoring obligation. Therefore, the driver of the vehicle can more easily recognize whether the autonomous driving mode is switched to autonomous driving in a hands-on mode or a hands-off mode from the difference in the display of the surrounding situation image. As a result, when the autonomous driving mode is switched from autonomous driving without a monitoring obligation to autonomous driving with a monitoring obligation, the driver can more easily recognize whether the autonomous driving mode after the switch is in a hands-on mode or a hands-off mode.

[0105] Furthermore, as described above, it is believed that the display format required differs between autonomous driving in hands-on mode and autonomous driving in hands-off mode. In contrast, the configuration of embodiment 1 makes it possible to display a surrounding situation image in a display format that corresponds to whether the host vehicle is in hands-on mode or hands-off mode. In this respect, when switching from autonomous driving without a monitoring obligation to autonomous driving with a monitoring obligation, the driver can more easily recognize whether the autonomous driving after the switch is in hands-on mode or hands-off mode.

[0106] (Embodiment 2) In the first embodiment, when the grip identification unit 105 identifies that the steering wheel is being gripped while the host vehicle has switched to autonomous driving in a hands-off mode, the display control unit 106 switches to displaying a surrounding situation image for when the mode identification unit 102 identifies that the host vehicle is in autonomous driving in a hands-on mode. However, this is not necessarily limited to this. For example, the configuration of the second embodiment described below may be used. An example of the second embodiment will be described below with reference to the drawings. The vehicle system 1 of the second embodiment is similar to the vehicle system 1 of the first embodiment, except that part of the processing by the display control unit 106 when the grip identification unit 105 identifies that the steering wheel is being gripped while the host vehicle has switched to autonomous driving in a hands-off mode is different.

[0107] In the second embodiment, when the grip identification unit 105 identifies that the steering wheel is being gripped while the host vehicle has switched to autonomous driving in hands-off mode, the display control unit 106 preferably continues to display the surrounding situation image for the case where the mode identification unit 102 identifies autonomous driving in hands-off mode for a predetermined time after the grip identification unit 105 identifies that the steering wheel is being gripped, and then switches to displaying the surrounding situation image for the case where the mode identification unit 102 identifies autonomous driving in hands-on mode, even if autonomous driving in hands-off mode continues. The predetermined time here is a time that can be set arbitrarily.

[0108] Here, an example of the flow of the first display control-related processing in the HCU 10 of the second embodiment will be described with reference to the flowchart of Fig. 13. The flowchart of Fig. 13 may be configured to be started under the same conditions as the flowchart of Fig. 12.

[0109] In step S21, the mode identification unit 102 identifies whether the vehicle will be operating in a hands-on mode or a hands-off mode after the driver changeover. If the mode is identified as the hands-on mode (YES in S21), the process proceeds to step S22. On the other hand, if the mode is identified as the hands-off mode (NO in S21), the process proceeds to step S23.

[0110] In step S22, the display control unit 106 causes the display 91 to display the surrounding situation image in the hands-on mode display mode described above in the first embodiment, and the process proceeds to step S29. Meanwhile, in step S23, the display control unit 106 causes the display 91 to display the surrounding situation image in the hands-off mode display mode described above in the first embodiment.

[0111] The processing from step S24 to step S26 may be the same as the processing from S1 to S6 described above. In step S27, if the time elapsed since the driver change has reached the specified time (YES in S27), the process proceeds to S28. On the other hand, if the time elapsed since the driver change has not reached the specified time (NO in S27), the process proceeds to step S29. In step S28, the display of the surrounding situation image in the hands-off mode display mode continues for a predetermined time after it is determined in S26 that the steering wheel is being gripped, and then the process proceeds to S22.

[0112] In step S29, if it is time to end the first display control-related process (YES in S29), the first display control-related process is ended. On the other hand, if it is not time to end the first display control-related process (NO in S29), the process returns to S21 and is repeated.

[0113] As with the first embodiment, the configuration of the second embodiment also makes it possible for the driver to easily recognize whether the autonomous driving mode after switching from autonomous driving without a monitoring obligation to autonomous driving with a monitoring obligation is a hands-on mode or a hands-off mode. Furthermore, according to the configuration of the second embodiment, when the vehicle is in hands-off mode, even if the driver is holding the steering wheel, the surrounding situation image is displayed in the hands-off mode display mode for a predetermined time. This makes it possible for the driver to recognize that it is not necessary to hold the steering wheel.

[0114] (Embodiment 3) In the first embodiment, a configuration was shown in which, in hands-off mode, a lane marking image of only the own lane out of the own lane and surrounding lanes was displayed. However, if an obstacle is detected in the surrounding lanes, the configuration of the following third embodiment may be used. An example of the third embodiment will be described below with reference to the drawings. In the following, a surrounding vehicle will be described as an example of the obstacle.

[0115] In the example of the third embodiment, a display example will be described in which the surrounding situation image also includes images of surrounding vehicles, as shown in Fig. 14. OVIH in Fig. 14 indicates an image representing surrounding vehicles located in the own lane. OVIO in Fig. 14 indicates an image representing surrounding vehicles located in lanes surrounding the own lane. In the third embodiment, as described in the first embodiment, when the mode specification unit 102 specifies the autonomous driving mode as hands-off mode, the display control unit 106 displays only the own lane out of the own lane and surrounding lanes. On the other hand, even when only the own lane is displayed, the display control unit 106 allows the display of both an image representing surrounding vehicles corresponding to the own lane and an image representing surrounding vehicles corresponding to surrounding lanes as the surrounding vehicle image.

[0116] According to the above configuration, as in the example shown in FIG. 4 of the first embodiment, narrowing down the display allows for more selective selection of necessary information, making it easier for the driver to understand, compared to when surrounding lanes are displayed. Even when the display of surrounding lanes is omitted, the driver can recognize the situation regarding the surrounding lanes by displaying an image showing surrounding vehicles located in the surrounding lanes. Omitting the display of surrounding lanes also increases the possibility of reducing the nuisance of the display. For example, assume that the positions of lanes are sequentially identified based on map data and the recognition results of the lane markings by the perimeter monitoring sensor 60 and the lanes are displayed. In this case, a problem of display blurring may occur when the lane display is updated. However, the more lanes are displayed, the more noticeable this blurring becomes and the more likely it is to be perceived as a nuisance. Therefore, omitting the display of surrounding lanes makes this blurring less noticeable and reduces the nuisance of the display.

[0117] (Embodiment 4) In the first embodiment, the case where the driving mode is changed from level 3 autonomous driving to level 2 autonomous driving has been described as an example, but this is not necessarily limited to this. For example, the present invention may be applied to a case where the driving mode is changed from level 4 or higher autonomous driving to level 2 or lower autonomous driving or manual driving.

[0118] (Embodiment 5) In the above-described embodiment, a configuration has been shown in which a surrounding situation image is not displayed while the host vehicle is autonomously driving at level 3 or higher, but this is not necessarily limited to this. For example, a configuration may be possible in which a surrounding situation image is displayed even while the host vehicle is autonomously driving at level 3 or higher (hereinafter referred to as embodiment 5). An example of embodiment 5 will be described below with reference to the drawings. The vehicle system 1 of embodiment 5 is similar to the vehicle system 1 of embodiment 1, except that it includes an HCU 10a instead of the HCU 10.

[0119] Here, the schematic configuration of the HCU 10a will be described with reference to Fig. 15. As shown in Fig. 15, the HCU 10a includes, as functional blocks, a change request acquisition unit 101, a mode identification unit 102, an interruption estimation unit 103, a lane change identification unit 104, a grip identification unit 105, and a display control unit 106a for controlling the display on the display device 91. The HCU 10a is similar to the HCU 10 of the first embodiment, except that the HCU 10a includes the display control unit 106a instead of the display control unit 106. This HCU 10a also corresponds to a vehicle display control device. Execution of the processing of each functional block of the HCU 10a by a computer also corresponds to execution of a vehicle display control method.

[0120] The display control unit 106a is similar to the display control unit 106 of the first embodiment, except that it is capable of displaying a surrounding situation image even when the vehicle is autonomously driving at level 3 or higher, and performs processing related to this. The following describes processing that differs from that of the display control unit 106 of the first embodiment.

[0121] For example, the display control unit 106a displays a surrounding situation image even when the host vehicle is autonomously driving at level 3 or higher. Autonomous driving at level 3 or higher can be rephrased as autonomous driving without a supervisory obligation. When the autonomous driving stage (i.e., automation level) is switched to a lower automation stage while the host vehicle is autonomously driving at level 3 or higher and a surrounding situation image is being displayed, the display control unit 106a changes the display of the surrounding situation image corresponding to the automation level before the switch to the display of the surrounding situation image corresponding to the automation level after the switch, regardless of whether the autonomous driving is in hands-on mode or hands-off mode. The predetermined time may be set arbitrarily. According to the above configuration, the display of the surrounding situation image is changed after the automation level is switched, thereby preventing the driver from becoming confused.

[0122] As with the first embodiment, the display of the surrounding situation image after the automation level is switched can be switched between automated driving in hands-on mode and automated driving in hands-off mode. Examples of the display of the surrounding situation image according to the automation level include the following: At level 3, it is sufficient to display a lane marking image of only the vehicle's own lane out of the vehicle's own lane and surrounding lanes. At level 2, it is sufficient to display lane marking images of both the vehicle's own lane and surrounding lanes. Regarding the surrounding vehicle images, it is sufficient to display only the vehicle's own lane at level 3, while it is also sufficient to display surrounding lanes at level 2. In this case, the example shown in FIG. 4 can be omitted for switching the display of the surrounding situation image between hands-on mode and hands-off mode at level 2.

[0123] Furthermore, as described in the first embodiment, if the surrounding situation image is not to be displayed while the vehicle is being driven at level 3 or higher, the following may be done. When the automation level is switched to a lower level while the surrounding situation image is not being displayed during automated driving at level 3 or higher, the display control unit 106 may change the display of the surrounding situation image to one that corresponds to the new automation level, regardless of whether the automated driving is in hands-on mode or hands-off mode, at the same time as the automation level is switched or before the automated driving level is switched. The term "simultaneous" here may include an error that can be considered substantially simultaneous. The above configuration makes it possible to convey information about the surroundings of the vehicle to the driver more quickly.

[0124] Here, using FIG. 16, differences in the timing of display switching depending on whether or not the surrounding situation image is displayed while the host vehicle is autonomously driving at level 3 or higher will be described. Y in FIG. 16 shows an example of a case where the surrounding situation image is displayed while the host vehicle is autonomously driving at level 3 or higher. N in FIG. 16 shows an example of a case where the surrounding situation image is not displayed while the host vehicle is autonomously driving at level 3 or higher. LC in FIG. 16 shows the timing of switching the automation level. S in FIG. 16 shows the timing of starting display of the surrounding situation image according to the post-switched automation level. As shown in FIG. 16, when the host vehicle is autonomously driving at level 3 or higher, the display of the surrounding situation image according to the post-switched automation level starts after the automation level is switched. On the other hand, when the host vehicle is autonomously driving at level 3 or higher and not displaying the surrounding situation image, the display of the surrounding situation image according to the post-switched automation level starts at least before the automation level is switched.

[0125] Note that the configuration is not limited to a fixed one in which whether or not to display a surrounding situation image while the host vehicle is autonomously driving at level 3 or higher. For example, the configuration may be such that the setting of whether or not to display a surrounding situation image while the host vehicle is autonomously driving at level 3 or higher is switchable. The setting may be switched in response to input from the user received by the user input device 93. In this case, the display control unit 106a may be configured to use the above-mentioned processing differently depending on whether or not to display a surrounding situation image.

[0126] (Embodiment 6) The configuration of the vehicle when switching from autonomous driving level 4 or higher to autonomous driving level 3 may be the configuration of the sixth embodiment, which will be described below as an example. An example of the sixth embodiment will be described below with reference to the drawings.

[0127] First, a vehicle system 1b according to the sixth embodiment will be described with reference to Fig. 17. As shown in Fig. 17, the vehicle system 1b includes an HCU 10b, a communication module 20, a locator 30, a map DB 40, a vehicle state sensor 50, a periphery monitoring sensor 60, a vehicle control ECU 70, an autonomous driving ECU 80, a display 91b, a grip sensor 92, a user input device 93, and a DSM (Driver Status Monitor) 94. The vehicle system 1b is similar to the vehicle system 1 according to the first embodiment, except that the vehicle system 1b includes an HCU 10b and a display 91b instead of the HCU 10 and the display 91, and that the vehicle system 1b includes a DSM 94. Note that the vehicle system 1b also corresponds to a vehicle display control system.

[0128] 17, display 91b has a driver's side display 911 and a passenger's side display 912. Display 91b is similar to display 91 of embodiment 1 except that it has two types of display devices, driver's side display 911 and passenger's side display 912.

[0129] The driver's side display 911 is a display device whose display surface is located in front of the driver's seat of the vehicle. The driver's side display 911 can be a meter MID (Multi Information Display) or a HUD (Head-Up Display). The meter MID is a display device provided in front of the driver's seat inside the vehicle cabin. As an example, the meter MID may be provided on a meter panel. The HUD is provided in the vehicle cabin, for example, on an instrument panel. The HUD projects a display image formed by a projector onto a predetermined projection area on the front windshield as a projection member. The light of the image reflected by the front windshield toward the interior of the vehicle is perceived by the driver sitting in the driver's seat. This allows the driver to view a virtual image of the display image formed in front of the front windshield superimposed on part of the foreground. The HUD may be configured to project a display image onto a combiner provided in front of the driver's seat instead of the front windshield. The display surface of the HUD is located above the display surface of the meter MID. The driver's side display 911 may use a plurality of display devices.

[0130] The passenger side display 912 is a display device other than the driver side display 911, whose display surface is located in a position visible to passengers in the vehicle. A passenger is a person in the vehicle other than the driver. Examples of the passenger side display 912 include a display device visible from the passenger seat and a display device visible from the rear seat. An example of a display device visible from the passenger seat is a CID (Center Information Display). A CID is a display device located in the center of the instrument panel of the vehicle. Examples of a display device visible from the rear seat include a display device provided on the seat back or ceiling of the front seat. Multiple display devices may be used as the passenger side display 912.

[0131] The DSM94 is composed of a near-infrared light source, a near-infrared camera, a control unit for controlling them, and the like. The DSM94 is mounted, for example, on the top surface of the instrument panel with the near-infrared camera facing the driver's seat side of the vehicle. The DSM94 uses the near-infrared camera to capture an image of the driver's head illuminated with near-infrared light from the near-infrared light source. The image captured by the near-infrared camera is analyzed by the control unit. The control unit detects the driver's alertness based on the driver's features extracted by analyzing the captured image. The alertness detection at least distinguishes between an alert state and a sleeping state.

[0132] Next, a schematic configuration of the HCU 10b will be described with reference to Fig. 18. As shown in Fig. 18, the HCU 10b includes, as functional blocks, a change request acquisition unit 101, a mode identification unit 102, an interruption estimation unit 103, a lane change identification unit 104, a grip identification unit 105, a display control unit 106b, and a state identification unit 107 for controlling the display on the display device 91b. The HCU 10b is similar to the HCU 10 of the first embodiment, except that the HCU 10b includes the display control unit 106b instead of the display control unit 106 and the state identification unit 107. This HCU 10b also corresponds to a vehicle display control device. Execution of the processing of each functional block of the HCU 10b by a computer also corresponds to execution of a vehicle display control method.

[0133] The state identification unit 107 identifies the state of the driver. The state identification unit 107 identifies the state of the driver's wakefulness from the wakefulness of the driver sequentially detected by the DSM94. The state identification unit 107 distinguishes at least between an wakefulness state in which the driver is wakeful and a sleep state in which the driver is asleep. Here, a configuration in which the wakefulness state of the driver is detected by a control unit of the DSM94 is shown, but the state identification unit 107 may perform part of the function of this control unit. Furthermore, the state identification unit 107 may identify the wakefulness state of the driver from a source other than the detection result of the DSM94. For example, the wakefulness state of the driver may be identified from the detection result of a biosensor that detects the pulse wave of the driver.

[0134] The display control unit 106b is similar to the display control units 106 and 106a, except for some differences in processing. The following describes processing that differs from the display control units 106 and 106a. The display control unit 106b displays information related to the driving of the vehicle (hereinafter referred to as driving-related information) on the display 91b. The driving-related information displayed on the display 91b includes surrounding situation images and images that do not fall under the category of surrounding situation images. In other words, the driving-related information also includes surrounding situation images in its classification. The images that do not fall under the category of surrounding situation images include an image explaining an action permitted as a second task (hereinafter referred to as an ST explanation image), a vehicle speed image, an image of the vehicle, and an image of the lane markings on the vehicle's lane (hereinafter referred to as the vehicle's lane image).

[0135] When switching from sleep-enabling autonomous driving to sleep-disabling autonomous driving, the display control unit 106b increases the amount of driving-related information displayed on the display unit 91b during sleep-enabling autonomous driving compared to the amount of driving-related information displayed on the display unit 91b during sleep-enabling autonomous driving. In this case, the comparison may be made between the amount of information displayed on the same display unit or the combined amount of information displayed on multiple display units. As described above, sleep-enabling autonomous driving is autonomous driving of level 4 or higher. Below, level 4 autonomous driving will be used as an example. As described above, sleep-disabling autonomous driving is autonomous driving of level 3. The amount of information referred to here may refer to the amount of elements of each type of information. For example, examples of elements of each type of information include an image of the vehicle, an image of the vehicle's lane, an image of the dividing lines of surrounding lanes (hereinafter referred to as surrounding lane images), images of surrounding vehicles in the vehicle's lane, images of surrounding vehicles in surrounding lanes, and a vehicle speed image.

[0136] For example, an example of increasing the amount of information displayed during LV3 autonomous driving compared to LV4 autonomous driving can be as follows: If an image of the vehicle itself and an image of the lane it is driving are displayed during LV4 autonomous driving, but images of surrounding vehicles are not displayed, then during LV3 autonomous driving, images of surrounding vehicles can be displayed in addition to images of the vehicle itself and its lane. Also, if an image of the vehicle itself is displayed during LV4 autonomous driving, but images of the lane it is not displayed, then during LV3 autonomous driving, images of the lane it is displayed in addition to an image of the vehicle itself.

[0137] Even when switching from sleep-enabling autonomous driving to a driving level at or below mandatory supervision autonomous driving, the display control unit 106b may increase the amount of driving-related information displayed on the display 91b after switching to the level at or below mandatory supervision autonomous driving compared to the amount of driving-related information displayed on the display 91b during sleep-enabling autonomous driving. Driving levels at or below mandatory supervision autonomous driving include automated driving with automation levels 1 to 2 and manual driving with automation level 0. In this case, it is preferable that the display control unit 106b increase the amount of driving-related information displayed on the display 91b after switching to a level at or below mandatory supervision autonomous driving compared to when sleep-enabling autonomous driving is in progress. This prevents the driver from neglecting to monitor their surroundings due to excessive attention to the display when switching to driving at or below level 2, which requires monitoring their surroundings.

[0138] For example, the following can be used to increase the amount of information displayed when driving at levels of automation below supervisory automated driving compared to sleep-enabled automated driving. If an image of the vehicle itself is displayed but an image of the lane in which the vehicle is traveling is not displayed during automated driving at level 4, images of the lane in which the vehicle is traveling and images of surrounding vehicles can be displayed in addition to an image of the vehicle itself during driving at an automated level of LV2 or lower. In this case, a configuration can be used in which an image of the lane in which the vehicle is traveling is displayed in addition to an image of the vehicle itself during automated driving at level 3.

[0139] During LV4 autonomous driving, it is preferable that the display control unit 106b increases the amount of driving-related information displayed on the display 91b when the state identification unit 107 identifies the driver as asleep compared to the amount of driving-related information displayed on the display 91b when the state identification unit 107 identifies the driver as awake. This allows passengers to check more detailed information about the vehicle's driving even when the driver is asleep during LV4 autonomous driving. Therefore, it is possible to provide passengers with a sense of security even when the driver is asleep during LV4 autonomous driving. While the example described here is one in which driving-related information is displayed on the display 91b, the same can also be applied to a case in which driving-related information is displayed on the display 91.

[0140] For example, an example of displaying more information when the driver is asleep than when the driver is awake during LV4 autonomous driving may be as follows: When the driver is asleep, a vehicle speed image may be displayed but an image of the vehicle and an image of the lane in question may not be displayed, whereas when the driver is asleep, an image of the vehicle and an image of the lane in question may be displayed in addition to the vehicle speed image. Alternatively, when the driver is asleep, an image of the vehicle, an image of the vehicle, and an image of the lane in question may be displayed but an image of vehicles around the vehicle in the lane in question may not be displayed, whereas when the driver is asleep, an image of the vehicle, an image of the vehicle, an image of the lane in question may be displayed in addition to the vehicle speed image, an image of the vehicle, and an image of the lane in question.

[0141] When the state identification unit 107 identifies the driver as asleep during LV4 autonomous driving, the display control unit 106b preferably increases the amount of driving-related information displayed on the passenger-side display 912 compared to the amount displayed on the driver-side display 911, compared to when the state identification unit 107 identifies the driver as awake. In this case, as an example, the driver-side display 911 may display the same amount of driving-related information whether the state identification unit 107 identifies the driver as asleep or awake. On the other hand, when the state identification unit 107 identifies the driver as awake during LV4 autonomous driving, the driver-side display 911 and the passenger-side display 912 may display the same amount of driving-related information. This makes it possible to efficiently provide passengers with information they need while eliminating unnecessary displays when the driver is asleep during LV4 autonomous driving.

[0142] For example, the amount of information displayed can be varied depending on the driver's state during LV4 autonomous driving as follows: When the driver is awake, both the driver's side display 911 and the passenger's side display 912 can be configured to display a vehicle speed image but not an image of the vehicle or the lane in question; when the driver is asleep, the driver's side display 911 can display a vehicle speed image but not an image of the vehicle or the lane in question, while the passenger's side display 912 can display an image of the vehicle speed image as well as an image of the vehicle or the lane in question.

[0143] When the state identification unit 107 determines that the driver is not asleep during LV4 autonomous driving, the display control unit 106b preferably changes the display of information to correspond to the level of LV3 autonomous driving after switching from LV4 autonomous driving to LV3 autonomous driving. If the driver is not asleep during LV4 autonomous driving, the driver is able to grasp the situation around the vehicle. Therefore, the driver can grasp the situation around the vehicle without increasing the amount of driving-related information displayed on the display 91b before switching to LV3 autonomous driving. Therefore, there is no problem with increasing the amount of driving-related information displayed on the display 91b after switching to LV3 autonomous driving.

[0144] On the other hand, if the state identification unit 107 identifies that the driver has transitioned from a sleep state to an awake state during LV4 autonomous driving, the display control unit 106b preferably changes the display of information to correspond to the level of LV3 autonomous driving after the switch from LV4 autonomous driving to LV3 autonomous driving before switching from LV4 autonomous driving to LV3 autonomous driving. If the driver falls asleep during LV4 autonomous driving, there is a possibility that the driver may not be able to grasp the situation around the vehicle. Therefore, by increasing the amount of driving-related information displayed on the display 91b before switching to LV3 autonomous driving, the driver can more easily grasp the situation around the vehicle. As a result, convenience for the driver is improved.

[0145] Here, an example of the flow of processing related to display control in the HCU 10b from sleep-enabling autonomous driving to sleep-disabling autonomous driving (hereinafter referred to as second display control processing) will be described using the flowchart of Fig. 19. The flowchart of Fig. 19 may be configured to be started, for example, when the host vehicle starts autonomous driving at level 4 or higher.

[0146] First, in step S41, the state identification unit 107 identifies the state of the driver. In step S42, if the driver is identified as asleep in S41 (YES in S42), the process proceeds to step S43. On the other hand, if the driver is identified as awake in S41 (NO in S42), the process proceeds to step S44.

[0147] In step S43, display control unit 106b increases the amount of driving-related information displayed on passenger side display 912 compared to driver side display 911, and then proceeds to step S45. On the other hand, in step S44, display control unit 106b displays the same amount of driving-related information on driver side display 911 and passenger side display 912, and then proceeds to step S45.

[0148] In step S45, if switching to LV3 autonomous driving is to occur (YES in S45), the process proceeds to step S46. On the other hand, if switching to LV3 autonomous driving is not to occur (NO in S45), the process returns to S41 and is repeated. Switching to LV3 autonomous driving refers to a state in which switching is about to occur but has not yet started. LV3 autonomous driving is an autonomous driving that does not allow sleep, so when switching to LV3 autonomous driving occurs, the driver is assumed to be awake.

[0149] In step S46, if the driver has been determined to be asleep in S41 (YES in S46), the process proceeds to step S47. On the other hand, if the driver has not been determined to be asleep in S41 (NO in S46), the process proceeds to step S48.

[0150] In step S47, the display control unit 106b changes the display of information according to the stage of LV3 autonomous driving after the switchover to LV3 autonomous driving before the switchover to LV3 autonomous driving, and ends the second display control-related processing. On the other hand, in step S48, the display control unit 106b changes the display of information according to the stage of LV3 autonomous driving after the switchover to LV3 autonomous driving, and ends the second display control-related processing.

[0151] (Embodiment 7) The configuration is not limited to that of the sixth embodiment, and may be that of the seventh embodiment, as exemplified below. An example of the seventh embodiment will be described below with reference to the drawings. The vehicle system 1b of the seventh embodiment is similar to the vehicle system 1b of the sixth embodiment, except that it includes an HCU 10c instead of the HCU 10b.

[0152] Here, the schematic configuration of the HCU 10c will be described with reference to Fig. 20. As shown in Fig. 20, the HCU 10c includes, as functional blocks, a changeover request acquisition unit 101, a mode identification unit 102, an interruption estimation unit 103, a lane change identification unit 104, a grip identification unit 105, a display control unit 106c, and a state identification unit 107 for controlling the display on the display device 91b. The HCU 10c is similar to the HCU 10b of the sixth embodiment, except that the HCU 10c includes a display control unit 106c instead of the display control unit 106b. This HCU 10c also corresponds to a vehicle display control device. Execution of the processing of each functional block of the HCU 10c by a computer also corresponds to execution of a vehicle display control method.

[0153] The display control unit 106c is similar to the display control unit 106b except for some differences in processing. The following describes processing that differs from that of the display control unit 106b. When switching from sleep-enabling autonomous driving to sleep-preventing autonomous driving, if the state identification unit 107 identifies the driver as being in an awake state before a predetermined time before the planned switching time, the display control unit 106c changes the display of information to correspond to the level of autonomous driving after the switch from sleep-enabling autonomous driving to sleep-preventing autonomous driving. On the other hand, if the state identification unit 107 identifies that the driver has transitioned from a sleep state to an awake state within a predetermined time before the planned switching time, the display control unit 106c changes the display of information to correspond to the level of autonomous driving after the switch from sleep-enabling autonomous driving to sleep-preventing autonomous driving. As described above, sleep-enabling autonomous driving is autonomous driving of level 4 or higher. The following description will be given using level 4 autonomous driving as an example. As described above, sleep-preventing autonomous driving is autonomous driving of level 3. The predetermined time may be set arbitrarily as long as it is equal to or longer than the estimated time required for the driver to recognize the surroundings of the vehicle after transitioning from a sleep state to an awake state.

[0154] In the seventh embodiment, the process of S46 in the flowchart of Fig. 19 may be modified as follows: In the seventh embodiment, if the state identification unit 107 has continuously identified the driver as awake from a predetermined time before the scheduled timing of switching to LV3 autonomous driving in the process of S46, the process may proceed to step S47. On the other hand, if the state identification unit 107 has identified the driver as asleep within the predetermined time before the scheduled timing of switching to LV3 autonomous driving, the process may proceed to step S48.

[0155] (Embodiment 8) In the sixth and seventh embodiments, the HCUs 10b and 10c are provided with the state identification unit 107, but this is not necessarily limited to this. For example, the HCUs 10b and 10c may not be provided with the state identification unit 107, and the display may not be controlled depending on whether the driver is awake or asleep.

[0156] The present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. The technical scope of the present disclosure also includes embodiments obtained by appropriately combining the technical means disclosed in different embodiments. The control unit and method described in the present disclosure may be implemented by a special-purpose computer comprising a processor programmed to execute one or more functions embodied in a computer program. Alternatively, the apparatus and method described in the present disclosure may be implemented by a special-purpose hardware logic circuit. Alternatively, the apparatus and method described in the present disclosure may be implemented by one or more special-purpose computers configured by combining a processor executing a computer program with one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by a computer on a computer-readable non-transitory tangible recording medium. [Explanation of symbols]

[0157] 1, 1b Vehicle system (vehicle display control system), 10, 10a, 10b, 10c HCU (vehicle display control device), 91, 91b Display, 93 User input device, 102 Mode identification unit, 103 Interruption estimation unit, 104 Lane change identification unit, 105 Grasp identification unit, 106, 106a, 106b, 106c Display control unit, 107 State identification unit, 911 Driver's side display, 912 Passenger's side display

Claims

1. a display control unit (106, 106a, 106b, 106c) that displays a surrounding situation image, which is an image for showing a surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification unit (102) that identifies whether the vehicle is to be executed in a supervising autonomous driving mode in which the driver is obligated to monitor the vehicle, that is, a hands-on mode in which the driver needs to hold the steering wheel of the vehicle, or a hands-off mode in which the driver does not need to hold the steering wheel, The display control unit changes the display of the surrounding situation image depending on whether the mode identification unit has identified the vehicle as being in the hands-on mode or the hands-off mode, The surrounding situation image includes an image of a lane, The display control unit is a vehicle display control device that, when the mode identification unit identifies the autonomous driving mode as the hands-on mode, displays the vehicle's own lane, which is the lane the vehicle is traveling in, and surrounding lanes other than the own lane, and, when the mode identification unit identifies the autonomous driving mode as the hands-off mode, displays only the vehicle's own lane out of the vehicle's own lane and the surrounding lanes.

2. In claim 1, The surrounding situation image includes an image showing an obstacle, When the mode identification unit identifies the autonomous driving mode as hands-off mode, the display control unit displays only the own lane out of the own lane and the surrounding lanes, while being able to display both an image showing an obstacle corresponding to the own lane and an image showing an obstacle corresponding to the surrounding lanes.

3. In claim 1 or 2, The surrounding situation image is an image of the surroundings of the vehicle viewed from a virtual viewpoint, When the mode identification unit identifies the autonomous driving mode as the hands-on mode, the display control unit displays the surrounding situation image viewed from the virtual viewpoint that is farther away from the display target in the surrounding situation image than when the mode identification unit identifies the autonomous driving mode as the hands-off mode, and when the mode identification unit identifies the autonomous driving mode as the hands-off mode, the display control unit displays the surrounding situation image viewed from the virtual viewpoint that is closer to the display target than when the mode identification unit identifies the autonomous driving mode as the hands-on mode.

4. a display control unit (106, 106a, 106b, 106c) that displays a surrounding situation image, which is an image for showing a surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification unit (102) that identifies whether the vehicle is to be executed in a supervising autonomous driving mode in which the driver is obligated to monitor the vehicle, that is, a hands-on mode in which the driver needs to hold the steering wheel of the vehicle, or a hands-off mode in which the driver does not need to hold the steering wheel, The display control unit changes the display of the surrounding situation image depending on whether the mode identification unit has identified the vehicle as being in the hands-on mode or the hands-off mode, The surrounding situation image is an image of the surroundings of the vehicle viewed from a virtual viewpoint, When the mode identification unit identifies the autonomous driving mode as the hands-on mode, the display control unit displays the surrounding situation image viewed from the virtual viewpoint that is farther away from the display target in the surrounding situation image than when the mode identification unit identifies the autonomous driving mode as the hands-off mode, and when the mode identification unit identifies the autonomous driving mode as the hands-off mode, the display control unit displays the surrounding situation image viewed from the virtual viewpoint that is closer to the display target than when the mode identification unit identifies the autonomous driving mode as the hands-on mode.

5. In any one of claims 1 to 4, The surrounding situation image is an image of the surroundings of the vehicle viewed from a virtual viewpoint, When the mode identification unit identifies the autonomous driving mode as the hands-on mode, the display control unit displays the surrounding situation image seen from the virtual viewpoint looking down from above more closely than when the mode identification unit identifies the autonomous driving mode as the hands-off mode, and when the mode identification unit identifies the autonomous driving mode as the hands-off mode, the display control unit displays the surrounding situation image seen from the virtual viewpoint looking down from below more closely than when the mode identification unit identifies the autonomous driving mode as the hands-on mode.

6. a display control unit (106, 106a, 106b, 106c) that displays a surrounding situation image, which is an image for showing a surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification unit (102) that identifies whether the vehicle is to be executed in a supervising autonomous driving mode in which the driver is obligated to monitor the vehicle, that is, a hands-on mode in which the driver needs to hold the steering wheel of the vehicle, or a hands-off mode in which the driver does not need to hold the steering wheel, The display control unit changes the display of the surrounding situation image depending on whether the mode identification unit has identified the vehicle as being in the hands-on mode or the hands-off mode, The surrounding situation image is an image of the surroundings of the vehicle viewed from a virtual viewpoint, When the mode identification unit identifies the autonomous driving mode as the hands-on mode, the display control unit displays the surrounding situation image seen from the virtual viewpoint looking down from above more closely than when the mode identification unit identifies the autonomous driving mode as the hands-off mode, and when the mode identification unit identifies the autonomous driving mode as the hands-off mode, the display control unit displays the surrounding situation image seen from the virtual viewpoint looking down from below more closely than when the mode identification unit identifies the autonomous driving mode as the hands-on mode.

7. In any one of claims 1 to 6, When the mode identification unit identifies the autonomous driving mode as the hands-on mode, the display control unit widens the area around the vehicle to be displayed as the surrounding situation image compared to when the mode identification unit identifies the autonomous driving mode as the hands-off mode, and when the mode identification unit identifies the autonomous driving mode as the hands-off mode, the display control unit narrows the area around the vehicle to be displayed as the surrounding situation image compared to when the mode identification unit identifies the autonomous driving mode as the hands-on mode.

8. a display control unit (106, 106a, 106b, 106c) that displays a surrounding situation image, which is an image for showing a surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification unit (102) that identifies whether the vehicle is to be executed in a supervising autonomous driving mode in which the driver is obligated to monitor the vehicle, that is, a hands-on mode in which the driver needs to hold the steering wheel of the vehicle, or a hands-off mode in which the driver does not need to hold the steering wheel, The display control unit changes the display of the surrounding situation image depending on whether the mode identification unit has identified the vehicle as being in the hands-on mode or the hands-off mode, When the mode identification unit identifies the autonomous driving mode as the hands-on mode, the display control unit widens the area around the vehicle to be displayed as the surrounding situation image compared to when the mode identification unit identifies the autonomous driving mode as the hands-off mode, and when the mode identification unit identifies the autonomous driving mode as the hands-off mode, the display control unit narrows the area around the vehicle to be displayed as the surrounding situation image compared to when the mode identification unit identifies the autonomous driving mode as the hands-on mode.

9. In any one of claims 1 to 8, The display control unit is a display control device for a vehicle that changes the color tone of at least a portion of the surrounding situation image depending on whether the mode identification unit has identified the autonomous driving as the hands-on mode or the hands-off mode.

10. In any one of claims 1 to 9, The surroundings image includes a plurality of image elements, The display control unit is a display control device for a vehicle that changes at least one of the arrangement and size ratio of the image elements depending on whether the mode identification unit identifies the autonomous driving as the hands-on mode or the hands-off mode.

11. In claim 10, The surrounding situation image includes, as one of the image elements, a hands-on / off image that is an image indicating whether the vehicle is in the hands-on mode or the hands-off mode, The display control unit is a vehicle display control device that, when the mode identification unit identifies the autonomous driving mode as the hands-on mode, increases the size ratio of the hands-on / off image compared to when the mode identification unit identifies the autonomous driving mode as the hands-off mode.

12. In any one of claims 1 to 11, The surrounding situation image also includes an image of a background, The display control unit is a display control device for a vehicle that changes the background image of the surrounding situation image depending on whether the mode identification unit has identified the autonomous driving as the hands-on mode or the hands-off mode.

13. In any one of claims 1 to 12, The display control unit is a vehicle display control device that switches the display to the surrounding situation image when the mode identification unit identifies the automatic driving in the hands-on mode, even if the automatic driving in the hands-off mode is continued, in at least one of the following cases: when the vehicle changes lanes by automatic driving when the vehicle has switched to the automatic driving in the hands-off mode, and when it is estimated that a vehicle surrounding the vehicle will cut into the lane in which the vehicle is traveling.

14. a display control unit (106, 106a, 106b, 106c) that displays a surrounding situation image, which is an image for showing a surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification unit (102) that identifies whether the vehicle is to be executed in a supervising autonomous driving mode in which the driver is obligated to monitor the vehicle, that is, a hands-on mode in which the driver needs to hold the steering wheel of the vehicle, or a hands-off mode in which the driver does not need to hold the steering wheel, The display control unit changes the display of the surrounding situation image depending on whether the mode identification unit has identified the vehicle as being in the hands-on mode or the hands-off mode, The display control unit is a vehicle display control device that switches the display to the surrounding situation image when the mode identification unit identifies the automatic driving in the hands-on mode, even if the automatic driving in the hands-off mode is continued, in at least one of the following cases: when the vehicle changes lanes by automatic driving when the vehicle has switched to the automatic driving in the hands-off mode, and when it is estimated that a vehicle surrounding the vehicle will cut into the lane in which the vehicle is traveling.

15. In any one of claims 1 to 14, The display control unit is a vehicle display control device that, when the vehicle has switched to autonomous driving in the hands-off mode and the elapsed time since this switch reaches a specified time, switches the display to the surrounding situation image that is displayed when the mode identification unit has identified autonomous driving in the hands-on mode, even if autonomous driving in the hands-off mode continues.

16. a display control unit (106, 106a, 106b, 106c) that displays a surrounding situation image, which is an image for showing a surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification unit (102) that identifies whether the vehicle is to be executed in a supervising autonomous driving mode in which the driver is obligated to monitor the vehicle, that is, a hands-on mode in which the driver needs to hold the steering wheel of the vehicle, or a hands-off mode in which the driver does not need to hold the steering wheel, The display control unit changes the display of the surrounding situation image depending on whether the mode identification unit has identified the vehicle as being in the hands-on mode or the hands-off mode, The display control unit is a vehicle display control device that, when the vehicle has switched to autonomous driving in the hands-off mode and the elapsed time since this switch reaches a specified time, switches the display to the surrounding situation image that is displayed when the mode identification unit has identified autonomous driving in the hands-on mode, even if autonomous driving in the hands-off mode continues.

17. In any one of claims 1 to 16, a gripping identification unit (105) that identifies the driver's gripping of the steering wheel, The display control unit is a vehicle display control device that, when the vehicle has switched to autonomous driving in the hands-off mode and the grip identification unit identifies that the driver is gripping the steering wheel, switches the display to the surrounding situation image that would be displayed if the mode identification unit identified autonomous driving in the hands-on mode, even if autonomous driving in the hands-off mode continues.

18. a display control unit (106, 106a, 106b, 106c) that displays a surrounding situation image, which is an image for showing a surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification unit (102) that identifies whether the vehicle is to be executed in a supervising autonomous driving mode in which the driver is obligated to monitor the vehicle, that is, a hands-on mode in which the driver needs to hold the steering wheel of the vehicle, or a hands-off mode in which the driver does not need to hold the steering wheel, The display control unit changes the display of the surrounding situation image depending on whether the mode identification unit has identified the vehicle as being in the hands-on mode or the hands-off mode, a gripping identification unit (105) that identifies the driver's gripping of the steering wheel, The display control unit is a vehicle display control device that, when the vehicle has switched to autonomous driving in the hands-off mode and the grip identification unit identifies that the driver is gripping the steering wheel, switches the display to the surrounding situation image that would be displayed if the mode identification unit identified autonomous driving in the hands-on mode, even if autonomous driving in the hands-off mode continues.

19. In any one of claims 1 to 16, a gripping identification unit (105) that identifies the driver's gripping of the steering wheel, When the grip identification unit identifies the driver's grip on the steering wheel when the vehicle has switched to autonomous driving in the hands-off mode, the display control unit continues to display the surrounding situation image for the case in which the mode identification unit identifies autonomous driving in the hands-off mode for a predetermined time after the grip identification unit identifies the driver's grip on the steering wheel, and then switches to displaying the surrounding situation image for the case in which the mode identification unit identifies autonomous driving in the hands-on mode, even if autonomous driving in the hands-off mode continues.

20. a display control unit (106, 106a, 106b, 106c) that displays a surrounding situation image, which is an image for showing a surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification unit (102) that identifies whether the vehicle is to be executed in a supervising autonomous driving mode in which the driver is obligated to monitor the vehicle, that is, a hands-on mode in which the driver needs to hold the steering wheel of the vehicle, or a hands-off mode in which the driver does not need to hold the steering wheel, The display control unit changes the display of the surrounding situation image depending on whether the mode identification unit has identified the vehicle as being in the hands-on mode or the hands-off mode, a gripping identification unit (105) that identifies the driver's gripping of the steering wheel, When the grip identification unit identifies the driver's grip on the steering wheel when the vehicle has switched to autonomous driving in the hands-off mode, the display control unit continues to display the surrounding situation image for the case in which the mode identification unit identifies autonomous driving in the hands-off mode for a predetermined time after the grip identification unit identifies the driver's grip on the steering wheel, and then switches to displaying the surrounding situation image for the case in which the mode identification unit identifies autonomous driving in the hands-on mode, even if autonomous driving in the hands-off mode continues.

21. In any one of claims 1 to 20, The display control unit (106a) is a vehicle display control device that, when the autonomous driving level switches to a lower level of automation while the surrounding situation image is not being displayed during autonomous driving without a monitoring obligation, which is autonomous driving without the driver's monitoring obligation, changes the display of the surrounding situation image to one that corresponds to the autonomous driving level after the switch, from either simultaneously with or before the switch of the autonomous driving level, regardless of whether the autonomous driving is in the hands-on mode or the hands-off mode.

22. In any one of claims 1 to 21, The vehicle is capable of switching between a no-monitoring-obligation autonomous driving, which is an autonomous driving without a driver's monitoring obligation, and a monitoring-obligation autonomous driving, as the stage of the autonomous driving, and is used in a vehicle that is capable of at least a sleep-enabled autonomous driving, in which the driver is permitted to sleep, and a sleep-disallowed autonomous driving, in which the driver is not permitted to sleep, as the no-monitoring-obligation autonomous driving, the display control unit (106b, 106c) causes the display device to display travel-related information relating to the travel of the vehicle, The display control unit is a vehicle display control device that, when switching from sleep-enabling autonomous driving to sleep-disabling autonomous driving, increases the amount of driving-related information displayed on the display when sleep-disabling autonomous driving is in progress compared to the amount of driving-related information displayed on the display when sleep-enabling autonomous driving is in progress.

23. a display control unit (106, 106a, 106b, 106c) that displays a surrounding situation image, which is an image for showing a surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification unit (102) that identifies whether the vehicle is to be executed in a supervising autonomous driving mode in which the driver is obligated to monitor the vehicle, that is, a hands-on mode in which the driver needs to hold the steering wheel of the vehicle, or a hands-off mode in which the driver does not need to hold the steering wheel, The display control unit changes the display of the surrounding situation image depending on whether the mode identification unit has identified the vehicle as being in the hands-on mode or the hands-off mode, The vehicle is capable of switching between a no-monitoring-obligation autonomous driving, which is an autonomous driving without a driver's monitoring obligation, and a monitoring-obligation autonomous driving, as the stage of the autonomous driving, and is used in a vehicle that is capable of at least a sleep-enabled autonomous driving, in which the driver is permitted to sleep, and a sleep-disallowed autonomous driving, in which the driver is not permitted to sleep, as the no-monitoring-obligation autonomous driving, the display control unit (106b, 106c) causes the display device to display travel-related information relating to the travel of the vehicle, The display control unit is a vehicle display control device that, when switching from sleep-enabling autonomous driving to sleep-disabling autonomous driving, increases the amount of driving-related information displayed on the display when sleep-disabling autonomous driving is in progress compared to the amount of driving-related information displayed on the display when sleep-enabling autonomous driving is in progress.

24. In claim 22 or 23, A state identification unit (107) is provided to identify the state of the driver, The display control unit (106b) changes the display of information to correspond to the stage of autonomous driving after switching from sleep-enabling autonomous driving to sleep-disabling autonomous driving when the state identification unit identifies that the driver is not asleep during the sleep-enabling autonomous driving, and changes the display of information to correspond to the stage of autonomous driving after switching when the state identification unit identifies that the driver has transitioned from a sleep state to an awake state during the sleep-enabling autonomous driving, before switching from sleep-enabling autonomous driving to sleep-disabling autonomous driving.

25. In claim 22 or 23, A state identification unit (107) is provided to identify the state of the driver, The display control unit (106c) is a vehicle display control device that, when switching from sleep-enabling autonomous driving to sleep-disabling autonomous driving, if the state identification unit has identified the driver as being in an awake state before a predetermined time before the scheduled switching time, changes the display of information to correspond to the stage of autonomous driving after the switch from sleep-enabling autonomous driving to sleep-disabling autonomous driving, and, if the state identification unit has identified that the driver has transitioned from a sleep state to an awake state within a predetermined time before the scheduled switching time, changes the display of information to correspond to the stage of autonomous driving after the switch before switching from sleep-enabling autonomous driving to sleep-disabling autonomous driving.

26. In any one of claims 1 to 25, The vehicle is capable of switching between a no-monitoring-obligation autonomous driving, which is an autonomous driving without a driver's monitoring obligation, and a monitoring-obligation autonomous driving, as the stage of the autonomous driving, and is used in a vehicle that is capable of at least a sleep-enabled autonomous driving, in which the driver is permitted to sleep, and a sleep-disallowed autonomous driving, in which the driver is not permitted to sleep, as the no-monitoring-obligation autonomous driving, the display control unit (106b, 106c) causes the display device to display travel-related information relating to the travel of the vehicle, A state identification unit (107) is provided to identify the state of the driver, The display control unit is a vehicle display control device that, during sleep-enabled automatic driving, increases the amount of driving-related information displayed on the display when the state identification unit identifies the driver as asleep compared to the amount of driving-related information displayed on the display when the state identification unit identifies the driver as awake.

27. a display control unit (106, 106a, 106b, 106c) that displays a surrounding situation image, which is an image for showing a surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification unit (102) that identifies whether the vehicle is to be executed in a supervising autonomous driving mode in which the driver is obligated to monitor the vehicle, that is, a hands-on mode in which the driver needs to hold the steering wheel of the vehicle, or a hands-off mode in which the driver does not need to hold the steering wheel, The display control unit changes the display of the surrounding situation image depending on whether the mode identification unit has identified the vehicle as being in the hands-on mode or the hands-off mode, The vehicle is capable of switching between a no-monitoring-obligation autonomous driving, which is an autonomous driving without a driver's monitoring obligation, and a monitoring-obligation autonomous driving, as the stage of the autonomous driving, and is used in a vehicle that is capable of at least a sleep-enabled autonomous driving, in which the driver is permitted to sleep, and a sleep-disallowed autonomous driving, in which the driver is not permitted to sleep, as the no-monitoring-obligation autonomous driving, the display control unit (106b, 106c) causes the display device to display travel-related information relating to the travel of the vehicle, A state identification unit (107) is provided to identify the state of the driver, The display control unit is a vehicle display control device that, during sleep-enabled automatic driving, increases the amount of driving-related information displayed on the display when the state identification unit identifies the driver as asleep compared to the amount of driving-related information displayed on the display when the state identification unit identifies the driver as awake.

28. In claim 26 or 27, The display control unit (106b, 106c) displays information on the display unit (91b), and controls the display on the driver's side display unit (911) whose display surface is located in front of the driver's seat of the vehicle, and the passenger's side display unit (912) whose display surface is located in a place other than the driver's side display unit that is visible to passengers in the vehicle. The display control unit is a vehicle display control device that, when the state identification unit identifies the driver as asleep during sleep-enabled automatic driving, increases the amount of driving-related information displayed on the passenger side display compared to the driver side display when the state identification unit identifies the driver as awake.

29. In any one of claims 1 to 28, The present invention is used in a vehicle that is capable of switching between automated driving without a monitoring obligation, which is automated driving without a driver's monitoring obligation, and the automated driving with a monitoring obligation, and is capable of at least sleep-enabled automated driving, in which the driver is permitted to sleep, and sleep-disallowed automated driving, in which the driver is not permitted to sleep, as the automated driving without a monitoring obligation, the display control unit (106b, 106c) causes the display device to display travel-related information relating to the travel of the vehicle, The display control unit is a vehicle display control device that, when automation switches from the sleep-enabled autonomous driving to a level of driving lower than the autonomous driving with monitoring obligation, increases the amount of driving-related information displayed on the display after switching to the level of driving lower than the autonomous driving with monitoring obligation when the automation switches from the sleep-enabled autonomous driving to a level of driving lower than the autonomous driving with monitoring obligation.

30. a display control unit (106, 106a, 106b, 106c) that displays a surrounding situation image, which is an image for showing a surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification unit (102) that identifies whether the vehicle is to be executed in a supervising autonomous driving mode in which the driver is obligated to monitor the vehicle, that is, a hands-on mode in which the driver needs to hold the steering wheel of the vehicle, or a hands-off mode in which the driver does not need to hold the steering wheel, The display control unit changes the display of the surrounding situation image depending on whether the mode identification unit has identified the vehicle as being in the hands-on mode or the hands-off mode, The present invention is used in a vehicle that is capable of switching between automated driving without a monitoring obligation, which is automated driving without a driver's monitoring obligation, and the automated driving with a monitoring obligation, and is capable of at least sleep-enabled automated driving, in which the driver is permitted to sleep, and sleep-disallowed automated driving, in which the driver is not permitted to sleep, as the automated driving without a monitoring obligation, the display control unit (106b, 106c) causes the display device to display travel-related information relating to the travel of the vehicle, The display control unit is a vehicle display control device that, when automation switches from the sleep-enabled autonomous driving to a level of driving lower than the autonomous driving with monitoring obligation, increases the amount of driving-related information displayed on the display after switching to the level of driving lower than the autonomous driving with monitoring obligation when the automation switches from the sleep-enabled autonomous driving to a level of driving lower than the autonomous driving with monitoring obligation.

31. a display (91, 91b) provided in the vehicle so that a display surface faces the interior of the vehicle; A vehicle display control system including the vehicle display control device (10, 10a, 10b, 10c) according to any one of claims 1 to 30.

32. Executed by at least one processor, a display control step of displaying a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification step of identifying whether the vehicle is to be executed in a hands-on mode of autonomous driving that requires the driver to hold the steering wheel of the vehicle, or in a hands-off mode of autonomous driving that does not require the driver to hold the steering wheel, in a supervised autonomous driving with supervision obligation, In the display control step, the display of the surrounding situation image is changed depending on whether the vehicle has been identified as being in the hands-on mode of autonomous driving or the hands-off mode of autonomous driving in the mode identification step, The surrounding situation image includes an image of a lane, In the display control process, if the mode identification process identifies the autonomous driving mode as hands-on mode, the vehicle's own lane, which is the lane the vehicle is traveling in, and surrounding lanes other than the own lane are displayed, while if the mode identification process identifies the autonomous driving mode as hands-off mode, only the own lane of the own lane and the surrounding lanes is displayed.

33. Executed by at least one processor, a display control step of displaying a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification step of identifying whether the vehicle is to be executed in a hands-on mode of autonomous driving that requires the driver to hold the steering wheel of the vehicle, or in a hands-off mode of autonomous driving that does not require the driver to hold the steering wheel, in a supervised autonomous driving with supervision obligation, In the display control step, the display of the surrounding situation image is changed depending on whether the vehicle has been identified as being in the hands-on mode of autonomous driving or the hands-off mode of autonomous driving in the mode identification step, The surrounding situation image is an image of the surroundings of the vehicle viewed from a virtual viewpoint, A display control method for a vehicle, wherein in the display control process, if the mode identification process identifies the autonomous driving as the hands-on mode, the surrounding situation image viewed from the virtual viewpoint farther away from the display target in the surrounding situation image is displayed than when the mode identification process identifies the autonomous driving as the hands-off mode, and, if the mode identification process identifies the autonomous driving as the hands-off mode, the surrounding situation image viewed from the virtual viewpoint closer to the display target is displayed than when the mode identification process identifies the autonomous driving as the hands-on mode.

34. Executed by at least one processor, a display control step of displaying a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification step of identifying whether the vehicle is to be executed in a hands-on mode of autonomous driving that requires the driver to hold the steering wheel of the vehicle, or in a hands-off mode of autonomous driving that does not require the driver to hold the steering wheel, in a supervised autonomous driving with supervision obligation, In the display control step, the display of the surrounding situation image is changed depending on whether the vehicle has been identified as being in the hands-on mode of autonomous driving or the hands-off mode of autonomous driving in the mode identification step, The surrounding situation image is an image of the surroundings of the vehicle viewed from a virtual viewpoint, In the display control process, if the mode identification process identifies the autonomous driving as the hands-on mode, the surrounding situation image seen from the virtual viewpoint looking down from above is displayed more clearly than when the mode identification process identifies the autonomous driving as the hands-off mode, and if the mode identification process identifies the autonomous driving as the hands-off mode, the surrounding situation image seen from the virtual viewpoint looking down from below is displayed more clearly than when the mode identification process identifies the autonomous driving as the hands-on mode.

35. Executed by at least one processor, a display control step of displaying a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification step of identifying whether the vehicle is to be executed in a hands-on mode of autonomous driving that requires the driver to hold the steering wheel of the vehicle, or in a hands-off mode of autonomous driving that does not require the driver to hold the steering wheel, in a supervised autonomous driving with supervision obligation, In the display control step, the display of the surrounding situation image is changed depending on whether the vehicle has been identified as being in the hands-on mode of autonomous driving or the hands-off mode of autonomous driving in the mode identification step, A display control method for a vehicle, wherein in the display control process, if the mode identification process identifies the autonomous driving as the hands-on mode, the area around the vehicle to be displayed as the surrounding situation image is made wider than when the mode identification process identifies the autonomous driving as the hands-off mode, and, if the mode identification process identifies the autonomous driving as the hands-off mode, the area around the vehicle to be displayed as the surrounding situation image is made narrower than when the mode identification process identifies the autonomous driving as the hands-on mode.

36. Executed by at least one processor, a display control step of displaying a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification step of identifying whether the vehicle is to be executed in a hands-on mode of autonomous driving that requires the driver to hold the steering wheel of the vehicle, or in a hands-off mode of autonomous driving that does not require the driver to hold the steering wheel, in a supervised autonomous driving with supervision obligation, In the display control step, the display of the surrounding situation image is changed depending on whether the vehicle has been identified as being in the hands-on mode of autonomous driving or the hands-off mode of autonomous driving in the mode identification step, In the display control process, when the vehicle has switched to the hands-off mode of autonomous driving, if the vehicle changes lanes by autonomous driving, or if it is estimated that a vehicle surrounding the vehicle will cut into the vehicle's driving lane, even if the hands-off mode of autonomous driving continues, the display control method for a vehicle switches to the display of the surrounding situation image when the mode identification process identifies the hands-on mode of autonomous driving.

37. Executed by at least one processor, a display control step of displaying a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification step of identifying whether the vehicle is to be executed in a hands-on mode of autonomous driving that requires the driver to hold the steering wheel of the vehicle, or in a hands-off mode of autonomous driving that does not require the driver to hold the steering wheel, in a supervised autonomous driving with supervision obligation, In the display control step, the display of the surrounding situation image is changed depending on whether the vehicle has been identified as being in the hands-on mode of autonomous driving or the hands-off mode of autonomous driving in the mode identification step, In the display control process, when the vehicle has switched to the hands-off mode of autonomous driving and the elapsed time since this switch reaches a specified time, even if the hands-off mode of autonomous driving continues, the display is switched to the surrounding situation image that was identified as the hands-on mode of autonomous driving in the mode identification process.

38. Executed by at least one processor, a display control step of displaying a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification step of identifying whether the vehicle is to be executed in a hands-on mode of autonomous driving that requires the driver to hold the steering wheel of the vehicle, or in a hands-off mode of autonomous driving that does not require the driver to hold the steering wheel, in a supervised autonomous driving with supervision obligation, In the display control step, the display of the surrounding situation image is changed depending on whether the vehicle has been identified as being in the hands-on mode of autonomous driving or the hands-off mode of autonomous driving in the mode identification step, a gripping identification step of identifying a gripping position of the steering wheel by the driver, In the display control process, when the vehicle has switched to the hands-off mode of autonomous driving and the grip identification process identifies the driver's grip on the steering wheel, the display control method for a vehicle switches to the display of the surrounding situation image when the mode identification process identifies the hands-on mode of autonomous driving, even if the hands-off mode of autonomous driving continues.

39. Executed by at least one processor, a display control step of displaying a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification step of identifying whether the vehicle is to be executed in a hands-on mode of autonomous driving that requires the driver to hold the steering wheel of the vehicle, or in a hands-off mode of autonomous driving that does not require the driver to hold the steering wheel, in a supervised autonomous driving with supervision obligation, In the display control step, the display of the surrounding situation image is changed depending on whether the vehicle has been identified as being in the hands-on mode of autonomous driving or the hands-off mode of autonomous driving in the mode identification step, a gripping identification step of identifying a gripping position of the steering wheel by the driver, In the display control process, when the vehicle has switched to automatic driving in the hands-off mode and the grip identification process identifies the driver's grip on the steering wheel, the display of the surrounding situation image for the case where the mode identification process identifies automatic driving in the hands-off mode is continued for a predetermined time after the driver's grip on the steering wheel is identified in the grip identification process, and then, even if automatic driving in the hands-off mode continues, the display of the surrounding situation image for the case where the mode identification process identifies automatic driving in the hands-on mode is switched to.

40. Executed by at least one processor, a display control step of displaying a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification step of identifying whether the vehicle is to be executed in a hands-on mode of autonomous driving that requires the driver to hold the steering wheel of the vehicle, or in a hands-off mode of autonomous driving that does not require the driver to hold the steering wheel, in a supervised autonomous driving with supervision obligation, In the display control step, the display of the surrounding situation image is changed depending on whether the vehicle has been identified as being in the hands-on mode of autonomous driving or the hands-off mode of autonomous driving in the mode identification step, The vehicle is capable of switching between a no-monitoring-obligation autonomous driving, which is an autonomous driving without a driver's monitoring obligation, and a monitoring-obligation autonomous driving, as the stage of the autonomous driving, and is used in a vehicle that is capable of at least a sleep-enabled autonomous driving, in which the driver is permitted to sleep, and a sleep-disallowed autonomous driving, in which the driver is not permitted to sleep, as the no-monitoring-obligation autonomous driving, The display control step displays travel-related information related to travel of the vehicle on the display device, In the display control process, when switching from sleep-enabling autonomous driving to sleep-disabling autonomous driving, the amount of driving-related information displayed on the display device during sleep-disabling autonomous driving is increased compared to the amount of driving-related information displayed on the display device during sleep-enabling autonomous driving.

41. Executed by at least one processor, a display control step of displaying a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification step of identifying whether the vehicle is to be executed in a hands-on mode of autonomous driving that requires the driver to hold the steering wheel of the vehicle, or in a hands-off mode of autonomous driving that does not require the driver to hold the steering wheel, in a supervised autonomous driving with supervision obligation, In the display control step, the display of the surrounding situation image is changed depending on whether the vehicle has been identified as being in the hands-on mode of autonomous driving or the hands-off mode of autonomous driving in the mode identification step, The vehicle is capable of switching between a no-monitoring-obligation autonomous driving, which is an autonomous driving without a driver's monitoring obligation, and a monitoring-obligation autonomous driving, as the stage of the autonomous driving, and is used in a vehicle that is capable of at least a sleep-enabled autonomous driving, in which the driver is permitted to sleep, and a sleep-disallowed autonomous driving, in which the driver is not permitted to sleep, as the no-monitoring-obligation autonomous driving, The display control step displays travel-related information related to travel of the vehicle on the display device, a state identification step of identifying the state of the driver, A display control method for a vehicle, in which the display control process increases the amount of driving-related information displayed on the display when the state identification process identifies the driver as being in a sleeping state, compared to the amount of driving-related information displayed on the display when the state identification process identifies the driver as being in an awake state, during sleep-enabled automatic driving.

42. Executed by at least one processor, a display control step of displaying a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; a mode identification step of identifying whether the vehicle is to be executed in a hands-on mode of autonomous driving that requires the driver to hold the steering wheel of the vehicle, or in a hands-off mode of autonomous driving that does not require the driver to hold the steering wheel, in a supervised autonomous driving with supervision obligation, In the display control step, the display of the surrounding situation image is changed depending on whether the vehicle has been identified as being in the hands-on mode of autonomous driving or the hands-off mode of autonomous driving in the mode identification step, The present invention is used in a vehicle that is capable of switching between automated driving without a monitoring obligation, which is automated driving without a driver's monitoring obligation, and the automated driving with a monitoring obligation, and is capable of at least sleep-enabled automated driving, in which the driver is permitted to sleep, and sleep-disallowed automated driving, in which the driver is not permitted to sleep, as the automated driving without a monitoring obligation, The display control step displays travel-related information related to travel of the vehicle on the display device, In the display control process, when automation switches from the sleep-enabled autonomous driving to a level of driving below the autonomous driving with monitoring obligation, after switching to the level of driving below the autonomous driving with monitoring obligation, the display control method for a vehicle increases the amount of driving-related information displayed on the display device compared to when the sleep-enabled autonomous driving is in progress.

Images