Vehicle display control device, vehicle display control system, and vehicle display control method

Abstract

To allow a driver to easily recognize whether an automatic driving after a switchover is a hands-on mode or a hands-off mode for the switchover from automatic driving without monitoring duty to automatic driving with monitoring duty.SOLUTION: A vehicle display control device includes: a display control unit 106 that displays images of the surrounding situation of an own vehicle on a display unit 91 used in a cabin of the vehicle; and a mode identification unit 102 that identifies whether the vehicle performs automatic driving in a hands-on mode, which requires grasping a steering wheel, or in a hands-off mode, which does not require grasping the steering wheel, in automatic driving with monitoring duty. When the own vehicle switches from automatic driving without monitoring duty to automatic driving with monitoring duty, the display control unit 106 displays different images of the surrounding situation depending on whether the own vehicle identified its driving as the hands-on mode or the hands-off mode by the mode identification unit 102.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a vehicle display control device, a vehicle display control system, and a vehicle display control method. [Background technology]

[0002] Patent Document 1 discloses a technology for gradually switching a vehicle from a manual driving mode to an autonomous driving mode. Patent Document 1 also discloses a technology for displaying the automation level when gradually switching from the manual driving mode to the autonomous driving mode by a notification indicator.

[0003] As the automation level, for example, the automation levels 0 to 5 defined by SAE are known. Level 0 is a level where the driver performs all driving tasks without intervention of the system. Level 0 corresponds to so-called manual driving. Level 1 is a level where the system assists with either steering or acceleration / deceleration. Level 2 is a level where the system assists with both steering and acceleration / deceleration. Autonomous driving at levels 1 to 2 is an automated driving where the driver has the duty to monitor (hereinafter simply the duty to monitor) for safe driving. Level 3 is a level where the system can perform all driving tasks in specific locations such as highways, and the driver performs driving operations in an emergency. Level 4 is a level where the system can perform all driving tasks except under specific circumstances such as unsupported roads and extreme environments. Level 5 is a level where the system can perform all driving tasks in any environment. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2015-24746 A Summary of the Invention [Problem to be solved by the invention]

[0005] Not only switching from a manual driving mode to an automatic driving mode as disclosed in Patent Document 1, but also switching to an automatic driving mode with a low automation level in the automatic driving mode is also conceivable. Here, when switching from an automatic driving mode of level 3 or higher with no monitoring obligation to an automatic driving mode of level 2 with monitoring obligation, it is conceivable that the tasks required of the driver may be different even if the automation level is the same. In detail, since the automatic driving of level 2 does not require the driver to operate the vehicle, it is conceivable that the mode may be a hands-on mode in which the driver is required to hold the steering wheel, or a hands-off mode in which the driver is not required to hold the steering wheel. In response to such a problem, in the configuration that displays the automation level as disclosed in Patent Document 1, it is not possible to make the driver recognize whether the automatic driving mode after the automation level is switched is a hands-on mode or a hands-off mode.

[0006] One objective of this disclosure is to provide a vehicle display control device, a vehicle display control system, and a vehicle display control method that, when switching from autonomous driving without a monitoring obligation to autonomous driving with a monitoring obligation, enables the driver to easily recognize whether the autonomous driving after the switch is in hands-on mode or hands-off mode. [Means for solving the problem]

[0007] The above object is achieved by the combination of features recited in the independent claims, and the subclaims define further advantageous embodiments of the disclosure. The reference characters in parentheses in the claims indicate a correspondence with specific means described in the following embodiment as one aspect, and do not limit the technical scope of the present disclosure.

[0008] In order to achieve the above object, the present disclosure CarThe dual-use display control device is used in a vehicle that can be switched from automatic driving without monitoring obligation, which is automatic driving without the driver's monitoring obligation, to automatic driving with monitoring obligation, which is automatic driving with the driver's monitoring obligation, and includes a display control unit (106, 106a, 106b, 106c) that displays a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display device (91, 91b) used in the cabin of the vehicle, and when the automatic driving stage is switched to a lower automation stage while the surrounding situation image is being displayed during automatic driving without monitoring obligation, the display control unit changes, after a predetermined time has elapsed since the switch of the automatic driving stage, the display of the surrounding situation image corresponding to the automatic driving stage before the switch to the surrounding situation image corresponding to the automatic driving stage after the switch 。

[0009] In order to achieve the above object, the present disclosure Car The dual-use display control method is a vehicle display control method used in a vehicle that can be switched from automatic driving without a monitoring obligation, which is automatic driving without a driver's monitoring obligation, to automatic driving with a monitoring obligation, which is automatic driving with a driver's monitoring obligation, and includes a display control step executed by at least one processor to display a surrounding situation image, which is an image for showing the surrounding situation of the vehicle, on a display (91, 91b) used in the cabin of the vehicle, and a mode identification step to identify whether the automatic driving performed by the vehicle in the automatic driving with a monitoring obligation is a hands-on mode automatic driving that requires the driver to hold the steering wheel of the vehicle, or a hands-off mode automatic driving that does not require the driver to hold the steering wheel, and in the display control step, when the automatic driving stage is switched to a lower automation stage while the surrounding situation image is being displayed during automatic driving without a monitoring obligation, after a predetermined time has elapsed since the switching of the automatic driving stage, the display of the surrounding situation image corresponding to the automatic driving stage before the switching is changed to the display of the surrounding situation image corresponding to the automatic driving stage after the switching. 。

[0010] According to the above configuration, the display of the surrounding situation image displayed on the display used in the vehicle cabin is changed depending on whether the autonomous driving without a monitoring obligation is switched to the autonomous driving of the hands-on mode or the autonomous driving of the hands-off mode of the autonomous driving with a monitoring obligation. Therefore, the driver of the vehicle can more easily recognize whether the autonomous driving is switched to the hands-on mode or the hands-off mode from the difference in the display of the surrounding situation image. As a result, when switching from the autonomous driving without a monitoring obligation to the autonomous driving with a monitoring obligation, it is possible to make the driver more easily recognize whether the autonomous driving after the switching is the hands-on mode or the hands-off mode.

[0011] In order to achieve the above object, the vehicle display control system of the present disclosure ,car The display device includes a display (91, 91b) provided in the vehicle with a display surface facing the interior of the vehicle, and the above-mentioned vehicle display control device (10, 10a, 10b, 10c).

[0012] According to this, since it includes the above-mentioned vehicle display control device, when switching from autonomous driving without a monitoring obligation to autonomous driving with a monitoring obligation, it becomes possible for the driver to more easily recognize whether the autonomous driving after the switch is in hands-on mode or hands-off mode. [Brief description of the drawings]

[0013] [Figure 1] 1 is a diagram showing an example of a schematic configuration of a vehicle system 1. FIG. [Diagram 2] FIG. 2 is a diagram illustrating an example of a schematic configuration of an HCU 10. [Diagram 3] FIG. 2 is a diagram for explaining an example of a surrounding situation image. [Figure 4] 10A and 10B are diagrams for explaining an example of a difference in the display manner of a surrounding situation image between a hands-on mode and a hands-off mode. [Diagram 5] 10A and 10B are diagrams for explaining an example of a difference in the display manner of a surrounding situation image between a hands-on mode and a hands-off mode. [Figure 6]10A and 10B are diagrams for explaining an example of a difference in the display manner of a surrounding situation image between a hands-on mode and a hands-off mode. [Figure 7] 10A and 10B are diagrams for explaining an example of a difference in the display manner of a surrounding situation image between a hands-on mode and a hands-off mode. [Figure 8] 10A and 10B are diagrams for explaining an example of a difference in the display manner of a surrounding situation image between a hands-on mode and a hands-off mode. [Figure 9] 10A and 10B are diagrams for explaining an example of a difference in the display manner of a surrounding situation image between a hands-on mode and a hands-off mode. [Figure 10] 10A and 10B are diagrams for explaining an example of a difference in the display manner of a surrounding situation image between a hands-on mode and a hands-off mode. [Figure 11] 10A and 10B are diagrams for explaining an example of a difference in the display manner of a surrounding situation image between a hands-on mode and a hands-off mode. [Figure 12] 11 is a flowchart showing an example of the flow of a first display control-related process in an HCU 10 according to the first embodiment. [Figure 13] 11 is a flowchart showing an example of the flow of a first display control-related process in an HCU 10 according to the second embodiment. [Figure 14] 10A and 10B are diagrams for explaining an example of a difference in the display manner of a surrounding situation image between a hands-on mode and a hands-off mode. [Figure 15] FIG. 2 is a diagram illustrating an example of a schematic configuration of an HCU 10a. [Figure 16] FIG. 13 is a diagram for explaining the difference in timing of switching the display depending on whether or not a surrounding situation image is displayed while the vehicle is autonomous driving at level 3 or higher. [Figure 17] FIG. 2 is a diagram showing an example of a schematic configuration of a vehicle system 1b. [Figure 18] FIG. 2 is a diagram illustrating an example of a schematic configuration of an HCU 10b. [Figure 19] 23 is a flowchart showing an example of the flow of a second display control-related process in an HCU 10b in the sixth embodiment. [Figure 20]FIG. 2 is a diagram illustrating an example of a schematic configuration of an HCU 10c. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] A number of embodiments for disclosure will be described with reference to the drawings. For convenience of description, in a number of embodiments, parts having the same functions as parts shown in the drawings used in the previous description may be given the same reference numerals and their description may be omitted. For parts given the same reference numerals, the description in other embodiments may be referred to.

[0015] (Embodiment 1) <Overall configuration of vehicle system 1> Hereinafter, a first embodiment of the present disclosure will be described with reference to the drawings. A vehicle system 1 shown in FIG. 1 is used in a vehicle capable of autonomous driving (hereinafter, autonomous driving vehicle). As shown in FIG. 1, the vehicle system 1 includes an HCU (Human Machine Interface Control Unit) 10, a communication module 20, a locator 30, a map database (hereinafter, map DB) 40, a vehicle state sensor 50, a surroundings monitoring sensor 60, a vehicle control ECU 70, an autonomous driving ECU 80, a display 91, a grip sensor 92, and a user input device 93. This vehicle system 1 corresponds to a vehicle display control system. A vehicle using the vehicle system 1 is not necessarily limited to an automobile, but the following description will be given taking an example of use in an automobile.

[0016] There are multiple levels of autonomous driving for autonomous vehicles (hereinafter referred to as "automation levels"), as defined by the SAE, for example. Automation levels are classified into levels 0 to 5, for example, as follows:

[0017] Level 0 is a level where the driver performs all driving tasks without the system intervening. The driving task may be rephrased as a dynamic driving task. The driving task may be, for example, steering, acceleration / deceleration, and surrounding monitoring. Level 0 corresponds to so-called manual driving. Level 1 is a level where the system assists with either steering or acceleration / deceleration. Level 1 corresponds to so-called driving assistance. Level 2 is a level where the system assists with both steering and acceleration / deceleration. Level 2 corresponds to so-called partial driving automation. Levels 1 and 2 are also considered to be part of autonomous driving.

[0018] For example, autonomous driving at levels 1 to 2 is autonomous driving in which the driver has a duty to monitor safe driving (hereinafter, simply referred to as the duty to monitor). The duty to monitor includes visual monitoring of the surroundings. Autonomous driving at levels 1 to 2 can be rephrased as autonomous driving in which a second task is not permitted. A second task is an act other than driving that is permitted for the driver and is a specific act that is specified in advance. A second task can also be rephrased as a secondary activity, other activity, etc. It is considered that a second task must not prevent the driver from responding to a request from the autonomous driving system to take over driving operations. As an example, actions such as watching content such as videos, operating a smartphone, reading, and eating are assumed to be second tasks.

[0019] Level 3 is a level where the system can perform all driving tasks in specific locations such as highways, and the driver takes over driving operations in an emergency. At level 3, the driver is required to be able to respond quickly when the system requests a handover of driving. This handover of driving can also be described as the transfer of the responsibility of monitoring the surroundings from the vehicle's system to the driver. Level 3 corresponds to so-called conditional driving automation. Level 4 is a level where the system can perform all driving tasks except under certain circumstances such as unmanageable roads and extreme environments. Level 4 corresponds to so-called high driving automation. Level 5 is a level where the system can perform all driving tasks in any environment. Level 5 corresponds to so-called full driving automation.

[0020] For example, autonomous driving at levels 3 to 5 is autonomous driving in which the driver does not have a monitoring obligation. Autonomous driving at levels 3 to 5 can be rephrased as autonomous driving in which a second task is permitted. Among autonomous driving at levels 3 to 5, autonomous driving at level 4 or higher is autonomous driving in which the driver is permitted to sleep (hereinafter, sleep-enabled autonomous driving). Among autonomous driving at levels 3 to 5, autonomous driving at level 3 is autonomous driving in which the driver is not permitted to sleep (hereinafter, sleep-disabled autonomous driving). In this embodiment, the presence or absence of a monitoring obligation is switched by switching between an automation level of level 3 or higher and an automation level of level 2 or lower. Therefore, when switching from an automation level of level 3 or higher to an automation level of level 2 or lower, the driver is required to monitor safe driving. On the other hand, the transfer of driving control to the driver may be configured to be required when switching from an automation level of level 2 or higher to an automation level of level 1 or lower, for example. In this embodiment, a case in which driving control is transferred to the driver when switching from an automation level of level 2 or higher to an automation level of level 1 or lower is taken as an example for explanation.

[0021] The autonomous vehicle of the present embodiment is assumed to be switchable in automation level. The automation level may be configured to be switchable only between some of levels 0 to 5. In the present embodiment, an example is given of a case in which an autonomous vehicle can switch between autonomous driving at automation level 3, autonomous driving at automation level 2, and autonomous driving or manual driving at automation level 1. In the present embodiment, for example, autonomous driving at automation level 3 is permitted only during traffic jams. In the present embodiment, autonomous driving at automation level 3 may be permitted only during traffic jams and when traveling on a specific road section such as an expressway or a motorway. In the following, an example is given of a case in which autonomous driving at automation level 3 is permitted only during traffic jams and when traveling on a specific road section such as an expressway or a motorway.

[0022] In addition, in this embodiment, the autonomous driving of the automation level 2 includes an autonomous driving of a hands-on mode that requires the driver to hold the steering wheel of the vehicle, and an autonomous driving of a hands-off mode that does not require the driver to hold the steering wheel of the vehicle. As an example, the hands-on mode and the hands-off mode may be used differently as follows. For example, when the switching from the automation level 3 to the automation level 2 is planned based on a situation that can be predicted in advance, the autonomous driving may be configured to switch to the hands-off mode. On the other hand, when the switching from the automation level 3 to the automation level 2 is unplanned (i.e., sudden) based on a situation that cannot be predicted in advance, the autonomous driving may be configured to switch to the hands-on mode. This is because, when the switching from the automation level 3 to the automation level 2 is sudden, it is highly likely that a relatively large vehicle behavior will occur, and it is considered that there is a high need for the driver to hold the steering wheel. Note that the autonomous driving of the automation level 1 corresponds to the autonomous driving of the hands-on mode.

[0023] In addition, the present invention is not limited to the above example, and may be configured to use the hands-on mode and the hands-off mode depending on whether or not the section has high-precision map data. For example, the hands-off mode may be used in a section where high-precision map data exists, while the hands-on mode may be used in a section where high-precision map data does not exist. High-precision map data will be described later. In addition, the hands-on mode and the hands-off mode may be used depending on whether or not the vehicle is approaching a specific point. For example, the hands-off mode may be used when the vehicle is not approaching a specific point, while the hands-on mode may be used when the vehicle is approaching a specific point. Whether or not the vehicle is approaching a specific point may be determined by whether or not the distance to the specific point is equal to or less than a predetermined value. Examples of the specific point include a toll booth in the above-mentioned specific road section, an exit in the above-mentioned specific road section, a junction, an intersection, a two-way traffic section, a point where the number of lanes is reduced, and the like. The specific point can also be rephrased as a point where it is estimated that the driver is more likely to need to grip the steering wheel.

[0024] The communication module 20 transmits and receives information to and from other vehicles via wireless communication. That is, it performs vehicle-to-vehicle communication. The communication module 20 may transmit and receive information to and from a roadside device installed on the roadside via wireless communication. That is, it may perform road-to-vehicle communication. When performing road-to-vehicle communication, the communication module 20 may receive information on surrounding vehicles transmitted from surrounding vehicles of the vehicle via the roadside device. Also, the communication module 20 may transmit and receive information to and from a center outside the vehicle via wireless communication. That is, it may perform wide-area communication. When performing wide-area communication, the communication module 20 may receive information on surrounding vehicles transmitted from surrounding vehicles of the vehicle via the center. In addition, when performing wide-area communication, the communication module 20 may receive traffic congestion information, weather information, and the like around the vehicle from the center.

[0025] Locator 30 includes a Global Navigation Satellite System (GNSS) receiver and an inertial sensor. The GNSS receiver receives positioning signals from a plurality of positioning satellites. The inertial sensor includes, for example, a gyro sensor and an acceleration sensor. Locator 30 sequentially measures the position of the vehicle (hereinafter, the vehicle position) on which locator 30 is mounted, by combining the positioning signals received by the GNSS receiver with the measurement results of the inertial sensor. The vehicle position is represented, for example, by latitude and longitude coordinates. Note that the vehicle position may also be measured using a travel distance calculated from a signal sequentially output from a vehicle speed sensor mounted on the vehicle.

[0026] The map DB 40 is a non-volatile memory and stores high-precision map data. The high-precision map data is map data with higher precision than map data used for route guidance in the navigation function. The map DB 40 may also store map data used for route guidance. The high-precision map data includes information that can be used for automatic driving, such as three-dimensional shape information of roads, information on the number of lanes, and information indicating the travel direction permitted for each lane. In addition, the high-precision map data may include node point information indicating the positions of both ends of road markings such as dividing lines. The locator 30 may be configured not to use a GNSS receiver by using the three-dimensional shape information of the road. For example, the locator 30 may be configured to specify the vehicle position using the three-dimensional shape information of the road and the detection results of a perimeter monitoring sensor 60 such as a LIDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging) or a perimeter monitoring camera that detects a point cloud of characteristic points of the road shape and structures. The three-dimensional shape information of the road may be generated based on a captured image by REM (Road Experience Management).

[0027] The communication module 20 may receive map data distributed from an external server, for example, by wide area communication, and store the data in the map DB 40. In this case, the map DB 40 may be a volatile memory, and the communication module 20 may successively acquire map data of an area corresponding to the vehicle position.

[0028] The vehicle state sensor 50 is a group of sensors for detecting various states of the vehicle. The vehicle state sensor 50 includes a vehicle speed sensor for detecting the vehicle speed, a steering sensor for detecting the steering angle, and the like. The vehicle state sensor 50 outputs the detected sensing information to an in-vehicle LAN. The sensing information detected by the vehicle state sensor 50 may be configured to be output to the in-vehicle LAN via an ECU mounted in the vehicle.

[0029] The perimeter monitoring sensor 60 monitors the environment around the vehicle. As an example, the perimeter monitoring sensor 60 detects obstacles around the vehicle, such as moving objects such as pedestrians and other vehicles, and stationary objects such as objects fallen on the road. In addition, the perimeter monitoring sensor 60 detects road markings such as lane markings around the vehicle. The perimeter monitoring sensor 60 is, for example, a perimeter monitoring camera that captures a predetermined range around the vehicle, a millimeter wave radar that transmits a search wave to a predetermined range around the vehicle, a sonar, a LIDAR, or other sensor. The perimeter monitoring camera sequentially outputs the captured images as sensing information to the automatic driving ECU 80. The sonar, millimeter wave radar, LIDAR, or other sensor that transmits a search wave sequentially outputs a scan result based on a received signal obtained when receiving a reflected wave reflected by an obstacle to the automatic driving ECU 80 as sensing information. The sensing information detected by the perimeter monitoring sensor 60 may be configured to be output to an in-vehicle LAN via the automatic driving ECU 80.

[0030] The vehicle control ECU 70 is an electronic control device that performs driving control of the vehicle. Examples of driving control include acceleration / deceleration control and / or steering control. The vehicle control ECU 70 includes a steering ECU that performs steering control, a power unit control ECU that performs acceleration / deceleration control, and a brake ECU. The vehicle control ECU 70 performs driving control by outputting control signals to each driving control device mounted on the vehicle, such as an electronically controlled throttle, a brake actuator, and an EPS (Electric Power Steering) motor.

[0031] The autonomous driving ECU 80 includes, for example, a processor, a memory, an I / O, and a bus connecting these, and executes a control program stored in the memory to execute processes related to autonomous driving. The memory referred to here is a non-transitory tangible storage medium that non-temporarily stores computer-readable programs and data. The non-transitory tangible storage medium is realized by a semiconductor memory, a magnetic disk, or the like.

[0032] The autonomous driving ECU 80 includes a first autonomous driving ECU 81 and a second autonomous driving ECU 82. The following description will be given assuming that the first autonomous driving ECU 81 and the second autonomous driving ECU 82 each include a processor, memory, I / O, and a bus connecting these. Note that a configuration may be adopted in which a common processor performs the functions of the first autonomous driving ECU 81 and the second autonomous driving ECU 82 using virtualization technology.

[0033] The first automatic driving ECU81 is responsible for the automatic driving function of the level 2 or lower mentioned above. In other words, the first automatic driving ECU81 enables automatic driving with a monitoring obligation to be performed. For example, the first automatic driving ECU81 can execute at least one of longitudinal control and lateral control of the host vehicle. The longitudinal direction is the direction that coincides with the front-rear direction of the host vehicle. The lateral direction is the direction that coincides with the width direction of the host vehicle. The first automatic driving ECU81 executes acceleration / deceleration control of the host vehicle as longitudinal control. The first automatic driving ECU81 executes steering control of the host vehicle as lateral control. The first automatic driving ECU81 includes a first environment recognition unit, an ACC control unit, an LTA control unit, an LCA control unit, and the like as functional blocks.

[0034] The first environment recognition unit recognizes the driving environment around the vehicle based on sensing information acquired from the surrounding monitoring sensor 60. As an example, the first environment recognition unit recognizes the detailed position of the vehicle in the driving lane from information such as the dividing lines on the left and right of the lane in which the vehicle is traveling (hereinafter, the vehicle lane). In addition, the first environment recognition unit recognizes the positions and speeds of obstacles such as vehicles around the vehicle. The first environment recognition unit recognizes the positions and speeds of obstacles such as vehicles in the vehicle lane. In addition, the first environment recognition unit recognizes the positions and speeds of obstacles such as vehicles in the surrounding lanes of the vehicle lane. The surrounding lanes may be, for example, lanes adjacent to the vehicle lane. In addition, the surrounding lanes may be lanes other than the vehicle lane in the road section in which the vehicle is located. The first environment recognition unit may have the same configuration as the second environment recognition unit described later.

[0035] The ACC control unit executes ACC (Adaptive Cruise Control) control that realizes constant speed driving of the host vehicle at a target speed or driving to follow a preceding vehicle. The ACC control unit executes ACC control by using the positions and speeds of vehicles around the host vehicle recognized by the first environment recognition unit. The ACC control unit executes ACC control by causing the vehicle control ECU 70 to perform acceleration / deceleration control.

[0036] The LTA control unit executes LTA (Lane Tracing Assist) control to keep the host vehicle traveling within the lane. The LTA control unit may execute the LTA control by using the detailed position of the host vehicle in the host vehicle lane recognized by the first environment recognition unit. The LTA control unit may execute the LTA control by causing the vehicle control ECU 70 to perform steering control. Note that the ACC control is an example of longitudinal control. The LTA control is an example of lateral control.

[0037] The LCA control unit executes LCA (Lane Change Assist) control to automatically change lanes of the host vehicle from the host vehicle's lane to an adjacent lane. The LCA control unit may execute LCA control using the positions and speeds of vehicles around the host vehicle recognized by the first environment recognition unit. For example, the LCA control may be executed when the speed of a vehicle preceding the host vehicle is low, equal to or lower than a predetermined value, and there are no surrounding vehicles approaching the host vehicle from the side or rear side. For example, the LCA control unit may execute LCA control by causing the vehicle control ECU 70 to perform acceleration / deceleration control and steering control.

[0038] The first automatic driving ECU 81 realizes level 2 automatic driving by executing both ACC control and LTA control. LCA control may be executable, for example, when ACC control and LTA control are being executed. The first automatic driving ECU 81 may realize level 1 automatic driving by executing either ACC control or LTA control.

[0039] On the other hand, the second automatic driving ECU 82 is responsible for the above-mentioned level 3 or higher automatic driving function. In other words, the second automatic driving ECU 82 enables automatic driving without the obligation to monitor. The second automatic driving ECU 82 includes, as functional blocks, a second environment recognition unit, a behavior determination unit, a trajectory generation unit, and the like.

[0040] The second environment recognition unit recognizes the driving environment around the vehicle based on the sensing information acquired from the surrounding monitoring sensor 60, the vehicle position acquired from the locator 30, the map data acquired from the map DB 40, and the information on other vehicles acquired by the communication module 20. As an example, the second environment recognition unit uses these pieces of information to generate a virtual space that reproduces the actual driving environment.

[0041] The second environment recognition unit discriminates manual driving areas (hereinafter, MD areas) in the area in which the host vehicle travels. The second environment recognition unit discriminates automatic driving areas (hereinafter, AD areas) in the area in which the host vehicle travels. The second environment recognition unit discriminates ST sections in the AD areas. The second environment recognition unit discriminates non-ST sections in the AD areas.

[0042] The MD area is an area where automated driving is prohibited. In other words, the MD area is an area where the driver is required to perform all of the vehicle's longitudinal control, lateral control, and periphery monitoring. For example, the MD area may be a general road.

[0043] An AD area is an area where autonomous driving is permitted. In other words, an AD area is an area where the vehicle can take over one or more of longitudinal control, lateral control, and perimeter monitoring. For example, an AD area may be an expressway or a road for automobiles only.

[0044] AD areas are divided into non-ST sections where automated driving at level 2 or lower is possible, and ST sections where automated driving at level 3 or higher is possible. In this embodiment, there is no separate division between non-ST sections where automated driving at level 1 is permitted and non-ST sections where automated driving at level 2 is permitted. An ST section may be, for example, a driving section where congestion occurs (hereinafter, congested section). An ST section may be, for example, a driving section for which high-precision map data has been developed. A non-ST section may be a section that does not fall under the category of an ST section.

[0045] The behavior determination unit determines a behavior planned for the vehicle (hereinafter, future behavior) based on the recognition result of the driving environment by the second environment recognition unit, etc. The behavior determination unit determines a future behavior for driving the vehicle by autonomous driving. The behavior determination unit may determine a type of behavior that the vehicle should take to arrive at the destination as the future behavior. Examples of this type include going straight, turning right, turning left, changing lanes, etc.

[0046] Furthermore, when the behavior determination unit determines that a driver changeover is necessary, it generates a changeover request and outputs it to the HCU 10. An example of a case in which a driver changeover is necessary is when the vehicle moves from an ST section to a non-ST section in an AD area. Another example of a case in which a driver changeover is necessary is when the vehicle moves from an ST section to an MD area in an AD area. Other causes of a driver changeover (hereinafter, changeover causes) include the elimination of traffic congestion and a lack of high-precision map data.

[0047] A shortage of high-precision map data can be predicted. The behavior determination unit may predict a shortage of high-precision map data for the planned route of the vehicle by using the vehicle position measured by the locator 30 and the high-precision map data stored in the map DB 40. When the behavior determination unit predicts a shortage of high-precision map data, it may determine that a driver change is necessary and output a change request to the HCU 10 before the vehicle reaches a point where the shortage of high-precision map data is predicted.

[0048] The elimination of congestion may or may not be predictable. In particular, when communication module 20 is able to receive congestion information and information on surrounding vehicles, the elimination of congestion can be predicted from this information. The behavior determination unit may predict the elimination of congestion on the planned route of the vehicle using the vehicle's own position measured by locator 30 and the congestion information received by communication module 20. Alternatively, the behavior prediction unit may predict the elimination of congestion on the planned route of the vehicle using the number and speed of surrounding vehicles identified from the information on the surrounding vehicles received by communication module 20. Then, when the behavior determination unit predicts the elimination of congestion, it may determine that a driver change is necessary.

[0049] On the other hand, if the communication module 20 cannot receive traffic congestion information and information on surrounding vehicles, it is assumed that the end of the traffic congestion cannot be predicted. If the end of the traffic congestion cannot be predicted, the end of the traffic congestion can be determined using the number and speed of surrounding vehicles recognized by the second environment recognition unit using the surrounding monitoring sensor 60. Then, when the behavior determination unit determines that the traffic congestion has been resolved, it can determine that a driver change is necessary.

[0050] In addition, there are cases where a driver change is required for reasons other than the elimination of traffic congestion and the lack of high-precision map data. For example, there are changes in road structure, sudden sensor loss, sudden bad weather, etc. Examples of changes in road structure that require a driver change include the end of a section with a median strip, a reduction in the number of lanes, and entry into a construction zone. The reason why these changes in road structure cause a driver change is that the recognition accuracy of the driving environment may decrease. Changes in road structure can be predicted. The behavior determination unit may predict changes in road structure such as the end of a section with a median strip on the planned route of the vehicle and a reduction in the number of lanes, using the vehicle's position measured by the locator 30 and the high-precision map data stored in the map DB 40. In addition, the behavior determination unit may predict changes in road structure such as the vehicle's entry into a construction zone from the presence of a signboard or the like under construction recognized by the second environment recognition unit using the surrounding monitoring sensor 60. Then, when the behavior determination unit predicts these changes in road structure, it may determine that a driver change is necessary.

[0051] Sudden sensor loss is a failure of the perimeter monitoring sensor 60, a failure to recognize the driving environment using the perimeter monitoring sensor 60, etc. Sudden bad weather is heavy rain, snow, fog, etc. Sudden bad weather can cause a change of driving because there is a possibility that the accuracy of recognition of the driving environment using the perimeter monitoring sensor 60 will decrease. In addition, sudden bad weather can cause a change of driving because there is a possibility that a problem will occur in communication in the communication module 20. Sudden sensor loss and sudden bad weather cannot be predicted. The behavior determination unit may determine a sudden sensor loss or sudden bad weather from the recognition result of the driving environment by the second environment recognition unit, etc. Furthermore, the behavior determination unit may determine that a change of driving is necessary when it determines a sudden sensor loss or sudden bad weather.

[0052] The trajectory generation unit generates a driving trajectory of the vehicle in a section where autonomous driving can be performed, based on the recognition result of the driving environment by the second environment recognition unit and the future action determined by the action determination unit. The driving trajectory includes, for example, a target position of the vehicle according to the progress and a target speed at each target position. The trajectory generation unit sequentially provides the generated driving trajectory to the vehicle control ECU 70 as a control command to be followed by the vehicle in autonomous driving.

[0053] An autonomous driving system including the autonomous driving ECU 80 described above enables the host vehicle to perform autonomous driving at level 2 or lower and level 3 or higher. Furthermore, for example, the autonomous driving ECU 80 may be configured to switch the automation level of the host vehicle's autonomous driving as necessary. As an example, when the host vehicle moves from an ST section to a non-ST section in an AD area, autonomous driving may be switched from level 3 autonomous driving to level 2 or lower. Furthermore, when the host vehicle moves from an ST section to an MD area in an AD area, the autonomous driving ECU 80 may switch from level 3 autonomous driving to manual driving.

[0054] When a cause for switching from level 3 autonomous driving to level 2 autonomous driving occurs and the cause for the switching has been predicted, the autonomous driving ECU 80 may switch to the hands-off mode of autonomous driving in the level 2 autonomous driving. On the other hand, when a cause for switching from level 3 autonomous driving to level 2 autonomous driving occurs and the cause for the switching has not been predicted, the autonomous driving ECU 80 may switch to the hands-on mode of autonomous driving in the level 2 autonomous driving. Note that when switching from level 3 autonomous driving to level 1 autonomous driving, the autonomous driving is switched to the hands-on mode. Whether the autonomous driving is switched to the hands-on mode or the hands-off mode due to the driving change may be determined, for example, by the action determination unit.

[0055] The display 91 is a display device provided in the vehicle. The display 91 is provided so that the display surface faces the interior of the vehicle. For example, the display 91 is provided so that the display surface is located in front of the driver's seat of the vehicle. As the display 91, various displays such as a liquid crystal display, an organic EL display, and a head-up display (hereinafter, HUD) can be used.

[0056] The grip sensor 92 detects whether the driver is gripping the steering wheel of the vehicle. The grip sensor 92 may be configured to be provided on the rim of the steering wheel. The user input device 93 receives input from a user. The user input device 93 may be an operation device that receives operation input from the user. The operation device may be a mechanical switch or a touch switch integrated with a display device. Note that the user input device 93 is not limited to an operation device that receives operation input, so long as it is a device that receives input from a user. For example, it may be a voice input device that receives voice command input from the user.

[0057] The HCU 10 is mainly composed of a computer including a processor, a volatile memory, a non-volatile memory, I / O, and a bus connecting these components, and is connected to the display 91 and the in-vehicle LAN. The HCU 10 controls the display on the display 91 by executing a control program stored in the non-volatile memory. This HCU 10 corresponds to a vehicle display control device. The configuration of the HCU 10 regarding the control of the display on the display 91 will be described in detail below.

[0058] <Schematic Configuration of HCU10> Subsequently, the schematic configuration of the HCU 10 will be described with reference to FIG. 2. Regarding the control of the display on the display 91, as shown in FIG. 2, the HCU 10 includes a shift request acquisition unit 101, a mode identification unit 102, an interruption estimation unit 103, a lane change identification unit 104, a grip identification unit 105, and a display control unit 106 as functional blocks. Also, the execution of the processing of each functional block of the HCU 10 by a computer corresponds to the execution of a vehicle display control method. Note that part or all of the functions executed by the HCU 10 may be configured hardware-wise by one or more ICs or the like. Also, part or all of the functional blocks included in the HCU 10 may be realized by a combination of software execution by a processor and hardware components.

[0059] The shift request acquisition unit 101 acquires a shift request output from the automatic driving ECU 80. The shift request acquisition unit 101 acquires this shift request when a shift request is output from the automatic driving ECU 80.

[0060] The mode identification unit 102 identifies whether the autonomous driving of the host vehicle is to be performed in the hands-on mode or the hands-off mode in the autonomous driving of the host vehicle at the automation level 2 or lower. This processing by the mode identification unit 102 corresponds to a mode identification process. The autonomous driving of the automation level 2 or lower can be rephrased as the autonomous driving with the monitoring obligation. The mode identification unit 102 may perform the above-mentioned identification based on the result of the determination by the action determination unit of the autonomous driving ECU 80 as to whether the autonomous driving is to be switched to the hands-on mode or the hands-off mode due to the driving change. The mode identification unit 102 may maintain the above-mentioned identification result until the automation level of the host vehicle is switched. Note that, when the autonomous driving of the host vehicle is switched from the autonomous driving of the automation level 2 and the hands-off mode to the autonomous driving of the automation level 1, the mode identification unit 102 may identify the autonomous driving as the hands-on mode.

[0061] The cut-in estimation unit 103 estimates the cut-in of a surrounding vehicle into the driving lane of the vehicle (i.e., the vehicle's own lane). The cut-in estimation unit 103 may estimate that a surrounding vehicle will cut in to the vehicle's own lane from, for example, a recognition result of a surrounding vehicle of the vehicle in the driving environment recognized by the first environment recognition unit of the automatic driving ECU 80. For example, when the acceleration of the surrounding vehicle toward the vehicle's own lane becomes equal to or greater than a threshold, it may estimate that a surrounding vehicle will cut in to the vehicle's own lane. In addition, it may estimate that a surrounding vehicle will cut in to the vehicle's own lane from the lighting of a blinker lamp on the vehicle's own lane side of the surrounding vehicle. The lighting of the blinker lamp of the surrounding vehicle may be configured to be recognized by the first environment recognition unit by image analysis of an image captured by a surrounding monitoring camera. In addition, when the information of the surrounding vehicle received by the communication module 20 includes information informing that the surrounding vehicle will cut in to the vehicle's own lane, this information may be used to estimate that a surrounding vehicle will cut in to the vehicle's own lane.

[0062] The lane change identification unit 104 identifies that the vehicle will change lanes by autonomous driving. Since the LCA control is executed by the LCA control unit of the autonomous driving ECU 80, for example, the lane change identification unit 104 may identify that the vehicle will change lanes by autonomous driving.

[0063] The grip identification unit 105 identifies whether the driver is gripping the steering wheel of the vehicle. For example, the grip identification unit 105 may identify whether the driver is gripping the steering wheel from the detection result of the grip sensor 92. Note that the grip identification unit 105 may identify whether the driver is gripping the steering wheel from a detection result other than that of the grip sensor 92. For example, the grip identification unit 105 may identify whether the driver is gripping the steering wheel by performing image recognition on an image of the driver captured by a DSM (Driver Status Monitor).

[0064] The display control unit 106 controls the display on the display 91. The processing in the display control unit 106 corresponds to a display control step. The display control unit 106 displays an image (hereinafter, a surrounding situation image) for showing the surrounding situation of the vehicle on the display 91 while the vehicle is being driven autonomously at level 2 or lower or while being driven manually. The display control unit 106 may display a surrounding situation image as an overhead image showing the positional relationship between the vehicle and the surrounding vehicles as seen from a virtual viewpoint above the vehicle, using the positional relationship between the vehicle and the surrounding vehicles in the driving environment recognized by the autonomous driving ECU 80. This virtual viewpoint may be directly above the vehicle, or may be a position shifted from directly above the vehicle. For example, it may be an overhead view seen from a virtual viewpoint above and behind the vehicle. The surrounding situation image may be a virtual image for showing the surrounding situation of the vehicle, or may be a processed image captured by a surrounding monitoring camera of the surrounding monitoring sensor 60.

[0065] An example of a surrounding situation image will now be described with reference to Fig. 3. Sc in Fig. 3 indicates the display screen of the display device 91. PLI in Fig. 3 indicates an image showing lane markings (hereinafter, marking line image). HVI in Fig. 3 indicates an image showing the host vehicle (hereinafter, host vehicle image). OVI in Fig. 3 indicates an image showing vehicles surrounding the host vehicle (hereinafter, surrounding vehicle image). Figs. 3 to 11 show an example in which the surrounding vehicles are vehicles preceding the host vehicle. Ve in Fig. 3 indicates an image showing the speed of the host vehicle (hereinafter, vehicle speed image).

[0066] As shown in Fig. 3, the surrounding situation image includes an image of the vehicle itself, an image of surrounding vehicles, an image of lane lines, and an image of vehicle speed. The image of the vehicle itself, the image of surrounding vehicles, the image of lane lines, and the image of vehicle speed correspond to the image elements of the surrounding situation image. As shown in Fig. 3, the surrounding situation image may include image elements other than the image of the vehicle itself, the image of surrounding vehicles, and the image of lane lines, which are images showing the surrounding situation of the vehicle itself.

[0067] In addition, when an image showing the view in front of the vehicle is used as the surrounding situation image, the surrounding situation image may not include the vehicle image. The surrounding situation image may also include image elements such as an assistance execution image, a hands-on / off image, and a background image. The assistance execution image is an image showing control related to driving assistance being executed in the vehicle. Examples of control related to driving assistance include the above-mentioned ACC control and LTA control. The hands-on / off image is an image showing whether the vehicle is autonomously driving in hands-on mode or hands-off mode. The background image is an image showing the background of the surrounding situation image.

[0068] On the other hand, when the vehicle is in autonomous driving at level 3 or higher, the display control unit 106 may, for example, not display the surrounding situation image, but instead display an image explaining the action permitted as the second task, an image showing the vehicle speed of the vehicle, etc., on the display 91. Another example of not displaying the surrounding situation image is to display an image of the vehicle itself and an image of the lane markings corresponding to the vehicle itself, but not to display images of the surrounding vehicles. This means that images of the surrounding vehicles are not displayed even if the surrounding monitoring sensor 60 detects surrounding vehicles.

[0069] When the vehicle switches from level 3 autonomous driving to level 2 or lower autonomous driving, the display control unit 106 changes the display of the surrounding situation image depending on whether the mode specification unit 102 specifies the autonomous driving as hands-on mode or hands-off mode. Note that autonomous driving at automation level 3 can be rephrased as autonomous driving without monitoring obligation. Hereinafter, an example of the difference in the display mode of the surrounding situation image between the hands-on mode and the hands-off mode when the vehicle switches from level 3 autonomous driving to level 2 autonomous driving will be described with reference to Figs. 4 to 11. HON in Figs. 4 to 11 indicates the display mode in the hands-on mode. Meanwhile, HOFF in Figs. 4 to 11 indicates the display mode in the hands-off mode.

[0070] When the mode specifying unit 102 specifies the autonomous driving mode as the hands-on mode, the display control unit 106 may display the own lane and the surrounding lanes. On the other hand, when the mode specifying unit 102 specifies the autonomous driving mode as the hands-off mode, the display control unit 106 may display only the own lane among the own lane and the surrounding lanes. The surrounding lanes may be, for example, lanes adjacent to the own lane. Alternatively, the surrounding lanes may be lanes other than the own lane in the road section in which the own vehicle is located. As a specific example, as shown in FIG. 4, in the hands-on mode, the lane marking images of both the own lane and the surrounding lanes may be displayed. On the other hand, in the hands-off mode, the lane marking image of only the own lane among the own lane and the surrounding lanes may be displayed.

[0071] In the hands-off mode, which is more likely to ensure safety than the hands-on mode, it is considered that it is sufficient for the driver to know the situation closer to the vehicle. Conversely, in the hands-on mode, it is considered that the driver has a demand to know the situation farther from the vehicle. In contrast, according to the above configuration, when the vehicle is in the hands-on mode, the situation of more lanes is displayed than when the vehicle is in the hands-off mode. Therefore, it is possible to display the surrounding situation image in a display mode according to whether the vehicle is in the hands-on mode or the hands-off mode. In addition, since the number of lanes displayed in the surrounding situation image is different depending on whether the vehicle is in the hands-on mode or the hands-off mode, the driver of the vehicle can more easily recognize whether the vehicle will switch to automatic driving in the hands-on mode or the hands-off mode from this difference.

[0072] When the mode specifying unit 102 specifies the automatic driving in the hands-on mode, the display control unit 106 may display a surrounding situation image seen from a virtual viewpoint farther away from the display target in the surrounding situation image than when the mode specifying unit 102 specifies the automatic driving in the hands-off mode. On the other hand, when the mode specifying unit 102 specifies the automatic driving in the hands-off mode, the display control unit 106 may display a surrounding situation image seen from a virtual viewpoint closer to the display target than when the mode specifying unit 102 specifies the automatic driving in the hands-on mode. The display target referred to here is an object, a lane mark, etc. represented in the surrounding situation image. As a specific example, as shown in FIG. 5, in the hands-on mode, a surrounding situation image as if the surrounding situation of the vehicle is seen from a farther distance than in the hands-off mode may be displayed. On the other hand, in the hands-off mode, a surrounding situation image as if the surrounding situation of the vehicle is seen from a closer distance than in the hands-on mode may be displayed.

[0073] According to the above configuration, when the vehicle is in hands-on mode, a wider range of the situation is displayed than when the vehicle is in hands-off mode. Therefore, it is possible to display the surrounding situation image in a display mode according to whether the vehicle is in hands-on mode or hands-off mode. In addition, since the perspective of the virtual viewpoint of the surrounding situation image is changed depending on whether the vehicle is in hands-on mode or hands-off mode, the driver of the vehicle can more easily recognize from this difference whether the vehicle is switching to automatic driving in hands-on mode or hands-off mode.

[0074] When the mode specifying unit 102 specifies the automatic driving in the hands-on mode, the display control unit 106 may display a surrounding situation image seen from a virtual viewpoint looking down from above, more than when the mode specifying unit 102 specifies the automatic driving in the hands-off mode. On the other hand, when the mode specifying unit 102 specifies the automatic driving in the hands-off mode, the display control unit 106 displays a surrounding situation image seen from a virtual viewpoint looking down from below, more than when the mode specifying unit 102 specifies the automatic driving in the hands-on mode. As a specific example, as shown in FIG. 6, in the hands-on mode, a surrounding situation image in which the situation of the host vehicle is viewed from a higher viewpoint than in the hands-off mode may be displayed. On the other hand, in the hands-off mode, a surrounding situation image in which the situation of the host vehicle is viewed from a lower viewpoint than in the hands-on mode may be displayed.

[0075] According to the above configuration, when the vehicle is in hands-on mode, a wider range of the situation is displayed than when the vehicle is in hands-off mode. Therefore, it is possible to display the surrounding situation image in a display mode according to whether the vehicle is in hands-on mode or hands-off mode. In addition, since the height of the virtual viewpoint of the surrounding situation image is changed depending on whether the vehicle is in hands-on mode or hands-off mode, the driver of the vehicle can more easily recognize whether the vehicle is switching to automatic driving in hands-on mode or hands-off mode from this difference.

[0076] When the mode specifying unit 102 specifies the autonomous driving in the hands-on mode, the display control unit 106 may display a wider area around the vehicle as a surrounding situation image than when the mode specifying unit 102 specifies the autonomous driving in the hands-off mode. On the other hand, when the mode specifying unit 102 specifies the autonomous driving in the hands-off mode, the display control unit 106 may display a narrower area around the vehicle as a surrounding situation image than when the mode specifying unit 102 specifies the autonomous driving in the hands-on mode. As a specific example, as shown in FIG. 7, in the hands-on mode, a surrounding situation image with a wider cut-out area around the vehicle may be displayed than in the hands-off mode. On the other hand, in the hands-off mode, a surrounding situation image with a narrower cut-out area around the vehicle may be displayed than in the hands-on mode.

[0077] According to the above configuration, when the vehicle is in hands-on mode, a wider range of the situation is displayed than when the vehicle is in hands-off mode. Therefore, it is possible to display the surrounding situation image in a display mode according to whether the vehicle is in hands-on mode or hands-off mode. In addition, since the range of the surroundings of the vehicle shown in the surrounding situation image is different depending on whether the vehicle is in hands-on mode or hands-off mode, the driver of the vehicle can more easily recognize from this difference whether the vehicle is switching to automatic driving in hands-on mode or hands-off mode.

[0078] The display control unit 106 may change the color tone of at least a part of the surrounding situation image depending on whether the mode specification unit 102 specifies the autonomous driving in the hands-on mode or the hands-off mode. As a specific example, as shown in FIG. 8, the color tone of the support execution image (see ACC and LTA in FIG. 8) may be changed between the hands-on mode and the hands-off mode. The ACC in FIG. 8 shows a support execution image indicating that the ACC control is being executed. The LTA in FIG. 8 shows a support execution image indicating that the LTA control is being executed. In the example of FIG. 8, an example in which the color tone of the support execution image is changed between the hands-on mode and the hands-off mode is shown, but this is not necessarily limited to this. For example, a configuration may be adopted in which the color tone of image elements other than the support execution image in the surrounding situation image is changed.

[0079] According to the above configuration, the color tones of the image elements in the surrounding situation image are made different depending on whether the vehicle is in hands-on mode or hands-off mode, so that the driver of the vehicle can more easily recognize from this difference whether the vehicle is switching to automated driving in hands-on mode or hands-off mode.

[0080] Furthermore, when the mode specifying unit 102 specifies the autonomous driving mode as the hands-on mode, it is preferable that the display control unit 106 displays the image elements of the surrounding situation image in a color tone that is more likely to call attention than when the mode specifying unit 102 specifies the autonomous driving mode as the hands-off mode. For example, when the hands-on mode is specified, the image elements may be displayed in an exciting color tone such as red. On the other hand, when the hands-off mode is specified, the image elements may be displayed in a calming color tone such as blue.

[0081] In the hands-on mode, it is considered that the driver needs to pay more attention to the driving of the vehicle than in the hands-off mode. In response to this, according to the above configuration, when the vehicle is in the hands-on mode, the image elements of the surrounding situation image are displayed in a color tone that is more likely to call attention than when the vehicle is in the hands-off mode. Therefore, it is possible to display the surrounding situation image in a display mode according to whether the vehicle is in the hands-on mode or the hands-off mode.

[0082] The display control unit 106 may change at least one of the layout and the size ratio of the image elements in the surrounding situation image depending on whether the mode specification unit 102 specifies the autonomous driving as the hands-on mode or the hands-off mode. As a specific example, the layout of the image elements may be changed between the hands-on mode and the hands-off mode, as shown in Fig. 9. HM in Fig. 9 indicates the hands-on / off image. In the example of Fig. 9, the layout of the image elements showing the surrounding situation of the vehicle in the surrounding situation image and the hands-on / off image on the left and right is changed between the hands-on mode and the hands-off mode.

[0083] According to the above configuration, the arrangement of image elements in the surrounding situation image is different depending on whether the vehicle is in hands-on mode or hands-off mode, so that the driver of the vehicle can more easily recognize from this difference whether the vehicle is switching to automated driving in hands-on mode or hands-off mode.

[0084] In addition, as shown in FIG. 10, when the mode identification unit 102 identifies autonomous driving in hands-on mode, it is preferable that the display control unit 106 increases the ratio of the size of the hands-on / off image compared to when the mode identification unit 102 identifies autonomous driving in hands-off mode.

[0085] In the hands-off mode, the driver does not need to grip the steering wheel, but in the hands-on mode, the driver must grip the steering wheel. Therefore, it is preferable that the driver is more likely to notice the hands-on / off image in the hands-on mode than in the hands-off mode. In contrast, according to the above configuration, when the host vehicle is in the hands-on mode, the hands-on / off image is displayed larger than when the host vehicle is in the hands-off mode, so that the driver is more likely to notice the hands-on / off image. Therefore, it is possible to display the surrounding situation image in a display mode according to whether the host vehicle is in the hands-on mode or the hands-off mode.

[0086] The display control unit 106 may change the background image in the surrounding situation image depending on whether the mode specification unit 102 specifies the autonomous driving as the hands-on mode or the hands-off mode. As a specific example, the background image may be changed between the hands-on mode and the hands-off mode as shown in FIG. 11. BI in FIG. 11 indicates the background image. As an example, when a pattern is to be displayed as the background image, the pattern may be changed. In addition, the background image may be displayed more clearly in the hands-on mode than in the hands-off mode.

[0087] According to the above configuration, the background image in the surrounding situation image is different depending on whether the vehicle is in hands-on mode or hands-off mode, so that the driver of the vehicle can more easily recognize from this difference whether the vehicle is switching to automated driving in hands-on mode or hands-off mode.

[0088] The display control unit 106 may be configured to switch between various display modes according to whether the vehicle is in the hands-on mode or the hands-off mode, as exemplified in Figures 4 to 11, only a part of the display modes, or a combination of a plurality of display modes. When the vehicle switches from level 3 autonomous driving to level 1 autonomous driving or manual driving, the display control unit 106 may display the surrounding situation image in the display mode of the hands-off mode.

[0089] In at least one of the cases where the vehicle changes lanes by automatic driving in the hands-off mode and where it is estimated that a surrounding vehicle will cut into the vehicle's lane, when the vehicle changes lanes by automatic driving in the hands-off mode and the display control unit 106 continues the hands-off mode automatic driving, it is preferable to switch the display of the surrounding situation image to a display mode similar to that of the hands-on mode, even if the mode specification unit 102 specifies that the vehicle is in the hands-off mode automatic driving. The lane change specification unit 104 may specify that the vehicle will change lanes by automatic driving. The cut-in estimation unit 103 may estimate that a surrounding vehicle will cut into the vehicle's lane.

[0090] When the vehicle changes lanes through autonomous driving or when it is estimated that a surrounding vehicle will cut into the vehicle's lane, it is considered that the possibility of a relatively large vehicle behavior occurring increases even in the hands-off mode, and the possibility of transitioning to the hands-on mode increases. In contrast, with the above configuration, even if the autonomous driving in the hands-off mode continues, if the possibility of transitioning to the hands-on mode increases, the driver can more easily prepare for the transition to the hands-on mode.

[0091] When the time elapsed since the host vehicle switched to the hands-off mode autonomous driving reaches a specified time, it is preferable for the display control unit 106 to switch to the display of the surrounding situation image for the case where the mode specification unit 102 specifies the hands-on mode autonomous driving, even if the hands-off mode autonomous driving continues. The specified time here is a time that can be set arbitrarily.

[0092] It is considered that the amount of information that the driver must confirm is greater when the vehicle is in hands-on mode than when the vehicle is in hands-off mode. In response to this, with the above configuration, the display of the surrounding situation image is switched to the same display as in the hands-on mode before switching from the hands-off mode to the hands-on mode. This reduces the amount of information that is newly added when switching from the hands-off mode to the hands-on mode, and reduces the burden on the driver.

[0093] When the vehicle has switched to autonomous driving in hands-off mode and the grip identification unit 105 identifies gripping of the steering wheel, it is preferable that the display control unit 106 switches to the display of a surrounding situation image that is identified as autonomous driving in hands-on mode by the mode identification unit 102, even if autonomous driving in hands-off mode is continued.

[0094] Even when the vehicle is in hands-off mode, if the driver holds the steering wheel, it is preferable to display a surrounding situation image similar to that in the hands-on mode, just as if the vehicle was in hands-on mode. In contrast, with the above configuration, even when the vehicle is in hands-off mode, if the driver holds the steering wheel, it is possible to display a surrounding situation image similar to that in the hands-on mode.

[0095] Further, the display control unit 106 may be configured to reverse or customize the display between the hands-on mode and the hands-off mode according to the driver's preference. As an example, it may be configured to reverse or customize the display between the hands-on mode and the hands-off mode according to the input received by the user input device 93.

[0096] <First display control related processing in HCU10> Here, using the flowchart of FIG. 12, an example of the flow of processing related to the control of the display according to whether it is the hands-on mode or the hands-off mode in HCU10 (hereinafter, the first display control related processing) will be described. The flowchart of FIG. 12 may be configured to start, for example, when a driving change is made after the host vehicle starts level 3 (LV3) automated driving. HCU10 may determine that a driving change has occurred from the fact that the change request acquisition unit 101 has acquired a change request. Further, as described above, during LV3 automated driving, the display control unit 106 may not display the surrounding situation image and may display, for example, an image explaining an action permitted as a second task on the display 91.

[0097] First, in step S1, the mode identification unit 102 identifies whether the host vehicle will execute hands-on mode automated driving or hands-off mode automated driving after the driving change. If it is identified as the hands-on mode (YES in S1), the process proceeds to step S2. On the other hand, if it is identified as the hands-off mode (NO in S1), the process proceeds to step S3.

[0098] In step S2, the display control unit 106 displays the surrounding situation image on the display 91 in the above-described hands-on mode display mode, and the process proceeds to step S8. On the other hand, in step S3, the display control unit 106 displays the surrounding situation image on the display 91 in the above-described hands-off mode display mode.

[0099] In step S4, if the lane change identification unit 104 identifies that the vehicle will change lanes by autonomous driving (YES in S4), the process proceeds to S2. On the other hand, if the lane change identification unit 104 does not identify that the vehicle will change lanes by autonomous driving (NO in S4), the process proceeds to step S5.

[0100] In step S5, if the cut-in estimation unit 103 estimates that a surrounding vehicle will cut in to the own lane (YES in S5), the process proceeds to S2. On the other hand, if the cut-in estimation unit 103 does not estimate that a surrounding vehicle will cut in to the own lane (NO in S5), the process proceeds to S6.

[0101] In step S6, if the gripping identification unit 105 identifies gripping of the steering wheel (YES in S6), the process proceeds to S2. On the other hand, if the gripping identification unit 105 does not identify gripping of the steering wheel (NO in S6), the process proceeds to S7.

[0102] In step S7, if the time elapsed since the driver changeover has reached the specified time (YES in S7), the process proceeds to S2. On the other hand, if the time elapsed since the driver changeover has not reached the specified time (NO in S7), the process proceeds to step S8.

[0103] In step S8, if it is time to end the first display control related process (YES in S8), the first display control related process is ended. On the other hand, if it is not time to end the first display control related process (NO in S8), the process returns to S1 and is repeated. Examples of timing to end the first display control related process include when the power switch is turned off, when the vehicle switches to automatic driving at level 3 or higher, etc.

[0104] <Summary of the First Embodiment> According to the configuration of the first embodiment, the display of the surrounding situation image displayed on the display 91 used in the vehicle cabin of the vehicle is changed depending on whether the autonomous driving without a monitoring obligation is switched to the autonomous driving of the hands-on mode or the hands-off mode of the autonomous driving with a monitoring obligation. Therefore, the driver of the vehicle can more easily recognize whether the autonomous driving is switched to the hands-on mode or the hands-off mode from the difference in the display of the surrounding situation image. As a result, when switching from the autonomous driving without a monitoring obligation to the autonomous driving with a monitoring obligation, it is possible to make the driver more easily recognize whether the autonomous driving after the switching is the hands-on mode or the hands-off mode.

[0105] In addition, as described above, it is considered that the display manner required is different between the hands-on mode and the hands-off mode. In response to this, according to the configuration of the first embodiment, it is possible to display the surrounding situation image in a display manner according to whether the vehicle is in the hands-on mode or the hands-off mode. In this respect, when switching from the autonomous driving without the monitoring obligation to the autonomous driving with the monitoring obligation, it is possible to make the driver more easily recognize whether the autonomous driving after the switching is in the hands-on mode or the hands-off mode.

[0106] (Embodiment 2) In the first embodiment, when the gripping identification unit 105 identifies gripping of the steering wheel in a state where the host vehicle has switched to autonomous driving in a hands-off mode, the display control unit 106 switches to a display of a surrounding situation image in a case where the mode identification unit 102 identifies autonomous driving in a hands-on mode. However, the present invention is not necessarily limited to this. For example, the configuration of the second embodiment shown below may be used. An example of the second embodiment will be described below with reference to the drawings. The vehicle system 1 of the second embodiment is similar to the vehicle system 1 of the first embodiment, except that a part of the processing in the display control unit 106 in a case where the gripping identification unit 105 identifies gripping of the steering wheel in a state where the host vehicle has switched to autonomous driving in a hands-off mode is different.

[0107] In the display control unit 106 of the second embodiment, when the gripping identification unit 105 identifies gripping of the steering wheel in a state in which the vehicle has switched to automatic driving in the hands-off mode, it is preferable to continue displaying a surrounding situation image for the case in which the mode identification unit 102 identifies automatic driving in the hands-off mode for a predetermined time after the gripping identification unit 105 identifies gripping of the steering wheel, and then switch to displaying a surrounding situation image for the case in which the mode identification unit 102 identifies automatic driving in the hands-on mode, even if automatic driving in the hands-off mode continues. The predetermined time referred to here is a time that can be set arbitrarily.

[0108] Here, an example of the flow of the first display control related process in the HCU 10 of the second embodiment will be described with reference to the flowchart of Fig. 13. The flowchart of Fig. 13 may be configured to be started under the same conditions as the flowchart of Fig. 12.

[0109] In step S21, the mode identification unit 102 identifies whether the vehicle will be in hands-on mode or hands-off mode after the driver handover. If the vehicle is in the hands-on mode (YES in S21), the process proceeds to step S22. On the other hand, if the vehicle is in the hands-off mode (NO in S21), the process proceeds to step S23.

[0110] In step S22, the display control unit 106 causes the display 91 to display the surrounding situation image in the display mode of the hands-on mode described above in the first embodiment, and the process proceeds to step S29. On the other hand, in step S23, the display control unit 106 causes the display 91 to display the surrounding situation image in the display mode of the hands-off mode described above in the first embodiment.

[0111] The processing from step S24 to step S26 may be the same as the processing from S1 to S6 described above. In step S27, if the time that has elapsed since the driver changeover has reached the specified time (YES in S27), the process proceeds to S28. On the other hand, if the time that has elapsed since the driver changeover has not reached the specified time (NO in S27), the process proceeds to step S29. In step S28, the display of the surrounding situation image in the display mode of the hands-off mode is continued for a predetermined time after the gripping of the steering wheel is identified in S26, and then the process proceeds to S22.

[0112] In step S29, if it is time to end the first display control related process (YES in S29), the first display control related process is ended. On the other hand, if it is not time to end the first display control related process (NO in S29), the process returns to S21 and is repeated.

[0113] As in the first embodiment, the configuration of the second embodiment also allows the driver to easily recognize whether the autonomous driving after switching is in a hands-on mode or a hands-off mode when switching from an autonomous driving without a monitoring obligation to an autonomous driving with a monitoring obligation. In addition, according to the configuration of the second embodiment, when the vehicle is in a hands-off mode, even if the driver holds the steering wheel, the surrounding situation image is displayed in the display mode of the hands-off mode for a predetermined time. Therefore, it is possible to make the driver recognize that it is not necessary to hold the steering wheel.

[0114] (Embodiment 3) In the first embodiment, a configuration is shown in which, in the hands-off mode, a lane marking image of only the own lane out of the own lane and surrounding lanes is displayed, but when an obstacle is detected in the surrounding lanes, the following configuration of the third embodiment may be used. An example of the third embodiment is described below with reference to the drawings. In the following, a surrounding vehicle is described as an example of an obstacle.

[0115] In the example of the third embodiment, as shown in FIG. 14, a display example in which the surrounding situation image also includes a surrounding vehicle image will be described. OVIH in FIG. 14 indicates an image representing a surrounding vehicle located in the own lane. OVIO in FIG. 14 indicates an image representing a surrounding vehicle located in a surrounding lane of the own lane. In the third embodiment, as described in the first embodiment, when the mode specification unit 102 specifies the automatic driving in the hands-off mode, the display control unit 106 displays only the own lane out of the own lane and the surrounding lanes. On the other hand, even when only the own lane is displayed, the display control unit 106 allows both an image showing a surrounding vehicle corresponding to the own lane and an image showing a surrounding vehicle corresponding to the surrounding lane to be displayed as the surrounding vehicle image.

[0116] According to the above configuration, as in the example shown in FIG. 4 of the first embodiment, the display is narrowed down to select necessary information and is easy for the driver to understand, compared to the case where the surrounding lanes are displayed. Even if the display of the surrounding lanes is omitted, the driver can recognize the situation of the surrounding lanes by displaying an image showing the surrounding vehicles located in the surrounding lanes. By omitting the display of the surrounding lanes, the possibility of suppressing the nuisance of the display is increased. For example, the position of the lane is sequentially identified from the map data and the recognition result of the division line by the surrounding monitoring sensor 60, and the lane is displayed. In this case, a problem of blurring of the display may occur when the display of the lane is updated. On the other hand, the more lanes are displayed, the more noticeable this blurring becomes, and the more likely it is to be felt as a nuisance. Therefore, by omitting the display of the surrounding lanes, it is possible to make the blurring less noticeable and suppress the nuisance of the display.

[0117] (Embodiment 4) In the first embodiment, the case where the driving is switched from level 3 autonomous driving to level 2 autonomous driving has been described as an example, but the present invention is not necessarily limited to this. For example, the present invention may be applied to a case where the driving is switched from level 4 or higher autonomous driving to level 2 or lower autonomous driving or manual driving.

[0118] (Embodiment 5) In the above-described embodiment, a configuration has been shown in which a surrounding situation image is not displayed while the vehicle is autonomously driving at level 3 or higher, but this is not necessarily limited to this. For example, a configuration may be possible in which a surrounding situation image is displayed even while the vehicle is autonomously driving at level 3 or higher (hereinafter, referred to as embodiment 5). An example of embodiment 5 will be described below with reference to the drawings. The vehicle system 1 of embodiment 5 is similar to the vehicle system 1 of embodiment 1, except that it includes an HCU 10a instead of the HCU 10.

[0119] Here, the schematic configuration of the HCU 10a will be described with reference to Fig. 15. As shown in Fig. 15, the HCU 10a includes a changeover request acquisition unit 101, a mode identification unit 102, an interruption estimation unit 103, a lane change identification unit 104, a grip identification unit 105, and a display control unit 106a as functional blocks for controlling the display on the display device 91. The HCU 10a is similar to the HCU 10 of the first embodiment, except that the HCU 10a includes the display control unit 106a instead of the display control unit 106. This HCU 10a also corresponds to a display control device for a vehicle. In addition, the execution of the processing of each functional block of the HCU 10a by a computer also corresponds to the execution of a display control method for a vehicle.

[0120] The display control unit 106a is similar to the display control unit 106 of the first embodiment, except that it is capable of displaying a surrounding situation image even when the vehicle is autonomously driving at level 3 or higher, and performs processing related to this. The following describes processing that differs from that of the display control unit 106 of the first embodiment.

[0121] For example, the display control unit 106a displays the surrounding situation image even when the vehicle is in autonomous driving at level 3 or higher. Autonomous driving at level 3 or higher can be rephrased as autonomous driving without a monitoring obligation. When the autonomous driving stage (i.e., the automation level) is switched to a lower level of automation while the vehicle is in autonomous driving at level 3 or higher and the surrounding situation image is displayed, the display control unit 106a changes the display of the surrounding situation image according to the automation level before the switch to the display of the surrounding situation image according to the automation level after the switch after a predetermined time has elapsed from the switch of the automation level, regardless of whether the autonomous driving is in hands-on mode or hands-off mode. The predetermined time mentioned here may be set arbitrarily. According to the above configuration, the display of the surrounding situation image is changed after the switch of the automation level, so that it is possible to prevent the driver from becoming confused.

[0122] As for the display of the surrounding situation image after the automation level is switched, it is sufficient to switch between the hands-on mode and the hands-off mode, as in the first embodiment. In addition, as an example of the display of the surrounding situation image according to the automation level, for example, the following may be performed. At level 3, it is sufficient to display the lane marking image only of the own lane among the own lane and the surrounding lanes. At level 2, it is sufficient to display both the own lane and the surrounding lanes. As for the surrounding vehicle image, it is sufficient to display only the own lane at level 3, while it is sufficient to display the surrounding lanes at level 2. In this case, the application of the example shown in FIG. 4 may be excluded as the switching of the display of the surrounding situation image between the hands-on mode and the hands-off mode at level 2.

[0123] Also, as described in the first embodiment, if the surrounding situation image is not displayed during the autonomous driving of the vehicle at level 3 or higher, the following may be performed. When the automation level is switched to a lower stage of automation while the surrounding situation image is not displayed during the autonomous driving of level 3 or higher, the display control unit 106 may change the display of the surrounding situation image to one corresponding to the automation level after the switch, regardless of whether the autonomous driving is in the hands-on mode or the hands-off mode, at the same time as the switching of the automation level or before the switching of the stage of the autonomous driving. The term "simultaneous" as used here may include an error that can be said to be substantially simultaneous. With the above configuration, it is possible to convey information about the surroundings of the vehicle to the driver more quickly.

[0124] Here, with reference to FIG. 16, differences in the timing of switching the display depending on whether or not the surrounding situation image is displayed during autonomous driving of the own vehicle at level 3 or higher will be described. Y in FIG. 16 shows an example of a case where the surrounding situation image is displayed during autonomous driving of level 3 or higher. N in FIG. 16 shows an example of a case where the surrounding situation image is not displayed during autonomous driving of level 3 or higher. LC in FIG. 16 shows the timing of switching the automation level. S in FIG. 16 shows the start timing of displaying the surrounding situation image according to the automation level after switching. As shown in FIG. 16, when the surrounding situation image is displayed during autonomous driving of level 3 or higher, the display of the surrounding situation image according to the automation level after switching is started after the automation level is switched. On the other hand, when the surrounding situation image is not displayed during autonomous driving of level 3 or higher, the display of the surrounding situation image according to the automation level after switching is started at least before the automation level is switched.

[0125] In addition, the present invention is not limited to a configuration in which whether or not to display a surrounding situation image during autonomous driving of the own vehicle at level 3 or higher is fixed. For example, a configuration may be adopted in which a setting as to whether or not to display a surrounding situation image during autonomous driving of the own vehicle at level 3 or higher is switchable. The setting may be switched in response to an input from a user received by the user input device 93. In this case, the display control unit 106a may be configured to use the above-mentioned processes selectively depending on whether or not to display a surrounding situation image.

[0126] (Embodiment 6) A configuration in which the vehicle switches from autonomous driving level 4 or higher to autonomous driving level 3 may be that of the sixth embodiment, which will be described below as an example. An example of the sixth embodiment will be described below with reference to the drawings.

[0127] First, a vehicle system 1b according to the sixth embodiment will be described with reference to Fig. 17. As shown in Fig. 17, the vehicle system 1b includes an HCU 10b, a communication module 20, a locator 30, a map DB 40, a vehicle state sensor 50, a surroundings monitoring sensor 60, a vehicle control ECU 70, an automatic driving ECU 80, a display 91b, a grip sensor 92, a user input device 93, and a DSM (Driver Status Monitor) 94. The vehicle system 1b is similar to the vehicle system 1 according to the first embodiment, except that the vehicle system 1b includes an HCU 10b and a display 91b instead of the HCU 10 and the display 91, and includes a DSM 94. The vehicle system 1b also corresponds to a vehicle display control system.

[0128] 17, the display 91b has a driver's side display 911 and a passenger's side display 912. The display 91b is similar to the display 91 of the first embodiment except that the display 91b has two types of display devices, the driver's side display 911 and the passenger's side display 912.

[0129] The driver's side display 911 is a display device whose display surface is located in front of the driver's seat of the vehicle. As the driver's side display 911, a meter MID (Multi Information Display) or a HUD (Head-Up Display) can be used. The meter MID is a display device provided in front of the driver's seat inside the vehicle cabin. As an example, the meter MID may be configured to be provided on a meter panel. The HUD is provided, for example, on an instrument panel inside the vehicle cabin. The HUD projects a display image formed by a projector onto a projection area that is set on a front windshield as a projection member. The light of the image reflected by the front windshield into the vehicle cabin is perceived by the driver sitting in the driver's seat. This allows the driver to view a virtual image of the display image formed in front of the front windshield, superimposed on a part of the foreground. The HUD may be configured to project a display image onto a combiner provided in front of the driver's seat instead of the front windshield. The display surface of the HUD is located above the display surface of the meter MID. The driver's side display 911 may use a plurality of display devices.

[0130] Passenger side display 912 is a display device whose display surface is located at a position visible to passengers of the vehicle other than driver side display 911. A passenger is a person in the vehicle other than the driver. Passenger side display 912 may be a display device visible from the passenger seat or a display device visible from the rear seat. An example of a display device visible from the passenger seat is a CID (Center Information Display). A CID is a display device disposed in the center of the instrument panel of the vehicle. An example of a display device visible from the rear seat is a display device provided on the seat back, ceiling, etc. of the front seat. Multiple display devices may be used as passenger side display 912.

[0131] The DSM94 is composed of a near-infrared light source, a near-infrared camera, and a control unit for controlling them. The DSM94 is disposed, for example, on the upper surface of the instrument panel with the near-infrared camera facing the driver's seat side of the vehicle. The DSM94 uses the near-infrared camera to capture an image of the driver's head illuminated with near-infrared light by the near-infrared light source. The image captured by the near-infrared camera is subjected to image analysis by the control unit. The control unit detects the driver's level of alertness based on the driver's features extracted by image analysis of the captured image. The level of alertness is detected by distinguishing at least an alert state from a sleeping state.

[0132] Next, a schematic configuration of the HCU 10b will be described with reference to FIG. 18. As shown in FIG. 18, the HCU 10b includes a changeover request acquisition unit 101, a mode identification unit 102, an interruption estimation unit 103, a lane change identification unit 104, a grip identification unit 105, a display control unit 106b, and a state identification unit 107 as functional blocks for controlling the display on the display device 91b. The HCU 10b is similar to the HCU 10 of the first embodiment, except that the HCU 10b includes a display control unit 106b instead of the display control unit 106, and that the HCU 10b includes a state identification unit 107. This HCU 10b also corresponds to a display control device for a vehicle. In addition, the execution of the processing of each functional block of the HCU 10b by a computer also corresponds to the execution of a display control method for a vehicle.

[0133] The state identification unit 107 identifies the state of the driver. The state identification unit 107 identifies the state related to the wakefulness of the driver from the wakefulness of the driver detected successively by the DSM94. The state identification unit 107 distinguishes at least between an wakefulness state in which the driver is wakeful and a sleep state in which the driver is asleep. Here, a configuration in which the wakefulness state of the driver is detected by a control unit of the DSM94 is shown, but the state identification unit 107 may take on part of the function of this control unit. Furthermore, the state identification unit 107 may identify the state related to the wakefulness of the driver from a detection result other than that of the DSM94. For example, the state related to the wakefulness of the driver may be identified from a detection result of a biosensor that detects the pulse wave of the driver.

[0134] The display control unit 106b is similar to the display control units 106 and 106a, except for some different processing. The following describes processing different from the display control units 106 and 106a. The display control unit 106b displays information related to the driving of the vehicle (hereinafter, driving-related information) on the display 91b. The driving-related information displayed on the display 91b includes surrounding situation images and images that do not correspond to surrounding situation images. In other words, the driving-related information also includes surrounding situation images in its classification. The images that do not correspond to surrounding situation images include images that explain the actions permitted as the second task (hereinafter, ST explanation images), vehicle speed images, vehicle images, and lane marking images of the vehicle's lane (hereinafter, vehicle lane images).

[0135] When switching from sleep-enabling autonomous driving to sleep-preventing autonomous driving, the display control unit 106b increases the amount of driving-related information displayed on the display 91b during sleep-enabling autonomous driving compared to the amount of driving-related information displayed on the display 91b during sleep-enabling autonomous driving. In this case, the comparison may be made between the amount of information displayed on the same display device, or the amount of information displayed on a plurality of display devices combined. As described above, sleep-enabling autonomous driving is autonomous driving of LV4 or higher. In the following, LV4 autonomous driving will be described as an example. As described above, sleep-preventing autonomous driving is autonomous driving of LV3. The amount of information referred to here may be the amount of elements by type of information. For example, examples of elements by type of information include an image of the vehicle itself, an image of the vehicle's lane, an image of the dividing lines of the surrounding lanes (hereinafter, surrounding lane images), an image of surrounding vehicles in the vehicle's lane, an image of surrounding vehicles in the surrounding lanes, and a vehicle speed image.

[0136] For example, an example of displaying more information during LV3 autonomous driving than during LV4 autonomous driving can be as follows: If an image of the vehicle itself and an image of the lane itself are displayed during LV4 autonomous driving, but images of surrounding vehicles are not displayed, then an image of the surrounding vehicles can be displayed in addition to an image of the vehicle itself and an image of the lane itself during LV3 autonomous driving. Also, if an image of the vehicle itself is displayed during LV4 autonomous driving, but images of the lane itself are not displayed, then an image of the lane itself can be displayed in addition to an image of the vehicle itself during LV3 autonomous driving.

[0137] Even when switching from sleep-enabling automatic driving to driving at a level below automatic driving with monitoring obligation, the display control unit 106b may increase the amount of driving-related information displayed on the display 91b after switching to driving at a level below automatic driving with monitoring obligation, compared to the amount of driving-related information displayed on the display 91b during sleep-enabling automatic driving. Driving at a level below automatic driving with monitoring obligation includes automatic driving with automation levels 1 to 2 and manual driving with automation level 0. In this case, it is preferable that the display control unit 106b increase the amount of driving-related information displayed on the display 91b after switching to driving at a level below automatic driving with monitoring obligation, compared to the case of sleep-enabling automatic driving. This makes it possible to prevent the driver from neglecting to monitor the surroundings by focusing too much on the display when switching to driving at LV2 or below, which requires monitoring the surroundings.

[0138] For example, the following can be used to increase the amount of information displayed when driving at a level of automation below automated driving with monitoring obligation compared to when driving at a level of automated driving that allows sleep. If an image of the vehicle itself is displayed but an image of the lane itself is not displayed during automated driving at LV4, an image of the lane itself and images of surrounding vehicles can be displayed in addition to an image of the vehicle itself during driving at an automation level of LV2 or lower. In this case, a configuration can be used in which an image of the lane itself is displayed in addition to an image of the vehicle itself during automated driving at LV3.

[0139] During LV4 automatic driving, the display control unit 106b preferably increases the amount of driving-related information displayed on the display 91b when the state identification unit 107 identifies the driver as asleep, compared to the amount of driving-related information displayed on the display 91b when the state identification unit 107 identifies the driver as awake. This allows the passenger to check more detailed information about the vehicle's driving even when the driver is asleep during LV4 automatic driving. Therefore, it is possible to give the passenger a sense of security even when the driver is asleep during LV4 automatic driving. Here, the case where the driving-related information is displayed on the display 91b is taken as an example, but it is also applicable to the case where the driving-related information is displayed on the display 91.

[0140] For example, an example of displaying more information when the driver is asleep than when the driver is awake during LV4 autonomous driving may be as follows: When the driver is asleep, a vehicle speed image may be displayed but an image of the vehicle and an image of the lane may not be displayed, whereas when the driver is asleep, an image of the vehicle and an image of the lane may be displayed in addition to the vehicle speed image. In addition, when the driver is asleep, a vehicle speed image, an image of the vehicle and an image of the lane may be displayed but an image of vehicles in the lane may not be displayed, whereas when the driver is asleep, an image of vehicles in the lane may be displayed in addition to the vehicle speed image, an image of the vehicle and an image of the lane.

[0141] When the state identification unit 107 identifies the driver as asleep during LV4 automatic driving, the display control unit 106b preferably increases the amount of driving-related information displayed on the passenger side display 912 compared to the driver side display 911, compared to when the state identification unit 107 identifies the driver as awake. In this case, as an example, the driver side display 911 may display the same amount of driving-related information whether the state identification unit 107 identifies the driver as asleep or awake. On the other hand, when the state identification unit 107 identifies the driver as awake during LV4 automatic driving, the driver side display 911 and the passenger side display 912 may display the same amount of driving-related information. According to this, when the driver is asleep during LV4 automatic driving, it is possible to efficiently provide the passenger with information necessary for the passenger while eliminating unnecessary displays.

[0142] For example, the amount of information displayed may be varied depending on the state of the driver during LV4 autonomous driving as follows: When the driver is awake, a vehicle speed image may be displayed on both the driver's display 911 and the passenger's display 912, but an image of the vehicle and an image of the lane in which the vehicle is traveling may not be displayed; when the driver is asleep, a vehicle speed image may be displayed on the driver's display 911, but an image of the vehicle and an image of the lane in which the vehicle is traveling may not be displayed, whereas an image of the vehicle speed image may be displayed on the passenger's display 912, as well as an image of the vehicle and an image of the lane in which the vehicle is traveling.

[0143] When the state identification unit 107 identifies that the driver is not asleep during LV4 autonomous driving, it is preferable that the display control unit 106b change the display of information to correspond to the stage of LV3 autonomous driving after switching from LV4 autonomous driving to LV3 autonomous driving. When the driver is not asleep during LV4 autonomous driving, the driver can grasp the situation around the vehicle. Therefore, the driver can grasp the situation around the vehicle without increasing the amount of driving-related information displayed on the display 91b before switching to LV3 autonomous driving. Therefore, there is no problem in increasing the amount of driving-related information displayed on the display 91b after switching to LV3 autonomous driving.

[0144] On the other hand, when the state identification unit 107 identifies that the driver has transitioned from a sleep state to an awakened state during LV4 autonomous driving, the display control unit 106b preferably changes the display of information to correspond to the stage of LV3 autonomous driving after switching from LV4 autonomous driving to LV3 autonomous driving before switching from LV4 autonomous driving to LV3 autonomous driving. If the driver is asleep during LV4 autonomous driving, there is a possibility that the driver is unable to grasp the situation around the vehicle. Therefore, by increasing the amount of driving-related information displayed on the display 91b before switching to LV3 autonomous driving, the driver can easily grasp the situation around the vehicle. As a result, convenience for the driver is improved.

[0145] Here, an example of the flow of processing related to the control of the display from sleep-allowed autonomous driving to sleep-unallowed autonomous driving in the HCU 10b (hereinafter, second display control related processing) will be described with reference to the flowchart of Fig. 19. The flowchart of Fig. 19 may be configured to be started when the host vehicle starts autonomous driving of LV4 or higher, for example.

[0146] First, in step S41, the state identification unit 107 identifies the state of the driver. In step S42, if the driver is identified as asleep in S41 (YES in S42), the process proceeds to step S43. On the other hand, if the driver is identified as awake in S41 (NO in S42), the process proceeds to step S44.

[0147] In step S43, the display control unit 106b increases the amount of driving-related information displayed on the passenger side display 912 compared to the driver side display 911, and then proceeds to step S45. On the other hand, in step S44, the display control unit 106b displays the same amount of driving-related information on the driver side display 911 and the passenger side display 912, and then proceeds to step S45.

[0148] In step S45, if switching to LV3 autonomous driving is to occur (YES in S45), proceed to step S46. On the other hand, if switching to LV3 autonomous driving is not to occur (NO in S45), return to S41 and repeat the process. Switching to LV3 autonomous driving refers to a state in which switching is about to occur but has not yet started. Since LV3 autonomous driving is an autonomous driving that does not allow sleep, it is assumed that the driver is awake when switching to LV3 autonomous driving is to occur.

[0149] In step S46, if the driver has been determined to be asleep in S41 (YES in S46), the process proceeds to step S47. On the other hand, if the driver has not been determined to be asleep in S41 (NO in S46), the process proceeds to step S48.

[0150] In step S47, the display control unit 106b changes the display of information according to the stage of LV3 automatic driving after switching to automatic driving of LV3 before switching to automatic driving of LV3, and ends the second display control related process. On the other hand, in step S48, the display control unit 106b changes the display of information according to the stage of LV3 automatic driving after switching to automatic driving of LV3, and ends the second display control related process.

[0151] (Embodiment 7) The configuration is not limited to that of the sixth embodiment, and may be that of the seventh embodiment, which will be described below as an example. An example of the seventh embodiment will be described below with reference to the drawings. The vehicle system 1b of the seventh embodiment is similar to the vehicle system 1b of the sixth embodiment, except that the vehicle system 1b of the seventh embodiment includes an HCU 10c instead of the HCU 10b.

[0152] Here, the schematic configuration of the HCU 10c will be described with reference to FIG. 20. As shown in FIG. 20, the HCU 10c includes a changeover request acquisition unit 101, a mode identification unit 102, an interruption estimation unit 103, a lane change identification unit 104, a grip identification unit 105, a display control unit 106c, and a state identification unit 107 as functional blocks for controlling the display on the display device 91b. The HCU 10c is similar to the HCU 10b of the sixth embodiment, except that the HCU 10c includes a display control unit 106c instead of the display control unit 106b. This HCU 10c also corresponds to a display control device for a vehicle. In addition, the execution of the processing of each functional block of the HCU 10c by a computer also corresponds to the execution of a display control method for a vehicle.

[0153] The display control unit 106c is similar to the display control unit 106b, except for some different processing. The following describes processing different from that of the display control unit 106b. When switching from sleep-enabling automatic driving to sleep-preventing automatic driving, if the state identification unit 107 identifies the driver as being in an awake state from a predetermined time before the planned switching timing, the display control unit 106c changes the display of information according to the stage of automatic driving after switching from sleep-enabling automatic driving to sleep-preventing automatic driving after switching. On the other hand, if the state identification unit 107 identifies that the driver has transitioned from a sleep state to an awake state within a predetermined time before the planned switching timing, the display control unit 106c changes the display of information according to the stage of automatic driving after switching before switching from sleep-enabling automatic driving to sleep-preventing automatic driving. As described above, sleep-enabling automatic driving is automatic driving of LV4 or higher. The following describes LV4 automatic driving as an example. As described above, sleep-preventing automatic driving is automatic driving of LV3. The predetermined time may be set arbitrarily as long as it is equal to or longer than the time required for the driver to recognize the surroundings of the vehicle after transitioning from a sleep state to an awake state.

[0154] In the seventh embodiment, the process of S46 in the flowchart of Fig. 19 may be changed as follows. In the seventh embodiment, if the state identification unit 107 has continuously identified the state as awake from a predetermined time before the scheduled timing of switching to LV3 automatic driving in the process of S46, the process may proceed to step S47. On the other hand, if the state identification unit 107 has identified the state as asleep within a predetermined time before the scheduled timing of switching to LV3 automatic driving, the process may proceed to step S48.

[0155] (Embodiment 8) In the sixth and seventh embodiments, the HCUs 10b and 10c are provided with the state identification unit 107, but this is not necessarily limited to the above. For example, the HCUs 10b and 10c may not be provided with the state identification unit 107, and the display may not be controlled depending on whether the driver is awake or asleep.

[0156] The present disclosure is not limited to the above-described embodiment, and various modifications are possible within the scope of the claims. The embodiments obtained by appropriately combining the technical means disclosed in the different embodiments are also included in the technical scope of the present disclosure. The control unit and the method described in the present disclosure may be realized by a dedicated computer constituting a processor programmed to execute one or more functions embodied by a computer program. Alternatively, the device and the method described in the present disclosure may be realized by a dedicated hardware logic circuit. Alternatively, the device and the method described in the present disclosure may be realized by one or more dedicated computers configured by a combination of a processor that executes a computer program and one or more hardware logic circuits. The computer program may be stored in a computer-readable non-transient tangible recording medium as instructions executed by the computer. [Explanation of symbols]

[0157] 1, 1b Vehicle system (vehicle display control system), 10, 10a, 10b, 10c HCU (vehicle display control device), 91, 91b Display, 93 User input device, 102 Mode identification unit, 103 Interruption estimation unit, 104 Lane change identification unit, 105 Grasping identification unit, 106, 106a, 106b, 106c Display control unit, 107 State identification unit, 911 Driver's side display, 912 Passenger's side display

Claims

1. It is used in vehicles that can switch from automated driving without a monitoring obligation, which is automated driving without the driver's monitoring obligation, to automated driving with a monitoring obligation, which is automated driving with the driver's monitoring obligation. a display control unit (106, 106a, 106b, 106c) for causing a surrounding situation image, which is an image for showing a surrounding situation of the vehicle, to be displayed on a display device (91, 91b) used in a cabin of the vehicle; The display control unit is a display control device for a vehicle that, when the surrounding situation image is displayed during the autonomous driving without monitoring obligation and the autonomous driving level switches to a lower level of automation, changes the display of the surrounding situation image from the image corresponding to the autonomous driving level before the switch to the image corresponding to the autonomous driving level after the switch after a predetermined time has elapsed since the switch of the autonomous driving level.

2. In claim 1, The vehicle is used in a vehicle in which, as a stage of the autonomous driving, at least a supervision-free autonomous driving, which is an autonomous driving without a supervision obligation of the driver, and the supervision-obligated autonomous driving are switchable, and the vehicle is capable of at least a sleep-enabled autonomous driving, in which the driver is permitted to sleep, and a sleep-prohibited autonomous driving, in which the driver is not permitted to sleep, as the autonomous driving without supervision obligation, The display control unit (106b, 106c) causes the display device to display driving-related information related to the driving of the vehicle, The display control unit is a display control device for a vehicle that, when switching from the sleep-enabling autonomous driving to the sleep-disabling autonomous driving, increases the amount of driving-related information to be displayed on the display when the sleep-enabling autonomous driving is in progress compared to the amount of driving-related information to be displayed on the display when the sleep-disabling autonomous driving is in progress.

3. In claim 2, A state identification unit (107) for identifying the state of the driver, The display control unit (106b) changes the display of information to correspond to the stage of autonomous driving after switching from the sleep-enabling autonomous driving to the sleep-preventing autonomous driving when the state identification unit determines that the driver is not in a sleep-enabling autonomous driving state during the sleep-enabling autonomous driving, and changes the display of information to correspond to the stage of autonomous driving after switching before switching from the sleep-enabling autonomous driving to the sleep-preventing autonomous driving.

4. In claim 2, A state identification unit (107) for identifying the state of the driver, The display control unit (106c), in the case of switching from the sleep-enabling autonomous driving to the sleep-preventing autonomous driving, if the state identification unit has identified the driver as in an awake state before a predetermined time before the planned switch time, changes the display of information to correspond to the stage of autonomous driving after the switch from the sleep-enabling autonomous driving to the sleep-preventing autonomous driving, while if the state identification unit has identified that the driver has transitioned from a sleep state to an awake state within a predetermined time before the planned switch time, changes the display of information to correspond to the stage of autonomous driving after the switch before switching from the sleep-enabling autonomous driving to the sleep-preventing autonomous driving.

5. In any one of claims 1 to 4, The vehicle is used in a vehicle in which, as a stage of the autonomous driving, at least a supervision-free autonomous driving, which is an autonomous driving without a supervision obligation of the driver, and the supervision-obligated autonomous driving are switchable, and the vehicle is capable of at least a sleep-enabled autonomous driving, in which the driver is permitted to sleep, and a sleep-prohibited autonomous driving, in which the driver is not permitted to sleep, as the autonomous driving without supervision obligation, The display control unit (106b, 106c) causes the display device to display driving-related information related to the driving of the vehicle, A state identification unit (107) for identifying the state of the driver, The display control unit is a display control device for a vehicle that, during the sleep-enabling automatic driving, causes the display to display a greater amount of driving-related information when the state identification unit identifies the driver as being in a sleeping state than the amount of driving-related information that is displayed on the display when the state identification unit identifies the driver as being in an awake state.

6. In claim 5, The display control unit (106b, 106c) displays information on the display (91b), and controls display on a driver's side display (911) having a display surface located in front of the driver's seat of the vehicle, and a passenger's side display (912) having a display surface located in a place other than the driver's side display that is visible to passengers of the vehicle, The display control unit is a vehicle display control device that, when the state identification unit identifies the driver as being asleep during sleep-enabled automatic driving, increases the amount of driving-related information displayed on the passenger's side display compared to the driver's side display when the state identification unit identifies the driver as being in an awake state.

7. In any one of claims 1 to 6, The present invention is used in a vehicle that is capable of at least switching between autonomous driving without a monitoring obligation, which is autonomous driving without a driver's monitoring obligation, and autonomous driving with a monitoring obligation, and is capable of at least sleep-enabled autonomous driving, in which the driver is permitted to sleep, and sleep-prohibited autonomous driving, in which the driver is not permitted to sleep, as the autonomous driving without a monitoring obligation, The display control unit (106b, 106c) causes the display device to display driving-related information related to the driving of the vehicle, The display control unit is a display control device for a vehicle that, when automation switches from the sleep-enabled autonomous driving to a level of driving lower than the autonomous driving with monitoring obligation, increases the amount of driving-related information displayed on the display after switching to the level of driving lower than the autonomous driving with monitoring obligation, compared to when the sleep-enabled autonomous driving is in progress.

8. A display (91, 91b) provided in the vehicle so that a display surface faces an interior of the vehicle; A vehicle display control system including the vehicle display control device (10, 10a, 10b, 10c) according to any one of claims 1 to 7.

9. A display control method for a vehicle used in a vehicle capable of switching from automatic driving without a monitoring obligation, which is automatic driving without a driver's monitoring obligation, to automatic driving with a monitoring obligation, which is automatic driving with a driver's monitoring obligation, Executed by at least one processor, a display control step of displaying a surrounding situation image, which is an image for showing a surrounding situation of the vehicle, on a display (91, 91b) used in a passenger compartment of the vehicle; A mode identification process is included which identifies whether the vehicle is to perform the autonomous driving with a monitoring obligation in a hands-on mode that requires the driver to hold the steering wheel of the vehicle or in a hands-off mode that does not require the driver to hold the steering wheel, In the display control process, when the surrounding situation image is being displayed during the autonomous driving without the obligation to monitor, if the autonomous driving level switches to a lower level of automation, after a predetermined time has elapsed since the switch in the autonomous driving level, the display of the surrounding situation image corresponding to the autonomous driving level before the switch is changed to the display of the surrounding situation image corresponding to the autonomous driving level after the switch.

Images