Presentation control device and presentation control program

Abstract

To provide a presentation control device and the like capable of performing highly convenient information presentation.SOLUTION: An HCU 100 controls information display with respect to a driver of a vehicle which can execute automatic driving. The HCU 100 includes a driving state determination part 120 which discriminates between a specific action permission state for permitting a specific action other than driving with respect to a drive during automatic driving, a specific action prohibition state in which the specific action is prohibited during automatic driving, and an automatic driving prohibition state for prohibiting automatic driving. The HCU 100 includes a presentation information adjustment part 140 for presenting in a presentation manner in which first transition information regarding the transition to the specific action permission state from the automatic driving prohibition state is further stressed than second transition information regarding the transition to the specific action permission state from the specific action prohibition state.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The disclosure in this specification relates to a technology for controlling the presentation of information to a driver of a vehicle capable of autonomous driving. [Background technology]

[0002] Patent Document 1 discloses a technique for switching from manual driving to automatic driving. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2017 / 154396 Summary of the Invention [Problem to be solved by the invention]

[0004] However, depending on the control contents of the autonomous driving, there is a possibility that a situation may occur in which the driver is permitted to perform a specific action other than driving, such as a second task, and a situation in which the driver is prohibited from performing the specific action. Patent Document 1 does not describe the presentation of highly convenient information in response to such a transition of the situation.

[0005] An object of the disclosure is to provide a presentation control device and a presentation control program capable of providing highly convenient information presentation. [Means for solving the problem]

[0006] The various aspects disclosed in this specification employ different technical means to achieve their respective objectives. The reference symbols in parentheses in the claims and in this section are merely examples showing the corresponding relationship with the specific means described in the embodiments below as one aspect, and do not limit the technical scope.

[0007] One of the disclosed presentation control devices is a presentation control device that controls the presentation of information to a driver of a vehicle (A) capable of performing autonomous driving, a determination unit (120) that distinguishes between a specific action permitted state in which the driver is permitted to perform a specific action other than driving while autonomous driving is being performed, a hands-off permitted state in which the driver is prohibited from performing a specific action while autonomous driving is being performed and is permitted to release the hands from the steering wheel, and a both-prohibited state in which the driver is prohibited from performing a specific action and releasing the hands from the steering wheel while autonomous driving is being performed, and determines whether or not a transition between each state is possible and whether or not the transition is to be performed; a presentation control unit (140) that presents: both-prohibited transition possible information indicating that a transition from the both-prohibited state to the specific action permitted state is possible; both-prohibited transition execution information indicating the execution of a transition from the both-prohibited state to the specific action permitted state; and hands-free transition execution information indicating the execution of a transition from the hands-free permitted state to the specific action permitted state; Equipped with The presentation control unit presents the both-prohibited transition possible information prior to the both-prohibited transition execution information when transitioning from the both-prohibited state to the specific action permitted state, presents the both-prohibited transition execution information after receiving an instruction from the driver to start automatic driving in the specific action permitted state, and presents the hands-free transition execution information without a start instruction when transitioning from the hands-free permitted state to the specific action permitted state. 。

[0008] One of the disclosed presentation control programs is a presentation control program stored in a storage medium (101) and including instructions to be executed by a processor (102) in order to control presentation of information to a driver of a vehicle (A) capable of performing autonomous driving, The command is, A judgment process (S132, S133, S135) that distinguishes between a specific action permission state that permits the driver to perform a specific action other than driving while autonomous driving is being performed, a hands-off permission state that prohibits a specific action while autonomous driving is being performed and permits the driver to release the steering wheel, and both prohibition states that prohibit specific actions and hands-off while autonomous driving is being performed, and judges whether a transition between each state is possible and whether the transition should be performed; a presentation control process (S134, S136, S137) for presenting both-prohibited transition possible information indicating that a transition from both-prohibited states to a specific action permitted state is possible, both-prohibited transition execution information indicating the execution of a transition from both-prohibited states to a specific action permitted state, and hands-free transition execution information indicating the execution of a transition from the hands-free permitted state to the specific action permitted state; Including, In the presentation control process, when transitioning from a both-prohibited state to a specific action permitted state, the both-prohibited transition possible information is presented prior to the both-prohibited transition execution information, the both-prohibited transition execution information is presented after receiving an instruction from the driver to start autonomous driving in the specific action permitted state, and when transitioning from the hands-free permitted state to the specific action permitted state, the hands-free transition execution information is presented without a start instruction. 。

[0009] According to these disclosures, a presentation control device and a presentation control program capable of providing highly convenient information presentation can be provided.

[0010]

[0011]

[0012]

[0013]

[0014]

[0015] [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 illustrates a system including an HCU. [Diagram 2] 2 is a block diagram showing an example of functions of an HCU. FIG. [Diagram 3] FIG. [Figure 4] 10 is a time chart showing an example of a notification executed on each display device. [Diagram 5] FIG. 11 is a diagram illustrating an example of information presentation. [Figure 6]FIG. 11 is a diagram illustrating an example of information presentation. [Figure 7] FIG. 11 is a diagram illustrating an example of information presentation. [Figure 8] FIG. 11 is a diagram illustrating an example of information presentation. [Figure 9] FIG. 11 is a diagram illustrating an example of information presentation. [Figure 10] FIG. 11 is a diagram illustrating an example of information presentation. [Figure 11] FIG. 11 is a diagram illustrating an example of information presentation. [Figure 12] FIG. 11 is a diagram illustrating an example of information presentation. [Figure 13] FIG. 11 is a diagram illustrating an example of information presentation. [Figure 14] FIG. 11 is a diagram illustrating an example of information presentation. [Figure 15] FIG. 11 is a diagram illustrating an example of information presentation. [Figure 16] FIG. 11 is a diagram illustrating an example of information presentation. [Figure 17] FIG. 11 is a diagram illustrating an example of information presentation. [Figure 18] FIG. 11 is a diagram illustrating an example of information presentation. [Figure 19] 11 is a flowchart showing an example of a presentation control method executed by the HCU. [Figure 20] 11 is a flowchart showing an example of a presentation control method executed by the HCU. [Figure 21] 11 is a flowchart showing an example of a presentation control method executed by the HCU. [Figure 22] 11 is a flowchart showing an example of a presentation control method executed by the HCU. [Diagram 23] 11 is a flowchart showing an example of a presentation control method executed by the HCU. [Figure 24] 11 is a flowchart showing an example of a presentation control method executed by the HCU. [Diagram 25] 13 is a flowchart showing an example of a presentation control method executed by an HCU in the second embodiment. [Figure 26] 13 is a flowchart showing an example of a presentation control method executed by an HCU in the third embodiment. [Figure 27] 13 is a flowchart showing an example of a presentation control method executed by an HCU in the fifth embodiment. [Figure 28] 23 is a flowchart showing an example of a presentation control method executed by an HCU in the sixth embodiment. [Figure 29] 23 is a flowchart showing an example of a presentation control method executed by an HCU in the seventh embodiment. [Diagram 30] 23 is a flowchart showing an example of a presentation control method executed by an HCU in the eighth embodiment. [Diagram 31] 23 is a flowchart showing an example of a presentation control method executed by an HCU in the eighth embodiment. [Diagram 32] 13 is a flowchart showing an example of a presentation control method executed by an HCU in the ninth embodiment. [Diagram 33] 13 is a flowchart showing an example of a presentation control method executed by an HCU in the ninth embodiment. [Diagram 34] 34 is a flowchart continuing from the flowchart of FIG. 33. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] (First embodiment) The presentation control device of the first embodiment will be described with reference to Figs. 1 to 24. The presentation control device of the first embodiment is provided by an HCU (Human Machine Interface Control Unit) 100 mounted on a vehicle A. The HCU 100 constitutes an HMI (Human Machine Interface) system used in the vehicle A together with a plurality of display devices, an audio device 24, an operation device 26, and the like. The HMI system has an input interface function for accepting operations by an occupant (e.g., a driver, etc.) of the vehicle A, and an output interface function for presenting information to the driver. The HCU 100 is connected to a locator 30, a surroundings monitoring sensor 40, an in-vehicle communication device 50, a first automatic driving ECU 60, a second automatic driving ECU 70, a DSM 27, and a vehicle control ECU 80 via a communication bus 99, etc.

[0021] Locator 30 generates vehicle position information and the like by composite positioning that combines multiple pieces of acquired information. Locator 30 includes a GNSS (Global Navigation Satellite System) receiver 31, an inertial sensor 32, a map database (hereinafter, "map DB") DB 33, and a locator ECU 34. GNSS receiver 31 receives positioning signals from multiple positioning satellites. Inertial sensor 32 is a sensor that detects an inertial force acting on vehicle A. Inertial sensor 32 includes, for example, a gyro sensor and an acceleration sensor.

[0022] The map DB 33 is a non-volatile memory and stores map data such as link data, node data, road shapes, structures, etc. The map data may be a three-dimensional map consisting of a point cloud of characteristic points of road shapes and structures. The three-dimensional map may be generated based on captured images by REM (Road Experience Management). The map data may also include traffic regulation information, road construction information, weather information, traffic signal information, etc. The map data stored in the map DB is updated periodically or as needed based on the latest information received by the in-vehicle communication device 50 described later.

[0023] The locator ECU 34 is mainly configured to include a microcomputer equipped with a processor, a memory, an input / output interface, and a bus connecting these. The locator ECU 34 sequentially locates the position of the vehicle A (hereinafter, the vehicle position) by combining the positioning signal received by the GNSS receiver 31, the map data of the map DB 33, and the measurement results of the inertial sensor 32. The vehicle position may be expressed by, for example, latitude and longitude coordinates. The vehicle position may be located using a travel distance calculated from a signal sequentially output from a vehicle speed sensor mounted on the vehicle A. When a three-dimensional map consisting of a point cloud of characteristic points of road shapes and structures is used as the map data, the locator ECU 34 may be configured to identify the vehicle position using the three-dimensional map and the detection results of the periphery monitoring sensor 40 without using the GNSS receiver 31.

[0024] The perimeter monitoring sensor 40 is an autonomous sensor that monitors the environment around the vehicle A. The perimeter monitoring sensor 40 can detect moving objects such as pedestrians, cyclists, non-human animals, and other vehicles from a detection range around the vehicle A, as well as fallen objects on the road, road markings such as guardrails, curbs, road signs, and lane lines, and stationary objects such as roadside structures. The perimeter monitoring sensor 40 provides detection information on objects around the vehicle A to the first automatic driving ECU 60, the second automatic driving ECU 70, and the like via the communication bus 99.

[0025] The perimeter monitoring sensor 40 has a front camera 41 and a millimeter wave radar 42 as detection components for object detection. The front camera 41 outputs at least one of imaging data of the front range of the vehicle A and an analysis result of the imaging data as detection information. For example, a plurality of millimeter wave radars 42 are arranged at intervals from each other on each of the front and rear bumpers of the vehicle A. The millimeter wave radar 42 irradiates millimeter waves or quasi-millimeter waves toward the front range, front side range, rear range, rear side range, etc. of the vehicle A. The millimeter wave radar 42 generates detection information by processing to receive reflected waves reflected by moving objects, stationary objects, etc. Note that other detection components such as LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging) that detects a point cloud of feature points of the ground, and sonar that receives reflected waves of ultrasonic waves may be included in the perimeter monitoring sensor 40.

[0026] The in-vehicle communication device 50 is a communication module mounted on the vehicle A. The in-vehicle communication device 50 has at least a function of V2N (Vehicle to cellular Network) communication according to communication standards such as LTE (Long Term Evolution) and 5G, and transmits and receives radio waves to and from base stations around the vehicle A. The in-vehicle communication device 50 may further have functions such as vehicle to roadside infrastructure (hereinafter "V2I") communication and vehicle to vehicle (hereinafter "V2V") communication. The in-vehicle communication device 50 enables cooperation between the cloud and the in-vehicle system (Cloud to Car) through V2N communication. By mounting the in-vehicle communication device 50, the vehicle A becomes a connected car that can be connected to the Internet. The in-vehicle communication device 50 acquires traffic congestion information distributed from a traffic information center or the like, and provides the information to the second automatic driving ECU 70, the HCU 100, and the like.

[0027] The first automatic driving ECU 60 and the second automatic driving ECU 70 are mainly composed of a computer equipped with a processor 62, 72, a memory 61, 71, an input / output interface, and a bus connecting these, etc. The first automatic driving ECU 60 and the second automatic driving ECU 70 are ECUs capable of executing automatic driving control that controls part or substantially all of the driving of the vehicle A.

[0028] The first automatic driving ECU 60 has a partial automatic driving function that partially takes over the driving operation of the driver. The second automatic driving ECU 70 has an automatic driving function that can take over the driving operation of the driver. As an example, in the automatic driving levels defined by the Society of Automotive Engineers, the first automatic driving ECU 60 enables partial automatic driving control (advanced driving assistance) of level 2 or lower. In other words, the first automatic driving ECU 60 makes it possible to implement automatic driving control that requires the driver to monitor the surroundings. In other words, the first automatic driving ECU 60 makes it possible to implement automatic driving in which the second task described below is prohibited.

[0029] For example, the first automatic driving ECU 60 can execute one or both of longitudinal control and lateral control of vehicle A. Here, the longitudinal direction is the direction that coincides with the front-to-rear direction of vehicle A, and the lateral direction is the direction that coincides with the width direction of vehicle A. The first automatic driving ECU 60 executes control of acceleration and deceleration of vehicle A as longitudinal control. Also, the first automatic driving ECU 60 executes steering angle control of the steering wheels of vehicle A as lateral control.

[0030] The first automatic driving ECU 60 constructs a plurality of functional units that realize the above-mentioned advanced driving assistance by causing the processor 62 to execute a plurality of commands in accordance with the driving assistance program stored in the memory 61. Specifically, as shown in Fig. 2, the first automatic driving ECU 60 constructs an environment recognition unit 63, an ACC control unit 64, an LTA control unit 65, etc. as functional units.

[0031] The environment recognition unit 63 recognizes the driving environment around the vehicle A based on the detection information acquired from the periphery monitoring sensor 40. The environment recognition unit 63 provides the analysis result of the detection information carried out for the driving environment recognition to the ACC control unit 64 and the LTA control unit 65 as analyzed detection information. As an example, the environment recognition unit 63 generates information (lane information) indicating the relative positions and shapes of the left and right dividing lines or road edges of the lane in which the vehicle A is currently traveling (hereinafter, the current lane) as analyzed detection information. In addition, the environment recognition unit 63 generates information (preceding vehicle information) indicating the presence or absence of a preceding vehicle preceding the vehicle A in the current lane and, if there is a preceding vehicle, its position and speed as analyzed detection information. The environment recognition unit 63 sequentially provides the preceding vehicle information to the ACC control unit 64 and sequentially provides the lane information to the LTA control unit 65. The environment recognition unit 63 may be configured to recognize the MD area, AD area, ST section, and non-ST section described later.

[0032] The ACC control unit 64 executes ACC (Adaptive Cruise Control) control to realize constant speed driving of the vehicle A at a target speed or driving to follow the preceding vehicle based on the preceding vehicle information. The LTA control unit 65 executes LTA (Lane Tracing Assist) control to keep the vehicle A traveling within the lane based on the lane information. Specifically, each of the control units 64, 65 generates control commands for acceleration / deceleration or steering angle, and sequentially provides them to the vehicle control ECU 80 described later. The ACC control is an example of longitudinal control, and the LTA control is an example of lateral control.

[0033] The first automatic driving ECU 60 executes both the ACC control and the LTA control to realize automatic driving of Level 2. Note that the first automatic driving ECU 60 may be capable of realizing automatic driving of Level 1 by executing either the ACC control or the LTA control.

[0034] On the other hand, the second automatic driving ECU 70 enables automatic driving control of level 3 or higher among the above-mentioned automatic driving levels. That is, the second automatic driving ECU 70 enables automatic driving in which the driver is permitted to suspend surrounding monitoring. In other words, the second automatic driving ECU 70 enables automatic driving in which the second task is permitted.

[0035] Here, the second task is an action other than driving permitted to the driver, and is a specific action that is specified in advance. During the automatic driving period in which the vehicle A is automatically driven by the level 3 automatic driving function of the second automatic driving ECU 70, the driver in this case is a person (passenger) who takes over the driving control from the automatic driving system when leaving a limited area or in an emergency. The driver may be legally permitted to perform the second task until a request to perform driving operation by the automatic driving system, that is, a request to take over driving (Take Over Request) occurs.

[0036] The second task may be called a secondary activity or other activity. The second task must not prevent the driver from responding to a request from the automated driving system to take over the driving operation. As an example, watching content such as a video, operating a smartphone, reading, eating, and other activities are assumed as the second task.

[0037] The second automatic driving ECU 70 constructs a plurality of functional units that realize the above-mentioned automatic driving by causing the processor 72 to execute a plurality of commands by the automatic driving program stored in the memory 71. Specifically, the second automatic driving ECU 70 constructs an environment recognition unit 73, a behavior determination unit 74, a trajectory generation unit 75, etc. as functional units.

[0038] The environment recognition unit 73 recognizes the driving environment around the vehicle A based on the detection information acquired from the perimeter monitoring sensor 40, the vehicle position and map data acquired from the locator ECU 34, the communication information acquired from the in-vehicle communication device 50, etc. As an example, the environment recognition unit 73 recognizes the current lane position of the vehicle A, the shape of the current lane, and the relative positions and relative speeds of moving objects around the vehicle A, etc. The environment recognition unit 73 sequentially provides the above recognition results to the action determination unit 74 and the trajectory generation unit 75.

[0039] In addition, the environment recognition unit 73 distinguishes between manual driving areas (MD areas) and automatic driving areas (AD areas) in the driving area of ​​vehicle A, and between ST sections and non-ST sections in the AD areas, and sequentially provides the recognition results to the HCU 100.

[0040] The MD area is an area where automated driving is prohibited. In other words, the MD area is an area where the driver is required to perform all of the longitudinal control, lateral control, and periphery monitoring of the vehicle A. For example, the MD area is an area where the driving route is a general road.

[0041] The AD area is an area where autonomous driving is permitted. In other words, the AD area is an area where vehicle A can take over one or more of longitudinal control, lateral control, and perimeter monitoring. For example, the AD area is an area where the driving road is an expressway or a motorway.

[0042] AD areas are divided into non-ST sections where automated driving of level 2 or lower is possible, and ST sections where automated driving of level 3 or higher is possible. Note that in the first embodiment, it is assumed that non-ST sections where automated driving of level 1 is permitted are equivalent to non-ST sections where automated driving of level 2 is permitted.

[0043] An ST section is, for example, a driving section where congestion occurs (congested section). Also, an ST section is, for example, a driving section for which a high-precision map has been developed. The surrounding condition grasping unit 110 determines that the section is an ST section when the driving speed of vehicle A continues to be within a threshold range for a predetermined period of time. Alternatively, the surrounding condition grasping unit 110 may determine whether the section is an ST section by combining the vehicle position and congestion information acquired from the in-vehicle communication device 50. Note that, in addition to congested sections, the surrounding condition grasping unit 110 may also determine that a section where specific conditions other than congestion are satisfied with respect to the surrounding environment of vehicle A is an ST section.

[0044] The behavior determination unit 74 determines a future behavior scheduled for the vehicle A based on the recognition result of the driving environment. Specifically, when the behavior determination unit 74 has acquired an instruction to start autonomous driving in cooperation with the HCU 100 described later, the behavior determination unit 74 determines a type of behavior (e.g., going straight, turning right, turning left, changing lanes, etc.) that the vehicle A should take to arrive at the destination as a future behavior. In addition, when the behavior determination unit 74 determines that it is necessary to transfer the driving control to the driver, it generates a handover request and provides it to the HCU 100.

[0045] The trajectory generation unit 75 generates a driving trajectory of the vehicle A in a section where autonomous driving is possible, based on the recognition result of the driving environment and the determined future action. The driving trajectory includes, for example, a target position of the vehicle A according to the progress and a target speed at each target position. The trajectory generation unit 75 sequentially provides the generated driving trajectory to the vehicle control ECU 80 as a control command to be followed by the vehicle A during autonomous driving.

[0046] The autonomous driving system including the autonomous driving ECUs 60, 70 described above enables autonomous driving equivalent to at least level 2 and level 3 to be performed in vehicle A. In what follows, the state in which autonomous driving equivalent to level 2 is being performed may be referred to as "level 2 execution mode," and the state in which autonomous driving equivalent to level 3 is being performed may be referred to as "level 3 execution mode."

[0047] The vehicle control ECU 80 is an electronic control device that performs acceleration / deceleration control and steering control of the vehicle A. Examples of the vehicle control ECU 80 include a steering ECU that performs steering control, a power unit control ECU that performs acceleration / deceleration control, and a brake ECU. The vehicle control ECU 80 acquires detection signals output from each sensor mounted on the vehicle A, such as a steering angle sensor and a vehicle speed sensor, and outputs control signals to each driving control device, such as an electronically controlled throttle, a brake actuator, and an EPS (Electric Power Steering) motor. The vehicle control ECU 80 acquires a control instruction for the vehicle A from the first automatic driving ECU 60 or the second automatic driving ECU 70, and controls each driving control device to realize automatic driving according to the control instruction.

[0048] The vehicle control ECU 80 is also connected to an on-board sensor 81 that detects driving operation information of the driving members by the driver. The on-board sensor 81 includes, for example, a pedal sensor that detects the amount of depression of the accelerator pedal, a steer sensor that detects the amount of steering, etc. The vehicle control ECU 80 sequentially provides the detected driving operation information to the HCU 100.

[0049] The DSM 27 includes a near-infrared light source, a near-infrared camera, and a control unit for controlling them. The DSM 27 is installed, for example, on the upper surface of the steering column or the upper surface of the instrument panel 9 with the near-infrared camera facing the headrest of the driver's seat. The DSM 27 uses the near-infrared camera to capture an image of the driver's head illuminated with near-infrared light from the near-infrared light source. The image captured by the near-infrared camera is analyzed by the control unit. The control unit extracts information such as the position and line of sight of the driver from the captured image, and provides the extracted driver's state information to the HCU 100, etc. via the communication bus 99.

[0050] Next, the multiple display devices, the audio device 24, the operation device 26, and the HCU 100 included in the HMI system will be described in detail.

[0051] The multiple display devices include a head-up display (hereinafter, HUD) 21, a meter display 22, and a center information display (hereinafter, CID) 23. The multiple display devices may further include displays EMB, EML, and EMR of an electronic mirror system. The HUD 21, the meter display 22, and the CID 23 are displays that present image content, such as still images or videos, to the driver as visual information.

[0052] Based on the control signal and image data acquired from the HCU 100, the HUD 21 projects the light of the image formed in front of the driver onto a projection area PA defined on the windshield WS or the like. The light of the image reflected by the windshield WS into the interior of the vehicle is perceived by the driver sitting in the driver's seat. In this way, the HUD 21 displays a virtual image in the space in front of the projection area PA. The driver visually recognizes the virtual image within the angle of view VA displayed by the HUD 21 superimposed on the foreground of the vehicle A.

[0053] The meter display 22 and the CID 23 are mainly configured to include, for example, a liquid crystal display or an OLED (Organic Light Emitting Diode) display. The meter display 22 and the CID 23 display various images on the display screen based on the control signal and video data acquired from the HCU 100. The meter display 22 is installed, for example, in front of the driver's seat. The CID 23 is provided in a central area in the vehicle width direction in front of the driver. For example, the CID 23 is installed above the center cluster in the instrument panel 9. The CID 23 has a touch panel function and detects, for example, touch operations and swipe operations on the display screen by the driver or the like. The CID 23 is an example of a "center display".

[0054] The audio device 24 has a plurality of speakers installed in the vehicle cabin. Based on the control signal and voice data acquired from the HCU 100, the audio device 24 presents an alarm sound, a voice message, or the like to the driver as auditory information. That is, the audio device 24 is an information presentation device capable of presenting information in a manner different from visual information. Auditory information is an example of "information in a specific manner different from visual information."

[0055] The operation device 26 is an input unit that accepts a user device by a driver or the like. User operations related to starting and stopping each level of an autonomous driving function, for example, are input to the operation device 26. The operation device 26 includes, for example, a steering switch provided on the spokes of the steering wheel, an operation lever provided on the steering column, and a voice input device that recognizes the contents of the driver's speech.

[0056] The HCU 100 controls the presentation of information to the driver based on information from the above-mentioned first automatic driving ECU 60 and second automatic driving ECU 70, etc. The HCU 100 mainly includes a computer equipped with a memory 101, a processor 102, an input / output interface, and a bus connecting these. The processor 102 is hardware for arithmetic processing. The processor 102 includes at least one type of core among a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a RISC (Reduced Instruction Set Computer)-CPU, for example.

[0057] The memory 101 is at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, an optical medium, etc., that non-temporarily stores or stores computer-readable programs and data, etc. The memory 101 stores various programs executed by the processor 102, such as a presentation control program described below.

[0058] The processor 102 executes a plurality of instructions included in the presentation control program stored in the memory 101. In this way, the HCU 100 constructs a plurality of functional units for controlling presentation to the driver. In this manner, in the HCU 100, the presentation control program stored in the memory 101 causes the processor 102 to execute a plurality of instructions, thereby constructing a plurality of functional units. Specifically, in the HCU 100, as shown in FIG. 2, functional units such as a surrounding state grasping unit 110, a driving state determining unit 120, a driver state estimating unit 130, and a presentation information adjusting unit 140 are constructed.

[0059] The surrounding state grasping unit 110 acquires the recognition result of the driving environment from the environment recognition unit 63 of the first automatic driving ECU 60 or the environment recognition unit 73 of the second automatic driving ECU 70. The surrounding state grasping unit 110 grasps the surrounding state of the vehicle A based on the acquired recognition result. Specifically, the surrounding state grasping unit 110 grasps the approach to the AD area, the entry into the AD area, the approach to the ST section, the entry into the ST section, and the like. The surrounding state grasping unit 110 sequentially provides the grasped surrounding state information to the driving state determination unit 120. Note that the surrounding state grasping unit 110 may grasp the surrounding state based on information acquired directly from the locator ECU 34, the surrounding monitoring sensor 40, and the like, instead of the recognition result acquired from each automatic driving ECU 60, 70.

[0060] Driver State Estimation The driver state estimation unit 130 estimates the driver state based on information from the DSM 27, the vehicle control ECU 80, etc. For example, the driver state estimation unit 130 estimates whether or not each body part of the driver is involved in the driving operation as the driver state. Specifically, the driver state estimation unit 130 determines whether or not the driver's eyes are monitoring the surroundings based on state information related to the line of sight of the driver acquired from the DSM 27, etc. Also, the driver state estimation unit 130 determines whether or not the driver is holding the steering wheel with his / her hands based on the steering amount acquired from the vehicle control ECU 80. In addition, the driver state estimation unit 130 determines whether or not the driver has his / her foot on the accelerator pedal based on the accelerator pedal depression amount acquired from the vehicle control ECU 80.

[0061] In the following description, the state in which the eyes are monitoring the surroundings may be referred to as eyes on, and the state in which they are not monitoring the surroundings may be referred to as eyes off. In addition, the state in which the hands are gripping the steering wheel may be referred to as hands on, and the state in which the hands are not gripping the steering wheel may be referred to as hands off. In addition, the state in which the feet are placed on the accelerator pedal may be referred to as legs on, and the state in which the feet are not placed on the accelerator pedal may be referred to as legs off. The driver state estimation unit 130 sequentially provides the estimated state of each body part to the driving state determination unit 120. The driver state estimation unit 130 may also determine whether or not the driver conditions described below are satisfied.

[0062] The driving state determination unit 120 determines the driving state related to autonomous driving in cooperation with the second autonomous driving ECU 70 and the first autonomous driving ECU 60. Specifically, the driving state determination unit 120 determines the autonomous driving level permitted to the driver and the autonomous driving level to be actually executed as the driving state.

[0063] Based on information from the surrounding condition grasping unit 110, the driving state judgment unit 120 determines that autonomous driving cannot be permitted when vehicle A is traveling in an MD area. On the other hand, the driving state judgment unit 120 determines that autonomous driving of level 2 or higher can be permitted when vehicle A is traveling in an AD area. Furthermore, the driving state judgment unit 120 determines that autonomous driving of level 2 can be permitted when vehicle A is traveling in a non-ST section of the AD area, and determines that autonomous driving of level 3 can be permitted when vehicle A is traveling in an ST section.

[0064] In the following, a state in which level 2 autonomous driving is permitted may be referred to as a "level 2 permitted state," and a state in which level 3 autonomous driving is permitted may be referred to as a "level 3 permitted state." Additionally, a state in which all autonomous driving at levels 1 and above is prohibited may be referred to as an "autonomous driving prohibited state." The level 2 permitted state is an example of a "specific action prohibited state," and the level 3 permitted state is an example of a "specific action permitted state."

[0065] In addition, when it is determined that level 2 autonomous driving is permitted, the driving state determination unit 120 determines whether or not to permit hands-off at level 2. Specifically, the driving state determination unit 120 determines whether or not to permit hands-off when specific conditions are met based on the execution state of LTA, the presence or absence of high-precision map data around vehicle A, lane conditions, the driver's surrounding monitoring state, the road shape around vehicle A, and the like.

[0066] The specific conditions include, for example, at least one of the following: LTA control is being executed; there is high-precision map data around vehicle A; at least one of the left and right dividing lines of the current lane can be detected; it can be determined that the driver is monitoring the surroundings; and the driving section is not a section with a complex road structure (for example, a merging section or a branching section). In the following, a state in which hands-off is permitted may be referred to as a "hands-off permitted state," and a state in which hands-off is prohibited may be referred to as a "hands-off prohibited state." The hands-off permitted state is an example of a "hands-off permitted state," and the hands-off prohibited state is an example of a "hands-off prohibited state."

[0067] Furthermore, the driving state determination unit 120 permits leg off when at least ACC is being executed in autonomous driving of level 2 or lower. In the following, a state in which leg off is permitted may be referred to as a "leg off permitted state," and a state in which leg off is prohibited may be referred to as a "leg off prohibited state." In addition, the driving state determination unit 120 permits eyes off when level 3 autonomous driving is permitted. In other words, the level 3 permitted state can also be referred to as an eyes off permitted state.

[0068] The driving state determination unit 120 determines the autonomous driving level to actually be executed based on the currently permitted autonomous driving level, the driver's state information, input information to the operation device 26, and the like. That is, when an instruction to start the currently permitted autonomous driving level is acquired as input information, the driving state determination unit 120 decides to execute the autonomous driving level. However, when level 3 autonomous driving is permitted while level 2 autonomous driving is being executed and the vehicle is in a hands-on state, the driving state determination unit 120 determines to execute level 3 without acquiring input information. The above driving state determination unit 120 is an example of a "discrimination unit."

[0069] The presentation information adjustment unit 140 has, as sub-functional units, a presentation information selection unit 141, a content arbitration unit 142, and a presentation information generation unit 143. The presentation information adjustment unit 140 is an example of a "presentation control unit."

[0070] The presentation information selection unit 141 controls the presentation of content related to autonomous driving in cooperation with the content arbitration unit 142, based on information acquired from the surrounding state grasping unit 110, the driving state determination unit 120, and the driver state estimation unit 130. Specifically, the presentation information selection unit 141 selects content to be presented on each presentation device based on various information.

[0071] The content arbitration unit 142 arbitrates the contents to be displayed on each display device. The content arbitration unit 142 comprehensively judges the priority of each content based on various information from the presentation information selection unit 141. The content arbitration unit 142 selects the content that is judged to have a high priority as the content to be presented. In addition, the content arbitration unit 142 can sequentially change the display size and display layout of each content to be displayed on each display device according to the priority. As an example, the content arbitration unit 142 increases the display size of the content with a higher priority. As another example, the content arbitration unit 142 positions the content with a higher priority toward the front of each display area.

[0072] The presentation information generation unit 143 generates a control signal and video data to be provided to each display device, and a control signal and audio data to be provided to the audio device 24, based on the selection result of the presentation information selection unit 141 and the arbitration result of the content arbitration unit 142. The presentation information generation unit 143 outputs the generated control signal and each data to each presentation device, thereby presenting information on each presentation device.

[0073] Next, the contents presented by the HCU 100 will be described below with reference to Fig. 5 to Fig. 18 while referring to the time chart of Fig. 4. The HCU 100 performs notifications such as AD area approach notification, AD area entry notification, level 2 start notification, level 3 approach notification, driver preparation notification, level 3 possible notification, level 3 start notification, and level 3 end notification in relation to autonomous driving.

[0074] The AD area approach notification is issued when the vehicle A approaches an AD area while traveling in an MD area. Specifically, the AD area approach notification is issued when the surrounding state grasping unit 110 determines that the distance from the vehicle A to the AD area is below a threshold distance. In the AD area approach notification, the presentation information adjustment unit 140 displays AD area approach messages N1h and N1m on the HUD 21 and the meter display 22, as shown in FIG.

[0075] The AD area approaching messages N1h, N1m are image contents that visually indicate the approach to an AD area. The AD area approaching message N1h displayed on the HUD 21 is displayed within the field of view VA together with the digital speedometer and the driving status STh, etc. The AD area approaching message N1h is positioned to the left or right of the driving status STh that is positioned in the central area within the field of view VA. The AD area approaching message N1h includes a message image and a frame-shaped window image that surrounds the message image. The message image is text information that describes the approach of the AD area, such as "You will soon be in an area where automatic driving is possible."

[0076] The AD area approaching message N1m displayed on the meter display 22 is displayed within the display screen together with the driving status STm, etc. The AD area approaching message N1m is arranged, for example, at the lower edge of the display screen. The AD area approaching message N1m is displayed in front of the driving status STm. The AD area approaching message N1m includes a message image and a band-shaped window image in which the message image is arranged in front. The message image, like the HUD 21, is text information indicating the approach of an AD area. The window image is displayed in a color (for example, blue) different from the background of the driving status STm.

[0077] The above AD area approach notification is implemented regardless of the level of autonomous driving available after entering the AD area. That is, as shown in Figure 4, the AD area approach notification is implemented not only when approaching an AD area where level 2 is possible from an MD area, but also when approaching an AD area where level 3 or higher autonomous driving is possible from an MD area.

[0078] The AD area entry notification is implemented when the surrounding condition grasping unit 110 determines that the vehicle A has entered the AD area after the AD area approach notification. In the AD area entry notification, the presentation information adjustment unit 140 displays the AD area entry messages N2h and N2m on the HUD 21 and the meter display 22, and displays the AD area notification content CTMc on the CID 23, as shown in FIGS.

[0079] The AD area entry messages N2h, N2m, and N2c are image contents that visually indicate entry into an AD area. The display manner of the AD area entry messages N2h and N2m displayed on the HUD 21 and the meter display 22 is changed when a predetermined change time has elapsed since the start of display. In the following, the AD area entry messages before the change are referred to as first AD area entry messages N2fh and N2fm, and the AD area entry messages after the change are referred to as second AD area entry messages N2sh and N2sm.

[0080] The first AD area entry message N2fh displayed on the HUD 21 is, for example, disposed at substantially the same position as the AD area approaching message N1h, and is configured to include a message image and a frame-shaped window image, similar to the AD area approaching message N1h. The message image is character information related to entry into the AD area, and for example, a message indicating a method for starting autonomous driving is described. Specifically, the message image includes the description "autonomous driving mode", the description "ENTER" indicating input to the operation device 26, and an arrow pointing to the "ENTER". As a result, the first AD area approaching message N1fh notifies the driver that the autonomous driving mode can be started by inputting to the operation device 26, and thus indicates to the driver that the vehicle A has entered an AD area where autonomous driving is possible.

[0081] The first AD area entry message N2fm displayed on the meter display 22 is, for example, arranged in substantially the same position as the AD area approaching message N1m, and similar to the AD area approaching message N1m, is configured to include a message image and a strip-shaped window image with the message image arranged in front. The message image is text information related to entry into the AD area, similar to the first AD area entry message N2fh. Specifically, the message image includes the description "Autonomous driving mode used", the description "ENTER" indicating input to the operation device 26, and an arrow pointing to the "ENTER".

[0082] The above-mentioned first AD area entry message N2fh, N2fm is changed to the second AD area entry message N2sh, N2sm after a predetermined change time (for example, several seconds) has elapsed. The second AD area entry message N2sh, N2sm is made to have a display mode with lower visibility than the first AD area entry message N2fh, N2fm. Specifically, the second AD area entry message N2sh, N2sm is made to have a display mode with lower visibility by applying at least one of the following: reduction in display size, change in display color, change in display position, reduction in display brightness, non-display of a part, etc. For example, the second AD area entry message N2sh displayed on the HUD 21 has its window image hidden, its display position changed to a lower display position within the angle of view VA, and its display size is further reduced. Also, the second AD area entry message N2sm displayed on the meter display 22 has its message image and window image hidden, and its display size is reduced.

[0083] The AD area notification content CTMc displayed on CID23 is a map image displayed in a different display mode from that before the AD area notification in the map display area Am. For example, the AD area notification content CTMc is a map image that distinguishes the AD area from the MD area among the surrounding areas of the vehicle A. Specifically, it is an area image displayed in a different display color. When it is determined that the vehicle A has entered the AD area, the AD area notification image N2c starts to be displayed substantially at the same time as the first AD area entry message N2fh, N2fm. The display mode of the AD area notification content CTMa is not changed even after a predetermined change time has elapsed.

[0084] Of the above AD area entry messages N2h, N2m, and CTMc, the AD area entry messages N2h, N2m displayed on the HUD 21 and the meter display 22 are displayed for a display allowable time (for example, about 5 to 10 seconds). If no input information for starting autonomous driving is acquired from the operation device 26 during the display allowable time, the AD area entry messages N2h, N2m are hidden after the allowable time has elapsed. On the other hand, the AD area notification content CTMc continues to be displayed even after the display allowable time has elapsed (see FIG. 8). In other words, after the display allowable time has elapsed, the entry notification is continued on fewer display devices. The AD area notification content CTMc continues to be displayed, for example, until input information for starting autonomous driving is acquired.

[0085] The level 2 start notification is implemented when the driving state determination unit 120 determines that level 2 autonomous driving has started. In the level 2 start notification, the presentation information adjustment unit 140 causes level 2 start messages N3h and N3m to be displayed on the HUD 21 and the meter display 22, as shown in FIG.

[0086] The level 2 start messages N3h and N3m are image contents that visually indicate the start of level 2 autonomous driving. The level 2 start messages N3h and N3m are configured to include, for example, a message image in which text information related to the start of autonomous driving is described, and a window image that partitions the message image within the display area. The message image is a description that indicates that the autonomous driving that is about to start may be canceled, such as "Autonomous driving will be canceled due to surrounding road conditions." Each level 2 start message N3h and N3m is located in the center of the display area. Each level 2 start message N3h and N3m is displayed in front of the driving status STh and STm.

[0087] The level 3 approach notification notifies the approach of an ST section where level 3 automated driving is possible. The level 3 approach notification is executed when the remaining distance from vehicle A to the ST section falls below a threshold (e.g., about 10 km) while traveling in a non-ST section of an MD area or AD area. In the level 3 approach notification, the presentation information adjustment unit 140 causes the HUD 21 and the meter display 22 to display level 3 approach messages N4h and N4m, as shown in FIG. 13.

[0088] The level 3 approaching messages N4h and N4m are image contents that visually indicate the approach to the ST section. The level 3 approaching message N4h displayed on the HUD 21 is, for example, located in substantially the same position as the AD area approaching message N1h, and is configured to include a message image and a frame-shaped window image, similar to the AD area approaching message N1h. The message image is text information that describes the approach of the ST section, such as "About 10 km ahead, traffic jam, 2nd task available."

[0089] The level 3 approaching message N4m displayed on the meter display 22 is, for example, disposed in substantially the same position as the AD area approaching message N1m. The level 3 approaching message N4m is configured to include a message image and a strip-shaped window image with the message image disposed in front, similar to the AD area approaching message N1m. The message image is text information descriptive of the approach of the ST section, similar to the display on the HUD 21, such as "Traffic jam about 10 km ahead. 2nd task is available during the traffic jam."

[0090] In addition, when transitioning from the autonomous driving prohibited state to the level 3 possible state (first transition), the level 3 approach notification and the AD area approach notification are performed simultaneously. At this time, the level 3 approach notification and the AD area approach notification may be performed as a single notification.

[0091] The driver preparation notification is a notification that prompts the driver to satisfy the conditions (driver conditions) for implementing level 3. The driver preparation notification is executed when the driver conditions are not satisfied at the start of the level 3 approach notification. The driver conditions are, for example, that the driver's wakefulness exceeds a threshold level, that the driver's posture type is a predefined posture type, and the like. Furthermore, the driver conditions may be that the stress level falls below a threshold level, that the elapsed time from the start of driving exceeds a predetermined time, and the like. The driver preparation notification may be executed simultaneously with the level 3 approach notification. For example, it is started after the execution of the level 3 approach notification. As shown in FIG. 14, the presentation information adjustment unit 140 displays the driver preparation messages Nph, Npm on the HUD 21 and the meter display 22 in the driver preparation notification.

[0092] Each driver preparation message Nph, Npm includes, for example, a window image and a message image. The message image is a description that prompts the driver to take a predetermined action so that the driver condition that is not satisfied is satisfied. The driver preparation message Nph displayed on the HUD 21 is displayed, for example, in the same position as the level 3 approach message N4h. The driver preparation message Npm displayed on the meter display 22 is displayed, for example, below the driver status STD in the same display area as the driver status STD.

[0093] The level 3 possible notification notifies that a level 3 possible state has been established by entering an ST section. The level 3 possible notification is executed when entering an ST section from a non-ST section of an AD area, i.e., when there is a transition from a level 2 possible state to a level 3 possible state (second transition). In addition, the level 3 possible notification is executed when entering an ST section from an MD area, i.e., when there is a transition from an autonomous driving prohibited state to a level 3 possible state (first transition). In other words, the level 3 possible notification may be executed simultaneously with the AD area intrusion notification. The level 3 possible notification and the AD area intrusion notification may also be executed as a single notification.

[0094] In the level 3 possible notification, the presentation information adjustment unit 140 causes the HUD 21 and the meter display 22 to display level 3 possible messages N5h and N5m, as shown in Fig. 15. In addition, when the first transition occurs, the presentation information adjustment unit 140 causes the audio device 24 to present a level 3 possible notification sound. In this way, the presentation information adjustment unit 140 presents information regarding the first transition (first transition information) to the driver in a more emphasized manner than information regarding the second transition (second transition information).

[0095] The level 3 possible messages N5h and N5m are image contents that visually indicate the entry into the ST section. The level 3 possible message N5h displayed on the HUD 21 is, for example, arranged in substantially the same position as the AD area approaching message N1h, and is configured to include a message image and a frame-shaped window image, similar to the AD area approaching message N1h. The message image is text information that describes the entry into the ST section, such as "2nd task is now available."

[0096] The level 3 possible message N5m displayed on the meter display 22 is disposed, for example, at substantially the same position as the AD area approaching message N1m. The level 3 approaching message N4m, like the AD area approaching message N1m, is configured to include a message image and a strip-shaped window image with the message image disposed in front. The message image is substantially the same as that displayed on the HUD 21, for example.

[0097] The level 3 possible notification sound is sound content that indicates, as auditory information, that level 3 autonomous driving has become possible. The level 3 possible notification sound may be a sound effect or a voice message.

[0098] The level 3 start notification notifies the start of level 3 automated driving. In the first embodiment, when there is a transition from manual driving, level 1, or level 2 hands-on to level 3, the level 3 start notification is executed after receiving a level 3 start instruction from the driver. On the other hand, when there is a transition from level 2 hands-off to level 3, the level 3 start notification is executed when entering the ST section. In this case, the level 3 start notification is executed simultaneously with the level 3 possible notification. In the level 3 start notification, the presentation information adjustment unit 140 displays level 3 start messages N6h, N6m, and N6c on the HUD 21, the meter display 22, and the CID 23 as shown in FIG. 16.

[0099] The level 3 start messages N6h, N6m, and N6c are image contents that visually indicate the start of level 3 autonomous driving. The level 3 start message N6h displayed on the HUD 21 is, for example, located in substantially the same position as the level 2 start message N3h, and is configured to include a message image and a frame-shaped window image. The message image is text information with a description related to the start of level 3 autonomous driving, such as "Pay attention to information from the vehicle. Automated driving will be canceled due to surrounding road conditions."

[0100] The level 3 start message N6m displayed on the meter display 22 is, for example, disposed at substantially the same position as the level 2 start message N3h, and includes a message image and a frame-shaped window image. The message image is, for example, substantially the same as that displayed on the HUD 21.

[0101] The level 3 start message N6c displayed on the CID 23 is displayed, for example, near the center area of ​​the entertainment display area Ae. The level 3 start message N6c is displayed closer to the driver than other display contents in the entertainment display area Ae. The level 3 start message N6c includes a message image and a window image arranged behind the message image. The message image is a description related to the start of level 3 autonomous driving, similar to the message images on the HUD 21 and the meter display 22, such as "Autonomous driving will be canceled due to the surrounding road environment. Please pay attention to information transmitted from the vehicle."

[0102] In addition, in the level 3 start notification, the presentation information adjustment unit 140 switches the map display area Am and the entertainment display area Ae of CID 23. Specifically, the presentation information adjustment unit 140 displays the entertainment display area Ae above the map display area Am.

[0103] The level 3 end notification is executed when transitioning from level 3 autonomous driving to level 2 or lower autonomous driving or manual driving. When transitioning from level 3 autonomous driving to a level 2 permitted state, the presentation information adjustment unit 140 transitions from the information presentation in FIG. 17 to the information presentation in FIG. 10 or FIG. 11 as a level 3 end notification. Also, when transitioning from level 3 autonomous driving to a manual driving state, the presentation information adjustment unit 140 transitions from the information presentation in FIG. 17 to the information presentation in FIG. 8 as a level 3 end notification. In addition, the presentation information adjustment unit 140 causes the meter display 22 to display a level 3 end message N7m as a level 3 end notification, as shown in FIG. 18. In addition, the presentation information adjustment unit 140 causes the audio device 24 to output a level 3 end notification sound.

[0104] The level 3 end message N7m is image content that visually indicates the end of level 3 autonomous driving. For example, when transitioning to manual driving, the level 3 end message N7m is a message image that includes text information related to the end of autonomous driving, such as "Autonomous driving mode deactivated" (see FIG. 18). When transitioning to autonomous driving of level 2 or lower, the message image may include text information that alerts the driver to the end of the second task or the start of necessary driving operations.

[0105] When an override is executed in which the driver operates the steering wheel during autonomous driving at level 2 or higher, the presentation information adjustment unit 140 issues a notification indicating that the autonomous driving mode is temporarily canceled, as shown in Fig. 18. During the override, the presentation information adjustment unit 140 stops or hides the display content in the entertainment display area Ae.

[0106] The level 3 end notification sound is sound content that indicates the end of level 3 autonomous driving as auditory information. The level 3 end notification sound may be a sound effect or a voice message.

[0107] However, when there is a transition to a hands-off permitted state (fourth transition), the presentation information adjustment unit 140 suspends the presentation of the level 3 end notification sound. That is, when there is a transition to a hands-off prohibited state after the transition (third transition), the presentation information adjustment unit 140 presents the level 3 end notification sound, and when hands-on is not required after the transition, the presentation information adjustment unit 140 does not present the level 3 end notification sound. In this way, the presentation information adjustment unit 140 presents information regarding the third transition (third transition information) to the driver with more emphasis than information regarding the fourth transition (fourth transition information).

[0108] Here, the contents that the presentation information adjustment unit 140 continuously displays while autonomous driving is being performed will be described with reference to Figs. 10 to 12, etc. The presentation information adjustment unit 140 displays the driving statuses STh, STm, status icons ICsh, ICsm, and the driver status STD, etc., while autonomous driving of level 2 or lower is being performed.

[0109] The driving statuses STh and STm are contents displayed on the HUD 21 and the meter display 22. The driving statuses STh and STm indicate the operation states of longitudinal control and lateral control in autonomous driving at level 2 or lower, i.e., the ACC function and the LTA function. Each driving status STh and STm includes an other vehicle icon ICv2, an ACC icon ICa, and an LTA icon ICl. The driving status STm displayed on the meter display 22 also displays the vehicle icon ICv1 in addition to the above-mentioned contents.

[0110] The other vehicle icon ICv2 is an icon modeled after another vehicle, and is displayed at least while the ACC function is being executed or while a preceding vehicle is being detected. The ACC icons ICah and ICam are icons indicating the execution of ACC control, and are displayed while the ACC control is being executed, and are hidden while the vehicle is stopped or are displayed in a different manner from when the control is being executed. The ACC icon ICa is a figure arranged below the other vehicle icon ICv2, and is visually recognized as being lined up next to the preceding vehicle indicated by the other vehicle icon ICv2 due to perspective. The LTA icons IClh and IClm are icons indicating the execution of LTA control, and are displayed while the LTA control is being executed, and are hidden while the vehicle is stopped or are displayed in a different manner from when the control is being executed. The LTA icon ICl is arranged on both the left and right sides of the other vehicle icon ICv2 and the ACC icon ICa, and is visually recognized as a pair of lines extending from the front side to the back side.

[0111] The driving status STm in the meter display 22 is displayed collectively in one of a plurality of display areas obtained by dividing the display screen of the meter display 22. The display area of ​​the meter display 22 is divided into, for example, four areas: an upper area DAu, a lower left area DAl, a lower center area DAc, and a lower right area DAr. Note that each display area is apparently divided by the presentation information adjustment unit 140 changing the background color for each display area, displaying a frame image, or the like. The driving status STm is displayed in the lower center area DAc.

[0112] The status icons ICsh, ICsm are icons indicating the execution status of longitudinal control and lateral control, similar to the driving statuses STh, STm. The status icons ICsm are displayed in the upper area DAu. The status icons ICsh, ICsm are simplified icons compared to the driving statuses STh, STm, such as icons imitating other vehicles, icons indicating the execution of ACC control, and icons indicating the execution of LTA control. The status icons ICsh, ICsm may be continuously displayed even in autonomous driving of level 3 or higher.

[0113] The future action content is content that notifies the vehicle A of a future action scheduled to be taken, and is displayed in the lower right area DAr. The future action content includes, for example, an icon indicating the future action and a numerical image indicating the distance to the point where the future action will be performed. The future action includes lane changes, right and left turns, merging, etc. In addition, the future action may include a transfer of control between the vehicle A and the driver, a change in the level of automated driving, etc.

[0114] The driver status STD is a content that presents the driver's state regarding the driving operation. The driver status STD is displayed in the lower left area DAl of the meter display 22. The driver status STD presents a corresponding state of a plurality of types of body parts of the driver, between an interruption permitted state in which interruption of the driving operation is permitted and an interruption prohibited state in which interruption of the driving operation is prohibited. For example, the driver status STD presents an interruption permitted state or an interruption prohibited state for driving operations such as monitoring the surroundings of the eyes, gripping the steering wheel with the hands, and placing the feet on the pedals. The driver status STD displays the state of each body part in the same display area.

[0115] Specifically, the driver status STD includes a driver icon ICd, a steer icon ICs, a pedal icon ICp, and a gaze icon ICv. The driver icon ICd is a human-shaped icon. In detail, the driver icon ICd is shaped like a human body including the head, which is a body part including the arms, legs, and eyes. That is, the driver icon ICd is a content that displays each body part together. As a result, the driver icon ICd presents each body part close to each other in the lower left area DAl. This driver icon ICd may be a single content that is integrated, or may be a content in which each body part is spaced apart.

[0116] The steer icon ICs is an icon image imitating a steering wheel. The pedal icon ICp is an icon imitating an accelerator pedal. The gaze icon ICv is an icon indicating the driver's gaze. For example, the gaze icon ICv is shaped like an arrow extending from near the head of the driver icon ICd in the direction in which the head is facing (leftward in the cases of Figs. 10 to 12, etc.).

[0117] The driver status STD indicates the state permitted to the driver with respect to driving operations by using each icon together. For example, in the hands-off prohibited state in which hands-off is prohibited, the hands of the driver icon ICd and the steering icon ICs are displayed overlapping each other. In the hands-off permitted state in which hands-off is permitted, the hands of the driver icon ICd and the steering icon ICs are displayed apart. In addition, the visibility of the steering icon ICs is reduced.

[0118] In addition, in the case of a leg-off prohibited state in which leg-off is prohibited, the feet of the driver icon ICd and the pedal icon ICp are displayed overlapping each other. In addition, the visibility of the pedal icon ICp is reduced. In the case of a leg-off permitted state in which leg-off is permitted, the feet of the driver icon ICd and the pedal icon ICp are displayed apart. In addition, in the case of an eyes-off prohibited state in which eyes-off is prohibited, the gaze icon ICv is displayed, and in the case of an eyes-off permitted state in which eyes-off is permitted, the visibility of the gaze icon ICv is reduced.

[0119] The presentation information adjustment unit 140 notifies (transition notification) the state transition between the interruption prohibited state (hands-off prohibited state, leg-off prohibited state, eyes-off prohibited state) and the interruption permitted state (hands-off permitted state, leg-off permitted state, eyes-off permitted state) for each body part by using the above-mentioned driver status STD. At this time, the presentation information adjustment unit 140 changes the manner of the driver status STD by displaying an animation. Specifically, the presentation information adjustment unit 140 presents the state transition by displaying an animation for the display part of the driver icon ICd that corresponds to the body part whose state is transitioning.

[0120] The animation display is stopped when it is estimated that the corresponding body part of the driver has actually transitioned to a state corresponding to the state. The presentation information adjustment unit 140 also stops the animation when the corresponding body part of the driver does not transition to an interrupted state (hands off, legs off, eyes off) within a predetermined period of time. In this case, the presentation information adjustment unit 140 may set the mode after the animation is stopped to a display mode of an interruption prohibited state, that is, a mode corresponding to the current state of the body part.

[0121] For example, when transitioning from a hands-off prohibited state to a hands-off permitted state, the presentation information adjustment unit 140 displays an animation in which the hands of the driver icon ICd move from a position in the hands-off prohibited state to a position in the hands-off permitted state (see FIG. 11). The presentation information adjustment unit 140 repeatedly displays the above animation. When the driver actually transitions to the hands-off state, the presentation information adjustment unit 140 stops the animation and sets the display mode to that of the hands-off state (see FIG. 12).

[0122] In addition, when the state transitions from the hands-off prohibited state to the hands-off permitted state, the presentation information adjustment unit 140 reduces the visibility of the steer icons ICs by reducing the luminance, etc. The presentation information adjustment unit 140 may repeatedly reduce the visibility of the steer icons ICs in synchronization with the animation display of the driver icon ICd, or may perform the reduction after the animation of the driver icon ICd stops.

[0123] Similarly, when the state transitions from the leg-off prohibited state to the leg-off permitted state, the presentation information adjustment unit 140 displays an animation in which the foot of the driver icon ICd moves from the position in the leg-off prohibited state to the position in the leg-off permitted state. In addition, the presentation information adjustment unit 140 reduces the visibility of the pedal icon ICp.

[0124] In addition, when the state transitions from the interruption permitted state to the interruption prohibited state, the presentation information adjustment unit 140 similarly presents the state transition by displaying an animation. In this case, the presentation information adjustment unit 140 may continue displaying the animation until the corresponding body part of the driver actually transitions to the driving involved state.

[0125] In addition, the driver status STD is displayed in a different manner in the level 3 execution state from that in the level 2 or lower states. For example, the driver status STD in the level 3 execution state is displayed in a display manner in which the driver icon ICd is seated in a chair icon ICc. At this time, the driver icon ICd is displayed as a human figure reclining further back than in the level 2 or lower states. The steer icon ICs and the pedal icon ICp are not displayed.

[0126] The line-of-sight icon ICv, the steering icon ICs, and the pedal icon ICp may be hidden separately from reducing the visibility. The line-of-sight icon ICv, the steering icon ICs, and the pedal icon ICp may be displayed when there is a change in driving behavior that requires display together with the driver icon ICd.

[0127] In addition, when transitioning from a level 3 autonomous driving state to a hands-on level 2 autonomous driving state, the gaze icon ICv and the steer icon ICs are changed from hidden to displayed, or from displayed to highlighted, in order to encourage the driver to put his / her eyes on and put his / her hands on. At this time, the gaze icon ICv and the steer icon ICs may be displayed simultaneously or in sequence.

[0128] By simultaneously displaying the line of sight icon ICv and the steer icon ICs, the state in which the driver is permitted to carry out driving operations can be quickly conveyed to the driver.

[0129] On the other hand, when transitioning from a level 3 autonomous driving state to a hands-on level 2 autonomous driving state, the driver is required to perform many driving operations, which increases the number of tasks the driver must perform and may cause confusion. Therefore, by displaying the gaze icon ICv and the steer icon ICs in sequence, the driver can perform the tasks one by one, allowing them to perform driving operations safely.

[0130] Next, the flow of the presentation control method executed by the HCU 100 in cooperation with the functional blocks will be described below with reference to Figs. 19 to 24 as well as Figs. 2 to 18. In Figs. 19 to 24, the meter display 22 is written as "MD." In the flow described below, "S" means multiple steps of the flow that are executed by multiple commands included in the presentation control program.

[0131] First, a flow in a scene in which the vehicle enters an AD area from an MD area will be described (see Figs. 19 to 21). In S101 in Fig. 19, the surrounding state grasping unit 110 judges whether or not the vehicle is approaching an AD area. If it is judged that the vehicle is not approaching an AD area, the surrounding state grasping unit 110 waits until the vehicle is approaching an AD area. If it is judged that the vehicle is approaching an AD area, in S102, the surrounding state grasping unit 110 judges whether or not level 3 autonomous driving is possible after entering the AD area. If it is judged that level 3 autonomous driving is not possible, that is, only autonomous driving at level 2 or lower is possible, the process proceeds to S103.

[0132] In S103, the presentation information adjustment unit 140 executes an AD area approach notification. Specifically, the HUD 21 and the meter display 22 display AD area approach messages N1h and N1m.

[0133] After S103, the process proceeds to S110 in Fig. 22. In S100, the surrounding state grasping unit 110 judges whether or not the vehicle has entered an AD area. If it is judged that the vehicle has entered an AD area, the process proceeds to S111, and an AD area entry notification is executed.

[0134] Next, in S112, the driving state determination unit 120 determines whether or not the driver has permitted level 2 automated driving. If it is determined that the driver has not permitted it, in S113, the presentation information adjustment unit 140 determines whether or not the permitted time has elapsed since entry into the AD area. If it is determined that the permitted time has not elapsed, the process returns to S112. On the other hand, if it is determined that the permitted time has elapsed, in S114, the presentation information adjustment unit 140 stops the AD area entry notification on the HUD 21 and the meter display 22 and continues the AD area entry notification on the CID 23, and the process returns to S112.

[0135] If it is determined in S112 that the driver has permitted Level 2 automated driving, the process proceeds to S115. In S115, the presentation information adjustment unit 140 issues a Level 2 start notification. After the process of S115 is executed, the series of processes ends.

[0136] On the other hand, in S102 of FIG. 21, if the surrounding state grasping unit 110 determines that level 3 automated driving is possible after entering the AD area, the process proceeds to S116 of FIG. 23. In S116, the surrounding state grasping unit 110 determines whether or not the vehicle has entered the AD area. If it is determined that the vehicle has entered the AD area, the process proceeds to S117, in which the presentation information adjustment unit 140 executes an AD area entry notification and a level 3 possible notification. Next, in S118, the driving state determination unit 120 determines whether or not a level 3 start instruction has been acquired from the driver. If it is determined that the start instruction has been acquired, in S119, the presentation information adjustment unit 140 executes a level 3 start notification. After the process of S119 has been executed, the series of processes ends.

[0137] Next, a flow of the driver state presentation control method in level 2 autonomous driving will be described with reference to Fig. 22. First, in S120, the driving state determination unit 120 determines whether or not the state (interruption permitted state and interruption prohibited state) related to the driver's body parts has changed. If it is determined that there has been no change, the display of the current state is maintained in S122. On the other hand, if it is determined that the state has changed, the presentation information adjustment unit 140 starts a state change notification in S121.

[0138] Thereafter, in S123, the driver state estimation unit 130 determines whether the actual operating state of the driver has transitioned to a state corresponding to the relevant state. If the transition has not occurred, in S124, it is determined whether a predetermined time has elapsed since the start of the transition notification, and if not, the process returns to S123. If it is determined that the predetermined time has elapsed, in S125, the presentation information adjustment unit 140 interrupts the transition notification and ends the series of processes. On the other hand, if it is determined in S123 that the transition has occurred, the state after the transition is displayed, and the series of processes ends.

[0139] Next, a presentation control method for transitioning from level 2 autonomous driving to level 3 autonomous driving will be described with reference to the flow in FIG.

[0140] First, in S130, the surrounding state grasping unit 110 judges whether or not the vehicle is approaching an ST section. If it is judged that the vehicle is not approaching an ST section, the surrounding state grasping unit 110 waits until the vehicle is approaching an ST section. If it is judged that the vehicle is approaching an ST section, the presentation information adjusting unit 140 executes a level 3 approach notification and a driver preparation notification in S131. Note that if the driver preparation notification is not necessary, the driver preparation notification may be interrupted.

[0141] After S131, the process proceeds to S132. In S132, the surrounding state grasping unit 110 judges whether or not the vehicle has entered an ST section. If it is judged that the vehicle has entered an ST section, the process proceeds to S133, in which the driving state judgment unit 120 judges whether or not the vehicle has entered an ST section in a hands-on state.

[0142] If it is determined in S133 that the vehicle has entered the ST section in a hands-on state, the process proceeds to S134. In S134, the presentation information adjustment unit 140 issues a level 3 possible notification. Next, in S135, the driving state determination unit 120 determines whether or not the driver has instructed level 3 automatic driving control. If it is determined that the driver has instructed level 3 automatic driving control, in S136, the presentation information adjustment unit 140 issues a level 3 start notification. After the process of S136 is executed, the series of processes ends.

[0143] On the other hand, if it is determined in S133 that the vehicle has entered the ST section in a hands-off state, the process proceeds to S137. In S137, a level 3 possible notification and a level 3 start notification are simultaneously issued, and the series of processes ends.

[0144] Next, a presentation control method when level 3 autonomous driving ends will be described with reference to the flow in FIG.

[0145] In S140, the driving state determination unit 120 determines whether or not to transition to driving control at level 2 or lower. If it is determined that the transition should be made, the process proceeds to S141, where the driving state determination unit 120 determines whether or not to transition to a level 2 hands-off permitted state. If it is determined that the transition should be made to a level 2 hands-off permitted state, the process proceeds to S142, where the presentation information adjustment unit 140 executes a level 3 end notification. Specifically, a level 3 end message N7m is displayed on the display device. After the process of S142 is executed, the series of processes is terminated.

[0146] On the other hand, if it is determined in S141 that the state will be changed to a state other than the level 2 hands-off, that is, to any of the level 2 hands-on, level 1, and level 0 (manual driving), the process proceeds to S143. In S143, the presentation information adjustment unit 140 executes a level 3 end notification. Specifically, in addition to the level 3 end message N7m on the display device, a level 3 end notification sound is output from the audio device 24.

[0147] Note that the above-mentioned S116 and S132 are an example of a "discrimination process", and S117, S134, and S137 are an example of a "presentation control process".

[0148] According to the first embodiment described above, the first transition information regarding the transition from the autonomous driving prohibited state to the specific action permitted state is presented in a presentation manner that emphasizes the first transition information regarding the transition from the specific action prohibited state to the specific action permitted state. Therefore, the driver can more reliably recognize the transition from the autonomous driving prohibited state to the specific action permitted state, which requires more attention. As a result, it is possible to present information in a highly convenient manner.

[0149] Second embodiment In the second embodiment, a modified example of the HCU 100 in the first embodiment will be described. In the second embodiment, when transitioning from a level 3 permitted state to a level 2 permitted state, the presentation information adjustment unit 140 also outputs a transition notification sound in the audio device 24 when transitioning to a hands-off permitted state. That is, when transitioning from the level 2 permitted state to the level 3 permitted state, the presentation information adjustment unit 140 performs notification by the display device, and when transitioning from the level 3 permitted state to the level 2 permitted state, performs notification by the audio device 24 in addition to notification by the display device.

[0150] As a result, the presentation information adjustment unit 140 presents information about each transition in different presentation modes for a transition from a state in which the second task is prohibited to a state in which the second task is permitted (permitted transition) and a transition from a state in which the second task is permitted to a state in which the second task is prohibited (prohibited transition). In particular, in this embodiment, information about prohibited transitions (prohibited transition information) is presented with more emphasis than information about permitted transitions (permitted transition information).

[0151] In the second embodiment, the flow of the presentation control method executed by the HCU 100 will be described below with reference to FIG. 25. The process of S240 is similar to the process of S140 in the first embodiment. If it is determined in S240 that the driving will be shifted to level 2 or lower, the presentation information adjustment unit 140 executes a level 3 end notification in S241. Specifically, in addition to displaying a level 3 end message by each display device, a level 3 end sound is output by the audio device 24. When the process of S241 is executed, the series of processes ends.

[0152] In the second embodiment, the above-mentioned S132 and S240 are an example of a "determination process", and S134, S137, and S241 are an example of a "presentation control process".

[0153] According to the second embodiment, the prohibited transition information regarding the transition from the specific action permitted state to the specific action prohibited state is presented in a more emphasized presentation manner than the permitted transition information regarding the transition from the specific action prohibited state to the specific action permitted state. Therefore, the driver can more reliably recognize the transition from the specific action permitted state to the specific action prohibited state, which requires more attention. As a result, it is possible to present information in a highly convenient manner.

[0154] Note that, instead of or in addition to outputting the level 3 end sound, the presentation information adjustment unit 140 may display a level 3 end message for a prohibited transition in a display mode that is emphasized more than a level 3 end message for a permitted transition. For example, the presentation information adjustment unit 140 may change the display color, size, or the like of the level 3 end message to emphasize the level 3 end message more than the permitted transition.

[0155] Third embodiment In the third embodiment, a modified example of the HCU 100 in the first embodiment will be described. In the third embodiment, the second autonomous driving ECU 70 transitions to autonomous driving of level 3 or higher when level 2 hands-off is permitted and there is no override by the driver, i.e., the vehicle is actually in a hands-off state.

[0156] In this case, when the driver state is determined to be a hands-on state in the hands-off permitted state, the presentation information adjustment unit 140 executes a transition impossibility notification to notify that transition to level 3 or higher autonomous driving is not possible. In the transition impossibility notification, for example, a message image is displayed on the HUD 21 and the meter display 22. The message image may directly state that transition is not possible, or may indirectly state that transition is not possible by stating that transition to level 3 is possible with hands-off.

[0157] In addition, even if the driver operates the steering wheel while in the level 2 hands-off state, the level 2 hands-off state is maintained up to a certain torque amount. On the other hand, if the torque amount is equal to or exceeds the certain torque amount, the vehicle enters an override state.

[0158] In the third embodiment, a flow of the presentation control method executed by the HCU 100 will be described below with reference to Fig. 26. The process of S330 is similar to the process of S130 in the first embodiment. When it is determined in S330 that the ST section is approaching, in S331, the driver state estimation unit 130 determines whether or not the current driver state is a hands-off state. When it is determined that the driver state is not a hands-off state (a hands-on state), in S332, the presentation information adjustment unit 140 executes a transition impossible notification.

[0159] On the other hand, if it is determined in S331 that the vehicle is in a hands-off state, the surrounding state grasping unit 110 determines in S333 whether or not the vehicle has not yet entered the ST section when the vehicle is in the hands-off state. If it is determined that the vehicle has not yet entered the ST section, the process proceeds to S334. S334 is the same as the process of S131. In S335, the surrounding state grasping unit 110 determines whether or not the vehicle has entered the ST section. If it is determined that the vehicle has entered the ST section, the process proceeds to S336. S336 is the same as the process of S137. On the other hand, if it is determined in S333 that the vehicle has already entered the ST section when the vehicle is in the hands-off state, the process skips S334 and S335 and proceeds to S336. When the process of S336 is executed, the series of processes ends.

[0160] In the present embodiment, the presentation information adjustment unit 140 presents the transition impossibility notification before entering the ST section, but may be configured to present the notification after entering the ST section. In the present embodiment, the presentation information adjustment unit 140 presents the transition impossibility notification if the driver is not in a hands-off state while approaching the ST section. Alternatively, the presentation information adjustment unit 140 may be configured to present the transition impossibility notification regardless of whether the driver is approaching the ST section or not if the driver is in a hands-on state in a hands-off permitted state.

[0161] (Fourth embodiment) In the level 3 end notification of the above embodiment, the presentation information adjustment unit 140 presents information about the first transition to the driver in a more emphasized manner than information about the second transition by having the audio device 24 present a level 3 possible notification sound when the first transition occurs. Alternatively, the presentation information adjustment unit 140 may emphasize information about the first transition more than information about the second transition by making the level 3 possible messages N5h and N5m displayed on the display device in the first transition more emphasized than the second transition. For example, the presentation information adjustment unit 140 may emphasize the level 3 possible messages N5h and N5m more than the second transition by changing the display color, size, etc. of the level 3 possible messages N5h and N5m.

[0162] Fifth embodiment In the fifth embodiment, a modified example of the HCU 100 in the first embodiment will be described. In Fig. 27, components denoted by the same reference numerals as those in the first embodiment are similar components and have the same functions and effects.

[0163] In the fifth embodiment, when the driver condition for implementing level 3 on the driver is determined not to be satisfied by the driver state estimation unit 130, the presentation information adjustment unit 140 stops highlighting information related to the first transition in the level 3 possible notification. Specifically, the presentation information adjustment unit 140 prohibits the presentation of the level 3 possible notification sound in the audio device 24. As a result, the level 3 possible notification when the driver condition is satisfied is made to have substantially the same form in the case of transition from the autonomous driving prohibited state to the level 3 possible state and in the case of transition from the level 2 possible state to the level 3 possible state. In the fifth embodiment, the driver state estimation unit 130 is an example of a "preparation determination unit". The satisfaction of the driver condition corresponds to the preparation for the execution of autonomous driving with a specific action permitted.

[0164] The process executed by the HCU 100 in the fifth embodiment will be described with reference to the flow of FIG. 27. When it is determined in S116 that the vehicle has entered an AD area, the flow proceeds to S116a. In S116a, the driver state estimation unit 130 determines whether or not the driver condition is satisfied. When it is determined that the driver condition is not satisfied, the flow proceeds to S117. That is, in S117, an AD area entry notification and a highlighted level 3 possible notification are issued.

[0165] On the other hand, if it is determined that the driver condition is satisfied, the flow proceeds to S117a. In S117a, the presentation information adjustment unit 140 executes an AD area entry notification and a level 3 possible notification with the emphasis stopped. That is, in S117a, the notification by the audio device 24 is not executed in the level 3 possible notification.

[0166] According to the fifth embodiment described above, when it is determined that the driver is ready for automatic driving, the presentation of the first transition information in an emphasized presentation manner is stopped. Therefore, the transition information can be emphasized and presented only when the driver is not ready for automatic driving. Therefore, it is possible to more appropriately use the presentation manner of the information.

[0167] Sixth embodiment In the sixth embodiment, a modified example of the HCU 100 in the first embodiment will be described. In Fig. 28, components denoted by the same reference numerals as those in the first embodiment are similar components and have the same functions and effects.

[0168] In the sixth embodiment, when the number of occurrences of the first transition (transition count) within a predetermined period falls outside the allowable range, the presentation information adjustment unit 140 stops highlighting the information related to the first transition after the number of occurrences falls outside the allowable range. The stop of highlighting corresponds to a relatively lower degree of highlighting than when the number of occurrences of the first transition falls within the allowable range. The predetermined period here may be a period from when the vehicle A starts to travel until it stops, or may be a period during which the vehicle travels a predetermined threshold distance or threshold time. The allowable range here is specified as a numerical range equal to or less than the threshold number of times or less than the threshold number of times. For example, the allowable range of the occurrence count is set to one time.

[0169] Specifically, as a process of canceling the emphasis, the presentation information adjustment unit 140 prohibits the presentation of the level 3 possible notification sound in the audio device 24. That is, the level 3 possible notification in the second and subsequent first transitions within a predetermined period is made in substantially the same manner as in the case of the second transition.

[0170] The process executed by the HCU 100 in the sixth embodiment will be described with reference to the flow in FIG. 28. If it is determined in S116 that the AD area has been entered, the flow proceeds to S116b. In S116b, the presentation information adjustment unit 140 determines whether or not the number of occurrences of the first transition is within an allowable range. If it is determined to be within the allowable range, the flow proceeds to S117. That is, in S117, an AD area entry notification and an emphasized level 3 possible notification are issued.

[0171] On the other hand, if it is determined that the number of occurrences of the first transition is outside the allowable range, the flow proceeds to S117a. In S117a, the presentation information adjustment unit 140 executes an AD area entry notification and a level 3 possible notification in which the emphasis is stopped. That is, in S117a, the notification by the audio device 24 is not executed in the level 3 possible notification.

[0172] According to the sixth embodiment described above, when it is determined that the number of occurrences of the first transition within a predetermined period is outside the allowable range, the degree of emphasis of the first transition information is relatively lower than when it is within the allowable range. Therefore, the transition information can be presented with emphasis only at a stage where the number of occurrences of the first transition is relatively small. Therefore, it is possible to more appropriately use the presentation mode of the information.

[0173] Seventh embodiment In the seventh embodiment, a modified example of the HCU 100 in the first embodiment will be described. In Fig. 29, components denoted by the same reference numerals as those in the first embodiment are similar components and have the same functions and effects.

[0174] In the seventh embodiment, the presentation information adjustment unit 140 judges whether or not the driver has performed an autonomous driving level 3 start operation before entering the ST section, which is an AD area where autonomous driving level 3 is possible. If it is judged that the driver has performed the autonomous driving level 3 start operation, the presentation information adjustment unit 140 emphasizes the information about the first transition. On the other hand, if it is judged that the driver has not performed the autonomous driving level 3 start operation before entering the ST section, the presentation information adjustment unit 140 stops emphasizing the information about the first transition. That is, if the driver has performed the autonomous driving level 3 start operation before entering the AD area where autonomous driving level 3 is possible, the presentation information adjustment unit 140 emphasizes the information about the first transition. Executing the start operation before entering the AD area where autonomous driving level 3 is possible corresponds to executing the start operation before transitioning to the specific action permitted state.

[0175] The process executed by the HCU 100 in the seventh embodiment will be described with reference to the flow of FIG. 29. If it is determined in S116 that the vehicle has entered an AD area, the flow proceeds to S116c. In S116c, the presentation information adjustment unit 140 determines whether the driver has performed an operation to start autonomous driving level 3 before entering the AD area where autonomous driving level 3 is possible. If it is determined that the driver has performed an operation to start autonomous driving level 3, the flow proceeds to S117. That is, in S117, an AD area entry notification and a highlighted level 3 possible notification are issued.

[0176] On the other hand, if it is determined that the driver has not performed the operation to start level 3 autonomous driving before entering an AD area where level 3 autonomous driving is possible, the flow proceeds to S117a. In S117a, the presentation information adjustment unit 140 executes an AD area entry notification and a level 3 possible notification with the highlighting stopped. That is, in S117a, the notification by the audio device 24 is not executed in the level 3 possible notification.

[0177] According to the sixth embodiment described above, when it is determined that the driver has not performed the start operation of autonomous driving level 3 before entering an AD area where autonomous driving level 3 is possible, the degree of emphasis of the first transition information is relatively lower than when it is determined that the driver has performed the operation. Therefore, the transition information can be presented with emphasis only when the start operation of autonomous driving level 3 is performed before entering an AD area where autonomous driving level 3 is possible. Therefore, when there is a time lag between the start operation of autonomous driving level 3 and when autonomous driving level 3 is actually executable, the driver can be more reliably notified that autonomous driving level 3 is executable.

[0178] Eighth embodiment In the eighth embodiment, a modified example of the HCU 100 in the first embodiment will be described. In Figures 30 and 31, components denoted by the same reference numerals as those in the first embodiment are similar components and have the same functions and effects.

[0179] In the eighth embodiment, the presentation information adjustment unit 140 presents information on the second transition to the driver in a more emphasized manner than information on the first transition. Specifically, the presentation information adjustment unit 140 emphasizes the level 3 possible notification in the second transition more than the level 3 possible notification in the first transition. For example, the presentation information adjustment unit 140 may present a level 3 possible notification sound by the audio device 24 in the level 3 possible notification in the second transition, and may stop the presentation in the level 3 possible notification in the first transition. Note that the presentation information adjustment unit 140 may emphasize the information on the second transition more than the information on the first transition by making the level 3 possible messages N5h and N5m displayed on the display device in the second transition more emphasized than the first transition.

[0180] The process executed by the HCU 100 in the eighth embodiment will be described with reference to the flows of Figs. 30 and 31. When it is determined in S116 of Fig. 30 that the user has entered an AD area, the process proceeds to S117b. In S117b, the presentation information adjustment unit 140 issues an AD area entry notification and a non-emphasized level 3 possible notification. That is, in S117b, the audio device 24 does not issue a level 3 possible notification. After the process of S117b, the process proceeds to S118.

[0181] 31, when it is determined that the ST section has been entered in a hands-on state, the process proceeds to S134a. In S134a, the presentation information adjustment unit 140 executes an emphasized level 3 possible notification. After the process of S134a is executed, the process proceeds to S135.

[0182] On the other hand, if it is determined in S133 that the vehicle has entered the ST section in a hands-off state, the process proceeds to S137a, in which the highlighted level 3 possible notification and level 3 start notification are simultaneously executed, and the series of processes ends.

[0183] According to the eighth embodiment described above, the second transition information regarding the transition from the specific action prohibited state to the specific action permitted state is presented in a presentation manner that emphasizes the transition from the automatic driving prohibited state to the specific action permitted state more than the first transition information regarding the transition from the automatic driving prohibited state to the specific action permitted state. Therefore, the driver can more reliably recognize the transition from the automatic driving prohibited state to the specific action permitted state, which requires more attention. As a result, it is possible to present information in a highly convenient manner.

[0184] Ninth embodiment In the ninth embodiment, a modified example of the HCU 100 in the first embodiment will be described. In Fig. 32 to Fig. 34, components denoted by the same reference numerals as those in the first embodiment are similar components and have the same functions and effects.

[0185] In the ninth embodiment, the surrounding condition grasping unit 110 determines the type of lane in which the vehicle is currently traveling. For example, the surrounding condition grasping unit 110 determines whether the lane in which the vehicle is currently traveling is an overtaking lane, a climbing lane, or a normal lane other than the above. The surrounding condition grasping unit 110 is an example of a "lane identification unit."

[0186] The presentation information adjustment unit 140 changes the degree of emphasis of the presentation of the first transition information and the second transition information according to the type of the determined lane. Specifically, the presentation information adjustment unit 140 changes the degree of emphasis of the level 3 possible notification when the current lane is an overtaking lane, a normal lane, or an uphill lane. For example, the presentation information adjustment unit 140 emphasizes the transition information of a lane in which overtaking or passing another vehicle traveling in a specific lane is permitted by traffic regulations, more than that of the specific lane. As an example, the presentation information adjustment unit 140 emphasizes and presents the level 3 possible notification in the order of the uphill lane, the normal lane, and the overtaking lane.

[0187] As an example of the change in the emphasis degree, when the presentation information adjustment unit 140 executes the level 3 possible notification while traveling on an uphill lane, the presentation information adjustment unit 140 displays the level 3 possible message in a normal color and does not execute the presentation of the level 3 possible notification sound by the audio device 24. Here, the normal color is a color (e.g., white, blue, etc.) that is not conspicuous compared to the emphasis color described below. Then, when the presentation information adjustment unit 140 executes the level 3 possible notification while traveling on a normal lane, the presentation information adjustment unit 140 displays the level 3 possible message in a normal color and executes the presentation of the level 3 possible notification sound by the audio device 24. In addition, when the presentation information adjustment unit 140 executes the level 3 possible notification while traveling on an overtaking lane, the presentation information adjustment unit 140 displays the level 3 possible message in an emphasis color (e.g., a warning color such as red or yellow) and executes the presentation of the level 3 possible notification sound by the audio device 24.

[0188] The process executed by the HCU 100 in the ninth embodiment will be described with reference to the flows in Figures 32 to 34. First, when it is determined in S116 of Figure 32 that the vehicle has entered an AD area, the flow proceeds to S116c. In S116c, the surrounding state grasping unit 110 determines the type of lane in which the vehicle is currently traveling.

[0189] When it is determined in S116c that the vehicle is traveling in the passing lane, the flow proceeds to S117c. In S117c, the presentation information adjustment unit 140 executes an AD area entry notification and a level 3 possible notification in a presentation mode that is more emphasized than when the vehicle is traveling in a normal lane or an uphill lane.

[0190] If it is determined in S116c that the vehicle is traveling in a normal lane, the flow proceeds to S117d. In S117d, the presentation information adjustment unit 140 executes an AD area entry notification and a level 3 possible notification in a presentation manner that is emphasized more than when the vehicle is traveling in a climbing lane and less emphasized than when the vehicle is traveling in an overtaking lane.

[0191] If it is determined in S116c that the vehicle is traveling on an uphill lane, the flow proceeds to S117e. In S117e, the presentation information adjustment unit 140 issues an AD area entry notification and a level 3 possible notification in a presentation manner in which the degree of emphasis is reduced compared to when the vehicle is traveling on a normal lane or an uphill lane.

[0192] When any of the processes of S117c, S117d, and S117e is executed, the flow proceeds to S118. Note that the above-mentioned S116c, S133a, and S133c are an example of a "lane identification process." Also, the above-mentioned S117c, S117d, S117e, S134c, S134d, S134e, S137c, S137d, and S137e are an example of a "presentation control process."

[0193] Next, the flowchart of Fig. 33 will be described. If it is determined in S133 of Fig. 33 that the vehicle has entered an AD area in a hands-on state, the process proceeds to S133a. In S133a, the surrounding state recognition unit 110 determines the type of lane in which the vehicle is currently traveling.

[0194] When it is determined in S133a that the vehicle is traveling in the passing lane, the flow proceeds to S134c. In S134c, the presentation information adjustment unit 140 executes the level 3 possible notification in a presentation mode that is more emphasized than when the vehicle is traveling in a normal lane or an uphill lane.

[0195] If it is determined in S133a that the vehicle is traveling in a normal lane, the flow proceeds to S134d. In S134d, the presentation information adjustment unit 140 executes the level 3 possible notification in a presentation mode in which the presentation information is emphasized more than when the vehicle is traveling in a climbing lane and is emphasized less than when the vehicle is traveling in an overtaking lane.

[0196] If it is determined in S133a that the vehicle is traveling on an uphill lane, the flow proceeds to S134e. In S134e, the presentation information adjustment unit 140 executes a level 3 possible notification in a presentation mode in which the degree of emphasis is weakened compared to when the vehicle is traveling on a normal lane or an uphill lane.

[0197] If it is determined in S133 that the vehicle is not in a hands-on state, the process proceeds to S133b in Fig. 34. In S133b, the surrounding state grasping unit 110 determines the type of lane in which the vehicle is currently traveling, similarly to S133a.

[0198] When it is determined in S133b that the vehicle is traveling in the passing lane, the flow proceeds to S137c. In S137c, the presentation information adjustment unit 140 executes a level 3 possible notification and a level 3 start notification in a presentation manner that is more emphasized than when the vehicle is traveling in a normal lane or an uphill lane.

[0199] If it is determined in S133b that the vehicle is traveling in a normal lane, the flow proceeds to S137d. In S137d, the presentation information adjustment unit 140 executes a level 3 possible notification and a level 3 start notification in a presentation manner that is more emphasized than when the vehicle is traveling in an uphill lane and less emphasized than when the vehicle is traveling in an overtaking lane.

[0200] Then, when it is determined in S133b that the vehicle is traveling on an uphill lane, the flow proceeds to S137e. In S137e, the presentation information adjustment unit 140 executes a level 3 possible notification and a level 3 start notification in a presentation manner in which the degree of emphasis is weakened compared to when the vehicle is traveling on a normal lane or an uphill lane.

[0201] According to the ninth embodiment, the degree of emphasis of the presentation of the first transition information regarding the transition from the autonomous driving prohibited state to the specific action permitted state and the second transition information regarding the transition from the specific action prohibited state to the specific action permitted state is changed according to the type of lane in which the vehicle is traveling. Therefore, the driver can recognize each piece of information with a degree of emphasis according to the lane in which the vehicle is traveling. As a result, it is possible to present information in a highly convenient manner.

[0202] (Other embodiments) The disclosure in this specification is not limited to the exemplified embodiments. The disclosure includes the exemplified embodiments and modifications based thereon by those skilled in the art. For example, the disclosure is not limited to the combination of parts and / or elements shown in the embodiments. The disclosure can be implemented by various combinations. The disclosure can have additional parts that can be added to the embodiments. The disclosure includes the omission of parts and / or elements of the embodiments. The disclosure includes the substitution or combination of parts and / or elements between one embodiment and another embodiment. The disclosed technical scope is not limited to the description of the embodiments. Some disclosed technical scopes are indicated by the description of the claims, and should be interpreted as including all modifications within the meaning and scope equivalent to the description of the claims.

[0203] In the above embodiment, the presentation information adjustment unit 140 presents the auditory information by the audio device 24 as information in a specific form different from the visual information, but the information in the specific form is not limited to this. For example, the presentation information adjustment unit 140 may be configured to present tactile information. Specifically, the presentation information adjustment unit 140 presents tactile information by vibrating a vibration member provided on the seat on which the driver sits, the steering wheel, the seat belt device, or the like. The presentation information adjustment unit 140 may emphasize the presentation of information by presenting both auditory information and tactile information. The presentation information adjustment unit 140 may also combine the visual information in the emphasized display form with the auditory information and / or tactile information. Alternatively, the presentation information adjustment unit 140 may emphasize the presentation of information by increasing the number of notifications. Alternatively, the presentation information adjustment unit 140 may emphasize the presentation of information by increasing the amount of information presented.

[0204] In the sixth embodiment described above, when the number of occurrences of the first transition within a predetermined period falls outside the allowable range, the presentation information adjustment unit 140 stops emphasizing the information related to the first transition after the number of occurrences falls outside the allowable range. Alternatively or in addition to this, when the number of occurrences of the second transition within a predetermined period falls outside the allowable range, the presentation information adjustment unit 140 may relatively lower the degree of emphasis of the information related to the second transition after the number of occurrences falls outside the allowable range.

[0205] In the above-mentioned ninth embodiment, the presentation information adjustment unit 140 changes the emphasis degree of each piece of transition information according to the type of lane to which the vehicle is traveling, among an overtaking lane, an uphill lane, and a normal lane. Alternatively, the presentation information adjustment unit 140 may be configured to change the emphasis degree of each piece of transition information according to the type of lane to which the vehicle is traveling, among a lane on a general road and a lane on a highway. In this case, the presentation information adjustment unit 140 may increase the emphasis degree of each piece of transition information when the lane to which the vehicle is traveling is a lane on a highway, compared to when the lane to which the vehicle is traveling is a lane on a general road. In other words, the presentation information adjustment unit 140 may increase the emphasis degree of each piece of transition information as the driving speed permitted for the lane to which the vehicle is traveling is increased.

[0206] In the above embodiment, the specific action permitted state has been described as a state in which autonomous driving level 3 is permitted, but it may also be a state in which autonomous driving level 4 or higher is permitted. Also, the specific action prohibited state has been described as a state in which autonomous driving level 2 is permitted, but it may also be a state in which autonomous driving level 1 is permitted.

[0207] The HCU 100 may be a dedicated computer including at least one of a digital circuit and an analog circuit as a processor. Here, the digital circuit is at least one of ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), SOC (System on a Chip), PGA (Programmable Gate Array), and CPLD (Complex Programmable Logic Device). Such a digital circuit may also include a memory that stores a program.

[0208] The HCU 100 may be provided by one computer or a set of computer resources linked by a data communication device. For example, some of the functions provided by the HCU 100 in the above-mentioned embodiment may be realized by another ECU. [Explanation of symbols]

[0209] 100 HCU (presentation control device), 101 memory (storage medium), 102 processor, 110 surrounding state grasping unit (lane identification unit), 120 driving state judgment unit (discrimination unit), 130 driver state estimation unit (preparation determination unit), 140 presentation information adjustment unit (presentation control unit), A vehicle.

Claims

1. A presentation control device that controls presentation of information to a driver of a vehicle (A) capable of performing autonomous driving, a determination unit (120) that distinguishes between a specific action permitted state that permits the driver to perform a specific action other than driving while the autonomous driving is being performed, a hands-off permitted state that prohibits the driver from performing the specific action while the autonomous driving is being performed and permits the driver to release the steering wheel, and a both-prohibited state that prohibits the driver from performing the specific action and the hands-off while the autonomous driving is being performed, and determines whether or not a transition between the states is possible and whether or not the transition is to be performed; a presentation control unit (140) that presents: both-prohibited transition possible information indicating that a transition from the both-prohibited states to the specific action permitted state is possible; both-prohibited transition execution information indicating execution of a transition from the both-prohibited states to the specific action permitted state; and hands-free transition execution information indicating execution of a transition from the hands-free permitted state to the specific action permitted state; Equipped with The presentation control device presents the both prohibited transition possible information prior to the both prohibited transition execution information when transitioning from the both prohibited states to the specific action permitted state, presents the both prohibited transition execution information after receiving an instruction from the driver to start the autonomous driving in the specific action permitted state, and presents the hands-free transition execution information without the start instruction when transitioning from the hands-free permitted state to the specific action permitted state.

2. The presentation control device of claim 1, wherein the presentation control unit shortens the time from when the transition to the specific action permitted state becomes possible to when the hands-off transition execution information is presented relative to the time from when the transition to the specific action permitted state becomes possible to when the hands-off transition execution information is presented.

3. A presentation control program stored in a storage medium (101) and including instructions to be executed by a processor (102) in order to control presentation of information to a driver of a vehicle (A) capable of performing autonomous driving, The instruction: a judgment process (S132, S133, S135) for distinguishing between a specific action permission state in which the driver is permitted to perform a specific action other than driving while the autonomous driving is being performed, a hands-off permission state in which the driver is prohibited from performing the specific action while the autonomous driving is being performed and is permitted to release the hands from the steering wheel, and both of the prohibited states in which the driver is prohibited from performing the specific action and releasing the hands from the steering wheel while the autonomous driving is being performed, and for determining whether or not a transition between each state is possible and whether the transition is to be performed; a presentation control process (S134, S136, S137) for presenting both-prohibited transition possible information indicating that a transition from the both-prohibited states to the specific action permitted state is possible, both-prohibited transition execution information indicating the execution of a transition from the both-prohibited states to the specific action permitted state, and hands-free transition execution information indicating the execution of a transition from the hands-free permitted state to the specific action permitted state; Including, In the presentation control process, when there is a transition from the both prohibited states to the specific action permitted state, the both prohibited transition possible information is presented prior to the both prohibited transition execution information, the both prohibited transition execution information is presented after receiving an instruction from the driver to start the autonomous driving in the specific action permitted state, and when there is a transition from the hands-free permitted state to the specific action permitted state, the hands-free transition execution information is presented without the start instruction.

Images