Driving control device and HMI control device

Abstract

To satisfactorily make a driver conscious of a transition between driving levels due to termination of automated driving.SOLUTION: An HMI control device (25) includes a driving level acquisition section (253) and a display control section (255). The driving level acquisition section acquires a determination result of a driving level corresponding to an execution state including whether or not driving automation is executed in a driving control device (18) for controlling driving of a vehicle. The display control section controls image display operation in an HMI device (20) according to the driving level acquired by the driving level acquisition section. The display control section causes the HMI device to display an action instruction display for instructing a driver to take a low-level corresponding state in which, while a high level included in automated driving is terminated, the driving level lower than the high level can be executed, as a driver state, and performs an action instruction display for instructing the driver to take a middle-level corresponding state in which the driving level of a middle level can be executed.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a driving control device that controls the driving of an autonomous vehicle, and an HMI control device that controls an HMI device mounted on the vehicle. HMI is an abbreviation for human machine interface. [Background technology]

[0002] Various autonomous driving systems for vehicles such as automobiles have been proposed (see, for example, Patent Document 1, etc.). During autonomous driving, the driver, who is an occupant in the driver's seat of the vehicle, is able to freely execute a second task. A second task is a task executed by the driver other than driving operations. Specifically, second tasks include, for example, operating a mobile terminal, watching video content, etc. A second task is also called a "non-driving task" or a "secondary activity." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-107502 A Summary of the Invention [Problem to be solved by the invention]

[0004] Automated driving may end, for example, when the vehicle reaches the end of a specific road section where autonomous driving is permitted. When autonomous driving ends, it is necessary to transfer authority to the driver for vehicle motion control and / or driving environment monitoring.

[0005] On the other hand, the driver performing the second task during autonomous driving is not aware of the driving state and driving environment of the vehicle. Therefore, in order to smoothly transfer authority, it is necessary to properly raise the driver's awareness toward the end of autonomous driving. The present invention has been made in consideration of the above-mentioned circumstances. That is, the present invention provides a technology that enables the driver to properly raise the driver's awareness toward the transition of the driving automation level due to the end of autonomous driving, for example. [Means for solving the problem]

[0006] The HMI control device (25) is configured to control the HMI device (20) that displays an image visible to the driver of the autonomously operable vehicle (1). The HMI control device according to claim 1 comprises: A driving level acquisition unit (253) that acquires a result of determining a driving level corresponding to an implementation state including whether or not driving automation is implemented in a driving control device (18) that controls driving of the vehicle; A display control unit (255) that controls an image display operation in the HMI device according to the driving level acquired by the driving level acquisition unit; Equipped with The display control unit causes the HMI device to display an operational instruction display that terminates a high level, which is the driving level included in the autonomous driving, and causes the driver to take a low level corresponding state in which the driving level that is lower than the high level can be executed as the driver state, and causes the driver to take a medium level corresponding state in which the driving level of a medium level, which is a driving level between the high level and the low level, can be executed. The HMI control program is a computer-readable program executed by an HMI control device (25) configured to control an HMI device (20) that displays images visible to the driver of an autonomous vehicle (1). The HMI control program according to claim 2, The process executed by the HMI control device is as follows: A driving level acquisition process for acquiring a result of a driving level determination corresponding to an implementation state including whether or not driving automation is implemented in a driving control device (18) that controls driving of the vehicle; a display control process for controlling an image display operation in the HMI device according to the driving level acquired in the driving level acquisition process; Including, The display control process displays an operational instruction display on the HMI device to terminate a high level, which is the driving level included in the autonomous driving, and to cause the driver to take a low level corresponding state in which the driving level lower than the high level can be executed as the driver state, and displays an operational instruction display to cause the driver to take a medium level corresponding state in which the driving level of a medium level, which is a driving level between the high level and the low level, can be executed.

[0007] In addition, in each section of the application documents, each element may be given a reference symbol in parentheses. In this case, the reference symbol merely indicates an example of the correspondence between the element and the specific configuration described in the embodiment described later. Therefore, the present invention is not limited in any way by the description of the reference symbol. [Brief description of the drawings]

[0008] [Figure 1] 1 is a schematic diagram showing the external appearance of the interior of a vehicle cabin in a vehicle equipped with an in-vehicle system including a driving control device and an HMI control device. [Diagram 2] 2 is a block diagram showing a schematic configuration of the in-vehicle system shown in FIG. 1. [Diagram 3] 2 is a schematic diagram showing a display example of an HMI device in the first embodiment. FIG. [Figure 4] 4 is a time chart showing the transition of driving automation levels in the first embodiment. [Diagram 5] 2 is a schematic diagram showing a display example of an HMI device in the first embodiment. FIG. [Figure 6] 2 is a schematic diagram showing a display example of an HMI device in the first embodiment. FIG. [Figure 7] 2 is a schematic diagram showing a display example of an HMI device in the first embodiment. FIG. [Figure 8] 4 is a time chart showing the transition of driving automation levels in the first embodiment. [Figure 9] 13 is a time chart showing the transition of driving automation levels in the second embodiment. [Figure 10] 13 is a time chart showing the transition of driving automation levels in the second embodiment. [Figure 11] 13 is a time chart showing the transition of driving automation levels in the third embodiment. [Figure 12] FIG. 13 is a schematic diagram showing a display example of an HMI device in a third embodiment. [Figure 13] 13 is a time chart showing the transition of driving automation levels in the third embodiment. [Figure 14] FIG. 13 is a schematic diagram showing a display example of an HMI device in a third embodiment. [Figure 15] 13 is a time chart showing the transition of driving automation levels in the fourth embodiment. [Figure 16] FIG. 13 is a schematic diagram showing a display example of an HMI device in a fourth embodiment. [Figure 17] 13 is a time chart showing the transition of driving automation levels in the fifth embodiment. [Figure 18] 13 is a time chart showing the transition of driving automation levels in the fifth embodiment. [Figure 19] 13 is a time chart showing the transition of driving automation levels in the fifth embodiment. [Figure 20] 13 is a time chart showing the transition of driving automation levels in the sixth embodiment. [Figure 21] 13 is a time chart showing the transition of driving automation levels in the sixth embodiment. [Figure 22] 13 is a time chart showing the transition of driving automation levels in the seventh embodiment. [Diagram 23] 13 is a time chart showing the transition of driving automation levels in the seventh embodiment. [Figure 24] 13 is a time chart showing the transition of driving automation levels in the seventh embodiment. [Diagram 25] 13 is a time chart showing the transition of driving automation levels in the eighth embodiment. [Figure 26] 13 is a time chart showing the transition of driving automation levels in the eighth and ninth embodiments. [Figure 27] 13 is a time chart showing the transition of driving automation levels in the ninth and tenth embodiments. [Figure 28] 10 is a time chart showing the transition of driving automation levels in the tenth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] (Embodiment) Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that if a description of various modified examples applicable to one embodiment is inserted in the middle of a series of descriptions related to the embodiment, it may hinder understanding of the embodiment. For this reason, the modified examples will be described together after the series of descriptions related to the embodiment, rather than in the middle of the series of descriptions related to the embodiment.

[0010] (First embodiment: configuration) Referring to FIG. 1, the vehicle 1 is a so-called standard automobile, and is equipped with multiple passenger seats, such as a driver's seat 2, in the vehicle cabin, which is the internal space of the box-shaped body. An occupant sitting in the driver's seat 2 is hereinafter referred to as the "driver." The line of sight of the driver in a state where the driver takes a standard driving posture in the driver's seat 2 is hereinafter referred to as the "forward." The "standard driving posture" refers to the posture of the driver when the driver sits in the driver's seat 2 with the eyeball arrangement direction of the left and right eyes and the arrangement direction of the left and right shoulders approximately parallel to each other while the vehicle 1 is moving forward and straight, and is appropriately seated in the driver's seat 2 so that he or she can manually drive the vehicle.

[0011] An accelerator pedal 3, a brake pedal 4, and a footrest 5 are provided in front of the driver's seat 2. In addition, a shift lever 6 is provided diagonally forward of the driver's seat 2. The accelerator pedal 3, the brake pedal 4, and the footrest 5 are disposed below a dashboard 7 provided in front of the driver's seat 2. A steering wheel 8 is attached to a steering column (not shown) that extends rearward from the dashboard 7 toward the driver's seat 2. A front windshield 9 is provided above the dashboard 7.

[0012] A vehicle 1 is equipped with an in-vehicle system 10. The vehicle 1 equipped with the in-vehicle system 10 may hereinafter be referred to as a "host vehicle." Fig. 2 shows a schematic block configuration of the in-vehicle system 10. Hereinafter, a schematic configuration of the in-vehicle system 10 will be described with reference to Figs. 1 and 2.

[0013] The in-vehicle system 10 is configured to function as a driving automation system in the vehicle by being installed in the vehicle. In the present embodiment, the in-vehicle system 10 has a configuration capable of automatic driving. That is, the vehicle is configured to be capable of automatic driving by installing the in-vehicle system 10.

[0014] "Autonomous driving" refers to a driving automation level corresponding to levels 3 to 5 in the standard "SAE J3016" published by SAE International, where the driving automation system is responsible for, i.e., executes, all dynamic driving tasks. SAE is an abbreviation for Society of Automotive Engineers. "Dynamic driving tasks" are all operational and tactical functions that need to be performed in real time when operating a vehicle 1 in road traffic, excluding strategic functions. "Strategic functions" include route planning, route selection, etc. Level X in "SAE J3016" is hereinafter simply referred to as "Level X". X is any one of 0 to 5. Hereinafter, the larger the value of X, or the more dynamic driving tasks the driving automation system is responsible for, i.e., executes, the higher the driving automation level is expressed. In contrast, the smaller the value of X, or the fewer dynamic driving tasks the driving automation system is responsible for, i.e., executes, the lower the driving automation level is expressed.

[0015] In "SAE J3016", the contents of levels 0 to 5 are specifically as follows. The names of the driving automation levels listed alongside each level are not those listed in "SAE J3016" but are used for convenience in this specification. In the following level content description, "OEDR" stands for Object and Event Detection and Response, and is also referred to as "detection and response of objects and events". OEDR includes monitoring of the driving environment. Monitoring of the driving environment includes detection, recognition, and classification of objects and events. Monitoring of the driving environment also includes preparation to respond to objects and events as necessary. A "limited domain" is a specific condition in which a certain driving automation system or its function is designed to operate, and is also referred to as an operational design domain or ODD. ODD stands for Operational Design Domain. The limited domain includes at least one of multiple constraints, such as geographical, environmental, speed, and time.

[0016] Level 0: Manual driving – the driver performs all dynamic driving tasks. Level 1: Driving assistance: The driving automation system continuously performs either the longitudinal vehicle motion control subtask (i.e., starting, accelerating / decelerating, and stopping) or the lateral vehicle motion control subtask (i.e., steering) of the dynamic driving task in a specific limited area. However, the driving automation system does not perform both the longitudinal vehicle motion control subtask and the lateral vehicle motion control subtask simultaneously. Level 2: Advanced driving assistance: The driving automation system continuously performs the longitudinal vehicle motion control subtask and the lateral vehicle motion control subtask of the dynamic driving task in a specific limited area. The driver is expected to supervise the driving automation system by performing the OEDR, which is a subtask of the dynamic driving task. Level 3: Conditional Autonomous Driving: The driving automation system continuously performs all dynamic driving tasks in specific limited areas. In principle, the driver is not obligated to perform OEDR such as monitoring the surroundings of the vehicle. However, if it becomes difficult to continue the driving automation level, the driving automation system will request the driver to take over driving with ample time to spare. The driver must respond appropriately to the request. Level 4: Highly automated driving: The driving automation system continuously performs all dynamic driving tasks in specific limited areas. If it becomes difficult to maintain the driving automation level in a limited area, the driving automation system will take action. Level 5: Fully automated driving: The driving automation system performs all dynamic driving tasks continuously, without being limited to specific restricted areas. If it becomes difficult to continue with this level of driving automation, the driving automation system will also respond without being limited to specific restricted areas.

[0017] In this embodiment, the in-vehicle system 10 is configured to be able to realize driving automation levels of levels 0 to 3 in the vehicle. Specifically, the in-vehicle system 10 is configured to be able to execute ACC and LKA corresponding to level 1. ACC is adaptive cruise control, i.e., vehicle distance control. LKA is an abbreviation for Lane Keeping Assistance, which is lane keeping assistance control. In addition, the in-vehicle system 10 is configured to be able to execute "hands-off driving" and "advanced safety driving assistance" corresponding to level 2. "Hands-off driving" refers to the driving automation system automatically executing starting, steering, acceleration / deceleration, and stopping control on the condition that the driver responds appropriately to an intervention request from the driving automation system. "Advanced safety driving assistance" refers to the driving assistance operation performed by the driving automation system operating in parallel at the appropriate time in a scene where there is a possibility of a collision, etc., on the premise that the driver drives the vehicle. Hereinafter, in this specification, unless otherwise specified, the automated driving of level 3 is simply referred to as "automated driving". In addition, "hands-off driving" may be simply referred to as "level 2". Furthermore, "Advanced Safety Driving Assistance" is called "Level 2 [G Mode]."

[0018] (Overall system configuration) 2, the in-vehicle system 10 is an in-vehicle network including an in-vehicle communication line 10A and a plurality of nodes interconnected via the in-vehicle communication line 10A, and is configured to be capable of performing various vehicle controls and various associated display operations while the vehicle is being driven. The in-vehicle system 10 is configured to comply with a predetermined communication standard such as CAN (international registered trademark: international registration number 1048262A). CAN (international registered trademark) is an abbreviation for Controller Area Network.

[0019] The in-vehicle system 10 includes a vehicle state sensor 11, an external state sensor 12, a surroundings monitoring sensor 13, a locator 14, a DCM 15, a navigation device 16, a driver state detection unit 17, a driving control device 18, and an HMI device 20. DCM is an abbreviation for Data Communication Module. The vehicle state sensor 11 to the HMI device 20 are connected to an in-vehicle communication line 10A.

[0020] The HMI device 20 is configured to display images that are visible to at least the vehicle occupants including the driver, and to output audio that is audible to the vehicle occupants. Specifically, the HMI device 20 is configured to provide various information and / or entertainment to the vehicle occupants using image and / or audio input / output devices including the meter panel 21, the HUD device 22, the CID device 23, and the terminal device 24. CID stands for Center Information Display. HUD stands for Head-Up Display. The terminal device 24 is a portable or wearable electronic device brought into the vehicle by the vehicle occupants including the driver, such as a mobile phone, a tablet device, a notebook computer, a portable game console, a smart watch, etc.

[0021] The HMI device 20 includes an HMI control device 25 configured to control the output of images and / or sounds on the meter panel 21 or the like. That is, the HMI control device 25 is configured to control the operation of the HMI device 20 constituting the in-vehicle infotainment system. The meter panel 21, the HUD device 22, and the CID device 23 are connected to the HMI control device 25 so as to be able to communicate information with each other via a sub-communication line different from the in-vehicle communication line 10A. When the terminal device 24 is brought into the vehicle, it is connected to the HMI control device 25 so as to be able to communicate information with each other via short-range wireless communication such as Bluetooth (registered trademark) or TransferJet (registered trademark). The HMI control device 25 is provided as a node connected to the in-vehicle communication line 10A. The configurations of the HMI device 20 and the HMI control device 25 will be described in detail later.

[0022] (Various sensors) The vehicle state sensor 11 is provided to generate outputs corresponding to quantities related to the driving state of the vehicle. The "various quantities related to the driving state" include, for example, accelerator opening, braking amount, shift position, steering angle, and other quantities related to the driving operation state by the driver or the driving automation system. The "various quantities related to the driving state" include, for example, physical quantities related to the behavior of the vehicle, such as vehicle speed, angular velocity, forward / rearward acceleration, and left / right acceleration. That is, the vehicle state sensor 11 is a collective term for well-known sensors necessary for vehicle driving control, such as an accelerator opening sensor, a steering angle sensor, a wheel speed sensor, an angular velocity sensor, and an acceleration sensor, for the sake of simplicity of illustration and explanation. The vehicle state sensor 11 is provided so as to be able to provide detection outputs to each part of the driving control device 18 and the like via the in-vehicle communication line 10A.

[0023] The external environment sensor 12 is provided to generate an output corresponding to various quantities related mainly to the natural environment among the driving environment of the vehicle. The "various quantities related to the natural environment" include physical quantities such as outside air temperature, rainfall, and illuminance. That is, the external environment sensor 12 is a collective name for well-known sensors such as an outside air temperature sensor, a raindrop sensor, and an illuminance sensor for the sake of simplicity of illustration and explanation. The external environment sensor 12 is provided to be able to provide detection outputs to each part such as the driving control device 18 via the in-vehicle communication line 10A.

[0024] The periphery monitoring sensor 13 is provided to detect mainly the driving environment of the vehicle other than those detectable by the external state sensor 12. Specifically, the periphery monitoring sensor 13 is configured to detect moving objects and stationary objects within a predetermined detection range around the vehicle. "Moving objects" include pedestrians, cyclists, animals, and other vehicles that are in operation. "Stationary objects" include objects fallen on the road, guardrails, curbs, parked vehicles, road signs, and road markings, as well as roadside structures (e.g., walls, buildings, etc.). The periphery monitoring sensor 13 may also be referred to as an "ADAS sensor." ADAS is an abbreviation for Advanced Driver-Assistance Systems.

[0025] In this embodiment, the perimeter monitoring sensor 13 has a front camera 131 and a radar sensor 132 as components for detecting moving and stationary objects. The front camera 131 is provided to capture images in front of and on the front sides of the vehicle. The front camera 131 is a digital camera device and includes an image sensor such as a CCD or CMOS. CCD stands for Charge Coupled Device. CMOS stands for Complementary MOS.

[0026] The radar sensor 132 is a millimeter wave radar sensor, a sub-millimeter wave radar sensor, or a laser radar sensor configured to transmit and receive radar waves, and is mounted on the front part of the body of the host vehicle. The radar sensor 132 is configured to output a signal corresponding to the position and relative speed of a reflection point. A "reflection point" is a point on the surface of an object present around the host vehicle that is estimated to have reflected radar waves. The "relative speed" is the relative speed of the reflection point, i.e., the object that reflected the radar waves, with respect to the host vehicle.

[0027] (locator) Locator 14 is configured to acquire highly accurate position information of the vehicle by so-called composite positioning. Specifically, locator 14 has GNSS receiver 141, inertia acquisition unit 142, high-precision map DB 143, and locator ECU 144. GNSS is an abbreviation for Global Navigation Satellite System. DB is an abbreviation for database. ECU is an abbreviation for Electronic Control Unit. "High-precision position information" is, for example, position information having a position accuracy that can be used for driving automation levels of level 2 or higher, specifically, an error of less than 10 cm.

[0028] The GNSS receiver 141 is provided to receive positioning signals transmitted from a plurality of positioning satellites, i.e., artificial satellites. In this embodiment, the GNSS receiver 141 is configured to be able to receive positioning signals from positioning satellites in at least one of satellite positioning systems such as GPS, QZSS, GLONASS, Galileo, IRNSS, and the Beidou satellite navigation system. GPS is an abbreviation for Global Positioning System. QZSS is an abbreviation for Quasi-Zenith Satellite System. GLONASS is an abbreviation for Global Navigation Satellite System. IRNSS is an abbreviation for Indian Regional Navigation Satellite System.

[0029] The inertia acquisition unit 142 is configured to acquire the acceleration and angular velocity acting on the host vehicle. In this embodiment, the inertia acquisition unit 142 is provided as a three-axis gyro sensor and a three-axis acceleration sensor built into a box-shaped housing of the locator 14.

[0030] The high-precision map DB 143 is mainly composed of a non-volatile rewritable memory so as to store high-precision map information in a rewritable manner and to retain the stored contents even during power interruption. The non-volatile rewritable memory is, for example, a hard disk, an EEPROM, a flash ROM, etc. EEPROM is an abbreviation for Electronically Erasable and Programmable ROM. ROM is an abbreviation for Read Only Memory. The high-precision map information may also be referred to as high-precision map data. The high-precision map information includes map information with higher accuracy than map information used in conventional car navigation systems that can handle position errors of about several meters. Specifically, the high-precision map DB 143 stores information that can be used for driving automation levels of level 2 or higher, such as three-dimensional road shape information, lane number information, and regulation information, in accordance with a predetermined standard such as the ADASIS standard. ADASIS is an abbreviation for Advanced Driver Assistance Systems Interface Specification.

[0031] The locator ECU 144 is configured as a so-called in-vehicle microcomputer equipped with a CPU, ROM, RAM, an input / output interface, etc. (not shown). CPU is an abbreviation for Central Processing Unit. RAM is an abbreviation for Random Access Memory. The locator ECU 144 is configured to sequentially determine the position, direction, etc. of the vehicle based on the positioning signal received by the GNSS receiver 141, the acceleration and angular velocity acquired by the inertia acquisition unit 142, the vehicle speed acquired from the vehicle state sensor 11, etc. The locator 14 is configured to be able to provide the results of the determination of the position, direction, etc. by the locator ECU 144 to each unit such as the navigation device 16, the driving control device 18, and the HMI control device 25 via the in-vehicle communication line 10A.

[0032] (DCM) The DCM 15 is an in-vehicle communication module that is provided to be capable of communicating with base stations around the vehicle by wireless communication conforming to a communication standard such as LTE or 5G. LTE is an abbreviation for Long Term Evolution. 5G is an abbreviation for 5th Generation.

[0033] Specifically, for example, the DCM 15 is configured to acquire the latest high-precision map information from a probe server on the cloud. The DCM 15 also stores the acquired latest high-precision map information in the high-precision map DB 143 by cooperating with the locator ECU 144. The DCM 15 is also configured to acquire traffic information such as congestion information from the above-mentioned probe server and / or a predetermined database. The "traffic congestion information" includes the location and length of a congestion section. Specifically, the traffic congestion information includes the head position of the congestion, the tail position of the congestion, an estimated congestion distance, an estimated congestion time, and the like. The traffic information is also referred to as "road traffic information."

[0034] (Navigation device) The navigation device 16 is provided to acquire a planned driving route from the current position of the vehicle to a predetermined destination. In this embodiment, the navigation device 16 is configured to calculate a planned driving route based on the destination set by the driver of the vehicle, high-precision map information acquired from the locator 14, and the position information and direction information of the vehicle acquired from the locator 14. The navigation device 16 is also provided so as to be able to provide various information including route information, which is the calculation result, to each part such as the driving control device 18 and the HMI control device 25 via the in-vehicle communication line 10A. That is, the navigation device 16 is configured to display a navigation screen display for displaying a map, a route, etc., on the HMI device 20.

[0035] (Driver status detection unit) The driver state detection unit 17 is provided to detect the driver state. The "driver state" is the state of the driver seated in the driver's seat 2 of the vehicle, and includes at least one of the line of sight, posture, behavior, psychological state, etc. The driver state detection unit 17 is also provided so as to be able to provide the detection result of the driver state to each unit such as the driving control device 18 and the HMI control device 25 via the in-vehicle communication line 10A.

[0036] In this embodiment, the driver state detection unit 17 includes a gaze detection unit 171, a posture detection unit 172, and an operation state detection unit 173. The gaze detection unit 171 is provided to detect the driver's face direction and / or gaze direction by image recognition based on an image captured by an in-vehicle camera equipped with an image sensor such as a CCD or CMOS. That is, the gaze detection unit 171 has a configuration similar to that of a DSM device that issues a warning regarding the driver's inattentive driving, etc. DSM is an abbreviation for Driver Status Monitor.

[0037] The posture detection unit 172 is provided to detect the seating posture of the driver in the driver's seat 2 using the above-mentioned in-vehicle camera and / or a physical quantity sensor such as a seating pressure sensor provided inside the driver's seat 2. The operation state detection unit 173 is provided to detect the resting states of the driver's feet on the accelerator pedal 3, the brake pedal 4, and the footrest 5, and the operation states of the accelerator pedal 3 and the brake pedal 4. The operation state detection unit 173 is also provided to detect the gripping state and operation state of the steering wheel 8 by the driver.

[0038] (Operation control device) The driving control device 18 has a configuration as an "automated driving ECU" or a "driving assistance ECU." That is, the driving control device 18 is configured to control the driving of the host vehicle based on signals and information acquired from the vehicle state sensor 11, the external state sensor 12, the surrounding monitoring sensor 13, the locator 14, and the like. Specifically, the driving control device 18 is configured to execute a predetermined driving control operation. In this embodiment, the "predetermined driving control operation" includes a vehicle control operation, i.e., a dynamic driving task execution operation, corresponding to levels 1 to 3. In this embodiment, the driving control device 18 is configured to be able to set the driving automation level of the host vehicle to any one of level 0, level 1 [ACC], level 1 [LKA], level 2, level 2 [G mode], and level 3.

[0039] The driving control device 18 has a configuration as a so-called in-vehicle microcomputer equipped with a CPU, ROM, non-volatile rewritable memory, RAM, an input / output interface, etc. (not shown). Specifically, the driving control device 18 has the following functional configurations or functional units realized on the in-vehicle microcomputer. That is, the driving control device 18 has a driving situation acquisition unit 181, a driver state acquisition unit 182, an operation state determination unit 183, an automation level determination unit 184, a vehicle control unit 185, and a display command transmission unit 186.

[0040] The driving condition acquisition unit 181 is provided to acquire at least the driving condition of the vehicle. The "driving condition" includes the driving condition and the driving environment detected or acquired by the vehicle condition sensor 11, the external environment condition sensor 12, the surroundings monitoring sensor 13, etc. The driving condition acquisition unit 181 is also provided to acquire high-precision map information of the current position of the vehicle and its surroundings, and traffic information of the road on which the vehicle is currently traveling. That is, the driving condition acquisition unit 181 is configured to acquire information necessary for vehicle control corresponding to levels 1 to 3 from the vehicle condition sensor 11, the external environment condition sensor 12, the surroundings monitoring sensor 13, the locator 14, the DCM 15, etc.

[0041] The driver state acquisition unit 182 is provided to acquire the driver state. Specifically, the driver state acquisition unit 182 is configured to acquire, i.e., receive, from the driver state detection unit 17, the detection result of the driver state by the driver state detection unit 17.

[0042] The operation state determination unit 183 is provided to determine the driving operation state by the driver. Specifically, the operation state determination unit 183 is adapted to acquire, i.e., receive, the detection result by the operation state detection unit 173 from the driver state detection unit 17. The operation state determination unit 183 is also adapted to acquire, i.e., receive, the input operation state of the driver in the HMI device 20 from the HMI device 20. Then, the operation state determination unit 183 is adapted to determine the driving operation state by the driver based on these acquired results.

[0043] The automation level determination unit 184 is configured to determine the driving automation level based on the driving conditions acquired by the driving condition acquisition unit 181, etc. The driving control device 18 is configured to be able to provide the results of the driving automation level determination by the automation level determination unit 184 to each component, such as the HMI control device 25, via the in-vehicle communication line 10A. Details of the determination of the driving automation level by the automation level determination unit 184 will be described in detail in the explanation of the operation overview and operation examples below.

[0044] The vehicle control unit 185 is configured to execute a vehicle motion control subtask according to the driving automation level. That is, the vehicle control unit 185 executes longitudinal and / or lateral motion control of the host vehicle based on the driving automation level determined by the automation level determination unit 184.

[0045] The display command transmission unit 186 is provided to cause the HMI device 20 to execute a level-related display related to the driving automation level by transmitting display command information to the HMI control device 25 that controls the HMI device 20. The level-related display includes an execution-related display related to the execution state of the driving automation level and a transition-related display related to the transition of the driving automation level. The execution-related display is mainly a display for clearly indicating the driving automation level being executed, for example, "automated driving in progress". The transition-related display is a display related to the end of the driving automation level being executed and / or the transition to another driving automation level when the driving automation level being executed is transitioned to another driving automation level due to the completion of the termination condition of the driving automation level being executed or the non-satisfaction of the continuation condition. The transition-related display is, for example, "automated driving ends 3 km ahead", "preparing for driving changeover", "hands-off driving is possible", etc. Further details and other specific examples of the execution-related display and the transition-related display will be described in detail in the operation overview and operation example described later.

[0046] (HMI device) The HMI device 20 is provided to at least visually present various information related to the vehicle to the driver and to accept input operations by the driver corresponding to the presented information. In this embodiment, the HMI device 20 mounted on the vehicle capable of automatic driving is configured to be able to present various information related to automatic driving and the like and to accept input operations by the driver. "Information presentation" is, for example, various kinds of guidance, input operation instructions, input operation content notification, warnings, and the like.

[0047] As described above, the HMI device 20 includes the meter panel 21, the HUD device 22, and the CID device 23, which are provided on the dashboard 7. That is, in this embodiment, the HMI device 20 has a configuration as a so-called "dashboard HMI." The HMI device 20 also includes a speaker (not shown) for presenting information by voice.

[0048] The meter panel 21 has a meter 211, a meter display 212, and a meter switch 213. The meter 211 is provided to execute meter display of the vehicle speed, engine RPM, coolant temperature, remaining fuel, etc. The meter display 212 is an information display section or information display area provided in the center of the meter panel 21 in the vehicle width direction, and is provided to be able to execute display of various information such as date and time, outside temperature, mileage, radio station, etc. In this embodiment, the meter display 212 has a configuration as a display device that is a liquid crystal display or an organic EL display having a substantially rectangular displayable area. EL is an abbreviation for electroluminescence. The meter switch 213 is provided to be able to accept various operations related to the display state or display contents of the meter 211 and / or the meter display 212, for example, a reset operation of the trip meter, etc.

[0049] The HUD device 22 is provided to display a display image including characters and / or symbols in front of the driver. That is, the HUD device 22 is configured to form a virtual display image in front of the driver using AR technology, thereby superimposing and displaying the display image on the foreground including the road surface ahead of the vehicle. The "superimposed display" refers to displaying related information (e.g., building name) of a superimposed object (e.g., building) included in the foreground in such a manner that the superimposed object and the related information are associated with each other by displaying the related information so as to be superimposed on the superimposed object or in the vicinity of the superimposed object. The "superimposed display" also includes a route display, a traveling direction display, a traffic information display, and the like for the road surface ahead. Specifically, the HUD device 22 projects a display image light constituting the display image onto a predetermined projection range PA on the front windshield 9, and allows the driver to visually recognize the reflected light of the display image light by the front windshield 9, thereby displaying the display image in AR.

[0050] The CID device 23 is provided at approximately the center of the dashboard 7 in the vehicle width direction. The CID device 23 is provided so as to be capable of displaying a navigation display screen for displaying a map, a route, and the like by the navigation device 16. The CID device 23 is also provided so as to be capable of displaying information and contents different from the navigation display screen. Specifically, the CID device 23 is configured so as to be capable of executing displays related to driving modes such as "comfort", "normal", "sports", "circuit", and the like.

[0051] The CID device 23 is also configured to be able to execute a display related to a second task that the driver can use during automatic driving. The CID device 23 is configured to be able to execute, for example, viewing video content as a second task. "Video content" is, for example, a movie, a concert video, a music video, a television broadcast, etc. The second task also includes the operation of the terminal device 24.

[0052] The CID device 23 has a CID display 231, an input device 232, and a CID switch 233. The CID display 231 is provided at a substantially central position in the vehicle width direction of the dashboard 7, i.e., at a position between the driver's seat 2 and the passenger seat, so as to be visible at least to the driver. The CID display 231 has a configuration as a display device that is a liquid crystal display or an organic EL display. When the second task is viewing video content, the CID display 231 is configured to display an image of the video content by the display device.

[0053] Input device 232 is a transparent touch panel, and is provided so as to cover CID display 231 by being placed on top of CID display 231. In other words, input device 232 is configured to be able to accept input operations by the driver or the like corresponding to the display on CID display 231 while allowing the driver or the like to visually recognize the display on CID display 231. CID switch 233 has a plurality of manually operated switches arranged around CID display 231 and input device 232.

[0054] The HMI device 20 has a steering switch and the like in addition to the meter switch 213 and the CID switch 233. The steering switch is provided on the spokes of the steering wheel 8. The HMI device 20 is provided so as to be able to provide the results of the input operation received by the driver to each part such as the driving control device 18 via the in-vehicle communication line 10A.

[0055] (HMI control device) The HMI control device 25 has a configuration as an HCU that controls the operations of the meter panel 21, the CID device 23, the HUD device 22, and the like, which are included in the HMI device 20. HCU is an abbreviation for HMI Control Unit.

[0056] The HMI control device 25 has a configuration as a so-called in-vehicle microcomputer equipped with a CPU, ROM, non-volatile rewritable memory, RAM, input / output interface, etc. (not shown). The HMI control device 25 has the following functional configuration or functional units realized on the microcomputer. That is, the HMI control device 25 has a vehicle information acquisition unit 251, a driving environment acquisition unit 252, an automation level acquisition unit 253, a driver state acquisition unit 254, a display control unit 255, and an operation reception unit 256.

[0057] The vehicle information acquisition unit 251 is provided to acquire information related to the driving state of the vehicle. Specifically, the vehicle information acquisition unit 251 acquires, from the vehicle state sensor 11, various quantities related to the driving state of the vehicle, which are detected or acquired by the vehicle state sensor 11.

[0058] The driving environment acquisition unit 252 is configured to acquire information related to the driving environment of the vehicle. Specifically, the driving environment acquisition unit 252 acquires various quantities related to the natural environment around the vehicle, which are detected or acquired by the external environment sensor 12, from the external environment sensor 12. The driving environment acquisition unit 252 also acquires an object detection result by the peripheral monitoring sensor 13 from the peripheral monitoring sensor 13. Furthermore, the driving environment acquisition unit 252 acquires the current position of the vehicle, a planned driving route, and traffic information including congestion information on the planned driving route from the locator 14 and the navigation device 16.

[0059] The automation level acquisition unit 253 is configured to acquire the result of the driving automation level determination made by the driving control device 18. Specifically, the automation level acquisition unit 253 is configured to acquire the result of the driving automation level determination made by the automation level determination unit 184 from the driving control device 18.

[0060] The driver state acquisition unit 254 is provided to acquire the driver state. Specifically, the driver state acquisition unit 254 is configured to acquire the detection result of the driver state by the driver state detection unit 17 from the driver state detection unit 17.

[0061] The display control unit 255 is provided to control the image and / or sound output operation by the HMI device 20. That is, the display control unit 255 is configured to present various information to vehicle occupants including the driver by controlling image output and sound output in the meter panel 21, the HUD device 22, the CID device 23, and the like. The "various information" includes driving state information, driving environment information, information related to the driving automation level, route information, traffic congestion information, various messages, and the like. In addition, the display control unit 255 is configured to cause the terminal device 24 to present various information by cooperating with the terminal device 24 while the driver is performing a second task using the terminal device 24.

[0062] In this embodiment, the display control unit 255 is configured to control the image display operation in the HMI device 20 according to the driving automation level acquired by the automation level acquisition unit 253. That is, the display control unit 255 is configured to cause an image display device such as the meter panel 21 included in the HMI device 20 to display an image according to the driving automation level determined by the automation level determination unit 184 and being executed or scheduled to be executed by the in-vehicle system 10.

[0063] The operation reception unit 256 is provided to receive input operations by vehicle occupants, including the driver, in the HMI device 20. Specifically, the operation reception unit 256 monitors the state or result of reception of input operations by the meter switch 213, the input device 232, the CID switch 233, the terminal device 24, etc., which correspond to various information presented by the display control unit 255.

[0064] (Operation overview) The following describes the operation of the operation control device 18 and the HMI control device 25 according to this embodiment, as well as an overview of the control method and control program executed by these devices, together with the effects achieved by this embodiment.

[0065] In the driving control device 18, the driving condition acquisition unit 181 acquires various information including the driving condition of the vehicle. Specifically, the driving condition acquisition unit 181 acquires the driving condition and driving environment of the vehicle from the vehicle condition sensor 11, the external environment condition sensor 12, and the surroundings monitoring sensor 13. In addition, the driving condition acquisition unit 181 acquires high-precision map information of the current position of the vehicle and its surroundings, a planned driving route, and traffic information on the planned driving route from the locator 14 and the navigation device 16.

[0066] The driver state acquisition unit 182 acquires the driver state. Specifically, the driver state acquisition unit 182 receives the detection result of the driver state by the driver state detection unit 17 from the driver state detection unit 17.

[0067] The operation state determination unit 183 determines the driving operation state by the driver. Specifically, the operation state determination unit 183 acquires the detection result by the operation state detection unit 173 from the driver state detection unit 17. The detection result by the operation state detection unit 173 includes the state of the driver's feet resting on the accelerator pedal 3, the brake pedal 4, and the footrest 5, the operation state of the accelerator pedal 3 and the brake pedal 4, the grip state and operation state of the steering wheel 8, and the like. In addition, the operation state determination unit 183 acquires the input operation state of the driver in the HMI device 20 from the HMI device 20. Then, the operation state determination unit 183 determines the driving operation state by the driver based on the detection result by the operation state detection unit 173 and the input operation state of the driver in the HMI device 20.

[0068] The automation level determination unit 184 determines the driving automation level to be executed by the in-vehicle system 10. The driving automation level is determined based on at least the driving conditions acquired by the driving condition acquisition unit 181. In this embodiment, the automation level determination unit 184 determines the driving automation level based on the results acquired by the driving condition acquisition unit 181 and the driver state acquisition unit 182, and the determination result by the operation state determination unit 183.

[0069] Specifically, the automation level determination unit 184 determines the start conditions for a driving automation level corresponding to levels 1 to 3, based on various information such as the driving conditions acquired by the driving condition acquisition unit 181. When the start conditions for a predetermined driving automation level are met, the automation level determination unit 184 determines that the driving automation level is executable. Then, when an approval operation is performed by the driver, the automation level determination unit 184 decides to execute the driving automation level. The vehicle control unit 185 executes vehicle speed control, steering control, braking control, and the like, according to the driving automation level determined to be executed by the automation level determination unit 184.

[0070] The display command transmission unit 186 causes the HMI device 20 to execute a level-related display related to the driving automation level by transmitting display command information to the HMI control device 25 that controls the HMI device 20. As a result, the HMI device 20 notifies the vehicle occupants, including the driver, of the driving automation level that is being executed or will be executed in the future by image display and / or voice. Specifically, for example, the HMI device 20 displays the driving automation level currently being executed on the meter display 212 or the like. In addition, when autonomous driving becomes possible, the HMI device 20 displays on the meter display 212 or the like that autonomous driving is enabled and an approval operation instruction for receiving an approval operation for starting autonomous driving.

[0071] During autonomous driving, the driver is not obligated to monitor the surroundings of the vehicle until a request to take over driving or to intervene in driving is made by the in-vehicle system 10. In addition, the driver is not required to perform steering control operations and acceleration / deceleration control operations in principle until a request to take over driving or to intervene in driving is made.

[0072] For this reason, during automated driving, the driver is not required to take a manual driving position that allows manual driving. "Driving position" includes driving posture and driving consciousness. "Driving posture" refers to the driver's riding posture, including the seating posture in the driver's seat 2, the positional relationship of the feet to the accelerator pedal 3, etc., the grip and operation state of the steering wheel 8, the direction of gaze, etc. On the other hand, "driving consciousness" refers to the driver's psychological state, including the state of awareness of the driving state and driving environment of the vehicle. Note that the direction of gaze can also be included in "driving consciousness." A level of driving consciousness in which the driver's attention state or monitoring state for the driving state and driving environment of the vehicle is high and which can correspond to a lower driving automation level is hereinafter referred to as "high driving consciousness." In contrast, when the driver's consciousness is away from driving, the driving consciousness is low.

[0073] While the in-vehicle system 10 is stably executing the automatic driving without a request to take over driving or to intervene in driving, the driver is not required to constantly hold the steering wheel 8. The driver is also not required to constantly maintain a driving posture that allows the driver to operate the accelerator pedal 3 and the brake pedal 4 at any time. Furthermore, until the in-vehicle system 10 requests a driver to take over driving or to intervene in driving, the driver may be away from the driving state and driving environment of the vehicle. Therefore, while the in-vehicle system 10 is stably executing the automatic driving, the driver can freely execute or use the second task.

[0074] 3 shows, as a typical example, a display example on CID display 231 and meter display 212 when viewing video content by CID device 23 is being executed as a second task during autonomous driving. In information display area DA1, which is a horizontally long strip-shaped area at the top end of CID display 231, an execution-related display saying "Autonomous driving in progress" that clearly indicates the driving automation level being executed is displayed. In screen area DA2, which is located below information display area DA1 and occupies most of the displayable area of ​​CID display 231, a video content screen, which is the second task screen, is displayed.

[0075] An execution-related display such as "Autonomous driving in progress", which clearly indicates the driving automation level being executed, is displayed in level information display area DB1, which is a horizontally long strip-shaped area at the upper end of meter display 212. In driving information display area DB2, which is located below level information display area DB1 and occupies most of the displayable area of ​​meter display 212, various information such as the lane in which the host vehicle is traveling during autonomous driving is displayed.

[0076] When the termination condition for a specific driving automation level currently being executed is satisfied, or when the continuation condition for that driving automation level is no longer satisfied, the automation level determination unit 184 determines a transition in driving automation level. That is, the automation level determination unit 184 determines the termination of the currently executed driving automation level and the next executable driving automation level.

[0077] The vehicle control unit 185 then executes vehicle speed control, steering control, braking control, and the like, depending on the transition mode of the driving automation level determined by the automation level determination unit 184. In addition, the display command sending unit 186 sends display command information to the HMI control device 25, thereby causing the HMI device 20 to execute transition-related displays related to the transition of the driving automation level.

[0078] (Example of operation) Fig. 4 shows, as an example of a typical operation, an example in which the driving automation level is shifted from autonomous driving to level 2, i.e., hands-off driving, when an autonomous driving possible section ends. In the figure, "LV" is an abbreviation of "level." Times T11 to T15 indicate the passage of time.

[0079] During autonomous driving, as described above, the driver is free to perform the second task without the obligation to monitor the surroundings, etc. Therefore, there is a high possibility that the driver's attention is away from driving during autonomous driving.

[0080] On the other hand, during hands-off driving, the driver is obligated to monitor the surroundings. However, the driver is not required to perform steering control operations and acceleration / deceleration control operations as in the case of autonomous driving. In other words, the driver's behavior regarding driving operations is similar in autonomous driving and hands-off driving. For this reason, when switching from autonomous driving to hands-off driving, the driver may misunderstand the level of driving automation currently being implemented.

[0081] In this regard, it is assumed that the driver's driving awareness can be improved more reliably as the driving automation level decreases if the driver not only receives the information and recognizes and understands it, but also exercises his / her body. Therefore, in this embodiment, when terminating automated driving and transitioning to hands-off driving, the automation level determination unit 184 transitions to a driving automation level lower than hands-off driving and then transitions to hands-off driving.

[0082] In this typical example, as shown in FIG. 4, when terminating autonomous driving and transitioning to hands-off driving, the automation level determination unit 184 transitions to level 0, i.e., manual driving, and then transitions to hands-off driving. This increases the driver's awareness of driving by having the driver perform manual driving once when terminating autonomous driving. Then, by transitioning to hands-off driving after the driver's awareness of driving has been sufficiently increased, the driver can reliably recognize the driving automation level being executed. This makes it possible to have the driver assume an appropriate driving posture adapted to the transition of driving automation levels.

[0083] Specific examples of driving control and display control in this typical example will be described below with reference to Figs. 4 to 7. First, at time T11, the in-vehicle system 10 executes a driving changeover request. Time T11 is set a predetermined time before the scheduled time when the vehicle will arrive at the end point of the autonomous driving possible section. In response to the driving changeover request, the display command transmission unit 186 transmits to the HMI control device 25 display command information for displaying the visual guidance display GA shown in Fig. 5 on the CID display 231, which is a display device that displays the second task screen.

[0084] The line-of-sight guidance display GA is a display that guides the driver's line of sight from the second task screen to the front display device. The "front display device" is a display device arranged in front of the driver, i.e., on the forward direction side of the vehicle with respect to the driver, and in this embodiment, is the meter panel 21 and the HUD device 22. That is, the front display device displays information or images in front of the driver, i.e., on the forward direction side of the vehicle with respect to the driver.

[0085] Specifically, the HMI control device 25 displays the gaze guidance display GA, "→→→Watch out ahead→→→", in the information display area DA1 located above the screen area DA2, which is the second task screen display area, on the CID display 231. At this time, by displaying the gaze guidance display GA as an animation, it is possible to further enhance the gaze guidance effect. In the animation display, for example, one of the multiple "→"s that is brighter than the others can be moved in the gaze guidance direction, i.e., to the right toward the front display device. Or, for example, the character string "Watch out ahead" can be moved in the gaze guidance direction.

[0086] In addition, in response to the driving change request, the display command transmission unit 186 transmits display command information to the HMI control device 25 to display the transition-related display and the operation instruction display shown in Fig. 5 on the front display device. As a result, the meter panel 21 displays a transition-related display, "Automatic driving ends 3 km ahead," in the top level information display area DB1 of the meter display 212. In addition, the meter panel 21 displays a first operation instruction display GB1, a second operation instruction display GB2, a third operation instruction display GB3, and a fourth operation instruction display GB4 in the driving information display area DB2 of the meter display 212. The first operation instruction display GB1, the second operation instruction display GB2, the third operation instruction display GB3, and the fourth operation instruction display GB4 are displays that instruct the driver to take a predetermined driving posture.

[0087] 5, the meter display 212 displays a first action instruction display GB1 saying "Prepare for driving changeover (manual)". The meter display 212 also displays a second action instruction display GB2 saying "Watch out ahead!" The meter display 212 also displays a third action instruction display GB3 saying "Hold the steering wheel!" The meter display 212 also displays a fourth action instruction display GB4 saying "Put your feet on the pedals!"

[0088] In the specific example shown in FIG. 5, the first action instruction display GB1, "Prepare for driving changeover (manual)", corresponds to the transition-related display. The first action instruction display GB1 also corresponds to the main action instruction display instructing the main action. The "main action" is an action necessary for transferring authority to the driver as the driving automation level decreases, and includes an action of adjusting the driving posture. The second action instruction display GB2, "Watch out ahead!" corresponds to the secondary action instruction display instructing the secondary action. The "secondary action" is an action that promotes or assists the main action, and includes an action of adjusting the driving consciousness. The second action instruction display GB2 also corresponds to the main action instruction display. The third action instruction display GB3, "Hold the steering wheel!" corresponds to the main action instruction display. The fourth action instruction display GB4, "Put your feet on the pedals!" corresponds to the main action instruction display. That is, in the driving changeover request, the display command transmission unit 186 transmits display command information to the HMI control device 25, which causes the main action instruction display and the secondary action instruction display shown in FIG. 5 to be displayed on the HMI device 20.

[0089] The HUD device 22 displays a transition-related display of "Preparing for Driving Switch (Manual)". The display by the HUD device 22 has a narrow visible range and can be easily viewed by approaching the manual driving posture. Therefore, the driver views the line-of-sight guidance display GA on the CID display 231, and the driver's line of sight is guided to the transition-related display by the HUD device 22, so that the driver's posture naturally approaches the manual driving posture. Then, the driver views the main action instruction display and the secondary action instruction display, which effectively makes the driver aware of taking the manual driving posture. In this way, by guiding the driver's line of sight to the transition-related display by the HUD device 22 as a front display device, the driver is effectively encouraged to quickly take the manual driving posture.

[0090] In this way, when the display shown in FIG. 5 is executed by the HMI device 20 at time T11, the driving changeover operation, i.e., the main operation and the secondary operation, is started at time T12. The time lag from the display start time T11 to the driving changeover operation start time T12 corresponds to the reaction time of a standard driver. After the display start time T11, the automation level determination unit 184 monitors whether the driving changeover operation has ended based on the driver state acquisition result by the driver state acquisition unit 182 and the driving operation state determination result by the operation state determination unit 183. Then, when the driving changeover operation ends at time T13, manual driving becomes possible. Therefore, the automation level determination unit 184 shifts the driving automation level from automatic driving to manual driving at time T13 before time T14 at which the end point of the automatic driving possible section is reached. During manual driving, as shown in FIG. 6, a navigation screen is displayed in the screen area DA2 on the CID display 231. On the other hand, in the information display area DA1, the visual guidance display GA continues to be displayed in order to prevent the driver from gazing at the navigation screen for a long period of time.

[0091] After manual driving starts at time T13, when the driver's driving operation state stabilizes, a smooth transition from manual driving to hands-off driving is possible. Therefore, the operation state determination unit 183 determines the driving operation state by the driver. Then, the automation level determination unit 184 permits the execution of hands-off driving on the condition that the operation state determination unit 183 determines that the driving operation state corresponding to manual driving has stabilized.

[0092] Specifically, when the driving operation state corresponding to the manual driving is stabilized, the display command transmission unit 186 transmits display command information for displaying a transition related display and an approval request display on the HMI device 20 to the HMI control device 25. The approval request display is a display that prompts or requests the driver to input an operation to approve the execution of hands-off driving.

[0093] 6, the meter display 212 displays a transition-related display saying "Hands-off driving is possible" in the level information display area DB1. The meter display 212 also displays a first input request display GC1 showing an approval button having the character string "Approve" and a second input request display GC2 showing a disapproval button having the character string "Do not approve" in the driving information display area DB2.

[0094] When the driver performs an approval operation at time T15, the automation level determination unit 184 determines to perform hands-off driving, and the in-vehicle system 10 starts hands-off driving.

[0095] As shown in Fig. 7, during hands-off driving, the CID display 231 displays an execution-related display of "hands-off driving" in the information display area DA1, and displays a navigation screen in the screen area DA2. Also, the meter display 212 displays an execution-related display of "hands-off driving" in the level information display area DB1, and displays various information such as route information in the driving information display area DB2. Fig. 7 shows an example in which route information to turn right at an intersection 800 m ahead, a radio station, an outside temperature, and the current date and time are displayed in the driving information display area DB2.

[0096] Figure 4 shows the transition of the driving automation level when the driver complies with the driving takeover request. In contrast, when the driver does not comply with the driving takeover request and does not perform the primary action, the driving automation level transitions as shown in Figure 8.

[0097] That is, if the driving changeover operation that should have been performed between times T12 and T13 is not performed, the automation level determination unit 184 continues autonomous driving until time T14, when the end point of the autonomous driving possible section is reached. During that time, the automation level determination unit 184 waits for the driving changeover operation to be performed. If the driving changeover operation is performed by time T14, the automation level determination unit 184 switches the driving automation level from autonomous driving to manual driving. On the other hand, if the driving changeover operation is not performed by time T14, the automation level determination unit 184 transitions the driving automation level from autonomous driving to hands-off driving at time T14.

[0098] Even if the driving changeover operation that should be performed at times T12 to T13 is not performed, the display command transmission unit 186 continues to transmit display command information to the HMI control device 25 to display the operation instruction display shown in FIG. 5 on the front display device. As a result, the driving changeover request for realizing a driving posture that allows hands-off driving to be continued satisfactorily continues for a predetermined time even after time T14 has passed. Then, if a driving posture that allows hands-off driving to be continued satisfactorily has not been realized even after a predetermined time has passed since time T14, the in-vehicle system 10 executes a predetermined emergency process at time T16. The emergency process is, for example, a rest suggestion or a so-called MRM for safely stopping the vehicle. MRM is an abbreviation for Minimal Risk Maneuver.

[0099] (First embodiment: summary) Thus, in this embodiment, when ending a high automation level and transitioning to a medium automation level, the automation level determination unit 184 transitions from the high automation level to a low automation level once, and then transitions to the medium automation level. The high automation level is a driving automation level included in automated driving levels 3 to 5. The medium automation level is a driving automation level lower than the high automation level. The low automation level is a driving automation level even lower than the medium automation level. This makes it possible to effectively raise the driver's awareness regarding the transition of driving automation levels due to the end of the high automation level included in automated driving levels 3 to 5.

[0100] If the in-vehicle system 10 is configured to support level 4 or higher, the "high automation level" can be level 4 or 5. In this case, the "low automation level" can be level 0 or 1.

[0101] In this embodiment, the automation level determination unit 184 permits execution of the medium automation level on the condition that at least the operation state determination unit 183 determines that the driving operation state corresponding to the low automation level has stabilized. Specifically, when the driving operation state corresponding to the low automation level has stabilized, the display command transmission unit 186 transmits display command information to the HMI control device 25 to cause the HMI device 20 to display an approval request display. The approval request display is a display that prompts the driver to input an operation to approve the execution of the medium automation level. The automation level determination unit 184 executes the medium automation level when a predetermined medium automation level execution condition is met, a predetermined driving operation state is stable, and driver approval is given. This makes it possible to transition the driving automation level even more stably.

[0102] Second Embodiment The second embodiment will be described below with reference to Figs. 9 and 10. In the following description of the second embodiment, the differences from the first embodiment will be mainly described. In the first and second embodiments, the same reference numerals are used for the same or equivalent parts. Therefore, in the following description of the second embodiment, the description of the first embodiment may be appropriately used for the components having the same reference numerals as those in the first embodiment, unless there is a technical contradiction or a special additional description. The same applies to the third embodiment described below.

[0103] The configuration of the in-vehicle system 10 according to this embodiment is the same as that of the first embodiment. That is, the vehicle 1 and the in-vehicle system 10 according to this embodiment have the configurations shown in Figures 1 and 2. However, the operation mode and the corresponding functional configuration of this embodiment are slightly different from those of the first embodiment.

[0104] In this embodiment, the automation level determination unit 184 sets the driving automation level at the low automation level according to the cause of termination of the high automation level. That is, the driving automation level at the low automation level that is executed once before transitioning from the high automation level to the medium automation level is variable according to the cause of termination of the high automation level. This makes it possible to allow the driver to adopt a more appropriate driving posture according to the cause of termination of the high automation level.

[0105] Specifically, for example, assume the first case in which the driving automation level is shifted from automated driving to hands-off driving when the automated driving section ends. In this first case, the driver's driving awareness does not need to be raised to a level where he or she can immediately handle manual driving, and it is sufficient if the awareness is raised to a level where the driver can smoothly shift to hands-off driving.

[0106] Therefore, in this case, as shown in Fig. 9, the automation level determination unit 184 transitions from automated driving to level 1, for example, ACC, and then transitions to hands-off driving. Note that times T11 to T15 shown in Fig. 9 are the same as those in Fig. 4.

[0107] In contrast, a second case is assumed in which the autonomous driving is interrupted due to the detection of an obstacle on the road while driving in an autonomous driving section. In this case, it is preferable that the driver has a higher driving consciousness than in the first case. Therefore, in this case, as shown in FIG. 10, the automation level determination unit 184 transitions from autonomous driving to level 0, i.e., manual driving, and then transitions to hands-off driving.

[0108] 10, first, at time T21, the in-vehicle system 10 executes a driving changeover request. Time T21 can be regarded as substantially the same as the time when the automation level determination unit 184 determines that the automated driving should be interrupted due to the detection of an obstacle on the road, etc.

[0109] When a driver changeover request is executed by the HMI device 20 at time T21, the driver changeover operation, i.e., the main operation and the secondary operation, is started at time T22, a predetermined time lag after the driver changeover request. Then, when the driver changeover operation ends at time T23, the driving automation level transitions from automatic driving to manual driving.

[0110] T24 indicates the time when the conditions for interrupting automated driving are no longer met, such as when a detected road obstacle is passed safely. After time T24 has elapsed, when the driver's driving operation state stabilizes, a smooth transition from manual driving to hands-off driving becomes possible.

[0111] Therefore, when the driving operation state corresponding to manual driving is stable, the display command sending unit 186 sends display command information to the HMI control device 25 to cause the HMI device 20 to display a transition-related display regarding the transition to hands-off driving and an approval request display. The automation level determination unit 184 transitions the driving automation level from manual driving to hands-off driving at time T25 when the approval operation by the driver is accepted.

[0112] Third embodiment Hereinafter, the third embodiment will be described with reference to Fig. 11 to Fig. 14. The configuration of the in-vehicle system 10 according to this embodiment is similar to that of the first and second embodiments. This embodiment is slightly different from the first and second embodiments in the operation mode and the corresponding functional configuration.

[0113] Figure 11 shows an example in which the driving automation level transitions from autonomous driving to Level 1 (i.e., ACC) once and then to hands-off driving as the autonomous driving possible section ends. Times T31 to T35 in Figure 11 are the same as times T11 to T15 in Figure 4, respectively. In this example, it is assumed that the first operation instruction display GB1 to the fourth operation instruction display GB4 shown in Figure 12 are displayed when a driving change request is made at time T31.

[0114] In this case, the first action instruction display GB1 "Prepare for driver handover (ACC)" corresponds to a main action instruction display instructing a main action required for driver handover. On the other hand, the second action instruction display GB2 "Watch out ahead!" corresponds to the main action instruction display and also effectively functions as a secondary action instruction display instructing a secondary action mainly for raising driving awareness.

[0115] In ACC, which is Level 1 and is the transitional level from autonomous driving, steering operation by the driver is essential, but accelerator and brake operation by the driver are not essential. Therefore, the third action instruction display GB3 "Hold the steering wheel!" corresponds to a main action instruction display that instructs a main action. On the other hand, the fourth action instruction display GB4 "Put your feet on the pedals!" corresponds to a secondary action instruction display that instructs a secondary action.

[0116] Ideally, both the primary action and the secondary action are executed accurately from the start of the driving changeover action at time T32 until the end point of the automatic driving possible section is reached at time T34. However, while the primary action is an action essential for continuing driving the vehicle, the secondary action is an auxiliary action for increasing driving awareness. Therefore, if the primary action is executed correctly, even if the secondary action is not executed or there is some deficiency, it is acceptable to execute the shift of the driving automation level. Conversely, if the primary action is executed correctly, but emergency processing such as MRM is executed every time due to non-execution or improper execution of the secondary action, the user of the vehicle may feel inconvenienced.

[0117] Therefore, the operation state detection unit 173 detects the execution state or execution status of the main action and secondary action by the driver. If the main action is not executed, the vehicle control unit 185 executes a predetermined emergency process such as MRM. Figure 13 shows the transition of the driving automation level in this case.

[0118] Specifically, if at least the main operation of the driving switching operations that should have been executed between times T32 and T33 is not executed, the automation level determination unit 184 continues the autonomous driving until time T34 when the end point of the autonomous driving possible section is reached. During that time, the automation level determination unit 184 waits for the execution of the main operation. If the main operation is executed by time T34, the automation level determination unit 184 switches the driving automation level from autonomous driving to ACC. On the other hand, if the main operation is not executed by time T34, the automation level determination unit 184 transitions the driving automation level from autonomous driving to manual driving at time T34. Note that even after transitioning to manual driving, collision avoidance braking, collision damage mitigation braking, etc. may be executed at any time by the vehicle control unit 185.

[0119] Even if the driver change operation that should have been performed between times T32 and T33 is not performed, the display command transmission unit 186 continues to transmit display command information to the HMI control device 25 to cause the front display device to display the operation instruction display shown in Fig. 12. This causes the driver change request to continue for a predetermined time even after time T34 has passed. Then, if the driver change has not been realized even after a predetermined time has passed since time T34, at time T36, the in-vehicle system 10 executes a predetermined emergency process such as MRM.

[0120] On the other hand, there may be cases where the secondary action is not executed even though the primary action is executed. That is, for example, the driver may be sitting in the driver's seat 2 with his legs crossed and his right shoulder raised off the backrest while looking ahead and gripping the steering wheel 8 with his right hand in preparation for switching to ACC. In this case, although the driver's driving posture is compatible with ACC, it is preferable to correct the sitting posture in order to further increase the driver's awareness of driving.

[0121] Therefore, in this case, the vehicle control unit 185 does not execute the above-mentioned safety stop control. Specifically, as shown in Fig. 11, the automation level determination unit 184 shifts the driving automation level from autonomous driving to level 1 (i.e., ACC) at time T33. As a result, as shown in Fig. 14, an execution-related display saying "ACC driving assistance in progress" is displayed in the information display area DA1 on the CID display 231, and a navigation screen is displayed in the screen area DA2.

[0122] Meanwhile, the display command transmitting unit 186 transmits display command information for displaying a warning display on the HMI device 20 to the HMI control device 25. As a result, the HUD device 22 displays a warning display saying "Put your feet on the pedals!" as shown in Fig. 14. Also, the meter display 212 continues to display the fourth action instruction display GB4 as a warning display saying "Put your feet on the pedals!"

[0123] (Fourth embodiment) The fourth embodiment will be described below with reference to Fig. 15 and Fig. 16. The configuration of the in-vehicle system 10 according to this embodiment is the same as that of each of the above-mentioned embodiments. That is, the vehicle 1 and the in-vehicle system 10 according to this embodiment have the configurations shown in Fig. 1 and Fig. 2. This embodiment is slightly different from each of the above-mentioned embodiments in terms of the operation mode and the corresponding functional configuration.

[0124] In each of the above embodiments, the transition of the driving automation level is adjusted so as to induce physical movement of the driver and effectively increase driving awareness when the driving automation level is lowered due to the end of the high automation level. However, the same effect can be achieved by making the driving automation level "apparently" transition by presenting information using the HMI device 20, even if the driving control device 18 does not "actually" transition the driving automation level to a low automation level as in each of the above embodiments.

[0125] Typically, when transitioning between driving automation levels, the driving control device 18 determines the actual driving automation level and performs the transition operation, and the HMI device 20 presents various related information. Therefore, as long as the information presented using the HMI device 20 can induce physical movements of the driver that enhance driving awareness, it is sufficient that the actual transition of the driving automation level by the driving control device 18 does not occur. Therefore, in this embodiment, the transition of the driving automation level in the first embodiment described above is performed "virtually" by presenting information such as a display on the HMI device 20.

[0126] That is, in this embodiment, when the high automation level is ended, the display control unit 255 causes the HMI device 20 to display an operational instruction display that causes the driver to assume a low-level compatible state. The low-level compatible state is a driver state capable of executing a low automation level. In this embodiment, the low automation level is a driving automation level in which the in-vehicle system 10 including the driving control device 18 does not execute at least one of lateral movement control by steering and longitudinal movement control by acceleration and deceleration. In other words, the low-level compatible state is a driver state, i.e., a driving posture, that is compatible with driving automation levels from level 0 to 1.

[0127] As an example of a typical operation, Fig. 15 shows an example in which the driving automation level is shifted from autonomous driving to hands-off driving when an autonomous driving possible section ends, similar to the first embodiment described above. Times T41 to T45 in Fig. 15 are respectively the same as times T11 to T15 in Fig. 4. In Fig. 15, the thin solid line indicates the transition of the actual driving automation level, and the thick dashed line indicates the transition of the driving automation level corresponding to the execution-related display on the HMI device 20.

[0128] 15, the driving control device 18 ends the automatic driving when the automatic driving possible section ends, and transitions to hands-off driving of level 2 without transitioning to level 1 or lower. Meanwhile, the display control unit 255 causes the HMI device 20 to display an operation instruction display for making the driver temporarily take a low-level compatible state where manual driving is possible.

[0129] Then, the driving automation level of the system actually executed by the driving control device 18 does not shift from automated driving to level 1 or lower, but shifts to hands-off driving of level 2. In contrast, the driving automation level in the driver's consciousness, based on recognition of the display content on the HMI device 20 and the accompanying physical movements, shifts from automated driving to a driving automation level lower than hands-off driving, and then shifts to hands-off driving.

[0130] In this way, when the driving automation level of the system transitions from automated driving to hands-off driving, the driver is made to temporarily assume a manual driving posture, which effectively increases the driver's awareness of driving. This makes it possible to have the driver assume an appropriate driving posture adapted to the transition of the driving automation level. This makes it possible to effectively raise the driver's awareness of the transition of the driving automation level due to the end of automated driving.

[0131] Specifically, at time T41, the in-vehicle system 10 executes a driving change request. In response to the driving change request, the display command transmission unit 186 transmits to the HMI control device 25 display command information for displaying the line-of-sight guidance display GA shown in FIG. 16 on the CID display 231, which is a display device that displays the second task screen. The line-of-sight guidance display GA shown in FIG. 16 is the same as that of the first embodiment shown in FIG. 5. The display command transmission unit 186 also transmits to the HMI control device 25 display command information for displaying the transition-related display and the operation instruction display shown in FIG. 16 on the front display device.

[0132] As a result, the meter panel 21 displays a transition-related display saying "Automatic driving ends 3 km ahead" in the level information display area DB1 on the meter display 212. The meter panel 21 also displays the first operation instruction display GB1 to the fourth operation instruction display GB4 similar to the first embodiment described above in the driving information display area DB2 on the meter display 212. The HUD device 22 displays a transition-related display saying "Prepare for driving changeover (manual)". The CID device 23 displays a line-of-sight guidance display GA in the information display area DA1 on the CID display 231.

[0133] In this way, when a driving changeover request is executed at time T41, a driving changeover operation is started at time T42. At this time, the first operation instruction display GB1, "Prepare for driving changeover (manual)", corresponds to a transition-related display. Furthermore, although the display of the driving automation level of the transition destination is manual driving, which is a lower driving automation level than the original hands-off driving, the first operation instruction display GB1 still displays the need to transfer authority from the system to the driver. For this reason, the first operation instruction display GB1 also corresponds to a main operation instruction display for instructing the main operation required for transferring authority to the driver as the driving automation level is lowered.

[0134] The second action instruction display GB2 "Watch out ahead!" corresponds to the main action instruction display and also functions effectively as a secondary action instruction display for instructing a secondary action including an action for adjusting driving awareness. As described above, the system transitions from the automatic driving to the hands-off driving. In the hands-off driving, the driver is not required to perform steering control operation and acceleration / deceleration control operation. For this reason, the third action instruction display GB3 "Hold the steering wheel!", which corresponds to manual driving, is not an essential action for transitioning from automatic driving to hands-off driving, but is related to an action for improving driving awareness. Therefore, the third action instruction display GB3 corresponds to a secondary action instruction display. Similarly, the fourth action instruction display GB4 "Put your feet on the pedals!" corresponds to a secondary action instruction display. In this way, the display control unit 255 causes the HMI device 20 to display the main action instruction display instructing the main action including the action for adjusting the driving posture and the secondary action instruction display instructing the secondary action including the action for adjusting the driving awareness as action instruction displays.

[0135] When the driving changeover operation ends at time T43 and the driver assumes a driving position that allows manual driving, the automation level determination unit 184 shifts the driving automation level on the system from automated driving to level 2 [G mode]. Meanwhile, the display control unit 255 shifts the driving automation level on the display from automated driving to manual driving. As a result, from time T43 to time T45 when hands-off driving actually starts, even though the vehicle appears to be driving manually, i.e., the display shows that the system is executing driving assistance equivalent to level 2 in "SAE J3016."

[0136] After the driving changeover operation is completed at time T43, when the driving operation state of the driver becomes stable, a smooth transition to hands-off driving is possible. Therefore, the operation state determination unit 183 determines the driving operation state of the driver. Then, the automation level determination unit 184 permits the execution of hands-off driving on the condition that the operation state determination unit 183 determines that the driving operation state corresponding to manual driving has become stable.

[0137] Specifically, when the driver state acquired by the driver state acquisition unit 254 is stabilized in a low-level compatible state, the display control unit 255 causes the HMI device 20 to display an approval request display that prompts an input operation to approve the execution of the medium automation level. The approval request display is the same as that shown in FIG. 6. Then, when the driver performs an approval operation at time T45, the automation level determination unit 184 determines to execute hands-off driving. As a result, the in-vehicle system 10 starts hands-off driving. That is, after time T45, the driving automation level on the system and the driving automation level on the display match, and both become hands-off driving. In this way, by transitioning to hands-off driving with the driver's driving awareness sufficiently enhanced, the transition from automated driving to hands-off driving can be performed even safely.

[0138] Thus, in this embodiment, when the high automation level is terminated, the display control unit 255 causes the HMI device 20 to display an operation instruction display that causes the driver to take a low-level corresponding state that can execute a low automation level as the driver state. Specifically, when the high automation level is terminated and a transition is made to a medium automation level that is a driving automation level between the high automation level and the low automation level, the display control unit 255 causes the HMI device 20 to display an operation instruction display that causes the driver to take a low-level corresponding state. The low automation level corresponding to the operation instruction display is an apparent driving automation level that is different from the driving automation level actually executed by the driving control device 18 between the end of the high automation level and the start of the medium automation level. This makes it possible to effectively raise the driver's awareness regarding the transition of driving automation levels due to the end of automated driving.

[0139] In this embodiment, the display control unit 255 causes the CID display 231 displaying the second task screen to display a visual guidance display GA for guiding the driver's line of sight from the second task screen to the HUD device 22 serving as the front display device. This effectively encourages the driver to quickly assume a predetermined driving posture by guiding the driver's line of sight to the HUD device 22 serving as the front display device.

[0140] Fifth embodiment Hereinafter, the fifth embodiment will be described with reference to Figs. 17 to 19. The configuration of the in-vehicle system 10 according to this embodiment is similar to that of the fourth embodiment. However, the operation mode and the corresponding functional configuration of this embodiment are slightly different from those of the fourth embodiment. That is, this embodiment is a partial modification of the fourth embodiment.

[0141] The fourth embodiment is an embodiment in which the actual driving automation level transition in the first embodiment is changed to a virtualized, i.e., apparent, one. Similarly, the present embodiment is an embodiment in which the actual driving automation level transition in the second embodiment is changed to a virtualized, i.e., apparent, one.

[0142] In this embodiment, the display control unit 255 sets the driving automation level at the low automation level according to the cause of ending the high automation level. That is, the driving automation level at the low automation level to which the vehicle apparently transitions from the high automation level upon ending the high automation level is variable according to the cause of ending the high automation level. This makes it possible to allow the driver to adopt a more appropriate driving posture according to the cause of ending the high automation level.

[0143] Figure 17 shows a case in which the driving automation level is shifted from autonomous driving to hands-off driving as the autonomous driving possible section ends, similar to Figure 9. Times T51 to T55 in Figure 17 are similar to times T11 to T15 in Figures 4 and 9, respectively. In Figure 17 and subsequent figures, thin solid lines indicate the transition of the actual driving automation level, and thick dashed lines indicate the transition of the driving automation level corresponding to the execution-related display on HMI device 20.

[0144] In the example shown in Figure 17, the driving automation level in the system transitions from automated driving to level 2 [G mode] at time T53. Then, at time T55 when the driving operation state becomes stable and an approval operation by the driver is received, the driving automation level in the system transitions from level 2 [G mode] to hands-off driving.

[0145] In contrast, the driver's driving awareness does not need to be raised to a level where he or she can immediately respond to manual driving, but is sufficient if it is raised to a level where he or she can smoothly transition to hands-off driving. Therefore, the driving automation level displayed transitions from automated driving to level 1 at time T53, and transitions from level 1 to hands-off driving at time T55. That is, the display mode when a driving handover request is made at time T51 is the same as that shown in FIG.

[0146] Fig. 18 shows a case where autonomous driving is interrupted due to detection of an obstacle on the road while traveling in an autonomous driving possible section, similar to Fig. 10. Times T61 to T65 in Fig. 18 are the same as times T21 to T25 in Fig. 10, respectively.

[0147] Even when autonomous driving is interrupted due to detection of an obstacle on the road, etc., it is preferable that the driving automation level on the system after the interruption be as high as possible. Therefore, in this example, the driving automation level on the system transitions from autonomous driving to level 2 [G mode] at time T63 when the driving handover operation is completed. Then, at time T65 when the driving operation state becomes stable and the driver's approval operation is accepted, the driving automation level on the system transitions from level 2 [G mode] to hands-off driving.

[0148] On the other hand, in this example, it is preferable that the driver has a higher level of driving awareness than in the example of the end of the autonomous driving possible section shown in Figure 17. Therefore, the driving automation level on the display temporarily shifts from autonomous driving to manual driving at time T63, and then shifts from manual driving to hands-off driving at time T65. That is, the display mode when a driving changeover request is made at time T61 corresponds to the shift from autonomous driving to manual driving, similar to that in Figure 16.

[0149] Fig. 19 shows a case where autonomous driving is terminated due to the occurrence of some kind of abnormality in the in-vehicle system 10. In this case as well, it is preferable that the driver has a higher level of driving awareness than in the example of the end of the autonomous driving possible section shown in Fig. 17. Therefore, the display control unit 255 shifts the driving automation level on the display from autonomous driving to manual driving.

[0150] On the other hand, it is preferable that the driving automation level or driving assistance level of the system is as high as possible. Therefore, the automation level determination unit 184 reduces the driving automation level of the system from autonomous driving to a predetermined level. The predetermined level is the highest level that can be implemented at that time.

[0151] Specifically, when the operation control device 18 determines that the automatic operation cannot be continued due to the occurrence of an abnormality, it executes a request to switch the operation to manual operation at time T71. That is, the display command transmission unit 186 transmits display command information to the HMI control device 25 so as to cause the HMI device 20 to display the request to switch the operation.

[0152] When a driving change request is executed by the HMI device 20 at time T71, a driving change operation is initiated at time T72. Then, when the driving change operation ends at time T73, the driving automation level displayed is shifted from automated driving to manual driving. Meanwhile, the driving automation level displayed in the system is shifted to the highest level that can be implemented at that time, although automated driving ends. Even if the driving automation level displayed in the system allows only manual driving, the driver can make an appropriate driving change by shifting the driving automation level displayed from automated driving to manual driving.

[0153] In this example, an abnormality occurs in the driving automation system that makes it impossible to continue autonomous driving. Therefore, even after T74 when the driver's manual driving operation is stable, the approval operation request to shift the driving automation level to a higher level is not executed. However, as described above, although the vehicle appears to be in manual driving mode, the system is allowed to execute as many driving assistance operations as possible.

[0154] In the examples shown in Fig. 17 to Fig. 19, the process when the driver changeover operation is not performed may be the same as that in Fig. 8 and Fig. 13. That is, a predetermined emergency process such as MRM may be executed after waiting for a predetermined time. Alternatively, if the minimum necessary safety control operation such as collision avoidance braking and collision damage mitigation braking is possible, the above emergency process may not be executed even if the driver changeover operation is not performed.

[0155] Sixth embodiment Hereinafter, the sixth embodiment will be described with reference to Fig. 20 and Fig. 21. The configuration of the in-vehicle system 10 according to this embodiment is similar to that of the fourth and fifth embodiments. However, the operation mode and the corresponding functional configuration of this embodiment are slightly different from those of the fourth and fifth embodiments. That is, this embodiment is a partial modification of the fourth and fifth embodiments.

[0156] In this example, although it is possible to switch to hands-off driving when the autonomous driving section ends, in order to improve driving awareness, the HMI device 20 suggests switching to level 1 (i.e., ACC). Fig. 20 shows a case where the suggestion to switch to ACC has been approved, and Fig. 21 shows a case where such approval has not been performed.

[0157] That is, in the example shown in Fig. 20, the cause of ending autonomous driving is the end of the section where autonomous driving is possible and a transition to ACC due to ACC approval. On the other hand, in the example shown in Fig. 21, the cause of ending autonomous driving is the end of the section where autonomous driving is possible and a transition to hands-off driving due to ACC not being approved. Due to such differences in the causes of ending, the transition manner from autonomous driving to the driving automation level on the system and the driving automation level on the display is different between Fig. 20 and Fig. 21.

[0158] Specifically, in the example shown in Fig. 20, first, at time T81, the in-vehicle system 10 executes a driving changeover request similar to time T41 in Fig. 15 or time T51 in Fig. 17. In addition, at time T82, the in-vehicle system 10 executes a proposal to switch to ACC. Then, at time T83, the driver executes an approval operation for the proposal to switch to ACC and starts a driving changeover operation to assume a driving posture compatible with ACC.

[0159] In this example, the driver has a clear intention to shift from autonomous driving to ACC. Therefore, when the driving switching operation ends at time T84, which is before time T85, the in-vehicle system 10 shifts both the driving automation level on the system and the driving automation level on the display from autonomous driving to ACC (level 1). Time T85 is the time when the end point of the autonomous driving possible section is reached. Thereafter, at time T86, when the driver's driving operation state is stable, if the conditions for shifting the driving automation level to a higher level, for example, level 2 [G mode], are met, a shift proposal can be appropriately executed.

[0160] On the other hand, in the example shown in Fig. 21, first, at time T91, the in-vehicle system 10 executes a driving change request similar to the example shown in Fig. 20. In addition, at time T92, the in-vehicle system 10 executes a proposal to shift to ACC. Times T91 and T92 are similar to times T81 and T82 shown in Fig. 20, respectively.

[0161] In this example, the driver does not execute the approval operation for the proposal to switch to ACC until time T93. In this case, the driver may not have a clear intention to switch from automated driving to a predetermined driving automation level. Therefore, at time T93, the in-vehicle system 10 executes a request to switch to manual driving, similar to the request at time T41 in FIG. 15.

[0162] When the driving switching operation to manual driving ends at time T94, prior to time T95, the automation level determination unit 184 shifts the driving automation level on the system from automated driving to level 2 [G mode]. Time T95 is the time at which the end of the section where automated driving is possible is reached. Meanwhile, the display control unit 255 shifts the driving automation level on the display from automated driving to manual driving. As a result, after time T94, even though the vehicle appears to be driving manually, i.e., on the display, the system is in a state where driving assistance equivalent to level 2 in "SAE J3016" is being executed.

[0163] Thereafter, when the driver's driving operation state stabilizes, a smooth transition to hands-off driving is possible. Therefore, when the driver's driving operation state stabilizes, a proposal to transition to hands-off driving is executed. When an approval operation for such a transition proposal is executed at time T96, the in-vehicle system 10 transitions both the driving automation level on the system and the driving automation level on the display to hands-off driving.

[0164] Seventh embodiment The seventh embodiment will be described below with reference to Fig. 22 to Fig. 24. The configuration of the in-vehicle system 10 according to this embodiment is similar to that of the first embodiment, etc. This embodiment is slightly different from the first embodiment, etc. in the operation mode and the corresponding functional configuration.

[0165] Similar to the example of FIG. 4, FIGS. 22 to 24 show an example of transitioning from autonomous driving (i.e., level 3) as a high automation level to level 2 as a medium automation level. In FIGS. 22 to 24, times TA1 to TA6 indicate the passage of time. In the example of FIG. 22, the example of FIG. 23, and the example of FIG. 24, the vehicle's travel route and travel conditions (e.g., vehicle speed, etc.) are the same, and only the duration of autonomous driving being performed at time TA1 is different. That is, FIG. 22 shows an example in which the duration of autonomous driving is short, i.e., less than a predetermined time. On the other hand, FIGS. 23 and 24 show examples in which the duration of autonomous driving is equal to or longer than a predetermined time. The duration of autonomous driving in the example of FIG. 23 is shorter than that in the example of FIG. 24.

[0166] The contents of the times TA1 to TA6 are as follows. The changeover request start time TA1 is the start time of the driving changeover request. The driving changeover request includes displaying a transition-related display and an action instruction display on the front display device. That is, the changeover request start time TA1 corresponds to the action instruction timing for displaying the action instruction display (i.e., at least the main action instruction display) on the HMI device 20. Therefore, the changeover request start time TA1 is the same as the display start time T11 in FIG. 4. The changeover start time TA2 is the time when the driver starts the driving changeover action in response to the driving changeover request, and is the same as the driving changeover action start time T12 in FIG. 4. The changeover end time TA3 is the time when the driver's driving changeover action started at the changeover start time TA2 ends, and is the same as the driving changeover action end time T13 in FIG. 4. The end point passing time TA4 is the time when the vehicle reaches the end point of the automatic driving possible section, and is the same as the time T14 in FIG. 4. For ease of explanation, Figures 22 to 24 are drawn so that the horizontal time axis is approximately the same scale and the horizontal axis direction position of the end point passing time TA4 is the same. The approval period start time TA5 is the timing at which the driver can perform the approval operation. Specifically, the approval period start time TA5 corresponds to, for example, the approval request timing at which an approval request display is displayed on the HMI device 20. Alternatively, the approval period start time TA5 corresponds to, for example, the approval permission timing at which an input operation to approve the execution of the medium automation level is permitted. The transition time TA6 is the approval execution timing at which the driver performs the approval operation to execute the medium automation level execution, and is the same as time T15 in Figure 4.

[0167] When the duration of autonomous driving is short, it is assumed that the driver's driving awareness has not deteriorated much compared to before the start of autonomous driving, and the driving operation sense (e.g., response to steering amount, etc.) has not been lost. In contrast, the longer the duration of autonomous driving, the more likely it is that the driver's driving awareness will deteriorate and the driving operation sense will be lost. Therefore, in order to effectively restore the driver's driving awareness and driving operation sense, it is preferable to have the driver take a driving posture corresponding to a lower driving automation level as the duration of autonomous driving is longer. Therefore, in this embodiment, the automation level determination unit 184 sets the change mode of the driving automation level from the high automation level to the medium automation level according to the duration of the high automation level.

[0168] Specifically, when the duration of the high automation level is less than a predetermined time, the automation level determination unit 184 transitions from the high automation level to the medium automation level without transitioning to the low automation level, as shown in FIG. 22. In addition, the automation level determination unit 184 sets the driving automation level at the low automation level according to the duration of the high automation level. More specifically, in the example of FIG. 24, where the duration of the automated driving is relatively long, it is assumed that the degree of decline in driving consciousness and loss of driving operation sense is greater than that in the example of FIG. 23, where the duration of the automated driving is not so long. Therefore, in the example of FIG. 24, in order to significantly recover the driving consciousness and driving operation sense, it is effective to have the driver experience level 0, which is a lower driving automation level, that is, manual driving. On the other hand, in the example of FIG. 23, a sufficient recovery of the driving consciousness and driving operation sense is expected by decreasing to level 1. Therefore, the automation level determination unit 184 sets the driving automation level at the low automation level lower the longer the duration of the high automation level. As a result, the longer the autonomous driving continues and the greater the possibility of the driver losing his / her driving awareness and / or driving operation sense, the longer the preparation period can be secured for the driver to regain his / her driving awareness and / or driving operation sense when the autonomous driving ends.

[0169] In addition, in this embodiment, the display command transmission unit 186 or the display control unit 255 sets the operation instruction timing, i.e., the changeover request start time TA1, according to the duration of the high automation level. That is, the longer the duration of the automatic driving, the earlier the changeover request start time TA1 is set. Specifically, referring to FIG. 22 to FIG. 24, the end point passing time TA4, the approval period start time TA5, and the transition time TA6 are almost the same or have almost no difference between FIG. 22 to FIG. 24. In contrast, the changeover request start time TA1 is the latest in the example of FIG. 22, in which the duration of the automatic driving is the shortest, and is the earliest in the example of FIG. 24, in which the duration of the automatic driving is the longest. As a result, the longer the duration of the automatic driving, the longer the time interval between the changeover request start time TA1 and the end point passing time TA4 can be secured. Such a time interval is a period that can be used as a preparation period for the driver to recover driving consciousness and / or driving operation sense when transitioning from a high automation level to a medium automation level. Therefore, according to this embodiment, the longer the duration of autonomous driving, the longer the preparation period possible for the driver to regain driving awareness and / or driving operation sense when autonomous driving ends can be secured.

[0170] As shown in Figs. 22 to 24, in this embodiment, the time interval from the changeover request start time TA1 to the approval period start time TA5 is set to be longer as the duration of the high automation level becomes longer. In other words, if the changeover request start time TA1 is taken as a reference time point, the timing of the approval period start time TA5 based on this reference time point is set later as the duration of the high automation level becomes longer. In this way, according to this embodiment, the approval request timing or approval permission timing specified by the approval period start time TA5 is set according to the duration of the high automation level. As a result, the longer the duration of the autonomous driving, the longer the preparation period for the driver to recover driving awareness and / or driving operation sense when the autonomous driving ends can be secured as long as possible.

[0171] 22 to 24 are drawn so that the positions of the approval period start time TA5 and the transition time TA6 in the horizontal axis direction are the same. However, this embodiment is not limited to such an aspect. That is, for example, the time interval from the end point passing time TA4 to the approval period start time TA5 can be set to be longer as the duration of the high automation level becomes longer. In other words, the positions of the approval period start time TA5 and the transition time TA6 in FIG. 23 in the horizontal axis direction can be to the right of the positions of the approval period start time TA5 and the transition time TA6 in FIG. 22 in the horizontal axis direction. Similarly, the positions of the approval period start time TA5 and the transition time TA6 in FIG. 24 in the horizontal axis direction can be to the right of the positions of the approval period start time TA5 and the transition time TA6 in FIG. 23 in the horizontal axis direction.

[0172] Eighth embodiment The eighth embodiment will be described below with reference to Fig. 25 and Fig. 26. The contents of times TA1 to TA6 in Fig. 25 and Fig. 26 are the same as those in Figs. 22 to 24.

[0173] In this embodiment, the in-vehicle system 10 including the driving control device 18 is configured to be capable of performing automated driving at levels 3 and 4. Other than the operational aspects related to these driving automation levels and the corresponding functional configurations, the configuration of the in-vehicle system 10 according to this embodiment is the same as that of the first embodiment and the like.

[0174] In level 4 autonomous driving, if it becomes difficult to continue the level (for example, when an obstacle on the road or a construction zone appears), the driving automation system will respond. In other words, while level 4 autonomous driving is being performed, the driver is not required to respond appropriately to a request from the driving automation system to take over driving. Therefore, while level 4 autonomous driving is being performed, the driver is allowed to sleep. On the other hand, in level 3 autonomous driving, if it becomes difficult to continue the level, the driver must respond appropriately to a request from the driving automation system to take over driving. Therefore, while level 3 autonomous driving is being performed, the driver is required to be awake to an extent that he or she can respond appropriately to a request from the driving automation system to take over driving.

[0175] In the example of Fig. 25 and the example of Fig. 26, the driving route and driving conditions of the vehicle are the same, and only the driver state during autonomous driving is different. Specifically, Fig. 25 shows a case where the driver is not asleep during level 4 autonomous driving, that is, before the change request start time TA1. On the other hand, Fig. 26 shows a case where the driver is asleep during level 4 autonomous driving.

[0176] Both Level 3 and Level 4 can be implemented in "specific limited areas," i.e., sections where autonomous driving is possible. For this reason, it is necessary to request the driver to take over driving with sufficient time to spare before the vehicle reaches the end point of the section where autonomous driving is possible. However, it is assumed that the driver after sleep (i.e., waking up from sleep) has a reduced awareness of driving and / or driving operation ability compared to before the start of sleep-enabled autonomous driving. The reduced ability to drive includes the loss of the above-mentioned sense of driving operation. It is assumed that the reduced awareness of driving and / or driving operation ability is particularly noticeable immediately after waking up from sleep.

[0177] Therefore, the driver state detection unit 17 detects whether the driver state while executing the high automation level is asleep or not. Then, the automation level determination unit 184 sets the change mode of the driving automation level from the high automation level to the medium automation level depending on whether the driver state while executing the high automation level is asleep or not. Specifically, when the driver state while executing level 4 as the high automation level is asleep, the automation level determination unit 184 ends the high automation level earlier than when the driver state is not asleep.

[0178] More specifically, when the driver's state is not asleep during automated driving at level 4 as a high automation level, level 4 is maintained until shift end time TA3, as shown in Figure 25. Then, at shift end time TA3, the driving automation level transitions from level 4 to level 1. After that, at transition time TA6, the driving automation level transitions from level 1 to level 2. This transition mode is similar to the level transition mode in the example of Figure 4, where the driving automation level transitions from automated driving to level 1 as a low automation level and then to level 2 as a medium automation level.

[0179] In contrast, if the driver's state during level 4 automated driving is a sleeping state, as shown in FIG. 26, at the change request start time TA1, level 4 ends and the driving automation level temporarily transitions to level 3 as a sub-high automation level. The sub-high automation level is a driving automation level included in automated driving between the high automation level and the medium automation level, and is level 3 in the example of FIG. 26. The driving automation level is maintained at level 3 from the change request start time TA1 to the change end time TA3. That is, a driving change operation is executed at level 3. Then, at the change end time TA3, the driving automation level transitions from level 3 to level 1. Then, at the transition time TA6, the driving automation level transitions from level 1 to level 2. That is, the driving automation level is set to level 3 from the change start time TA2 to the change end time TA3, which indicate the period of operation during which the driver's state transitions to a low-level compatible state corresponding to level 1. This allows a driver who has fallen asleep under Level 4 autonomous driving to gradually assume an appropriate driving posture under Level 3 autonomous driving, without being forced to suddenly transition to a driving posture corresponding to Level 1.

[0180] Thus, in this embodiment, when the driver state while executing a high automation level is asleep, the automation level determination unit 184 transitions from the high automation level to a sub-high automation level, and then transitions to a low automation level. In contrast, when the driver state while executing a high automation level is not asleep, the automation level determination unit 184 transitions from the high automation level to a low automation level without transitioning to the sub-high automation level. This allows the driving automation level transition control for restoring the driver's driving awareness and / or driving operation ability when autonomous driving ends to be appropriately executed depending on whether the driver actually fell asleep during autonomous driving.

[0181] Ninth embodiment The ninth embodiment will be described below with reference to Figures 26 and 27. The contents of times TA1 to TA6 in Figures 26 and 27 are the same as those in Figures 25 and 26. This embodiment is a partial modification of the eighth embodiment. That is, in this embodiment, the driving automation level transition mode in the example of Figure 25 in the eighth embodiment is modified to the mode shown in Figure 27.

[0182] In the example of Fig. 26 and the example of Fig. 27, the driving route and driving conditions of the vehicle are the same, and only the driver state during autonomous driving is different. Specifically, Fig. 26 shows a case where the driver is asleep during level 4 autonomous driving, that is, before the change request start time TA1. On the other hand, Fig. 27 shows a case where the driver is not asleep during level 4 autonomous driving.

[0183] As mentioned above, it is assumed that the driver's driving awareness and / or driving operation ability is reduced after falling asleep compared to before automated driving began. Therefore, if the driver is not asleep during automated driving, the driver will be able to handle a lower level of driving automation than if the driver was asleep.

[0184] Therefore, in this embodiment, the automation level determination unit 184 sets the driving automation level at the low automation level depending on whether the driver state during execution of the high automation level is a sleeping state. As a result, specifically, when the driver state during automated driving at level 4 as the high automation level is a sleeping state, the low automation level between the changeover end time TA3 and the transition time TA6 is set to level 1 as shown in FIG. 26. On the other hand, when the driver state during automated driving at level 4 as the high automation level is a non-asleep state, the low automation level is set to level 0 as shown in FIG. 27. As a result, driving automation level transition control for restoring the driver's driving awareness and / or driving operation ability when the automated driving ends can be appropriately executed depending on whether the driver actually fell asleep during the automated driving.

[0185] Tenth embodiment The tenth embodiment will be described below with reference to Figures 27 and 28. The contents of times TA1 to TA6 in Figures 26 and 27 are the same as those in Figures 26 and 27. This embodiment is a partial modification of the ninth embodiment. That is, in this embodiment, the driving automation level transition mode in the example of Figure 26 in the ninth embodiment is modified to the mode shown in Figure 28.

[0186] In the example of Fig. 27 and the example of Fig. 28, the driving route and driving conditions of the vehicle are the same, and only the driver state during autonomous driving is different. Specifically, Fig. 27 shows a case where the driver is not asleep during level 4 autonomous driving, that is, before the change request start time TA1. On the other hand, Fig. 28 shows a case where the driver is asleep during level 4 autonomous driving.

[0187] In this embodiment, the display command transmission unit 186 or the display control unit 255 sets the operation instruction timing, i.e., the change request start time TA1, depending on whether the driver state during execution of the high automation level is a sleeping state or not. That is, the change request start time TA1 is set earlier when the driver state during execution of the high automation level is a sleeping state. Specifically, referring to FIG. 27 and FIG. 28, the end point passing time TA4, the approval period start time TA5, and the transition time TA6 are almost the same or have almost no difference between FIG. 27 and FIG. 28. In contrast, the change request start time TA1 is earlier in the example of FIG. 28 where the driver state during execution of the high automation level is a sleeping state than in the example of FIG. 27. This allows the preparation period for restoring the driver's driving awareness and / or driving operation ability, which has been reduced due to sleep during automatic driving, at the end of automatic driving to be secured as long as possible.

[0188] In this embodiment, the time interval from the changeover request start time TA1 to the approval period start time TA5 is set to be longer when the driver state during execution of the high automation level is in a sleeping state. In other words, when the changeover request start time TA1 is set as a reference time point, the timing of the approval period start time TA5 based on the reference time point is set later as the duration of the high automation level becomes longer. Thus, according to this embodiment, the approval request timing or approval permission timing specified by the approval period start time TA5 is set depending on whether the driver state during execution of the high automation level is in a sleeping state or not. This allows the preparation period for restoring the driver's driving awareness and / or driving operation ability, which has decreased due to sleep during autonomous driving, at the end of autonomous driving to be as long as possible.

[0189] 27 and 28 are drawn so that the horizontal axis positions of the approval period start time TA5 and the transition time TA6 are the same. However, this embodiment is not limited to such an aspect. That is, for example, the time interval from the end point passing time TA4 to the approval period start time TA5 can be set depending on whether the driver state during execution of the high automation level is a sleeping state or not. In other words, the horizontal axis positions of the approval period start time TA5 and the transition time TA6 in FIG. 28 can be to the right of the horizontal axis positions of the approval period start time TA5 and the transition time TA6 in FIG. 27.

[0190] Eleventh Embodiment In the seventh to tenth embodiments, the transition of the driving automation level between the high automation level and the medium automation level, i.e., from the shift request start time TA1 to the transition time TA6, was the actual driving automation level. However, the transition of the driving automation level in the seventh to tenth embodiments may be an apparent one, as in the fourth embodiment, etc.

[0191] 23 to 28 may be "apparent" transitions due to information presented using the HMI device 20. In this case, the actual driving automation level determined and executed by the automation level determination unit 184 transitions from automated driving to level 2 at shift end time TA3, in the manner shown in FIG. 22. In addition, the functional configuration or operation of the automation level determination unit 184 and / or the display command transmission unit 186 in each of the above embodiments may be interpreted as the functional configuration or operation of the display control unit 255.

[0192] (Modification) The present invention is not limited to the above embodiment. Therefore, the above embodiment can be modified as appropriate. Representative modified examples will be described below. In the following description of the modified examples, differences from the above embodiment will be mainly described. In addition, the same reference numerals are used for parts that are the same or equivalent in the above embodiment and the modified examples. Therefore, in the following description of the modified examples, the description of the above embodiment can be appropriately used for components having the same reference numerals as the above embodiment, unless there is a technical contradiction or special additional explanation.

[0193] The present invention is not limited to the specific device configuration shown in the above embodiment. That is, for example, the vehicle 1 equipped with the in-vehicle system 10 is not limited to a normal automobile. Specifically, the vehicle 1 may be a large automobile such as a cargo truck. There is no particular limitation on the number of wheels, and the vehicle may be a three-wheeled automobile, or a six- or eight-wheeled automobile such as a cargo truck. The type of the vehicle 1 may be a conventional automobile equipped with only an internal combustion engine, an electric automobile or a fuel cell automobile not equipped with an internal combustion engine, or a so-called hybrid automobile. The shape and structure of the body of the vehicle 1 are not limited to a box shape, i.e., a substantially rectangular shape in a plan view. There are also no particular limitations on the use of the vehicle 1, the position of the steering wheel 8, the number of occupants, and the like.

[0194] The communication standard for the in-vehicle system 10 may be other than CAN (international registered trademark), such as FlexRay (international registered trademark). The communication standard for the in-vehicle system 10 is not limited to one type. For example, the in-vehicle system 10 may have a sub-network line that complies with a communication standard such as LIN. LIN is an abbreviation for Local Interconnect Network.

[0195] The vehicle state sensor 11, the external state sensor 12, and the periphery monitoring sensor 13 are not limited to the above examples. For example, the periphery monitoring sensor 13 may include a sonar, i.e., an ultrasonic sensor. Alternatively, the periphery monitoring sensor 13 may include two or more types of sensors selected from a millimeter wave radar sensor, a sub-millimeter wave radar sensor, a laser radar sensor, and an ultrasonic sensor. There is also no particular limit to the number of each type of sensor to be installed.

[0196] The locator 14 is not limited to the above example. For example, the locator 14 does not need to have a built-in gyro sensor and an acceleration sensor. Specifically, the inertia acquisition unit 142 may receive output signals from an angular velocity sensor and an acceleration sensor provided outside the locator 14 as the vehicle state sensor 11.

[0197] The DCM 15 may be omitted, that is, the traffic information may be acquired by the navigation device 16. Alternatively, the navigation device 16 may have a configuration including the locator 14 and the DCM 15.

[0198] The navigation device 16 may be connected to the HMI control device 25 so as to be capable of communicating information via a sub-communication line different from the in-vehicle communication line 10A.

[0199] The navigation device 16 may have a display screen dedicated to displaying a navigation screen, separate from the HMI device 20. Alternatively, the navigation device 16 may be provided as a part of the HMI device 20. Specifically, for example, the navigation device 16 may be integrated with the CID device 23.

[0200] The driver state detection unit 17 may be connected to the HMI control device 25 so as to be capable of communicating information via a sub-communication line different from the in-vehicle communication line 10A.

[0201] The driver state detection unit 17 is not limited to a configuration including the gaze detection unit 171, the posture detection unit 172, and the operation state detection unit 173. That is, for example, a function corresponding to the posture detection unit 172 can be achieved by image recognition using the configuration of the gaze detection unit 171. The driver state detection unit 17 may also include a bioinformation sensor that detects bioinformation such as the driver's pulse. In this case, components such as detection electrodes in the bioinformation sensor can be shared with components in the operation state detection unit 173 that detect the gripping state of the steering wheel 8.

[0202] In the above embodiment, the driving control device 18 is configured to be able to execute vehicle control operations corresponding to levels 1 to 3. However, the present invention is not limited to this aspect. That is, for example, the present invention can be suitably applied to cases where vehicle control operations corresponding to levels 1 to 5 are executable.

[0203] Furthermore, the levels or categories of driving automation in the present invention are not limited to those specified in "SAE J3016". Specifically, in "SAE J3016", the higher the driving automation level, the larger the level numerical value. However, the present invention is not limited to such an embodiment. That is, for example, the present invention can be similarly applied to a standard in which the highest driving automation level is "Level 1" and the lower the driving automation level, the larger the level numerical value.

[0204] The HMI device 20 is not limited to a configuration including the meter panel 21, the HUD device 22, and the CID device 23. That is, for example, the meter panel 21 and the CID device 23 may be integrated together.

[0205] The meter 211 and the meter display 212 may be realized by one display device. In this case, the meter 211 may be provided as display areas at both the left and right ends of one display device that is a liquid crystal or organic EL display. That is, the meter 211 may be realized by displaying images of a bezel, a pointer, a scale, etc. corresponding to a tachometer, a speedometer, a water temperature gauge, etc. Also, the meter display 212 may be provided as a display area other than the meter 211 in such a display device.

[0206] The input device 232 may have a pointing device or the like operated by the driver instead of or in addition to the touch panel superimposed on the CID display 231. The input device 232 may have a voice input device that detects the driver's speech.

[0207] In the above embodiment, the operation control device 18 and the HMI control device 25 have a configuration as a so-called in-vehicle microcomputer equipped with a CPU, etc. However, the present invention is not limited to such a configuration.

[0208] For example, all or part of the driving control device 18 may be configured to include a digital circuit configured to enable the above-mentioned operations, such as an ASIC or FPGA. ASIC stands for Application Specific Integrated Circuit. FPGA stands for Field Programmable Gate Array. In other words, in the driving control device 18, an on-board microcomputer portion and a digital circuit portion may coexist. The same applies to the HMI control device 25.

[0209] The program according to the present invention, which enables the execution of various operations, procedures, or processes described in the above embodiment, can be downloaded or upgraded via V2X communication by the DCM 15 or the like. V2X is an abbreviation for Vehicle to X. Alternatively, such a program can be downloaded or upgraded via a terminal device provided in a manufacturing plant, a maintenance plant, a dealer, or the like of the vehicle 1. Such a program may be stored on a memory card, an optical disk, a magnetic disk, or the like.

[0210] In this way, each of the above functional configurations and methods may be realized by a dedicated computer provided by configuring a processor and a memory programmed to execute one or more functions embodied in a computer program. Alternatively, each of the above functional configurations and methods may be realized by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, each of the above functional configurations and methods may be realized by one or more dedicated computers configured by combining a processor and a memory programmed to execute one or more functions with a processor configured with one or more hardware logic circuits. Furthermore, the computer program may be stored in a computer-readable non-transient tangible storage medium as instructions to be executed by a computer. In other words, each of the above functional configurations and methods may be expressed as a computer program including a procedure for realizing the same, or as a non-transient tangible storage medium storing the program.

[0211] The present invention is not limited to the specific functional configurations and operation examples shown in the above embodiment. That is, for example, in accordance with the driving automation level defined in "SAE J3016", in the above embodiment, the transition mode of the driving automation level on the system and / or display was 3-0-2, 3-0-1, 3-1-2, 4-0-2, 4-1-2, etc. However, the present invention is not limited to such a mode. That is, for example, such a transition mode may be 4-0-1, 5-2-3, etc. That is, when the transition mode is LA-LB-LC, it is sufficient that LA≧3, LB≦2, and LA>LC>LB.

[0212] In the above embodiment, the display for guiding the line of sight and instructing the operation is executed by the CID device 23. However, the present invention is not limited to such an embodiment. That is, for example, such display may be executed by the terminal device 24. That is, the HMI control device 25 may cause the display screen of the terminal device 24 to execute various displays.

[0213] The visual guidance display GA may be displayed in a screen area DA2, which is a second task screen display area, in the CID display 231. The display form of the visual guidance display GA is not limited to the specific example exemplified in the above embodiment. That is, for example, the visual guidance display GA may be displayed like a meteor or comet that flows in the visual guidance direction.

[0214] In the example shown in FIG. 8 etc., the in-vehicle system 10 may selectively execute only one of the rest suggestion and MRM as the predetermined emergency processing depending on the road conditions. Alternatively, the in-vehicle system 10 may execute the rest suggestion first and then execute MRM. Then, the in-vehicle system 10 may execute MRM if the driver does not assume a driving posture corresponding to level 2 or does not perform any response operation to the rest suggestion even after a predetermined time has elapsed since the driver received the rest suggestion. There are no particular limitations on the content of the predetermined emergency processing.

[0215] 4 and 9, the transition timing at which the driving automation level transitions from a high automation level, i.e., level 3, to a low automation level, i.e., level 0 or level 1, can be changed from the timing at which the driving change operation ends (i.e., time T13). Specifically, such transition timing can be set appropriately between the timing at which the driving change operation ends and the timing at which the end of the autonomous driving section is reached (i.e., between times T13 and T14).

[0216] In the example of Fig. 15, the transition timing at which the actual driving automation level transitions from a high automation level, i.e., level 3, to a medium automation level, i.e., level 2, can be changed from the timing at which the driving changeover operation ends (i.e., time T43). Specifically, such transition timing can be set appropriately between times T43 and T44. The same applies to the examples of Fig. 17, etc.

[0217] Similar expressions such as "obtain," "calculate," "estimate," "detect," "sensing," and "determine" may be substituted for each other as appropriate to the extent that they are not technically inconsistent. "Detect" or "detection" and "extract" may also be substituted for each other as appropriate to the extent that they are not technically inconsistent.

[0218] Needless to say, the elements constituting the above-mentioned embodiment are not necessarily essential, except when expressly stated as essential or when considered to be obviously essential in principle. Furthermore, when the numbers, amounts, ranges, etc. of the components are mentioned, the present invention is not limited to the specific values, except when expressly stated as essential or when clearly limited to a specific value in principle. Similarly, when the shapes, directions, positional relationships, etc. of the components are mentioned, the present invention is not limited to the shapes, directions, positional relationships, etc., except when expressly stated as essential or when clearly limited to a specific shape, direction, positional relationship, etc. in principle.

[0219] The modified examples are not limited to the above examples. For example, all or part of one of the multiple embodiments and all or part of the other may be combined with each other as long as they are not technically inconsistent. There is no particular limit to the number of combinations. Similarly, all or part of one of the multiple modified examples and all or part of the other may be combined with each other as long as they are not technically inconsistent. Furthermore, all or part of the above embodiment and all or part of the above modified example may be combined with each other as long as they are not technically inconsistent. In short, the above embodiments and modified examples may be combined arbitrarily as long as they are not technically inconsistent.

[0220] (summary) The present disclosure described above by the embodiment and the modified examples includes the following aspects related to an HMI control method and an HMI control program. Note that the following aspects may be combined with each other as long as there is no technical contradiction between them.

[0221] The HMI control method is a method for controlling an HMI device (20) that displays images visible to the driver of an autonomously-drivable vehicle (1). The HMI control program is a program executed by an HMI control device (25) configured to control the HMI device (20) that displays images visible to the driver of the autonomously-drivable vehicle (1).

[0222] According to a first aspect, the HMI control method and the process executed by the HMI control device include: An automation level acquisition process for acquiring a result of a driving automation level determination in a driving control device (18) that controls the driving of the vehicle; A display control process that controls an image display operation in the HMI device according to the driving automation level acquired in the automation level acquisition process; having The display control process is a process for displaying, on the HMI device, an operational instruction display that causes the driver to take a low-level corresponding state in which the driving automation level, which is the driving automation level included in the autonomous driving, can execute a low-level corresponding state in which the in-vehicle system (10) including the driving control device does not execute at least one of lateral movement control by steering and longitudinal movement control by acceleration / deceleration, as the driver state that is the state of the driver, when the high automation level, which is the driving automation level included in the autonomous driving, is terminated.

[0223] According to a second aspect, the display control process is a process of displaying, on the HMI device, the operational instruction display that causes the driver to take the low level corresponding state when the high automation level is terminated and a transition is made to a medium automation level, which is the driving automation level between the high automation level and the low automation level.

[0224] According to a third aspect, the HMI control method and the process executed by the HMI control device include: A driver status acquisition process for acquiring the driver status; an operation reception process for receiving an input operation by the driver; and The display control process is a process that, when the driver state acquired by the driver state acquisition process stabilizes in the low-level corresponding state, displays an approval request display on the HMI device prompting the input operation to approve the execution of the medium automation level.

[0225] According to a fourth aspect, the display control process is a process that sets the approval request timing for displaying the approval request display on the HMI device, or the approval permission timing for permitting the input operation to approve the execution of the medium automation level, depending on the duration of the high automation level or whether the driver state while executing the high automation level is a sleeping state or not.

[0226] According to a fifth aspect, the display control process is a process for setting the driving automation level at the low automation level depending on the cause of termination of the high automation level.

[0227] According to a sixth aspect, the display control process is a process that sets the driving automation level at the low automation level depending on the duration of the high automation level or whether the driver state while executing the high automation level is a sleeping state or not.

[0228] According to the seventh aspect, the display control process is a process for setting the operation instruction timing for displaying the operation instruction display on the HMI device depending on the duration of the high automation level or whether the driver state while executing the high automation level is a sleeping state or not.

[0229] According to an eighth aspect, the display control process is a process for displaying, as the operation instruction displays, a main action instruction display instructing a main action including an adjustment action of driving posture as the driver state, and a secondary action instruction display instructing a secondary action including an adjustment action of driving awareness as the driver state, on the HMI device.

[0230] According to a ninth aspect, the display control process is a process of displaying, on a display device (23) that displays the second task screen, a visual guidance display that guides the driver's gaze from a second task screen that displays a second task that the driver can execute while the autonomous driving is being performed to a front display device (21, 22) that is positioned on the vehicle's forward direction.

[0231] According to a tenth aspect, the front display device is a head-up display device (22).

[0232] According to an eleventh aspect, the low automation level corresponding to the operation instruction display is an apparent driving automation level that is different from the driving automation level actually executed by the driving control device between the end of the high automation level and the start of the medium automation level.

[0233] The present disclosure as described above with reference to the embodiment and the modified examples includes the following aspects related to an operation control method and an operation control program. Note that the following aspects may be combined with each other as long as there is no technical contradiction between them.

[0234] The driving control method is a method for controlling driving of an automatically-drivable vehicle (1). The driving control program is a program executed by a driving control device (18) configured to control driving of the automatically-drivable vehicle (1).

[0235] According to a first aspect, the operation control method and the process executed by the operation control device include: A driving status acquisition process for acquiring a driving status of the vehicle; An automation level determination process that determines a driving automation level based on the driving condition acquired by the driving condition acquisition process; having The automation level determination process is a process in which, when a high automation level, which is the driving automation level included in the autonomous driving, is terminated and a medium automation level, which is a driving automation level lower than the high automation level, is transitioned from the high automation level to a low automation level, which is a driving automation level lower than the medium automation level, and then transitioned to the medium automation level.

[0236] According to the second aspect, The driving operation state by the driver is determined by an operation state determination process. The automation level determination process is a process that allows execution of the medium automation level on the condition that the operation state determination process determines that the driving operation state corresponding to the low automation level is stable.

[0237] According to a third aspect, the automation level determination process is a process for setting the driving automation level at the low automation level depending on the cause of termination of the high automation level.

[0238] According to a fourth aspect, the automation level determination process is a process that sets the manner in which the driving automation level changes from the high automation level to the medium automation level depending on the duration of the high automation level or whether the driver state while executing the high automation level is a sleep state or not.

[0239] According to a fifth aspect, the automation level determination process is a process that sets the driving automation level at the low automation level depending on the duration of the high automation level or whether the driver state while executing the high automation level is a sleep state or not.

[0240] According to a sixth aspect, the automation level determination process is a process that terminates the high automation level sooner when the driver state while executing the high automation level is a sleeping state than when the driver state is a non-sleeping state.

[0241] According to the seventh viewpoint, The automation level determination process includes: When the driver state while executing the high automation level is a sleep state, the system first transitions from the high automation level to a sub-high automation level, which is a driving automation level between the high automation level and the medium automation level and is included in the automated driving, and then transitions to the low automation level; When the driver state while executing the high automation level is a non-asleep state, the process transitions from the high automation level to the low automation level without transitioning to the sub-high automation level.

[0242] According to an eighth aspect, the automation level determination process transitions from the high automation level to the medium automation level without transitioning to the low automation level if the duration is less than a predetermined time.

[0243] According to a ninth aspect, the driving control method and the processing executed by the driving control device further include a display command sending processing for sending display command information to an HMI control device (25) that controls the HMI device so as to cause the HMI device (20) to execute a transition-related display related to the transition of the driving automation level.

[0244] According to the tenth aspect, the display command transmission process is a process of transmitting the display command information that causes the HMI device to display an approval request display that prompts the driver to perform an input operation to approve the execution of the medium automation level when the driving operation state corresponding to the low automation level becomes stable.

[0245] According to an eleventh aspect, the display command transmission process is a process of setting an approval request timing for displaying the approval request display on the HMI device, or an approval permission timing for permitting the input operation to approve the execution of the medium automation level, depending on the duration of the high automation level or whether the driver state while executing the high automation level is a sleeping state or not.

[0246] According to a twelfth aspect, the display command transmission process is a process of transmitting the display command information that causes the HMI device to display a main action instruction display that instructs a main action including an adjustment action of driving posture as a driver state, which is the state of the driver, and a secondary action instruction display that instructs a secondary action including an adjustment action of driving awareness as the driver state.

[0247] According to a thirteenth aspect, the display command transmission process is a process for setting an operation instruction timing for displaying the main operation instruction display on the HMI device depending on the duration of the high automation level or whether the driver state while executing the high automation level is a sleeping state or not.

[0248] According to a fourteenth aspect, the operation control method and the process executed by the operation control device include: A driver status acquisition process for acquiring the driver status; A vehicle control process that executes motion control by steering or acceleration / deceleration of the vehicle based on the driving automation level determined by the automation level determination process. and When the main action is not performed, the vehicle control process executes a predetermined safety stop control, If the main operation is executed but the secondary operation is not executed, the vehicle control process does not execute the safety stop control, while the display command transmission process transmits the display command information to display a warning display on the HMI device.

[0249] According to the fifteenth aspect, the display command transmission process is a process of transmitting the display command information for causing a visual guidance display to be displayed on a display device (23) displaying the second task screen, which displays a second task that the driver can execute while the autonomous driving is being executed, to guide the driver's gaze to a front display device (21, 22) arranged on the vehicle's forward direction.

[0250] According to a sixteenth aspect, the front display device is a head-up display device (22). [Explanation of symbols]

[0251] 10. In-vehicle systems 18 Operation control device 20 HMI device 21 Meter panel 22 Head-up display device 25 HMI control device 253 Automation Level Acquisition Department 254 Driver status acquisition unit 255 Display control section 256 Operation reception section

Claims

1. An HMI control device (25) configured to control an HMI device (20) that displays an image visible to a driver of an autonomously operable vehicle (1), A driving level acquisition unit (253) that acquires a result of determining a driving level corresponding to an implementation state including whether or not driving automation is implemented in a driving control device (18) that controls driving of the vehicle; A display control unit (255) that controls an image display operation in the HMI device according to the driving level acquired by the driving level acquisition unit; Equipped with The display control unit causes the HMI device to display an operation instruction display for terminating a high level, which is the driving level included in the autonomous driving, and causing the driver to take a low-level corresponding state in which the driving level that is lower than the high level can be executed as a driver state that is the state of the driver, and performs an operation instruction display for causing the driver to take a medium-level corresponding state in which the driving level of a medium level, which is a driving level between the high level and the low level, can be executed. HMI control device.

2. An HMI control program executed by an HMI control device (25) configured to control an HMI device (20) that displays an image visible to a driver of an autonomously driven vehicle (1), comprising: The process executed by the HMI control device is as follows: A driving level acquisition process for acquiring a driving level determination result corresponding to an implementation state including whether or not driving automation is implemented in a driving control device (18) that controls driving of the vehicle; a display control process for controlling an image display operation in the HMI device according to the driving level acquired in the driving level acquisition process; Including, The display control process displays, on the HMI device, an operation instruction display for terminating a high level, which is the driving level included in the autonomous driving, and causing the driver to take a low-level corresponding state in which the driving level that is lower than the high level can be executed as a driver state that is the state of the driver, and displays an operation instruction display for causing the driver to take a medium-level corresponding state in which the driving level of a medium level, which is a driving level between the high level and the low level, can be executed. HMI control program.

Images