Operation control device and HMI control device

Abstract

To satisfactorily make a driver conscious of a transition between driving levels due to termination of automated driving.SOLUTION: An HMI control device (25) includes a driving level acquisition section (253) and a display control section (255). The driving level acquisition section acquires a determination result of a driving level corresponding to an execution state including whether or not driving automation is executed in a driving control device (18) for controlling driving of a vehicle. The display control section controls image display operation in an HMI device (20) according to the driving level acquired by the driving level acquisition section. The display control section causes the HMI device to display an action instruction display for instructing a driver to take a low-level corresponding state in which, while a high level included in automated driving is terminated, the driving level lower than the high level can be executed, as a driver state, and performs an action instruction display for instructing the driver to take a middle-level corresponding state in which the driving level of a middle level can be executed.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a driving control device that controls the driving of an autonomously driven vehicle, and an HMI control device that controls an HMI device installed in the vehicle. HMI stands for Human Machine Interface. [Background technology]

[0002] Various automated driving systems for vehicles such as automobiles have been proposed (see, for example, Patent Document 1). During automated driving, the driver, who is the occupant in the driver's seat of the vehicle, is able to freely perform a second task. A second task is a task performed by the driver other than driving operations. Specifically, second tasks include, for example, operating a mobile terminal, watching video content, etc. A second task is also called a "non-driving task" or a "secondary activity." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2017-107502 Summary of the Invention [Problem to be solved by the invention]

[0004] Automated driving may end, for example, when the vehicle reaches the end of a specific road section where automated driving is permitted. When automated driving ends, it is necessary to transfer authority for vehicle dynamics control and / or driving environment monitoring back to the driver.

[0005] On the other hand, a driver performing a second task during automated driving is not aware of the driving state and driving environment of the vehicle. Therefore, in order to smoothly transfer authority, it is necessary to properly raise the driver's awareness toward the end of automated driving. The present invention has been made in consideration of the circumstances exemplified above. That is, the present invention provides a technology that enables properly raising the driver's awareness toward the transition of driving automation levels due to the end of automated driving, for example. [Means for solving the problem]

[0006] The HMI control device (25) is configured to control the HMI device (20) that displays an image visible to the driver of the automatically drivable vehicle (1). The HMI control device according to claim 1 comprises: a driving level acquisition unit (253) that acquires a result of determining a driving level corresponding to an implementation state, including whether or not automated driving is implemented, in a driving control device (18) that controls the driving of the vehicle; a display control unit (255) that controls an image display operation in the HMI device according to the driving level acquired by the driving level acquisition unit; Equipped with The display control unit displays an operational instruction display on the HMI device that terminates the high level, which is the driving level included in the autonomous driving, and causes the driver to take a low level corresponding state, which is the driver state of the driver, in which the driving level is lower than the high level, and displays an operational instruction display that causes the driver to take a medium level corresponding state, which is a driving level between the high level and the low level, in which the driving level is able to be performed. The HMI control program is a computer-readable program executed by an HMI control device (25) configured to control an HMI device (20) that displays images visible to the driver of an automatically drivable vehicle (1). The HMI control program according to claim 2 As a process executed by the HMI control device, a driving level acquisition process for acquiring a result of determining a driving level corresponding to an implementation state, including whether or not automated driving is implemented, in a driving control device (18) that controls the driving of the vehicle; and a display control process for controlling an image display operation in the HMI device according to the driving level acquired in the driving level acquisition process; Including, The display control process displays an operational instruction display on the HMI device that terminates the high level, which is the driving level included in the autonomous driving, and causes the driver to assume a low level corresponding state, which is a driver state that can execute the driving level that is lower than the high level, as the driver state of the driver, and displays an operational instruction display that causes the driver to assume a medium level corresponding state, which is a driving level between the high level and the low level, and which can execute the driving level.

[0007] In addition, in each section of the application documents, each element may be assigned a reference symbol in parentheses. In this case, the reference symbol merely indicates an example of the correspondence between the element and the specific configuration described in the embodiment described below. Therefore, the present invention is not limited in any way by the description of the reference symbol. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a schematic diagram showing the exterior of the interior of a vehicle cabin equipped with an in-vehicle system including a driving control device and an HMI control device. [Figure 2] FIG. 2 is a block diagram showing a schematic configuration of the in-vehicle system shown in FIG. [Figure 3] FIG. 2 is a schematic diagram showing a display example of an HMI device in the first embodiment. [Figure 4] 1 is a time chart showing the transition of the driving automation level in the first embodiment. [Figure 5] FIG. 2 is a schematic diagram showing a display example of an HMI device in the first embodiment. [Figure 6] FIG. 2 is a schematic diagram showing a display example of an HMI device in the first embodiment. [Figure 7] FIG. 2 is a schematic diagram showing a display example of an HMI device in the first embodiment. [Figure 8] 1 is a time chart showing the transition of the driving automation level in the first embodiment. [Figure 9] 10 is a time chart showing the transition of the driving automation level in the second embodiment. [Figure 10] 10 is a time chart showing the transition of the driving automation level in the second embodiment. [Figure 11] 10 is a time chart showing the transition of the driving automation level in the third embodiment. [Figure 12] FIG. 10 is a schematic diagram showing a display example of an HMI device in a third embodiment. [Figure 13] 10 is a time chart showing the transition of the driving automation level in the third embodiment. [Figure 14] FIG. 10 is a schematic diagram showing a display example of an HMI device in a third embodiment. [Figure 15] 10 is a time chart showing the transition of driving automation levels in the fourth embodiment. [Figure 16] FIG. 10 is a schematic diagram showing a display example of an HMI device in a fourth embodiment. [Figure 17] 10 is a time chart showing the transition of the driving automation level in the fifth embodiment. [Figure 18] 10 is a time chart showing the transition of the driving automation level in the fifth embodiment. [Figure 19] 10 is a time chart showing the transition of the driving automation level in the fifth embodiment. [Figure 20] 13 is a time chart showing the transition of the driving automation level in the sixth embodiment. [Figure 21] 13 is a time chart showing the transition of the driving automation level in the sixth embodiment. [Figure 22] 13 is a time chart showing the transition of the driving automation level in the seventh embodiment. [Figure 23] 13 is a time chart showing the transition of the driving automation level in the seventh embodiment. [Figure 24] 13 is a time chart showing the transition of the driving automation level in the seventh embodiment. [Figure 25] 13 is a time chart showing the transition of the driving automation level in the eighth embodiment. [Figure 26] 10 is a time chart showing the transition of driving automation levels in the eighth and ninth embodiments. [Figure 27] 10 is a time chart showing the transition of driving automation levels in the ninth and tenth embodiments. [Figure 28] 10 is a time chart showing the transition of driving automation levels in the tenth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] (Embodiment) Hereinafter, embodiments of the present invention will be described with reference to the drawings. It should be noted that if a description of various modifications applicable to one embodiment is inserted in the middle of a series of descriptions relating to the embodiment, it may hinder understanding of the embodiment. Therefore, the modifications will be described together after the series of descriptions relating to the embodiment, rather than in the middle of the series of descriptions relating to the embodiment.

[0010] (First embodiment: configuration) Referring to FIG. 1, vehicle 1 is a so-called standard automobile, and is equipped with multiple passenger seats, including a driver's seat 2, in the cabin, which is the internal space of a box-shaped vehicle body. An occupant sitting in driver's seat 2 will be referred to as the "driver" below. The direction of the driver's line of sight when the driver is in a standard driving position in driver's seat 2 will be referred to as the "forward" below. The "standard driving position" refers to the posture of the driver when the driver is seated in driver's seat 2 with the alignment of his or her eyes and the alignment of his or her shoulders approximately parallel while the vehicle 1 is moving forward and straight, and the driver is able to properly manually drive the vehicle.

[0011] An accelerator pedal 3, a brake pedal 4, and a footrest 5 are provided in front of the driver's seat 2. A shift lever 6 is also provided diagonally in front of the driver's seat 2. The accelerator pedal 3, the brake pedal 4, and the footrest 5 are disposed below a dashboard 7 provided in front of the driver's seat 2. A steering wheel 8 is attached to a steering column (not shown) that extends rearward from the dashboard 7 toward the driver's seat 2. A front windshield 9 is provided above the dashboard 7.

[0012] A vehicle 1 is equipped with an in-vehicle system 10. Hereinafter, the vehicle 1 equipped with the in-vehicle system 10 may be referred to as the "host vehicle." FIG. 2 shows a schematic block configuration of the in-vehicle system 10. The schematic configuration of the in-vehicle system 10 will be described below with reference to FIGS. 1 and 2.

[0013] The in-vehicle system 10 is configured to function as an automated driving system for the vehicle when installed in the vehicle. In this embodiment, the in-vehicle system 10 is configured to be capable of automatic driving. That is, the vehicle is configured to be capable of automatic driving when the in-vehicle system 10 is installed in the vehicle.

[0014] "Automated driving" refers to driving automation levels 3 to 5 in the "SAE J3016" standard published by SAE International, in which the driving automation system is responsible for, or performs, all dynamic driving tasks. SAE stands for Society of Automotive Engineers. "Dynamic driving tasks" refer to all operational and tactical functions that must be performed in real time when operating a vehicle 1 in road traffic, excluding strategic functions. "Strategic functions" include route planning, route selection, etc. Level X in "SAE J3016" is hereinafter referred to simply as "Level X." X ranges from 0 to 5. Hereinafter, the higher the value of X, or the more dynamic driving tasks the driving automation system is responsible for, or performs, the higher the level of driving automation. Conversely, the lower the value of X, or the fewer dynamic driving tasks the driving automation system is responsible for, or performs, the lower the level of driving automation.

[0015] The specific details of Levels 0 to 5 in "SAE J3016" are as follows. Note that the names of the driving automation levels listed next to each level are not those specified in "SAE J3016" but are used for convenience in this specification. In the following level descriptions, "OEDR" stands for Object and Event Detection and Response, and is also referred to as "detection and response of objects and events." OEDR includes monitoring of the driving environment. Driving environment monitoring includes detection, recognition, and classification of objects and events. Driving environment monitoring also includes preparation for responding to objects and events as necessary. A "limited domain" is a specific condition in which a driving automation system or its functions are designed to operate, and is also referred to as an operational design domain (ODD). ODD stands for Operational Design Domain. A limited domain includes at least one of multiple constraints, such as geographical, environmental, speed, and time constraints.

[0016] Level 0: Manual driving: The driver performs all dynamic driving tasks. Level 1: Driving assistance: The driving automation system continuously performs either the longitudinal vehicle motion control subtask (i.e., starting, accelerating, decelerating, and stopping) or the lateral vehicle motion control subtask (i.e., steering) of the dynamic driving task in a specific limited area. However, the driving automation system does not perform both the longitudinal vehicle motion control subtask and the lateral vehicle motion control subtask simultaneously. Level 2: Advanced driving assistance: The driving automation system continuously performs the longitudinal vehicle motion control subtask and the lateral vehicle motion control subtask of the dynamic driving task in a specific limited area. The driver is expected to supervise the driving automation system by performing the OEDR subtask of the dynamic driving task. Level 3: Conditional Autonomous Driving: The driving automation system continuously performs all dynamic driving tasks within specific limited areas. In principle, the driver is not obligated to perform OEDR, such as monitoring the area around the vehicle. However, if it becomes difficult to maintain this level of driving automation, the driving automation system will request the driver to take over driving with ample time to spare. The driver must respond appropriately to this request. Level 4: Highly automated driving: The driving automation system continuously performs all dynamic driving tasks within specific limited areas. If it becomes difficult to maintain this level of driving automation within the limited areas, the driving automation system will take action. Level 5: Fully automated driving: The driving automation system performs all dynamic driving tasks continuously, without being limited to specific limited areas. If it becomes difficult to maintain this level of driving automation, the driving automation system will also respond, without being limited to specific limited areas.

[0017] In this embodiment, the in-vehicle system 10 is configured to achieve driving automation levels 0 to 3 in the host vehicle. Specifically, the in-vehicle system 10 is configured to execute ACC and LKA, which correspond to Level 1. ACC is adaptive cruise control, i.e., inter-vehicle distance control. LKA is an abbreviation for Lane Keeping Assistance, which is a lane-keeping assist control. The in-vehicle system 10 is also configured to execute "hands-off driving" and "advanced safe driving assistance," which correspond to Level 2. "Hands-off driving" refers to the automated driving system automatically performing starting, steering, acceleration / deceleration, and stopping control, provided that the driver appropriately responds to intervention requests from the automated driving system. "Advanced safe driving assistance" refers to the automated driving system operating in parallel with the driver's driving of the host vehicle, and performing driving assistance operations in a timely manner in situations where there is a risk of a collision, etc., assuming that the driver drives the host vehicle. Hereinafter, in this specification, level 3 automated driving will simply be referred to as "automated driving" unless otherwise specified. Furthermore, "hands-off driving" may also be simply referred to as "Level 2." Furthermore, "Advanced Safety Driving Assistance" is called "Level 2 [G Mode]."

[0018] (Overall system configuration) 2, the in-vehicle system 10 is an in-vehicle network including an in-vehicle communication line 10A and a plurality of nodes interconnected via the in-vehicle communication line 10A, and is configured to be able to perform various vehicle controls and various associated display operations while the vehicle is being driven. The in-vehicle system 10 is configured to comply with a predetermined communication standard such as CAN (international registered trademark: international registration number 1048262A). CAN (international registered trademark) is an abbreviation for Controller Area Network.

[0019] The in-vehicle system 10 includes a vehicle state sensor 11, an external environment state sensor 12, a surroundings monitoring sensor 13, a locator 14, a DCM 15, a navigation device 16, a driver state detection unit 17, a driving control device 18, and an HMI device 20. DCM stands for Data Communication Module. The vehicle state sensor 11 to the HMI device 20 are connected to an in-vehicle communication line 10A.

[0020] The HMI device 20 is configured to display images visible to at least the vehicle occupants, including the driver, and to output audio audible to the vehicle occupants. Specifically, the HMI device 20 is configured to provide various information and / or entertainment to the vehicle occupants using image and / or audio input / output devices including a meter panel 21, a HUD device 22, a CID device 23, and a terminal device 24. CID stands for Center Information Display. HUD stands for Head-Up Display. The terminal device 24 is a portable or wearable electronic device brought into the vehicle by the vehicle occupants, including the driver, such as a mobile phone, a tablet device, a laptop computer, a portable game console, or a smart watch.

[0021] The HMI device 20 includes an HMI control device 25 configured to control the output of images and / or sounds on the meter panel 21 or the like. That is, the HMI control device 25 is configured to control the operation of the HMI device 20 constituting an in-vehicle infotainment system. The meter panel 21, the HUD device 22, and the CID device 23 are connected to the HMI control device 25 so as to be able to communicate information with each other via a sub-communication line different from the in-vehicle communication line 10A. When the terminal device 24 is brought into the vehicle, it is connected to the HMI control device 25 so as to be able to communicate information with each other via short-range wireless communication such as Bluetooth (registered trademark) or TransferJet (registered trademark). The HMI control device 25 is provided as a node connected to the in-vehicle communication line 10A. The configurations of the HMI device 20 and the HMI control device 25 will be described in detail later.

[0022] (various sensors) The vehicle state sensor 11 is configured to generate outputs corresponding to various quantities related to the driving state of the vehicle. The "various quantities related to the driving state" include, for example, accelerator position, braking amount, shift position, steering angle, and other quantities related to the driving operation state by the driver or the driving automation system. The "various quantities related to the driving state" also include, for example, vehicle speed, angular velocity, longitudinal acceleration, lateral acceleration, and other physical quantities related to the behavior of the vehicle. For the sake of simplicity of illustration and explanation, the vehicle state sensor 11 is a collective term for well-known sensors necessary for vehicle driving control, such as an accelerator position sensor, steering angle sensor, wheel speed sensor, angular velocity sensor, and acceleration sensor. The vehicle state sensor 11 is configured to provide detection outputs to various components, such as the driving control device 18, via the in-vehicle communication line 10A.

[0023] The external environment sensor 12 is configured to generate outputs corresponding to various quantities related to the natural environment within the driving environment of the vehicle. The "various quantities related to the natural environment" include physical quantities such as outside temperature, rainfall, and illuminance. For the sake of simplicity of illustration and explanation, the external environment sensor 12 is a general term for well-known sensors such as an outside temperature sensor, a raindrop sensor, and an illuminance sensor. The external environment sensor 12 is configured to provide detection outputs to various components, such as the driving control device 18, via the in-vehicle communication line 10A.

[0024] The perimeter monitoring sensor 13 is configured to detect mainly the driving environment of the vehicle other than those detectable by the external state sensor 12. Specifically, the perimeter monitoring sensor 13 is configured to be able to detect moving and stationary objects within a predetermined detection range around the vehicle. "Moving objects" include pedestrians, cyclists, animals, and other vehicles in operation. "Stationary objects" include fallen objects on the road, guardrails, curbs, parked vehicles, road signs, and road markings, as well as roadside structures (e.g., walls, buildings, etc.). The perimeter monitoring sensor 13 may also be referred to as an "ADAS sensor." ADAS stands for Advanced Driver-Assistance Systems.

[0025] In this embodiment, the perimeter monitoring sensor 13 has a front camera 131 and a radar sensor 132 as components for detecting moving and stationary objects. The front camera 131 is provided to capture images in front of and on the sides of the vehicle. The front camera 131 is a digital camera device and includes an image sensor such as a CCD or CMOS. CCD stands for Charge Coupled Device. CMOS stands for Complementary MOS.

[0026] The radar sensor 132 is a millimeter wave radar sensor, a submillimeter wave radar sensor, or a laser radar sensor configured to transmit and receive radar waves, and is mounted on the front of the vehicle body. The radar sensor 132 is configured to output signals corresponding to the position and relative speed of a reflection point. A "reflection point" is a point on the surface of an object around the vehicle that is estimated to have reflected radar waves. The "relative speed" is the speed of the reflection point, i.e., the object that reflected the radar waves, relative to the vehicle.

[0027] (locator) Locator 14 is configured to acquire highly accurate position information of the vehicle by so-called composite positioning. Specifically, locator 14 has a GNSS receiver 141, an inertial acquisition unit 142, a high-accuracy map DB 143, and a locator ECU 144. GNSS stands for Global Navigation Satellite System. DB stands for database. ECU stands for Electronic Control Unit. "High-accuracy position information" is, for example, position information having a position accuracy that can be used for driving automation levels of level 2 or higher, specifically, with an error of less than 10 cm.

[0028] The GNSS receiver 141 is configured to receive positioning signals transmitted from a plurality of positioning satellites, i.e., artificial satellites. In this embodiment, the GNSS receiver 141 is configured to be able to receive positioning signals from positioning satellites in at least one of satellite positioning systems such as GPS, QZSS, GLONASS, Galileo, IRNSS, and the Beidou satellite navigation system. GPS is an abbreviation for Global Positioning System. QZSS is an abbreviation for Quasi-Zenith Satellite System. GLONASS is an abbreviation for Global Navigation Satellite System. IRNSS is an abbreviation for Indian Regional Navigation Satellite System.

[0029] The inertia acquisition unit 142 is configured to acquire the acceleration and angular velocity acting on the host vehicle. In this embodiment, the inertia acquisition unit 142 is provided as a three-axis gyro sensor and a three-axis acceleration sensor built into a box-shaped housing of the locator 14.

[0030] The high-precision map DB 143 is mainly composed of non-volatile rewritable memory so that it can rewritably store high-precision map information and retain the stored contents even when the power is turned off. Examples of non-volatile rewritable memory include a hard disk, EEPROM, flash ROM, etc. EEPROM stands for Electronically Erasable and Programmable ROM. ROM stands for Read Only Memory. High-precision map information may also be referred to as high-precision map data. High-precision map information contains map information with higher accuracy than map information used in conventional car navigation systems, which can handle position errors of approximately several meters. Specifically, the high-precision map DB 143 stores information usable for driving automation levels 2 and above, such as three-dimensional road shape information, lane number information, and traffic regulation information, in accordance with predetermined standards such as the ADASIS standard. ADASIS stands for Advanced Driver Assistance Systems Interface Specification.

[0031] The locator ECU 144 is configured as a so-called in-vehicle microcomputer equipped with a CPU, ROM, RAM, input / output interface, etc. (not shown). CPU stands for Central Processing Unit. RAM stands for Random Access Memory. The locator ECU 144 is configured to sequentially determine the position, direction, etc. of the vehicle based on the positioning signal received by the GNSS receiver 141, the acceleration and angular velocity acquired by the inertia acquisition unit 142, the vehicle speed acquired from the vehicle state sensor 11, etc. The locator 14 is configured to be able to provide the results of the determination of the position, direction, etc. by the locator ECU 144 to each unit, such as the navigation device 16, the driving control device 18, and the HMI control device 25, via the in-vehicle communication line 10A.

[0032] (DCM) The DCM 15 is an in-vehicle communication module that is capable of communicating with base stations around the vehicle via wireless communication compliant with communication standards such as LTE or 5G. LTE stands for Long Term Evolution, and 5G stands for 5th Generation.

[0033] Specifically, for example, the DCM 15 is configured to acquire the latest high-precision map information from a probe server on the cloud. The DCM 15 also stores the acquired latest high-precision map information in the high-precision map DB 143 in cooperation with the locator ECU 144. The DCM 15 is also configured to acquire traffic information such as congestion information from the probe server and / or a predetermined database. The "congestion information" includes the location and length of a congestion section. Specifically, the congestion information includes the location of the beginning of the congestion, the location of the end of the congestion, an estimated congestion distance, an estimated congestion time, etc. The traffic information is also referred to as "road traffic information."

[0034] (Navigation device) Navigation device 16 is configured to acquire a planned driving route from the current position of the vehicle to a predetermined destination. In this embodiment, navigation device 16 is configured to calculate a planned driving route based on a destination set by the driver of the vehicle, high-precision map information acquired from locator 14, and vehicle position information and direction information acquired from locator 14. Navigation device 16 is also configured to provide various information, including route information resulting from the calculation, to various components, such as driving control device 18 and HMI control device 25, via in-vehicle communication line 10A. That is, navigation device 16 displays a navigation screen on HMI device 20 for displaying a map, a route, and the like.

[0035] (Driver status detection unit) The driver state detection unit 17 is configured to detect the driver state. The "driver state" refers to the state of the driver sitting in the driver's seat 2 of the vehicle, and includes at least one of the following: line of sight, posture, behavior, psychological state, etc. The driver state detection unit 17 is also configured to be able to provide the driver state detection results to each component, such as the driving control device 18 and the HMI control device 25, via the in-vehicle communication line 10A.

[0036] In this embodiment, the driver status detection unit 17 includes a gaze detection unit 171, a posture detection unit 172, and an operation status detection unit 173. The gaze detection unit 171 is configured to detect the direction of the driver's face and / or the direction of the driver's gaze by image recognition based on an image captured by an in-vehicle camera equipped with an image sensor such as a CCD or CMOS. In other words, the gaze detection unit 171 has a configuration similar to that of a DSM device that issues a warning to the driver regarding inattentive driving, etc. DSM is an abbreviation for Driver Status Monitor.

[0037] The posture detection unit 172 is configured to detect the seating posture of the driver in the driver's seat 2 using the above-mentioned in-vehicle camera and / or a physical quantity sensor such as a seating pressure sensor provided inside the driver's seat 2. The operation state detection unit 173 is configured to detect the state of the driver's feet resting on the accelerator pedal 3, the brake pedal 4, and the footrest 5, as well as the operation states of the accelerator pedal 3 and the brake pedal 4. The operation state detection unit 173 is also configured to detect the state of gripping and operation of the steering wheel 8 by the driver.

[0038] (Operation control device) The driving control device 18 is configured as an "autonomous driving ECU" or a "driving assistance ECU." That is, the driving control device 18 is configured to control the driving of the host vehicle based on signals and information acquired from the vehicle state sensor 11, the external state sensor 12, the surrounding monitoring sensor 13, the locator 14, etc. Specifically, the driving control device 18 is configured to execute predetermined driving control operations. In this embodiment, the "predetermined driving control operations" include vehicle control operations, i.e., dynamic driving task execution operations, corresponding to levels 1 to 3. In this embodiment, the driving control device 18 is configured to be able to set the driving automation level of the host vehicle to any one of level 0, level 1 [ACC], level 1 [LKA], level 2, level 2 [G mode], and level 3.

[0039] The driving control device 18 has a configuration as a so-called in-vehicle microcomputer, including a CPU, ROM, non-volatile rewritable memory, RAM, input / output interface, etc. (not shown). Specifically, the driving control device 18 has the following functional configurations or functional units realized on the in-vehicle microcomputer. That is, the driving control device 18 has a driving condition acquisition unit 181, a driver state acquisition unit 182, an operation state determination unit 183, an automation level determination unit 184, a vehicle control unit 185, and a display command transmission unit 186.

[0040] The driving condition acquisition unit 181 is configured to acquire at least the driving condition of the vehicle. The "driving condition" includes the driving condition and the driving environment detected or acquired by the vehicle condition sensor 11, the external environment condition sensor 12, the periphery monitoring sensor 13, etc. The driving condition acquisition unit 181 is also configured to acquire high-precision map information of the current position of the vehicle and its surroundings, and traffic information of the road on which the vehicle is currently traveling. That is, the driving condition acquisition unit 181 acquires information necessary for vehicle control corresponding to levels 1 to 3 from the vehicle condition sensor 11, the external environment condition sensor 12, the periphery monitoring sensor 13, the locator 14, the DCM 15, etc.

[0041] The driver state acquisition unit 182 is configured to acquire the driver state. Specifically, the driver state acquisition unit 182 acquires, i.e., receives, from the driver state detection unit 17, the detection result of the driver state by the driver state detection unit 17.

[0042] The operation state determination unit 183 is provided to determine the driving operation state by the driver. Specifically, the operation state determination unit 183 acquires, i.e., receives, the detection result by the operation state detection unit 173 from the driver state detection unit 17. The operation state determination unit 183 also acquires, i.e., receives, from the HMI device 20, the driver's input operation state in the HMI device 20. Then, the operation state determination unit 183 determines the driving operation state by the driver based on these acquired results.

[0043] The automation level determination unit 184 is configured to determine the driving automation level based on the driving conditions acquired by the driving condition acquisition unit 181, etc. The driving control device 18 is configured to be able to provide the results of the driving automation level determination by the automation level determination unit 184 to each component, such as the HMI control device 25, via the in-vehicle communication line 10A. Details of the determination of the driving automation level by the automation level determination unit 184 will be described in detail in the operation overview and operation example explanations below.

[0044] The vehicle control unit 185 is configured to execute vehicle motion control subtasks according to the driving automation level. That is, the vehicle control unit 185 executes longitudinal and / or lateral motion control of the host vehicle based on the driving automation level determined by the automation level determination unit 184.

[0045] The display command transmission unit 186 is configured to cause the HMI device 20 to execute level-related displays related to the driving automation level by transmitting display command information to the HMI control device 25 that controls the HMI device 20. The level-related displays include execution-related displays related to the execution status of the driving automation level and transition-related displays related to transitions between driving automation levels. The execution-related displays are primarily displays that indicate the driving automation level currently being executed, such as "Autonomous driving in progress." The transition-related displays are displays related to the termination of the currently executing driving automation level and / or transition to another driving automation level when a transition to another driving automation level occurs due to the satisfaction of the termination condition for the currently executing driving automation level or the non-satisfaction of the continuation condition. The transition-related displays are, for example, "Autonomous driving ends 3 km ahead," "Preparing for driver changeover," "Hands-off driving is possible," etc. Further details and other specific examples of the execution-related displays and transition-related displays will be described in detail in the operation overview and operation example descriptions below.

[0046] (HMI device) The HMI device 20 is configured to present various pieces of information about the vehicle to the driver at least visually and to accept input operations by the driver corresponding to the presented information. In this embodiment, the HMI device 20 mounted on the vehicle capable of autonomous driving is configured to be able to present various pieces of information about autonomous driving and the like and to accept input operations by the driver. "Information presentation" includes, for example, various kinds of guidance, input operation instructions, input operation content notifications, warnings, etc.

[0047] As described above, the HMI device 20 includes the meter panel 21, the HUD device 22, and the CID device 23, which are provided on the dashboard 7. That is, in this embodiment, the HMI device 20 has a configuration as a so-called "dashboard HMI." The HMI device 20 also includes a speaker (not shown) for presenting information by voice.

[0048] The meter panel 21 has a meter 211, a meter display 212, and a meter switch 213. The meter 211 is configured to display the vehicle speed, engine RPM, coolant temperature, remaining fuel, etc. of the vehicle. The meter display 212 is an information display unit or information display area provided in the center of the meter panel 21 in the vehicle width direction, and is configured to be able to display various information such as the date and time, outside temperature, mileage, radio station, etc. In this embodiment, the meter display 212 is configured as a display device that is a liquid crystal display or an organic EL display with a substantially rectangular displayable area. EL stands for electroluminescence. The meter switch 213 is configured to be able to accept various operations related to the display state or display content of the meter 211 and / or the meter display 212, such as an operation to reset the trip meter.

[0049] The HUD device 22 is configured to display a display image including letters and / or symbols in front of the driver. That is, the HUD device 22 is configured to form a virtual display image in front of the driver using AR technology, thereby superimposing the display image on a foreground including the road surface ahead of the vehicle. "Superimposed display" refers to displaying related information (e.g., building name) of a superimposed object (e.g., a building) included in the foreground while associating the superimposed object with the related information by superimposing the superimposed object on the superimposed object or displaying the information near the superimposed object. Displaying a route, a direction of travel, traffic information, and the like for the road surface ahead also falls under the category of "superimposed display." Specifically, the HUD device 22 projects display image light constituting the display image onto a predetermined projection range PA on the front windshield 9, allowing the driver to visually recognize the light of the display image light reflected by the front windshield 9, thereby displaying the display image in AR.

[0050] The CID device 23 is provided in the approximate center of the dashboard 7 in the vehicle width direction. The CID device 23 is provided to be able to display a navigation display screen for displaying maps, routes, etc. by the navigation device 16. The CID device 23 is also provided to be able to display information and content different from the navigation display screen. Specifically, the CID device 23 is configured to be able to execute displays related to driving modes such as "Comfort," "Normal," "Sport," and "Circuit."

[0051] The CID device 23 is also configured to be able to execute a display related to a second task that the driver can use during autonomous driving. The CID device 23 is configured to be able to execute, for example, viewing video content as a second task. "Video content" includes, for example, movies, concert footage, music videos, television broadcasts, etc. The second task also includes operating the terminal device 24.

[0052] The CID device 23 has a CID display 231, an input device 232, and a CID switch 233. The CID display 231 is provided in an approximate center position in the vehicle width direction of the dashboard 7, i.e., in a position between the driver's seat 2 and the passenger seat, so as to be visible at least to the driver. The CID display 231 is configured as a display device that is a liquid crystal display or an organic EL display. When the second task is viewing video content, the CID display 231 is configured to display an image of the video content on the display device.

[0053] The input device 232 is a transparent touch panel that is placed on top of the CID display 231 to cover it. That is, the input device 232 is configured to allow the driver or the like to visually recognize the display on the CID display 231, while accepting input operations by the driver or the like corresponding to the display. The CID switch 233 has a plurality of manually operated switches arranged around the CID display 231 and the input device 232.

[0054] The HMI device 20 has a steering switch and the like in addition to the meter switch 213 and the CID switch 233. The steering switch is provided on the spoke portion of the steering wheel 8. The HMI device 20 is configured to be able to provide the results of the input operation received by the driver to each part such as the driving control device 18 via the in-vehicle communication line 10A.

[0055] (HMI control device) The HMI control device 25 has a configuration as an HCU that controls the operations of the meter panel 21, the CID device 23, the HUD device 22, and the like included in the HMI device 20. HCU is an abbreviation for HMI Control Unit.

[0056] The HMI control device 25 has a configuration as a so-called in-vehicle microcomputer, including a CPU, ROM, nonvolatile rewritable memory, RAM, input / output interface, etc. (not shown). The HMI control device 25 has the following functional configuration or functional units realized on the microcomputer. That is, the HMI control device 25 has a vehicle information acquisition unit 251, a driving environment acquisition unit 252, an automation level acquisition unit 253, a driver state acquisition unit 254, a display control unit 255, and an operation reception unit 256.

[0057] The vehicle information acquisition unit 251 is configured to acquire information related to the driving state of the vehicle. Specifically, the vehicle information acquisition unit 251 acquires, from the vehicle state sensor 11, various quantities related to the driving state of the vehicle, which are detected or acquired by the vehicle state sensor 11.

[0058] The driving environment acquisition unit 252 is configured to acquire information related to the driving environment of the vehicle. Specifically, the driving environment acquisition unit 252 acquires, from the external environment sensor 12, various quantities related to the natural environment around the vehicle, which are detected or acquired by the external environment sensor 12. The driving environment acquisition unit 252 also acquires, from the periphery monitoring sensor 13, the object detection results of the periphery monitoring sensor 13. Furthermore, the driving environment acquisition unit 252 acquires, from the locator 14 and the navigation device 16, the current position of the vehicle, the planned driving route, and traffic information including congestion information on the planned driving route.

[0059] The automation level acquisition unit 253 is configured to acquire the result of the driving automation level determination made by the driving control device 18. Specifically, the automation level acquisition unit 253 is configured to acquire the result of the driving automation level determination made by the automation level determination unit 184 from the driving control device 18.

[0060] The driver state acquisition unit 254 is provided to acquire the driver state. Specifically, the driver state acquisition unit 254 is configured to acquire the detection result of the driver state by the driver state detection unit 17 from the driver state detection unit 17.

[0061] The display control unit 255 is configured to control the image and / or audio output operation of the HMI device 20. That is, the display control unit 255 controls the image output and audio output of the meter panel 21, the HUD device 22, the CID device 23, etc., to present various information to vehicle occupants, including the driver. The "various information" includes driving state information, driving environment information, information related to driving automation levels, route information, traffic congestion information, various messages, etc. Furthermore, the display control unit 255 cooperates with the terminal device 24 to cause the terminal device 24 to present various information while the driver is performing a second task using the terminal device 24.

[0062] In this embodiment, the display control unit 255 is configured to control the image display operation in the HMI device 20 according to the driving automation level acquired by the automation level acquisition unit 253. In other words, the display control unit 255 is configured to display an image display according to the driving automation level determined by the automation level determination unit 184 and currently being executed or scheduled to be executed by the in-vehicle system 10 on an image display device such as the meter panel 21 included in the HMI device 20.

[0063] The operation reception unit 256 is provided to receive input operations by vehicle occupants, including the driver, on the HMI device 20. Specifically, the operation reception unit 256 monitors the state or results of reception of input operations by the meter switch 213, the input device 232, the CID switch 233, the terminal device 24, etc., which correspond to various information presented by the display control unit 255.

[0064] (Operation overview) The following describes the operation of the operation control device 18 and the HMI control device 25 according to this embodiment, as well as an overview of the control method and control program executed by these devices, along with the effects achieved by this embodiment.

[0065] In the driving control device 18, the driving condition acquisition unit 181 acquires various information including the driving condition of the vehicle. Specifically, the driving condition acquisition unit 181 acquires the driving condition and driving environment of the vehicle from the vehicle condition sensor 11, the external environment condition sensor 12, and the surroundings monitoring sensor 13. The driving condition acquisition unit 181 also acquires high-precision map information of the current position of the vehicle and its surroundings, a planned driving route, and traffic information on the planned driving route from the locator 14 and the navigation device 16.

[0066] The driver state acquisition unit 182 acquires the driver state. Specifically, the driver state acquisition unit 182 receives the detection result of the driver state by the driver state detection unit 17 from the driver state detection unit 17.

[0067] The operation state determination unit 183 determines the driving operation state of the driver. Specifically, the operation state determination unit 183 acquires the detection result of the operation state detection unit 173 from the driver state detection unit 17. The detection result of the operation state detection unit 173 includes the state of the driver's feet on the accelerator pedal 3, the brake pedal 4, and the footrest 5, the operation state of the accelerator pedal 3 and the brake pedal 4, the grip state and operation state of the steering wheel 8, etc. The operation state determination unit 183 also acquires the driver's input operation state in the HMI device 20 from the HMI device 20. Then, the operation state determination unit 183 determines the driving operation state of the driver based on the detection result of the operation state detection unit 173 and the driver's input operation state in the HMI device 20.

[0068] The automation level determination unit 184 determines the driving automation level to be implemented by the in-vehicle system 10. The driving automation level is determined based on at least the driving conditions acquired by the driving condition acquisition unit 181. In this embodiment, the automation level determination unit 184 determines the driving automation level based on the results acquired by the driving condition acquisition unit 181 and the driver state acquisition unit 182, and the determination result by the operation state determination unit 183.

[0069] Specifically, the automation level determination unit 184 determines the start conditions for a driving automation level corresponding to levels 1 to 3 based on various information such as the driving conditions acquired by the driving condition acquisition unit 181. When the start conditions for a predetermined driving automation level are met, the automation level determination unit 184 determines that the driving automation level is executable. Then, when the driver performs an approval operation, the automation level determination unit 184 decides to execute the driving automation level. The vehicle control unit 185 executes vehicle speed control, steering control, braking control, etc. according to the driving automation level determined to be executed by the automation level determination unit 184.

[0070] The display command transmission unit 186 causes the HMI device 20 to execute a level-related display related to the driving automation level by transmitting display command information to the HMI control device 25 that controls the HMI device 20. As a result, the HMI device 20 notifies the vehicle occupants, including the driver, of the driving automation level that is being implemented or will be implemented in the future by visual display and / or audio. Specifically, for example, the HMI device 20 displays the currently implemented driving automation level on the meter display 212 or the like. Furthermore, when autonomous driving becomes possible, the HMI device 20 displays on the meter display 212 or the like that autonomous driving is now possible and an approval operation instruction for accepting an approval operation to start autonomous driving.

[0071] During autonomous driving, the driver is not obligated to monitor the surroundings of the vehicle until a request to take over driving or a request to intervene in driving is made by the in-vehicle system 10. Furthermore, the driver is not required to perform steering control operations or acceleration / deceleration control operations in principle until a request to take over driving or a request to intervene in driving is made.

[0072] Therefore, during automated driving, the driver is not required to assume a manual driving position that allows for manual driving. "Driving position" includes both driving posture and driving awareness. "Driving posture" refers to the driver's riding position, including the seating position in the driver's seat 2, the position of the feet relative to the accelerator pedal 3, the grip and operation state of the steering wheel 8, and the direction of gaze. On the other hand, "driving awareness" refers to the driver's psychological state, including the driver's awareness of the driving state and driving environment of the vehicle. Note that gaze direction can also be included in "driving awareness." A level of driving awareness in which the driver's attention or monitoring state regarding the driving state and driving environment of the vehicle is high and which corresponds to a lower level of driving automation is hereinafter referred to as "high driving awareness." In contrast, when the driver's attention is away from driving, the driver's driving awareness is low.

[0073] While the in-vehicle system 10 is stably performing automated driving without a request to take over driving or intervehicle driving, the driver is not required to constantly grip the steering wheel 8. The driver is also not required to constantly maintain a driving posture that allows the accelerator pedal 3 and the brake pedal 4 to be operated at any time. Furthermore, until the in-vehicle system 10 requests a driver to take over driving or intervehicle driving, the driver may be unaware of the driving state and driving environment of the vehicle. Therefore, while the in-vehicle system 10 is stably performing automated driving, the driver can freely perform or utilize a second task.

[0074] 3 shows, as a typical example, an example of the display on CID display 231 and meter display 212 when viewing video content on CID device 23 is being performed as a second task during autonomous driving. Information display area DA1, a horizontally elongated strip at the top of CID display 231, displays an execution-related display saying "Autonomous driving in progress," which indicates the level of driving automation currently being performed. Screen area DA2, located below information display area DA1 and occupying most of the displayable area of ​​CID display 231, displays a video content screen, which is the second task screen.

[0075] An execution-related display such as "Autonomous driving in progress," which indicates the driving automation level currently being implemented, is displayed in level information display area DB1, a horizontally elongated strip-shaped area at the top of meter display 212. Driving information display area DB2, located below level information display area DB1 and occupying most of the displayable area of ​​meter display 212, displays various information such as the lane in which the vehicle is traveling during autonomous driving.

[0076] When the termination condition for a specific driving automation level currently being executed is met, or when the continuation condition for that driving automation level is no longer met, the automation level determination unit 184 decides to transition to a new driving automation level. That is, the automation level determination unit 184 decides to end the currently executed driving automation level and to select the next executable driving automation level.

[0077] The vehicle control unit 185 then performs vehicle speed control, steering control, braking control, etc. in accordance with the driving automation level transition mode determined by the automation level determination unit 184. In addition, the display command sending unit 186 sends display command information to the HMI control device 25, causing the HMI device 20 to execute transition-related displays related to the transition of the driving automation level.

[0078] (Example of operation) Figure 4 shows an example of a typical operation in which the driving automation level is shifted from autonomous driving to Level 2, i.e., hands-off driving, as the autonomous driving section ends. In the figure, "LV" is an abbreviation for "level." Times T11 to T15 indicate the passage of time.

[0079] As mentioned above, during automated driving, the driver is free to perform secondary tasks without the obligation to monitor the surroundings, etc. Therefore, there is a high possibility that the driver's attention will be diverted from driving during automated driving.

[0080] On the other hand, during hands-off driving, the driver is obligated to monitor the surroundings, etc. However, the driver is not required to perform steering control operations or acceleration / deceleration control operations, as is the case during automated driving. In other words, the driver's behavior regarding driving operations is similar in automated driving and hands-off driving. For this reason, when switching from automated driving to hands-off driving, the driver may misunderstand the level of driving automation currently being implemented.

[0081] In this regard, it is expected that the driver's driving awareness can be more reliably improved as the driving automation level decreases if the driver not only receives information and recognizes and understands it, but also engages in physical movement. Therefore, in this embodiment, when terminating automated driving and transitioning to hands-off driving, the automation level determination unit 184 first transitions to a driving automation level lower than hands-off driving and then transitions to hands-off driving.

[0082] In this typical example, as shown in FIG. 4, when terminating automated driving and transitioning to hands-off driving, the automation level determination unit 184 first transitions to level 0, i.e., manual driving, and then transitions to hands-off driving. This increases the driver's awareness of driving by having the driver temporarily perform manual driving when terminating automated driving. Then, by transitioning to hands-off driving after the driver's awareness of driving has sufficiently increased, the driver can reliably recognize the driving automation level being implemented. This makes it possible for the driver to adopt an appropriate driving posture that is adapted to the transition in driving automation level.

[0083] Specific examples of driving control and display control in this typical example will be described below with reference to FIGS. 4 to 7. First, at time T11, the in-vehicle system 10 executes a driving change request. Time T11 is set a predetermined time before the scheduled time when the vehicle will arrive at the end point of the autonomous driving possible section. In response to the driving change request, the display command transmitter 186 transmits display command information to the HMI control device 25 to cause the CID display 231, which is a display device that displays the second task screen, to display the visual guidance display GA shown in FIG. 5.

[0084] The line-of-sight guidance display GA is a display that guides the driver's line of sight from the second task screen to the front display device. The "front display device" is a display device that is arranged in front of the driver, i.e., on the side of the vehicle's travel destination relative to the driver, and in this embodiment, is the meter panel 21 and the HUD device 22. That is, the front display device displays information or images in front of the driver, i.e., on the side of the vehicle's travel destination relative to the driver.

[0085] Specifically, the HMI control device 25 displays the gaze guidance display GA, "→→→Watch ahead→→→," in the information display area DA1 located above the screen area DA2, which is the second task screen display area, on the CID display 231. At this time, by displaying the gaze guidance display GA as an animation, it is possible to further enhance the gaze guidance effect. For example, in the animation display, it is possible to move one of the multiple "→"s that is brighter than the others in the gaze guidance direction, i.e., to the right toward the front display device. Alternatively, for example, it is possible to move the character string "Watch ahead" in the gaze guidance direction.

[0086] Furthermore, in response to the driver change request, the display command transmitter 186 transmits display command information to the HMI control device 25 to cause the front display device to display the transition-related display and the operation instruction display shown in FIG. 5. As a result, the meter panel 21 displays a transition-related display saying "Automated driving ends 3 km ahead" in the top level information display area DB1 of the meter display 212. The meter panel 21 also displays a first operation instruction display GB1, a second operation instruction display GB2, a third operation instruction display GB3, and a fourth operation instruction display GB4 in the driving information display area DB2 of the meter display 212. The first operation instruction display GB1, the second operation instruction display GB2, the third operation instruction display GB3, and the fourth operation instruction display GB4 are displays that instruct the driver to take a predetermined driving posture.

[0087] Specifically, in the example shown in FIG. 5, the meter display 212 displays a first operational instruction display GB1 that reads, "Prepare for (manual) driver changeover." The meter display 212 also displays a second operational instruction display GB2 that reads, "Watch out ahead!" The meter display 212 also displays a third operational instruction display GB3 that reads, "Hold on to the steering wheel!" The meter display 212 also displays a fourth operational instruction display GB4 that reads, "Put your feet on the pedals!"

[0088] In the specific example shown in FIG. 5, the first action instruction display GB1, "Prepare for (manual) driving changeover," corresponds to a transition-related display. The first action instruction display GB1 also corresponds to a main action instruction display instructing a main action. The "main action" is an action necessary to transfer authority to the driver as the level of driving automation decreases, and includes an action to adjust driving posture. The second action instruction display GB2, "Watch out ahead!" corresponds to a secondary action instruction display instructing a secondary action. The "secondary action" is an action that promotes or assists the main action, and includes an action to adjust driving awareness. The second action instruction display GB2 also corresponds to a main action instruction display. The third action instruction display GB3, "Hold the steering wheel!" corresponds to a main action instruction display. The fourth action instruction display GB4, "Put your feet on the pedals!" corresponds to a main action instruction display. That is, in the driving change request, the display command sending unit 186 transmits display command information to the HMI control device 25 to cause the HMI device 20 to display the main action instruction display and secondary action instruction display shown in FIG. 5.

[0089] The HUD device 22 displays a transition-related display saying "Prepare for driving changeover (manual)." The display by the HUD device 22 has a narrow visible range and becomes easily visible as the driver approaches a manual driving posture. Therefore, the driver visually recognizes the line-of-sight guidance display GA on the CID display 231, which guides the driver's line of sight to the transition-related display by the HUD device 22, thereby naturally bringing the driver's posture closer to a manual driving posture. Then, the driver visually recognizes the main action instruction display and the secondary action instruction display, which effectively raises the driver's awareness of adopting a manual driving posture. In this way, guiding the driver's line of sight to the transition-related display by the HUD device 22 as a front display device effectively encourages the driver to quickly adopt a manual driving posture.

[0090] Thus, when the display shown in FIG. 5 is executed on the HMI device 20 at time T11, the driver switching operation, i.e., the main operation and secondary operation, begins at time T12. The time lag from the display start time T11 to the driver switching operation start time T12 corresponds to the reaction time of a typical driver. After the display start time T11, the automation level determination unit 184 monitors whether the driver switching operation has been completed based on the driver state acquisition result by the driver state acquisition unit 182 and the driving operation state determination result by the operation state determination unit 183. Then, when the driver switching operation is completed at time T13, manual driving becomes possible. Therefore, the automation level determination unit 184 transitions the driving automation level from automated driving to manual driving at time T13, before time T14, when the vehicle reaches the end of the automated driving section. During manual driving, a navigation screen is displayed in the screen area DA2 on the CID display 231, as shown in FIG. 6. On the other hand, in the information display area DA1, the visual guidance display GA continues to be displayed to prevent the driver from gazing at the navigation screen for a long period of time.

[0091] After manual driving begins at time T13, when the driver's driving operation state stabilizes, a smooth transition from manual driving to hands-off driving is possible. Therefore, the operation state determination unit 183 determines the driving operation state by the driver. Then, the automation level determination unit 184 permits the execution of hands-off driving on the condition that the operation state determination unit 183 determines that the driving operation state corresponding to manual driving has stabilized.

[0092] Specifically, when the driving operation state corresponding to manual driving is stabilized, the display command sending unit 186 sends display command information to the HMI control device 25 to cause the HMI device 20 to display a transition-related display and an approval request display. The approval request display is a display that prompts or requests the driver to perform an input operation to approve the execution of hands-off driving.

[0093] 6, the meter display 212 displays a transition-related display saying "Hands-off driving is possible" in the level information display area DB1. The meter display 212 also displays a first input request display GC1 showing an approval button with the character string "Approve" and a second input request display GC2 showing a disapproval button with the character string "Do not approve" in the driving information display area DB2.

[0094] When the driver performs an approval operation at time T15, the automation level determination unit 184 determines to perform hands-off driving, and the in-vehicle system 10 then starts hands-off driving.

[0095] As shown in Fig. 7, during hands-off driving, the CID display 231 displays an execution-related message "Hands-off driving in progress" in the information display area DA1, and displays a navigation screen in the screen area DA2. The meter display 212 also displays an execution-related message "Hands-off driving in progress" in the level information display area DB1, and displays various information such as route information in the driving information display area DB2. Fig. 7 shows an example in which route information indicating a right turn at an intersection 800 meters ahead, radio stations, outside temperature, and the current date and time are displayed in the driving information display area DB2.

[0096] Figure 4 shows the transition of driving automation level when the driver complies with the driving takeover request. In contrast, if the driver does not comply with the driving takeover request and does not perform the primary action, the driving automation level will transition as shown in Figure 8.

[0097] That is, if the driving changeover operation that should have been performed between times T12 and T13 is not performed, the automation level determination unit 184 continues autonomous driving until time T14, when the end point of the autonomous driving possible section is reached. During this time, the automation level determination unit 184 waits for the driving changeover operation to be performed. If the driving changeover operation is performed by time T14, the automation level determination unit 184 switches the driving automation level from autonomous driving to manual driving. On the other hand, if the driving changeover operation is not performed by time T14, the automation level determination unit 184 transitions the driving automation level from autonomous driving to hands-off driving at time T14.

[0098] Even if the driving changeover operation that should have been performed between times T12 and T13 is not performed, the display command transmission unit 186 continues to transmit display command information to the HMI control device 25 to cause the operation instruction display shown in FIG. 5 to be displayed on the front display device. As a result, the driving changeover request to achieve a driving posture that allows hands-off driving to be continued satisfactorily continues for a predetermined time even after time T14 has passed. If a driving posture that allows hands-off driving to be continued satisfactorily has not been achieved even after a predetermined time has passed since time T14, the in-vehicle system 10 executes a predetermined emergency process at time T16. The emergency process is, for example, a rest suggestion or a so-called MRM for safely stopping the vehicle. MRM stands for Minimal Risk Maneuver.

[0099] (First embodiment: summary) As such, in this embodiment, when ending a high automation level and transitioning to a medium automation level, the automation level determination unit 184 first transitions from the high automation level to a low automation level, and then transitions to the medium automation level. A high automation level is a driving automation level included in automated driving levels 3 to 5. A medium automation level is a driving automation level lower than the high automation level. A low automation level is a driving automation level even lower than the medium automation level. This makes it possible to effectively raise the driver's awareness regarding the transition in driving automation level due to the end of the high automation level included in automated driving levels 3 to 5.

[0100] If the in-vehicle system 10 is configured to support level 4 or higher, the "high automation level" can be level 4 or 5. In this case, the "low automation level" can be level 0 or 1.

[0101] In this embodiment, the automation level determination unit 184 allows execution of the medium automation level on the condition that at least the operation state determination unit 183 determines that the driving operation state corresponding to the low automation level has stabilized. Specifically, when the driving operation state corresponding to the low automation level has stabilized, the display command sending unit 186 sends display command information to the HMI control device 25 to display an approval request display on the HMI device 20. The approval request display is a display that prompts the driver to input an operation to approve the execution of the medium automation level. The automation level determination unit 184 executes the medium automation level when specified medium automation level execution conditions are met, specified driving operation states have stabilized, and driver approval has been given. This enables the transition of driving automation levels to be carried out more stably.

[0102] Second Embodiment The second embodiment will be described below with reference to FIGS. 9 and 10. In the following description of the second embodiment, differences from the first embodiment will be mainly described. In addition, identical or equivalent parts in the first and second embodiments are denoted by the same reference numerals. Therefore, in the following description of the second embodiment, the description of the first embodiment can be appropriately applied to components having the same reference numerals as those in the first embodiment, unless there is a technical contradiction or a special additional explanation. The same applies to the third embodiment described below.

[0103] The configuration of the in-vehicle system 10 according to this embodiment is the same as that of the first embodiment. That is, the vehicle 1 and the in-vehicle system 10 according to this embodiment have the configurations shown in Figures 1 and 2. However, the operation mode and the corresponding functional configuration of this embodiment are slightly different from those of the first embodiment.

[0104] In this embodiment, the automation level determination unit 184 sets the driving automation level at a low automation level depending on the cause of termination of the high automation level. In other words, the driving automation level at a low automation level, which is once executed before transitioning from a high automation level to a medium automation level, is variable depending on the cause of termination of the high automation level. This makes it possible for the driver to adopt a more appropriate driving posture depending on the cause of termination of the high automation level.

[0105] Specifically, for example, assume the first case where the driving automation level is shifted from automated driving to hands-off driving when the automated driving section ends. In this first case, the driver's driving awareness does not need to be raised to a level where they can immediately respond to manual driving; it is sufficient if they are raised to a level where they can smoothly shift to hands-off driving.

[0106] Therefore, in this case, as shown in Fig. 9, the automation level determination unit 184 transitions from automated driving to level 1, for example, ACC, and then transitions to hands-off driving. Note that times T11 to T15 shown in Fig. 9 are the same as those in Fig. 4.

[0107] In contrast, consider a second case in which autonomous driving is interrupted due to the detection of an obstacle on the road while driving in an area where autonomous driving is possible. In this case, it is preferable that the driver's awareness of driving is higher than in the first case. Therefore, in this case, as shown in Figure 10, the automation level determination unit 184 first transitions from autonomous driving to level 0, i.e., manual driving, and then transitions to hands-off driving.

[0108] 10, first, at time T21, the in-vehicle system 10 executes a driver change request. Time T21 can be considered to be roughly the same as the time when the automation level determination unit 184 determines that automated driving should be suspended due to the detection of an obstacle on the road, etc.

[0109] When a driver changeover request is executed by the HMI device 20 at time T21, the driver changeover operation, i.e., the main operation and secondary operation, is initiated at time T22, a predetermined time lag after the driver changeover request. Then, when the driver changeover operation is completed at time T23, the driving automation level transitions from automatic driving to manual driving.

[0110] T24 indicates the time when the conditions for interrupting automated driving are resolved, such as when a detected road obstacle is passed without incident. After time T24 has passed, if the driver's driving operation state stabilizes, a smooth transition from manual driving to hands-off driving becomes possible.

[0111] Therefore, when the driving operation state corresponding to manual driving is stabilized, the display command sending unit 186 sends display command information to the HMI control device 25 to cause the HMI device 20 to display a transition-related display regarding the transition to hands-off driving and an approval request display. The automation level determination unit 184 transitions the driving automation level from manual driving to hands-off driving at time T25 when the driver's approval operation is received.

[0112] (Third embodiment) The third embodiment will be described below with reference to Figures 11 to 14. The configuration of the in-vehicle system 10 according to this embodiment is similar to that of the first and second embodiments. This embodiment differs slightly from the first and second embodiments in terms of the operating mode and the corresponding functional configuration.

[0113] Figure 11 shows an example in which, as the autonomous driving section ends, the driving automation level is first shifted from autonomous driving to Level 1 (i.e., ACC) and then shifted to hands-off driving. Times T31 to T35 in Figure 11 are the same as times T11 to T15 in Figure 4, respectively. In this example, it is assumed that the first operational instruction display GB1 to the fourth operational instruction display GB4 shown in Figure 12 are displayed when a driver change request is made at time T31.

[0114] In this case, the first action instruction display GB1 "Prepare for driver changeover (ACC)" corresponds to the main action instruction display instructing the main action required for driver changeover. On the other hand, the second action instruction display GB2 "Watch out ahead!" corresponds to the main action instruction display and also effectively functions as a secondary action instruction display instructing the secondary action mainly to increase driving awareness.

[0115] In ACC, which is Level 1, the transition from autonomous driving, steering operation by the driver is required, but accelerator and brake operation by the driver are not required. Therefore, the third action instruction display GB3, "Hold the steering wheel!", corresponds to a primary action instruction display that instructs the primary action. On the other hand, the fourth action instruction display GB4, "Put your feet on the pedals!", corresponds to a secondary action instruction display that instructs the secondary action.

[0116] Ideally, both the primary and secondary actions would be executed accurately between the start of the driver changeover operation at time T32 and the time the vehicle reaches the end point of the autonomous driving section at time T34. However, while the primary action is essential for the vehicle to continue driving, the secondary action is an auxiliary action to increase driving awareness. Therefore, as long as the primary action is executed correctly, it is acceptable to execute a transition in driving automation level even if the secondary action is not executed or is executed with some deficiency. Conversely, if emergency processing such as MRM is executed every time an emergency processing is executed due to non-execution or improper execution of a secondary action even when the primary action is executed correctly, the user of the vehicle may experience inconvenience.

[0117] Therefore, the operation state detection unit 173 detects the execution state or execution status of the primary action and secondary action by the driver. If the primary action is not executed, the vehicle control unit 185 executes a predetermined emergency process such as MRM. Figure 13 shows the transition of the driving automation level in this case.

[0118] Specifically, if at least the primary operation of the driving changeover operations that should have been performed between times T32 and T33 is not performed, the automation level determination unit 184 continues autonomous driving until time T34, when the end point of the autonomous driving possible section is reached. During this time, the automation level determination unit 184 waits for the primary operation to be performed. If the primary operation is performed by time T34, the automation level determination unit 184 switches the driving automation level from autonomous driving to ACC. On the other hand, if the primary operation is not performed by time T34, the automation level determination unit 184 transitions the driving automation level from autonomous driving to manual driving at time T34. Note that even after transitioning to manual driving, collision avoidance braking, collision damage mitigation braking, etc. may be performed at any time by the vehicle control unit 185.

[0119] Even if the driver change operation that should have been performed between times T32 and T33 is not performed, the display command transmission unit 186 continues to transmit display command information to the HMI control device 25 to cause the front display device to display the operation instruction display shown in Fig. 12. As a result, the driver change request continues for a predetermined time even after time T34 has passed. Then, if the driver change has not been realized even after a predetermined time has passed since time T34, at time T36 the in-vehicle system 10 executes a predetermined emergency process such as MRM.

[0120] On the other hand, there may be cases where the secondary action is not executed even though the primary action has been executed. That is, for example, there may be a case where the driver is sitting in the driver's seat 2 with his legs crossed and his right shoulder raised from the backrest while looking ahead and gripping the steering wheel 8 with his right hand in preparation for switching to ACC. In this case, although the driver's driving posture is compatible with ACC, it is preferable for the driver to correct his sitting posture in order to further increase his driving awareness.

[0121] Therefore, in this case, the vehicle control unit 185 does not execute the above-mentioned safety stop control. Specifically, as shown in Figure 11, the automation level determination unit 184 transitions the driving automation level from autonomous driving to Level 1 (i.e., ACC) at time T33. As a result, as shown in Figure 14, the CID display 231 displays an execution-related display saying "ACC driving assistance in progress" in the information display area DA1, and displays a navigation screen in the screen area DA2.

[0122] Meanwhile, the display command sending unit 186 sends display command information to the HMI control device 25 to display a warning display on the HMI device 20. As a result, the HUD device 22 displays a warning display saying "Put your feet on the pedals!" as shown in Fig. 14. Also, the meter display 212 continues to display the fourth action instruction display GB4 as a warning display saying "Put your feet on the pedals!"

[0123] (Fourth embodiment) The fourth embodiment will be described below with reference to Figures 15 and 16. The configuration of the in-vehicle system 10 according to this embodiment is the same as that of the above-described embodiments. That is, the vehicle 1 and the in-vehicle system 10 according to this embodiment have the configurations shown in Figures 1 and 2. This embodiment differs slightly from the above-described embodiments in terms of the operation mode and the corresponding functional configuration.

[0124] In each of the above embodiments, the transition of the driving automation level was adjusted so as to induce physical movement in the driver and effectively increase driving awareness when the driving automation level is reduced following the end of the high automation level. However, the same effect can be achieved by "apparently" transitioning the driving automation level by presenting information using the HMI device 20, rather than "actually" transitioning the driving automation level to a low automation level using the driving control device 18 as in each of the above embodiments.

[0125] Typically, when the driving automation level is changed, the driving control device 18 determines the actual driving automation level and performs the transition operation, and the HMI device 20 presents various related information. Therefore, as long as the information presented using the HMI device 20 can induce physical movements in the driver that increase driving awareness, it is sufficient that the actual change in driving automation level by the driving control device 18 is not performed. Therefore, in this embodiment, the change in driving automation level in the first embodiment described above is performed "virtually" by presenting information such as a display on the HMI device 20.

[0126] That is, in this embodiment, when the high automation level is terminated, the display control unit 255 causes the HMI device 20 to display an operational instruction display that causes the driver to assume a low-level compatible state. The low-level compatible state is a driver state that can execute a low automation level. In this embodiment, the low automation level is a driving automation level in which the in-vehicle system 10 including the driving control device 18 does not execute at least one of lateral movement control by steering and longitudinal movement control by acceleration and deceleration. In other words, the low-level compatible state is a driver state, i.e., a driving posture, that is compatible with driving automation levels 0 to 1.

[0127] As an example of a typical operation, Figure 15 shows an example in which the driving automation level is transitioned from automated driving to hands-off driving as the automated driving enabled section ends, similar to the first embodiment. Times T41 to T45 in Figure 15 are the same as times T11 to T15 in Figure 4. In Figure 15, the thin solid line indicates the transition of the actual driving automation level, and the thick dashed line indicates the transition of the driving automation level corresponding to the execution-related display on the HMI device 20.

[0128] 15, the driving control device 18 ends the autonomous driving when the autonomous driving possible section ends, and transitions to hands-off driving of level 2 without transitioning to level 1 or lower. Meanwhile, the display control unit 255 causes the HMI device 20 to display an operation instruction display to prompt the driver to temporarily assume a low-level compatible state where manual driving is possible.

[0129] As a result, the driving automation level of the system actually implemented by the driving control device 18 does not transition from automated driving to level 1 or lower, but transitions to hands-off driving of level 2. In contrast, the driving automation level in the driver's consciousness, based on the recognition of the display content on the HMI device 20 and the accompanying physical movements, transitions from automated driving to a driving automation level lower than hands-off driving, and then transitions to hands-off driving.

[0130] In this way, when the driving automation level of the system transitions from automated driving to hands-off driving, the driver's driving awareness is effectively increased by temporarily having the driver assume a manual driving posture. This makes it possible for the driver to assume an appropriate driving posture adapted to the transition in driving automation level. This makes it possible to effectively raise the driver's awareness regarding the transition in driving automation level when automated driving ends.

[0131] Specifically, at time T41, the in-vehicle system 10 executes a driver change request. In response to the driver change request, the display command transmitter 186 transmits to the HMI control device 25 display command information for displaying the visual guidance display GA shown in FIG. 16 on the CID display 231, which is a display device for displaying the second task screen. The visual guidance display GA shown in FIG. 16 is the same as that of the first embodiment shown in FIG. 5. The display command transmitter 186 also transmits to the HMI control device 25 display command information for displaying the transition-related display and the action instruction display shown in FIG. 16 on the front display device.

[0132] As a result, the meter panel 21 displays a transition-related display saying "Automatic driving ends 3 km ahead" in the level information display area DB1 on the meter display 212. The meter panel 21 also displays the first to fourth operation instruction displays GB1 to GB4, similar to those of the first embodiment, in the driving information display area DB2 on the meter display 212. The HUD device 22 displays a transition-related display saying "Prepare for driving changeover (manual)". The CID device 23 displays a line-of-sight guidance display GA in the information display area DA1 on the CID display 231.

[0133] In this way, when a driving change request is executed at time T41, the driving change operation is initiated at time T42. At this time, the first operation instruction display GB1, "Prepare for driving change (manual)," corresponds to a transition-related display. Furthermore, although the first operation instruction display GB1 indicates the driving automation level to be transitioned to manual driving, which is a lower level of driving automation than the original hands-off driving, it still indicates the need for transfer of authority from the system to the driver. Therefore, the first operation instruction display GB1 also corresponds to a main operation instruction display for instructing the main operation required to transfer authority to the driver as the driving automation level decreases.

[0134] The second action instruction display GB2, "Watch out ahead!", corresponds to a main action instruction display and also effectively functions as a secondary action instruction display for instructing secondary actions, including actions to adjust driving awareness. As described above, the system transitions from automated driving to hands-off driving. In hands-off driving, the driver is not required to perform steering control operations or acceleration / deceleration control operations. Therefore, the third action instruction display GB3, "Hold the steering wheel!", corresponding to manual driving, is not an action required for transitioning from automated driving to hands-off driving, but rather relates to an action for improving driving awareness. Therefore, the third action instruction display GB3 corresponds to a secondary action instruction display. Similarly, the fourth action instruction display GB4, "Put your feet on the pedals!", corresponds to a secondary action instruction display. In this way, the display control unit 255 causes the HMI device 20 to display, as action instruction displays, a main action instruction display instructing a main action, including an action to adjust driving posture, and a secondary action instruction display instructing a secondary action, including an action to adjust driving awareness.

[0135] When the driving changeover operation ends at time T43 and the driver assumes a driving position that allows for manual driving, the automation level determination unit 184 shifts the driving automation level on the system from automated driving to level 2 [G mode]. Meanwhile, the display control unit 255 shifts the driving automation level on the display from automated driving. As a result, from time T43 until time T45 when hands-off driving actually begins, even though the vehicle appears to be in manual driving mode, i.e., the display shows that the system is providing driving assistance equivalent to level 2 in "SAE J3016."

[0136] After the driver switching operation is completed at time T43, if the driver's driving operation state stabilizes, a smooth transition to hands-off driving is possible. Therefore, the operation state determination unit 183 determines the driving operation state by the driver. Then, the automation level determination unit 184 permits the execution of hands-off driving on the condition that the operation state determination unit 183 determines that the driving operation state corresponding to manual driving has stabilized.

[0137] Specifically, when the driver state acquired by the driver state acquisition unit 254 is stabilized in a low-level corresponding state, the display control unit 255 causes the HMI device 20 to display an approval request display prompting an input operation to approve the execution of the medium automation level. The approval request display is the same as that shown in FIG. 6. Then, when the driver performs an approval operation at time T45, the automation level determination unit 184 determines to execute hands-off driving. As a result, the in-vehicle system 10 starts hands-off driving. That is, after time T45, the driving automation level on the system and the driving automation level on the display match, and both result in hands-off driving. In this way, by transitioning to hands-off driving when the driver's driving awareness is sufficiently enhanced, the transition from automated driving to hands-off driving can be made even safer.

[0138] As described above, in this embodiment, when the high automation level is terminated, the display control unit 255 causes the HMI device 20 to display an operational instruction display that causes the driver to assume a low-level corresponding state that can execute a low automation level as the driver state. Specifically, when the high automation level is terminated and the driving automation level is transitioned to a medium automation level, which is a driving automation level between the high automation level and the low automation level, the display control unit 255 causes the HMI device 20 to display an operational instruction display that causes the driver to assume a low-level corresponding state. The low automation level corresponding to the operational instruction display is an apparent driving automation level that is different from the driving automation level actually executed by the driving control device 18 between the end of the high automation level and the start of the medium automation level. This makes it possible to effectively raise the driver's awareness regarding the transition of driving automation levels due to the end of automated driving.

[0139] Furthermore, in this embodiment, the display control unit 255 displays a line-of-sight guidance display GA, which guides the driver's line of sight from the second task screen to the HUD device 22 serving as the front display device, on the CID display 231 that displays the second task screen. This effectively encourages the driver to quickly assume a predetermined driving posture by guiding the driver's line of sight to the HUD device 22 serving as the front display device.

[0140] Fifth Embodiment The fifth embodiment will be described below with reference to Figures 17 to 19. The configuration of the in-vehicle system 10 according to this embodiment is the same as that of the fourth embodiment. However, the operation mode and the corresponding functional configuration of this embodiment are slightly different from those of the fourth embodiment. In other words, this embodiment is a partial modification of the fourth embodiment.

[0141] The fourth embodiment is a modification of the first embodiment in which the actual driving automation level transition is virtualized, i.e., apparent. Similarly, the present embodiment is a modification of the second embodiment in which the actual driving automation level transition is virtualized, i.e., apparent.

[0142] In this embodiment, the display control unit 255 sets the driving automation level at a low automation level according to the cause of termination of the high automation level. That is, the driving automation level at a low automation level to which the vehicle apparently transitions from the high automation level upon termination of the high automation level is variable according to the cause of termination of the high automation level. This makes it possible for the driver to adopt a more appropriate driving posture according to the cause of termination of the high automation level.

[0143] Figure 17 shows a case where the driving automation level is shifted from automated driving to hands-off driving as the automated driving possible section ends, similar to Figure 9. Times T51 to T55 in Figure 17 are the same as times T11 to T15 in Figures 4 and 9, respectively. In Figure 17 and subsequent figures, thin solid lines indicate the transition of the actual driving automation level, and thick dashed lines indicate the transition of the driving automation level corresponding to the execution-related display on HMI device 20.

[0144] In the example shown in Figure 17, the driving automation level of the system transitions from automated driving to level 2 [G mode] at time T53. Then, at time T55 when the driving operation state stabilizes and the driver's approval operation is received, the driving automation level of the system transitions from level 2 [G mode] to hands-off driving.

[0145] In contrast, the driver's driving awareness does not need to be raised to a level where they can immediately respond to manual driving; it is sufficient if they are raised to a level where they can smoothly transition to hands-off driving. Therefore, the driving automation level displayed transitions from automated driving to level 1 at time T53, and then transitions from level 1 to hands-off driving at time T55. In other words, the display format when a driver handover request is made at time T51 is the same as that shown in Figure 12.

[0146] Fig. 18 shows a case where autonomous driving is interrupted due to the detection of an obstacle on the road while traveling in an autonomous driving enabled section, similar to Fig. 10. Times T61 to T65 in Fig. 18 are the same as times T21 to T25 in Fig. 10, respectively.

[0147] Even when automated driving is interrupted due to the detection of an obstacle on the road, etc., it is preferable that the driving automation level on the system after the interruption be as high as possible. Therefore, in this example, the driving automation level on the system transitions from automated driving to Level 2 [G mode] at time T63 when the driving handover operation is completed. Then, at time T65 when the driving operation state stabilizes and the driver's approval operation is received, the driving automation level on the system transitions from Level 2 [G mode] to hands-off driving.

[0148] On the other hand, in this example, it is preferable that the driver's driving awareness is higher than in the example of the end of the autonomous driving possible section shown in Figure 17. Therefore, the driving automation level on the display temporarily shifts from autonomous driving to manual driving at time T63, and then shifts from manual driving to hands-off driving at time T65. In other words, the display mode when a driver change request is made at time T61 corresponds to the shift from autonomous driving to manual driving, similar to Figure 16.

[0149] Figure 19 shows a case where autonomous driving is terminated due to the occurrence of some kind of abnormality in the in-vehicle system 10. In this case, too, it is preferable that the driver's driving awareness is higher than in the example of the end of the autonomous driving possible section shown in Figure 17. Therefore, the display control unit 255 shifts the driving automation level on the display from autonomous driving to manual driving.

[0150] On the other hand, it is preferable that the driving automation level or driving assistance level of the system is as high as possible. Therefore, the automation level determination unit 184 reduces the driving automation level of the system from autonomous driving to a predetermined level. The predetermined level is the highest level that can be implemented at that time.

[0151] Specifically, when the operation control device 18 determines that the automatic operation cannot be continued due to the occurrence of an abnormality, it executes a request for switching the operation to manual operation at time T71. That is, the display command sending unit 186 sends display command information to the HMI control device 25 to cause the HMI device 20 to display the request for switching the operation.

[0152] When a driver changeover request is executed by the HMI device 20 at time T71, the driver changeover operation is initiated at time T72. Then, when the driver changeover operation is completed at time T73, the displayed driving automation level transitions from automated driving to manual driving. Meanwhile, the system's driving automation level transitions to the highest level feasible at that time, even though automated driving is terminated. Even if the system's driving automation level only allows manual driving, the driver can appropriately take over driving by transitioning the displayed driving automation level from automated driving to manual driving.

[0153] In this example, an abnormality occurs in the driving automation system that makes it impossible to continue autonomous driving. Therefore, even after T74, when the driver's manual driving operation has stabilized, the approval operation request to shift the driving automation level to a higher level is not executed. However, as mentioned above, even though the vehicle appears to be driving manually, the system is still able to perform as many driving assistance operations as possible.

[0154] 17 to 19, the processing when the driver changeover operation is not performed may be the same as in the cases of Fig. 8 and Fig. 13. That is, a predetermined emergency processing such as MRM may be performed after waiting for a predetermined time. Alternatively, if the minimum necessary safety control operations such as collision avoidance braking and collision damage mitigation braking are possible, the above emergency processing may not be performed even if the driver changeover operation is not performed.

[0155] (Sixth embodiment) The sixth embodiment will be described below with reference to Figures 20 and 21. The configuration of the in-vehicle system 10 according to this embodiment is similar to that of the fourth and fifth embodiments. However, the operation mode and the corresponding functional configuration of this embodiment are slightly different from those of the fourth and fifth embodiments. In other words, this embodiment is a partial modification of the fourth and fifth embodiments.

[0156] In this example, although it is possible to switch to hands-off driving when the autonomous driving section ends, in order to improve driving awareness, the HMI device 20 suggests switching to level 1 (i.e., ACC). Fig. 20 shows a case where the suggestion to switch to ACC has been approved, and Fig. 21 shows a case where such approval has not been made.

[0157] That is, in the example shown in Figure 20, the reasons for terminating autonomous driving are the end of the section where autonomous driving is possible and a transition to ACC due to ACC approval. On the other hand, in the example shown in Figure 21, the reasons for terminating autonomous driving are the end of the section where autonomous driving is possible and a transition to hands-off driving due to ACC not being approved. Due to these differences in the reasons for termination, the transition patterns from autonomous driving to the driving automation level on the system and the driving automation level on the display differ between Figure 20 and Figure 21.

[0158] Specifically, in the example shown in Fig. 20, first, at time T81, the in-vehicle system 10 executes a driving changeover request similar to time T41 in Fig. 15 or time T51 in Fig. 17. Then, at time T82, the in-vehicle system 10 executes a proposal to switch to ACC. Then, at time T83, the driver executes an operation to approve the proposal to switch to ACC, and starts a driving changeover operation to adopt a driving posture that can accommodate ACC.

[0159] In this example, the driver has a clear intention to transition from automated driving to ACC. Therefore, when the driving changeover operation is completed at time T84, before time T85, the in-vehicle system 10 transitions both the system- and display-based driving automation levels from automated driving to level 1 ACC. Time T85 is the time when the end point of the automated driving enabled section is reached. Thereafter, at time T86, when the driver's driving operation state has stabilized, if the conditions for transitioning the driving automation level to a higher level, for example, level 2 [G mode], are met, a transition suggestion can be executed as appropriate.

[0160] On the other hand, in the example shown in Fig. 21, first, at time T91, the in-vehicle system 10 executes a driving change request similar to the example shown in Fig. 20. Then, at time T92, the in-vehicle system 10 executes a proposal to transition to ACC. Times T91 and T92 are similar to times T81 and T82 shown in Fig. 20, respectively.

[0161] In this example, the driver does not approve the proposal to switch to ACC until time T93. In this case, the driver may not be clearly aware that he or she is switching from automated driving to a predetermined driving automation level. Therefore, at time T93, the in-vehicle system 10 requests a shift to manual driving, similar to the request at time T41 in FIG. 15.

[0162] When the operation of switching to manual driving is completed at time T94, before time T95, the automation level determination unit 184 shifts the driving automation level on the system from automated driving to level 2 [G mode]. Time T95 is the time when the end point of the section where automated driving is possible is reached. Meanwhile, the display control unit 255 shifts the driving automation level on the display from automated driving. As a result, from time T94 onwards, although the vehicle appears to be driving manually, the system is executing driving assistance equivalent to level 2 in "SAE J3016."

[0163] Thereafter, when the driver's driving operation state stabilizes, a smooth transition to hands-off driving is possible. Therefore, when the driver's driving operation state stabilizes, a proposal to transition to hands-off driving is executed. When an approval operation for this transition proposal is executed at time T96, the in-vehicle system 10 transitions both the driving automation level on the system and the driving automation level on the display to hands-off driving.

[0164] Seventh Embodiment The seventh embodiment will be described below with reference to Figures 22 to 24. The configuration of the in-vehicle system 10 according to this embodiment is similar to that of the first embodiment, etc. The present embodiment differs slightly from the first embodiment, etc. in terms of the operation mode and the corresponding functional configuration.

[0165] Similar to the example of FIG. 4, FIGS. 22 to 24 show an example of transitioning from autonomous driving at a high automation level (i.e., level 3) to level 2 at a medium automation level. In FIGS. 22 to 24, times TA1 to TA6 indicate the passage of time. In the example of FIG. 22, the example of FIG. 23, and the example of FIG. 24, the vehicle's driving route and driving conditions (e.g., vehicle speed, etc.) are the same, and only the duration of autonomous driving being performed at time TA1 differs. That is, FIG. 22 shows an example in which the duration of autonomous driving is short, i.e., less than a predetermined time. On the other hand, FIGS. 23 and 24 show examples in which the duration of autonomous driving is equal to or longer than a predetermined time. The duration of autonomous driving in the example of FIG. 23 is shorter than that in the example of FIG. 24.

[0166] The contents of times TA1 to TA6 are as follows. The changeover request start time TA1 is the start time of the driver changeover request. The driver changeover request includes displaying a transition-related display and an action instruction display on the front display device. In other words, the changeover request start time TA1 corresponds to the action instruction timing at which the action instruction display (i.e., at least the main action instruction display) is displayed on the HMI device 20. Therefore, the changeover request start time TA1 is the same as the display start time T11 in FIG. 4. The changeover start time TA2 is the time at which the driver starts the driver changeover action in response to the driver changeover request, and is the same as the driver changeover action start time T12 in FIG. 4. The changeover end time TA3 is the time at which the driver's driver changeover action, which started at the changeover start time TA2, ends, and is the same as the driver changeover action end time T13 in FIG. 4. The end point passing time TA4 is the time at which the vehicle reaches the end point of the autonomously drivable section, and is the same as time T14 in FIG. 4. For ease of explanation, Figures 22 to 24 are drawn so that the horizontal time axis is approximately the same scale and the horizontal axis position of the end point passing time TA4 is the same. The approval period start time TA5 is the timing at which the driver can perform the approval operation. Specifically, the approval period start time TA5 corresponds to, for example, the approval request timing at which an approval request display is displayed on the HMI device 20. Alternatively, the approval period start time TA5 corresponds to, for example, the approval permission timing at which an input operation to approve the execution of the medium automation level is permitted. The transition time TA6 is the approval execution timing at which the driver performs the approval operation to approve the execution of the medium automation level, and is the same as time T15 in Figure 4.

[0167] When the duration of autonomous driving is short, it is assumed that the driver's driving awareness has not deteriorated significantly since the start of autonomous driving, and that the driver's driving operation sense (e.g., response to steering input, etc.) has not been lost. In contrast, the longer the duration of autonomous driving, the more likely it is that the driver's driving awareness will decline and the driver's driving operation sense will be lost. Therefore, in order to effectively restore the driver's driving awareness and driving operation sense, it is preferable to have the driver assume a driving posture corresponding to a lower driving automation level the longer the duration of autonomous driving. Therefore, in this embodiment, the automation level determination unit 184 sets the change mode of the driving automation level from the high automation level to the medium automation level depending on the duration of the high automation level.

[0168] Specifically, when the duration of a high automation level is less than a predetermined time, the automation level determination unit 184 transitions from the high automation level to a medium automation level without transitioning to a low automation level, as shown in FIG. 22. The automation level determination unit 184 also sets the driving automation level at a low automation level according to the duration of the high automation level. More specifically, in the example of FIG. 24, where the duration of automated driving is relatively long, it is expected that the degree of decline in driving awareness and loss of driving operation sense will be greater than in the example of FIG. 23, where the duration of automated driving is not so long. Therefore, in the example of FIG. 24, it is effective to have the driver experience manual driving, which is a lower driving automation level, i.e., level 0, in order to significantly restore driving awareness and driving operation sense. On the other hand, in the example of FIG. 23, a reduction to level 1 is expected to sufficiently restore driving awareness and driving operation sense. Therefore, the automation level determination unit 184 sets a lower driving automation level at a low automation level the longer the duration of the high automation level. As a result, the longer the duration of autonomous driving, the greater the possibility that the driver will lose their driving awareness and / or driving operation sense, and the longer the preparation period can be secured to allow the driver to regain their driving awareness and / or driving operation sense when autonomous driving ends.

[0169] Furthermore, in this embodiment, the display command sending unit 186 or the display control unit 255 sets the operation instruction timing, i.e., the changeover request start time TA1, according to the duration of the high automation level. That is, the longer the duration of the automated driving, the earlier the changeover request start time TA1 is set. Specifically, with reference to FIGS. 22 to 24, the end point passing time TA4, the approval period start time TA5, and the transition time TA6 are almost the same or have little difference between FIGS. 22 to 24. In contrast, the changeover request start time TA1 is latest in the example of FIG. 22, where the duration of the automated driving is shortest, and is earliest in the example of FIG. 24, where the duration of the automated driving is longest. As a result, the longer the duration of the automated driving, the longer the time interval between the changeover request start time TA1 and the end point passing time TA4 can be secured. This time interval can be used as a preparation period for the driver to regain driving awareness and / or driving operation sense when transitioning from a high automation level to a medium automation level. Therefore, according to this embodiment, the longer the duration of autonomous driving, the longer the preparation period possible for the driver to regain driving awareness and / or driving operation sense when autonomous driving ends can be secured.

[0170] As shown in Figures 22 to 24, in this embodiment, the time interval from the changeover request start time TA1 to the approval period start time TA5 is set to be longer as the duration of the high automation level increases. In other words, if the changeover request start time TA1 is taken as a reference point, the timing of the approval period start time TA5 relative to this reference point is set later as the duration of the high automation level increases. In this way, according to this embodiment, the approval request timing or approval permission timing specified by the approval period start time TA5 is set according to the duration of the high automation level. As a result, the longer the duration of automated driving, the longer the preparation period for the driver to recover their driving awareness and / or driving operation sense when automated driving ends can be secured as long as possible.

[0171] 22 to 24 are drawn so that the approval period start time TA5 and the transition time TA6 are positioned at the same horizontal axis. However, this embodiment is not limited to this. That is, for example, the time interval from the end point passing time TA4 to the approval period start time TA5 can be set to be longer as the duration of the high automation level increases. In other words, the horizontal axis positions of the approval period start time TA5 and the transition time TA6 in FIG. 23 can be located to the right of the horizontal axis positions of the approval period start time TA5 and the transition time TA6 in FIG. 22. Similarly, the horizontal axis positions of the approval period start time TA5 and the transition time TA6 in FIG. 24 can be located to the right of the horizontal axis positions of the approval period start time TA5 and the transition time TA6 in FIG. 23.

[0172] Eighth Embodiment The eighth embodiment will be described below with reference to Figures 25 and 26. The contents of times TA1 to TA6 in Figures 25 and 26 are the same as those in Figures 22 to 24.

[0173] In this embodiment, the in-vehicle system 10 including the driving control device 18 is configured to be able to perform automated driving at levels 3 and 4. Other than the operational aspects related to these driving automation levels and the corresponding functional configurations, the configuration of the in-vehicle system 10 according to this embodiment is the same as that of the first embodiment and the like.

[0174] In level 4 autonomous driving, if it becomes difficult to continue at that level (for example, when an obstacle on the road or a construction zone appears), the driving automation system will respond. In other words, while level 4 autonomous driving is being performed, the driver is not required to respond appropriately to requests from the driving automation system to take over driving. Therefore, the driver is allowed to sleep while level 4 autonomous driving is being performed. On the other hand, in level 3 autonomous driving, if it becomes difficult to continue at that level, the driver must respond appropriately to requests from the driving automation system to take over driving. Therefore, while level 3 autonomous driving is being performed, the driver is required to be awake enough to be able to respond appropriately to requests from the driving automation system to take over driving.

[0175] In the example of Figure 25 and the example of Figure 26, the vehicle's driving route and driving conditions are the same, and only the driver's state during autonomous driving is different. Specifically, Figure 25 shows a case where the driver was not asleep during level 4 autonomous driving, i.e., before the change request start time TA1. On the other hand, Figure 26 shows a case where the driver was asleep during level 4 autonomous driving.

[0176] Both Level 3 and Level 4 can be implemented in "specific limited areas," i.e., sections where autonomous driving is possible. For this reason, it is necessary to request the driver to take over driving with sufficient time to spare before the vehicle reaches the end of the section where autonomous driving is possible. However, it is assumed that the driver after falling asleep (i.e., waking up from sleep) will have reduced driving awareness and / or driving operation ability compared to before the start of sleep-enabled autonomous driving. This reduced driving operation ability includes the loss of the driving operation sense described above. It is assumed that the reduced driving awareness and / or driving operation ability will be particularly noticeable immediately after waking up from sleep.

[0177] Therefore, the driver state detection unit 17 detects whether the driver state while executing the high automation level is asleep. Then, the automation level determination unit 184 sets the change mode of the driving automation level from the high automation level to the medium automation level depending on whether the driver state while executing the high automation level is asleep. Specifically, if the driver state while executing level 4 as a high automation level is asleep, the automation level determination unit 184 ends the high automation level earlier than if the driver state is not asleep.

[0178] More specifically, if the driver's state is not asleep during automated driving at level 4 (high automation level), level 4 is maintained until shift end time TA3, as shown in Figure 25. Then, at shift end time TA3, the driving automation level transitions from level 4 to level 1. After that, at transition time TA6, the driving automation level transitions from level 1 to level 2. This transition pattern is similar to the level transition pattern in the example of Figure 4, where the vehicle transitions from automated driving to level 1 (low automation level) and then to level 2 (medium automation level).

[0179] In contrast, if the driver's state is asleep during level 4 automated driving, as shown in Figure 26, level 4 ends at change request start time TA1, and the driving automation level temporarily transitions to level 3, which is a sub-high automation level. The sub-high automation level is a driving automation level included in automated driving between the high automation level and the medium automation level, and in the example of Figure 26, it is level 3. From the change request start time TA1 to the change end time TA3, the driving automation level is maintained at level 3. That is, a driving change operation is executed at level 3. Then, at the change end time TA3, the driving automation level transitions from level 3 to level 1. Then, at transition time TA6, the driving automation level transitions from level 1 to level 2. That is, the driving automation level is set to level 3 from the change start time TA2 to the change end time TA3, which indicate the period of operation during which the driver's state transitions to a low-level compatible state corresponding to level 1. This allows a driver who has fallen asleep under Level 4 autonomous driving to gradually assume an appropriate driving posture under Level 3 autonomous driving, without being forced to suddenly transition to a driving posture corresponding to Level 1.

[0180] As described above, in this embodiment, when the driver state while executing a high automation level is asleep, the automation level determination unit 184 first transitions from the high automation level to a sub-high automation level and then transitions to a low automation level. In contrast, when the driver state while executing a high automation level is not asleep, the automation level determination unit 184 transitions from the high automation level to a low automation level without transitioning to the sub-high automation level. This allows driving automation level transition control for restoring the driver's driving awareness and / or driving operation ability when autonomous driving ends to be appropriately executed depending on whether the driver actually fell asleep during autonomous driving.

[0181] Ninth embodiment The ninth embodiment will be described below with reference to Figures 26 and 27. The details of times TA1 to TA6 in Figures 26 and 27 are the same as those in Figures 25 and 26. This embodiment is a partial modification of the eighth embodiment. That is, in this embodiment, the driving automation level transition mode in the example of Figure 25 in the eighth embodiment is modified to the mode shown in Figure 27.

[0182] In the example of Figure 26 and the example of Figure 27, the vehicle's driving route and driving conditions are the same, and only the driver's state during autonomous driving is different. Specifically, Figure 26 shows a case where the driver was asleep during level 4 autonomous driving, i.e., before the change request start time TA1. On the other hand, Figure 27 shows a case where the driver was not asleep during level 4 autonomous driving.

[0183] As mentioned above, it is assumed that a driver's driving awareness and / or driving operation ability is reduced after falling asleep compared to before automated driving began. Therefore, if a driver is not asleep during automated driving, they will be able to adapt to a lower level of driving automation than if they were asleep.

[0184] Therefore, in this embodiment, the automation level determination unit 184 sets the driving automation level at a low automation level depending on whether the driver's state is asleep while the high automation level is being executed. Specifically, if the driver's state is asleep during automated driving at level 4 as a high automation level, the low automation level between the changeover end time TA3 and the transition time TA6 is set to level 1, as shown in FIG. 26. On the other hand, if the driver's state is not asleep during automated driving at level 4 as a high automation level, the low automation level is set to level 0, as shown in FIG. 27. This allows driving automation level transition control for restoring the driver's driving awareness and / or driving operation ability when automated driving ends to be appropriately executed depending on whether the driver actually fell asleep during automated driving.

[0185] Tenth Embodiment The tenth embodiment will be described below with reference to Figures 27 and 28. The details of times TA1 to TA6 in Figures 26 and 27 are the same as those in Figures 26 and 27. This embodiment is a partial modification of the ninth embodiment. That is, in this embodiment, the driving automation level transition mode in the example of Figure 26 in the ninth embodiment is modified to the mode shown in Figure 28.

[0186] In the example of Figure 27 and the example of Figure 28, the vehicle's driving route and driving conditions are the same, and only the driver's state during autonomous driving is different. Specifically, Figure 27 shows a case where the driver was not asleep during level 4 autonomous driving, i.e., before the change request start time TA1. On the other hand, Figure 28 shows a case where the driver was asleep during level 4 autonomous driving.

[0187] In this embodiment, the display command sending unit 186 or the display control unit 255 sets the operation instruction timing, i.e., the change request start time TA1, depending on whether the driver state while the high automation level is being executed is a sleeping state. That is, the change request start time TA1 is set earlier when the driver state while the high automation level is being executed is a sleeping state. Specifically, with reference to FIGS. 27 and 28, the end point passing time TA4, the approval period start time TA5, and the transition time TA6 are almost the same or have little difference between FIGS. 27 and 28. In contrast, the change request start time TA1 is earlier in the example of FIG. 28, where the driver state while the high automation level is being executed is a sleeping state, than in the example of FIG. 27. This allows the preparation period for restoring the driver's driving awareness and / or driving operation ability, which have deteriorated due to sleep during automated driving, when automated driving ends to be as long as possible.

[0188] In this embodiment, the time interval from the changeover request start time TA1 to the approval period start time TA5 is set to be longer when the driver's state while the high automation level is being executed is a sleep state. In other words, if the changeover request start time TA1 is set as a reference point, the timing of the approval period start time TA5 relative to that reference point is set later as the duration of the high automation level becomes longer. As such, according to this embodiment, the approval request timing or approval permission timing specified by the approval period start time TA5 is set depending on whether the driver's state while the high automation level is being executed is a sleep state. This makes it possible to ensure as long as possible a preparation period for restoring the driver's driving awareness and / or driving operation ability, which have deteriorated due to sleep during automated driving, when automated driving ends.

[0189] 27 and 28 are drawn so that the approval period start time TA5 and the transition time TA6 are positioned at the same horizontal axis. However, this embodiment is not limited to this. That is, for example, the time interval from the end point passing time TA4 to the approval period start time TA5 can be set depending on whether the driver state during execution of the high automation level is a sleep state or not. In other words, the horizontal axis positions of the approval period start time TA5 and the transition time TA6 in FIG. 28 can be to the right of the horizontal axis positions of the approval period start time TA5 and the transition time TA6 in FIG. 27.

[0190] Eleventh Embodiment In the seventh to tenth embodiments, the transition of driving automation levels between the high automation level and the medium automation level, i.e., from the shift request start time TA1 to the transition time TA6, was the actual driving automation level. However, the transition of driving automation levels in the seventh to tenth embodiments may be an apparent transition, as in the fourth embodiment, etc.

[0191] 23 to 28 may be an "apparent" transition of the driving automation level due to information presentation using the HMI device 20. In this case, the actual driving automation level determined and executed by the automation level determination unit 184 transitions from automated driving to level 2 at shift end time TA3, as shown in FIG. 22. Furthermore, the functional configuration or operation of the automation level determination unit 184 and / or the display command transmission unit 186 in each of the above embodiments may be interpreted as the functional configuration or operation of the display control unit 255.

[0192] (Variation) The present invention is not limited to the above-described embodiment. Therefore, the above-described embodiment can be modified as appropriate. Representative modifications will be described below. In the following description of the modifications, differences from the above-described embodiment will be mainly described. Furthermore, identical or equivalent parts between the above-described embodiment and the modifications are given the same reference numerals. Therefore, in the following description of the modifications, the description of the above-described embodiment can be used as appropriate for components having the same reference numerals as those in the above-described embodiment, unless there is a technical contradiction or special additional explanation.

[0193] The present invention is not limited to the specific device configurations shown in the above embodiments. For example, the vehicle 1 equipped with the in-vehicle system 10 is not limited to a standard automobile. Specifically, the vehicle 1 may be a large vehicle such as a cargo truck. The number of wheels is also not particularly limited, and the vehicle 1 may be a three-wheeled vehicle, or a six- or eight-wheeled vehicle such as a cargo truck. The type of vehicle 1 may be a conventional vehicle equipped with only an internal combustion engine, an electric vehicle or a fuel cell vehicle without an internal combustion engine, or a so-called hybrid vehicle. The shape and structure of the body of the vehicle 1 are also not limited to a box shape, i.e., a substantially rectangular shape in a plan view. The use of the vehicle 1, the position of the steering wheel 8, the number of occupants, etc. are also not particularly limited.

[0194] The communication standard for the in-vehicle system 10 may be other than CAN (internationally registered trademark), such as FlexRay (internationally registered trademark). The communication standard for the in-vehicle system 10 is not limited to one type. For example, the in-vehicle system 10 may have a sub-network line that complies with a communication standard such as LIN. LIN is an abbreviation for Local Interconnect Network.

[0195] The vehicle condition sensor 11, the external environment condition sensor 12, and the perimeter monitoring sensor 13 are not limited to the above examples. For example, the perimeter monitoring sensor 13 may include a sonar, i.e., an ultrasonic sensor. Alternatively, the perimeter monitoring sensor 13 may include two or more types of sensors selected from a millimeter wave radar sensor, a submillimeter wave radar sensor, a laser radar sensor, and an ultrasonic sensor. There is also no particular limitation on the number of sensors to be installed.

[0196] The locator 14 is not limited to the above example. For example, the locator 14 does not need to have a built-in gyro sensor and acceleration sensor. Specifically, the inertia acquisition unit 142 may receive output signals from an angular velocity sensor and an acceleration sensor provided outside the locator 14 as the vehicle state sensor 11.

[0197] The DCM 15 may be omitted, that is, traffic information may be acquired by the navigation device 16. Alternatively, the navigation device 16 may have a configuration including the locator 14 and the DCM 15.

[0198] The navigation device 16 may be connected to the HMI control device 25 so as to be capable of communicating information via a sub-communication line different from the in-vehicle communication line 10A.

[0199] The navigation device 16 may have a display screen dedicated to displaying the navigation screen, separate from the HMI device 20. Alternatively, the navigation device 16 may be provided as a part of the HMI device 20. Specifically, for example, the navigation device 16 may be integrated with the CID device 23.

[0200] The driver state detection unit 17 may be connected to the HMI control device 25 so as to be capable of communicating information via a sub-communication line different from the in-vehicle communication line 10A.

[0201] The driver state detection unit 17 is not limited to a configuration including the gaze detection unit 171, the posture detection unit 172, and the operation state detection unit 173. That is, for example, a function corresponding to the posture detection unit 172 can be achieved by image recognition using the configuration of the gaze detection unit 171. The driver state detection unit 17 may also include a biometric information sensor that detects biometric information such as the driver's pulse. In this case, components such as detection electrodes in the biometric information sensor may be shared with components in the operation state detection unit 173 that detect the grip state of the steering wheel 8.

[0202] In the above embodiment, the driving control device 18 is configured to be able to execute vehicle control operations corresponding to levels 1 to 3. However, the present invention is not limited to this aspect. That is, for example, the present invention can be suitably applied to cases where vehicle control operations corresponding to levels 1 to 5 are able to be executed.

[0203] Furthermore, the levels or categories of driving automation in the present invention are not limited to those specified in "SAE J3016." Specifically, "SAE J3016" specifies that the higher the level of driving automation, the larger the level numerical value. However, the present invention is not limited to such an embodiment. That is, for example, the present invention can be similarly applied to a standard in which the highest level of driving automation is defined as "Level 1," and the lower the level of driving automation, the larger the level numerical value.

[0204] The HMI device 20 is not limited to a configuration including the meter panel 21, the HUD device 22, and the CID device 23. That is, for example, the meter panel 21 and the CID device 23 may be integrated.

[0205] The meter 211 and the meter display 212 may be realized by a single display device. In this case, the meter 211 may be provided as display areas at both the left and right ends of a single display device that is a liquid crystal or organic EL display. That is, the meter 211 may be realized by displaying images of a bezel, pointer, scale, etc. corresponding to a tachometer, speedometer, water temperature gauge, etc. Furthermore, the meter display 212 may be provided as a display area other than the meter 211 on such a display device.

[0206] The input device 232 may have a pointing device or the like that is operated by the driver's hand instead of or in addition to the touch panel that is superimposed on the CID display 231. The input device 232 may have a voice input device that detects the driver's speech.

[0207] In the above embodiment, the operation control device 18 and the HMI control device 25 have a configuration as a so-called in-vehicle microcomputer equipped with a CPU, etc. However, the present invention is not limited to such a configuration.

[0208] For example, all or part of the driving control device 18 may be configured with a digital circuit, such as an ASIC or FPGA, configured to enable the above-described operations. ASIC stands for Application Specific Integrated Circuit. FPGA stands for Field Programmable Gate Array. In other words, the driving control device 18 may have both an on-board microcomputer and a digital circuit. The same applies to the HMI control device 25.

[0209] The program according to the present invention, which enables the execution of the various operations, procedures, or processes described in the above embodiments, can be downloaded or upgraded via V2X communication by the DCM 15 or the like. V2X stands for Vehicle to X. Alternatively, such a program can be downloaded or upgraded via a terminal device provided in a vehicle 1 manufacturing plant, a repair shop, a dealer, or the like. Such a program may be stored on a memory card, an optical disk, a magnetic disk, or the like.

[0210] In this manner, each of the above functional configurations and methods may be implemented by a special-purpose computer configured by configuring a processor and memory programmed to execute one or more functions embodied in a computer program. Alternatively, each of the above functional configurations and methods may be implemented by a special-purpose computer configured by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, each of the above functional configurations and methods may be implemented by one or more special-purpose computers configured by combining a processor and memory programmed to execute one or more functions with a processor configured with one or more hardware logic circuits. Furthermore, a computer program may be stored in a computer-readable, non-transitory storage medium as instructions to be executed by a computer. In other words, each of the above functional configurations and methods may be expressed as a computer program including procedures for implementing the same, or as a non-transitory storage medium storing the program.

[0211] The present invention is not limited to the specific functional configurations and operational examples shown in the above embodiment. For example, in accordance with the driving automation levels specified in "SAE J3016," in the above embodiment, the transition pattern of the driving automation level on the system and / or display was 3-0-2, 3-0-1, 3-1-2, 4-0-2, 4-1-2, etc. However, the present invention is not limited to such a pattern. For example, such a transition pattern may be 4-0-1, 5-2-3, etc. In other words, if the transition pattern is LA-LB-LC, it is sufficient that LA≧3, LB≦2, and LA>LC>LB.

[0212] In the above embodiment, the display for guiding the line of sight and instructing the operation is executed by the CID device 23. However, the present invention is not limited to this aspect. For example, such display may be executed by the terminal device 24. That is, the HMI control device 25 may cause various displays to be executed on the display screen of the terminal device 24.

[0213] The visual guidance display GA may be displayed in the screen area DA2, which is the second task screen display area, on the CID display 231. The display form of the visual guidance display GA is not limited to the specific example exemplified in the above embodiment. For example, the visual guidance display GA may be displayed as a meteor or comet that flows in the visual guidance direction.

[0214] In the example shown in FIG. 8 etc., the in-vehicle system 10 may selectively execute only one of the rest suggestion and MRM as the predetermined emergency process depending on the road conditions. Alternatively, the in-vehicle system 10 may first execute the rest suggestion and then execute MRM. Then, the in-vehicle system 10 may execute MRM if the driver does not adopt a driving posture corresponding to Level 2 or does not perform any operation in response to the rest suggestion even after a predetermined time has elapsed since the driver received the rest suggestion. There are no particular limitations on the content of the predetermined emergency process.

[0215] 4 and 9, the timing at which the driving automation level transitions from a high automation level, i.e., level 3, to a low automation level, i.e., level 0 or level 1, can be changed from the timing at which the driving change operation ends (i.e., time T13). Specifically, this transition timing can be set appropriately between the timing at which the driving change operation ends and the timing at which the end of the autonomous driving possible section is reached (i.e., between times T13 and T14).

[0216] In the example of Figure 15, the timing at which the actual driving automation level transitions from high automation level, i.e., level 3, to medium automation level, i.e., level 2, can be changed from the timing at which the driving change operation ends (i.e., time T43). Specifically, this transition timing can be set appropriately between times T43 and T44. The same applies to the example of Figure 17, etc.

[0217] Similar expressions such as "obtain," "calculate," "estimate," "detect," "sensing," and "determine" may be substituted for each other as appropriate within the scope of technical inconsistency. "Detect" or "detection" and "extract" may also be substituted for each other as appropriate within the scope of technical inconsistency.

[0218] It goes without saying that the elements constituting the above-described embodiments are not necessarily essential unless expressly stated as essential or clearly considered essential in principle. Furthermore, when numerical values ​​such as the number, amount, range, etc. of components are mentioned, the present invention is not limited to those specific numerical values ​​unless expressly stated as essential or clearly limited to specific numerical values ​​in principle. Similarly, when the shape, direction, positional relationship, etc. of components are mentioned, the present invention is not limited to those shapes, directions, positional relationship, etc. unless expressly stated as essential or clearly limited to specific shapes, directions, positional relationship, etc. in principle.

[0219] The modified examples are not limited to the above examples. For example, all or part of one of the multiple embodiments may be combined with all or part of another embodiment, provided that there is no technical contradiction. There is no particular limit to the number of combinations. Similarly, all or part of one of the multiple modified examples may be combined with all or part of another embodiment, provided that there is no technical contradiction. Furthermore, all or part of the above embodiment may be combined with all or part of the above modified example, provided that there is no technical contradiction. In short, the above embodiments and modified examples may be combined in any way, provided that there is no technical contradiction.

[0220] (summary) The present disclosure, as described in the above embodiments and modifications, includes the following aspects related to an HMI control method and an HMI control program. Note that the following aspects may be combined with one another as long as there is no technical contradiction between them.

[0221] The HMI control method is a method for controlling an HMI device (20) that displays images visible to the driver of an automatically drivable vehicle (1). The HMI control program is a program executed by an HMI control device (25) configured to control the HMI device (20) that displays images visible to the driver of the automatically drivable vehicle (1).

[0222] According to a first aspect, the HMI control method and the process executed by the HMI control device include: An automation level acquisition process for acquiring a result of a driving automation level determination in a driving control device (18) that controls the driving of the vehicle; A display control process that controls an image display operation in the HMI device according to the driving automation level acquired in the automation level acquisition process. and The display control process is a process in which, when terminating the high automation level, which is the driving automation level included in the autonomous driving, the HMI device displays an operational instruction display that causes the driver to take a low-level corresponding state, which is the driving automation level, in which the on-board system (10) including the driving control device does not perform at least one of lateral movement control by steering and longitudinal movement control by acceleration and deceleration, as the driver state, and which is capable of performing the low automation level, which is the driving automation level.

[0223] According to a second aspect, the display control process is a process of displaying, on the HMI device, the operational instruction display that causes the driver to adopt the low-level corresponding state when the high automation level is terminated and a transition is made to a medium automation level, which is the driving automation level between the high automation level and the low automation level.

[0224] According to a third aspect, the HMI control method and the process executed by the HMI control device include: a driver status acquisition process for acquiring the driver status; an operation reception process for receiving an input operation by the driver; and The display control process is a process that displays an approval request display on the HMI device to prompt the input operation to approve the execution of the medium automation level when the driver state acquired by the driver state acquisition process stabilizes in the low-level corresponding state.

[0225] According to a fourth aspect, the display control process is a process for setting the approval request timing for displaying the approval request display on the HMI device, or the approval permission timing for permitting the input operation to approve the execution of the medium automation level, depending on the duration of the high automation level or whether the driver's state while executing the high automation level is asleep or not.

[0226] According to a fifth aspect, the display control process is a process for setting the driving automation level at the low automation level depending on the cause of termination of the high automation level.

[0227] According to a sixth aspect, the display control process is a process for setting the driving automation level at the low automation level depending on the duration of the high automation level or whether the driver's state while executing the high automation level is a sleep state or not.

[0228] According to the seventh aspect, the display control process is a process for setting the timing of the operation instruction display on the HMI device depending on the duration of the high automation level or whether the driver's state while executing the high automation level is asleep or not.

[0229] According to an eighth aspect, the display control process is a process of displaying, as the operation instruction displays, a main action instruction display that instructs a main action including an action to adjust driving posture as the driver state, and a secondary action instruction display that instructs a secondary action including an action to adjust driving awareness as the driver state on the HMI device.

[0230] According to a ninth aspect, the display control process is a process of displaying, on a display device (23) that displays the second task screen, a visual guidance display that guides the driver's gaze from a second task screen that displays a second task that the driver can perform while the automatic driving is being performed to a front display device (21, 22) that is positioned on the front side of the vehicle.

[0231] According to a tenth aspect, the front display device is a head-up display device (22).

[0232] According to the 11th aspect, the low automation level corresponding to the operation instruction display is an apparent driving automation level that is different from the driving automation level actually executed by the driving control device between the end of the high automation level and the start of the medium automation level.

[0233] The present disclosure, as described in the above embodiments and modifications, includes the following aspects related to an operation control method and an operation control program. Note that the following aspects may be applied in combination with one another as long as there is no technical contradiction between them.

[0234] The driving control method is a method for controlling the driving of an automatically drivable vehicle 1. The driving control program is a program executed by a driving control device 18 configured to control the driving of the automatically drivable vehicle 1.

[0235] According to a first aspect, the operation control method and the process executed by the operation control device include: a driving status acquisition process for acquiring a driving status of the vehicle; an automation level determination process that determines a driving automation level based on the driving conditions acquired by the driving condition acquisition process; and The automation level determination process is a process in which, when a high automation level, which is the driving automation level included in the autonomous driving, is terminated and a medium automation level, which is a driving automation level lower than the high automation level, is transitioned from the high automation level to a low automation level, which is a driving automation level lower than the medium automation level, and then transitioned to the medium automation level.

[0236] According to the second viewpoint, The driving operation state determination process further includes determining a driving operation state by the driver. The automation level determination process is a process that allows execution of the medium automation level on the condition that the operation state determination process determines that the driving operation state corresponding to the low automation level has stabilized.

[0237] According to a third aspect, the automation level determination process is a process for setting the driving automation level at the low automation level depending on the cause of termination of the high automation level.

[0238] According to a fourth aspect, the automation level determination process is a process that sets the change pattern of the driving automation level from the high automation level to the medium automation level depending on the duration of the high automation level or whether the driver's state while executing the high automation level is sleepy or not.

[0239] According to a fifth aspect, the automation level determination process is a process for setting the driving automation level at the low automation level depending on the duration of the high automation level or whether the driver's state while executing the high automation level is a sleep state or not.

[0240] According to a sixth aspect, the automation level determination process is a process that terminates the high automation level earlier when the driver state while the high automation level is being executed is a sleep state than when the driver state is a non-sleep state.

[0241] According to the seventh aspect, The automation level determination process includes: If the driver state while the high automation level is being executed is a sleep state, the system first transitions from the high automation level to a sub-high automation level, which is a driving automation level between the high automation level and the medium automation level and is included in the automated driving, and then transitions to the low automation level; When the driver state while executing the high automation level is a non-sleep state, the process transitions from the high automation level to the low automation level without transitioning to the sub-high automation level.

[0242] According to an eighth aspect, the automation level determination process transitions from the high automation level to the medium automation level without transitioning to the low automation level if the duration is less than a predetermined time.

[0243] According to a ninth aspect, the driving control method and the processing executed by the driving control device further include a display command sending processing for sending display command information to an HMI control device (25) that controls the HMI device (20) so as to cause the HMI device to execute a transition-related display related to the transition of the driving automation level.

[0244] According to the tenth aspect, the display command sending process is a process of sending the display command information that causes the HMI device to display an approval request display that prompts the driver to perform an input operation to approve the execution of the medium automation level when the driving operation state corresponding to the low automation level becomes stable.

[0245] According to the eleventh aspect, the display command sending process is a process of setting the approval request timing for displaying the approval request display on the HMI device, or the approval permission timing for permitting the input operation to approve the execution of the medium automation level, depending on the duration of the high automation level or whether the driver's state while executing the high automation level is asleep or not.

[0246] According to the twelfth aspect, the display command transmission process is a process of transmitting the display command information that causes the HMI device to display a main action instruction display that instructs a main action including an action to adjust driving posture as a driver state, which is the state of the driver, and a secondary action instruction display that instructs a secondary action including an action to adjust driving awareness as the driver state.

[0247] According to the thirteenth aspect, the display command transmission process is a process for setting the operation instruction timing for displaying the main operation instruction display on the HMI device depending on the duration of the high automation level or whether the driver's state while executing the high automation level is asleep or not.

[0248] According to a fourteenth aspect, the operation control method and the processing executed by the operation control device include: a driver status acquisition process for acquiring the driver status; a vehicle control process that performs motion control by steering or acceleration / deceleration of the vehicle based on the driving automation level determined by the automation level determination process; and If the primary action is not performed, the vehicle control process executes a predetermined safety stop control, If the primary operation is executed but the secondary operation is not executed, the vehicle control process does not execute the safety stop control, while the display command transmission process transmits the display command information to display a warning display on the HMI device.

[0249] According to the fifteenth aspect, the display command transmission process is a process of transmitting the display command information for causing a visual guidance display to be displayed on a display device (23) that displays the second task screen, which displays a second task that the driver can perform while the automatic driving is being performed, to guide the driver's gaze to a front display device (21, 22) that is positioned on the front side of the vehicle.

[0250] According to a sixteenth aspect, the front display device is a head-up display device (22). [Explanation of symbols]

[0251] 10 In-Vehicle Systems 18 Operation control device 20 HMI device 21 Meter panel 22 Head-up display device 25 HMI control devices 253 Automation Level Acquisition Department 254 Driver status acquisition unit 255 Display control unit 256 Operation reception section

Claims

1. An HMI control device (25) configured to control an HMI device (20) that displays an image visible to a driver of an automatically drivable vehicle (1), a driving level acquisition unit (253) that acquires a determination result of a driving level corresponding to an implementation state including whether or not driving automation is implemented in a driving control device (18) that controls driving of the vehicle; a display control unit (255) that controls an image display operation in the HMI device according to the driving level acquired by the driving level acquisition unit; Equipped with The display control unit causes the HMI device to display an operation instruction display that causes the driver to terminate a high level, which is the driving level included in the autonomous driving, and take a low-level corresponding state, which is a driver state of the driver, in which the driving level is lower than the high level, and performs an operation instruction display that causes the driver to take a medium-level corresponding state, which is a driving level between the high level and the low level, in which the driving level is higher. HMI control device.

2. An HMI control program executed by an HMI control device (25) configured to control an HMI device (20) that displays an image visible to a driver of an automatically drivable vehicle (1), comprising: The process executed by the HMI control device is a driving level acquisition process for acquiring a determination result of a driving level corresponding to an implementation state including whether or not driving automation is implemented in a driving control device (18) that controls the driving of the vehicle; a display control process for controlling an image display operation in the HMI device according to the driving level acquired in the driving level acquisition process; Including, The display control process displays, on the HMI device, an operation instruction display that causes the driver to terminate a high level, which is the driving level included in the autonomous driving, and take a low-level corresponding state, which is a driver state of the driver, in which the driving level is lower than the high level, and displays an operation instruction display that causes the driver to take a medium-level corresponding state, which is a driving level between the high level and the low level, in which the driving level is higher. HMI control program.

Images