HMI control device and HMI control program

Abstract

To provide a technique which enables a second task to be utilized further comfortably during automatic driving.SOLUTION: An HMI control device (23) configured to control an HMI device (20) mounted on an automatically drivable vehicle comprises a second task presentation section (234) for presenting a second task that an occupant on a driver seat can execute during automatic driving, using the HMI device. Such a second task presentation section presents a plurality of second tasks with different execution required time selectably.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an HMI control device and an HMI control program for controlling an HMI device mounted on an autonomously driven vehicle. HMI stands for Human Machine Interface. [Background technology]

[0002] Various automated driving systems for vehicles such as automobiles have been proposed (see, for example, Patent Document 1). During automated driving, the driver, who is an occupant in the driver's seat of the vehicle, is able to freely perform a second task. A second task is a task other than driving that is performed by the driver. Specifically, second tasks include, for example, operating a mobile communication terminal, watching video content, etc. A second task is also called a "non-driving task" or a "secondary activity." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2017-107502 Summary of the Invention [Problem to be solved by the invention]

[0004] The time required to execute a second task varies. For example, when operating a mobile communication terminal such as a mobile phone, the time required to execute the second task is relatively short. In contrast, when watching a movie, the time required to execute the second task may be long. Therefore, for example, if a second task that requires a long execution time is interrupted midway through execution, or if selection operations for second tasks that require a short execution time are frequently required, convenience may be impaired.

[0005] The present invention has been made in consideration of the above-mentioned circumstances, etc. That is, the present invention provides a technique that enables, for example, a second task to be used more comfortably during autonomous driving. [Means for solving the problem]

[0006] The HMI control device (23) is configured to control the HMI device (20) mounted on an autonomously driven vehicle. The HMI control device according to claim 1 comprises: a second task presentation unit (234) that presents a second task that can be performed by a driver's seat occupant during automatic driving by the HMI device; When the second task is being executed and the duration of the second task from a predetermined continuation reference point is equal to or longer than a threshold time, Information about the presence of other vehicles in the vicinity is configured to be presented by the HMI device. The HMI control program is a program executed by an HMI control device (23) configured to control an HMI device (20) mounted on an autonomously driven vehicle. In the HMI control program according to claim 2, the processing executed by the HMI control device comprises: A process of presenting a second task that can be performed by a driver's seat occupant during autonomous driving by the HMI device; When the second task is being executed and the duration of the second task from a predetermined continuation reference point is equal to or longer than a threshold time, Information about the presence of other vehicles in the vicinity a process of presenting the above by the HMI device; Includes.

[0007] In addition, in each section of the application documents, each element may be assigned a reference symbol in parentheses. In this case, the reference symbol merely indicates an example of the correspondence between the element and the specific configuration described in the embodiment described below. Therefore, the present invention is not limited in any way by the description of the reference symbol. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a schematic configuration of an in-vehicle system including a driving control ECU and an HMI control device according to an embodiment. [Figure 2] 2 is a schematic diagram showing an example of a display mode in the HMI device shown in FIG. 1. FIG. [Figure 3] 1. FIG. 4 is a schematic diagram showing another example of a display mode in the HMI device shown in FIG. [Figure 4] 1. FIG. 4 is a schematic diagram showing yet another example of a display mode in the HMI device shown in FIG. [Figure 5] 1. FIG. 4 is a schematic diagram showing yet another example of a display mode in the HMI device shown in FIG. [Figure 6] 2 is a flowchart showing an example of the operation of the in-vehicle system shown in FIG. [Figure 7] 2 is a flowchart showing an example of the operation of the operation control ECU shown in FIG. 1. [Figure 8] 2 is a flowchart showing an example of the operation of the HMI control device shown in FIG. 1. [Figure 9] 2 is a flowchart showing an example of the operation of the operation control ECU shown in FIG. 1. [Figure 10] 1. FIG. 4 is a schematic diagram showing yet another example of a display mode in the HMI device shown in FIG. [Figure 11] 10 is a flowchart showing another example of the operation of the HMI control device shown in FIG. [Figure 12] 10 is a flowchart showing yet another example of the operation of the HMI control device shown in FIG. [Figure 13] 10 is a flowchart showing another example of the operation of the in-vehicle system shown in FIG. [Figure 14] 10 is a flowchart showing another example of the operation of the operation control ECU shown in FIG. [Figure 15] 10 is a flowchart showing yet another example of the operation of the HMI control device shown in FIG. DETAILED DESCRIPTION OF THE INVENTION

[0009] (Embodiment) Hereinafter, embodiments of the present invention will be described with reference to the drawings. It should be noted that various modifications applicable to one embodiment may be hindered from being understood if they are introduced in the middle of a series of explanations relating to the embodiment. Therefore, the modifications will be explained together after the series of explanations relating to the embodiment, rather than in the middle of the series of explanations relating to the embodiment.

[0010] (composition) Referring to FIG. 1, the in-vehicle system 10 is mounted on a vehicle capable of autonomous driving. "Autonomous driving" refers to a driving automation level corresponding to levels 3 to 5 in the "SAE J3016" standard published by SAE International, in which the in-vehicle system 10 is responsible for all dynamic driving tasks. SAE stands for Society of Automotive Engineers. The definition of "dynamic driving task" will be described later. Level X in "SAE J3016" is hereinafter referred to as "SAE Level X." X is any one of 0 to 5. In this embodiment, the in-vehicle system 10 is mounted on an automobile as a vehicle, and is configured to be able to set the driving automation level of the vehicle to any one of SAE Levels 0 to 3. A vehicle equipped with the in-vehicle system 10 may hereinafter be referred to as the "host vehicle."

[0011] (Overall system configuration) The in-vehicle system 10 is an in-vehicle network including an in-vehicle communication line 10A and a plurality of nodes interconnected via the in-vehicle communication line 10A, and is configured to be able to perform various vehicle controls and associated display operations while the vehicle is being driven. The in-vehicle system 10 is configured to comply with a predetermined communication standard such as CAN (international registered trademark: international registration number 1048262A). CAN (international registered trademark) is an abbreviation for Controller Area Network.

[0012] The in-vehicle system 10 includes a vehicle state sensor 11, an external state sensor 12, a surroundings monitoring sensor 13, a locator 14, a DCM 15, a navigation device 16, a driving control ECU 17, a DSM 18, an operation unit 19, and an HMI device 20. DCM stands for Data Communication Module. ECU stands for Electronic Control Unit. DSM stands for Driver Status Monitor. The vehicle state sensor 11 to the HMI device 20 are connected to each other via an in-vehicle communication line 10A.

[0013] The HMI device 20 includes a meter panel 21, a CID device 22, and an HMI control device 23. CID stands for Center Information Display. The meter panel 21 and the CID device 22 are connected to the HMI control device 23 via a sub-communication line different from the in-vehicle communication line 10A so as to be able to communicate information with the HMI control device 23. The HMI control device 23 is provided as a node connected to the in-vehicle communication line 10A.

[0014] (various sensors) The vehicle state sensor 11 is configured to generate outputs corresponding to various quantities related to the driving state of the vehicle. The "various quantities related to the driving state" include, for example, various quantities related to the driving operation state by the driver, such as accelerator operation amount, brake operation amount, shift position, and steering angle. The "driver" refers to a vehicle occupant who is in charge of or performs a dynamic driving task, typically a vehicle occupant in the driver's seat of the vehicle, and may also be referred to as a "driver's seat occupant." The "various quantities related to the driving state" include physical quantities related to the vehicle's behavior, such as driving speed, angular velocity, longitudinal acceleration, and lateral acceleration. For the sake of simplicity and explanation, the vehicle state sensor 11 is a collective term for well-known sensors required for driving control, such as an accelerator position sensor, steering angle sensor, wheel speed sensor, angular velocity sensor, and acceleration sensor. The vehicle state sensor 11 is configured to provide detection outputs to various components, such as the driving control ECU 17, via the in-vehicle communication line 10A.

[0015] The external environment sensor 12 is configured to generate outputs corresponding to various quantities related to the natural environment around the vehicle. The "various quantities related to the natural environment" include physical quantities such as outside temperature, rainfall, and illuminance. That is, the external environment sensor 12 is a general term for well-known sensors such as an outside temperature sensor, a raindrop sensor, and an illuminance sensor for the sake of simplicity of illustration and explanation. The external environment sensor 12 is configured to provide detection outputs to various components such as the driving control ECU 17 via the in-vehicle communication line 10A.

[0016] The perimeter monitoring sensor 13 is configured to detect mainly the traffic environment around the vehicle other than those detectable by the external state sensor 12. Specifically, the perimeter monitoring sensor 13 is configured to detect moving and stationary objects within a predetermined detection range around the vehicle. "Moving objects" include pedestrians, cyclists, animals, and other moving vehicles. "Stationary objects" include fallen objects on the road, guardrails, curbs, parked vehicles, road signs, road markings, as well as roadside structures (e.g., medians, buildings, etc.). The perimeter monitoring sensor 13 may also be referred to as an "ADAS sensor." ADAS stands for Advanced Driver-Assistance Systems. In this embodiment, the perimeter monitoring sensor 13 has a front camera 131 and a radar sensor 132 as components for detecting moving and stationary objects.

[0017] The front camera 131 is provided to capture images of the front and sides of the vehicle. In this embodiment, the front camera 131 is a digital camera device and includes an image sensor such as a CCD or CMOS. CCD stands for Charge Coupled Device. CMOS stands for Complementary MOS.

[0018] The radar sensor 132 is a millimeter wave radar sensor, a submillimeter wave radar sensor, or a laser radar sensor configured to transmit and receive radar waves, and is mounted on the front of the vehicle body. The radar sensor 132 is configured to output signals corresponding to the position and relative speed of a reflection point. A "reflection point" is a point on the surface of an object around the vehicle that is estimated to have reflected radar waves. The "relative speed" is the speed of the reflection point, i.e., the object that reflected the radar waves, relative to the vehicle.

[0019] (locator) Locator 14 is configured to determine highly accurate position information of the vehicle by so-called composite positioning. Specifically, locator 14 has a GNSS receiver 141, an inertial acquisition unit 142, a high-accuracy map DB 143, and a locator ECU 144. GNSS stands for Global Navigation Satellite System. DB stands for database. "High-accuracy position information" refers to position information having a position accuracy of SAE Level 2 or higher that can be used for advanced driving assistance or automated driving, specifically, with an error of less than 10 cm.

[0020] The GNSS receiver 141 is configured to receive positioning signals transmitted from a plurality of positioning satellites, i.e., artificial satellites. In this embodiment, the GNSS receiver 141 is configured to be able to receive positioning signals from positioning satellites in at least one of satellite positioning systems such as GPS, QZSS, GLONASS, Galileo, IRNSS, and the Beidou satellite navigation system. GPS is an abbreviation for Global Positioning System. QZSS is an abbreviation for Quasi-Zenith Satellite System. GLONASS is an abbreviation for Global Navigation Satellite System. IRNSS is an abbreviation for Indian Regional Navigation Satellite System.

[0021] The inertia acquisition unit 142 is configured to acquire the acceleration and angular velocity acting on the host vehicle. In this embodiment, the inertia acquisition unit 142 is provided as a three-axis gyro sensor and a three-axis acceleration sensor built into a box-shaped housing of the locator 14.

[0022] The high-precision map DB 143 is mainly composed of non-volatile rewritable memory so that it can rewritably store high-precision map information and retain the stored contents even when the power is turned off. Examples of non-volatile rewritable memory include a hard disk, EEPROM, flash ROM, etc. EEPROM stands for Electronically Erasable and Programmable ROM. ROM stands for Read Only Memory. High-precision map information may also be referred to as high-precision map data. High-precision map information contains map information with higher accuracy than map information used in conventional car navigation systems, which can handle position errors of approximately several meters. Specifically, the high-precision map DB 143 stores information usable for advanced driving assistance or automated driving, such as three-dimensional road shape information, lane number information, and traffic regulation information, in accordance with predetermined standards such as the ADASIS standard. ADASIS stands for Advanced Driver Assistance Systems Interface Specification.

[0023] The locator ECU 144 is configured as a so-called in-vehicle microcomputer equipped with a CPU, ROM, RAM, input / output interface, etc. (not shown). CPU stands for Central Processing Unit. RAM stands for Random Access Memory. The locator ECU 144 is configured to sequentially determine the position, direction, etc. of the vehicle based on the positioning signal received by the GNSS receiver 141, the acceleration and angular velocity acquired by the inertia acquisition unit 142, the traveling speed acquired from the vehicle state sensor 11, etc. The locator 14 is configured to be able to provide the results of the determination of the position, direction, etc. by the locator ECU 144 to each component, such as the navigation device 16, the driving control ECU 17, and the HMI control device 23, via the in-vehicle communication line 10A.

[0024] (DCM) The DCM 15 is an in-vehicle communication module that is capable of communicating information with base stations around the vehicle via wireless communication compliant with a communication standard such as LTE or 5G. LTE stands for Long Term Evolution. 5G stands for 5th Generation. Specifically, for example, the DCM 15 is configured to acquire the latest high-precision map information from a probe server on the cloud. The DCM 15 also stores the acquired latest high-precision map information in the high-precision map DB 143 in cooperation with the locator ECU 144. The DCM 15 is also configured to acquire traffic information such as congestion information from the probe server and / or a predetermined database. The traffic information is also referred to as "road traffic information."

[0025] (Navigation device) Navigation device 16 is configured to calculate a planned driving route from the current position of the vehicle to a predetermined destination. In this embodiment, navigation device 16 is configured to calculate a planned driving route based on a destination set by the driver or the like of the vehicle, high-precision map information acquired from locator 14, and vehicle position information and direction information acquired from locator 14. Navigation device 16 is also configured to provide various information, including route information resulting from the calculation, to various components such as driving control ECU 17 and HMI control device 23 via in-vehicle communication line 10A. That is, navigation device 16 is configured to display a navigation screen on HMI device 20 for displaying a map, a route, and the like.

[0026] (Drive control ECU) The driving control ECU 17 is configured to control the driving of the vehicle based on signals and information acquired from the vehicle state sensor 11, the external environment state sensor 12, the surroundings monitoring sensor 13, the locator 14, etc. In other words, the driving control ECU 17 is configured to execute predetermined driving control operations. In this embodiment, the "predetermined driving control operations" are vehicle control operations, i.e., dynamic driving task (DDT) execution operations, corresponding to SAE Levels 1 to 3. The definition of the "DDT" conforms to "SAE J3016." In other words, the "DDT" refers to all operational and tactical functions that must be performed in real time when operating a vehicle in road traffic, excluding strategic functions. The "strategic functions" include route planning, route selection, etc.

[0027] The driving control ECU 17 as a driving control device of the present invention is configured to control driving of a vehicle capable of SAE Level 3 autonomous driving while traveling in a specific section and / or during traffic congestion. A "specific section" is a driving section that has been set in advance to be capable of SAE Level 3 autonomous driving. In the specific section, driving at a predetermined high speed range is possible during autonomous driving. The predetermined high speed range is, for example, 60 km / h or more and less than the legal speed limit. Such SAE Level 3 autonomous driving, which is capable of driving at high speeds in a specific section, is hereinafter referred to as "high-speed range autonomous driving." On the other hand, SAE Level 3 autonomous driving performed during traffic congestion is hereinafter referred to as "traffic congestion autonomous driving." With SAE Level 3 autonomous driving, the driver is not required to monitor the surroundings of the vehicle.

[0028] The driving control ECU 17 has a configuration as a so-called in-vehicle microcomputer, including a CPU, ROM, non-volatile rewritable memory, RAM, input / output interface, etc. (not shown). Specifically, the driving control ECU 17 has an information acquisition unit 171, a driving level determination unit 172, and a vehicle speed control unit 173 as functional configurations or functional units realized on the in-vehicle microcomputer.

[0029] The information acquisition unit 171 is configured to acquire at least the running state of the host vehicle. The "running state" includes the driving state, natural environment, traffic environment, etc., which are detected or acquired by the vehicle state sensor 11, the external environment sensor 12, the periphery monitoring sensor 13, etc. The information acquisition unit 171 is also configured to acquire the current position of the host vehicle and traffic information for the road on which the host vehicle is currently traveling. That is, the information acquisition unit 171 is configured to acquire information necessary for vehicle control corresponding to SAE Levels 1 to 3 from the vehicle state sensor 11, the external environment sensor 12, the periphery monitoring sensor 13, the locator 14, the DCM 15, etc.

[0030] The driving level determination unit 172 is configured to determine whether or not to execute autonomous driving based on the driving conditions and the like acquired by the information acquisition unit 171. That is, the driving level determination unit 172 is configured to set the driving automation level of the host vehicle to one of SAE levels 0 to 3. Specifically, the driving level determination unit 172 determines whether or not an autonomous driving start condition is met, and starts autonomous driving at SAE level 3 when an approval operation from the driver is received while such condition is met. Furthermore, the driving level determination unit 172 is configured to execute control necessary to terminate autonomous driving when an autonomous driving continuation condition is not met while SAE level 3 autonomous driving is being executed. The driving control ECU 17 is configured to be able to provide the driving automation level setting result by the driving level determination unit 172 to each unit, such as the HMI control device 23, via the in-vehicle communication line 10A.

[0031] The vehicle speed control unit 173 is configured to control the driving speed of the host vehicle depending on whether autonomous driving is executable, as determined by the driving level determination unit 172. Specifically, when the driving automation level is set to one of SAE levels 1 to 3, the vehicle speed control unit 173 executes driving speed control according to the set driving automation level. The driving control ECU 17 is also provided with a steering control unit, a braking control unit, etc. (not shown) that execute steering control, braking control, etc. according to the set driving automation level. In other words, the driving control ECU 17 is configured as a vehicle control unit that executes vehicle motion control subtasks, such as vehicle speed control, steering control, and braking control, according to the driving automation level determined by the driving level determination unit 172.

[0032] (DSM) The DSM 18 is configured to detect the driver's condition by image recognition based on images captured by an in-vehicle camera equipped with an image sensor such as a CCD or CMOS. The DSM 18 is also configured to issue a warning about inattentive driving or the like using a speaker (not shown) or the like provided on the dashboard or the like based on the detection result of the driver's condition. The DSM 18 is also configured to be able to provide the detection result of the driver's condition to each part, such as the driving control ECU 17 and the HMI control device 23, via the in-vehicle communication line 10A.

[0033] (Operation unit) The operation unit 19 is configured to accept manual input operations from the driver other than motion control inputs for the vehicle. "Movement control inputs" are control inputs directly related to lateral or longitudinal motion control when the vehicle is being manually driven, and specifically include, for example, shift operation, accelerator operation, brake operation, steering operation, etc. Note that the operation of the turn signal switch 191 in the lane change function linked to the operation of the turn signal switch 191, which is included in SAE Level 2, is different from the above-mentioned "motion control input."

[0034] Specifically, the operation unit 19 has a turn signal switch 191. The turn signal switch 191 is configured to output a signal corresponding to the operation state of a turn signal lever (not shown), which is an operating lever provided on the steering column. The operation unit 19 also has a steering switch (not shown) provided on a spoke of the steering wheel. The operation unit 19 also has a seat position adjustment switch, a seating posture adjustment switch, a window opening / closing switch, etc., which are provided corresponding to each passenger seat including the driver's seat. The operation unit 19 is configured to be able to provide the results of the input operation received by the driver to each component, such as the driving control ECU 17 and the HMI control device 23, via the in-vehicle communication line 10A.

[0035] (HMI device) The HMI device 20 is configured to present various pieces of information related to the vehicle to the driver at least visually, and to accept input operations by the driver corresponding to the presented information. In this embodiment, the HMI device 20 mounted on the vehicle capable of autonomous driving is configured to be able to present various pieces of information related to autonomous driving and to accept input operations by the driver. "Information presentation" includes, for example, various types of guidance, input operation instructions, input operation content notifications, warnings, etc.

[0036] As described above, the HMI device 20 includes the meter panel 21 and the CID device 22, which are provided on a dashboard (not shown). That is, in this embodiment, the HMI device 20 is configured as a so-called "dashboard HMI" that is attached to a dashboard (not shown). The HMI device 20 also includes a speaker (not shown) for presenting information by voice.

[0037] The meter panel 21 has a meter 211, a meter display 212, and a meter switch 213. The meter 211 is provided to display the vehicle's traveling speed, coolant temperature, remaining fuel, etc. The meter display 212 is an information display unit provided in the center of the meter panel 21, and is provided to display various information such as the date and time, outside temperature, mileage, radio station, etc. The meter switch 213 is provided to be able to accept various operations related to the display state or display content of the meter 211 and / or meter display 212, such as an operation to reset the trip meter.

[0038] The CID device 22 is mounted on the dashboard. The CID device 22 is configured to be able to display a navigation display screen for displaying maps, routes, and the like by the navigation device 16. The CID device 22 is also configured to be able to display information and content different from that displayed on the navigation display screen. Specifically, the CID device 22 is configured to be able to execute displays related to driving modes such as "Comfort," "Normal," "Sport," and "Circuit." The CID device 22 is also configured to execute displays related to a second task that the driver can execute during SAE Level 3 autonomous driving.

[0039] The CID device 22 has a CID display 221, an input device 222, and a CID switch 223. The CID display 221 is provided in approximately the center of the dashboard in the vehicle width direction so as to be visible at least to the driver. The CID display 221 is configured as a display device that is a liquid crystal display or an organic EL display. EL stands for electroluminescence.

[0040] The input device 222 is a transparent touch panel that is provided to cover the CID display 221. That is, the input device 222 is configured to accept input operations by the driver or the like while allowing the driver or the like to visually recognize the display on the CID display 221. The CID switch 223 has a plurality of manually operated switches that are arranged around the CID display 221 and the input device 222.

[0041] 2 to 5 show examples of display screens related to a second task during autonomous driving on the CID display 221. FIGS. 2 and 3 show a case where a movie is being watched, which is an example of long-duration content. FIG. 4 shows a case where a mobile communication terminal operation, which is an example of short-duration content, is permitted to be executed. FIG. 5 shows a menu screen presenting a selection of multiple contents with different execution times.

[0042] "Long-duration content" is registered content that takes a relatively long time to execute (for example, 30 minutes or more). "Registered content" is content that can be used as a second task, and that has been registered or made available in advance by the manufacturer or seller of the vehicle or the user of the vehicle, including the driver. "Short-duration content" is registered content that takes a shorter time to execute than long-duration content (for example, less than 30 minutes). "Content" includes, for example, video content, text content, etc. "Video content" includes, for example, movies, concert footage, music videos, television broadcasts, games, videophones, etc. "Text content" includes, for example, books, magazines, text broadcasts, etc.

[0043] 2 to 5, the display screen of CID display 221 has a first display area DA and a second display area DB. The first display area DA is an area where video content and text content are mainly displayed on a relatively large screen, and may also be referred to as the "main display area."

[0044] The second display area DB is provided above the first display area DA in the vertical direction of the screen. The "vertical direction of the screen" refers to the vertical direction based on the driver or the like facing the CID display 221, and is the vertical direction in Figures 2 to 5. The second display area DB is a display area provided at the top of the display screen of the CID display 221, and is capable of displaying various text information such as traffic information. The second display area DB may also be referred to as an "information display bar." The second display area DB may be provided anywhere within the display screen of the CID display 221. The second display area DB may also be provided on a display device adjacent to the CID display 221.

[0045] 2 and 3, the first display area DA has a central display area DA1, a left margin area DA2, and a right margin area DA3. The central display area DA1 is an area for displaying video content and text content, and is provided between the left margin area DA2 and the right margin area DA3 in the left-right direction of the screen. The "left-right direction of the screen" refers to the left-right direction based on the driver or the like facing the CID display 221, and is the left-right direction in FIGS. 2 to 5.

[0046] The left margin area DA2 is provided to the left of the central display area DA1 in the left-right direction of the screen. An arrow-shaped type change button DA4 is provided in the left margin area DA2. The type change button DA4 is an operation button operated to change the type of content among movies, music videos, television broadcasts, games, books, etc., and is located approximately in the center of the left margin area DA2.

[0047] The right margin area DA3 is provided to the right of the central display area DA1 in the left-right direction of the screen. A type change button DA4 is also provided in the right margin area DA3. The type change button DA4 is located approximately in the center of the right margin area DA3. The type change button DA4 in the right margin area DA3 is provided so that it can be operated to change the type of content in the reverse order of the type change button DA4 in the left margin area DA2.

[0048] The right margin area DA3 also has a content change button DA5. The content change button DA5 is an operation button that is operated to change content within the same time category, and is located at the bottom edge of the right margin area DA3 in the vertical direction of the screen. The "time category" indicates whether the content is short-term content or long-term content.

[0049] As shown in Fig. 3, the right margin area DA3 is configured to display an approval button DA6 as needed. The approval button DA6 is an operation button operated to accept various approval operations by the driver, and is arranged at the top end of the right margin area DA3 in the vertical direction of the screen. Note that the approval button DA6 is configured to display text such as "OK," "START," or "CONTINUE" inside the rectangular button shape depending on the content of the approval operation.

[0050] The second display area DB has a status display area DB1 and an information display area DB2. The status display area DB1 is located at the left end of the screen in the left-right direction so as to display information corresponding to the current driving automation level of the vehicle. The information displayed in the status display area DB1 is, for example, "driving in a dedicated section" in the case of high-speed automated driving, and "driving in a traffic jam" in the case of automated driving in a traffic jam.

[0051] The information display area DB2 is located to the right of the status display area DB1 in the left-right direction of the screen. The information display area DB2 displays various information and messages related to traffic information and driving status in text as needed. Specifically, the display in the information display area DB2 includes an information display box DB3 and a message display box DB4. The information display box DB3 displays traffic information such as congestion information. The message display box DB4 displays various messages or warnings other than traffic information.

[0052] Referring to FIG. 4, when the second task is to permit operation of a mobile communication terminal such as a mobile phone, the first display area DA displays a mobile device permission screen DA7. The mobile device permission screen DA7 is configured to display text and / or images indicating that the driver is permitted to operate the mobile communication terminal. Note that, even in the display screen shown in FIG. 4, the second display area DB has the same display function as above. Furthermore, when the input device 222 detects that any location on the mobile device permission screen DA7 has been touched, the CID display 221 displays the menu screen shown in FIG. 5.

[0053] 5, on the menu screen, the first display area DA displays a plurality of content selection buttons DA8 and a plurality of type selection buttons DA9. The content selection buttons DA8 present frequently used individual content in each content type, such as movies, concert videos, television broadcasts, games, etc., in thumbnail display. The type selection buttons DA9 are provided corresponding to each content type, such as movies, concert videos, television broadcasts, games, etc. Note that the second display area DB on the menu screen shown in FIG. 5 also has the same display function as above.

[0054] (HMI control device) 1 again, the HMI control device 23 is configured to control the operation of the HMI device 20. That is, the HMI control device 23 is configured as an HCU that controls the operation of the meter panel 21, the CID device 22, and the like included in the HMI device 20. HCU is an abbreviation for HMI Control Unit.

[0055] The HMI control device 23 has a configuration as a so-called in-vehicle microcomputer, including a CPU, ROM, nonvolatile rewritable memory, RAM, input / output interface, etc. (not shown). The HMI control device 23 has, as functional configurations or functional units realized on the microcomputer, a vehicle information acquisition unit 231, a behavior acquisition unit 232, a time acquisition unit 233, a second task presentation unit 234, a continuous operation acceptance unit 235, and a display control unit 236.

[0056] The vehicle information acquisition unit 231 is configured to acquire the driving state of the host vehicle detected or acquired by the vehicle state sensor 11, etc. The vehicle information acquisition unit 231 is also configured to acquire the current position of the host vehicle, the planned driving route, and traffic information on the planned driving route including the road on which the host vehicle is currently traveling. Furthermore, the vehicle information acquisition unit 231 is configured to acquire the result of the driving automation level determination by the driving level determination unit 172 from the driving control ECU 17.

[0057] The behavior acquisition unit 232 is configured to acquire the detection result of the driver's behavior. That is, the behavior acquisition unit 232 is configured to acquire the behavior detection result by the DSM 18 from the DSM 18 via the in-vehicle communication line 10A. Note that the behavior acquisition unit 232 may also acquire the behavior detection result by operating the input device 222.

[0058] The time acquisition unit 233 is configured to acquire a possible continuous time for autonomous driving based on the content acquired by the vehicle information acquisition unit 231. Specifically, when traveling through a specific section that is a traveling section that has been set in advance to be capable of autonomous driving, the time acquisition unit 233 acquires a possible continuous time based on the distance of the specific section. On the other hand, when autonomous driving is initiated due to congestion, the time acquisition unit 233 acquires a possible continuous time based on a predicted duration of the congestion.

[0059] The second task presentation unit 234 is configured to present, via the HMI device 20, a second task that can be executed by the driver during autonomous driving, based on the allowable continuation time acquired by the time acquisition unit 233. That is, the second task presentation unit 234 displays one of a long-time content presentation screen, a short-time content presentation screen, and a menu screen on the CID display 221, depending on the acquisition result of the allowable continuation time. The "long-time content presentation screen" and the "short-time content presentation screen" are display screens corresponding to FIG. 2 or FIG. 3. That is, the long-time content presentation screen is a display screen that presents or displays, in the central display area DA1, one of the long-time contents that can be completed within the allowable continuation time. On the other hand, the short-time content presentation screen is a display screen that presents or displays, in the central display area DA1, one of the short-time contents that can be completed within the allowable continuation time.

[0060] Specifically, in this embodiment, when the acquired possible continuation time is less than a predetermined threshold, the second task presentation unit 234 is configured to display a short-time content presentation screen on the CID display 221. On the other hand, when the acquired possible continuation time is equal to or greater than the predetermined threshold, the second task presentation unit 234 is configured to display a long-time content presentation screen on the CID display 221. Furthermore, if the time acquisition unit 233 is unable to acquire the possible continuation time for some reason, the second task presentation unit 234 is configured to display a menu screen on the CID display 221, thereby presenting a plurality of second tasks with different required execution times in a selectable manner.

[0061] The continuation operation receiving unit 235 is configured to receive a continuation operation, which is an input operation for continuing the second task being executed, when a temporary resolution of the traffic jam is detected. "When a temporary resolution of the traffic jam is detected" refers to, for example, when the current position of the host vehicle is within a congested section based on the acquired traffic jam information, and the periphery monitoring sensor 13 detects an acceleration-enabled section ahead of the host vehicle that is a predetermined distance or more. Alternatively, "when a temporary resolution of the traffic jam is detected" refers to, for example, when the current position of the host vehicle is within a congested section based on the acquired traffic jam information, and when it is detected that the host vehicle's speed is equal to or greater than a predetermined resolution lower limit vehicle speed (e.g., 10 km / h) and less than a predetermined resolution upper limit vehicle speed (e.g., 60 km / h). Note that when the host vehicle's speed is equal to or greater than the resolution upper limit vehicle speed, the traffic jam is detected to have been completely resolved, and the driving automation level becomes such that the second task cannot be performed.

[0062] Specifically, when a temporary relief of congestion is detected, the continued operation accepting unit 235 displays traffic information indicating the temporary relief of congestion in the information display box DB3 and also displays the approval button DA6. The continued operation accepting unit 235 also detects whether or not a continued operation, i.e., a touch input operation on the input device 222, has been performed at a position corresponding to the display position of the approval button DA6.

[0063] Furthermore, in this embodiment, when a temporary relief of traffic congestion is detected, the continuous operation acceptance unit 235 displays a message "Watch out ahead" in the message display box DB4 to prompt the driver to check the road conditions ahead, i.e., ahead of the vehicle. The continuous operation acceptance unit 235 is configured to enable the continuous operation when the behavior acquisition unit 232 acquires a detection result indicating that the driver is checking the road conditions ahead of the vehicle. Furthermore, when the continuous operation acceptance unit 235 has enabled acceptance of the continuous operation, the continuous operation acceptance unit 235 notifies the driving control ECU 17 that the continuous operation has been enabled, so that the driving control ECU 17 can control the traveling speed of the vehicle so that it does not exceed a predetermined speed range.

[0064] The display control unit 236 is configured to execute display control other than the display control of the second task presentation unit 234 and the continuous operation reception unit 235. Specifically, for example, when a change occurs in traffic information such as congestion information, the display control unit 236 displays the latest information in the information display box DB3. Furthermore, the display control unit 236 executes various display controls by the CID device 22 for driving automation levels of SAE Levels 0 to 2, where second tasks cannot be executed.

[0065] (Operation Overview: First Embodiment) The following describes the operation of the operation control ECU 17 and the HMI control device 23 according to this embodiment, as well as an overview of the methods and programs executed by them, along with the effects achieved by the configuration of this embodiment.

[0066] The information acquisition unit 171 in the driving control ECU 17 acquires various information including the driving state of the vehicle. Specifically, the information acquisition unit 171 acquires the driving state and traffic environment from the vehicle state sensor 11, the external state sensor 12, and the surroundings monitoring sensor 13. The information acquisition unit 171 also acquires highly accurate position information of the vehicle and information on traffic regulations around the vehicle's position from the locator 14. The information acquisition unit 171 also acquires a planned driving route from the current position of the vehicle to a predetermined destination from the navigation device 16.

[0067] The information acquisition unit 171 also acquires a traffic congestion state as a driving state from the periphery monitoring sensor 13 and the DCM 15. The "traffic congestion state" refers to the state of traffic congestion occurring ahead of the host vehicle within the planned driving route of the host vehicle. That is, the "traffic congestion state" includes, for example, the head position of the traffic congestion, the tail position of the traffic congestion, the length of the traffic congestion, and the like within the planned driving route of the host vehicle. The "traffic congestion state" also includes whether the host vehicle has entered a traffic congestion, whether the traffic congestion has temporarily cleared, and the like.

[0068] The information acquisition unit 171 also acquires the state of the driver from the DSM 18. The information acquisition unit 171 also acquires the operation state of the operation unit 19 by the driver from the operation unit 19. Furthermore, the information acquisition unit 171 acquires the execution state of the second task by the driver from the HMI control device 23.

[0069] The driving level determination unit 172 determines whether or not to execute autonomous driving based on various information such as the driving conditions acquired by the information acquisition unit 171. Specifically, the driving level determination unit 172 determines the start conditions for driving automation levels corresponding to SAE Levels 1 to 3 based on various information such as the information acquired by the information acquisition unit 171. For example, in this embodiment, the start conditions for autonomous driving at SAE Level 3 include driving in a specific section or in a traffic jam, a traffic environment that allows autonomous driving, etc. "A traffic environment that allows autonomous driving" includes, for example, being able to properly check the traffic environment using various sensors or visually, and not having any obstructions such as accidents or road construction within the planned autonomous driving section.

[0070] The driving control ECU 17 performs vehicle speed control, steering control, braking control, and the like depending on whether autonomous driving is possible, as determined by the driving level determination unit 172. For example, the vehicle speed control unit 173 controls the traveling speed of the host vehicle depending on whether autonomous driving is possible, as determined by the driving level determination unit 172. Specifically, during SAE Level 3 autonomous driving, the vehicle speed control unit 173 controls the traveling speed of the host vehicle so as to maintain the traveling speed of the host vehicle within a predetermined speed range and to maintain a predetermined distance from a preceding vehicle according to the traveling speed. Furthermore, during SAE Level 3 autonomous driving, the driving control ECU 17 performs steering control so as to prevent the host vehicle from deviating from the lane in which it is currently traveling.

[0071] In order for the driving level determination unit 172 to set the driving automation level to SAE Level 3, the driver's approval operation using the HMI device 20 is required. Therefore, the HMI device 20 presents various information related to autonomous driving and accepts driver operations corresponding to the presented information.

[0072] Specifically, when the conditions for starting SAE Level 3 autonomous driving are met, the HMI control device 23 displays on the meter panel 21 and the CID display 221 that autonomous driving is possible. In addition, the HMI control device 23 displays on the CID display 221 an execute button for approving the execution of autonomous driving and starting autonomous driving of the vehicle, and a not execute button for not approving the execution of autonomous driving and avoiding the start of autonomous driving of the vehicle.

[0073] The HMI control device 23 then monitors the operation state of the input device 222 to determine whether or not either the execute button or the no-execution button has been selected and operated within a predetermined time. When either the execute button or the no-execution button has been selected and operated within the predetermined time, the HMI control device 23 erases the display of both the execute button and the no-execution button, and notifies, i.e., transmits, the result of acceptance of the operation to the driving control ECU 17. When neither the execute button nor the no-execution button has been selected and operated within the predetermined time, the HMI control device 23 performs the same processing as when the no-execution button has been selected and operated. Note that instead of operating the input device 222, the driver may operate an input button such as a steering switch provided on the steering wheel to select whether or not to execute autonomous driving.

[0074] As described above, SAE Level 3 autonomous driving is initiated by selecting the execute button, etc. Then, the second task can be executed by the driver. The HMI control device 23 displays content available as the second task on the CID display 221 so that the driver can select and operate it. Such content can be registered in advance by a user, including the current driver of the vehicle, for example. In addition, such content can be provided from a server via V2X communication using the DCM 15. V2X stands for Vehicle to X.

[0075] The HMI control device 23 accepts a selection operation by the driver by monitoring the operation state of the input device 222. When the driver selects content presented, i.e., displayed, on the CID display 221, use of the selected content begins.

[0076] The time required to execute a second task varies. For example, operating a mobile communication terminal such as a mobile phone takes a relatively short time to execute. In contrast, watching a movie can take a long time to execute.

[0077] For example, there may be cases where the time available for executing the second task, i.e., the time available for continuing autonomous driving, is relatively short, such as about 30 minutes. In such a case, if the only content presented on CID display 221 is a movie that requires more than an hour to run, the driver, who is the user, may feel inconvenienced because the movie will end midway.

[0078] In contrast, there may be cases where the second task can be executed for a relatively long time, for example, one hour or more. In such a case, if the only content presented on the CID display 221 is a music video that takes about five minutes, the driver, who is the user, may feel inconvenienced because he or she will have to repeatedly select the content.

[0079] On the other hand, the executable time of the second task can often be predicted in advance with a certain degree of accuracy. That is, for example, in the case of high-speed autonomous driving, the distance of a specific section where autonomous driving is possible is known. Therefore, in the case of high-speed autonomous driving, it is possible to roughly estimate the duration of autonomous driving. Furthermore, for example, in the case of autonomous driving in traffic jams, the duration of autonomous driving, i.e., the predicted duration of the traffic jam, can be roughly estimated based on the average traveling speed of the vehicle during the traffic jam and the traffic jam distance obtained from traffic jam information. Alternatively, for example, the predicted duration of the traffic jam can be directly obtained, i.e., read from traffic jam information such as "traffic jam between ABC interchange and XYZ interchange for ** minutes." The executable time of the second task can then be predicted based on the approximate value of the duration of autonomous driving.

[0080] Therefore, the HMI control device 23 acquires the possible continuation time of the autonomous driving by the time acquisition unit 233. Then, the HMI control device 23 presents, by the HMI device 20, a second task that the driver can perform during the autonomous driving, based on the possible continuation time acquired by the time acquisition unit 233.

[0081] Specifically, the vehicle information acquisition unit 231 acquires the driving state of the vehicle detected or acquired by the vehicle state sensor 11, etc. The vehicle information acquisition unit 231 also acquires the current position of the vehicle, the planned driving route, and traffic information on the planned driving route including the road on which the vehicle is currently traveling.

[0082] The time acquisition unit 233 acquires the possible continuation time of autonomous driving based on the content acquired by the vehicle information acquisition unit 231. The second task presentation unit 234 presents, via the HMI device 20, a second task that the driver can perform during autonomous driving, based on the possible continuation time acquired by the time acquisition unit 233.

[0083] As described above, the HMI control device 23 according to this embodiment, and the HMI control method and HMI control program executed thereby, have the following advantages. That is, for example, it is possible to present to the driver second task content that matches the executable time of the second task, i.e., the available duration of autonomous driving. In other words, it is possible to present to the driver second task content with an execution time required according to the cause or situation that made autonomous driving possible. Therefore, it is possible to use the second task more comfortably during autonomous driving.

[0084] For example, the vehicle may enter a traffic jam while traveling in a section outside a specific section of a motorway with a minimum speed limit. Alternatively, the vehicle may enter a traffic jam while traveling on a general road without a minimum speed limit. Alternatively, a traffic jam may occur due to an accident, construction work, or the like in a specific section of a motorway with a minimum speed limit. In these cases, the start conditions for high-speed range automated driving, which allows the vehicle to travel at a specified high-speed range, may not be met.

[0085] On the other hand, even in these cases, because the vehicle is in a traffic jam, the start condition for automated driving in a traffic jam, which allows the vehicle to travel at a predetermined medium-to-low speed range, for example, less than 10 km / h, may be met. Therefore, in these cases, if other conditions included in the start condition are met, it becomes possible to perform SAE Level 3 automated driving in a traffic jam and use the second task. Therefore, in this embodiment, the driving level determination unit 172 determines to perform automated driving in a traffic jam when the vehicle is traveling on a motorway with a minimum speed limit set within a predetermined speed range for automated driving in a traffic jam and is in a traffic jam. Note that there is also a driving control ECU 17 that continues automated driving in a high-speed range without switching to automated driving in a traffic jam when the vehicle enters a traffic jam during automated driving in a high-speed range. This driving control ECU 17 simply controls the traveling speed of the vehicle to maintain a safe distance from the vehicle ahead when the vehicle enters a traffic jam during automated driving in a high-speed range. This allows the second task to continue.

[0086] As in the above example, after SAE Level 3 automated driving in a traffic jam is performed and the second task is started, the traffic jam may temporarily clear. In such a case, when the host vehicle reaches a section where the traffic jam is temporarily cleared, the host vehicle is controlled to accelerate. Note that the "speed acceleration control" may also be referred to as "acceleration control on the acceleration side," "acceleration control in the acceleration direction," or simply "acceleration control." After that, when the host vehicle reaches the end of the traffic jam again, the host vehicle is controlled to decelerate.

[0087] In the above-mentioned situation where traffic congestion is temporarily resolved, if the vehicle's speed increases and temporarily exceeds the upper limit for automated driving in traffic congestion, the conditions for continuing automated driving in traffic congestion will no longer be met. This will cause the driving automation level to change from SAE Level 3 to SAE Level 2 in traffic congestion, making the second task unavailable. However, upon subsequent re-entry into the congested section, SAE Level 3 automated driving in traffic congestion and the accompanying use of the second task will become available.

[0088] In this way, when the vehicle's automated driving during congestion is temporarily suspended due to speed increase control when the traffic jam is temporarily resolved, the use of the second task is also temporarily suspended, which is inconvenient. Therefore, when the driving control ECU 17 and the HMI control device 23 detect that the traffic jam has been temporarily resolved, they execute the processing required to continue the automated driving during congestion and the associated use of the second task while the traffic jam is temporarily resolved.

[0089] Specifically, when a temporary relief of the traffic jam is detected, the continuation operation receiving unit 235 in the HMI control device 23 receives a continuation operation, which is an input operation for continuing the second task being executed. The HMI control device 23 transmits the result of receiving the continuation operation by the continuation operation receiving unit 235 to the operation control ECU 17.

[0090] The information acquisition unit 171 in the driving control ECU 17 acquires the traffic congestion state. The information acquisition unit 171 also acquires the execution state of the second task by the driver from the HMI control device 23. The information acquisition unit 171 also acquires the acceptance state of a continuation operation from the HMI control device 23. That is, the information acquisition unit 171 acquires from the HMI control device 23 whether or not there has been a request to continue the second task being executed. Then, when the acquired traffic congestion state is a temporary relief of the traffic congestion and there has been a request to continue the second task being executed, the vehicle speed control unit 173 controls the driving speed to increase within a predetermined speed range, that is, within a speed limit range that is faster than the minimum speed limit of the expressway and allows automatic driving to continue in traffic congestion.

[0091] In this embodiment, when the driver wishes to continue the second task when the traffic congestion is temporarily alleviated, temporary interruptions of the automated driving in a traffic jam and the second task against the driver's wishes can be effectively avoided. Therefore, this embodiment makes it possible to use the second task more comfortably during automated driving.

[0092] Even when a temporary relief from congestion is detected, it is not guaranteed that the automated driving system will continue as is. When automated driving system is released due to a temporary relief from congestion, it is necessary for the system to smoothly transfer control of the vehicle back to the driver. For this reason, it is desirable for the driver to fully understand the traffic conditions ahead of the vehicle, i.e., the road conditions, when requesting continuation of the automated driving system and the second task.

[0093] Therefore, in this embodiment, when a temporary relief of traffic congestion is detected, the continuous operation acceptance unit 235 displays a message saying "Watch out ahead" to prompt the driver to check the road conditions ahead (i.e., the road ahead of the vehicle). Furthermore, when the behavior acquisition unit 232 acquires a detection result indicating that the driver is checking the road conditions ahead of the vehicle, the continuous operation acceptance unit 235 enables the continuous operation. Furthermore, when the acceptance of the continuous operation is enabled, the continuous operation acceptance unit 235 notifies the driving control ECU 17 that the continuous operation has been enabled, in order to cause the driving control ECU 17 to control the traveling speed of the vehicle so that it does not exceed a predetermined speed range.

[0094] When the driving control ECU 17 receives a notification from the continuation operation receiving unit 235 that the continuation operation has been enabled, the driving control ECU 17 executes processing to continue autonomous driving. That is, the driving level determination unit 172 decides to continue autonomous driving when the continuation operation, which is an input operation for continuing the second task being executed, is validly received by the HMI device 20 and a continuation request is notified from the HMI device 20. Specifically, when the detection result of the driver's behavior indicates that the road conditions ahead of the vehicle are being checked, the driving level determination unit 172 determines that the continuation operation has been validly received by the HMI device 20 and decides to continue autonomous driving in traffic jams.

[0095] (Example of operation) The above-described control operations or control methods, and a specific example of a control program corresponding thereto, will be described using the flowcharts shown in Figures 6 to 9. In the illustrated flowcharts, "S" is an abbreviation for "step."

[0096] (Automatic driving control) 6 shows a flow of a series of operations related to automatic driving of the in-vehicle system 10. The series of operations shown in FIG. 6 will be hereinafter referred to as "automatic driving control operations."

[0097] In step 601, the in-vehicle system 10 determines whether the autonomous driving start conditions are met. The processing of step 601 is executed by the driving control ECU 17, for example, as a processing corresponding to the operation of the driving level determination unit 172. Details of the processing content of step 601 will be described later. After the processing of step 601, the in-vehicle system 10 advances the processing to step 602.

[0098] In step 602, the in-vehicle system 10 determines whether autonomous driving can be started. That is, the in-vehicle system 10 determines whether the determination result in step 601 is that the autonomous driving start condition is met or not met.

[0099] If autonomous driving can be started (i.e., step 602 = YES), the in-vehicle system 10 executes the processing of step 603. In step 603, the in-vehicle system 10 outputs an autonomous driving possible message to the HMI device 20. The autonomous driving possible message is a message including the reason why autonomous driving has become possible and the fact that autonomous driving can be started, and is displayed on the CID display 221 and output as audio from a speaker (not shown). The in-vehicle system 10 also displays the above-mentioned execute button and not execute button on the CID display 221.

[0100] On the other hand, if it is not possible to start autonomous driving (i.e., step 602=NO), the in-vehicle system 10 skips all processes from step 603 onwards and terminates the autonomous driving control operation. The process of step 602 is executed by the HMI control device 23, for example, as corresponding to the operation of the vehicle information acquisition unit 231 and the display control unit 236. The process of step 603 is executed by the HMI control device 23, for example, as corresponding to the operation of the display control unit 236.

[0101] After processing step 603, the in-vehicle system 10 proceeds to step 604. In step 604, the in-vehicle system 10 determines whether the driver has approved the start of autonomous driving by operating the execute button to approve the execution of autonomous driving and start autonomous driving of the vehicle. The processing of step 604 corresponds to the operation of the HMI control device 23 and is executed by the HMI control device 23.

[0102] If the driver selects and operates the execute button to approve the start of autonomous driving (i.e., step 604 = YES), the in-vehicle system 10 sequentially executes the processes of steps 605 to 607. On the other hand, if the driver does not approve the start of autonomous driving (i.e., step 604 = NO), the in-vehicle system 10 skips all processes from step 605 onwards and terminates the autonomous driving control operation. Note that even after the autonomous driving control operation has temporarily terminated because the determination result in step 602 or step 604 is "NO," the process from step 601 may be resumed if a predetermined resumption condition is met. Such a resumption condition may be, for example, a predetermined autonomous driving start trigger operation (e.g., operation of a steering switch (not shown) included in the operation unit 19), the passage of a predetermined interval, or detection of the occurrence of traffic congestion in the vehicle's destination. Specifically, for example, the start of autonomous driving may be rejected in step 604 immediately after the vehicle enters a specific section where high-speed autonomous driving is possible. In such cases, it may take a considerable distance or time for the vehicle to exit the specific section. Therefore, in such cases, if the driver changes his / her mind after the non-approval operation or gets caught in traffic, and subsequently desires to perform automated driving, automated driving becomes fully usable.

[0103] In step 605, the in-vehicle system 10 starts autonomous driving. Specifically, for example, if the start condition for high-speed range autonomous driving is met in step 601, the in-vehicle system 10 starts executing high-speed range autonomous driving. On the other hand, if the start condition for traffic jam autonomous driving is met in step 601, the in-vehicle system 10 starts executing traffic jam autonomous driving. The processing of step 605 is executed by the driving control ECU 17, for example, as corresponding to the operation of the driving level determination unit 172.

[0104] In step 606, the in-vehicle system 10 selects second task content to be used during autonomous driving. The processing of step 606 corresponds to the operation of the HMI control device 23 and is executed by the HMI control device 23. The processing content of step 606 will be described in detail later.

[0105] In step 607, the in-vehicle system 10 starts executing, i.e., using, the second task content selected in step 606. The process of step 607 corresponds to the operation of the HMI control device 23 and is executed by the HMI control device 23.

[0106] During autonomous driving, the in-vehicle system 10 repeatedly executes the processes of steps 608 and 609. In step 608, the in-vehicle system 10 determines whether to continue autonomous driving. The process of step 608 is executed by the driving control ECU 17 and the HMI control device 23. The process of step 608 will be described in detail later.

[0107] In step 609, the in-vehicle system 10 determines whether the determination result in step 608 is to end autonomous driving. If autonomous driving is to be continued (i.e., step 609=NO), the in-vehicle system 10 returns the process to step 608. Note that if the second task content selected in step 606 is completed while autonomous driving is continuing, the process returns to step 606. On the other hand, if autonomous driving is to be ended (i.e., step 609=YES), the in-vehicle system 10 executes the processes from step 610 onwards. The process of step 609 is executed by the driving control ECU 17, for example, as corresponding to the operation of the driving level determination unit 172.

[0108] In step 610, the in-vehicle system 10 ends the second task. In the following step 611, the in-vehicle system 10 displays a changeover instruction message on the meter display 212 and the CID display 221 and outputs the message as audio from a speaker (not shown). The changeover instruction message is a message for urging the driver to change over to driving. "Driving changeover" refers to the handover of driving operations from the system to the driver, that is, the handover of responsibility for executing the dynamic driving task from the system to the driver. The processing of steps 610 and 611 corresponds to the operation of the HMI control device 23 and is executed by the HMI control device 23.

[0109] Following the processing of step 611, the in-vehicle system 10 executes the processing of step 612. In step 612, the in-vehicle system 10 determines whether the preparation for the driver changeover is complete and the driver changeover is possible. The preparation for the driver changeover refers to the driver getting ready for the driver changeover. Specifically, the in-vehicle system 10 determines whether the preparation for the driver changeover is complete based on the detection results of the driver's grip on the steering wheel and the driver's line of sight. The processing of step 612 is executed by the driving control ECU 17, for example, as corresponding to the operations of the information acquisition unit 171 and the driving level determination unit 172.

[0110] If the preparation for driver handover is complete and the driver handover is possible (i.e., step 612 = YES), the in-vehicle system 10 proceeds to step 613. In step 613, the in-vehicle system 10 executes the driver handover. That is, the in-vehicle system 10 ends the autonomous driving and sets the driving automation level to SAE Level 2. The processing of step 613 is executed by the driving control ECU 17, for example, as corresponding to the operation of the driving level determination unit 172. Thereafter, the in-vehicle system 10 ends the autonomous driving control operation.

[0111] If the preparation for the driver changeover has not been completed (i.e., step 612=NO), the in-vehicle system 10 causes the process to proceed to step 614. In step 614, the in-vehicle system 10 determines whether a predetermined time T1 has elapsed since the start of displaying the changeover instruction message in step 611. If the predetermined time T1 has not yet elapsed (i.e., step 614=NO), the in-vehicle system 10 returns the process to step 612. On the other hand, if the predetermined time T1 has elapsed (i.e., step 614=YES), the in-vehicle system 10 causes the process to proceed to steps 615 and 616. The process of step 614 is executed by the HMI control device 23 as a process corresponding to the operation of the HMI control device 23.

[0112] In step 615, the in-vehicle system 10 displays a driver change warning message on the meter display 212 and the CID display 221 and outputs it as audio from the speaker. The driver change warning message is a message to warn that a driver change is not possible because preparations for the driver change are not complete. The processing of step 615 is executed by the HMI control device 23, for example, as a processing corresponding to the operation of the display control unit 236.

[0113] In step 616, the in-vehicle system 10 determines whether a predetermined time T2 has elapsed since the start of displaying the changeover instruction message in step 611. T2>T1. If the predetermined time T2 has not yet elapsed (i.e., step 616=NO), the in-vehicle system 10 returns the process to step 612. On the other hand, if the predetermined time T2 has elapsed (i.e., step 616=YES), the in-vehicle system 10 proceeds with the process to steps 617 and 618. The process of step 616 is executed by the HMI control device 23 as a corresponding operation of the HMI control device 23.

[0114] In step 617, the in-vehicle system 10 displays an emergency stop message instructing the driver to bring the vehicle to an emergency stop on the meter display 212 and the CID display 221, and outputs the message by voice from the speaker. The process of step 617 is executed by the HMI control device 23, for example, as a process corresponding to the operation of the display control unit 236.

[0115] In step 618, the in-vehicle system 10 executes the processing required to bring the vehicle to an emergency stop in a safe place. Specifically, the information acquisition unit 171 in the driving control ECU 17 receives information from the HMI control device 23 that an emergency stop message has been displayed. As a result, the driving control ECU 17 detects that preparation for a driver changeover has not been completed before the predetermined time T2 has elapsed. Then, the driving control ECU 17 executes the vehicle control required to bring the vehicle to an emergency stop in a safe place (for example, an emergency parking zone). Thereafter, the in-vehicle system 10 ends the automatic driving control operation.

[0116] (Automatic driving start condition determination) Fig. 7 shows details of the processing content of step 601 shown in Fig. 6, which determines whether the automatic driving start condition is met. In the processing for determining whether the automatic driving start condition is met, the driving control ECU 17 first executes the processing of step 701.

[0117] In step 701, the driving control ECU 17 determines whether or not a section condition is met. The "section condition" means that the vehicle is currently traveling in a specific section. If the section condition is met (i.e., step 701 = YES), the driving control ECU 17 causes the process to proceed to step 702.

[0118] In step 702, the driving control ECU 17 determines whether or not the environmental conditions are met. The "environmental conditions" refer to a driving environment in which autonomous driving is possible. The environmental conditions include, for example, a traffic environment in which autonomous driving is possible, and the absence of abnormalities in the sensors and other components required for autonomous driving. Specifically, the traffic environment includes, for example, the absence of poor visibility due to fog or heavy rain, the absence of obstructing events such as accidents or road construction within the planned autonomous driving section, and the like.

[0119] If the environmental conditions are met (i.e., step 702 = YES), the drive control ECU 17 executes the process of step 703 and then ends the process of determining whether the automatic driving start condition is met. In step 703, the drive control ECU 17 determines whether the automatic driving start condition is met. Specifically, in this case, the drive control ECU 17 determines whether the start condition for high-speed range automatic driving is met.

[0120] On the other hand, if the environmental conditions are not met (i.e., step 702=NO), the drive control ECU 17 ends the process of determining whether the automatic driving start condition is met after executing the process of step 704. In step 704, the drive control ECU 17 determines that the automatic driving start condition is not met.

[0121] If the section condition is not met (i.e., step 701 = NO), the driving control ECU 17 proceeds to step 705. In step 705, the driving control ECU 17 determines whether or not a congestion condition is met. The "congestion condition" means that the host vehicle is currently traveling through a congestion section, or that the host vehicle is currently about to enter a congestion section. Specifically, the congestion condition is met when it is determined based on traffic information that the current position of the host vehicle is within or near the congestion section, and when it is detected that any of the following (A) to (D) applies: (A) The distance between the host vehicle and the vehicle ahead is rapidly decreasing. (B) The speed of the vehicle ahead is rapidly decreasing. (C) The vehicle ahead is flashing its hazard lights. (D) The distance between the host vehicle and the vehicle ahead is a predetermined short distance (e.g., 10 m or less), and the vehicle's traveling speed is in a predetermined low speed range (e.g., less than 20 km / h).

[0122] If the congestion condition is met (i.e., step 705=YES), the drive control ECU 17 proceeds to step 706. On the other hand, if the congestion condition is not met (i.e., step 705=NO), the drive control ECU 17 executes the process of step 704 and then ends the process of determining the automatic driving start condition. In step 704, the drive control ECU 17 determines that the automatic driving start condition is not met.

[0123] In step 706, the driving control ECU 17 determines whether the latest traffic information is currently being acquired by the information acquisition unit 171. "Latest" means, for example, that the traffic information provided by the Japan Road Traffic Information Center or the like is within N minutes from the current time. N is, for example, 5 to 10.

[0124] If the latest traffic information has been acquired (i.e., step 706 = YES), the drive control ECU 17 executes the process of step 703 and then ends the process of determining whether the autonomous driving start condition is met. In step 703, the drive control ECU 17 determines whether the autonomous driving start condition is met. Specifically, in this case, the drive control ECU 17 determines whether the start condition for autonomous driving in traffic congestion is met.

[0125] On the other hand, if the latest traffic information has not been acquired (i.e., step 706=NO), the drive control ECU 17 proceeds to step 707. In step 707, the drive control ECU 17 determines whether the state in which the congestion condition is met has continued for a predetermined time.

[0126] If the congestion condition has been met for a predetermined time (i.e., step 707=YES), the drive control ECU 17 executes the process of step 703 and then ends the process of determining whether the autonomous driving start condition is met. In step 703, the drive control ECU 17 determines whether the autonomous driving start condition is met. Specifically, in this case, the drive control ECU 17 determines whether the condition for starting autonomous driving in a traffic jam is met.

[0127] On the other hand, if the state in which the congestion condition is satisfied has not continued for the predetermined time (i.e., step 707=NO), the drive control ECU 17 ends the process of determining whether the automatic driving start condition is satisfied after executing the process of step 704. In step 704, the drive control ECU 17 determines that the automatic driving start condition is not satisfied.

[0128] (Content selection) Fig. 8 shows details of the processing content of step 606 shown in Fig. 6, which selects second task content to be used during autonomous driving. In this content selection processing, the HMI control device 23 first executes the processing of steps 801 and 802.

[0129] In step 801, the HMI control device 23 displays a second task permission message on the CID display 221. The second task permission message is a message for notifying the driver that use of the second task is now possible. In step 802, the HMI control device 23 determines whether the section condition is met. That is, the HMI control device 23 determines whether the automated driving that has just started execution is high-speed range automated driving.

[0130] If the currently started autonomous driving is high-speed range autonomous driving (i.e., step 802=YES), the HMI control device 23 executes the processes of steps 803, 804, and 805. On the other hand, if the currently started autonomous driving is traffic jam autonomous driving (i.e., step 802=NO), the HMI control device 23 executes the processes of steps 806 and 807.

[0131] Steps 803 to 805 are processes for acquiring the continuation time Tc of high-speed range autonomous driving, which is SAE Level 3 autonomous driving that allows driving at high speeds in a specific section. In step 803, the HMI control device 23 acquires the remaining distance in the specific section in which the vehicle is currently traveling. The remaining distance is the planned traveling distance from the current position of the vehicle to the point where the vehicle will exit the specific section (for example, the exit interchange where the vehicle will exit, or the end point of the specific section).

[0132] In step 804, the HMI control device 23 calculates the section travel time Tv based on the remaining distance acquired in step 803. Specifically, the section travel time Tv can be calculated, for example, by dividing the remaining distance acquired in step 803 by the average travel speed of the host vehicle in the specific section in which it is currently traveling.

[0133] In step 805, the HMI control device 23 acquires the continuation time Tc based on the section traveling time Tv calculated in step 804. Specifically, the continuation time Tc can be calculated, for example, by multiplying or subtracting the section traveling time Tv calculated in step 804 by a predetermined correction constant.

[0134] It should be noted that, for some reason, it may be impossible to calculate the section traveling time Tv in step 804. In this case, the HMI control device 23 sets the continuation time Tc to the maximum value that can be set. Specifically, for example, in a specification in which the value of the continuation time Tc is set using a four-digit hexadecimal number from "0000" to "FFFF," the value of the continuation time Tc is set to "FFFF" when the section traveling time Tv cannot be calculated.

[0135] Steps 806 and 807 are processes for acquiring the possible continuation time Tc of autonomous driving in traffic jams, which is SAE Level 3 autonomous driving that allows driving at medium to low speeds during traffic jams. In step 806, the HMI control device 23 acquires a predicted traffic jam time Tj, which is a predicted duration of traffic jams. The predicted traffic jam time Tj is an approximate value of the time required for the host vehicle to finish driving through the traffic jam section in which it is currently traveling. Specifically, the predicted traffic jam time Tj can be calculated, for example, based on the average driving speed of the host vehicle during the current traffic jam and the remaining distance through the traffic jam section acquired from the traffic jam information.

[0136] In step 807, the HMI control device 23 acquires the possible continuation time Tc based on the predicted congestion time Tj acquired in step 806. Specifically, the possible continuation time Tc can be calculated, for example, by multiplying or subtracting a predetermined correction constant from the predicted congestion time Tj acquired in step 806. Note that, for some reason, it may be impossible to acquire the predicted congestion time Tj in step 806. In this case, the HMI control device 23 sets the possible continuation time Tc to the maximum value that can be set. A specific example in this case is, for example, "FFFF", as described above.

[0137] Once the acquisition of the continuation time Tc according to the cause or type of the start of autonomous driving is completed as described above, the HMI control device 23 proceeds to step 808. In step 808, the HMI control device 23 determines whether the continuation time Tc is less than the first threshold value Tc1.

[0138] If the continuation time Tc is less than the first threshold value Tc1 (i.e., step 808=YES), the HMI control device 23 proceeds to step 809. In step 809, the HMI control device 23 determines the short-time content as suggested content. The suggested content is second task content that is suggested to the driver to use.

[0139] If the allowable continuation time Tc is equal to or greater than the first threshold value Tc1 (i.e., step 808 = NO), the HMI control device 23 advances the process to step 810. In step 810, the HMI control device 23 determines whether the allowable continuation time Tc is less than the second threshold value Tc2. That is, the determination in step 810 is whether the allowable continuation time Tc is equal to or greater than the first threshold value Tc1 and less than the second threshold value Tc2. The second threshold value Tc2 is a threshold for determining whether the allowable continuation time Tc is set to the maximum value. The first threshold value Tc1<<the second threshold value Tc2. Therefore, the determination in step 810 is whether the time acquisition unit 233 has acquired the allowable continuation time Tc.

[0140] If the allowable continuation time Tc is less than the second threshold value Tc2 (that is, step 810=YES), the HMI control device 23 advances the process to step 811. In step 811, the HMI control device 23 determines the long-duration content as the suggested content.

[0141] If the possible continuation time Tc is equal to or greater than the second threshold value Tc2 (i.e., step 810=NO), it means that the time acquisition unit 233 was unable to acquire the possible continuation time Tc. That is, for some reason, the section travel time Tv or the predicted traffic congestion time Tj could not be acquired, and the possible continuation time Tc is set to the maximum value. In this case, the HMI control device 23 advances the process to step 812. In step 812, the HMI control device 23 determines the composite content as the suggested content. The composite content is a combination of both short-term content and long-term content.

[0142] After the type of suggested content is determined as described above, the HMI control device 23 proceeds to step 813 and step 814. In step 813, the HMI control device 23 causes the CID display 221 to display a presentation screen for the determined suggested content.

[0143] Specifically, when the proposed content is short-time content, the HMI control device 23 displays a short-time content presentation screen on the CID display 221. The short-time content presentation screen is a display screen for presenting short-time content, and is a screen that displays "recommended" short-time content in the central display area DA1 in Fig. 2 so that it can be executed or played.

[0144] Furthermore, when the proposed content is long-duration content, the HMI control device 23 displays a long-duration content presentation screen on the CID display 221. The short-duration content presentation screen is a display screen for presenting long-duration content, and is a screen that displays "recommended" long-duration content in the central display area DA1 in Fig. 2 so that it can be executed or played.

[0145] Furthermore, when the proposed content is composite content, the HMI control device 23 presents a selection of second task contents, namely, short-term content and long-term content, each of which has a different execution time. Specifically, the HMI control device 23 displays the menu screen shown in FIG. 5 on the CID display 221.

[0146] In step 814, the HMI control device 23 determines whether the content selection operation by the driver or the like has ended. While the content selection operation is not yet completed, the determination in step 814 is "NO," and the HMI control device 23 returns the process to step 813. That is, the HMI control device 23 waits for the content selection operation to end. When the content selection operation ends (that is, step 814 = YES), the HMI control device 23 ends the content selection process. This starts the use of the selected second task content.

[0147] (Autonomous driving continuation decision) Fig. 9 shows details of the processing content of step 608 shown in Fig. 6, which determines whether or not to continue autonomous driving. In the processing for determining whether to continue autonomous driving, the driving control ECU 17 first executes the processing of step 901.

[0148] In step 901, the drive control ECU 17 determines whether it has been determined in step 705 that the congestion condition is met. That is, the drive control ECU 17 determines whether the SAE Level 3 autonomous driving currently being executed is the congestion autonomous driving or the high-speed range autonomous driving.

[0149] If the currently executing automated driving is high-speed range automated driving (i.e., step 901 = NO), the driving control ECU 17 causes the process to proceed to step 902. In step 902, the driving control ECU 17 determines whether or not an end condition for high-speed range automated driving is met. The end condition includes, for example, that the distance or required time to the point where the vehicle exits the specific section (e.g., the exit interchange from which the vehicle exits, or the end point of the specific section) is less than a predetermined value. The end condition also includes, for example, that the behavior of the vehicle up to a predetermined time (e.g., 15 seconds) ahead becomes unpredictable, taking into account the traffic environment or road conditions ahead of the vehicle.

[0150] If the termination condition is not met (i.e., step 902=NO), the drive control ECU 17 proceeds to step 903 and then temporarily terminates the process for determining whether to continue the autonomous driving. In step 903, the drive control ECU 17 determines whether to continue the autonomous driving and the second task.

[0151] If the termination condition is met (i.e., step 902 = YES), the drive control ECU 17 causes the process to proceed to step 904, and then terminates the process for determining whether to continue the autonomous driving. In step 904, the drive control ECU 17 determines to terminate the autonomous driving and the second task.

[0152] If the currently executed autonomous driving is autonomous driving in traffic congestion (i.e., step 901 = YES), the driving control ECU 17 proceeds to step 905. In step 905, the driving control ECU 17 determines whether a scene of temporary relief from traffic congestion has occurred. In other words, the driving control ECU 17 determines whether temporary relief from traffic congestion has been detected.

[0153] If a scene of temporary relief of traffic congestion has not occurred (i.e., step 905=NO), the drive control ECU 17 proceeds to step 903 and then temporarily ends the process of determining whether to continue the autonomous driving. In step 903, the drive control ECU 17 determines whether to continue the autonomous driving and the second task.

[0154] If a scene of temporary relief of traffic congestion occurs (i.e., step 905=YES), the drive control ECU 17 advances the process to step 906. In step 906, the drive control ECU 17 determines whether the information acquisition unit 171 has currently acquired the latest traffic congestion information.

[0155] If the latest traffic congestion information has not been acquired (i.e., step 906 = NO), the drive control ECU 17 proceeds to step 904 and then terminates the process for determining whether to continue the autonomous driving. In step 904, the drive control ECU 17 determines whether to terminate the autonomous driving and the second task.

[0156] If the latest congestion information has been acquired (i.e., step 906=YES), the drive control ECU 17 proceeds to step 907. In step 907, the drive control ECU 17 determines whether the current driving position of the host vehicle is in a congestion section.

[0157] If the current driving position of the vehicle is not in a congested section (i.e., step 907=NO), the driving control ECU 17 proceeds to step 904 and then terminates the process for determining whether to continue the autonomous driving. In step 904, the driving control ECU 17 determines to terminate the autonomous driving and the second task.

[0158] If the current driving position of the host vehicle is within a congested section (i.e., step 907 = YES), the driving control ECU 17 causes the processing to proceed to steps 908 and 909. In step 908, the driving control ECU 17 sends a command to the HMI control device 23 to cause the CID display 221 to execute a continuation proposal display. As a result, the HMI control device 23 causes the CID display 221 to execute the continuation proposal display. The continuation proposal display includes a continuation proposal message proposing the continuation of automated driving and an approval button DA6 shown in FIG. 3 for approval of the continuation. For example, the continuation proposal message is displayed alternately with a warning message saying "Watch out ahead" in the message display box DB4 shown in FIG. 3. In addition, the HMI control device 23 executes audio guidance corresponding to the continuation proposal display content using a speaker (not shown).

[0159] In step 909, the driving control ECU 17 determines whether or not an approval operation has been performed by the driver or the like on the HMI device 20. That is, the driving control ECU 17 receives the determination result as to whether or not the approval button DA6 shown in FIG. 3 has been selected and operated from the HMI control device 23.

[0160] If the approval operation has not been performed (i.e., step 909=NO), the drive control ECU 17 causes the process to proceed to step 910. In step 910, the drive control ECU 17 determines whether a predetermined time has elapsed since the start of the continuation proposal display. If the predetermined time has not yet elapsed, the determination in step 910 becomes "NO," and the drive control ECU 17 returns the process to step 909. In other words, the drive control ECU 17 waits for the approval operation for a predetermined time.

[0161] If a predetermined time has elapsed without an approval operation being performed, the determination in step 910 becomes "YES." In this case, the drive control ECU 17 proceeds to step 904 and then terminates the process for determining whether to continue autonomous driving. In step 904, the drive control ECU 17 determines to terminate autonomous driving and the second task.

[0162] If the approval operation is performed within the predetermined time, the determination in step 909 becomes "YES." In this case, the driving control ECU 17 causes the process to proceed to step 911. In step 911, the driving control ECU 17 determines whether the driver is checking the road conditions ahead, i.e., the road ahead of the vehicle.

[0163] If the driver is checking the road ahead (i.e., step 911=YES), the drive control ECU 17 proceeds to step 903 and then temporarily ends the process for determining whether to continue the autonomous driving. In step 903, the drive control ECU 17 determines whether to continue the autonomous driving and the second task.

[0164] If the driver is not checking ahead (i.e., step 911 = NO), the driving control ECU 17 causes the process to proceed to step 912. In step 912, the driving control ECU 17 determines whether a predetermined time has elapsed since the approval operation. If the predetermined time has not yet elapsed, the determination in step 912 becomes "NO," and the driving control ECU 17 returns the process to step 911. In other words, the driving control ECU 17 waits for the driver to check ahead for a predetermined time.

[0165] If a predetermined time has passed since the approval operation without the driver checking ahead, the determination in step 912 becomes "YES." In this case, the driving control ECU 17 advances the process to step 904 and then ends the process for determining whether to continue autonomous driving. In step 904, the driving control ECU 17 determines to end autonomous driving and the second task.

[0166] Second Embodiment The second embodiment will be described below with reference to FIGS. 1 to 9 as well as FIGS. 10 and 11. The following description of the second embodiment will mainly focus on differences from the first embodiment. In the first and second embodiments, identical or equivalent parts are designated by the same reference numerals. Therefore, in the following description of the second embodiment, the explanation of the first embodiment can be applied appropriately to components having the same reference numerals as those in the first embodiment, unless there is a technical contradiction or a special additional explanation. The same applies to the third and subsequent embodiments described below.

[0167] The basic configuration of the in-vehicle system 10 according to this embodiment is the same as that shown in Fig. 1. The basic operation of the in-vehicle system 10 according to this embodiment is the same as that shown in Figs. 2 to 9. In this embodiment, the following functions are added to those of the first embodiment.

[0168] For example, because there is a traffic obstruction such as an accident, traffic jam, broken-down vehicle, or construction ahead of the vehicle, the driver may take his / her eyes off the CID display 221 and check ahead multiple times even while performing the second task. Therefore, in this embodiment, when the behavior acquisition unit 232 acquires a detection result indicating that the driver is checking the road conditions ahead of the vehicle, the display control unit 236 displays the road conditions on the HMI device 20. Specifically, when the driver checks ahead at a predetermined frequency, the display control unit 236 displays the road conditions in the information display box DB3, as shown in FIG. 10 .

[0169] 11 shows the display control processing of the second task executed by the HMI control device 23. The display control processing of the second task is repeatedly executed at predetermined time intervals (for example, 100 msec) from the time the processing proceeds to step 607 to the time the processing proceeds to step 610 in the flowchart shown in FIG.

[0170] In the display control processing for the second task, first, in step 1101, HMI control device 23 reads display content data into a temporary storage area such as RAM. The display content is second task content to be displayed on CID display 221. Next, in step 1102, HMI control device 23 uses the data read in step 1101 to display the display content on CID display 221.

[0171] Next, the HMI control device 23 determines whether or not to execute the approval process this time in step 1103. The approval process is a process for accepting various approval operations by the driver.

[0172] If approval processing is to be executed (i.e., step 1103=YES), the HMI control device 23 executes the processing of steps 1104 and 1105, and then temporarily terminates the display control processing of the second task. In step 1104, the HMI control device 23 displays an approval request message on the CID display 221. In step 1105, the HMI control device 23 displays an approval button DA6 on the CID display 221.

[0173] If the approval process is not to be executed (i.e., step 1103 = NO), the HMI control device 23 advances the process to step 1106. In step 1106, the HMI control device 23 determines whether or not to execute the warning process this time. The warning process is a process for issuing various warnings to the driver.

[0174] If the warning process is to be executed (i.e., step 1106=YES), the HMI control device 23 executes the process of step 1107 and then temporarily terminates the display control process of the second task. In step 1107, the HMI control device 23 displays warning content on the CID display 221. The warning content includes messages and / or graphics for various warnings.

[0175] If the warning process is not to be executed (that is, step 1106=NO), the HMI control device 23 advances the process to step 1108. In step 1108, the HMI control device 23 determines whether the driver is checking ahead.

[0176] If the driver is not checking ahead (i.e., step 1108=NO), the HMI control device 23 temporarily ends the display control processing of the second task. On the other hand, if the driver is checking ahead (i.e., step 1108=YES), the HMI control device 23 advances the processing to step 1109.

[0177] In step 1109, the HMI control device 23 determines whether the driver's forward checks have reached a predetermined frequency. The "predetermined frequency" is the number of times per predetermined reference time, for example, three or more times per five seconds. If the driver's forward checks have not reached the predetermined frequency (i.e., step 1109 = NO), the HMI control device 23 temporarily terminates the display control processing of the second task. On the other hand, if the driver's forward checks have reached the predetermined frequency (i.e., step 1109 = YES), the HMI control device 23 causes the processing to proceed to step 1110.

[0178] In step 1110, the HMI control device 23 determines whether traffic information relating to the surroundings of the vehicle has been acquired by the vehicle information acquisition unit 231. The "surroundings of the vehicle" refers to the destination of the vehicle, within a predetermined estimated travel distance (for example, 5 km) from the current position of the vehicle.

[0179] If traffic information about the vicinity of the vehicle has not been acquired (i.e., step 1110=NO), the HMI control device 23 temporarily terminates the display control processing of the second task. On the other hand, if traffic information about the vicinity of the vehicle has been acquired (i.e., step 1110=YES), the HMI control device 23 executes the processing of step 1111 and then temporarily terminates the display control processing of the second task. In step 1111, the HMI control device 23 displays the acquired traffic information about the vicinity of the vehicle in the information display box DB3.

[0180] In this manner, in this embodiment, when the driver checks ahead multiple times at a predetermined frequency during the second task, traffic information related to the surroundings of the vehicle is displayed on the HMI device 20. Therefore, according to this embodiment, it is possible to display appropriate traffic information in accordance with the actual traffic situation on the HMI device 20.

[0181] (Additional embodiment 1: Third embodiment) The third embodiment will be described below. In the description of this embodiment, as in the description of the second embodiment, the description of the preceding embodiment will be used as appropriate, and the description will mainly focus on the parts that are unique to this embodiment.

[0182] This embodiment relates to processing when the possible continuous autonomous driving time Tc acquired by the time acquisition unit 233 is an extremely short time. An "extremely short time" is, for example, less than several tens of seconds, or a time equivalent to a driving distance of less than 1000 m. In this embodiment, when the possible continuous time Tc is an extremely short time, the driving control ECU 17 allows the driver to select whether to perform autonomous driving in traffic jams, but does not allow the execution of autonomous driving in high-speed ranges.

[0183] Specifically, the operation of this embodiment will be described with reference to Fig. 7. In this embodiment, the "environmental conditions" when determining whether or not the environmental conditions are met in step 702, which is executed when the section condition is met (i.e., step 701 = YES), include whether the continuation time Tc is equal to or greater than a predetermined execution lower limit time. Here, the "execution lower limit time" is a determination reference value for determining whether the continuation time Tc corresponds to the extremely short time described above, and is a value (e.g., 30 seconds) that is sufficiently smaller than the first threshold value Tc1 in step 808.

[0184] If the allowable continuation time Tc is shorter than the lower execution limit time, the allowable continuation time Tc corresponds to the extremely short time described above, so the determination result in step 702 is "NO," and the drive control ECU 17 determines in step 704 that the autonomous driving start condition is not met. In this case, the determination result in step 602 in the flowchart shown in FIG. 6 is "NO," and the in-vehicle system 10 skips all processing from step 603 onwards and terminates the autonomous driving control operation. At this time, it is preferable that the HMI control device 23 presents, via the HMI device 20, that autonomous driving and execution of the second task are not permitted because the allowable continuation time Tc is shorter than the lower execution limit time.

[0185] On the other hand, if the congestion condition is met instead of the section condition (i.e., step 705 = YES), the process does not proceed to the determination of whether the "environmental conditions" are met, including whether the continuation time Tc is equal to or greater than the lower execution time limit (i.e., step 702). In other words, for automated driving in a traffic jam, the execution condition does not include whether the continuation time Tc is equal to or greater than the predetermined lower execution time limit. This enables the HMI control device 23 to execute a second task during automated driving in a traffic jam for an extremely short period of time. In this case, it is preferable that the HMI control device 23 presents, via the HMI device 20, in step 603 that the time required for executing automated driving and using the associated second task is extremely short.

[0186] Thus, according to this embodiment, the execution of autonomous driving for a very short period of time and the accompanying use of the second task are possible during traffic jams, but are prohibited under driving conditions that correspond to high-speed autonomous driving. Therefore, according to this embodiment, the execution of autonomous driving and the use of the second task can be performed more appropriately.

[0187] Incidentally, when the autonomous driving continuation time Tc is short, by using the second task to keep the driver's attention on driving the vehicle (i.e., the running state and driving operation) as much as possible, a smooth handover of driving when the autonomous driving ends is possible. Therefore, in such a case, it is preferable to limit the HMI device 20 that executes the second task to the in-vehicle device, i.e., the CID device 22.

[0188] 12 shows the modified portion of the flowchart in FIG. 8 in accordance with this example. Specifically, referring to FIG. 12, if the available continuation time Tc is less than the first threshold value Tc1 (i.e., step 808=YES), the HMI control device 23 advances the process to step 1201. In step 1201, the HMI control device 23 determines whether the available continuation time Tc is greater than or equal to the model-specific threshold value TcS. The model-specific threshold value TcS is less than the first threshold value Tc1. In other words, the model-specific threshold value TcS is a value that is sufficiently smaller than the first threshold value Tc1 in step 808.

[0189] If the allowable duration Tc is equal to or greater than the model-restricted threshold TcS (i.e., step 1201=YES), the HMI control device 23 advances the process to step 809 and determines the short content as the suggested content. On the other hand, if the allowable duration Tc is less than the model-restricted threshold TcS (i.e., step 1201=NO), the HMI control device 23 advances the process to step 1202. In step 1202, the HMI control device 23 determines the restricted content as the suggested content. Restricted content is short-term content that is restricted to execution by the CID device 22, which is an in-vehicle device, is suitable for extremely short-term use, and allows the driver to easily focus their attention on driving the vehicle. Specifically, restricted content is, for example, a music video that takes a relatively short time to execute and can be played without any problem even if the audio is heard, or a television broadcast whose execution time is inconceivable.

[0190] The example shown in Figure 12 is particularly effective when the possible continuation time Tc of autonomous driving is extremely short. That is, when the possible continuation time Tc acquired based on the predicted traffic jam time Tj is shorter than a predetermined model-limited time, the second task presentation unit 234 preferably limits the HMI device 20 that executes the second task during traffic congestion to the in-vehicle device, i.e., the CID device 22. The "predetermined model-limited time" is the model-limited threshold TcS. In this case, the model-limited threshold TcS may be equal to the above-mentioned lower limit time, or there may be a slight difference between the model-limited threshold TcS and the lower limit time.

[0191] (Additional Embodiment 2: Fourth Embodiment) The fourth embodiment will be described below. This embodiment is a slight modification of the first and / or third embodiments. Specifically, this embodiment presents the second task permission message earlier when automated driving in traffic congestion is performed in an extremely short time.

[0192] That is, in this embodiment, when the possible continuation time Tc of autonomous driving is shorter than a predetermined time limit, the second task presentation unit 234 operates as follows: Instead of presenting an executable second task based on the possible continuation time Tc, or prior to such presentation, the second task presentation unit 234 presents, via the HMI device 20, information indicating that the second task is executable.

[0193] Figures 13 and 14 show an example of operation according to this embodiment. Figure 13 shows the vicinity of the modified part when a part of the flowchart in Figure 6 is modified to correspond to this example of operation. Figure 14 shows the part of the flowchart in Figure 7 modified to correspond to this example of operation.

[0194] 13, in step 601, the in-vehicle system 10 determines whether the autonomous driving start condition is met. At this time, as will be described later, a process for setting an extremely short time flag Fs, which indicates whether the execution time of autonomous driving is an extremely short time, is executed. The process contents of steps 602 to 605 are the same as those in the first embodiment.

[0195] Following the processing of step 605, the in-vehicle system 10 advances the processing to step 1301. In step 1301, the in-vehicle system 10 determines the setting state of the extremely short time flag Fs. Specifically, if the extremely short time flag Fs is reset (i.e., Fs=0), the in-vehicle system 10 determines the determination result of step 1301 as "YES," whereas if the extremely short time flag Fs is set (i.e., Fs=1), the in-vehicle system 10 determines the determination result of step 1301 as "NO."

[0196] If the extremely short time flag Fs is not set (i.e., step 1301=YES), the execution time of the autonomous driving started this time in step 605 is not extremely short. Therefore, in this case, the in-vehicle system 10 proceeds to step 606. In step 606, the in-vehicle system 10 executes an operation for selecting second task content to be used during autonomous driving. Note that in this embodiment, the content selection process in step 606 can be the process shown in FIG. 8 itself, or FIG. 8 modified by FIG. 12.

[0197] If the extremely short time flag Fs is set (i.e., step 1301=NO), the execution time of the current autonomous driving started in step 605 is extremely short. Therefore, in this case, the in-vehicle system 10 causes the process to proceed to steps 1302 and 1303.

[0198] In step 1302, the in-vehicle system 10 displays a second task permission message on the CID display 221. The processing content in step 1302 is the same as that in step 801 in the flowchart shown in FIG.

[0199] The driver can understand that the second task can be started by presenting the second task permission message in step 1302. Therefore, the driver or the like can select and execute the desired second task content, i.e., start using it, by operating the HMI device 20 as desired. Therefore, in step 1303, the in-vehicle system 10 determines whether or not the driver or the like has performed an operation to start the second task.

[0200] If the second task start operation has not yet been performed (i.e., step 1303=NO), the in-vehicle system 10 waits for such an operation for a predetermined time. Specifically, if the determination result of step 1303 is "NO," the in-vehicle system 10 causes the process to proceed to step 1304. In step 1304, the in-vehicle system 10 determines whether or not a predetermined waiting time has elapsed since the timing of presenting the second task permission message in step 1302. If the waiting time has not yet elapsed (i.e., step 1304=NO), the in-vehicle system 10 returns the process to step 1303. On the other hand, if the waiting time has elapsed (i.e., step 1304=YES), the in-vehicle system 10 causes the process to proceed to step 606. As a result, the content selection process is executed as usual.

[0201] If the second task start operation is performed before the waiting time has elapsed (i.e., step 1303=YES), the in-vehicle system 10 proceeds to step 1305. In step 1305, the in-vehicle system 10 determines whether the second task instructed to be executed by the second task start operation is executable. Specifically, for example, the in-vehicle system 10 determines whether the second task instructed to be executed by the second task start operation is the above-mentioned restricted content that can be executed during extremely short-term autonomous driving.

[0202] If the second task instructed to be executed by the second task start operation is executable (i.e., step 1305=YES), the in-vehicle system 10 proceeds to step 607. In step 607, the in-vehicle system 10 starts executing, i.e., using, the second task content selected by the second task start operation. In other words, in this case, the content selection process in step 606 is skipped. On the other hand, if the second task instructed to be executed by the second task start operation is not executable (i.e., step 1305=NO), the in-vehicle system 10 proceeds to step 1304.

[0203] 14, the processing content of step 701 is the same as that of the first embodiment. When the processing proceeds to step 702, it means that the host vehicle is currently traveling in a specific section where high-speed range automated driving is possible.

[0204] If it is determined in step 702 that the environmental conditions are not met (i.e., step 702=NO), the drive control ECU 17 executes the process of step 704 and then ends the process of determining the automatic driving start conditions. In step 704, the drive control ECU 17 determines that the automatic driving start conditions are not met. On the other hand, if it is determined in step 702 that the environmental conditions are met (i.e., step 702=YES), the drive control ECU 17 causes the process to proceed to steps 1401 and 1402.

[0205] Note that the "environmental conditions" in this case do not include the absence of obstructing events such as accidents or road construction within the planned autonomous driving section, or the fact that the allowable continuation time Tc is equal to or greater than a predetermined lower limit time, for the following reasons. This is because whether or not there are obstructing events such as accidents or road construction within the planned autonomous driving section is reflected in the length of the allowable continuation time Tc. Also, in this embodiment, whether or not the allowable continuation time Tc is equal to or greater than the lower limit time is processed as whether or not the extremely short time flag Fs is set, rather than as whether or not the environmental conditions are met in step 702.

[0206] In step 1401, the drive control ECU 17 acquires, that is, calculates, the continuation possible time Tc of high-speed range automatic driving. The processing content of step 1401 is the same as that of steps 803 to 805 in the flowchart shown in FIG.

[0207] In step 1402, the operation control ECU 17 determines whether the calculated continuation time Tc is equal to or greater than the time limit TcL. The time limit TcL may be equal to the above-mentioned model-specific threshold TcS or the execution lower limit time, or may be slightly different from these. The time limit TcL is less than the first threshold Tc1. In other words, the time limit TcL is a value sufficiently smaller than the first threshold Tc1 in step 808.

[0208] If the allowable continuation time Tc of high-speed range automatic driving is shorter than the time limit TcL (i.e., step 1402 = NO), the drive control ECU 17 executes the process of step 704 and then ends the process of determining the automatic driving start condition. In step 704, the drive control ECU 17 determines that the automatic driving start condition is not met. In other words, if the allowable continuation time Tc of high-speed range automatic driving is shorter than the time limit TcL, high-speed range automatic driving is not executed, and therefore execution of the second task is also not permitted.

[0209] On the other hand, if the continuation time Tc of high-speed range automatic driving is equal to or longer than the time limit TcL (i.e., step 1402=YES), the drive control ECU 17 ends the process of determining whether the automatic driving start condition is met after executing the process of step 703. In step 703, the drive control ECU 17 determines whether the start condition of high-speed range automatic driving is met.

[0210] The processing content of step 705 in Figure 14 is the same as in the first embodiment. The determination content of steps 706 and 707 is the same as in the first embodiment. If the conditions for starting autonomous driving in traffic congestion are met, the determination result in step 705 is "YES", and the determination result in step 706 or step 707 is also "YES".

[0211] In this embodiment, when the start condition for autonomous driving in traffic congestion is met, the process of setting the extremely short time flag Fs is executed in step 1403 etc. before the process proceeds to step 703. Specifically, first, in step 1403, the drive control ECU 17 acquires, i.e., calculates, the possible continuation time Tc of autonomous driving in traffic congestion. The process content of step 1403 is the same as step 806 and step 807 in the flowchart shown in FIG. 8.

[0212] Next, in step 1404, the drive control ECU 17 determines whether the possible continuous time Tc of automated driving in traffic congestion is equal to or greater than the time limit TcL. If the possible continuous time Tc of automated driving in traffic congestion is equal to or greater than the time limit TcL (i.e., step 1404=YES), the drive control ECU 17 resets the extremely short time flag Fs (i.e., Fs=0) in step 1405 and then causes the process to proceed to step 703. On the other hand, if the possible continuous time Tc of automated driving in traffic congestion is shorter than the time limit TcL (i.e., step 1404=NO), the drive control ECU 17 sets the extremely short time flag Fs (i.e., Fs=1) in step 1406 and then causes the process to proceed to step 703.

[0213] As described above, in this embodiment, if the possible continuous autonomous driving time Tc is an extremely short time, the extremely short time flag Fs is set (i.e., Fs = 1 in step 1406). Then, immediately after the start of autonomous driving and before the content selection process (i.e., step 606) is started, a second task permission message is presented (i.e., step 1302). This allows the second task permission message to be presented earlier to notify the driver or other person that the second task is available when autonomous driving is performed for an extremely short time. Therefore, according to this embodiment, even during an extremely short autonomous driving period, it is possible to adequately ensure the time during which the driver or other person can use the second task, improving convenience.

[0214] (Additional Embodiment 3: Fifth Embodiment) A fifth embodiment will be described below. This embodiment relates to a technology that enables a smooth handover of driving when autonomous driving ends by using a second task to prevent the driver's attention from being too far away from driving the vehicle (i.e., the driving state and driving operation) when autonomous driving is performed for a long time or long distance. Specifically, in this embodiment, the in-vehicle system 10, i.e., the HMI control device 23, is configured to present the driving state of the vehicle or surrounding information via the HMI device 20 when autonomous driving can be continued for a predetermined interrupt waiting time or longer from the start of execution of the second task. The "surrounding information" includes information about the traffic environment around the vehicle (e.g., the presence of other vehicles in the front and adjacent lanes) and / or traffic information about the vehicle's surroundings. In other words, the "surrounding information" typically includes road conditions ahead of the vehicle.

[0215] 15 shows an example of an operation according to this embodiment. The HMI control device 23 repeatedly starts the information interruption presentation routine shown in FIG. 15 at predetermined intervals (e.g., 100 msec) while the second task is being executed. "While the second task is being executed" refers to the period from when the processing in the flowchart shown in FIG. 6 or FIG. 13 proceeds to step 607 to when the processing in the flowchart shown in FIG. 6 proceeds to step 610. Each process in this routine corresponds to the operation of the second task presentation unit 234 and / or the display control unit 236.

[0216] When this routine is started, first, in step 1501, the HMI control device 23 determines whether the continuation time Tc of automatic operation is equal to or longer than the predetermined interrupt waiting time Tw1. If the continuation time Tc is shorter than the interrupt waiting time Tw1 (i.e., step 1501 = NO), all processing from step 1502 onwards is skipped, and this routine is temporarily terminated.

[0217] If the possible continuation time Tc is equal to or greater than the interrupt waiting time Tw1 (i.e., step 1501 = YES), the HMI control device 23 proceeds to step 1502. In step 1502, the HMI control device 23 determines whether the remaining time Tr is equal to or greater than the interrupt waiting time Tw1. The remaining time Tr is the time required from the present time until the time when the currently executing automatic driving is expected to end. In other words, the remaining time Tr is calculated by subtracting the automatic driving duration Tw from the possible continuation time Tc calculated at the start of automatic driving.

[0218] If the remaining time Tr is shorter than the interrupt waiting time Tw1 (i.e., step 1502=NO), all processing from step 1503 onwards is skipped and this routine ends. On the other hand, if the remaining time Tr is equal to or greater than the interrupt waiting time Tw1 (i.e., step 1502=YES), the HMI control device 23 advances the processing to step 1503.

[0219] In step 1503, the HMI control device 23 determines whether the duration Tw is equal to or greater than the interrupt waiting time Tw1. The duration Tw is the time required from a predetermined continuation reference point to the present time. The continuation reference point is the point closest to the present time between the point at which the currently executing autonomous driving started and the point at which the HMI device 20 last performed the operation of presenting the vehicle's driving status or surrounding information.

[0220] If the duration Tw is shorter than the interrupt waiting time Tw1 (i.e., step 1503=NO), all processing from step 1504 onwards is skipped and this routine ends. On the other hand, if the duration Tw is equal to or longer than the interrupt waiting time Tw1 (i.e., step 1503=YES), the HMI control device 23 advances the processing to step 1504.

[0221] In step 1504, the HMI control device 23 determines whether specific content is currently being executed or used. "Specific content" refers to a type of second task content that would be annoying to the occupant if an interruption in information presentation occurred during its use or execution, such as a movie or a game.

[0222] If the specific content is currently being executed or used (i.e., step 1504 = YES), presenting information about the vehicle's running status or surrounding information on the HMI device 20 at this time would be annoying to the occupant who is using the second task. Therefore, in this case, it is preferable to wait until the specific content has finished before presenting such information. Therefore, in this case, the HMI control device 23 skips the processing of steps 1505 and 1506 and temporarily ends this routine.

[0223] If the specific content is not currently being executed or used (i.e., step 1504 = NO), presenting information about the vehicle's driving status or surrounding area information on the HMI device 20 at this time is unlikely to annoy the occupant who is using the second task. Therefore, in this case, it is preferable to present such information at this time. Therefore, in this case, the HMI control device 23 executes the processes of steps 1505 and 1506 related to the information presentation operation and then temporarily terminates this routine. Note that if the specific content is a game, there may be situations where the progress of the game temporarily stops depending on the state of the character controlled by the player. For example, this may occur when the character controlled by the player is defeated by an enemy character or when the input of the selection result based on the player's decision is awaited. If such a situation occurs, the determination result of step 1504 may be "NO."

[0224] In step 1505, the HMI control device 23 presents information on the driving status or surrounding information of the vehicle on the HMI device 20. In step 1506, the HMI control device 23 resets the duration time Tw. This makes it possible to repeatedly present the driving status or surrounding information of the vehicle at intervals of the interrupt waiting time Tw1 until the currently executing autonomous driving ends.

[0225] As described above, according to this embodiment, when autonomous driving is performed for a long time or a long distance, at least a portion of the vehicle information or surrounding information of the vehicle, i.e., the driving state, traffic environment, and traffic information, is presented at least once before the driver handover period. This enables a smooth driver handover when autonomous driving ends. Furthermore, by presenting information when the second task content is interrupted or ended, it is possible to effectively avoid causing annoyance to the occupant using the second task. From this perspective, it is preferable, for example, for the second task presentation unit 234 to alternately present long-duration content and short-duration content.

[0226] (Summary of additional embodiments) The period during which "autonomous driving" corresponding to SAE Levels 3 to 5, during which occupants such as the driver are not required to monitor their surroundings, is performed may vary depending on the road type, the conditions around the vehicle, etc. Therefore, whether the period during which autonomous driving is performed without the driver being required to monitor their surroundings is long or short, it is necessary to realize a driving environment that is convenient for the driver, etc. In this regard, according to the third to fifth embodiments, it is possible to provide a driving environment that is convenient for the driver, etc., even when the period during which autonomous driving is performed without the driver being required to monitor their surroundings is extremely short.

[0227] (Variation) The present invention is not limited to the above-described embodiment. Therefore, the above-described embodiment can be modified as appropriate. Representative modifications will be described below. In the following description of the modifications, differences from the above-described embodiment will be mainly described. Furthermore, identical or equivalent parts between the above-described embodiment and the modifications are given the same reference numerals. Therefore, in the following description of the modifications, the description of the above-described embodiment can be used as appropriate for components having the same reference numerals as those in the above-described embodiment, unless there is a technical contradiction or special additional explanation.

[0228] The present invention is not limited to the specific device configurations shown in the above embodiments. That is, for example, the vehicle equipped with the in-vehicle system 10 is not limited to a four-wheeled vehicle. Specifically, the vehicle may be a three-wheeled vehicle, or a six- or eight-wheeled vehicle such as a cargo truck. The type of vehicle may be a conventional vehicle equipped with only an internal combustion engine, an electric vehicle or a fuel cell vehicle without an internal combustion engine, or a so-called hybrid vehicle. The shape and structure of the vehicle body are not limited to a box shape, i.e., a substantially rectangular shape in a plan view. There are also no particular limitations on the vehicle's use, steering wheel position, number of occupants, etc.

[0229] The communication standard for the in-vehicle system 10 may be other than CAN (internationally registered trademark), such as FlexRay (internationally registered trademark). The communication standard for the in-vehicle system 10 is not limited to one type. For example, the in-vehicle system 10 may have a sub-network line that complies with a communication standard such as LIN. LIN is an abbreviation for Local Interconnect Network.

[0230] The vehicle condition sensor 11, the external environment condition sensor 12, and the perimeter monitoring sensor 13 are not limited to the above examples. For example, the perimeter monitoring sensor 13 may include a sonar, i.e., an ultrasonic sensor. Alternatively, the perimeter monitoring sensor 13 may include two or more types of sensors selected from a millimeter wave radar sensor, a submillimeter wave radar sensor, a laser radar sensor, and an ultrasonic sensor. There is also no particular limitation on the number of sensors to be installed.

[0231] The locator 14 is not limited to the above example. For example, the locator 14 does not need to have a built-in gyro sensor and acceleration sensor. Specifically, the inertia acquisition unit 142 may receive output signals from an angular velocity sensor and an acceleration sensor provided outside the locator 14 as the vehicle state sensor 11.

[0232] The DCM 15 may be omitted, that is, traffic information may be acquired by the navigation device 16. Alternatively, the navigation device 16 may have a configuration including the locator 14 and the DCM 15.

[0233] The navigation device 16 may be connected to the HMI control device 23 so as to be capable of communicating information via a sub-communication line different from the in-vehicle communication line 10A.

[0234] The navigation device 16 may have a display screen dedicated to displaying the navigation screen, separate from the HMI device 20. Alternatively, the navigation device 16 may be provided as a part of the HMI device 20. Specifically, for example, the navigation device 16 may be integrated with the CID device 22.

[0235] In each country's road traffic system, the conditions for executing autonomous driving, such as the type of autonomous driving and the maximum speed during autonomous driving, may be appropriately considered in accordance with domestic circumstances, etc. Therefore, the above embodiment may be appropriately modified to conform to the specifications of each country's road traffic system. Specifically, for example, in the above embodiment, the driving control ECU 17 is configured to be able to execute both high-speed range autonomous driving, which allows driving at high speeds in specific sections, and traffic jam autonomous driving, which allows driving at medium to low speeds during traffic jams. However, the present invention is not limited to this aspect. That is, for example, the driving control ECU 17 may be capable of executing only high-speed range autonomous driving. Alternatively, the driving control ECU 17 may be capable of executing only traffic jam autonomous driving.

[0236] Automated driving in traffic jams may be possible only on expressways, regardless of whether a specific section is designated. Alternatively, automated driving in traffic jams may be possible only on specific sections. In these cases, vehicle speed control that prevents the driving speed from exceeding the upper limit for automated driving in traffic jams when traffic congestion is temporarily resolved is particularly useful. Furthermore, the driving conditions for automated driving in traffic jams may be appropriately set according to each country's road traffic regulations. Specifically, for example, in the above embodiment, the lower limit for traffic congestion resolution was 10 km / h and the upper limit for traffic congestion resolution was 60 km / h. However, the present invention is not limited to such an embodiment. That is, for example, automated driving in traffic jams may be performed stably without requiring continuous operation at speeds of 60 km / h or less, including when traffic congestion is temporarily resolved. Automated driving under such conditions may be referred to as "low-speed automated driving." "Low-speed automated driving" may be performed regardless of whether traffic congestion occurs.

[0237] In the above embodiment, the driving control ECU 17 is configured to be able to execute vehicle control operations corresponding to SAE Levels 1 to 3. However, the present invention is not limited to this aspect. That is, for example, the present invention may be suitably applied to cases where vehicle control operations corresponding to SAE Levels 1 to 5 can be executed. Furthermore, the driving automation levels or categories in the present invention are not limited to those specified in "SAE J3016." Furthermore, the vehicle speed control unit 173 may be provided as a vehicle control unit that executes vehicle motion control subtasks such as steering control, braking control, etc. in addition to vehicle speed control.

[0238] The DSM 18 may be connected to the HMI control device 23 so as to be capable of communicating information via a sub-communication line different from the in-vehicle communication line 10A.

[0239] The DSM 18 is not limited to a configuration that detects the driver's line of sight or the direction of the driver's face by image recognition. For example, the DSM 18 may be configured to detect the driver's seating posture and the state of gripping the steering wheel by a sensor of a type other than an image sensor.

[0240] The operation unit 19 may have a voice input device that detects the driver's speech.

[0241] The operation unit 19 may be provided as a part of the HMI device 20 .

[0242] The HMI device 20 is not limited to a configuration including the meter panel 21 and the CID device 22. That is, for example, the HMI device 20 may be provided with a head-up display. Therefore, the HMI control device 23 may display the various information and messages displayed on the CID display 221 in the above embodiment on a head-up display instead of or in addition to the CID display 221. There are also no particular limitations on the function and configuration of the head-up display.

[0243] For example, a type of head-up display that superimposes a virtual image on the foreground, including the road surface ahead of the vehicle, may be used. "Superimposed display" refers to displaying related information (e.g., the name of a building) of an object in the foreground so that the related information is superimposed on the object or displayed near the object, thereby associating the object with the related information. Display of a route to the road surface ahead, a direction of travel, traffic information, etc. also fall under the category of "superimposed display."

[0244] The meter 211 and the meter display 212 may be realized by a single display device. In this case, the meter 211 may be provided as display areas at both the left and right ends of a single display device that is a liquid crystal or organic EL display. That is, the meter 211 may be realized by displaying images of a bezel, pointer, scale, etc. corresponding to a tachometer, speedometer, water temperature gauge, etc. Furthermore, the meter display 212 may be provided as a display area other than the meter 211 on such a display device.

[0245] The input device 222 may have a pointing device or the like that is operated by the driver's hand instead of or in addition to the touch panel that is superimposed on the CID display 221. The input device 222 may have a voice input device that detects the driver's speech.

[0246] In the above embodiment, the operation control ECU 17 and the HMI control device 23 have a configuration as a so-called in-vehicle microcomputer including a CPU, etc. However, the present invention is not limited to such a configuration.

[0247] For example, all or part of the driving control ECU 17 may be configured with a digital circuit, such as an ASIC or FPGA, configured to enable the above-described operations. ASIC stands for Application Specific Integrated Circuit. FPGA stands for Field Programmable Gate Array. In other words, the driving control ECU 17 may include both an on-board microcomputer and a digital circuit. The same applies to the HMI control device 23.

[0248] The program according to the present invention, which enables the execution of the various operations, procedures, or processes described in the above embodiments, can be downloaded or upgraded via V2X communication using the DCM 15 or the like. Alternatively, the program can be downloaded or upgraded via a terminal device provided in a vehicle manufacturing plant, repair shop, dealer, or the like. The program can be stored on a memory card, optical disk, magnetic disk, or the like.

[0249] In this manner, each of the above functional configurations and methods may be implemented by a special-purpose computer configured by configuring a processor and memory programmed to execute one or more functions embodied in a computer program. Alternatively, each of the above functional configurations and methods may be implemented by a special-purpose computer configured by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, each of the above functional configurations and methods may be implemented by one or more special-purpose computers configured by combining a processor and memory programmed to execute one or more functions with a processor configured with one or more hardware logic circuits. Furthermore, a computer program may be stored in a computer-readable, non-transitory storage medium as instructions to be executed by a computer. In other words, each of the above functional configurations and methods may be expressed as a computer program including procedures for implementing the same, or as a non-transitory storage medium storing the program.

[0250] The present invention is not limited to the specific functional configurations and operational examples shown in the above embodiment. For example, various messages such as a continuation suggestion message, a warning message ahead, etc. may be displayed on another display device instead of or in addition to the CID display 221. The "other display device" may be, for example, the meter display 212 and / or a head-up display (not shown).

[0251] The continuing operation receiving unit 235 may be provided as an operation receiving unit that receives various input operations other than the continuing operation.

[0252] The design and operation functions of the display screens shown in Fig. 2 etc. may also be changed as appropriate. For example, referring to Fig. 2, a swipe operation within the central display area DA1 may realize a function similar to the type change button DA4 or the content change button DA5.

[0253] The flowcharts shown in Figure 6 and other figures may also be modified as appropriate. For example, if the determination result in step 604 is "NO," a process of waiting for an approval operation for a predetermined time is normally executed, similar to step 910. However, in Figure 6, due to space limitations and to simplify the description in the specification, the illustration and description of such waiting process are omitted.

[0254] In the processing of step 607, the HMI control device 23 may execute a notification of the start of the second task. The notification of the start of the second task may be performed, for example, by displaying text and / or outputting audio.

[0255] The boundary between short-term content and long-term content is not limited to 30 minutes. That is, for example, the boundary between the two may be 45 minutes, 60 minutes, or 75 minutes. Furthermore, the types of second task content are not limited to two, short-term content and long-term content. That is, for example, the types of second task content may be classified into three or more depending on the time required for execution. In this case, in the processing from step 808 onwards in FIG. 8, a determination of whether the continuation time Tc is less than a third threshold value TcZ may be added, where Z is a natural number equal to or greater than 3.

[0256] In the process of determining the autonomous driving start conditions shown in FIG. 7, first, a process of determining whether or not the section conditions are met (i.e., step 701) is executed. However, the present invention is not limited to this aspect. That is, for example, a process of determining whether or not the traffic congestion conditions are met (i.e., step 705) may be executed first. In such a modified example, if the traffic congestion conditions are not met (i.e., step 705 = NO), the driving control ECU 17 advances the process to step 701 to determine whether or not the section conditions are met. The process of step 702 may also be executed when the determination result in step 706 or step 707 is "YES."

[0257] The congestion condition in step 705 is not limited to the above specific example. For example, the congestion condition may be determined to be met when there is another vehicle within a predetermined distance ahead of the host vehicle, there is another vehicle traveling at a low speed in an adjacent lane to the lane in which the host vehicle is traveling, and the traveling speed of the host vehicle is less than a predetermined speed threshold. The predetermined speed threshold is, for example, 10 km / h.

[0258] Step 706 and / or step 707 may be omitted. That is, the fact that the latest traffic information is currently being acquired by the information acquisition unit 171 and / or the fact that the congestion condition has been met for a predetermined period of time may be excluded from the conditions for starting automated driving in congestion.

[0259] Even if congestion information cannot be obtained from traffic information, it is possible to calculate the predicted congestion time Tj. Specifically, for example, a specific driving section where congestion frequently occurs can be statistically identified on map information. Such a driving section is called a "predicted congestion section." Therefore, if the host vehicle enters an actual congestion section while traveling through a predicted congestion section, the predicted congestion time Tj can be calculated based on the average traveling speed of the host vehicle during the current congestion and the remaining distance of the predicted congestion section.

[0260] The display control process of the second task shown in Fig. 11 and the interruptive information presentation process shown in Fig. 15 may be combined. Specifically, for example, in the flowchart shown in Fig. 11, if the determination result of any of steps 1108 to 1110 is "NO", the interruptive information presentation process shown in Fig. 15 may be executed. The processing content of step 1505 in Fig. 15 may be the same as the processing content of step 1111 in Fig. 11.

[0261] "Less than the threshold" can be changed to "less than or equal to the threshold." Similarly, "greater than or equal to the threshold" and "exceed the threshold" are interchangeable. That is, "<" and "≦" are interchangeable. Similarly, ">" and "≧" are interchangeable.

[0262] Similar expressions such as "obtain," "calculate," "estimate," "detect," "sensing," and "determine" may be substituted for each other as appropriate within the scope of technical inconsistency. "Detect" or "detection" and "extract" may also be substituted for each other as appropriate within the scope of technical inconsistency.

[0263] It goes without saying that the elements constituting the above-described embodiments are not necessarily essential unless expressly stated as essential or clearly considered essential in principle. Furthermore, when numerical values such as the number, amount, range, etc. of components are mentioned, the present invention is not limited to those specific numerical values unless expressly stated as essential or clearly limited to specific numerical values in principle. Similarly, when the shape, direction, positional relationship, etc. of components are mentioned, the present invention is not limited to those shapes, directions, positional relationship, etc. unless expressly stated as essential or clearly limited to specific shapes, directions, positional relationship, etc. in principle.

[0264] The modified examples are not limited to the above examples. For example, multiple embodiments may be combined with each other as long as there is no technical contradiction. Also, multiple modified examples may be combined with each other as long as there is no technical contradiction. Furthermore, all or part of the above embodiment and all or part of the modified examples may be combined with each other as long as there is no technical contradiction.

[0265] (summary) (1) The present disclosure, as described in the above embodiments and modifications, includes the following aspects related to an HMI control method and an HMI control program. Note that the following aspects may be combined with one another as long as they are not technically inconsistent.

[0266] According to a first aspect, an HMI control method is a method for controlling an HMI device (20) mounted on an automatically driven vehicle, the method comprising: A process of acquiring the possible continuous time of autonomous driving; a process of presenting, by the HMI device, a second task that can be performed by the driver's seat occupant during autonomous driving based on the acquired continuation time; Includes. The HMI control program is a program executed by an HMI control device (23) configured to control an HMI device (20) mounted on an autonomously driven vehicle, The process executed by the HMI control device is A process of acquiring the possible continuous time of autonomous driving; a process of presenting, by the HMI device, a second task that can be performed by the driver's seat occupant during autonomous driving based on the acquired continuation time; Includes.

[0267] According to a second aspect, the process of acquiring the remaining continuation time includes a process of acquiring the remaining continuation time based on the distance of a travel section that has been set in advance to be capable of automatic driving.

[0268] According to a third aspect, the process of acquiring the possible continuation time includes a process of acquiring the possible continuation time based on a predicted duration of traffic congestion when autonomous driving is initiated due to traffic congestion.

[0269] The fourth aspect further includes a continuation operation acceptance process for accepting an input operation for continuing the second task being executed when a temporary relief of the congestion is detected.

[0270] A fifth aspect of the present invention further includes a behavior acquisition process for acquiring a detection result of a behavior of an occupant in the driver's seat, The continuous operation acceptance process includes a process of validating the input operation when the behavior acquisition process acquires the detection result that the driver's seat occupant is checking the road conditions ahead of the vehicle.

[0271] According to the sixth aspect, The vehicle is capable of autonomous driving within a predetermined speed range during traffic congestion, The continuous operation acceptance process includes a process of notifying a driving control device (17) that controls the driving of the vehicle that the input operation has been accepted, in order to have the driving control device (17) control the traveling speed of the vehicle so that the traveling speed does not exceed the predetermined speed range, when acceptance of the input operation has been accepted.

[0272] A seventh aspect further includes processing for displaying the road conditions by the HMI device when the behavior acquisition unit acquires the detection result indicating that the driver's seat occupant is checking the road conditions.

[0273] The eighth perspective is: a behavior acquisition process for acquiring a detection result of the behavior of the driver's seat occupant; a display control process for displaying the road conditions on the HMI device when the detection result indicating that the driver's seat occupant is checking the road conditions ahead of the vehicle is acquired by the behavior acquisition process; and Further includes:

[0274] According to a ninth aspect, the process of presenting the second task by the HMI device includes a process of presenting a plurality of second tasks having different required execution times in a selectable manner if the available continuation time cannot be obtained.

[0275] The tenth aspect includes processing for disallowing execution of the second task in a travel section that has been set in advance to be capable of automatic driving if the continuation time for travel in that travel section is shorter than a predetermined lower execution limit time.

[0276] An eleventh aspect includes a process of permitting execution of a second task while traveling in traffic congestion, regardless of the length of the possible continuation time acquired based on the predicted duration of traffic congestion.

[0277] According to a twelfth aspect, the process of presenting the second task by the HMI device includes a process of limiting the HMI device that executes the second task while driving in traffic to an in-vehicle device (22) if the possible continuation time acquired based on the predicted duration of traffic congestion is shorter than a predetermined model-limited time.

[0278] According to a twelfth aspect, the process of presenting the second task by the HMI device includes, when the continuation time is shorter than a predetermined time limit, a process of presenting by the HMI device information indicating that the second task is executable, instead of or prior to presenting an executable second task based on the continuation time.

[0279] A thirteenth aspect includes processing for presenting the vehicle's running state or surrounding information by the HMI device when autonomous driving can be continued for a predetermined interrupt waiting time or longer from the start of execution of the second task.

[0280] (2) The present disclosure, as described in the above embodiments and modifications, includes the following aspects related to an operation control method and an operation control program. Note that the following aspects may be applied in combination with one another as long as there is no technical contradiction between them.

[0281] According to a first aspect, a driving control method is a method for controlling driving of an automatically drivable vehicle within a predetermined speed range during traffic congestion, an information acquisition process for acquiring at least a running state of the vehicle; a driving level determination process for determining whether or not autonomous driving can be performed based on the driving state acquired by the information acquisition process; a vehicle speed control process that controls a traveling speed of the vehicle depending on whether autonomous driving is executable or not determined by the driving level determination process; Including, the information acquisition process includes a process of acquiring a traffic congestion state as the driving state and acquiring an execution state of a second task by a driver's seat occupant from an HMI control device (23) that controls an HMI device (20) mounted on the vehicle, The vehicle speed control process includes a process of controlling the driving speed to increase within the specified speed range when the acquired congestion state is a temporary relief of congestion and there is a request to continue the second task being executed. The driving control program is a driving control program executed by a driving control device (17) configured to control driving of an automatically drivable vehicle within a predetermined speed range during traffic congestion, The process executed by the operation control device is an information acquisition process for acquiring at least a running state of the vehicle; a driving level determination process for determining whether or not autonomous driving can be performed based on the driving state acquired by the information acquisition process; a vehicle speed control process that controls a traveling speed of the vehicle depending on whether autonomous driving is executable or not determined by the driving level determination process; Including, the information acquisition process includes a process of acquiring a traffic congestion state as the driving state and acquiring an execution state of a second task by a driver's seat occupant from an HMI control device (23) that controls an HMI device (20) mounted on the vehicle, The vehicle speed control process includes a process of controlling the driving speed to increase within the specified speed range when the acquired congestion state is a temporary relief of congestion and there is a request to continue the second task being executed.

[0282] According to the second viewpoint, the driving level determination process includes a process of determining to perform autonomous driving when the vehicle is traveling on a dedicated road where a minimum speed limit included in the predetermined speed range is set and is in a traffic jam, The vehicle speed control process includes a process of controlling the driving speed to be equal to or higher than the minimum speed limit within the specified speed range when the acquired driving condition is a temporary relief of congestion while driving on the dedicated road and the continuation request is made.

[0283] According to a third aspect, the driving level determination process includes a process of determining to continue autonomous driving when an input operation to continue the second task currently being executed is validly accepted by the HMI device and the continuation request is notified from the HMI device.

[0284] According to a fourth aspect, the driving level determination process includes a process of determining to continue autonomous driving, assuming that the input operation has been validly accepted by the HMI device, when the detection result of the behavior of the driver's seat occupant indicates that the road conditions ahead of the vehicle are being checked.

[0285] According to a fifth aspect, the driving level determination process includes a process of not permitting the execution of autonomous driving in a driving section that has been previously set to be autonomously driven if the continuation time for driving in that driving section is shorter than a predetermined execution lower limit time.

[0286] According to a sixth aspect, the driving level determination process includes a process of permitting the execution of automatic driving while traveling in traffic congestion, regardless of the length of the possible continuation time acquired based on the predicted duration of traffic congestion. [Explanation of symbols]

[0287] 17 Driving control ECU (driving control device) 171 Information Acquisition Department 172 Driving level determination unit 173 Vehicle speed control unit 23 HMI control device 232 Behavior Acquisition Unit 233 Time Acquisition Department 234 Second Task Presentation 235 Continuation operation reception section 236 Display control unit

Claims

1. An HMI control device (23) configured to control an HMI device (20) mounted on an autonomously driven vehicle, a second task presentation unit (234) that presents a second task that can be performed by a driver's seat occupant during automatic driving by the HMI device; and when the second task is being executed and a duration of the second task from a predetermined continuation reference point is equal to or longer than a threshold time, the HMI device presents information regarding the presence of other vehicles around the vehicle. HMI control device.

2. An HMI control program executed by an HMI control device (23) configured to control an HMI device (20) mounted on an autonomously driven vehicle, The process executed by the HMI control device is A process of presenting a second task that can be performed by a driver's seat occupant during autonomous driving by the HMI device; a process of presenting information on the presence of other vehicles around the vehicle by the HMI device when the second task is being executed and the duration of the second task from a predetermined continuation reference point in time is equal to or longer than a threshold time; An HMI control program including:

Images