HMI control device, hmi control method, hmi control program, and driving control device

Abstract

The present application provides an HMI control device, an HMI control method, an HMI control program, and a driving control device. The HMI control device (23) configured to control an HMI device (20) mounted on an automatically drivable vehicle includes a time acquisition unit (233) and a second task prompting unit (234). The time acquisition unit acquires a drivable time of automatic driving. The second task prompting unit prompts, through the HMI device, a second task that a passenger in a driver seat can perform in automatic driving, based on the drivable time acquired by the time acquisition unit.

Description

[0001] Cross-references to related applications

[0002] This application is based on Japanese Patent Application No. 2019-214229 filed on November 27, 2019, and Japanese Patent Application No. 2020-184660 filed on November 4, 2020, the contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to an HMI control device, an HMI control method, and an HMI control program for controlling an HMI device installed in an autonomous vehicle. HMI is an abbreviation for Human-Machine Interface. Additionally, this disclosure relates to a driving control device configured to control the driving of an autonomous vehicle. Background Technology

[0004] Various autonomous driving systems for automobiles and other vehicles have been proposed (see, for example, Patent Document 1). In autonomous driving, the driver, who is a passenger in the driver's seat of the vehicle, is free to perform a secondary task. The so-called secondary task is a task other than driving operations performed by the driver. Specifically, the secondary task includes, for example, operating a mobile communication terminal, watching or listening to video content, etc. The secondary task is also referred to as "non-driving task" or "secondary activity".

[0005] Patent Document 1: Japanese Patent Application Publication No. 2017-107502

[0006] The time required to perform a second task varies. For example, when operating a mobile communication terminal such as a mobile phone, the execution time is relatively short. In contrast, when watching a movie, the execution time may be long. Therefore, convenience may be compromised, for example, if a second task requiring a long execution time is interrupted midway, or if there are frequent requests to select a second task requiring a short execution time. Summary of the Invention

[0007] This disclosure was made in view of the circumstances illustrated above. Specifically, this disclosure provides, for example, a technique for more comfortably utilizing a second task in autonomous driving.

[0008] According to one aspect of this disclosure, the HMI control device is configured to control an HMI device mounted on an autonomous vehicle.

[0009] The HMI control device has the following features:

[0010] The time acquisition unit acquires the duration of autonomous driving; and

[0011] The second task prompting unit, based on the available duration obtained by the time acquisition unit, prompts the passenger in the driver's seat to perform a second task in autonomous driving via the HMI device.

[0012] According to another aspect of this disclosure, the driving control device is configured to control the driving of a vehicle capable of automatic driving within a specified speed range during traffic congestion.

[0013] The driving control device has the following features:

[0014] The information acquisition department shall at least acquire the driving status of the aforementioned vehicles;

[0015] The driving level determination unit determines whether autonomous driving can be performed based on the driving state obtained by the information acquisition unit.

[0016] The vehicle speed control unit controls the vehicle's speed based on whether automatic driving can be performed, as determined by the driving level determination unit.

[0017] The aforementioned information acquisition unit acquires the traffic congestion status as the aforementioned driving state, and acquires the execution status of the second task performed by the passenger in the driver's seat from the HMI control device that controls the HMI device mounted on the aforementioned vehicle.

[0018] If the traffic congestion status is temporarily relieved and there is a request to continue the second task being performed, the speed control unit will increase the driving speed within the specified speed range.

[0019] According to another aspect of this disclosure, the HMI control method is a method for controlling an HMI device installed in an autonomous vehicle, comprising the following steps or processes:

[0020] Obtain the duration of autonomous driving; and

[0021] Based on the obtained duration, the HMI device prompts the passenger in the driver's seat with a second task that can be performed during autonomous driving.

[0022] According to another aspect of this disclosure, the HMI control program is a program executed by an HMI control device configured to control an HMI device mounted in an autonomous driving vehicle.

[0023] The processes performed by the aforementioned HMI control device include:

[0024] Processing to obtain the duration of autonomous driving; and

[0025] Based on the obtained duration, the HMI device prompts the passenger in the driver's seat to perform a second task during autonomous driving.

[0026] Furthermore, in the various sections of the application documents, there are instances where elements are marked with reference numerals in parentheses. In such cases, the reference numerals merely indicate an example of the correspondence between that element and the specific structure described in the embodiments described later. Therefore, this disclosure is not limited in any way by the reference numerals used. Attached Figure Description

[0027] Figure 1 This is a block diagram illustrating the schematic structure of an in-vehicle system that includes a driving control ECU and an HMI control device.

[0028] Figure 2 It means Figure 1 A schematic diagram illustrating an example of the display method in an HMI device.

[0029] Figure 3 It means Figure 1 A schematic diagram of another example of the display method in the HMI device shown.

[0030] Figure 4 It means Figure 1 A schematic diagram illustrating another example of the display method in an HMI device.

[0031] Figure 5 It means Figure 1 A schematic diagram illustrating another example of the display method in an HMI device.

[0032] Figure 6 It means Figure 1 The flowchart shows an example of an operation of the vehicle system.

[0033] Figure 7 It means Figure 1 The flowchart shown is an example of an action of the driving control ECU.

[0034] Figure 8 It means Figure 1 A flowchart illustrating an example of an operation of the HMI control device.

[0035] Figure 9 It means Figure 1 The flowchart shown is an example of an action of the driving control ECU.

[0036] Figure 10 It means Figure 1 A schematic diagram illustrating another example of the display method in an HMI device.

[0037] Figure 11 It means Figure 1 A flowchart of another example of the operation of the HMI control device shown.

[0038] Figure 12 It means Figure 1 The flowchart shows another example of the operation of the HMI control device.

[0039] Figure 13 It means Figure 1 The flowchart shows another example of the operation of the vehicle system.

[0040] Figure 14 It means Figure 1 The flowchart shows another example of the operation of the driving control ECU.

[0041] Figure 15 It means Figure 1 The flowchart shows another example of the operation of the HMI control device. Detailed Implementation

[0042] (Implementation Method)

[0043] Hereinafter, embodiments of the present disclosure will be described based on the accompanying drawings. Furthermore, there is a concern that inserting various modifications applicable to one embodiment into a series of descriptions related to that embodiment could hinder understanding of the embodiment. Therefore, modifications are not described in the middle of a series of descriptions related to that embodiment, but rather described in detail later.

[0044] (structure)

[0045] Reference Figure 1 The vehicle system 10 is installed in an autonomous driving vehicle. "Autonomous driving" refers to a level of driving automation equivalent to Levels 3 to 5 in the SAE International standard "SAE J3016," where the vehicle system 10 is responsible for all dynamic driving tasks. SAE is an abbreviation for the Society of Automotive Engineers. The definition of "dynamic driving tasks" will be described later. Hereinafter, Level X in "SAE J3016" will be referred to as "SAE Level X." X can be any value from 0 to 5. In this embodiment, the vehicle system 10 is configured to be installed in a car, and the driving automation level in that car is set to any level from SAE Level 0 to 3. Hereinafter, the vehicle equipped with the vehicle system 10 will be referred to as "this vehicle."

[0046] (System Overall Structure)

[0047] The vehicle system 10 is a vehicle network comprising a vehicle communication line 10A and multiple nodes interconnected via the vehicle communication line 10A, configured to perform various vehicle controls and accompanying display actions during vehicle operation. The vehicle system 10 is configured according to communication standards such as CAN (International Registered Trademark: International Registration No. 1048262A). CAN (International Registered Trademark) is an abbreviation for Controller Area Network.

[0048] The vehicle system 10 includes a vehicle status sensor 11, an external status sensor 12, a surrounding monitoring sensor 13, a locator 14, a DCM 15, a navigation device 16, a driving control ECU 17, a DSM 18, an operating unit 19, and an HMI device 20. DCM is an abbreviation for Data Communication Module. ECU is an abbreviation for Electronic Control Unit. DSM is an abbreviation for Driver Status Monitor. The vehicle status sensor 11 to the HMI device 20 are interconnected via vehicle communication line 10A.

[0049] HMI device 20 includes instrument panel 21, CID device 22, and HMI control device 23. CID is an abbreviation for Center Information Display. Instrument panel 21 and CID device 22 are communicatively connected to HMI control device 23 via a sub-communication line different from vehicle communication line 10A. HMI control device 23 is configured as a node connected to vehicle communication line 10A.

[0050] (Various sensors)

[0051] The vehicle status sensor 11 is configured to generate outputs corresponding to various quantities related to the driving state of the vehicle. These "various quantities related to the driving state" include, for example, acceleration operation quantities, braking operation quantities, gear position, steering angle, and other quantities related to the driver's driving operation state. The term "driver" refers to the occupant responsible for or performing dynamic driving tasks; typically, it refers to the occupant sitting in the driver's seat of the vehicle, also known as the "driver's seat passenger." Furthermore, these "various quantities related to the driving state" include, for example, physical quantities related to the vehicle's behavior such as speed, angular velocity, longitudinal acceleration, and lateral acceleration. In other words, the term "vehicle status sensor 11" is a general term for known sensor classes required for driving control, such as accelerometer opening sensors, steering angle sensors, wheel speed sensors, angular velocity sensors, and acceleration sensors, for the sake of illustration and explanation. The vehicle status sensor 11 is configured to provide check outputs to various components, including the driving control ECU 17, via the vehicle communication line 10A.

[0052] The external condition sensor 12 is configured to generate outputs corresponding to various quantities related to the natural environment surrounding the vehicle. "Various quantities related to the natural environment" include, for example, physical quantities such as external temperature, rainfall, and illuminance. That is, "external condition sensor 12" is a general term for known sensor types such as external temperature sensors, rain sensors, and illuminance sensors for the sake of illustration and explanation. The external condition sensor 12 is configured to provide check outputs to various components such as the driving control ECU 17 via the vehicle communication line 10A.

[0053] The surrounding monitoring sensor 13 is configured to detect the traffic environment around the vehicle, primarily in addition to the traffic environment detectable by the external state sensor 12. Specifically, the surrounding monitoring sensor 13 is configured to detect moving and stationary objects within a defined detection range around the vehicle. "Moving objects" include pedestrians, cyclists, animals, and other vehicles in motion. "Stationary objects" include road debris, guardrails, curbs, parked vehicles, road signs, and road markings, as well as roadside structures (e.g., median strips, buildings, etc.). The surrounding monitoring sensor 13 may also be referred to as an "ADAS sensor." ADAS is an abbreviation for Advanced Driver-Assistance Systems. In this embodiment, the surrounding monitoring sensor 13, as a structure for detecting moving and stationary objects, includes a front-facing camera 131 and a radar sensor 132.

[0054] The front-facing camera 131 is configured to capture images of the front and front sides of the vehicle. In this embodiment, the front-facing camera 131 is a digital camera device equipped with an image sensor such as CCD or CMOS. CCD is an abbreviation for Charge Coupled Device. CMOS is an abbreviation for Complementary MOS.

[0055] Radar sensor 132 is a millimeter-wave radar sensor, sub-millimeter-wave radar sensor, or lidar sensor configured to transmit and receive radar waves, and is installed on the front of the vehicle body. Radar sensor 132 is configured to output signals corresponding to the position of the reflection point and the relative speed. The "reflection point" is a point on the surface of an object surrounding the vehicle that is presumed to reflect radar waves. The "relative speed" is the relative speed of the reflection point, i.e., the object reflecting the radar waves, relative to the vehicle.

[0056] (Locator)

[0057] The locator 14 is configured to determine the vehicle's high-precision position information through so-called composite positioning. Specifically, the locator 14 includes a GNSS receiver 141, an inertial acquisition unit 142, a high-precision map DB 143, and a locator ECU 144. GNSS is an abbreviation for Global Navigation Satellite System. DB is an abbreviation for database. "High-precision position information" refers to position information with an SAE level of 2 or higher, sufficient for advanced driver assistance or autonomous driving, specifically, position information with an error of less than 10 cm.

[0058] GNSS receiver 141 is configured to receive positioning signals transmitted from multiple positioning satellites, i.e., artificial satellites. In this embodiment, GNSS receiver 141 is configured to receive positioning signals from positioning satellites in at least one of the following satellite positioning systems: GPS, QZSS, GLONASS, Galileo, IRNSS, and BeiDou Navigation Satellite System. GPS is an abbreviation for Global Positioning System. QZSS is an abbreviation for Quasi-Zenith Satellite System. GLONASS is an abbreviation for Global Navigation Satellite System. IRNSS is an abbreviation for Indian Regional Navigation Satellite System.

[0059] The inertial acquisition unit 142 is configured to acquire the acceleration and angular velocity acting on the vehicle. In this embodiment, the inertial acquisition unit 142 is configured as a 3-axis gyroscope sensor and a 3-axis accelerometer sensor built into a box-shaped frame inside the positioner 14.

[0060] The high-precision map DB143 is primarily constructed using non-volatile, rewritable memory to allow for the rewritable storage of high-precision map information and to retain its contents during power-off. Non-volatile, rewritable memory includes, for example, hard disks, EEPROMs, and flash ROMs. EEPROM stands for Electronically Erasable and Programmable ROM. ROM stands for Read Only Memory. High-precision map information can also be referred to as high-precision map data. High-precision map information contains map information with higher accuracy than that used in conventional car navigation systems, which correspond to positional errors of approximately several meters. Specifically, the high-precision map DB143 stores information such as three-dimensional road shape information, lane number information, and restriction information—information that can be used for advanced driver assistance or autonomous driving—according to standards such as ADASIS. ADASIS stands for Advanced Driver Assistance Systems Interface Specification.

[0061] The locator ECU 144 is configured as a so-called vehicle-mounted microcomputer, equipped with a CPU, ROM, RAM, input / output interfaces, etc. (not shown). CPU is an abbreviation for Central Processing Unit. RAM is an abbreviation for Random Access Memory. The locator ECU 144 is configured to determine the vehicle's position and orientation based on the positioning signal received by the GNSS receiver 141, the acceleration and angular velocity acquired by the inertial acquisition unit 142, and the driving speed acquired by the vehicle status sensor 11. Furthermore, the locator 14 is configured to provide the position and orientation determination results determined by the locator ECU 144 to various units such as the navigation device 16, the driving control ECU 17, and the HMI control device 23 via the vehicle communication line 10A.

[0062] (DCM)

[0063] The DCM15 is an in-vehicle communication module configured to communicate with base stations around the vehicle via wireless communication standards such as LTE or 5G. LTE stands for Long Term Evolution, and 5G stands for 5th Generation. Specifically, for example, the DCM15 is configured to obtain the latest high-precision map information from a cloud-based detection server. Furthermore, the DCM15, in cooperation with the locator ECU144, stores the acquired latest high-precision map information in a high-precision map database DB143. Additionally, the DCM15 is configured to obtain traffic information, such as traffic congestion information, from the aforementioned detection server and / or a designated database. This traffic information is also referred to as "road traffic information."

[0064] (Navigation device)

[0065] The navigation device 16 is configured to calculate a predetermined driving route from the vehicle's current location to a specified destination. In this embodiment, the navigation device 16 is configured to calculate the predetermined driving route based on the destination set by the vehicle's driver, high-precision map information obtained from the locator 14, and the vehicle's location and orientation information obtained from the locator 14. Furthermore, the navigation device 16 is configured to provide various information, including the route information as a result of the calculation, to the driving control ECU 17 and the HMI control device 23, etc., via the vehicle communication line 10A. That is, the navigation device 16 causes the HMI device 20 to display a navigation screen for map display, route display, etc.

[0066] (Driving control ECU)

[0067] The driving control ECU 17 is configured to control the driving of the vehicle based on signals and information acquired from the vehicle status sensor 11, the external status sensor 12, the surrounding monitoring sensor 13, the locator 14, etc. That is, the driving control ECU 17 is configured to execute prescribed driving control actions. In this embodiment, the "prescribed driving control actions" correspond to vehicle control actions of SAE levels 1 to 3, i.e., dynamic driving task execution actions. The definition of "dynamic driving task" is based on "SAE J3016". That is, the so-called "dynamic driving task" refers to all operational and tactical functions that need to be performed in real time when operating a vehicle in road traffic, excluding strategic functions. "Strategic functions" include trip planning, route selection, etc.

[0068] The driving control ECU 17, as the driving control device disclosed herein, is configured to control the driving of a vehicle capable of SAE Level 3 automated driving during driving in a specific area and / or during traffic congestion. The term "specific area" refers to a pre-set driving area where SAE Level 3 automated driving is possible. Within this specific area, automated driving is possible within a prescribed high-speed range. The prescribed high-speed range is, for example, above 60 km / h but below the legal speed limit. Hereinafter, this SAE Level 3 automated driving capability, capable of driving within a high-speed range in a specific area, will be referred to as "high-speed range automated driving." On the other hand, SAE Level 3 automated driving performed during traffic congestion will be referred to as "automated driving during traffic congestion." In SAE Level 3 automated driving, the driver has no obligation to monitor the vehicle's surroundings.

[0069] The driving control ECU 17 has a structure that serves as a so-called vehicle-mounted microcomputer, including components such as a CPU (not shown), ROM, non-volatile rewritable memory, RAM, and input / output interfaces. Specifically, the driving control ECU 17 includes an information acquisition unit 171, a driving level determination unit 172, and a vehicle speed control unit 173, which are functional structures or units implemented on the vehicle-mounted microcomputer.

[0070] The information acquisition unit 171 is configured to acquire at least the driving status of the vehicle. "Driving status" includes driving conditions, natural environment, traffic environment, etc., detected or acquired by the vehicle status sensor 11, external status sensor 12, and surrounding monitoring sensor 13. Furthermore, the information acquisition unit 171 is configured to acquire the vehicle's current location and traffic information on the road the vehicle is currently traveling on. That is, the information acquisition unit 171 acquires information required for vehicle control corresponding to SAE levels 1 to 3 from the vehicle status sensor 11, external status sensor 12, surrounding monitoring sensor 13, locator 14, and DCM 15.

[0071] The driving level determination unit 172 is configured to determine whether autonomous driving can be performed based on the driving status and other information obtained by the information acquisition unit 171. That is, the driving level determination unit 172 sets the driving automation level in the vehicle to any one of SAE levels 0 to 3. Specifically, the driving level determination unit 172 determines whether the conditions for starting autonomous driving are met, and if the conditions are met and the driver's approval is received, it starts SAE level 3 autonomous driving. Furthermore, if the conditions for continuing autonomous driving are not met during SAE level 3 autonomous driving, the driving level determination unit 172 performs the controls necessary to terminate such autonomous driving. The driving control ECU 17 is configured to provide the driving automation level setting result determined by the driving level determination unit 172 to various units such as the HMI control device 23 via the vehicle communication line 10A.

[0072] The vehicle speed control unit 173 is configured to control the vehicle's speed based on whether autonomous driving can be performed, as determined by the driving level determination unit 172. Specifically, when the driving automation level is set to any one of SAE levels 1 to 3, the vehicle speed control unit 173 performs driving speed control corresponding to the set driving automation level. Furthermore, the driving control ECU 17 also includes a steering control unit (not shown) and a braking control unit that perform steering control, braking control, etc., corresponding to the set driving automation level. In other words, the driving control ECU 17 has a structure that functions as a vehicle control unit performing vehicle motion control sub-tasks such as vehicle speed control, steering control, and braking control, corresponding to the driving automation level determined by the driving level determination unit 172.

[0073] (DSM)

[0074] The DSM18 is configured to check the driver's state through image recognition based on images captured by an in-vehicle camera, which is equipped with an image sensor such as a CCD or CMOS. Furthermore, based on the driver's state check results, the DSM18 is configured to issue warnings related to negligent driving using a speaker (not shown) located on the dashboard or other similar device. The DSM18 is also configured to provide the driver state check results to the driving control ECU 17 and the HMI control unit 23, among other components, via the vehicle communication line 10A.

[0075] (Operations Department)

[0076] The operating unit 19 is configured to accept manual input operations from the driver, other than motion control inputs in the vehicle. "Motion control inputs" are control inputs directly related to lateral or longitudinal motion control during manual driving of the vehicle; specifically, they include operations such as gear shifting, acceleration, braking, and steering. Furthermore, the operation of the traffic light switch 191 in the lane change function included in SAE Level 2, which is linked to the operation of the traffic light switch 191, differs from the aforementioned "motion control inputs."

[0077] Specifically, the operation unit 19 includes a turn signal switch 191. The turn signal switch 191 is configured to output a signal corresponding to the operating state of the operating lever (not shown) provided on the steering column. Additionally, the operation unit 19 includes a turn signal switch (not shown) provided on the spokes of the steering wheel. Furthermore, the operation unit 19 includes seat position adjustment switches, seating posture adjustment switches, window opening / closing switches, etc., corresponding to each passenger seat, including the driver's seat. The operation unit 19 is configured to provide the results of input operations performed by the driver to various units, such as the driving control ECU 17 and the HMI control device 23, via the vehicle communication line 10A.

[0078] (HMI device)

[0079] The HMI device 20 is configured to provide the driver with various information related to the vehicle, at least visually, and to accept driver input operations corresponding to the provided information. In this embodiment, the HMI device 20 mounted on the autonomous vehicle is configured to perform various information prompts related to autonomous driving and to accept input operations performed by the driver. "Information prompts" include, for example, various guidance, input operation instructions, input operation content reports, warnings, etc.

[0080] As described above, the HMI device 20 includes a dashboard 21 (not shown) and a CID device 22 installed on a dashboard (not shown). That is, in this embodiment, the HMI device 20 has a structure called a "dashboard HMI" installed on a dashboard (not shown). Additionally, the HMI device 20 includes a speaker (not shown) for performing voice-based information prompts.

[0081] The instrument panel 21 includes an instrument cluster 211, an instrument display 212, and an instrument switch 213. The instrument cluster 211 is configured to display information such as the vehicle's speed, coolant temperature, and fuel level. The instrument display 212 is an information display unit located in the center of the instrument panel 21, configured to display various information such as date and time, outside temperature, driving distance, and radio reception station. The instrument switch 213 is configured to handle various operations related to the display status or content of the instrument cluster 211 and / or the instrument display 212, such as resetting the odometer.

[0082] The CID device 22 is installed on the instrument panel. The CID device 22 is configured to display a navigation display screen, such as a map display and route display, for the navigation device 16. Furthermore, the CID device 22 is also configured to display information and content different from such a navigation display screen. Specifically, the CID device 22 is configured to perform displays related to driving modes such as "Comfort," "Normal," "Sport," and "Loop." Additionally, the CID device 22 is configured to perform displays related to secondary tasks that the driver can perform in SAE Level 3 automated driving.

[0083] The CID device 22 includes a CID display 221, an input device 222, and a CID switch 223. The CID display 221 is positioned approximately at the center of the dashboard in the vehicle width direction in a manner that allows at least the driver to visually confirm it. The CID display 221 has a structure that serves as a display device, wherein the display device is a liquid crystal display or an organic EL display. EL is an abbreviation for Electroluminescence.

[0084] Input device 222 is a transparent touch panel configured to cover CID display 221. That is, input device 222 is configured to allow the driver or other personnel to visually confirm the display on CID display 221 and to respond to input operations performed by the driver or other personnel. CID switch 223 has multiple manually operated switches disposed around CID display 221 and input device 222.

[0085] Figures 2-5 This refers to an example of a display screen in CID display 221 related to a second task in autonomous driving. Figure 2 as well as Figure 3 This refers to the situation of watching a movie as an example of long-duration content. Figure 4 This represents an example of a situation where mobile communication terminal operations are permitted to be performed within a short period of time. Figure 5 The menu screen displays multiple options with varying execution times.

[0086] "Long-duration content" refers to registered content that requires a relatively long execution time (e.g., more than 30 minutes). "Registered content" refers to content that has been pre-registered or made available to users of the vehicle, including the manufacturer, seller, or driver, and can be used as a secondary task. "Short-duration content" refers to registered content that requires a shorter execution time than long-duration content (e.g., less than 30 minutes). "Content" includes, for example, video content and text content. Examples of "video content" include movies, concert videos, music videos, television broadcasts, games, and video calls. Examples of "text content" include books, magazines, and text broadcasts.

[0087] Reference Figures 2-5 The CID display 221 has a first display area DA and a second display area DB. The first display area DA is the area that mainly displays video content and text content with a relatively large screen size, and can also be called the "main display area".

[0088] The second display area DB is positioned above the first display area DA in the vertical direction of the screen. The term "vertical direction of the screen" refers to the vertical direction relative to the driver or other person directly facing the CID display 221. Figures 2-5 The second display area DB is located at the top of the CID display 221's screen and can display various text information such as traffic information. The second display area DB can also be called an "information display bar." Furthermore, the second display area DB can be placed anywhere within the CID display 221's screen. Alternatively, the second display area DB can be placed on a display device adjacent to the CID display 221.

[0089] Reference Figure 2 as well as Figure 3 The first display area DA comprises a central display area DA1, a left edge area DA2, and a right edge area DA3. The central display area DA1 is used to display video and text content, and is located between the left edge area DA2 and the right edge area DA3 in the left-right direction of the screen. The term "left-right direction of the screen" refers to the left-right direction relative to the driver or other person facing the CID monitor 221. Figures 2-5 The left and right directions in the middle.

[0090] The left edge area DA2 is positioned to the left of the central display area DA1 in the horizontal direction of the screen. Within the left edge area DA2, there is an arrow-shaped category change button DA4. The category change button DA4 is an operation button used to change the content category between movies, music videos, television broadcasts, games, books, etc., and is located approximately in the center of the left edge area DA2.

[0091] The right edge area DA3 is located to the right of the central display area DA1 in the horizontal direction of the screen. A category change button DA4 is also located in the right edge area DA3. The category change button DA4 in the right edge area DA3 is configured to operate in the reverse order of the category change button DA4 in the left edge area DA2, allowing users to change the category of content.

[0092] A content change button DA5 is also located in the right edge area DA3. The content change button DA5 is an operation button for changing content within the same time category, and it is located at the lower end of the vertical axis of the right edge area DA3. "Time Category" refers to whether the content is short-term or long-term.

[0093] like Figure 3 As shown, the right edge area DA3 displays the approval button DA6 as needed. The approval button DA6 is an operation button for various approval operations performed by the driver, and it is located at the upper part of the right edge area DA3 in the vertical direction. In addition, depending on the content of the approval operation, the approval button DA6 displays the strings such as "OK", "START", "CONTINUE" inside the rectangular button shape.

[0094] The second display area DB has a status display area DB1 and an information display area DB2. The status display area DB1 is located at the left end of the screen in the left-right direction to display information corresponding to the current driving automation level of the vehicle. The information displayed in the status display area DB1 is, for example, "Dedicated Zone Driving" in the case of high-speed range automated driving and "Traffic Congestion Driving" in the case of traffic congestion automated driving.

[0095] The information display area DB2 is positioned to the right of the status display area DB1 in the horizontal direction of the screen. Information display area DB2 displays various information and messages related to traffic and driving status as needed. Specifically, the display in information display area DB2 includes an information display box DB3 and a message display box DB4. Information display box DB3 displays traffic information such as traffic congestion information. Message display box DB4 displays various messages or warnings other than traffic information.

[0096] Reference Figure 4 As a second task, when operation of a mobile communication terminal such as a mobile phone is permitted, the first display area DA displays the mobile terminal permission screen DA7. The mobile terminal permission screen DA7 is configured as a text- and / or image-based display that allows the driver to operate the mobile communication terminal. Furthermore, in Figure 4 In the displayed screen, the second display area DB has the same display function as described above. Additionally, when the input device 222 detects that any location on the mobile terminal's permitted screen DA7 has been touched, the CID display 221 displays... Figure 5 The menu screen shown.

[0097] Reference Figure 5In the menu screen, the first display area DA shows multiple content selection buttons DA8 and multiple category selection buttons DA9. Content selection buttons DA8 use thumbnails to highlight frequently used content from various categories such as movies, concert videos, television broadcasts, and games. Category selection buttons DA9 are correspondingly set for each content category, such as movies, concert videos, television broadcasts, and games. Furthermore, in... Figure 5 In the menu screen shown, the second display area DB has the same display function as described above.

[0098] (HMI control device)

[0099] Refer again Figure 1 The HMI control unit 23 is configured to control the operation of the HMI device 20. That is, the HMI control unit 23 is configured as the HCU that controls the operation of the instrument panel 21 and CID device 22 included in the HMI device 20. HCU is an abbreviation for HMI Control Unit (Human Machine Interface Control Unit).

[0100] The HMI control device 23 has a structure that serves as a so-called vehicle-mounted microcomputer, including components such as a CPU (not shown), ROM, non-volatile rewritable memory, RAM, and input / output interfaces. The HMI control device 23 includes a vehicle information acquisition unit 231, an action acquisition unit 232, a time acquisition unit 233, a second task prompting unit 234, a continuation operation acceptance unit 235, and a display control unit 236, which are functional structures or units implemented on a microcomputer.

[0101] The vehicle information acquisition unit 231 acquires the vehicle's driving status as detected or obtained by the vehicle status sensor 11, etc. Additionally, the vehicle information acquisition unit 231 acquires the vehicle's current location, the planned driving route, and traffic information along the planned driving route, including the road the vehicle is currently traveling on. Furthermore, the vehicle information acquisition unit 231 obtains the driving automation level determination result from the driving control ECU 172.

[0102] The behavior acquisition unit 232 is configured to acquire the results of the driver's behavior checks. That is, the behavior acquisition unit 232 acquires the behavior check results of the DSM18 from the DSM18 via the vehicle communication line 10A. In addition, the behavior acquisition unit 232 can also acquire the behavior check results caused by the operation of the input device 222.

[0103] The time acquisition unit 233 is configured to acquire the duration of autonomous driving based on the information acquired by the vehicle information acquisition unit 231. Specifically, when driving within a pre-defined autonomous driving zone, i.e., a specific zone, the time acquisition unit 233 acquires the duration based on the distance of this specific zone. On the other hand, when autonomous driving begins due to traffic congestion, the time acquisition unit 233 acquires the duration based on the predicted duration of such traffic congestion.

[0104] The second task prompting unit 234 is configured to prompt the driver for a second task that can be performed during autonomous driving via the HMI device 20 based on the available duration acquired by the time acquisition unit 233. Specifically, the second task prompting unit 234 displays any one of the following screens on the CID display 221: a long-duration content prompting screen, a short-duration content prompting screen, and a menu screen, based on the acquisition result of the available duration. The "long-duration content prompting screen" and the "short-duration content prompting screen" correspond to... Figure 2 or Figure 3 The display screens are as follows: A long-duration content prompt screen is displayed in the central display area DA1, indicating or showing one of the long-duration content segments that can be completed within the specified duration. Conversely, a short-duration content prompt screen is displayed in the central display area DA1, indicating or showing one of the short-duration content segments that can be completed within the specified duration.

[0105] Specifically, in this embodiment, if the acquired duration is less than a predetermined threshold, the second task prompting unit 234 displays a short-duration content prompt screen on the CID display 221. Conversely, if the acquired duration is greater than or equal to the predetermined threshold, the second task prompting unit 234 displays a long-duration content prompt screen on the CID display 221. Furthermore, if the time acquisition unit 233 is unable to acquire the duration for some reason, the second task prompting unit 234 can selectively prompt multiple second tasks with different execution times by displaying a menu screen on the CID display 221.

[0106] The Continuation Operation Acceptance Unit 235 is configured to accept input operations for continuing the ongoing second task, i.e., Continuation Operation, when a temporary lifting of traffic congestion is detected. "When a temporary lifting of traffic congestion is detected" is, for example, when the vehicle's current position is within a traffic congestion zone based on acquired traffic congestion information, and the surrounding monitoring sensor 13 detects an acceleration zone at a predetermined distance or more ahead of the vehicle. Alternatively, "When a temporary lifting of traffic congestion is detected" is, for example, when the vehicle's current position is within a traffic congestion zone based on acquired traffic congestion information, and the vehicle's speed is detected to be at or above the predetermined lower lifting speed limit (e.g., 10 km / h) but below the predetermined upper lifting speed limit (e.g., 60 km / h). Furthermore, if the vehicle's speed is above or above the aforementioned upper lifting speed limit, the traffic congestion is detected as completely lifted, resulting in a driving automation level where the second task cannot be performed.

[0107] Specifically, when a temporary easing of traffic congestion is detected, the continuation operation acceptance unit 235 displays traffic information indicating the temporary easing of traffic congestion on the information display box DB3, and also displays the approval button DA6. Additionally, the continuation operation acceptance unit 235 checks for touch input operations in the input device 222 at the location corresponding to the display position of the approval button DA6.

[0108] Furthermore, in this embodiment, when a temporary easing of traffic congestion is detected, the continuation operation acceptance unit 235 displays a message such as "Attention Ahead" in the message display box DB4 to help the driver confirm the road conditions ahead, which is the vehicle's destination. Moreover, if the behavior acquisition unit 232 obtains a check result indicating that the driver is confirming the road conditions at the vehicle's destination, the continuation operation acceptance unit 235 activates the continuation operation. Furthermore, having activated the continuation operation acceptance, the continuation operation acceptance unit 235 notifies the driving control ECU 17 that the continuation operation has been activated in order to control the vehicle's speed by controlling the driving control ECU 17 to a speed not exceeding a predetermined range.

[0109] The display control unit 236 is configured to perform display controls other than those in the second task prompt unit 234 and the continue operation acceptance unit 235. Specifically, for example, when traffic information such as traffic congestion information changes, the display control unit 236 displays the latest information via the information display box DB3. Furthermore, the display control unit 236 performs various display controls based on the CID device 22 at SAE levels 0-2 of driving automation, where the second task cannot be performed.

[0110] (Action Summary: First Implementation Method)

[0111] Hereinafter, the operation of the driving control ECU 17 and HMI control device 23 of this embodiment, as well as the methods and procedures executed by these devices, will be described together with the effects of the structure of this embodiment.

[0112] The information acquisition unit 171 in the driving control ECU 17 acquires various information including the driving status of the vehicle. Specifically, the information acquisition unit 171 acquires driving status and traffic environment information from the vehicle status sensor 11, the external status sensor 12, and the surrounding monitoring sensor 13. Additionally, the information acquisition unit 171 acquires high-precision location information of the vehicle, as well as traffic restriction information regarding the vehicle's location and its surroundings, from the locator 14. Furthermore, the information acquisition unit 171 acquires a predetermined driving route from the vehicle's current location to a designated destination from the navigation device 16.

[0113] In addition, the information acquisition unit 171 acquires traffic congestion status as the driving status from the surrounding monitoring sensor 13 and the DCM 15. The so-called "traffic congestion status" refers to the occurrence of traffic congestion at the vehicle's destination within the vehicle's predetermined driving path. That is, the "traffic congestion status" includes, for example, the position of the beginning of the traffic congestion, the position of the end of the traffic congestion, and the length of the traffic congestion within the vehicle's predetermined driving path. Furthermore, the "traffic congestion status" includes whether the vehicle has entered a traffic congestion and whether the traffic congestion has been temporarily relieved.

[0114] Additionally, the information acquisition unit 171 acquires the driver's status from the DSM 18. Furthermore, the information acquisition unit 171 acquires the operation status of the operation unit 19 performed by the driver from the operation unit 19. Also, the information acquisition unit 171 acquires the execution status of the second task performed by the driver from the HMI control device 23.

[0115] The driving level determination unit 172 determines whether autonomous driving can be performed based on various information, such as the driving status, acquired by the information acquisition unit 171. Specifically, the driving level determination unit 172 determines the start conditions for driving automation levels equivalent to SAE levels 1 to 3 based on various information acquired by the information acquisition unit 171. For example, in this embodiment, the start conditions for autonomous driving at SAE level 3 include driving in a specific area or during traffic congestion, and a traffic environment suitable for autonomous driving. "A traffic environment suitable for autonomous driving" includes, for example, the ability to confirm the traffic environment well through various sensors or visual observation, and the absence of obstacles such as accidents or road construction within the predetermined autonomous driving range.

[0116] The driving control ECU 17 performs vehicle speed control, steering control, and braking control based on whether autonomous driving can be performed, as determined by the driving level determination unit 172. For example, the vehicle speed control unit 173 controls the vehicle's speed based on whether autonomous driving can be performed, as determined by the driving level determination unit 172. Specifically, in SAE Level 3 autonomous driving, the vehicle speed control unit 173 controls the vehicle's speed to maintain it within a specified speed range and to ensure a distance from the vehicle ahead is maintained at a specified distance corresponding to the vehicle speed. Furthermore, in SAE Level 3 autonomous driving, the driving control ECU 17 performs steering control to prevent the vehicle from leaving its current lane.

[0117] In order to set the driving automation level to SAE Level 3 via the driving level determination unit 172, the driver's approval operation using the HMI device 20 is required. Therefore, the HMI device 20 provides various information related to autonomous driving and accepts the driver's operation corresponding to the provided information.

[0118] Specifically, when the conditions for starting SAE Level 3 autonomous driving are met, the HMI control unit 23 displays the indication that autonomous driving can be performed on the instrument panel 21 and the CID display 221. Additionally, the HMI control unit 23 displays an execute button for approving autonomous driving and initiating autonomous driving of the vehicle, and a non-execute button for denying autonomous driving and preventing the initiation of autonomous driving of the vehicle on the CID display 221.

[0119] Furthermore, the HMI control unit 23 monitors the operating status of the input device 222 to determine whether either the execute or non-execute button has been selected within a specified time. If either the execute or non-execute button is selected within the specified time, the HMI control unit 23 deactivates the display of either button and sends the operation result notification to the driving control ECU 17. If neither the execute nor non-execute button is selected within the specified time, the HMI control unit 23 performs the same processing as if the non-execute button had been selected. Alternatively, instead of operating the input device 222, the driver can select whether to execute or not autopilot by operating input buttons such as the steering wheel's turn signal switch.

[0120] As described above, SAE Level 3 autonomous driving is initiated by performing button selection operations, etc. Thus, a secondary task can be performed through the driver's operation. The HMI control unit 23 displays content that can be used as a secondary task on the CID display 221 in a manner selectable by the driver. Such content can be pre-registered by, for example, the user including the current driver of the vehicle. Alternatively, such content can be provided from a server via V2X communication using the DCM 15. V2X is an abbreviation for Vehicle to X.

[0121] The HMI control unit 23 monitors the operating status of the input device 222 to receive selections made by the driver. The selected content is displayed on the CID display 221 based on the driver's selection, and the selected content is then utilized.

[0122] However, the time required to perform the second task varies. For example, operating a mobile communication terminal such as a mobile phone takes relatively little time. In contrast, watching a movie may take a long time.

[0123] For example, there might be a relatively short period where the second task can be performed, i.e., the duration of autonomous driving, is only about 30 minutes. In such a case, if the content displayed on the CID display 221 is only a movie with a required duration of more than 1 hour, the driver, as the user, might feel inconvenienced. This is because the movie screening ended midway.

[0124] Conversely, there might be situations where the second task has a relatively long execution time of over one hour. In such cases, if the content displayed on the CID display 221 is only a music video with a required duration of about five minutes, the driver, as the user, might find it inconvenient because they would have to repeatedly select the content.

[0125] On the other hand, the executable time of the second task can, in most cases, be predicted with a certain degree of accuracy. That is, for example, in the case of high-speed range autonomous driving, the distance of the specific section where autonomous driving is possible is known. Therefore, in the case of high-speed range autonomous driving, the duration of autonomous driving can be estimated. Furthermore, for example, in the case of autonomous driving during traffic congestion, the duration of autonomous driving, i.e., the predicted duration of traffic congestion, can be estimated based on the vehicle's average speed during traffic congestion and the distance of traffic congestion obtained from traffic congestion information. Alternatively, for example, the predicted duration of traffic congestion can be directly obtained and read from traffic congestion information such as "traffic congestion between intersection ABC and intersection XYZ for ** minutes". Moreover, the executable time of the second task can be predicted based on an approximation of the duration of autonomous driving.

[0126] Therefore, the HMI control unit 23 acquires the possible duration of autonomous driving through the time acquisition unit 233. Furthermore, based on the possible duration acquired by the time acquisition unit 233, the HMI control unit 23 prompts the driver for a second task that can be performed during autonomous driving through the HMI device 20.

[0127] Specifically, the vehicle information acquisition unit 231 acquires the vehicle's driving status as detected or obtained by the vehicle status sensor 11, etc. Additionally, the vehicle information acquisition unit 231 acquires the vehicle's current location, the planned driving route, and traffic information along the planned driving route, including the road the vehicle is currently traveling on.

[0128] The time acquisition unit 233 acquires the duration of autonomous driving based on the information acquired by the vehicle information acquisition unit 231. The second task prompting unit 234 prompts the driver for a second task that can be performed during autonomous driving via the HMI device 20 based on the duration acquired by the time acquisition unit 233.

[0129] Thus, the HMI control device 23 according to this embodiment, and the HMI control method and HMI control program executed therefrom, have the following effects. For example, it is possible to prompt the driver with the content of a second task, which has a suitable executable time for the second task, i.e., the duration of autonomous driving. In other words, it is possible to prompt the driver with the content of a second task, which has a required execution time corresponding to the reason or situation that allows autonomous driving. Therefore, the second task can be utilized more comfortably during autonomous driving.

[0130] For example, there might be situations where the vehicle enters traffic congestion while traveling outside a specific section of a dedicated motorway with a minimum speed limit. Alternatively, there might be situations where the vehicle enters traffic congestion while traveling on a general road without a minimum speed limit. Or, there might be situations where traffic congestion occurs in a specific section of a dedicated motorway with a minimum speed limit due to an accident or construction. In these situations, the conditions for initiating high-speed range automated driving, which allows operation within the prescribed high-speed range, may not be met.

[0131] On the other hand, even in these situations, since it is during traffic congestion, the start conditions for automatic driving during traffic congestion, which are within the specified low-to-medium speed range (e.g., less than 10 km / h), may be met. Therefore, in these situations, if other conditions included in the start conditions are met, SAE Level 3 automatic driving during traffic congestion can be executed, and the second task can be utilized. Therefore, in this embodiment, when driving on a dedicated road with a minimum speed limit within the specified speed range for automatic driving during traffic congestion and during a traffic congestion, the driving level determination unit 172 determines the execution of automatic driving during traffic congestion. Furthermore, there is also a driving control ECU 17 that, when entering a traffic congestion while in automatic driving within the high-speed range, does not switch to automatic driving during traffic congestion but continues with automatic driving within the high-speed range. In this driving control ECU 17, when entering a traffic congestion while in automatic driving within the high-speed range, the vehicle's speed is controlled only to ensure the inter-vehicle distance from the vehicle in front. Thus, the second task continues.

[0132] As in the example above, after the autonomous driving system begins its second task during SAE Level 3 traffic congestion, there may be periods when the traffic congestion temporarily eases. In such cases, if the vehicle reaches the area where the traffic congestion has temporarily eased, acceleration control is applied. This "acceleration control" can also be called "acceleration-side acceleration control," "acceleration-direction acceleration control," or simply "acceleration control." Subsequently, when the vehicle reaches the very end of the traffic congestion again, deceleration control is applied.

[0133] In scenarios where traffic congestion is temporarily relieved, as described above, if the vehicle's speed temporarily exceeds the upper speed limit for autonomous driving during traffic congestion, the conditions for continuing autonomous driving during traffic congestion are not met. Therefore, the driving automation level changes from SAE Level 3 to SAE Level 2 during traffic congestion, and the second task cannot be utilized. However, upon re-entering a congested area after the congestion has passed, SAE Level 3 autonomous driving during traffic congestion and the accompanying second task can be performed.

[0134] Thus, in scenarios where traffic congestion is temporarily relieved, if autonomous driving is temporarily interrupted due to the vehicle's acceleration control during traffic congestion, the use of the second task is also temporarily interrupted, resulting in poor convenience. Therefore, when a temporary easing of traffic congestion is detected, the driving control ECU 17 and HMI control device 23 perform the processing required to continue autonomous driving during traffic congestion and the accompanying use of the second task during the temporary easing of traffic congestion.

[0135] Specifically, when a temporary relief of traffic congestion is detected, the continue operation acceptance unit 235 in the HMI control unit 23 accepts an input operation for continuing the second task being performed, i.e., a continue operation. The HMI control unit 23 sends the acceptance result of the continue operation by the continue operation acceptance unit 235 to the driving control ECU 17.

[0136] The information acquisition unit 171 in the driving control ECU 17 acquires the traffic congestion status. Additionally, the information acquisition unit 171 acquires the execution status of the second task performed by the driver from the HMI control device 23. Furthermore, the information acquisition unit 171 acquires the acceptance status of continuing operation from the HMI control device 23. That is, the information acquisition unit 171 acquires from the HMI control device 23 whether there is a request to continue the ongoing second task. Moreover, if the acquired traffic congestion status is that the traffic congestion has temporarily eased, and there is a request to continue the ongoing second task, the speed control unit 173 controls the vehicle speed to increase within a specified speed range, i.e., faster than the minimum speed limit for dedicated motor vehicle roads and within the speed limit for autonomous driving during continued traffic congestion.

[0137] In this embodiment, temporary interruptions to autonomous driving and the second task that violate the driver's wishes can be effectively avoided when traffic congestion occurs again, even if the driver wishes to continue the second task after the traffic congestion has temporarily eased. Therefore, according to this embodiment, the second task can be utilized more comfortably in autonomous driving.

[0138] Even if traffic congestion is temporarily relieved, automated driving during traffic congestion may not continue as before. When automated driving during traffic congestion is deactivated along with the temporary relief of congestion, a smooth handover from the system to the driver is necessary. Therefore, the execution of automated driving during traffic congestion and the request to continue the second task are preferably performed when the driver has a full understanding of the traffic conditions ahead of the vehicle, i.e., the road conditions.

[0139] Therefore, in this embodiment, when a temporary easing of traffic congestion is detected, the continuation operation acceptance unit 235 displays a message such as "Caution Ahead" to help the driver confirm the road conditions ahead (i.e., the vehicle's destination). Furthermore, if the behavior acquisition unit 232 obtains a check result indicating that the driver is confirming the road conditions at the vehicle's destination, the continuation operation acceptance unit 235 activates the continuation operation. Moreover, after activating the continuation operation acceptance, the continuation operation acceptance unit 235 notifies the driving control ECU 17 of the activation status in order to control the vehicle's speed by ensuring it does not exceed a predetermined speed range.

[0140] If the driving control ECU 17 receives a notification from the continue operation acceptance unit 235 that the continue operation has been validated, it executes processing for continuing autonomous driving. That is, if the input operation for continuing the second task being performed is validly accepted by the HMI device 20, i.e., a continue operation is notified of the continue request by the HMI device 20, the driving level determination unit 172 decides to continue autonomous driving. Specifically, if the driving level determination unit 172 determines that the autonomous driving will continue in traffic congestion after the driver's behavior check results indicate that the road conditions at the vehicle's destination are being confirmed, and the continue operation has been validly accepted by the HMI device 20.

[0141] (Example of action)

[0142] use Figures 6-9 The flowchart shown illustrates a specific example of the control actions or methods described above, and the control procedures corresponding to these control actions or methods. Furthermore, in the flowchart shown, "S" is an abbreviation for "step".

[0143] (Autonomous driving control)

[0144] Figure 6 This illustrates a series of actions performed by the in-vehicle system 10 related to autonomous driving. The following will... Figure 6 The series of actions shown are called "autopilot control actions".

[0145] In step 601, the vehicle system 10 determines whether the conditions for starting autonomous driving are met. The processing in step 601 is, for example, processing corresponding to the action of the driving level determination unit 172, and is executed by the driving control ECU 17. Details of the processing in step 601 will be described later. After the processing in step 601, the vehicle system 10 proceeds to step 602.

[0146] In step 602, the vehicle system 10 determines whether autonomous driving can begin. That is, the vehicle system 10 determines whether the determination result in step 601 indicates that the conditions for starting autonomous driving are met or not.

[0147] If autonomous driving can be initiated (i.e., step 602 = Yes), the vehicle system 10 executes the processing in step 603. In step 603, the vehicle system 10 outputs an autonomous driving capability message to the HMI device 20. The autonomous driving capability message is a message containing the reason for autonomous driving and the main point of initiating autonomous driving, displayed on the CID display 221, and output as sound through a speaker (not shown). Additionally, the vehicle system 10 displays the aforementioned execute and disobey buttons on the CID display 221.

[0148] On the other hand, if autonomous driving cannot be initiated (i.e., step 602 = No), the vehicle system 10 skips step 603 and all subsequent processing, and terminates the autonomous driving control operation. The processing of step 602, for example, is performed by the HMI control device 23 as processing corresponding to the actions of the vehicle information acquisition unit 231 and the display control unit 236. The processing of step 603, for example, is performed by the HMI control device 23 as processing corresponding to the actions of the display control unit 236.

[0149] After the processing in step 603, the vehicle system 10 proceeds to step 604. In step 604, the vehicle system 10 determines whether the driver has approved the start of autonomous driving by operating the execution button for approving autonomous driving and starting the vehicle's autonomous driving. The processing in step 604 is performed by the HMI control device 23 as a process corresponding to the action of the HMI control device 23.

[0150] If the driver selects the operation execution button and approves the start of autonomous driving (i.e., step 604 = Yes), the vehicle system 10 sequentially executes steps 605 to 607. Conversely, if the driver does not approve the start of autonomous driving (i.e., step 604 = No), the vehicle system 10 skips step 605 and all subsequent processes, and terminates the autonomous driving control operation. Furthermore, since the determination result in step 602 or step 604 is "No," even after the autonomous driving control operation has temporarily ended, the processing from step 601 can be restarted if a predetermined restart condition is met. Such a restart condition could be, for example, a predetermined autonomous driving start trigger operation (e.g., operation of a turn signal switch not shown in the operation unit 19), the passage of a predetermined interval, or the detection of traffic congestion at the vehicle's destination. Specifically, for example, in the stage immediately after entering a specific area where high-speed autonomous driving is possible, the driver might not approve the start of autonomous driving in step 604. In such a case, a considerable distance or time may be required before the vehicle exits this specific area. Therefore, in such cases, by allowing the driver to change their mind or get stuck in traffic after the operation is not approved, autonomous driving can be fully utilized when the driver wishes to perform autonomous driving afterwards.

[0151] In step 605, the vehicle system 10 initiates autonomous driving. Specifically, for example, when the start condition for high-speed range autonomous driving in step 601 is met, the vehicle system 10 initiates high-speed range autonomous driving. On the other hand, when the start condition for traffic congestion autonomous driving in step 601 is met, the vehicle system 10 initiates traffic congestion autonomous driving. The processing in step 605 is, for example, a process corresponding to the operation of the driving level determination unit 172, and is executed by the driving control ECU 17.

[0152] In step 606, the vehicle system 10 performs a selection action for the second task content utilized in autonomous driving. The processing in step 606 is performed by the HMI control device 23 as a corresponding action of the HMI control device 23. Details regarding the processing content of step 606 will be described later.

[0153] In step 607, the vehicle system 10 begins to execute, or utilize, the second task selected in step 606. The processing in step 607 is performed by the HMI control device 23 as a process corresponding to the action of the HMI control device 23.

[0154] In autonomous driving, the vehicle system 10 repeatedly executes steps 608 and 609. In step 608, the vehicle system 10 determines whether to continue autonomous driving. The processing of step 608 is performed by the driving control ECU 17 and the HMI control device 23. Details of the processing of step 608 will be described later.

[0155] In step 609, the vehicle system 10 determines whether the determination result in step 608 indicates the end of autonomous driving. If autonomous driving continues (i.e., step 609 = No), the vehicle system 10 returns to step 608. Furthermore, during continued autonomous driving, when the second task selected in step 606 ends, the process returns to step 606. On the other hand, if autonomous driving ends (i.e., step 609 = Yes), the vehicle system 10 executes step 610 and subsequent processing. The processing in step 609, for example, is executed by the driving control ECU 17 as processing corresponding to the operation of the driving level determination unit 172.

[0156] In step 610, the vehicle system 10 terminates the second task. In the next step 611, the vehicle system 10 displays a switching instruction message on the instrument cluster display 212 and the CID display 221, and outputs an audio signal through a speaker (not shown). The switching instruction message is a message urging the driver to switch driving duties. "Switching driving duties" refers to the transfer of driving operations from the system to the driver; that is, the transfer of responsibility for the dynamic driving task from the system to the driver. The processing in steps 610 and 611 is performed by the HMI control device 23 as corresponding to the actions of the HMI control device 23.

[0157] Following step 611, the vehicle system 10 executes step 612. In step 612, the vehicle system 10 determines whether driving handover preparation is complete and driving handover can proceed. Driving handover preparation refers to the driver adjusting their driving posture for driving handover. Specifically, the vehicle system 10 determines whether driving handover preparation is complete based on the driver's steering wheel grip and line of sight. Step 612 is executed by the driving control ECU 17, for example, as a process corresponding to the actions of the information acquisition unit 171 and the driving level determination unit 172.

[0158] When the driving handover preparation is complete and a driving handover can be performed (i.e., step 612 = Yes), the vehicle system 10 advances the processing to step 613. In step 613, the vehicle system 10 performs a driving handover. That is, the vehicle system 10 terminates automatic driving and sets the driving automation level to SAE level 2. The processing in step 613 is performed by the driving control ECU 17, for example, as processing corresponding to the action of the driving level determination unit 172. Afterward, the vehicle system 10 terminates the automatic driving control operation.

[0159] If the driving alternation preparation is not completed (i.e., step 612 = No), the vehicle system 10 proceeds the process to step 614. In step 614, the vehicle system 10 determines whether a predetermined time T1 has elapsed since the display of the alternation instruction message in step 611. If the predetermined time T1 has elapsed (i.e., step 614 = No), the vehicle system 10 returns the process to step 612. On the other hand, if the predetermined time T1 has elapsed (i.e., step 614 = Yes), the vehicle system 10 proceeds the process to steps 615 and 616. The processing in step 614 is performed by the HMI control device 23 as processing corresponding to the operation of the HMI control device 23.

[0160] In step 615, the vehicle system 10 displays an alternation warning message on the instrument cluster display 212 and the CID display 221, and outputs sound through the speaker. The alternation warning message is a message used to warn that a driving alternation cannot be performed because driving alternation preparation has not been completed. The processing in step 615 is performed by the HMI control device 23, for example, as a process corresponding to the operation of the display control unit 236.

[0161] In step 616, the vehicle system 10 determines whether a predetermined time T2 has elapsed since the display of the alternating instruction message in step 611. If T2 > T1, and the predetermined time T2 has elapsed (i.e., step 616 = No), the vehicle system 10 returns the process to step 612. Conversely, if the predetermined time T2 has elapsed (i.e., step 616 = Yes), the vehicle system 10 advances the process to steps 617 and 618. The processing in step 616 is executed by the HMI control device 23 as processing corresponding to the operation of the HMI control device 23.

[0162] In step 617, the vehicle system 10 displays an emergency stop message indicating that the vehicle should be brought to an emergency stop on the instrument panel display 212 and the CID display 221, and outputs sound through the speaker. The processing in step 617 is performed by the HMI control device 23, for example, as a process corresponding to the operation of the display control unit 236.

[0163] In step 618, the vehicle system 10 performs the processing required to bring the vehicle to an emergency stop at a safe location. Specifically, the information acquisition unit 171 in the driving control ECU 17 receives information from the HMI control device 23 displaying an emergency stop message. As a result, the driving control ECU 17 detects that the driving handover preparation has not been completed before a predetermined time T2 has elapsed. Therefore, the driving control ECU 17 performs the vehicle control required to bring the vehicle to an emergency stop at a safe location (e.g., an emergency parking area). Afterwards, the vehicle system 10 terminates the automatic driving control operation.

[0164] (Determination of conditions for starting autonomous driving)

[0165] Figure 7 This indicates whether the conditions for starting autonomous driving are met. Figure 6 The details of step 601 are shown below. In the process of determining the conditions for starting autonomous driving, the driving control ECU 17 first executes the process of step 701.

[0166] In step 701, the driving control ECU 17 determines whether the interval condition is met. The "interval condition" is that the vehicle is currently traveling in a specific interval. If the interval condition is met (i.e., step 701 = yes), the driving control ECU 17 advances the process to step 702.

[0167] In step 702, the driving control ECU 17 determines whether the environmental conditions are met. "Environmental conditions" refers to the driving environment suitable for automated driving. Environmental conditions include, for example, a traffic environment suitable for automated driving, and the absence of abnormalities in sensors and other components required for automated driving. Specifically, a traffic environment includes, for example, the absence of poor visibility due to fog or heavy rain, and the absence of obstacles such as accidents or road construction within the predetermined automated driving range.

[0168] If the environmental conditions are met (i.e., step 702 = Yes), the driving control ECU 17 terminates the process of determining the autonomous driving start condition after executing the processing in step 703. In step 703, the driving control ECU 17 determines that the autonomous driving start condition is met. Specifically, in this case, the driving control ECU 17 determines that the start condition for high-speed range autonomous driving is met.

[0169] On the other hand, if the environmental conditions are not met (i.e., step 702 = No), the driving control ECU 17 terminates the process of determining the autonomous driving start condition after executing the processing in step 704. In step 704, the driving control ECU 17 determines that the autonomous driving start condition is not met.

[0170] If the zone condition is not met (i.e., step 701 = No), the driving control ECU 17 proceeds the process to step 705. In step 705, the driving control ECU 17 determines whether the traffic congestion condition is met. The "traffic congestion condition" is that the vehicle is currently traveling in a traffic congestion zone or is about to enter a traffic congestion zone. Specifically, the traffic congestion condition is met if, based on traffic information, the vehicle's current position is determined to be within or near a traffic congestion zone, and any one of the following (A) to (D) is detected: (A) The distance to the vehicle ahead decreases sharply. (B) The speed of the vehicle ahead decreases sharply. (C) The vehicle ahead flashes its hazard warning lights. (D) The distance to the vehicle ahead is a specified short distance (e.g., less than 10m), and the vehicle's speed is within a specified low speed range (e.g., less than 20km / h).

[0171] If the traffic congestion condition is met (i.e., step 705 = Yes), the driving control ECU 17 proceeds the process to step 706. Conversely, if the traffic congestion condition is not met (i.e., step 705 = No), the driving control ECU 17, after executing the process in step 704, terminates the process for determining the autonomous driving start condition. In step 704, the driving control ECU 17 determines that the autonomous driving start condition is not met.

[0172] In step 706, the driving control ECU 17 determines whether the latest traffic information has been obtained through the information acquisition unit 171. "Latest" refers, for example, to traffic information provided by the Japan Road Traffic Information Center or similar organizations within N minutes of the current time. N is, for example, 5 to 10 minutes.

[0173] Upon receiving the latest traffic information (i.e., step 706 = Yes), the driving control ECU 17, after executing the processing in step 703, terminates the process for determining the autonomous driving start condition. In step 703, the driving control ECU 17 determines that the autonomous driving start condition is met. Specifically, in this case, the driving control ECU 17 determines that the autonomous driving start condition is met during traffic congestion.

[0174] On the other hand, if the latest traffic information is not obtained (i.e., step 706 = No), the driving control ECU 17 advances the process to step 707. In step 707, the driving control ECU 17 determines whether the traffic congestion condition has persisted for a predetermined time.

[0175] If the traffic congestion condition persists for a predetermined time (i.e., step 707 = Yes), the driving control ECU 17, after executing the processing in step 703, ends the process of determining the autonomous driving start condition. In step 703, the driving control ECU 17 determines that the autonomous driving start condition has been met. Specifically, in this case, the driving control ECU 17 determines that the autonomous driving start condition has been met during traffic congestion.

[0176] On the other hand, if the traffic congestion condition does not persist for the specified time (i.e., step 707 = No), the driving control ECU 17 terminates the process of determining the autonomous driving start condition after executing the processing in step 704. In step 704, the driving control ECU 17 determines that the autonomous driving start condition is not met.

[0177] (Content Selection)

[0178] Figure 8 This indicates the selection of the second task content to be utilized in autonomous driving. Figure 6 The details of the processing content in step 606 are shown below. In this content selection process, the HMI control device 23 first executes the processes in steps 801 and 802.

[0179] In step 801, the HMI control device 23 displays a second task permission message on the CID display 221. The second task permission message is used to notify the driver that a second task can be utilized. In step 802, the HMI control device 23 determines whether the interval condition is met. That is, the HMI control device 23 determines whether the currently initiated automatic driving is high-speed range automatic driving.

[0180] If the autonomous driving being executed this time is high-speed range autonomous driving (i.e., step 802 = Yes), the HMI control device 23 executes steps 803, 804, and 805. On the other hand, if the autonomous driving being executed this time is traffic jam autonomous driving (i.e., step 802 = No), the HMI control device 23 executes steps 806 and 807.

[0181] Steps 803 to 805 are processes used to obtain the duration Tc of SAE Level 3 automated driving, i.e., high-speed range automated driving, which allows for high-speed driving within a specific section. In step 803, the HMI control device 23 obtains the remaining distance within the currently traveling specific section. The remaining distance is the predetermined distance traveled from the vehicle's current position to the point where the vehicle exits the specific section (e.g., the exit junction where the vehicle exits, or the end point of the specific section).

[0182] In step 804, the HMI control device 23 calculates the interval travel time Tv based on the remaining distance obtained in step 803. Specifically, the interval travel time Tv can be calculated, for example, by dividing the remaining distance obtained in step 803 by the average travel speed of the vehicle in the specific interval it is currently traveling in.

[0183] In step 805, the HMI control device 23 obtains the duration Tc based on the section travel time Tv calculated in step 804. Specifically, the duration Tc can be calculated, for example, by multiplying or subtracting the section travel time Tv calculated in step 804 using a specified correction amount.

[0184] Furthermore, for some reason, there may be a situation where the interval travel time Tv cannot be calculated in step 804. In this case, the HMI control device 23 sets the allowable duration Tc to a settable maximum value. Specifically, for example, in a specification that uses a 4-digit hexadecimal number from "0000" to "FFFF" to set the value of the allowable duration Tc, the value of the allowable duration Tc in the case where the interval travel time Tv cannot be calculated is set to "FFFF".

[0185] Steps 806 and 807 are processes for obtaining the duration Tc of SAE Level 3 automated driving, i.e., automated driving during traffic congestion, which allows for driving at low to medium speeds during traffic congestion. In step 806, the HMI control unit 23 obtains the predicted duration of traffic congestion, i.e., the traffic congestion prediction time Tj. The traffic congestion prediction time Tj is an approximation of the time required until the vehicle finishes driving through the currently congested traffic section. Specifically, the traffic congestion prediction time Tj can be calculated, for example, based on the vehicle's average speed during the current traffic congestion and the remaining distance of the traffic congestion section obtained from traffic congestion information.

[0186] In step 807, the HMI control device 23 obtains the duration Tc based on the traffic congestion prediction time Tj obtained in step 806. Specifically, the duration Tc can be calculated, for example, by multiplying or subtracting the traffic congestion prediction time Tj obtained in step 806 using a specified correction factor. Furthermore, for some reason, there may be a situation where the traffic congestion prediction time Tj cannot be obtained in step 806. In this case, the HMI control device 23 sets the duration Tc to a settable maximum value. A specific example in this case is the same as described above, "FFFF".

[0187] As described above, once the duration Tc corresponding to the reason or type of autonomous driving initiation is obtained, the HMI control device 23 advances the process to step 808. In step 808, the HMI control device 23 determines whether the duration Tc is less than a first threshold Tc1.

[0188] If the duration Tc is less than the first threshold Tc1 (i.e., step 808 = Yes), the HMI control device 23 advances the process to step 809. In step 809, the HMI control device 23 determines the content to be used within a short period as the recommended content. The recommended content is a second task content suggested to the driver.

[0189] If the duration Tc is greater than or equal to the first threshold Tc1 (i.e., step 808 = No), the HMI control device 23 proceeds the process to step 810. In step 810, the HMI control device 23 determines whether the duration Tc is less than the second threshold Tc2. That is, the determination in step 810 is whether the duration Tc is greater than or equal to the first threshold Tc1 and less than the second threshold Tc2. The second threshold Tc2 is a threshold used to determine whether the duration Tc is set to its maximum value. The first threshold Tc1 << the second threshold Tc2. Therefore, the determination in step 810 is a determination of whether the time acquisition unit 233 can acquire the duration Tc.

[0190] If the duration Tc is less than the second threshold Tc2 (i.e., step 810 = yes), the HMI control device 23 advances the process to step 811. In step 811, the HMI control device 23 determines the long-duration content as the recommended content.

[0191] If the duration Tc is greater than or equal to the second threshold Tc2 (i.e., step 810 = No), the time acquisition unit 233 fails to acquire the duration Tc. That is, if the interval travel time Tv or the traffic congestion prediction time Tj cannot be acquired for some reason, the duration Tc is set to its maximum value. Therefore, in this case, the HMI control device 23 advances the processing to step 812. In step 812, the HMI control device 23 determines the composite content as the recommended content. The composite content is a combination of both short-term and long-term content.

[0192] After determining the type of suggested content as described above, the HMI control device 23 advances the process to steps 813 and 814. In step 813, the HMI control device 23 displays a prompt screen showing the determined suggested content on the CID display 221.

[0193] Specifically, when the suggested content is a short-term content, the HMI control device 23 displays a short-term content prompt screen on the CID display 221. The short-term content prompt screen is a display screen used to indicate short-term content, and it is displayed on... Figure 2 The central display area DA1 can execute or start playing a short clip of "recommended" content.

[0194] Additionally, when the suggested content is long-term, the HMI control device 23 displays a long-term content prompt screen on the CID display 221. The short-term content prompt screen is a display screen used to prompt for long-term content, and it is displayed on... Figure 2 The central display area DA1 can display a long-running "recommended" content screen, which can be executed or started playing.

[0195] Furthermore, when the suggested content is composite, the HMI control device 23 can selectively prompt multiple second task contents with different execution times, i.e., short-duration content and long-duration content. Specifically, the HMI control device 23 enables... Figure 5 The menu screen shown is displayed on CID monitor 221.

[0196] In step 814, the HMI control device 23 determines whether the content selection operation performed by the driver or others has ended. If the content selection operation is not completed, and the determination in step 814 is "No," the HMI control device 23 returns to step 813. That is, the HMI control device 23 waits for the content selection operation to end. If the content selection operation ends (i.e., step 814 = Yes), the HMI control device 23 ends the content selection process. Thus, the selected second task content begins to be used.

[0197] (Autonomous driving continues to be assessed)

[0198] Figure 9 Shows the decision on whether to continue autonomous driving. Figure 6 The details of step 608 are shown below. In this process of continuing autonomous driving, the driving control ECU 17 first executes the process of step 901.

[0199] In step 901, the driving control ECU 17 determines whether the traffic congestion condition was met in step 705. That is, the driving control ECU 17 determines whether the currently executing SAE Level 3 automated driving is automated driving during traffic congestion or automated driving at high speeds.

[0200] If the currently executing autonomous driving is high-speed range autonomous driving (i.e., step 901 = No), the driving control ECU 17 advances the process to step 902. In step 902, the driving control ECU 17 determines whether the termination condition for high-speed range autonomous driving is met. The termination condition may include, for example, that the distance or time required to exit the specific section (e.g., the exit junction where the vehicle exits, or the end of the specific section) is less than a predetermined value. Alternatively, the termination condition may include, for example, that, considering the traffic environment or road conditions ahead of the vehicle, the vehicle's behavior becomes unpredictable up to a predetermined time (e.g., 15 seconds).

[0201] If the termination condition is not met (i.e., step 902 = No), the driving control ECU 17 temporarily terminates the decision-making process for continuing autonomous driving after advancing the process to step 903. In step 903, the driving control ECU 17 decides whether to continue autonomous driving and the second task.

[0202] If the termination condition is met (i.e., step 902 = Yes), the driving control ECU 17 terminates the autonomous driving continuation determination process after advancing the process to step 904. In step 904, the driving control ECU 17 determines the termination of autonomous driving and the second task.

[0203] If the currently executing autonomous driving mode is in traffic congestion (i.e., step 901 = Yes), the driving control ECU 17 advances the process to step 905. In step 905, the driving control ECU 17 determines whether a temporary traffic congestion relief scenario has occurred. That is, the driving control ECU 17 determines whether a temporary traffic congestion relief has been detected.

[0204] If no traffic congestion is temporarily relieved (i.e., step 905 = No), the driving control ECU 17 temporarily terminates the decision-making process for continuing autonomous driving after advancing to step 903. In step 903, the driving control ECU 17 decides whether to continue autonomous driving and the second task.

[0205] In the event of a temporary relief of traffic congestion (i.e., step 905 = Yes), the driving control ECU 17 proceeds the process to step 906. In step 906, the driving control ECU 17 determines whether the information acquisition unit 171 has acquired the latest traffic congestion information.

[0206] If the latest traffic congestion information is not currently available (i.e., step 906 = No), the driving control ECU 17, after advancing the process to step 904, terminates the decision-making process for continuing autonomous driving. In step 904, the driving control ECU 17 determines the termination of autonomous driving and the second task.

[0207] If the latest traffic congestion information is obtained (i.e., step 906 = Yes), the driving control ECU 17 proceeds to step 907. In step 907, the driving control ECU 17 determines whether the current driving position of the vehicle is within a traffic congestion zone.

[0208] If the current driving location of the vehicle is not within a traffic congestion zone (i.e., step 907 = No), the driving control ECU 17, after advancing the process to step 904, terminates the decision-making process for continuing autonomous driving. In step 904, the driving control ECU 17 determines the termination of autonomous driving and the second task.

[0209] If the current driving location of the vehicle is within a traffic congestion zone (i.e., step 907 = Yes), the driving control ECU 17 advances the process to steps 908 and 909. In step 908, the driving control ECU 17 sends a command to the HMI control unit 23 to instruct the CID display 221 to perform a continue suggestion display. Consequently, the HMI control unit 23 instructs the CID display 221 to perform a continue suggestion display. The continue suggestion display includes a continue suggestion message recommending continued autonomous driving and instructions for... Figure 3 The "Continue Approval" button DA6 is shown. Continue suggestion messages, such as "Pay Attention Ahead," or similar messages, alternate between these options. Figure 3 The message display box DB4 is shown. Additionally, the HMI control unit 23 uses a speaker (not shown) to provide audio guidance corresponding to the continued suggested display content.

[0210] In step 909, the driving control ECU 17 determines whether an approval operation performed by the driver or others has been executed on the HMI device 20. That is, the driving control ECU 17 receives a notification from the HMI control device 23 indicating whether an operation has been selected. Figure 3 The result of the approval button DA6 shown.

[0211] If the approved operation is not performed (i.e., step 909 = No), the driving control ECU 17 advances the process to step 910. In step 910, the driving control ECU 17 determines whether a predetermined time has elapsed since the start of the continued suggestion display. If the predetermined time has elapsed, the determination in step 910 is "No," and the driving control ECU 17 returns the process to step 909. That is, the driving control ECU 17 waits for the predetermined time for the approved operation.

[0212] If the state of not performing the approved operation remains unchanged for a specified time, the determination in step 910 is "yes". In this case, the driving control ECU 17, after advancing the process to step 904, ends the decision process for continuing autonomous driving. In step 904, the driving control ECU 17 determines the end of autonomous driving and the second task.

[0213] If the approval operation is performed within the specified time, the determination in step 909 is "yes". In this case, the driving control ECU 17 advances the process to step 911. In step 911, the driving control ECU 17 determines whether the driver is confirming the road conditions ahead, i.e., the destination of the vehicle's journey.

[0214] While the driver is confirming the path ahead (i.e., step 911 = Yes), the driving control ECU 17 temporarily terminates the decision-making process for continuing autonomous driving after advancing the process to step 903. In step 903, the driving control ECU 17 decides whether to continue autonomous driving and the second task.

[0215] If the driver is not currently confirming the road ahead (i.e., step 911 = No), the driving control ECU 17 advances the process to step 912. In step 912, the driving control ECU 17 determines whether a predetermined time has elapsed since the approved operation. If the predetermined time has elapsed, the determination in step 912 is "NO," and the driving control ECU 17 returns the process to step 911. That is, the driving control ECU 17 waits for the driver to confirm the road ahead for the predetermined time.

[0216] If the driver has not confirmed that the specified time has elapsed since the approved operation, the determination in step 912 is "yes". In this case, the driving control ECU 17, after advancing the process to step 904, ends the decision process for continuing autonomous driving. In step 904, the driving control ECU 17 terminates autonomous driving and the second task.

[0217] (Second Implementation)

[0218] Below, except Figures 1-9 In addition, refer to Figure 10 as well as Figure 11 The second embodiment will be described below. Furthermore, in the following description of the second embodiment, the parts that differ from the first embodiment described above will be primarily explained. Additionally, in both the first and second embodiments, the same reference numerals are used to denote identical or equivalent parts. Therefore, in the following description of the second embodiment, unless there is a technical inconsistency or specific additional explanation, the description in the first embodiment may be appropriately referenced for constituent elements having the same reference numerals as in the first embodiment. The same applies to the third and subsequent embodiments described later.

[0219] The basic structure of the vehicle system 10 in this embodiment is the same as Figure 1 The same. Furthermore, the basic operation of the vehicle system 10 in this embodiment is the same as... Figures 2-9 Same. In this embodiment, in addition to the first embodiment described above, the following functions are added.

[0220] For example, if there are traffic obstacles such as accidents, traffic jams, broken-down vehicles, or construction at the vehicle's destination, the driver may take their eyes off the CID display 221 to check the road ahead multiple times, even if they are performing a secondary task. Therefore, in this embodiment, when the behavior acquisition unit 232 obtains a check result indicating that the driver is checking the road conditions at the vehicle's destination, the display control unit 236 displays the road conditions via the HMI device 20. Specifically, when the driver performs road checks at a predetermined frequency, such as... Figure 10 As shown, the display control unit 236 displays the road conditions in the information display box DB3.

[0221] Figure 11 This is the display control processing of the second task, performed by the HMI control device 23. Such display control processing of the second task... Figure 6 The process in the flowchart shown is repeatedly executed at specified time intervals (e.g., 100 msec) from the moment the process advances to step 607 to the moment the process advances to step 610.

[0222] In the display control processing of the second task, firstly, in step 1101, the HMI control device 23 reads the data of the display content into a temporary storage area such as RAM. The display content is the second task content displayed on the CID display 221. Next, in step 1102, the HMI control device 23 uses the data read in step 1101 to display the content on the CID display 221.

[0223] Next, in step 1103, the HMI control device 23 determines whether to perform the approval process. The approval process is used for various approval operations performed by the driver.

[0224] If the approval process is executed (i.e., step 1103 = Yes), the HMI control device 23 temporarily terminates the display control process for the second task after executing steps 1104 and 1105. In step 1104, the HMI control device 23 displays the approval request message on the CID display 221. In step 1105, the HMI control device 23 displays the approval button DA6 on the CID display 221.

[0225] If no approval process is performed (i.e., step 1103 = No), the HMI control unit 23 advances the process to step 1106. In step 1106, the HMI control unit 23 determines whether to perform the current warning process. The warning process is used to issue various warnings to the driver.

[0226] If warning processing is performed (i.e., step 1106 = Yes), the HMI control device 23 temporarily terminates the display control processing of the second task after performing the processing in step 1107. In step 1107, the HMI control device 23 displays the warning content on the CID display 221. The warning content includes messages and / or graphics for various warnings.

[0227] If no warning action is taken (i.e., step 1106 = No), the HMI control unit 23 proceeds the process to step 1108. In step 1108, the HMI control unit 23 determines whether the driver is performing a forward confirmation.

[0228] If the driver does not perform forward confirmation (i.e., step 1108 = No), the HMI control unit 23 temporarily terminates the display control processing of the second task. On the other hand, if the driver is performing forward confirmation (i.e., step 1108 = Yes), the HMI control unit 23 advances the processing to step 1109.

[0229] In step 1109, the HMI control device 23 determines whether the driver's forward confirmation has reached a predetermined frequency. "Predetermined frequency" is the number of times per predetermined reference time interval, for example, more than 3 times every 5 seconds. If the driver's forward confirmation has not reached the predetermined frequency (i.e., step 1109 = No), the HMI control device 23 temporarily terminates the display control processing of the second task. On the other hand, if the driver's forward confirmation has reached the predetermined frequency (i.e., step 1109 = Yes), the HMI control device 23 advances the processing to step 1110.

[0230] In step 1110, the HMI control device 23 determines whether traffic information related to the vehicle's surroundings has been obtained through the vehicle information acquisition unit 231. "The vehicle's surroundings" refers to the vehicle's destination, which is within a predetermined driving distance (e.g., 5 km) from the vehicle's current location.

[0231] If no traffic information related to the vehicle's surroundings is obtained (i.e., step 1110 = No), the HMI control device 23 temporarily terminates the display control processing of the second task. On the other hand, if traffic information related to the vehicle's surroundings is obtained (i.e., step 1110 = Yes), the HMI control device 23 temporarily terminates the display control processing of the second task after executing the processing in step 1111. In step 1111, the HMI control device 23 displays the obtained traffic information related to the vehicle's surroundings in the information display frame DB3.

[0232] Thus, in this embodiment, when the driver checks the road ahead multiple times at a predetermined frequency during the second task, traffic information related to the vehicle's surroundings is displayed on the HMI device 20. Therefore, according to this embodiment, appropriate traffic information display based on actual traffic conditions can be performed on the HMI device 20.

[0233] (Additional Implementation Method 1: Third Implementation Method)

[0234] The third embodiment will now be described. In this description, the same principles as those of the second embodiment described above will be applied, with appropriate references to the descriptions in the preceding embodiments, and primarily focusing on the parts unique to this embodiment.

[0235] This embodiment relates to processing when the duration Tc of autonomous driving, acquired by the time acquisition unit 233, is an extremely short time. An "extremely short time" is, for example, equivalent to less than tens of seconds or a travel distance of less than 1000m. In this embodiment, when the duration Tc is extremely short, the driving control ECU 17 can execute autonomous driving during traffic jams based on the driver's selection, but cannot execute autonomous driving at high speeds.

[0236] Specifically, refer to Figure 7 The operation of this embodiment will be explained. In this embodiment, when determining whether the environmental conditions are met in step 702, which is executed when the interval condition is met (i.e., step 701 = yes), the "environmental conditions" include a duration Tc that is greater than or equal to a predetermined lower limit execution time. Here, the "lower limit execution time" is a criterion value used to determine whether the duration Tc is equivalent to the aforementioned extremely short time, and is a value that is sufficiently smaller than the first threshold Tc1 in step 808 (e.g., 30 seconds).

[0237] Furthermore, if the duration Tc is shorter than the lower limit of execution time, since the duration Tc is equivalent to the aforementioned extremely short time, the determination result in step 702 is "no," and in step 704, the driving control ECU 17 determines that the automatic driving start condition is not met. In this case, Figure 6 If the determination result of step 602 in the flowchart shown is "No", the vehicle system 10 skips step 603 and all subsequent processing, and terminates the autonomous driving control action. At this time, the HMI control device 23 preferably prompts through the HMI device 20 that autonomous driving and the execution of the second task are not allowed because the duration Tc is shorter than the lower limit of execution time.

[0238] On the other hand, if the interval condition is not met but the traffic congestion condition is met (i.e., step 705 = yes), the process does not proceed to the determination of whether the "environmental condition" containing a duration Tc of more than the minimum execution time is met (i.e., step 702). That is, for automatic driving in traffic congestion, the execution condition does not contain a duration Tc of more than the specified minimum execution time. Thus, in the HMI control device 23, the execution of the second task in automatic driving in traffic congestion can be performed for a very short time. At this time, in step 603, the HMI control device 23 preferably prompts the HMI device 20 that the time required for the execution of automatic driving and the utilization of the accompanying second task is a very short time.

[0239] Thus, according to this embodiment, the execution of autonomous driving for a very short time and the utilization of the accompanying second task can be carried out during traffic congestion, while driving conditions corresponding to high-speed autonomous driving are prohibited. Therefore, according to this embodiment, the execution of autonomous driving and the utilization of the second task can be performed more appropriately.

[0240] However, when the duration Tc of autonomous driving is short, a smooth driving transition at the end of autonomous driving can be achieved by minimizing the driver's awareness from leaving the vehicle's driving (i.e., driving state and driving operation) due to the second task. Therefore, in such cases, it is preferable to limit the HMI device 20 that performs the second task to the on-board equipment, i.e., the CID device 22.

[0241] Figure 12 Extract and show the changes corresponding to such examples. Figure 8 The changes are part of the flowchart. Specifically, refer to... Figure 12 If the duration Tc is less than the first threshold Tc1 (i.e., step 808 = Yes), the HMI control device 23 advances the process to step 1201. In step 1201, the HMI control device 23 determines whether the duration Tc is greater than or equal to the model-limited threshold TcS. The model-limited threshold TcS is less than the first threshold Tc1. That is, the model-limited threshold TcS is a value that is sufficiently smaller than the first threshold Tc1 in step 808.

[0242] If the duration Tc is greater than or equal to the model-limited threshold TcS (i.e., step 1201 = Yes), the HMI control device 23 proceeds the process to step 809, determining the short-term content as the recommended content. Conversely, if the duration Tc is less than the model-limited threshold TcS (i.e., step 1201 = No), the HMI control device 23 proceeds the process to step 1202. In step 1202, the HMI control device 23 determines the restricted content as the recommended content. The restricted content is limited to execution by the CID device 22, which is an in-vehicle device, for a very short period of time, allowing the driver to easily focus on driving the vehicle. Specifically, restricted content includes, for example, music videos with relatively short execution times that are acceptable even for listening to only the audio, and television broadcasts with execution times that cannot be conceptualized.

[0243] Figure 12 The example shown is particularly effective when the duration Tc of autonomous driving is extremely short. That is, if the duration Tc obtained by the second task prompting unit 234 based on the traffic congestion prediction time Tj is shorter than the specified vehicle-specific time limit, it preferably restricts the HMI device 20 performing the second task during traffic congestion to the on-board unit, i.e., the CID device 22. The "specified vehicle-specific time limit" is the vehicle-specific threshold TcS. In this case, the vehicle-specific threshold TcS can be equal to the aforementioned lower limit execution time, or a difference can be set between the vehicle-specific threshold TcS and the lower limit execution time.

[0244] (Additional Implementation Method 2: Fourth Implementation Method)

[0245] The fourth embodiment will now be described. This embodiment makes some changes to the first and / or third embodiments described above. Specifically, in the case of performing autonomous driving in traffic congestion in a very short time, this embodiment provides a second task permission message earlier.

[0246] That is, in this embodiment, when the duration Tc of autonomous driving is shorter than a predetermined time limit, the second task prompting unit 234 operates as follows: the second task prompting unit 234, instead of prompting for a second task that can be executed based on the duration Tc, or prior to such a prompt, prompts information indicating that the second task can be executed via the HMI device 20.

[0247] Figure 13 and Figure 14 This represents an example of an action corresponding to this implementation method. Figure 13 Extract and show the changes corresponding to this action example. Figure 6 The surrounding area of ​​the part in the flowchart is changed. Figure 14 Corresponding to this action example, the changes were... Figure 7 It is part of the flowchart.

[0248] Specifically, refer to Figure 13 In step 601, the vehicle system 10 determines whether the conditions for starting autonomous driving are met. At this time, as described later, the setting process of the extremely short time flag Fs, which indicates whether the execution time of autonomous driving is extremely short, is performed. The processing content of steps 602 to 605 is the same as that in the first embodiment described above.

[0249] Following step 605, the vehicle system 10 proceeds to step 1301. In step 1301, the vehicle system 10 determines the setting state of the extremely short time flag Fs. Specifically, if the extremely short time flag Fs is reset (i.e., Fs = 0), the vehicle system 10 determines step 1301 as "yes," and if the extremely short time flag Fs is set (i.e., Fs = 1), the vehicle system 10 determines step 1301 as "no."

[0250] Without setting the extremely short time flag Fs (i.e., step 1301 = yes), the execution time of the autonomous driving that begins in this step 605 is not extremely short. Therefore, in this case, the vehicle system 10 advances the processing to step 606. In step 606, the vehicle system 10 performs a selection action for the second task content utilized in autonomous driving. Furthermore, in this embodiment, as the content selection process in step 606, the following can be applied: Figure 8 itself or based on Figure 12 The changed Figure 8 .

[0251] If the extremely short time flag Fs is set (i.e., step 1301 = No), the execution time of the autonomous driving that begins in this step 605 is extremely short. Therefore, in this case, the vehicle system 10 advances the processing to steps 1302 and 1303.

[0252] In step 1302, the vehicle system 10 displays a second task permission message on the CID display 221. The processing content in step 1302 is related to... Figure 8 The same applies to step 801 in the flowchart shown.

[0253] The second task permission message in step 1302 indicates that the driver can begin the second task. Therefore, the driver can freely operate the HMI device 20, select the desired second task content, and begin execution. Therefore, in step 1303, the vehicle system 10 determines whether a second task can be started by the driver.

[0254] If the second task has not yet started (i.e., step 1303 = No), the vehicle system 10 waits for a predetermined time. Specifically, if the determination result of step 1303 is "No", the vehicle system 10 proceeds to step 1304. In step 1304, the vehicle system 10 determines whether a predetermined standby time has elapsed since the second task permission message was displayed in step 1302. Before the standby time has elapsed (i.e., step 1304 = No), the vehicle system 10 returns the process to step 1303. On the other hand, after the standby time has elapsed (i.e., step 1304 = Yes), the vehicle system 10 proceeds the process to step 606. Thus, content selection processing is performed as usual.

[0255] If the second task start operation is performed before the standby time expires (i.e., step 1303 = Yes), the vehicle system 10 advances the processing to step 1305. In step 1305, the vehicle system 10 determines whether the second task executed by the second task start operation instruction can be performed. Specifically, for example, the vehicle system 10 determines whether the second task executed by the second task start operation instruction is subject to the aforementioned limitations that can be performed in a very short period of autonomous driving.

[0256] If the second task can be executed via the second task start operation instruction (i.e., step 1305 = Yes), the vehicle system 10 proceeds to step 607. In step 607, the vehicle system 10 begins execution of the second task content selected via the second task start operation. That is, in this case, the content selection process of step 606 is skipped. On the other hand, if the second task cannot be executed via the second task start operation instruction (i.e., step 1305 = No), the vehicle system 10 proceeds to step 1304.

[0257] Reference Figure 14 The processing in step 701 is the same as in the first embodiment described above. If the process proceeds to step 702, the vehicle is currently traveling within a specific area where high-speed autonomous driving can be performed.

[0258] If the environmental conditions are determined not to be met in step 702 (i.e., step 702 = No), the driving control ECU 17 terminates the autonomous driving start condition determination process after executing the processing in step 704. In step 704, the driving control ECU 17 determines that the autonomous driving start condition is not met. On the other hand, if the environmental conditions are determined to be met in step 702 (i.e., step 702 = Yes), the driving control ECU 17 advances the processing to steps 1401 and 1402.

[0259] Furthermore, the "environmental conditions" in this case do not include the absence of obstacles such as accidents or road construction within the predetermined autonomous driving range, and the duration Tc being above the specified lower limit of execution time, for the following reasons. This is because whether or not obstacles such as accidents or road construction occur within the predetermined autonomous driving range will be reflected in the length of the duration Tc. Additionally, in this embodiment, whether the duration Tc is above the lower limit of execution time is not determined by whether the environmental conditions in step 702 are met, but rather by whether an extremely short time flag Fs has been set.

[0260] In step 1401, the driving control ECU 17 acquires and calculates the duration Tc of the high-speed range automated driving. The processing content of step 1401 is related to... Figure 8 Steps 803 to 805 in the flowchart shown are the same.

[0261] In step 1402, the driving control ECU 17 determines whether the calculated duration Tc is greater than or equal to the limit time TcL. The limit time TcL can be equal to or equal to the aforementioned model-limited threshold TcS or the lower limit execution time, or it can be set with some difference between these. The limit time TcL < the first threshold Tc1. That is, the limit time TcL is a value that is sufficiently smaller than the first threshold Tc1 in step 808.

[0262] If the duration Tc of high-speed range autonomous driving is shorter than the time limit TcL (i.e., step 1402 = No), the driving control ECU 17 terminates the autonomous driving start condition determination process after executing the processing in step 704. In step 704, the driving control ECU 17 determines that the autonomous driving start condition is not met. That is, if the duration Tc of high-speed range autonomous driving is shorter than the time limit TcL, high-speed range autonomous driving is not executed, and therefore the execution of the second task is not allowed.

[0263] In contrast, if the duration Tc of high-speed range autonomous driving is greater than or equal to the limit time TcL (i.e., step 1402 = yes), the driving control ECU 17 terminates the autonomous driving start condition determination process after executing the processing in step 703. In step 703, the driving control ECU 17 determines that the start condition of high-speed range autonomous driving is met.

[0264] Figure 14 The processing content of step 705 is the same as in the first embodiment described above. Furthermore, the determination content of steps 706 and 707 is the same as in the first embodiment described above. If the conditions for starting autonomous driving are met during traffic congestion, the determination result in step 705 is "yes," and the determination result in either step 706 or step 707 is also "yes."

[0265] In this embodiment, when the start condition for autonomous driving during traffic congestion is met, the setting process of the extremely short time flag Fs, such as in step 1403, is performed before proceeding to step 703. Specifically, firstly, in step 1403, the driving control ECU 17 acquires and calculates the permissible duration Tc of autonomous driving during traffic congestion. The processing content of step 1403 is similar to... Figure 8 Steps 806 and 807 in the flowchart shown are the same.

[0266] Next, in step 1404, the driving control ECU 17 determines whether the duration Tc of automated driving during traffic congestion is greater than or equal to the limit time TcL. If the duration Tc of automated driving during traffic congestion is greater than or equal to the limit time TcL (i.e., step 1404 = Yes), the driving control ECU 17, after resetting the extremely short time flag Fs in step 1405 (i.e., Fs = 0), proceeds to step 703. On the other hand, if the duration Tc of automated driving during traffic congestion is shorter than the limit time TcL (i.e., step 1404 = No), the driving control ECU 17, after setting the extremely short time flag Fs in step 1406 (i.e., Fs = 1), proceeds to step 703.

[0267] In this embodiment, when the duration Tc of autonomous driving is extremely short, an extremely short time flag Fs is set (i.e., Fs = 1 in step 1406). Therefore, after autonomous driving begins but before content selection processing begins (i.e., step 606), a second task permission message is displayed (i.e., step 1302). Thus, even when autonomous driving is performed for an extremely short time, a second task permission message is displayed earlier to notify the driver or other personnel that they can utilize the second task. Therefore, according to this embodiment, even in extremely short periods of autonomous driving, the time available for the driver or other personnel to utilize the second task can be well ensured, improving convenience.

[0268] (Additional Implementation Method 3: Fifth Implementation Method)

[0269] The fifth embodiment will now be described. This embodiment relates to a technique that enables a smooth transition of driving when autonomous driving is performed for an extended period or over a long distance, by preventing the driver's awareness from being excessively distracted from driving the vehicle (i.e., driving status and driving operations) due to a second task. Specifically, in this embodiment, when the vehicle system 10, i.e., the HMI control device 23, can continue autonomous driving for a predetermined interruption standby time or longer from the start of the second task, the HMI device 23 provides information about the vehicle's driving status or surrounding information. "Surrounding information" includes information about the traffic environment related to the vehicle's surroundings (e.g., the presence of other vehicles ahead and in adjacent lanes) and / or traffic information related to the vehicle's surroundings. That is, "surrounding information" typically includes the road conditions of the vehicle's destination.

[0270] Figure 15 This represents an example of an operation corresponding to this embodiment. During the execution of the second task, the HMI control device 23 repeatedly starts at predetermined time intervals (e.g., 100 msec). Figure 15 The example program shows an interruption message. The so-called "second task in progress" is from... Figure 6 or Figure 13 The process in the flowchart shown advances to step 607. Figure 6 The flowchart shown illustrates the period during which the process progresses to step 610. Each process in such a routine corresponds to the action of the second task prompt unit 234 and / or the display control unit 236.

[0271] If such a routine is initiated, firstly, in step 1501, the HMI control device 23 determines whether the duration Tc of the automatic driving is greater than or equal to the specified interruption standby time Tw1. If the duration Tc is shorter than the interruption standby time Tw1 (i.e., step 1501 = No), step 1502 and all subsequent processing are skipped, and the current routine is temporarily terminated.

[0272] If the available duration Tc is greater than or equal to the interruption standby time Tw1 (i.e., step 1501 = Yes), the HMI control device 23 proceeds the process to step 1502. In step 1502, the HMI control device 23 determines whether the remaining time Tr is greater than or equal to the interruption standby time Tw1. The remaining time Tr is the time required from the current moment to the expected end time of the currently executing automatic driving. That is, the remaining time Tr is calculated by subtracting the duration Tw of the automatic driving from the available duration Tc calculated at the start of the automatic driving.

[0273] If the remaining time Tr is shorter than the interruption standby time Tw1 (i.e., step 1502 = No), step 1503 and all subsequent processing are skipped, and the current routine is temporarily terminated. On the other hand, if the remaining time Tr is greater than or equal to the interruption standby time Tw1 (i.e., step 1502 = Yes), the HMI control device 23 advances the processing to step 1503.

[0274] In step 1503, the HMI control device 23 determines whether the duration Tw is greater than or equal to the interruption standby time Tw1. The duration Tw is the time required from a predetermined continuous reference time to the current time. The continuous reference time is the closer to the current time between the start time of the currently executing autonomous driving and the time when the HMI device 20 last performed a prompt action regarding the vehicle's driving status or surrounding information.

[0275] If the duration Tw is shorter than the interruption standby time Tw1 (i.e., step 1503 = No), step 1504 and all subsequent processing are skipped, and the routine is temporarily terminated. On the other hand, if the duration Tw is longer than the interruption standby time Tw1 (i.e., step 1503 = Yes), the HMI control device 23 advances the processing to step 1504.

[0276] In step 1504, the HMI control device 23 determines whether the current activity is in the middle of the execution or use of specific content. The so-called "specific content" refers to a type of secondary task content that would cause annoyance to the occupant if an interruption with an information prompt occurred during its use or execution, such as movies, games, etc.

[0277] If, during the execution or use of a specific task (i.e., step 1504 = yes), the HMI device 20 displays information about the vehicle's driving status or surroundings, it would annoy the occupant currently performing the second task. Therefore, in this case, such information display is preferably delayed until the specific task temporarily ends. Consequently, in this situation, the HMI control device 23 skips steps 1505 and 1506 and temporarily terminates the current routine.

[0278] If the current activity is not in the middle of executing or utilizing the specific content (i.e., step 1504 = No), even if the HMI device 20 provides information about the vehicle's driving status or surroundings at the current moment, the likelihood of the occupants using the second task feeling annoyed is low. Therefore, in this case, it is preferable to provide such information at the current moment. Thus, in this case, the HMI control device 23 temporarily terminates the routine after executing steps 1505 and 1506 related to the information prompt action. Furthermore, if the specific content is a game, there are scenarios where the game temporarily stops depending on the state of the player's character. For example, a scenario where the player's character is defeated by an enemy character, or a scenario awaiting input based on the player's choice, corresponds to this scenario. In the event of such a scenario, the determination result of step 1504 may be "No".

[0279] In step 1505, the HMI control device 23 displays information about the vehicle's driving status or surrounding environment via the HMI device 20. In step 1506, the HMI control device 23 resets the duration Tw. Thus, the vehicle's driving status or surrounding environment can be repeatedly displayed at intervals of interrupted standby time Tw1 until the currently executing autonomous driving mode ends.

[0280] In this embodiment, when autonomous driving is performed for a long time or over a long distance, at least some of the vehicle information or surrounding information, namely driving status, traffic environment, and traffic information, is prompted at least once before the driving transition. This enables a smooth driving transition at the end of autonomous driving. Furthermore, by providing information prompts at the moment of interruption or end of the second task content, annoyance to occupants currently using the second task can be effectively avoided. Moreover, from this perspective, it is preferable, for example, to alternately suggest long-duration and short-duration content via the second task prompting unit 234.

[0281] (Summary of Additional Implementation Methods)

[0282] The duration of "autopilot" operation, equivalent to SAE Levels 3-5, which does not require mandatory monitoring of the surroundings of the driver or other occupants, can vary depending on road type and the conditions around the vehicle. Therefore, regardless of whether the duration of autopilot operation without mandatory monitoring of the surroundings is long or short, a convenient driving environment for the driver or other occupants is necessary. In this regard, according to the third to fifth embodiments described above, even when the duration of autopilot operation without mandatory monitoring of the surroundings is extremely short, a convenient driving environment can still be provided to the driver or other occupants.

[0283] (Modified Example)

[0284] This disclosure is not limited to the embodiments described above. Therefore, appropriate modifications can be made to the embodiments described above. Hereinafter, representative modifications will be described. In the following description of the modifications, the differences from the embodiments described above will be mainly explained. In addition, in the embodiments and modifications described above, the same reference numerals are used to mark the same or equivalent parts. Therefore, in the following description of the modifications, for constituent elements having the same reference numerals as those in the embodiments described above, the descriptions in the embodiments described above can be appropriately referenced unless there is a technical contradiction or special additional explanation.

[0285] This disclosure is not limited to the specific device structure shown in the above embodiments. That is, for example, the vehicle equipped with the vehicle system 10 is not limited to a four-wheeled car. Specifically, such a vehicle can also be a three-wheeled car, or a six-wheeled or eight-wheeled car such as a truck. The type of vehicle can also be a conventional car with only an internal combustion engine, or an electric car or fuel cell vehicle without an internal combustion engine, or a so-called hybrid vehicle. The shape and structure of the vehicle body are not limited to a box shape, i.e., roughly rectangular when viewed from above. There are also no particular limitations on the purpose of the vehicle, the position of the steering wheel, the number of passengers, etc.

[0286] As the communication standard constituting the vehicle system 10, standards other than CAN (International Registered Trademark) can also be used, such as FlexRay (International Registered Trademark). Furthermore, the communication standard constituting the vehicle system 10 is not limited to one type. For example, the vehicle system 10 may also have sub-network lines based on communication standards such as LIN. LIN is an abbreviation for Local Interconnect Network.

[0287] The vehicle status sensor 11, the external status sensor 12, and the surrounding monitoring sensor 13 are not limited to the examples described above. For instance, the surrounding monitoring sensor 13 may also include a sonar, i.e., an ultrasonic sensor. Alternatively, the surrounding monitoring sensor 13 may also include two or more of the following sensors: a millimeter-wave radar sensor, a sub-millimeter-wave radar sensor, a lidar sensor, and an ultrasonic sensor. There is no particular limitation on the number of each type of sensor.

[0288] The locator 14 is not limited to the example described above. For example, the locator 14 may not have a structure that integrates a gyroscope sensor and an accelerometer sensor. Specifically, the inertial acquisition unit 142 may also receive output signals from an external angular velocity sensor and an accelerometer sensor, which are provided on the locator 14 as vehicle state sensors 11.

[0289] DCM15 can be omitted. That is, traffic information can be obtained by navigation device 16. Alternatively, navigation device 16 may also have a structure that includes locator 14 and DCM15.

[0290] The navigation device 16 can also be connected to the HMI control device 23 via a sub-communication line different from the vehicle communication line 10A.

[0291] The navigation device 16 may also have a dedicated navigation screen display, different from that of the HMI device 20. Alternatively, the navigation device 16 may be configured as part of the HMI device 20. Specifically, for example, the navigation device 16 may also be integrated with the CID device 22.

[0292] In the road traffic regulations of various countries, the types of autonomous driving and the maximum speed for autonomous driving, as well as the execution conditions, can be appropriately considered according to domestic circumstances. Therefore, the above-described implementation can be appropriately modified according to the specifications of the road traffic regulations of each country. Specifically, for example, in the above-described implementation, the driving control ECU 17 is configured to perform both high-speed range autonomous driving, which allows driving at high speeds within a specific area, and traffic congestion autonomous driving, which allows driving at medium to low speeds during traffic jams. However, this disclosure is not limited to this approach. That is, for example, the driving control ECU 17 may only be able to perform high-speed range autonomous driving. Alternatively, the driving control ECU 17 may only be able to perform traffic congestion autonomous driving.

[0293] Automated driving during traffic congestion can also operate only on dedicated motor vehicle lanes, regardless of whether a specific zone is set. Alternatively, automated driving during traffic congestion can also operate only within a specific zone. In these cases, speed control that ensures the driving speed during temporary traffic congestion relief does not exceed the upper speed limit for automated driving during traffic congestion is particularly beneficial. Furthermore, the driving conditions in automated driving during traffic congestion can be appropriately set according to the road traffic regulations of each country. Specifically, for example, in the above embodiment, the lower limit speed for relieving automated driving during traffic congestion is 10 km / h, and the upper limit speed is 60 km / h. However, this disclosure is not limited to this approach. That is, for example, automated driving during traffic congestion also includes a temporary traffic congestion relief state, where it can stably operate without further operation at speeds below or less than 60 km / h. Automated driving under such conditions can also be called "low-speed automated driving." "Low-speed automated driving" can also be performed regardless of whether traffic congestion occurs.

[0294] In the above embodiments, the driving control ECU 17 is configured to perform vehicle control actions corresponding to SAE levels 1 to 3. However, this disclosure is not limited to this configuration. That is, for example, this disclosure can be appropriately applied to situations where vehicle control actions corresponding to SAE levels 1 to 5 can be performed. Furthermore, the level or category of driving automation in this disclosure is not limited to the level or category specified in "SAE J3016". Moreover, the vehicle speed control unit 173 can be configured as a vehicle control unit that performs vehicle motion control sub-tasks such as steering control and braking control in addition to vehicle speed control.

[0295] DSM18 can also be connected to HMI control unit 23 via a sub-communication line different from vehicle communication line 10A in an information communication manner.

[0296] DSM18 is not limited to a structure that checks the driver's gaze or facial orientation through image recognition. That is, for example, DSM18 can also have a structure that checks the driver's seating posture and steering wheel grip through a different type of sensor than an image sensor.

[0297] The operation unit 19 may also have a voice input device for checking the driver's speech.

[0298] The operation unit 19 can also be configured as part of the HMI device 20.

[0299] The HMI device 20 is not limited to a structure that includes an instrument panel 21 and a CID device 22. That is, for example, the HMI device 20 may also include a head-up display. Therefore, the HMI control device 23 may replace the CID display 221 or, together with the CID display 221, display various information and messages displayed on the CID display 221 in the above embodiments on the head-up display. Furthermore, there are no particular limitations on the function and structure of the head-up display.

[0300] As a head-up display, for example, a display that overlays a virtual image onto the foreground containing the road surface ahead of the vehicle can be used. "Overlay display" refers to displaying the associated information (e.g., building name) of overlapping objects (e.g., buildings) contained in the foreground in a manner that overlaps with or near the overlapping objects, thereby establishing a mutual association between the overlapping objects and the associated information. Displaying the path ahead of the road surface, displaying the direction of travel, and displaying traffic information are also equivalent to "overlay display."

[0301] Instrument 211 and instrument display 212 can be implemented by a single display device. In this case, instrument 211 can be configured as the display area at the left and right ends of a display device that is a liquid crystal or organic EL display. That is, instrument 211 can display the bezels, pointers, scales, etc., corresponding to those of a tachometer, speedometer, water thermometer, etc., through image display. Furthermore, instrument display 212 can be configured as a display area other than instrument 211 within such a display device.

[0302] Input device 222 may also replace the touch panel superimposed on CID display 221, or together with the touch panel, have an indicator device that can be manually operated by the driver. Input device 222 may also have a voice input device for checking the driver's speech.

[0303] In the above embodiments, the driving control ECU 17 and the HMI control device 23 have a structure that serves as a so-called on-board microcomputer equipped with a CPU or the like. However, this disclosure is not limited to such a structure.

[0304] For example, all or part of the driving control ECU 17 can also be configured as digital circuits capable of performing the actions described above, for example, with an ASIC or FPGA structure. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field Programmable Gate Array. That is, in the driving control ECU 17, the on-board microcomputer part and the digital circuit part can coexist. The same applies to the HMI control device 23.

[0305] The various actions, steps, or programs of this disclosure capable of performing processing described in the above embodiments can be downloaded or upgraded via V2X communication based on DCM15, etc. Alternatively, such programs can be downloaded or upgraded via terminal devices installed in vehicle manufacturing plants, installation plants, sales outlets, etc. The storage destination for such programs can also be a memory card, optical disc, disk, etc.

[0306] In this way, the aforementioned functional structures and methods can also be implemented by a dedicated computer, which is provided by a processor and memory programmed to perform one or more functions embodied in a computer program. Alternatively, the aforementioned functional structures and methods can also be implemented by a dedicated computer provided by a processor composed of one or more dedicated hardware logic circuits. Alternatively, the aforementioned functional structures and methods can also be implemented by one or more dedicated computers composed of a combination of a processor and memory programmed to perform one or more functions and a processor composed of one or more hardware logic circuits. In addition, the computer program can also be stored as instructions executed by the computer in a computer-readable, non-transferable storage medium. That is, the aforementioned functional structures and methods can also be represented as a computer program containing steps for implementing each functional structure and method, or a storage medium storing the program in a non-transferable entity.

[0307] This disclosure is not limited to the specific functional structures and operational examples shown in the above embodiments. For example, various messages such as "continue suggestion message" and "attention ahead message" may replace CID display 221 or be displayed together with CID display 221 on other display devices. "Other display devices" include, for example, instrument display 212 and / or head-up display (not shown).

[0308] The Continuing Operation Acceptance Unit 235 can also be configured as an operation acceptance unit that accepts various input operations other than continuing operations.

[0309] Figure 2 The appearance design and operating functions shown in the display screen can also be appropriately changed. For example, refer to Figure 2 By sliding within the central display area DA1, the same function as the category change button DA4 or content change button DA5 can be achieved.

[0310] Figure 6 The flowchart shown can also be modified appropriately. For example, if the determination result in step 604 is "no", the same waiting time for approval operation as in step 910 is usually performed. However, in Figure 6 In the simplified relationship between the accompanying drawings and the description in the instruction manual, the illustrations and explanations of such standby processing are omitted.

[0311] In step 607, the HMI control device 23 may also execute a report indicating the start of the second task. This report may be given via text display and / or audio output.

[0312] The boundary between short-duration and long-duration content is not limited to 30 minutes. For example, the boundary could be 45 minutes, 60 minutes, or 75 minutes. Furthermore, the types of second task content are not limited to short-duration and long-duration content. For example, the types of second task content can be categorized into three or more types based on the execution time required. In this case, Figure 8 In the processing steps below 808, a determination can be added to check whether the duration Tc is less than the third threshold TcZ. Z is a natural number greater than or equal to 3.

[0313] exist Figure 7 In the process of determining the start condition of the automated driving system, the process of determining whether the interval condition is met is first performed (i.e., step 701). However, this disclosure is not limited to this approach. For example, the process of determining whether the traffic congestion condition is met may be performed first (i.e., step 705). In such a variation, if the traffic congestion condition is not met (i.e., step 705 = no), the driving control ECU 17 advances the process to step 701 to determine whether the interval condition is met. The process of step 702 may also be performed if the determination result in step 706 or step 707 is "yes".

[0314] The traffic congestion conditions in step 705 are not limited to the specific examples mentioned above. For instance, a traffic congestion condition can also be determined if there are other vehicles within a specified distance ahead of the vehicle, other vehicles traveling at low speed in the adjacent lane of the vehicle's lane, and the vehicle's speed is less than a specified speed threshold. The specified speed threshold is, for example, 10 km / h.

[0315] Steps 706 and / or 707 can be omitted. That is, the current state of having obtained the latest traffic information through the information acquisition unit 171 and / or the traffic congestion condition lasting for a specified time can be removed from the starting conditions for autonomous driving during traffic congestion.

[0316] Even when traffic congestion information is unavailable from traffic data sources, the predicted traffic congestion time Tj can still be calculated. Specifically, for example, specific driving intervals with a high frequency of traffic congestion can be statistically identified on a map. These intervals are called "traffic congestion prediction intervals." Therefore, if a vehicle enters an actual traffic congestion interval while traveling within a traffic congestion prediction interval, the predicted traffic congestion time Tj can be calculated based on the vehicle's average speed during the current traffic congestion period and the remaining distance within the traffic congestion prediction interval.

[0317] Figure 11 The second task shown includes display control processing and Figure 15The information interruption prompt processing shown can be combined. Specifically, for example, in Figure 11 In the flowchart shown, if the result of any one of steps 1108 to 1110 is "no", the process can still be executed. Figure 15 The information interruption prompt handling is shown below. Figure 15 The processing content of step 1505 in the middle can also be related to Figure 11 The processing content of step 1111 is the same.

[0318] "Less than the threshold" can be changed to "below the threshold". Similarly, "above the threshold" and "exceeding the threshold" can be interchanged. That is, "<" and "≤" can be interchanged. Similarly, ">" and "≥" can be interchanged.

[0319] Expressions such as "acquire," "calculate," "infer," "check," "detect," and "determine" can be appropriately substituted for each other within the scope of technical inconsistency. Similarly, "check," "detect," and "extract" can be appropriately substituted for each other within the scope of technical inconsistency.

[0320] Of course, the elements constituting the above embodiments are not necessarily essential, except where they are specifically stated to be necessary or where they are obviously necessary in principle. Furthermore, when referring to numerical values ​​such as the number, quantity, or range of constituent elements, this disclosure is not limited to those specific values, except where they are specifically stated to be necessary or where they are obviously limited to specific values ​​in principle. Similarly, when referring to the shape, orientation, or positional relationship of constituent elements, this disclosure is not limited to those shapes, orientations, or positional relationships, except where they are specifically stated to be necessary or where they are limited to specific shapes, orientations, or positional relationships in principle.

[0321] The variations are not limited to the examples described above. For instance, multiple embodiments can be combined with each other as long as they are not technically contradictory. Furthermore, multiple variations can be combined with each other as long as they are not technically contradictory. Moreover, all or part of the above embodiments can be combined with all or part of the variations, as long as they are not technically contradictory.

[0322] (Summarize)

[0323] (1) The present disclosure, as illustrated by the above embodiments and variations, includes the following viewpoints related to HMI control methods and HMI control procedures. Furthermore, the following viewpoints can be combined and applied to each other as long as they are not technically contradictory.

[0324] According to the first viewpoint, the HMI control method is a method for controlling an HMI device (20) installed in an autonomous vehicle, comprising:

[0325] Processing to obtain the duration of autonomous driving; and

[0326] Based on the obtained duration, the HMI device prompts the passenger in the driver's seat to perform a second task during autonomous driving.

[0327] Furthermore, the HMI control program is a program executed by the HMI control device (23), which is configured to control the HMI device (20) installed in the autonomous driving vehicle.

[0328] The processes performed by the aforementioned HMI control device include:

[0329] Processing to obtain the duration of autonomous driving; and

[0330] Based on the obtained duration, the HMI device prompts the passenger in the driver's seat to perform a second task during autonomous driving.

[0331] According to the second viewpoint, the process of obtaining the aforementioned duration includes: obtaining the aforementioned duration based on a distance of a pre-set autonomous driving range.

[0332] According to the third viewpoint, the aforementioned process for obtaining the duration includes: when autonomous driving is initiated due to traffic congestion, obtaining the aforementioned duration based on the predicted duration of such traffic congestion.

[0333] The fourth point also includes accepting the input operation acceptance processing for continuing the aforementioned second task when traffic congestion is detected to be temporarily relieved.

[0334] The fifth point also includes the process of obtaining and processing the results of the inspection of the behavior of the aforementioned driver's seat passenger.

[0335] The aforementioned continued operation acceptance process includes: when the aforementioned action acquisition process obtains the aforementioned inspection result indicating that the driver's seat passenger is confirming the road conditions at the vehicle's destination, the process of validating the aforementioned input operation.

[0336] According to the sixth point, the aforementioned vehicles are capable of autonomous driving within the prescribed speed range during traffic congestion.

[0337] The above-mentioned continued operation acceptance process includes: when the acceptance of the above-mentioned input operation has been valid, in order to control the driving speed of the vehicle by the driving control device (17) that controls the driving of the vehicle in a manner that does not exceed the above-mentioned prescribed speed range, the process of notifying the driving control device that the above-mentioned input operation has been valid.

[0338] The seventh point also includes: when the above-mentioned behavior acquisition unit obtains the above-mentioned inspection results indicating that the driver's seat passenger is confirming the above-mentioned road conditions, the above-mentioned HMI device displays the above-mentioned road conditions.

[0339] The eighth point also includes:

[0340] Behavior acquisition and processing: acquiring the inspection results of the aforementioned behavior of the driver's seat passenger; and

[0341] When the display control processing obtains the above-mentioned inspection results indicating that the driver's seat passenger is confirming the road conditions at the vehicle's destination through the above-mentioned behavior acquisition processing, the above-mentioned road conditions are displayed through the above-mentioned HMI device.

[0342] According to the ninth point of view, the above-mentioned processing of prompting the above-mentioned second task through the above-mentioned HMI device includes: when the above-mentioned available duration cannot be obtained, selectively prompting the processing of multiple above-mentioned second tasks with different execution times.

[0343] The tenth point includes: when the duration of driving in a pre-set autonomous driving zone is shorter than the specified minimum execution time, the execution of a second task in such a driving zone is not allowed.

[0344] The eleventh point includes: regardless of the length of the aforementioned duration obtained based on the continuous prediction time of traffic congestion, the processing of the execution of the second task during traffic congestion is permitted.

[0345] According to the twelfth point of view, the above-mentioned processing of prompting the second task through the above-mentioned HMI device includes: when the above-mentioned duration obtained based on the continuous prediction time of traffic congestion is shorter than the specified model-limited time, the above-mentioned HMI device performing the second task in traffic congestion is limited to vehicle-mounted equipment (22).

[0346] According to the twelfth point of view, the above-mentioned processing of prompting the second task through the HMI device includes: when the available duration is shorter than the predetermined time limit, prompting information indicating that the second task can be performed through the HMI device, instead of prompting the second task based on the available duration, or processing prior to such prompting.

[0347] The thirteenth point includes: when the autonomous driving can continue for more than the specified interruption standby time after the start of the second task mentioned above, the HMI device is used to prompt the vehicle's driving status or surrounding information.

[0348] (2) The present disclosure, as illustrated by the above embodiments and variations, includes the following viewpoints related to the driving control method and driving control program. Furthermore, the following viewpoints can be combined and applied to each other as long as they are not technically contradictory.

[0349] According to the first viewpoint, driving control methods are methods for controlling a vehicle capable of automatic driving within a specified speed range during traffic congestion, including:

[0350] Information acquisition and processing, at least the driving status of the aforementioned vehicles;

[0351] The driving level determination process, based on the driving status obtained through the aforementioned information acquisition process, determines whether autonomous driving can be performed; and

[0352] The vehicle speed control process determines whether autonomous driving can be performed based on the aforementioned driving level decision process, and then controls the vehicle's speed accordingly.

[0353] The aforementioned information acquisition process includes: acquiring the traffic congestion status as the aforementioned driving state, and acquiring the execution status of the second task performed by the passenger in the driver's seat from the HMI control device (23) controlling the HMI device (20) mounted on the aforementioned vehicle.

[0354] The aforementioned speed control process includes: when the acquired traffic congestion status is that the traffic congestion has been temporarily relieved, and there is a request to continue the aforementioned second task being performed, the process of increasing the driving speed within the aforementioned specified speed range.

[0355] Furthermore, the driving control program is a driving control program executed by the driving control device (17), which is configured to control the driving of a vehicle that can automatically drive within a specified speed range during traffic congestion.

[0356] The processes performed by the aforementioned driving control device include:

[0357] Information acquisition and processing, at least the driving status of the aforementioned vehicles;

[0358] The driving level determination process, based on the driving status obtained through the aforementioned information acquisition process, determines whether autonomous driving can be performed; and

[0359] The vehicle speed control process determines whether autonomous driving can be performed based on the aforementioned driving level decision process, and then controls the vehicle's speed accordingly.

[0360] The aforementioned information acquisition process includes: acquiring the traffic congestion status as the aforementioned driving state, and acquiring the execution status of the second task performed by the passenger in the driver's seat from the HMI control device (23) controlling the HMI device (20) mounted on the aforementioned vehicle.

[0361] The aforementioned speed control process includes: when the acquired traffic congestion status is that the traffic congestion has been temporarily relieved, and there is a request to continue the aforementioned second task being performed, the process of increasing the driving speed within the aforementioned specified speed range.

[0362] According to the second viewpoint,

[0363] The aforementioned driving level determination process includes: when driving on a dedicated road with a minimum speed limit within the aforementioned speed range during traffic congestion, deciding to engage automatic driving.

[0364] The aforementioned speed control process includes: when the obtained driving status is that traffic congestion on the dedicated road has been temporarily relieved and there is a request to continue, controlling the driving speed within the specified speed range to be above the minimum speed limit.

[0365] According to the third viewpoint, the aforementioned driving level determination process includes: when the input operation for continuing the aforementioned second task is effectively accepted by the aforementioned HMI device and the aforementioned continuation request is notified from the aforementioned HMI device, the process of deciding to continue autonomous driving is included.

[0366] According to the fourth point of view, the aforementioned driving level decision process includes: when the result of checking the behavior of the driver's seat passenger is to confirm the road conditions of the destination of the vehicle, as the input operation has been effectively accepted by the aforementioned HMI device, the process of deciding to continue autonomous driving is carried out.

[0367] According to the fifth point of view, the above-mentioned driving level determination process includes: when the duration of driving in a pre-set autonomous driving zone is shorter than the prescribed minimum execution time, the execution of autonomous driving in such a driving zone is not allowed.

[0368] According to the sixth point, the aforementioned driving level determination process includes: a process that allows the execution of autonomous driving while driving in traffic congestion, regardless of the duration of the forecast based on the duration of traffic congestion.

Claims

1. A driving control device configured to control the driving of a vehicle capable of automatic driving within a specified speed range during traffic congestion, comprising: The information acquisition department shall at least acquire the driving status of the aforementioned vehicles; The driving level determination unit determines whether autonomous driving can be performed based on the driving state obtained by the information acquisition unit. The vehicle speed control unit controls the vehicle's speed based on whether automatic driving can be performed, as determined by the driving level determination unit. The aforementioned information acquisition unit acquires the traffic congestion status as the aforementioned driving state, and acquires the execution status of the second task performed by the passenger in the driver's seat from the HMI control device, wherein... The aforementioned HMI control device controls the HMI device mounted on the aforementioned vehicle. If the traffic congestion is temporarily relieved and there is a request to continue the second task in progress, the speed control unit will increase the driving speed within the specified speed range.

2. The driving control device according to claim 1, wherein, When driving on a dedicated road with a minimum speed limit within the aforementioned speed range during periods of traffic congestion, the driving level determination unit decides to engage automatic driving. If the obtained driving status is that the traffic congestion on the dedicated road has been temporarily relieved and there is a request to continue, the speed control unit will control the driving speed within the specified speed range to be above the minimum speed limit.

3. The driving control device according to claim 1, wherein, If the input operation for continuing the second task is effectively accepted by the HMI device and the continued request is notified by the HMI device, the driving level determination unit decides to continue automatic driving.

4. The driving control device according to claim 3, wherein, If the inspection of the driver's seat passenger's behavior indicates that the main objective is to confirm the road conditions at the vehicle's destination, the driving level determination unit, having effectively processed the input operation through the HMI device, decides to continue automatic driving.

5. A storage medium storing a driving control program executed by a driving control device configured to control the driving of a vehicle capable of automatic driving within a specified speed range during traffic congestion. The driving control program executed by the aforementioned driving control device includes: At least the processing of the driving status of the aforementioned vehicles; Based on the obtained driving status, a decision is made as to whether autonomous driving processing can be performed; and Based on the determination of whether autonomous driving can be performed, the processing controls the speed of the aforementioned vehicles. In the process of at least obtaining the aforementioned driving state of the vehicle, the traffic congestion state, which is the aforementioned driving state, is obtained, and the execution state of the second task performed by the passenger in the driver's seat is obtained from the HMI control device that controls the HMI device mounted on the vehicle. In the process of controlling the aforementioned driving speed, if the traffic congestion is temporarily relieved and there is a request to continue the second task being performed, the driving speed is increased within the aforementioned specified speed range.

Images