control device

Abstract

To provide a control device capable of appropriately notifying warnings while in autonomous driving mode.SOLUTION: A control unit is configured to provide control to notify both a vehicle and a mobile terminal when the mobile terminal is connected while in autonomous driving mode (106), and block notifications to the mobile terminal to notify only the vehicle side when predetermined conditions are met (104).SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a control device that controls notifications to an in-vehicle terminal mounted on a vehicle and a mobile terminal carried by a passenger.

Background Art

[0002] Patent Document 1 discloses a driver monitoring system that communicates between a mobile terminal including a display screen and a camera arranged to be able to photograph the side facing the display screen, and a vehicle control device that controls the host vehicle, monitors the driver of the host vehicle, and determines whether the driver of the host vehicle is looking at the display screen of the mobile terminal based on an image captured by the camera while the host vehicle is running. When it is determined that the driver is looking at the display screen of the mobile terminal, an alarm is issued. Specifically, it is disclosed that when the driver is at an awareness level that requires a warning during autonomous driving, a warning is given by display or voice from the mobile terminal and the vehicle.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When the driver is performing a secondary task other than driving during autonomous driving, notifying a warning to the mobile terminal may cause an act of searching for the mobile terminal, and there is a possibility of delay in responses such as driver changeovers. In addition, constantly notifying a warning to the mobile terminal in the vehicle may be troublesome.

[0005] The present invention has been made in consideration of the above facts, and an object thereof is to provide a control device capable of appropriately notifying a warning during autonomous driving.

Means for Solving the Problems

[0006] The control device according to the first embodiment includes a notification unit that notifies an in-vehicle terminal mounted in the vehicle and a portable terminal carried by the occupant of a warning during autonomous driving, and a control unit that controls the notification unit to mute the portable terminal and notify the in-vehicle terminal of a warning when predetermined conditions are met.

[0007] According to the first embodiment, the notification unit notifies an in-vehicle terminal installed in the vehicle and a handheld terminal carried by the occupant of a warning during autonomous driving.

[0008] The control unit then controls the notification unit to mute the mobile terminal and send a warning to the in-vehicle terminal when predetermined conditions are met. This makes it possible to appropriately notify warnings during autonomous driving.

[0009] The control device according to the second embodiment, in the control device according to the first embodiment, if the driver is performing a secondary task other than driving and does not have the mobile terminal in their hand, the control unit will not notify the driver of the warning on the mobile terminal.

[0010] According to the second embodiment, it is possible to prevent the act of searching for a mobile device and eliminate the cause of delays in responding to situations such as driver changes.

[0011] The control device according to the third embodiment, in the control device according to the first or second embodiment, if the driver's mobile terminal is not placed in a predetermined location, the control unit will not notify the driver's mobile terminal of the warning.

[0012] According to the third embodiment, if the mobile device is not in its designated place such as a holder, the act of searching for the mobile device will occur. However, by making the notification muted, the act of searching for the mobile device can be prevented, thereby reducing delays in responding to situations such as driver changes.

[0013] The control device according to the fourth embodiment further comprises an identification unit that identifies the driver's mobile terminal in any one of the first to third embodiments, and the control unit, when there are multiple mobile terminals in the vehicle, disables the notification of warnings to mobile terminals other than the driver's mobile terminal identified by the identification unit.

[0014] According to the fourth aspect, unnecessary notifications to passengers other than the driver can be reduced, thereby alleviating inconvenience.

[0015] In the control device according to the fifth embodiment, in the fourth embodiment, the control unit disables notifications of warnings below a predetermined severity level for the mobile terminal other than the driver.

[0016] According to the fifth aspect, less important notifications to passengers other than the driver can be suppressed, thereby reducing annoyance. [Effects of the Invention]

[0017] As described above, the present invention provides a control device that can appropriately notify warnings during autonomous driving. [Brief explanation of the drawing]

[0018] [Figure 1] This figure shows the schematic configuration of the vehicle information processing system according to this embodiment. [Figure 2] This is a functional block diagram showing the schematic configuration of the in-vehicle unit and mobile terminal of the vehicle information processing system according to this embodiment. [Figure 3] This is a block diagram showing the configuration of the in-vehicle device's control unit and mobile terminal. [Figure 4] This flowchart shows an example of the processing flow performed by the control unit of the in-vehicle device in the vehicle information processing system according to this embodiment. [Figure 5] This diagram illustrates an example where an in-vehicle device obtains identification information from the driver's mobile device via a server. [Modes for carrying out the invention]

[0019] Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a diagram showing a schematic configuration of a vehicle information processing system according to the present embodiment.

[0020] The vehicle information processing system 10 according to the present embodiment includes a control device mounted on the vehicle 12 and an in-vehicle device 14 as an example of an in-vehicle terminal, and a portable terminal carried by a passenger of the vehicle 12.

[0021] The in-vehicle device 14 and the portable terminal 16 can be connected by various short-range wireless communications such as Wi-Fi (registered trademark) and Bluetooth (registered trademark).

[0022] FIG. 2 is a functional block diagram showing a schematic configuration of the in-vehicle device 14 and the portable terminal 16 of the vehicle information processing system 10 according to the present embodiment.

[0023] As shown in FIG. 2, the in-vehicle device 14 has functions of a vehicle information detection unit 20, a photographing unit 22, an automatic driving control unit 24, a communication unit 26, a display unit 28 as an example of a notification unit, and a control unit 30.

[0024] The vehicle information detection unit 20 detects vehicle information related to the vehicle 12. For example, it detects vehicle information such as the vehicle's position, acceleration, accelerator pedal position, distance to obstacles around the vehicle, and route. Specifically, the vehicle information detection unit 20 can apply multiple types of sensors and devices to acquire information representing the conditions of the surrounding environment of the vehicle 12. Examples of sensors and devices include sensors mounted on the vehicle 12, such as a vehicle speed sensor, steering angle sensor, and acceleration sensor, as well as a GNSS (Global Navigation Satellite System) device, an in-vehicle communication device, a navigation system, and a radar device. The GNSS device determines the position of the vehicle 12 by receiving GNSS signals containing time information from multiple GNSS satellites. The accuracy of positioning improves as the number of receivable GNSS signals increases. The in-vehicle communication device is a communication device that performs at least one of vehicle-to-vehicle communication with other vehicles 12 and vehicle-to-infrastructure communication with roadside devices. The navigation system includes a map information storage unit that stores map information, and performs processing to display the position of the vehicle 12 on a map and to guide the vehicle to a destination based on the position information obtained from the GNSS device and the map information stored in the map information storage unit. The radar device includes multiple radars with different detection ranges, and detects objects such as pedestrians and other vehicles 12 that are present around the vehicle 12, and acquires the relative position and relative speed of the detected objects and the vehicle 12. The radar device also incorporates a processing unit that processes the detection results of surrounding objects. This processing unit excludes noise and roadside objects such as guardrails from the monitoring target based on changes in the relative position and relative speed of individual objects included in the most recent multiple detection results, and tracks and monitors pedestrians and other vehicles 12 as monitored objects. The radar device then outputs information such as the relative position and relative speed of each monitored object. In this embodiment, all of this information may be detected as vehicle information, or any of the information may be detected as vehicle information.

[0025] The camera unit 22 is mounted inside the vehicle and captures images of the driver and the area around the vehicle 12, generating image data that represents the captured images for the video. The camera unit 22 also captures images of at least one area around the vehicle 12, including the front, side, and rear of the vehicle 12.

[0026] The communications unit 26 establishes communication with the mobile terminal 16 inside the vehicle via short-range wireless communication and transmits and receives various types of information.

[0027] The automatic driving control unit 24 controls the automatic driving of the vehicle 12 based on the detection results of the vehicle information detection unit 20 and the shooting results of the shooting unit 22. The automatic driving control unit 24 is assumed to be capable of automatic driving at least level 3 (conditional automatic driving). In this embodiment, the definition of automatic driving level is adopted from that defined by the SAE (Society of Automotive Engineers).

[0028] Specifically, the automatic driving control unit 24 controls the switching between automatic driving, which controls the autonomous driving of the vehicle 12 by judging the surrounding conditions of the vehicle 12 and the condition of the vehicle 12, and manual driving by the occupant. When automatic driving is enabled, the automatic driving control unit 24 judges the conditions of the vehicle 12 and its surroundings based on information obtained from the vehicle information detection unit 20 and the imaging unit 22 in order to enable the vehicle 12 to drive autonomously. Then, according to the judgment result, it performs automatic driving control processing to control actuators that drive the accelerator amount, brake amount, steering angle, etc. In the automatic driving control processing, a driving plan along a pre-set target route is generated based on the surrounding conditions of the vehicle 12 and map information, and the driving is controlled so that the vehicle 12 drives autonomously according to the generated driving plan.

[0029] Furthermore, the automatic driving control unit 24 can automatically start automatic driving control processing in response to the behavior of the vehicle 12 and the surrounding conditions while the occupant is manually driving the vehicle 12, thereby assisting the occupant in driving the vehicle 12. In this case, the automatic driving control unit 24 outputs information indicating the start of automatic driving control processing. Also, in cases where the system switches from manual driving by the occupant to automatic driving by automatic driving control processing at the occupant's instruction, the automatic driving control unit 24 outputs information indicating that the transfer of sovereignty of driving has occurred, that is, information indicating the start of automatic driving control processing. In addition, while the automatic driving control processing is being executed, the automatic driving control unit 24 can automatically terminate (cancel) the automatic driving control processing in response to the behavior of the vehicle 12 and the surrounding conditions, thereby prompting the occupant to manually drive the vehicle 12. In this case, the automatic driving control unit 24 outputs information indicating the end of automatic driving control processing in order to prompt the occupant to manually drive the vehicle 12. Also, in cases where the system switches from automatic driving to manual driving by the occupant at the occupant's instruction, the automatic driving control unit 24 outputs information indicating that the transfer of sovereignty of driving has occurred, that is, information indicating the end of automatic driving control processing. In the case of Level 3 autonomous driving, it is necessary to confirm the driver, but depending on the position of the handheld mobile device 16 and the driver's posture, the position of the camera unit 22 that photographs the driver may cause the mobile device 16 to obscure the driver's face within the field of view, preventing the system from understanding the driver's status and causing it to fail to meet the conditions for operating autonomous driving. In this case, if the driver can be confirmed from the image captured by the camera of the mobile device 16, autonomous driving may be maintained.

[0030] The communications unit 26 communicates with the mobile terminal 16 inside the vehicle 12 using various short-range wireless communication technologies such as Wi-Fi (registered trademark) and Bluetooth (registered trademark).

[0031] The display unit 28 provides various information to the occupants by displaying various information such as warnings. In this embodiment, based on the information output from the automatic driving control unit 24, it displays warnings such as the end of automatic driving and warns the driver to switch to manual driving. The display unit 28 may display information on a display provided on the instrument panel, for example, or it may display information on a head-up display that projects images onto the glass.

[0032] As shown in Figure 3, the control unit 30 is composed of a general-purpose microcomputer including a CPU (Central Processing Unit) 30A, ROM (Read Only Memory) 30B, RAM (Random Access Memory) 30C, storage 30D, interface (I / F) 30E, and bus 30F. The control unit 30 performs notification control during autonomous driving by having the CPU 30A execute a program stored in the ROM 30B. Specifically, when a mobile terminal 16 is connected during autonomous driving, the control unit 30 notifies both the vehicle 12 and the mobile terminal 16. When predetermined conditions are met, it controls the system to not notify the mobile terminal 16 and only notify the vehicle 12.

[0033] On the other hand, as shown in Figure 2, the mobile terminal 16 includes a control unit 40, a communication unit 42, and a display unit 44.

[0034] As shown in Figure 3, the control unit 40 is composed of a general-purpose microcomputer including a CPU 40A, ROM 40B, RAM 40C, storage 40D, interface (I / F) 40E, and bus 40F.

[0035] The control unit 40 performs processing in conjunction with the in-vehicle device 14 by having the CPU 40A load the information processing program stored in the ROM 40B into the RAM 40C and execute it.

[0036] The communication unit 42 communicates with the in-vehicle unit 14 using various short-range wireless communication technologies such as Wi-Fi (registered trademark) and Bluetooth (registered trademark).

[0037] The display unit 44 displays various information, such as information for operating the mobile terminal 16 and screens provided by the application. In this embodiment, it is possible to display warnings such as the end of automatic driving in conjunction with the in-vehicle unit 14 to warn the driver to switch to manual driving.

[0038] Although not shown in Figure 2, the mobile terminal 16 also includes a position detection unit that detects the location of the mobile terminal 16 using GPS (Global Positioning System), a camera or other imaging unit, and other components.

[0039] Incidentally, in a vehicle 12 capable of autonomous driving, as in this embodiment, it is necessary to transmit information to the occupants depending on the situation. For example, advance notification of driver changes, notification of system status and transitions, function suggestions from the system, and notification of the status of the vehicle 12 are necessary. Specifically, if the level of autonomous driving needs to be reduced or a transition from autonomous driving to manual driving becomes necessary due to some factor, it is necessary to notify the driver of information to prompt them to take action such as monitoring the surroundings or operating the vehicle. In addition, there are cases where the system needs to confirm the driver's intentions when executing a function.

[0040] In order to transmit information to the occupants as needed, the vehicle information processing system 10 according to this embodiment is capable of notifying information via an in-vehicle device 14 and a portable terminal 16 carried by the driver.

[0041] However, if the driver is performing a secondary task other than driving during autonomous driving, the mobile device 16 may receive a warning notification, causing the driver to search for the mobile device 16, which could delay responses such as a change of driver. In addition, constantly receiving warning notifications on the mobile device 16 in the vehicle may be annoying.

[0042] Therefore, in this embodiment, the control unit 30 notifies both the vehicle 12 and the mobile terminal 16 when the mobile terminal 16 is connected during autonomous driving, and when predetermined conditions are met, it controls the system to not notify the mobile terminal 16 and notify only the vehicle 12.

[0043] For example, if the driver is performing a secondary task other than driving and does not have the mobile device 16 in their possession, sending a notification to the mobile device 16 may cause the driver to search for the mobile device 16, potentially delaying actions such as a change of driver. Therefore, the notification of the warning to the driver's mobile device 16 is disabled.

[0044] Similarly, if the driver's mobile device 16 is not placed in a predetermined location, the notification to the mobile device 16 will be muted because sending a notification to the mobile device 16 may cause the driver to pick up the device, potentially delaying responses such as driver changes. For example, if a holder capable of detecting the placement of the mobile device 16 is provided and the mobile device 16 is not placed in the holder, the notification to the driver's mobile device 16 will be muted.

[0045] Furthermore, if there are multiple mobile devices 16 in the vehicle, notifying mobile devices 16 other than the driver's would be bothersome to the other passengers, so the warning notification is muted for mobile devices 16 other than the driver's.

[0046] Furthermore, if there are multiple mobile devices 16 in the vehicle, it can be bothersome for passengers other than the driver to receive notifications of unnecessary information. Therefore, for mobile devices 16 other than the driver, notifications of warnings below a predetermined level of importance are muted.

[0047] Furthermore, when activating autonomous driving level 3 (when the autonomous driving level increases), the vehicle 12 may display on a display unit 28 or similar that which of the detected mobile terminals 16 it will connect to, connect to the selected mobile terminal 16, and notify the selected mobile terminal 16.

[0048] Next, we will describe the specific processing performed by the control unit 30 of the in-vehicle unit 14 in the vehicle information processing system 10 according to this embodiment, which is configured as described above. Figure 4 is a flowchart showing an example of the processing flow performed by the control unit 30 of the in-vehicle unit 14 in the vehicle information processing system 10 according to this embodiment. Note that the processing in Figure 4 starts, for example, when communication between the in-vehicle unit 14 and the mobile terminal 16 is established.

[0049] In step 100, the CPU 30A determines whether or not it has received a driving change notification. This determination, for example, determines whether or not it has received a notification from the automatic driving control unit 24 to switch from automatic driving to manual driving, or from manual driving to automatic driving. If the determination is affirmative, the process proceeds to step 102; if negative, the process ends and other processes are performed. Here, as an example, we describe the determination of whether or not a driving change notification has been received, but it may also be used to determine whether or not other notifications that require control over the notification destination have been received.

[0050] In step 102, the CPU 30A determines whether a predetermined condition has been met. This determination may involve, for example, whether the driver is not holding the mobile terminal 16 while performing a secondary task other than driving during autonomous driving, or whether the driver's mobile terminal 16 is not placed in a predetermined location, or whether multiple mobile terminals 16 are present in the vehicle. If the determination is affirmative, the system proceeds to step 104; otherwise, it proceeds to step 106.

[0051] In step 104, the CPU 30A disables notifications to the mobile terminal 16 and terminates the series of processes by issuing a warning notification via the in-vehicle unit 14. For example, if the driver is performing a secondary task other than driving and does not have the mobile terminal 16 in their possession, notifying the mobile terminal 16 may cause the driver to search for the mobile terminal, potentially delaying responses such as a change of driver, but this prevents such delays. Similarly, if the driver's mobile terminal 16 is not placed in a predetermined location, notifying the mobile terminal 16 may cause the driver to pick up the mobile terminal 16, potentially delaying responses such as a change of driver, but this prevents such delays. Furthermore, if there are multiple mobile terminals 16 in the vehicle, notifying all of them may be bothersome for passengers other than the driver, but it suppresses unnecessary notifications. In this case, notifications to the mobile terminal 16 may be disabled only for warnings of a predetermined importance level or lower, while warnings of a higher importance level may be issued via both the mobile terminal and the in-vehicle unit 14.

[0052] Meanwhile, in step 106, the CPU 30A terminates the series of processes by issuing a warning notification via the mobile terminal 16 and the in-vehicle unit 14.

[0053] By performing this process, the recipient of warning notifications is changed depending on the situation, making it possible to appropriately notify warnings during autonomous driving.

[0054] For example, if the driver is performing a secondary task other than driving and does not have the mobile device 16 in their possession, the warning notification to the mobile device 16 will be muted. This prevents the driver from searching for the mobile device 16 and eliminates the cause of delays in taking over driving duties.

[0055] Furthermore, if the driver's mobile terminal 16 is not placed in a predetermined location, the warning notification to the mobile terminal 16 will be muted, thus preventing the driver from searching for the mobile terminal 16 and reducing delays in responding to driver changes and other related matters.

[0056] Furthermore, if there are multiple mobile devices in the vehicle, the warning notification to the mobile device 16 will be muted, thus reducing annoyance.

[0057] In the above embodiment, an example was shown where multiple mobile terminals 16 are present in the vehicle and the mobile terminals 16 are made silent, but the system is not limited to this. For example, the control unit 30 may function as a identification unit to identify the driver's mobile terminal 16, make all other mobile terminals 16 silent, and notify the driver's mobile terminal 16 of a warning. This reduces unnecessary notifications to passengers other than the driver and alleviates inconvenience. In this case, the driver's mobile terminal 16 can be identified by pre-registering the identification information of the mobile terminal 16. For example, the identification information of the driver's mobile terminal 16 may be pre-registered, and the identification may be confirmed when short-range wireless communication is established. Alternatively, for example, if there is a mobile terminal 16 registered as a digital key, the identification unit may identify that mobile terminal 16 as the driver's mobile terminal 16. Furthermore, as shown in Figure 5, the in-vehicle unit 14 may acquire identification information of the driver's mobile terminal 16 via the server 50. In this case, the driver's mobile terminal 16 may be identified by a signal from the mobile terminal 16 that detected the vehicle's smart key 52. ​​Specifically, the smart key 52 emits an authentication signal to its surroundings. A mobile terminal 16 located near the smart key 52 receives the authentication signal emitted from the smart key 52. ​​The mobile terminal 16 notifies the server 50 of a signal indicating that it has detected the smart key 52. ​​The server 50 transmits the identification information of the mobile terminal 16 that detected the smart key 52 to the communication unit 26. The communication unit 26 acquires the identification information of the driver's mobile terminal 16 by receiving the identification information of the mobile terminal 16 transmitted from the server 50. Furthermore, the timing for the identification unit to identify the driver's mobile device 16 can be set as appropriate. For example, the identification unit may identify the driver's mobile device 16 when the vehicle's power is turned on from an off state. Alternatively, the identification unit may identify the driver's mobile device 16 each time the vehicle is stopped, even if the vehicle's power is on. A stopped state is, for example, when the vehicle's shift position is in parking. Alternatively, the identification unit may identify the driver's mobile device 16 at regular intervals. Alternatively, the identification unit may identify the driver's mobile device 16 when the vehicle's autonomous driving reaches Level 3.

[0058] Furthermore, while the above embodiment describes an example of controlling the recipient of a warning notification, it is not limited to warning notifications. For example, when providing various types of information to the crew, the recipient of the notification may be controlled according to predetermined conditions, similar to the above embodiment.

[0059] Furthermore, although the processing performed by the vehicle information processing system 10 in each of the above embodiments has been described as software processing performed by executing a program, it is not limited to this. For example, the processing may be performed by hardware such as a GPU (Graphics Processing Unit), ASIC (Application Specific Integrated Circuit), and FPGA (Field-Programmable Gate Array). Alternatively, the processing may be a combination of both software and hardware. In the case of software processing, the program may be stored on various storage media and distributed.

[0060] Furthermore, the present invention is not limited to the above, and it is of course possible to implement it in various modified forms without departing from its spirit. [Explanation of symbols]

[0061] 10. Vehicle Information Processing System 12 vehicles 14. In-vehicle devices (control devices and in-vehicle terminals) 16 Mobile devices 24. Automated Driving Control Unit 28 Display section (notification section) 30 Control Unit (Control Unit and Specific Unit)

Claims

1. A notification unit that notifies an in-vehicle terminal installed in the vehicle and a handheld terminal carried by the occupant of warnings during autonomous driving, A control unit controls the notification unit to mute the mobile terminal and notify the in-vehicle terminal of a warning when predetermined conditions are met, Equipped with, The control unit is a control device that disables the notification of a warning to the driver's mobile device if the driver is not holding the mobile device while performing a secondary task other than driving.

2. A notification unit that notifies an in-vehicle terminal installed in the vehicle and a handheld terminal carried by the occupant of warnings during autonomous driving, A control unit controls the notification unit to mute the mobile terminal and notify the in-vehicle terminal of a warning when predetermined conditions are met, Equipped with, The control unit will disable the notification of a warning to the driver's mobile device if the driver's mobile device is not placed in a predetermined location.

3. A notification unit that notifies an in-vehicle terminal installed in the vehicle and a handheld terminal carried by the occupant of warnings during autonomous driving, A control unit controls the notification unit to mute the mobile terminal and notify the in-vehicle terminal of a warning when predetermined conditions are met, A special unit that identifies the driver's mobile device, Equipped with, The control unit is a control device that, when multiple mobile terminals are present in the vehicle, disables the notification of warnings to mobile terminals other than those of the driver identified by the identification unit.

4. The control device according to claim 3, wherein the control unit disables notifications of warnings below a predetermined severity level for the mobile terminal other than the driver.

Images