Vehicle control device and vehicle control method

Abstract

To obtain a vehicle controlling device and a vehicle controlling method that can prompt a driver to confirm that a portable terminal which receives notification of vehicle information is located within a vehicle cabin in a vehicle capable of executing autonomous travel control where a driver is permitted to take an action that does not allow peripheral confirmation of the vehicle.SOLUTION: A control device 10 comprises: a determination unit for determining whether a portable terminal 30 which receives notification of vehicle information is present in a vehicle cabin before an autonomous travel control is executed in a vehicle 20 capable of executing the autonomous travel control where a driver P is permitted to perform an action that does not allow peripheral confirmation of the vehicle; and a notification unit that notifies the driver P of information for carrying the portable terminal 30 via an on-vehicle device when the determination unit determines that the portable terminal 30 is not present in the vehicle interior.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a vehicle control device and a vehicle control method.

Background Art

[0002] The following Patent Document 1 describes a technique for issuing an alarm from a speaker mounted on a mobile terminal and a vehicle when a driver operates the mobile terminal during running by autonomous driving.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, in the autonomous driving (automated driving) of a vehicle performed under predetermined conditions, it is assumed that the driver is temporarily released from the obligation to pay attention to the front, and actions that cannot confirm the surroundings of the vehicle, such as operating a mobile terminal, are permitted. On the other hand, in such autonomous driving, the driver must be able to return to driving at any time in response to a request from the autonomous driving system. In this case, it is conceivable to notify the driver of information regarding autonomous driving and requests to the driver using a mobile terminal.

[0005] In order to appropriately notify the driver via the mobile terminal, it is desirable to prompt the driver to confirm that the mobile terminal is in the vehicle interior before executing autonomous driving control at a predetermined level or higher that permits actions that cannot confirm the surroundings of the vehicle by the driver.

[0006] In consideration of the above facts, the present invention aims to provide a vehicle control device and a vehicle control method that can prompt the driver to confirm that a mobile terminal for receiving notifications of vehicle information is inside the vehicle, in a vehicle capable of autonomous driving control in which actions that prevent the driver from checking the surroundings of the vehicle are permitted. [Means for solving the problem]

[0007] The vehicle control device according to claim 1 is for a vehicle capable of autonomous driving control in which the driver is permitted to perform actions that prevent the driver from checking the surroundings of the vehicle, and includes a determination unit that determines whether or not a mobile terminal for receiving notifications of vehicle information is inside the vehicle before the autonomous driving control is performed, and a notification unit that, if the determination unit determines that the mobile terminal is not inside the vehicle, the mobile terminal cannot send notifications of vehicle information via an in-vehicle device, The determination unit determines that the mobile terminal is inside the vehicle when the detected value of an on-board sensor mounted on the vehicle matches the detected value of a terminal sensor mounted on the mobile terminal for a predetermined physical quantity. .

[0008] The vehicle control device according to claim 1 determines whether a mobile terminal for receiving vehicle information notifications is inside the vehicle before autonomous driving control is performed in which the driver is permitted to perform actions that prevent the driver from checking the surroundings of the vehicle. If it is determined that the mobile terminal is not inside the vehicle, the mobile terminal notifies the driver via the in-vehicle device that vehicle information notifications cannot be sent. This prompts the driver to confirm that the mobile terminal is inside the vehicle. Furthermore, predetermined physical quantities are detected using on-board sensors mounted in the vehicle and terminal sensors mounted on the mobile device. If the detected values ​​from these sensors match, it is determined that the mobile device is inside the vehicle. In other words, if the detected values ​​from the sensors match, it indicates that the mobile device is in the same environment as the vehicle. Therefore, by considering whether the environment surrounding the mobile device matches the environment inside the vehicle, it is possible to determine whether or not the mobile device is inside the vehicle with high accuracy. It should be noted that the phrase "when the sensor's detected values ​​match" used here is a broad concept that includes not only cases where the detected values ​​perfectly match, but also cases where they match within a predetermined error range.

[0009] The vehicle control device according to claim 2, in the configuration described in claim 1, determines that the mobile terminal is inside the vehicle when the vehicle and the mobile terminal are connected in a manner that enables mutual communication according to a predetermined communication standard.

[0010] In the vehicle control device according to claim 2 of the present invention, it is determined that the mobile terminal is inside the vehicle when the vehicle and the mobile terminal are connected in a manner that enables mutual communication. As a result, it is determined that the mobile terminal is inside the vehicle when it is in a state where it can receive notifications of vehicle information, so that vehicle information can be reliably received on the mobile terminal, thereby improving safety during autonomous driving.

[0014] Claim 3 The vehicle control device according to the present invention described in claim 1 or claim 2 In the configuration described above, the system further includes an execution unit that performs the autonomous driving control, and the execution unit enables the execution of the autonomous driving control when the determination unit determines that the mobile terminal is inside the vehicle.

[0015] Claim 3 The vehicle control device according to the present invention, as described above, enables the execution of autonomous driving control that allows the driver to perform actions that prevent the driver from checking the surroundings of the vehicle when it is determined that a mobile terminal is inside the vehicle. As a result, since the autonomous driving control is executed only after it is ensured that it is possible to notify the vehicle information via the mobile terminal, safety during autonomous driving can be enhanced.

[0016] Claim 4 The vehicle control device according to the present invention described herein is Claim 3 In the configuration described above, when the determination unit determines that the mobile terminal is inside the vehicle, the execution unit activates the camera mounted on the mobile terminal, refers to the image data captured by the camera and the facial recognition data previously registered for the driver, attempts to detect the driver's face from the image data, and when the driver's face is detected from the image data, determines that the execution of the autonomous driving control has been approved, and enables the execution of the autonomous driving control.

[0017] Claim 4In the vehicle control device according to the present invention described herein, when it is determined that a mobile device is inside the vehicle, the camera mounted on the mobile device is activated. The vehicle control device also refers to the image data captured by the camera and pre-registered facial recognition data for the driver, and attempts to detect the driver's face from the image data. When the driver's face is detected from the image data, the vehicle control device determines that the driver has consented to the execution of autonomous driving control, and enables the execution of autonomous driving control that allows the driver to perform actions that prevent them from checking the surroundings of the vehicle. As a result, the execution of autonomous driving control can be enabled based on the fact that the mobile device is inside the vehicle and the fact that the driver has consented to the execution of autonomous driving control through facial recognition. As a result, vehicle information can be appropriately notified to the mobile device held by the driver, and safety during autonomous driving can be enhanced.

[0018] Claim 5 The vehicle control method according to the present invention, as described above, is for a vehicle capable of autonomous driving control in which the driver is permitted to perform actions that prevent the driver from checking the surroundings of the vehicle, and it determines whether or not a mobile terminal for receiving notifications of vehicle information is inside the vehicle before the autonomous driving control is performed. A vehicle control method in which a computer performs the following process, wherein the computer determines that the mobile terminal is inside the vehicle when the detected value of an on-board sensor mounted on the vehicle matches the detected value of a terminal sensor mounted on the mobile terminal with respect to a predetermined physical quantity. If it is determined that the mobile device is not inside the vehicle, the in-vehicle device will notify the user that the vehicle information cannot be sent from the mobile device.

[0019] Claim 5 As described above, the vehicle control method according to the present invention allows the driver to be prompted to confirm that the mobile device is inside the vehicle. [Effects of the Invention]

[0020] As described above, the vehicle control device and vehicle control method according to the present invention can prompt the driver to confirm that a portable terminal for receiving vehicle information notifications is inside the vehicle, in a vehicle capable of autonomous driving control in which the driver is permitted to perform actions that prevent the driver from checking the surroundings of the vehicle. [Brief explanation of the drawing]

[0021] [Figure 1]It is a diagram showing a schematic configuration of a system constructed by a vehicle control device according to the present embodiment. [Figure 2] It is a block diagram showing a hardware configuration of a vehicle control device. [Figure 3] It is a block diagram showing an example of a functional configuration of a vehicle control device. [Figure 4] It is a diagram showing an example of a notification performed via an in-vehicle device. [Figure 5] It is a diagram showing an example of face authentication performed via a mobile terminal. [Figure 6] It is a diagram showing an example of notification information performed via a mobile terminal. [Figure 7] It is a diagram showing an example of a notification performed via an in-vehicle device. [Figure 8] It is a diagram showing an example of a notification performed via a mobile terminal. [Figure 9] It is a flowchart showing the flow of notification processing.

Mode for Carrying Out the Invention

[0022] Hereinafter, the system S according to the present embodiment will be described based on FIGS. 1 to 6. The system S according to the present embodiment is a system that prompts the driver P to confirm that the mobile terminal 30 for receiving notification of vehicle information is in the vehicle interior before a predetermined autonomous driving control is executed in the vehicle 20.

[0023] As shown in FIG. 1, in the system S, the vehicle 20 and the driver P's mobile terminal 30 are connected to be able to communicate with each other according to a predetermined communication standard. Also, in the system S, the smart key 50 of the vehicle 20 is connected to be able to communicate with the vehicle 20 and the mobile terminal 30 according to a predetermined wireless communication standard. The vehicle 20 can execute various operations such as locking and unlocking of the door and starting of the engine remotely based on a control command transmitted from the smart key 50 by an operation of the driver P or the like. Further, in the system S, the vehicle 20, the mobile terminal 30, and the server 40 are connected to be able to communicate with each other via the network N.

[0024] In the above system S, vehicle information transmitted from the vehicle 20 is notified to the driver P via the in-vehicle unit 25 and the mobile terminal 30. In this embodiment, the transmission of notification information to the in-vehicle unit 25 and the mobile terminal 30 installed in the vehicle 20 is controlled by the control device 10. The control device 10 is an example of a "vehicle control device".

[0025] (Hardware configuration of control device 10) The hardware configuration of the control device 10 will now be described. Figure 2 is a block diagram showing the hardware configuration of system S. The control device 10 consists of one or more ECUs (Electronic Control Units) mounted on, for example, a vehicle 20. As shown in Figure 2, the control device 10 includes a CPU (Central Processing Unit) 11, ROM (Read Only Memory) 12, RAM (Random Access Memory) 13, a storage unit 14, and an input / output interface (I / F) 15. Each component is connected to each other so as to be able to communicate with each other via a bus 18.

[0026] The CPU 11 is a central processing unit that executes various programs and controls various components. Specifically, the CPU 11 reads a program from the ROM 12 or memory unit 14 and executes the program using the RAM 13 as a working area. The CPU 11 controls each of the above components and performs various calculations according to the program recorded in the ROM 12 or memory unit 14. ROM12 stores various programs and data. RAM13 temporarily stores programs or data as a working area. The storage unit 14 is composed of a storage device such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory, and stores various programs and data. The storage unit 14 stores a program 14A that causes the CPU 11 to execute the notification process described later. The storage unit 14 also stores a notification information database 14B that stores vehicle information to be notified to the in-vehicle device 25 of the vehicle 20 and the driver P's mobile terminal 30.

[0027] The input / output interface 15 is an interface for connecting the control device 10 to other devices. The input / output interface 15 is connected to the automatic driving ECU 21, navigation system 22, communication unit 23, on-board sensor 24, and on-board device 25, which are mounted on the vehicle 20.

[0028] The autonomous driving ECU 21 is an electronic control unit that performs various controls related to the autonomous driving of the vehicle 20, and constitutes the autonomous driving system of the vehicle 20. The vehicle ECU 20 is connected to the vehicle 20's powertrain system 21A, brake system 21B, and steering system 21C, and controls the autonomous driving of the vehicle 20 through these systems. Specifically, the autonomous driving ECU 20 is connected to sensors or a navigation system, which will be described later, and controls the autonomous driving of the vehicle 20 based on information acquired from the sensors or navigation system 22, which will be described later. Autonomous driving is a driving state in which some or all of the driving operations are performed automatically, and the vehicle 20 automatically drives along the road on which it is traveling.

[0029] The autonomous driving ECU21 performs autonomous driving control that conforms to a predetermined level of autonomous driving as defined by organizations such as the SAE (Society of Automotive Engineers), NHTSA (National Highway Traffic Safety Administration), and JSAE (Japan Society of Automotive Engineers). This predetermined level of autonomous driving includes a state in which the autonomous driving system, including the autonomous driving ECU21, completely replaces all driving operations within a limited area that meets specific driving environment conditions. At this level of autonomous driving, and at higher levels, the driver is permitted to take actions that prevent them from checking the vehicle's surroundings during autonomous driving. Here, actions that prevent checking the vehicle's surroundings refer to any work or tasks other than those that allow checking the vehicle's surroundings, such as operating the vehicle itself or checking and monitoring the vehicle's surroundings for the purpose of driving or operating it. For example, eating or drinking inside the vehicle, or operating a mobile device, are considered actions that prevent checking the vehicle's surroundings.

[0030] The autonomous driving ECU 21 enables autonomous driving, where the vehicle 20 is driven automatically toward a preset destination without the driver P having to operate the vehicle, based on the vehicle's location information, map information, and guidance route information from the navigation system 22, which will be described later. The autonomous driving ECU 21 also transmits information to the navigation system 22, for example, regarding the execution or modification of autonomous driving control.

[0031] Alternatively, the automatic driving ECU 21 may be omitted, and the control device 10 may constitute the automatic driving ECU.

[0032] The navigation system 22 provides driving guidance for the vehicle 20 to a destination set by the driver P. The navigation system 22 includes, for example, a GPS receiver 22A for measuring the location information of the vehicle 20 and a map database 22B that stores map information. The navigation system 22 recognizes the road and lane the vehicle 20 is traveling on based on the location information of the vehicle 20 measured by the GPS receiver 22A and the map information of the map database 22B. The navigation system 22 calculates the route from the location of the vehicle 20 to the destination and provides guidance to the driver P along the route by displaying it on the display included in the in-vehicle unit 25 (described later) and by outputting voice from the speaker. For example, the navigation system 22 transmits the location information of the vehicle 20, the information of the road the vehicle 20 is traveling on, and the information of the guided route for the vehicle 20 to the control device 10. Furthermore, the navigation system 22 transmits vehicle information, including information regarding the execution or modification of autonomous driving control, information regarding warnings to the driver during autonomous driving, and requests to the driver regarding autonomous driving, to the control unit 23 (described later) and the mobile terminal 30 via the communication unit 23.

[0033] The communication unit 23 is an interface for communicating with other devices. For this communication, for example, wired communication standards such as Ethernet® or FDDI, or wireless communication standards such as 4G, 5G, Bluetooth®, or Wi-Fi® are used. These standards are examples of "predetermined communication standards." In this embodiment, the vehicle 20 and the mobile terminal 30 of a user pre-registered as the driver of the vehicle 20 are connected via the communication unit 23 to enable mutual communication.

[0034] The on-board sensor 24 includes a peripheral sensor 24A for acquiring information about the area around the vehicle 20 (front, rear, right, and left) and an interior sensor 24B for acquiring information about the interior of the vehicle. The peripheral sensor 24A is configured to include at least one of the following, for example, a camera that takes pictures of the area around the vehicle, a millimeter-wave radar, an ultrasonic sensor, or LiDAR (Light Detection and Ranging), and detects the environment around the vehicle. The interior sensor 24B is configured to include at least one of the following, for example, a camera that takes pictures of the interior of the vehicle, an acceleration sensor, a yaw rate sensor, a temperature sensor, or a microphone.

[0035] The above-mentioned on-board sensor 24 is capable of detecting predetermined physical quantities related to the state of the vehicle 20. In this embodiment, it is possible to determine whether or not the driver P's terminal 30 is inside the vehicle 20 based on the physical quantities detected by the on-board sensor 24. The predetermined physical quantities related to the state of the vehicle are not particularly limited. For example, they may be physical quantities related to the driving state, such as the vehicle's acceleration or tilt angle. Alternatively, they may be detectable physical quantities related to the temperature inside the vehicle, the sound inside the vehicle, or the environment around the vehicle.

[0036] The in-vehicle unit 25 consists of devices capable of outputting information transmitted from the control device 10 to the driver P inside the vehicle. For example, it can consist of a display, speaker, indicator light, steering wheel, seat, seat belt, etc.

[0037] (Hardware configuration of mobile terminal 30) Next, the hardware configuration of the mobile terminal 30 will be described. The mobile terminal 30 consists of, for example, a smartphone or a tablet device, and can be brought into and taken out of the vehicle. The mobile terminal 30 includes a CPU 31, ROM 32, RAM 33, storage unit 34, input unit 35, output unit 36, communication unit 37, camera 38, and terminal sensor 40. Each component is connected to the others via a bus 39 so that they can communicate with each other.

[0038] The CPU 31 is a central processing unit that reads a program from the ROM 32 or memory unit 34 and executes the program using the RAM 33 as a working area. The CPU 31 controls each of the above components and performs various calculations according to the program recorded in the ROM 32 or memory unit 34. ROM32 stores various programs and data. RAM33 temporarily stores programs or data as a working area. The storage unit 34 is composed of a storage device such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory, and stores various programs and data. As will be described later, the storage unit 34 stores a program 34A that causes the CPU 31 to output various information transmitted from the control device 10 and the vehicle 20 to the mobile terminal 30. The input unit 35 includes a pointing device such as a mouse, a keyboard, a microphone, and a camera, and is used for various types of input. The output unit 36 ​​is, for example, a liquid crystal display or a speaker, and outputs various information. If the output unit 36 ​​is composed of a display, a touch panel system may be adopted and it may also function as the input unit 35. The communication unit 37 is an interface for communicating with other devices. For such communication, a wired communication standard such as Ethernet® or FDDI, or a wireless communication standard such as 4G, 5G, Bluetooth®, or Wi-Fi® may be used. The terminal sensor 40 consists of sensors for detecting predetermined physical quantities related to the state of the mobile terminal. The terminal sensor 40 includes at least one of the following: an acceleration sensor, a yaw rate sensor, a camera, a millimeter-wave radar, an ultrasonic sensor, LiDAR (Light Detection and Ranging), a temperature sensor, a microphone, etc. The predetermined physical quantities related to the state of the mobile terminal 30 are not particularly limited. For example, various physical quantities such as the acceleration and tilt angle of the mobile terminal 30, the temperature around the terminal, ambient sound, and the physical quantities of objects present in the surroundings can be considered "predetermined physical quantities related to the state of the mobile terminal 30".

[0039] (Functional configuration of the control device 10) Next, the functional configuration of the control device 10 will be described. Figure 3 is a block diagram showing an example of the functional configuration of the control device 10.

[0040] The determination unit 11A determines whether a mobile terminal 30 that receives notifications of vehicle information is inside the vehicle before autonomous driving control is executed in which the driver P is permitted to perform actions that prevent the vehicle 20 from checking its surroundings. "Autonomous driving control in which the driver P is permitted to perform actions that prevent the vehicle 20 from checking its surroundings" refers, for example, to Level 3 or higher autonomous driving as defined by the SAE (Society of Automotive Engineers).

[0041] The method for determining whether or not the mobile terminal 30 is inside the vehicle is not particularly limited, but for example, the determination unit 11A may determine that the driver P's mobile terminal 30 is inside the vehicle if the vehicle 20 and the mobile terminal 30 are connected in a manner that enables mutual communication according to a predetermined communication standard.

[0042] Furthermore, the determination unit 11A may determine that the mobile terminal 30 is inside the vehicle when the detected value of the on-board sensor 24 mounted on the vehicle 20 matches the detected value of the terminal sensor 40 mounted on the mobile terminal 30 for a predetermined physical quantity. Here, "match" is a broad concept that is not limited to cases where the detected values ​​perfectly match, but also includes cases where they match within a predetermined error range.

[0043] If the determination unit 11A determines that the mobile terminal 30 is not inside the vehicle, the notification unit 11B notifies the driver via the in-vehicle unit 25 that the mobile terminal 30 cannot send vehicle information regarding the autonomous driving of the vehicle 20. In one example shown in Figure 4, the message "Notifications cannot be sent from the mobile terminal." is displayed on the display 25A installed on the front side of the vehicle in the driver's seat (not shown), prompting the driver P to carry the mobile terminal 30 in an appropriate state. However, the notification unit 11B may also notify the driver by outputting audio information to the speaker of the in-vehicle unit 25, or by vibrating the steering wheel or seat to prompt the driver to carry the mobile terminal in an appropriate state.

[0044] Furthermore, when autonomous driving control is performed in the vehicle 20 in a manner that prevents the driver P from checking the surroundings of the vehicle, the notification unit 11B causes vehicle information related to autonomous driving to be notified from at least one of the onboard unit 25 installed in the vehicle 20 and the driver P's mobile terminal 30. The vehicle information (notification information) notified from the onboard unit 25 and the mobile terminal 30 is stored in the notification information database 14B. As shown in Figure 6●●, the notification information database 14B stores various notification information associated with items such as "notification classification," "event item," and "user action." "Notification classification" includes, as an example, "driver change warning," "driver change notification," "system state, transition (MRM)," "system state, transition (MRC)," "function suggestion from the system," and "vehicle status notification." Specific events corresponding to each item included in these "notification classifications" are set in "event items."

[0045] The events corresponding to the "Driver Change Warning" are set to situations where a sudden change of driving occurs, requiring a change of driving from vehicle 20 to driver P. Specifically, the events that correspond to this are set to include vehicle 20 leaving the autonomous driving area, encountering a dangerous situation, system failure or malfunction, sudden deterioration of weather, a shift in the autonomous driving level to a lower level than the current autonomous driving level, and an accident. These events are set to situations where it becomes suddenly difficult to continue autonomous driving at the current autonomous driving level, and driver P is required to take immediate action. Therefore, in the "User Action" section, these are associated with items that require "immediate action."

[0046] The events corresponding to the "Driver Change Notification" are defined as situations where a driver change occurs from vehicle 20 to driver P under non-sudden circumstances. Specifically, these include vehicle 20 leaving the automated driving area, encountering a dangerous situation, system failure or malfunction, sudden deterioration of weather, a transition in the automated driving level to a lower level than the current level, and an accident. These events are set as situations where manual driving is expected to be necessary when driving through a designated area, and driver P's actions are required before reaching that area. Therefore, the "User Action" item is associated with the item "Action required before reaching the manual driving area."

[0047] As an event corresponding to "System State Transition (MRM)," the vehicle 20's driving control state is set to transition to MRM (Minimal Risk Maneuver). MRM refers to the vehicle motion control that leads to MRC (Minimal Risk Control), which will be described later, as a response to an event that makes it impossible to drive safely.

[0048] As an event corresponding to "System State Transition (MRM)," the driving control state of vehicle 20 is set to transition to MRC (Minimal Risk Condition). MRC is the safe state that the vehicle ultimately aims for as a countermeasure when an event occurs that prevents safe driving due to a malfunction in the autonomous driving system, etc. Generally, it refers to a stopped state in a situation where the risk of an accident is sufficiently low.

[0049] When the events corresponding to the above "System State, Transition (MRM)" occur, it is unlikely that the user (driver P) will take any specific action, so in the "User Action" section, it is associated with the "None" item. However, since "System State, Transition (MRM)" is a stage before reaching MRC, user action may be required as needed.

[0050] The events corresponding to "function suggestions from the system" include overtaking, passing, lane changes, transitioning the autonomous driving level to a higher level than the current autonomous driving level, route selection via the navigation system 22, and deceleration at toll booths on toll roads. These events can be executed using autonomous driving control and do not require a switch to manual driving; therefore, approval from driver P is sufficient. Accordingly, the "User Actions" section is associated with the "Approval" section.

[0051] The events corresponding to "Notification of Driving Status" are set to include battery level and fuel level. These events do not involve the execution of autonomous driving control, and only notification to the driver P is sufficient. Therefore, in the "User Actions" section, they are associated with the "Notification Only" item.

[0052] The notification unit 11B refers to vehicle information transmitted from the autonomous driving ECU 21 and the navigation system 22, and the notification information database 14B, and transmits notification information to the in-vehicle unit 25 and the mobile terminal 30. The transmitted notification information is then communicated to the driver P via the in-vehicle unit 25 and the mobile terminal 30.

[0053] Specifically, with reference to Figures 7 and 8, an example of information being notified from vehicle 20 requesting a change of driver from driver P will be explained. Here, we will explain the case where, in conjunction with the navigation system 22, vehicle 20 is scheduled to enter a manual driving area from an automated driving area, and a change of driver is requested from driver P. Since this notification does not request a sudden change of driver, it is classified as a "change of driver notification" in the notification information database 14B and corresponds to the event "leaving the automated driving area". Therefore, a user action is required in which driver P takes over driving before vehicle 20 reaches the manual driving area.

[0054] As shown in Figure 7, inside the vehicle, the display 25A, which is an in-vehicle device 25, displays text information Q1 requesting a driver change and an icon image Q2 indicating the driver change request as a notification from the vehicle. The message requesting a driver change also includes information about the time until the manual driving area is reached. Meanwhile, the driver P's mobile terminal 30 displays the icon image indicating the driver change request and a confirmation operation object Q3 requesting confirmation of the driver change notification as a notification from the vehicle. When driver P performs an input operation on the confirmation operation object Q3, confirmation information is sent from the mobile terminal 30. When vehicle 20 receives the confirmation information, it determines that the driver change request from vehicle 20 has been confirmed by driver P.

[0055] As described above, it is preferable that the information notified via the mobile terminal 30 contains less information than the information notified via the in-vehicle unit 25. In the example above, the information output from the display 25A includes text information Q1 indicating a driver change, whereas the information displayed on the mobile terminal 30 omits the text information Q1 indicating a driver change and displays only the icon image Q2. Therefore, the information can be presented in a way that is easy to understand visually, even on a small mobile terminal. Furthermore, as in the notification described above, when the vehicle 20 requests driver P to take involvement in driving operations, it is preferable to request driver P's confirmation operation in response to the notification via the mobile terminal 30. In the example above, the confirmation operation object Q3 is displayed on the mobile terminal 30, and the vehicle 20 receives the input operation for the confirmation operation object Q3. This improves the accuracy of notifications to driver P and enhances driving safety.

[0056] The execution unit 11C enables the execution of autonomous driving control that allows the driver to perform actions that prevent them from checking the surroundings of the vehicle, when the determination unit 11A determines that the mobile terminal 30 is inside the vehicle. Here, if the determination unit 11A determines that the driver P's mobile terminal 30 is inside the vehicle, the execution unit 11C activates the camera 38 mounted on the mobile terminal 30 and performs facial recognition of the driver P. For example, as shown in Figure 5, the camera 38 of the mobile terminal 30 is automatically activated, and the output unit 36 ​​(display in Figure 5) displays the text message "Please bring the driver's face closer," causing the driver operating the mobile terminal 30 to take a picture of their own face.

[0057] The execution unit 11C refers to the image data captured by the camera 38 of the mobile terminal 30 and the facial recognition data that has been registered in advance for the driver P, and attempts to detect the face of the driver P from the captured image data. The facial recognition data may be stored in the storage unit 14 of the control device 10, or it may be obtained from the storage of the vehicle 20 or an external server.

[0058] When the execution unit 11C detects the driver P's face from the image data captured by the camera 38, it determines that the driver P has consented to the execution of autonomous driving control and transmits permission information for the execution of autonomous driving control to the autonomous driving ECU 21. Upon receiving the permission information for the execution of autonomous driving control, the autonomous driving ECU 21 executes autonomous driving control that allows the driver to perform actions that prevent them from checking the surroundings of the vehicle at a predetermined timing.

[0059] Figure 9 is a flowchart showing an example of a notification process in which the control device 10 prompts the driver P to confirm that the mobile terminal 30, which receives notifications of vehicle information, is inside the vehicle. The notification process is performed by the CPU 11 reading program 14A from the storage unit 14, loading it into RAM 13, and executing it, thereby performing the output process.

[0060] As shown in Figure 9, in step S10, the CPU 11 determines, based on the vehicle information transmitted from the navigation system 22, whether or not autonomous driving control is scheduled to be performed in the driving plan for the vehicle 20 to its destination. The CPU 11 may also determine whether or not autonomous driving control is scheduled to be performed in the driving plan for a predetermined time from the current time. If it is determined that autonomous driving control is scheduled to be performed, the determination in step S10 is affirmed, and the process proceeds to step S11. On the other hand, if it is determined that autonomous driving control is not scheduled to be performed, the determination in step S10 is denied, and the notification process ends.

[0061] In step S11, the CPU 11 determines whether the driver is permitted to perform actions that prevent the driver from checking the vehicle's surroundings during the planned autonomous driving control. If it is determined that actions that prevent the driver from checking the vehicle's surroundings are permissible, the determination in step S11 is affirmed, and the process proceeds to step S12. On the other hand, if it is determined that actions that prevent the driver from checking the vehicle's surroundings are not permissible, the determination in step S11 is denied, and the notification process ends.

[0062] In step S12, the CPU 11 confirms the position of the mobile terminal 30 based on the function of the determination unit 11A.

[0063] In step S13, the CPU 11 determines whether the mobile terminal 30 is inside the vehicle based on the function of the determination unit 11A. If it is determined that the mobile terminal 30 is not inside the vehicle, the determination in step S13 is rejected, and the process proceeds to step S14. In step S14, the CPU 11 sends information to the in-vehicle unit 25 notifying that vehicle information cannot be sent from the mobile terminal 30, and the in-vehicle unit 25 notifies the driver P (Figure 4).

[0064] On the other hand, if it is determined in step S13 that the mobile terminal 30 is inside the vehicle, the determination in step S13 is affirmed, and the process proceeds to step S15.

[0065] In step S15, the CPU 11 initiates facial recognition of driver P via the mobile terminal 30 based on the functions of the execution unit 11C. Specifically, it activates the camera 38 mounted on the mobile terminal 30, and attempts to detect driver P's face from the captured image data by referring to the image data captured by the camera 38 and the facial recognition data previously registered for driver P (Figure 5).

[0066] In step S16, the CPU 11 determines whether or not the driver P's face has been detected from the image data captured by the camera 38. If the driver P's face is detected, the determination in step S16 is affirmed, and the process proceeds to step S17. On the other hand, if the driver P's face is not detected, the determination in step S16 is denied, and the notification process ends.

[0067] In step S17, the CPU 11 sends permission information to the vehicle 20's autonomous driving ECU 21 regarding the execution of autonomous driving control that is permitted to the driver even if it prevents the driver from checking the surroundings of the vehicle.

[0068] (Mechanism of action and effect) As described above, in the control device 10 according to this embodiment, before autonomous driving control is performed in which the driver is permitted to take actions that prevent the driver from checking the surroundings of the vehicle, the control device 10 determines whether the mobile terminal 30 that receives notifications of vehicle information is inside the vehicle. If it is determined that the mobile terminal 30 is not inside the vehicle, the control device 10 notifies the driver via the in-vehicle unit 25 that vehicle information cannot be sent from the mobile terminal 30. This prompts the driver P to confirm that the mobile terminal 30 is inside the vehicle.

[0069] For example, the control device 10 can determine that the mobile terminal 30 is inside the vehicle when the vehicle 20 and the mobile terminal 30 are connected in a way that enables mutual communication. As a result, the control device 10 determines that the mobile terminal 30 is inside the vehicle when it is in a state where it can receive notifications of vehicle information, so the mobile terminal 30 can reliably receive vehicle information, thereby improving safety during autonomous driving.

[0070] Furthermore, for example, a predetermined physical quantity is detected using an on-board sensor 24 mounted on the vehicle 20 and a terminal sensor 40 mounted on the mobile terminal 30. If the detected values ​​from these sensors match, it may be determined that the mobile terminal 30 is inside the vehicle. In other words, when the detected values ​​from the on-board sensor 24 and the terminal sensor 40 match, it indicates that the mobile terminal 30 is in the same environment as the vehicle 20. Therefore, by considering whether the environment surrounding the mobile terminal 30 matches the environment inside the vehicle, it is possible to determine whether the mobile terminal 30 is inside the vehicle with high accuracy.

[0071] Furthermore, in this embodiment, when it is determined that the mobile terminal 30 is inside the vehicle, it enables the execution of autonomous driving control that allows the driver to perform actions that prevent them from checking the surroundings of the vehicle. As a result, since autonomous driving control is executed only after it is ensured that vehicle information can be notified via the mobile terminal 30, safety during autonomous driving can be enhanced.

[0072] Furthermore, in this embodiment, when it is determined that the mobile terminal 30 is inside the vehicle, the camera 38 mounted on the mobile terminal 30 is activated. The control device 10 also refers to the image data captured by the camera 38 and the facial recognition data pre-registered for the driver P, and detects the driver P's face from the image data. When the driver P's face is detected from the captured image data, the control device 10 determines that the driver has consented to the execution of autonomous driving control, and enables the execution of autonomous driving control that allows the driver to perform actions that prevent them from checking the surroundings of the vehicle. In this way, the execution of autonomous driving control can be enabled based on the fact that the mobile terminal 30 is inside the vehicle and the fact that the driver P has consented to the execution of autonomous driving control through facial recognition. As a result, vehicle information can be appropriately notified to the mobile terminal 30 held by the driver P, thereby improving safety during autonomous driving.

[0073] In the above embodiment, the control device 10 is configured to be mounted on the vehicle 20, but this is not limited to that configuration. It may also be configured as an external device such as a server.

[0074] Furthermore, the above notification process may be executed not only at predetermined control cycles, but also when the communication status between the vehicle 20 and the mobile terminal 30 deteriorates. Alternatively, for example, the notification process may be executed when the vehicle's ignition switch is turned ON or when the autonomous driving system is activated, and the location of the mobile terminal may be confirmed in advance before sending the notification. Furthermore, the navigation system 22 may be configured to set a driving route to the destination, and if the driving route includes a section where autonomous driving is permitted and the driver is allowed to perform actions that prevent the vehicle from checking its surroundings, a notification process may be executed.

[0075] Furthermore, in the present invention, the method for determining whether or not the mobile terminal 30 is inside the vehicle is not limited to the method described in the above embodiment. For example, the determination may be made by analyzing images captured by a camera that monitors the inside of the vehicle or a camera that photographs the driver. In addition, even if it is confirmed that the mobile terminal is inside the vehicle by such an in-vehicle monitoring system or by comparing the detection value of the in-vehicle sensor with the detection value of the terminal sensor, the system may be configured to determine that the mobile terminal is not inside the vehicle if the communication state of the mobile terminal is weak, the power of the mobile terminal is turned off, or the battery level of the mobile terminal is below a threshold.

[0076] Furthermore, the smart key 50 for the vehicle 20 typically has a built-in battery and can emit an authentication signal so that the vehicle can lock and unlock the doors of the vehicle 20 from a distance and for the vehicle to confirm that the smart key 50 is inside the vehicle. Using this function, the mobile terminal 30 may detect that the smart key 50 of the vehicle 20 is nearby, the mobile terminal 30 may notify the server 40 that the smart key 50 has been detected by the mobile terminal 30, and the server 30 may send a signal to the control device 10 via the communication unit 23 to determine that the mobile terminal 30 is inside the vehicle.

[0077] In addition, the notification processing that the CPU 11 reads and executes in the above embodiment may be executed by various processors other than the CPU. Examples of such processors include PLDs (Programmable Logic Devices) such as FPGAs (Field-Programmable Gate Arrays) whose circuit configuration can be changed after manufacturing, and dedicated electrical circuits that are processors with circuit configurations specifically designed to execute specific processing, such as ASICs (Application Specific Integrated Circuits). Furthermore, the notification processing may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (for example, multiple FPGAs, and a combination of a CPU and an FPGA). More specifically, the hardware structure of these various processors is an electrical circuit that combines circuit elements such as semiconductor elements.

[0078] Furthermore, although the above embodiment describes a configuration in which program 14A is pre-stored (installed) in the storage unit 14, the invention is not limited to this configuration. Program 14A may be provided in the form of a recording medium such as a CD-ROM (Compact Disk Read Only Memory), DVD-ROM (Digital Versatile Disk Read Only Memory), or USB (Universal Serial Bus) memory. Alternatively, program 14A may be provided in the form of a download from an external device via a network. [Explanation of Symbols]

[0079] 10 Control devices (vehicle control devices) 11A Judgment section 11B Notification Department 11C Execution Unit 20 vehicles 24 In-vehicle sensors 25 Onboard equipment 30 Mobile devices 38 Cameras 40 Terminal sensors

Claims

1. In a vehicle capable of autonomous driving control that allows the driver to perform actions that prevent them from checking the surroundings of the vehicle, a determination unit determines whether or not a mobile terminal for receiving vehicle information notifications is inside the vehicle before the autonomous driving control is executed, The system includes a notification unit that, when the determination unit determines that the mobile terminal is not inside the vehicle, notifies the vehicle via the in-vehicle device that the mobile terminal cannot send the vehicle information notification, The determination unit determines that the mobile terminal is inside the vehicle when the detected value of the on-board sensor mounted on the vehicle matches the detected value of the terminal sensor mounted on the mobile terminal for a predetermined physical quantity. Vehicle control device.

2. The determination unit determines that the mobile terminal is inside the vehicle when the vehicle and the mobile terminal are connected in a manner that enables mutual communication according to a predetermined communication standard. The vehicle control device according to claim 1.

3. The system further includes an execution unit that performs the aforementioned autonomous driving control, The execution unit enables the execution of the autonomous driving control when the determination unit determines that the mobile terminal is inside the vehicle. A vehicle control device according to claim 1 or claim 2.

4. The execution unit, when the determination unit determines that the mobile terminal is inside the vehicle, activates the camera mounted on the mobile terminal, refers to the image data captured by the camera and the facial recognition data previously registered for the driver, and attempts to detect the driver's face from the image data. When the driver's face is detected from the image data, it is determined that the execution of the autonomous driving control has been approved, and the execution of the autonomous driving control is enabled. The vehicle control device according to claim 3.

5. A vehicle control method for a vehicle capable of autonomous driving control in which the driver is permitted to perform actions that prevent the driver from checking the surroundings of the vehicle, wherein, before the autonomous driving control is executed, the computer performs a process to determine whether or not a mobile terminal that receives notifications of vehicle information is inside the vehicle. The computer determines that the mobile terminal is inside the vehicle when the detected value of the on-board sensor mounted on the vehicle matches the detected value of the terminal sensor mounted on the mobile terminal for a predetermined physical quantity. If it is determined that the aforementioned mobile device is not inside the vehicle, the in-vehicle device will notify the user that the vehicle information cannot be sent from the mobile device. Vehicle control method.

Images