Vehicle controlling device and vehicle controlling method
The vehicle control system ensures safe autonomous driving by verifying the presence and readiness of the driver using a mobile device within the vehicle, addressing distractions and enhancing safety during transitions.
Patent Information
- Application Number
- JP2023075848
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-01
- Publication Date
- 2024-11-14
- Estimated Expiration
- 2043-05-01
AI Technical Summary
In autonomous vehicle driving, drivers may be distracted by using mobile devices, and there is a need to ensure they are present and capable of resuming control when required, while ensuring safe transitions between autonomous and manual driving modes.
A vehicle control system that determines the presence of a mobile device within the vehicle using communication and sensor matching, and enables autonomous driving only when the device is present and the driver is authenticated, ensuring they are not distracted.
Enhances safety by ensuring drivers are prepared to take control during autonomous driving transitions, reducing distractions and improving overall driving safety.
Smart Images

Figure 2024160630000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a vehicle control device and a vehicle control method. [Background technology]
[0002] The following Patent Document 1 describes a technology in which, when a driver operates a mobile terminal while the vehicle is traveling in an autonomous driving mode, an alarm is issued from a speaker mounted on the mobile terminal and the vehicle. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2020-102252 A Summary of the Invention [Problem to be solved by the invention]
[0004] Incidentally, in autonomous driving (automated driving) of a vehicle under certain conditions, it is assumed that the driver is temporarily released from the obligation to pay attention to the road ahead and is permitted to engage in actions that do not allow the driver to check the surroundings of the vehicle, such as operating a mobile device. On the other hand, in such autonomous driving, the driver must be able to return to driving at any time upon request from the autonomous driving system. In this case, it is conceivable that notifications of information regarding autonomous driving and requests to the driver may be sent to the driver using a mobile device.
[0005] In order to provide the driver with appropriate notification via the mobile device, it is desirable to prompt the driver to confirm that the mobile device is within the vehicle cabin before autonomous driving control at a level or above that is permitted, which would prevent the driver from checking the vehicle's surroundings.
[0006] Taking into consideration the above facts, the present invention aims to provide a vehicle control device and a vehicle control method that can prompt the driver to confirm that a mobile terminal that receives vehicle information notifications is inside the vehicle in a vehicle capable of autonomous driving control that allows the driver to perform actions that do not allow the driver to check the vehicle's surroundings. [Means for solving the problem]
[0007] The vehicle control device of the present invention described in claim 1 is provided in a vehicle capable of autonomous driving control in which the driver is permitted to perform actions that do not allow the driver to check the vehicle's surroundings, and includes a judgment unit that judges whether or not a mobile terminal that receives notification of vehicle information is present in the vehicle cabin before the autonomous driving control is executed, and a notification unit that, when the judgment unit determines that the mobile terminal is not present in the vehicle cabin, notifies via an in-vehicle device that the vehicle information cannot be notified from the mobile terminal.
[0008] In the vehicle control device according to the present invention described in claim 1, before autonomous driving control that allows the driver to perform actions that prevent the driver from checking the surroundings of the vehicle is executed, it is determined whether or not a mobile terminal that receives notifications of vehicle information is present in the vehicle. If it is determined that the mobile terminal is not present in the vehicle, the mobile terminal notifies the driver via the in-vehicle device that notification of vehicle information cannot be performed. This makes it possible to prompt the driver to confirm that the mobile terminal is present in the vehicle.
[0009] The vehicle control device of the present invention described in claim 2 has the configuration described in claim 1, and the determination unit determines that the mobile terminal is located inside the vehicle cabin when the vehicle and the mobile terminal are connected so as to be able to communicate with each other using a predetermined communication standard.
[0010] In the vehicle control device according to the present invention described in claim 2, when the vehicle and the mobile terminal are connected to be able to communicate with each other, it is determined that the mobile terminal is inside the vehicle. As a result, when the mobile terminal is in a state where it can receive notifications of vehicle information, it is determined that the mobile terminal is inside the vehicle, so that the mobile terminal can reliably receive vehicle information, thereby improving safety during autonomous driving.
[0011] The vehicle control device of the present invention described in claim 3 has the configuration described in claim 1, and when a detection value of a predetermined physical quantity from an on-board sensor mounted on the vehicle matches a detection value of a terminal sensor mounted on the mobile terminal, the determination unit determines that the mobile terminal is inside the vehicle cabin.
[0012] In the vehicle control device according to the present invention described in claim 3, a predetermined physical quantity is detected using an on-board sensor mounted on the vehicle and a terminal sensor mounted on the mobile terminal. If the detection values of these sensors match, it is determined that the mobile terminal is inside the vehicle. In other words, if the detection values of the sensors match, this indicates that the mobile terminal is placed in the same environment as the vehicle. Therefore, it is possible to determine whether the mobile terminal is inside the vehicle by considering whether the environment around the mobile terminal matches the environment inside the vehicle, thereby making it possible to perform a highly accurate determination.
[0013] Note that the phrase "when the sensor detection values match" is a broad concept that includes not only a perfect match but also a match within a predetermined error range.
[0014] The vehicle control device of the present invention described in claim 4, in the configuration described in any one of claims 1 to 3, further has an execution unit that executes the autonomous driving control, and the execution unit enables the execution of the autonomous driving control when the determination unit determines that the mobile terminal is inside the vehicle cabin.
[0015] In the vehicle control device according to the present invention described in claim 4, when it is determined that the mobile terminal is in the vehicle interior, it is possible to execute autonomous driving control that allows the driver to perform actions that do not allow the driver to check the surroundings of the vehicle. This allows the driver to execute autonomous driving control after ensuring a state in which notification of vehicle information via the mobile terminal is possible, thereby improving safety during autonomous driving.
[0016] The vehicle control device of the present invention described in claim 5 has the configuration described in claim 4, and when the determination unit determines that the mobile terminal is within the vehicle cabin, the execution unit activates a camera mounted on the mobile terminal, refers to image data captured by the camera and facial recognition data pre-registered for the driver, and attempts to detect the driver's face from the image data, and when the driver's face is detected from the image data, determines that the execution of the autonomous driving control has been approved, and enables the execution of the autonomous driving control.
[0017] In the vehicle control device according to the present invention described in claim 5, when it is determined that the mobile terminal is in the vehicle interior, the camera mounted on the mobile terminal is activated. The vehicle control device also refers to image data captured by the camera and face authentication data registered in advance for the driver, and attempts to detect the driver's face from the image data. When the driver's face is detected from the image data, the vehicle control device determines that the execution of the autonomous driving control has been approved, and enables the execution of the autonomous driving control in which the driver is permitted to perform actions that do not allow the driver to check the surroundings of the vehicle. This makes it possible to enable the execution of the autonomous driving control based on the fact that the mobile terminal is in the vehicle interior and the fact that the execution of the autonomous driving control has been approved by the driver's face authentication. As a result, vehicle information can be appropriately notified to the mobile terminal held by the driver, and safety during autonomous driving can be improved.
[0018] The vehicle control method of the present invention described in claim 6, in a vehicle capable of autonomous driving control in which the driver is permitted to perform actions that do not allow the driver to check the vehicle's surroundings, determines whether or not a mobile terminal that receives notifications of vehicle information is present in the vehicle cabin before the autonomous driving control is executed, and if it is determined that the mobile terminal is not present in the vehicle cabin, notifies the vehicle driver via an in-vehicle device that the mobile terminal cannot notify the vehicle information.
[0019] In the vehicle control method according to the present invention recited in claim 6, as described above, it is possible to prompt the driver to confirm that the mobile terminal is inside the vehicle. Effect of the Invention
[0020] As described above, the vehicle control device and vehicle control method of the present invention can prompt the driver in a vehicle capable of autonomous driving control in which the driver is permitted to perform actions that do not allow the driver to check the vehicle's surroundings to check that a mobile terminal that receives vehicle information notifications is inside the vehicle. [Brief description of the drawings]
[0021] [Figure 1] 1 is a diagram showing a schematic configuration of a system constructed by a vehicle control device according to an embodiment of the present invention; [Diagram 2] 2 is a block diagram showing a hardware configuration of the vehicle control device; FIG. [Diagram 3] 2 is a block diagram showing an example of a functional configuration of a vehicle control device; [Figure 4] FIG. 13 is a diagram showing an example of a notification performed via an in-vehicle device. [Diagram 5] FIG. 13 is a diagram illustrating an example of face authentication performed via a mobile terminal. [Figure 6] FIG. 11 is a diagram showing an example of notification information sent via a mobile terminal. [Figure 7] FIG. 13 is a diagram showing an example of a notification performed via an in-vehicle device. [Figure 8] FIG. 11 is a diagram illustrating an example of a notification sent via a mobile terminal. [Figure 9] 13 is a flowchart showing the flow of a notification process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0022] Hereinafter, the system S according to this embodiment will be described with reference to Fig. 1 to Fig. 6. The system S according to this embodiment is a system that prompts the driver P to confirm that the mobile terminal 30 that receives notifications of vehicle information is in the vehicle interior before a predetermined autonomous driving control is executed in the vehicle 20.
[0023] As shown in Fig. 1, in the system S, the vehicle 20 and the mobile terminal 30 of the driver P are connected to be able to communicate with each other according to a predetermined communication standard. In addition, in the system S, the smart key 50 of the vehicle 20 is connected to the vehicle 20 and the mobile terminal 30 to be able to communicate with each other according to a predetermined wireless communication standard. The vehicle 20 can remotely perform various operations such as locking and unlocking the doors and starting the engine based on control commands transmitted from the smart key 50 by operation of the driver P, etc. Furthermore, in the system S, the vehicle 20, the mobile terminal 30, and the server 40 are connected to be able to communicate with each other via a network N.
[0024] In the above system S, vehicle information transmitted from the vehicle 20 is notified to the driver P via the vehicle-mounted device 25 and the mobile terminal 30. In this embodiment, the transmission of notification information to the vehicle-mounted device 25 and the mobile terminal 30 mounted in the vehicle 20 is controlled by the control device 10. The control device 10 is an example of a "vehicle control device."
[0025] (Hardware configuration of the control device 10) The hardware configuration of the control device 10 will be described. Fig. 2 is a block diagram showing the hardware configuration of the system S. The control device 10 is composed of, for example, one or more ECUs (Electronic Control Units) mounted on a vehicle 20. As shown in Fig. 2, the control device 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage unit 14, and an input / output interface (I / F) 15. Each component is connected to each other via a bus 18 so as to be able to communicate with each other.
[0026] The CPU 11 is a central processing unit that executes various programs and controls each part. That is, the CPU 11 reads a program from the ROM 12 or the storage unit 14, and executes the program using the RAM 13 as a working area. The CPU 11 controls each of the above components and performs various arithmetic processing according to the program recorded in the ROM 12 or the storage unit 14. The ROM 12 stores various programs and various data. The RAM 13 serves as a working area for temporarily storing programs or data. The storage unit 14 is configured with a storage device such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory, and stores various programs and various data. The storage unit 14 stores a program 14A for causing the CPU 11 to execute a notification process described later. The storage unit 14 also stores a notification information database 14B that stores vehicle information to be notified to the vehicle-mounted device 25 of the vehicle 20 and the mobile terminal 30 of the driver P.
[0027] The input / output interface 15 is an interface for connecting the control device 10 to other devices. An automatic driving ECU 21, a navigation system 22, a communication unit 23, an on-board sensor 24, and an on-board device 25 mounted on a vehicle 20 are connected to the input / output interface 15.
[0028] The autonomous driving ECU 21 is an electronic control unit that executes various controls related to the autonomous driving of the vehicle 20, and constitutes an autonomous driving system for the vehicle 20. The automobile ECU 20 is connected to a powertrain system 21A, a brake system 21B, and a steering system 21C of the vehicle 20, and controls the autonomous driving of the vehicle 20 via these systems. Specifically, the autonomous driving ECU 20 is connected to sensors or a navigation system, which will be described later, and controls the autonomous driving of the vehicle 20 based on information acquired from sensors or a navigation system 22, which will be described later. Autonomous driving is a driving state in which some or all of the driving operations are performed automatically, and the vehicle 20 is caused to automatically travel along the road on which the vehicle 20 travels.
[0029] The autonomous driving ECU 21 executes control of autonomous driving that conforms to a predetermined autonomous driving level defined by organizations such as the SAE (Society of Automotive Engineers), NHTSA (National Highway Traffic Safety Administration), and JSAE (Society of Automotive Engineers of Japan). The predetermined autonomous driving level includes an autonomous driving level in which an autonomous driving device including the autonomous driving ECU 21 takes over all driving operations in a limited area that satisfies specific driving environment conditions. At this autonomous driving level and any autonomous driving level higher, the driver is permitted to perform actions that do not allow the driver to check the surroundings of the vehicle while autonomously driving. Here, actions that do not allow for confirmation of the surroundings of the vehicle are tasks or operations other than those that allow for confirmation of the surroundings of the vehicle, such as operating the vehicle or driving the vehicle, or checking or monitoring the surroundings of the vehicle for driving or driving. For example, actions that do not allow for confirmation of the surroundings of the vehicle include the driver eating or drinking in the vehicle and operating a mobile terminal.
[0030] The autonomous driving ECU 21 enables autonomous driving, in which the vehicle 20 automatically drives toward a preset destination, without the driver P performing any driving operation, based on position information, map information, and guided route information of the vehicle 20 from the navigation system 22 described below. In addition, the autonomous driving ECU 21 transmits, for example, information regarding the execution or change of autonomous driving control to the navigation system 22.
[0031] In addition, the automatic driving ECU 21 may be omitted, and the control device 10 may constitute the automatic driving ECU.
[0032] The navigation system 22 provides driving guidance for the vehicle 20 to a destination set by the driver P. The navigation system 22 includes, for example, a GPS receiver 22A for measuring position information of the vehicle 20, and a map database 22B that stores map information. The navigation system 22 recognizes the roads and lanes on which the vehicle 20 is traveling, based on the position information of the vehicle 20 measured by the GPS receiver 22A and the map information in the map database 22B. The navigation system 22 calculates a route from the position of the vehicle 20 to the destination, and provides guidance on the route to the driver P by displaying on a display included in the in-vehicle device 25 described later and outputting audio from a speaker. The navigation system 22 transmits, for example, the position information of the vehicle 20, information on the roads on which the vehicle 20 is traveling, and information on the guided route for the vehicle 20 to the control device 10. In addition, the navigation system 22 transmits vehicle information, including information regarding the execution or change of autonomous driving control, information to alert the driver during autonomous driving, and requests to the driver regarding autonomous driving, to the control device 10 and the mobile terminal 30 via the communication unit 23 described later.
[0033] The communication unit 23 is an interface for communicating with other devices. For example, a wired communication standard such as Ethernet (registered trademark) or FDDI, or a wireless communication standard such as 4G, 5G, Bluetooth (registered trademark), or Wi-Fi (registered trademark) is used for the communication. These standards are examples of the "predetermined communication standard." In this embodiment, the vehicle 20 and the mobile terminal 30 of a user who is registered in advance as the driver of the vehicle 20 are connected to each other via the communication unit 23 so as to be able to communicate with each other.
[0034] The on-board sensor 24 includes a surrounding sensor 24A for acquiring information about the surroundings (front, rear, right, left) of the vehicle 20, and an interior sensor 24B for acquiring information about the interior of the vehicle. The surrounding sensor 24A includes, for example, at least one of a camera that captures images of the surroundings of the vehicle, a millimeter wave radar, an ultrasonic sensor, a LiDAR (Light Detection and Ranging), etc., and detects the surrounding environment of the vehicle. The interior sensor 24B includes, for example, at least one of a camera that captures images of the interior of the vehicle, an acceleration sensor, a yaw rate sensor, a temperature sensor, a microphone, etc.
[0035] The above-mentioned on-board sensor 24 is capable of detecting a predetermined physical quantity related to the state of the vehicle 20. In this embodiment, it is possible to determine whether or not the mobile terminal 30 of the driver P is present inside the vehicle 20 based on the physical quantity detected by the on-board sensor 24. The predetermined physical quantity related to the state of the vehicle is not particularly limited. For example, it may be a physical quantity related to the running state of the vehicle 20, such as the acceleration or tilt angle of the vehicle 20. In addition, it may be a detectable physical quantity related to the temperature inside the vehicle cabin, the sound inside the vehicle cabin, or the environment around the vehicle.
[0036] The vehicle-mounted device 25 is configured with devices capable of outputting information transmitted from the control device 10 to the driver P in the vehicle cabin. For example, the vehicle-mounted device 25 may be configured with a display, a speaker, an indicator light, a steering wheel, a seat, a seat belt, and the like.
[0037] (Hardware configuration of mobile terminal 30) Next, a hardware configuration of the mobile terminal 30 will be described. The mobile terminal 30 is, for example, a smartphone or a tablet terminal, and can be brought into and taken out of the vehicle. The mobile terminal 30 includes a CPU 31, a ROM 32, a RAM 33, a storage unit 34, an input unit 35, an output unit 36, a communication unit 37, a camera 38, and a terminal sensor 40. Each component is connected to each other via a bus 39 so as to be able to communicate with each other.
[0038] The CPU 31 is a central processing unit that reads out a program from the ROM 32 or the storage unit 34, and executes the program using the RAM 33 as a working area. The CPU 31 controls the above-mentioned components and performs various types of calculation processing according to the program recorded in the ROM 32 or the storage unit 34. The ROM 32 stores various programs and various data. The RAM 33 serves as a working area for temporarily storing programs or data. The storage unit 34 is configured with a storage device such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory, and stores various programs and various data. The storage unit 34 stores a program 34A for causing the CPU 31 to output various information transmitted from the control device 10 and the vehicle 20 to the mobile terminal 30, as described below. The input unit 35 includes a pointing device such as a mouse, a keyboard, a microphone, a camera, and the like, and is used to perform various input operations. The output unit 36 is, for example, a liquid crystal display or a speaker, and outputs various information. When the output unit 36 is configured as a display, it may function as the input unit 35 by adopting a touch panel system. The communication unit 37 is an interface for communicating with other devices. For this communication, for example, a wired communication standard such as Ethernet (registered trademark) or FDDI, or a wireless communication standard such as 4G, 5G, Bluetooth (registered trademark), or Wi-Fi (registered trademark) is used. The terminal sensor 40 is configured with a sensor for detecting a predetermined physical quantity related to the state of the mobile terminal. The terminal sensor 40 includes at least one of an acceleration sensor, a yaw rate sensor, a camera, a millimeter wave radar, an ultrasonic sensor, a LiDAR (Light Detection and Ranging), a temperature sensor, a microphone, and the like. The predetermined physical quantity related to the state of the mobile terminal 30 is not particularly limited. For example, various physical quantities such as the acceleration and tilt angle of the mobile terminal 30, the temperature around the terminal, surrounding sounds, and the physical quantities of objects present in the vicinity correspond to the "predetermined physical quantity related to the state of the mobile terminal 30".
[0039] (Functional configuration of the control device 10) Next, a description will be given of the functional configuration of the control device 10. Fig. 3 is a block diagram showing an example of the functional configuration of the control device 10.
[0040] The determination unit 11A determines whether or not the mobile terminal 30 that receives notification of vehicle information is present inside the vehicle cabin before autonomous driving control that allows the driver P to act in a way that prevents the driver P from checking the vehicle's surroundings is executed in the vehicle 20. "Autonomous driving control that allows the driver P to act in a way that prevents the driver P from checking the vehicle's surroundings" is, for example, autonomous driving of level 3 or higher defined by the Society of Automotive Engineers (SAE).
[0041] The method of determining whether or not the mobile terminal 30 is inside the vehicle cabin is not particularly limited, but for example, the determination unit 11A may determine that the driver P's mobile terminal 30 is inside the vehicle cabin when the vehicle 20 and the mobile terminal 30 are connected so as to be able to communicate with each other using a predetermined communication standard.
[0042] Furthermore, the determination unit 11A may determine that the mobile terminal 30 is inside the vehicle cabin when a detection value of a predetermined physical quantity between the on-board sensor 24 mounted on the vehicle 20 and a detection value between the terminal sensor 40 mounted on the mobile terminal 30 match. Note that the term "match" here is not limited to a case where the detection values match completely, but is a broad concept including a case where the detection values match within a predetermined error range.
[0043] When the determination unit 11A determines that the mobile terminal 30 is not inside the vehicle cabin, the notification unit 11B notifies the driver P through the in-vehicle device 25 that the mobile terminal 30 cannot notify vehicle information related to the autonomous driving of the vehicle 20. In an example shown in FIG. 4, text information "Notification from mobile terminal not possible" is displayed on a display 25A installed on the vehicle front side of the driver's seat (not shown), thereby urging the driver P to carry the mobile terminal 30 in an appropriate state. However, without being limited to this, the notification unit 11B may notify by outputting audio information to a speaker as the in-vehicle device 25, or may vibrate the steering wheel or the seat to urge the driver P to carry the mobile terminal in an appropriate state.
[0044] In addition, when autonomous driving control is executed in the vehicle 20, in which the driver P is permitted to perform an action that does not allow the driver P to check the surroundings of the vehicle, the notification unit 11B causes at least one of the vehicle-mounted device 25 mounted on the vehicle 20 and the mobile terminal 30 of the driver P to notify vehicle information related to the autonomous driving. The vehicle information (notification information) notified from the vehicle-mounted device 25 and the mobile terminal 30 is stored in the notification information database 14B. As shown in FIG. 6●●, the notification information database 14B stores various notification information in association with items such as "notification classification", "event item", and "user action". As an example, the "notification classification" includes "driving change warning", "driving change notification", "system state, transition (MRM)", "system state, transition (MRC)", "function proposal from the system", and "vehicle state notification". Then, specific events corresponding to each item included in these "notification classifications" are set in the "event item".
[0045] The event corresponding to the "driving change warning" is set to a situation in which driving changeover from the vehicle 20 to the driver P occurs suddenly. Specifically, the events corresponding to the event are set to include the vehicle 20 going out of the autonomous driving area, encountering a dangerous situation, a system failure or malfunction, a sudden deterioration in weather, a transition of the autonomous driving level to a level lower than the current autonomous driving level, and an accident. These events are set as situations in which it becomes suddenly difficult to continue autonomous driving at the current autonomous driving level, and it is necessary for the driver P to respond immediately. Therefore, in the "user action" item, the event is associated with an item that "immediate response is required."
[0046] The event corresponding to "notification of driving change" is set to a situation in which driving changeover from the vehicle 20 to the driver P occurs in a non-sudden situation. Specifically, it includes the vehicle 20 going out of the automated driving area, encountering a dangerous situation, a system failure or malfunction, a sudden deterioration in weather, a transition of the automated driving level to a level lower than the current automated driving level, and an accident. These events are set as cases in which manual driving is predicted to be necessary when traveling in a specified area, and a response from the driver P is required before reaching the specified area. Therefore, in the "user action" item, it is associated with the item "response required before reaching manual driving area".
[0047] As an event corresponding to "System State Transition (MRM)", it is set that the driving control state of the vehicle 20 transitions to MRM (Minimal Risk Maneuver). MRM is a vehicle motion control up to MRC, which will be described later, as a measure to be taken when an event occurs that makes it impossible to drive safely.
[0048] As an event corresponding to "System State,Transition (MRM)", the driving control state of the vehicle 20 is set to transition to MRC (Minimal Risk Condition). MRC is a safe state that the vehicle ultimately aims for as a measure in case an event occurs in which the vehicle cannot be driven safely due to a malfunction of the autonomous driving function, etc. In general, it refers to a stopped state in a situation where the risk of an accident is sufficiently low.
[0049] When an event corresponding to the above "System State,Transition (MRM)" and "System State,Transition (MRM)" occurs, it is unlikely that a specific action will be taken by the user (driver P), so it is associated with the "None" item in the "User Action" item. Note that, since "System State,Transition (MRM)" is a stage before reaching MRC, an action from the user may be required if necessary.
[0050] Events corresponding to "function suggestions from the system" include overtaking, passing, changing lanes, transitioning the autonomous driving level to a level higher than the current autonomous driving level, route selection via the navigation system 22, and deceleration at a toll booth on a toll road. These events can be executed by autonomous driving control and do not require a shift to manual driving, so approval from the driver P is sufficient. Therefore, in the "user action" category, they are associated with the "approval" category.
[0051] The remaining battery charge and the remaining fuel charge are set as events corresponding to "notification of driving state". These events are not related to the execution of autonomous driving control, and only need to be notified to the driver P. Therefore, in the "user action" category, they are associated with the "notification only" category.
[0052] The notification unit 11B refers to the vehicle information transmitted from the autonomous driving ECU 21 and the navigation system 22 and the notification information database 14B, and transmits notification information to the vehicle-mounted device 25 and the mobile terminal 30. The transmitted notification information is notified to the driver P via the vehicle-mounted device 25 and the mobile terminal 30.
[0053] Specifically, with reference to Figs. 7 and 8, an example will be described in which information requesting a driver changeover is notified from the vehicle 20 to the driver P. Here, a case will be described in which the driver P is requested to change over when the vehicle 20 is scheduled to enter a manual driving area from an automated driving area in conjunction with the navigation system 22. Since this notification is not a request for a sudden driver changeover, it is classified as a "driving changeover notification" in the notification information database 14B and corresponds to the event of "leaving the automated driving area". Therefore, a user action of the driver P to change over to a driving changeover is required before the vehicle 20 reaches the manual driving area.
[0054] As shown in FIG. 7, in the vehicle cabin, text information Q1 requesting a driver changeover and an icon image Q2 indicating the driver changeover request are displayed on the display 25A as the in-vehicle device 25 as a notification from the vehicle. The message requesting a driver changeover also includes information regarding the time required to reach the manual driving area. Meanwhile, an icon image indicating a driver changeover request and a confirmation operation object Q3 requesting confirmation of the driver changeover notification are displayed on the mobile terminal 30 of the driver P as a notification from the vehicle. When the driver P performs an input operation on the confirmation operation object Q3, confirmation information is transmitted from the mobile terminal 30. When the vehicle 20 receives the confirmation information, the vehicle 20 determines that the driver changeover request from the vehicle 20 has been confirmed by the driver P.
[0055] It is preferable that the amount of information notified via the mobile terminal 30, such as the above notification, is set to be smaller than that of information notified via the in-vehicle device 25. In the above example, the information output from the display 25A includes text information Q1 indicating a driver change, whereas the information displayed on the mobile terminal 30 omits the text information Q1 indicating a driver change and displays only the icon image Q2. Therefore, the information can be presented as easy-to-understand visual information even on a small mobile terminal. Furthermore, when the vehicle 20 requests the driver P to participate in the driving operation as in the above notification, it is preferable to request the driver P to perform a confirmation operation in response to the notification via the mobile terminal 30. In the above example, the confirmation operation object Q3 is displayed on the mobile terminal 30, and the input operation to the confirmation operation object Q3 is received by the vehicle 20. This can improve the accuracy of the notification to the driver P and enhance the safety of driving.
[0056] When the determination unit 11A determines that the mobile terminal 30 is inside the vehicle cabin, the execution unit 11C enables the execution of autonomous driving control that permits the driver to perform actions that do not allow the driver to check the surroundings of the vehicle. Here, when the determination unit 11A determines that the mobile terminal 30 of the driver P is inside the vehicle cabin, the execution unit 11C activates the camera 38 mounted on the mobile terminal 30 to perform face authentication of the driver P. For example, as shown in Fig. 5, the camera 38 of the mobile terminal 30 is automatically activated and text information such as "Please bring the driver's face closer" is displayed on the output unit 36 (display in Fig. 5) to prompt the driver operating the mobile terminal 30 to take a picture of his or her own face.
[0057] The execution unit 11C refers to image data captured by the camera 38 of the mobile terminal 30 and face authentication data registered in advance for the driver P, and attempts to detect the face of the driver P from the captured image data. The face authentication data may be stored in the memory unit 14 of the control device 10, or may be acquired from a storage of the vehicle 20 or an external server.
[0058] When the face of the driver P is detected from image data captured by the camera 38, the execution unit 11C determines that the driver P has consented to the execution of the autonomous driving control, and transmits execution permission information for the autonomous driving control to the autonomous driving ECU 21. By accepting the execution permission information for the autonomous driving control, the autonomous driving ECU 21 executes the autonomous driving control in which the driver is permitted to perform an action that does not allow the driver to check the surroundings of the vehicle at a predetermined timing.
[0059] 9 is a flowchart showing an example of a notification process in which the control device 10 notifies the driver P to check that the mobile terminal 30 that receives notifications of vehicle information is within the vehicle cabin. In the notification process, the CPU 11 reads out the program 14A from the storage unit 14, expands it in the RAM 13, and executes it, thereby performing output processing.
[0060] 9, in step S10, the CPU 11 determines whether or not the execution of autonomous driving control is scheduled in the driving plan to the destination of the vehicle 20 based on the vehicle information transmitted from the navigation system 22. The CPU 11 may also determine whether or not the execution of autonomous driving control is scheduled in the driving plan within a predetermined time from the current time. If it is determined that the execution of autonomous driving control is scheduled, the determination in step S10 is affirmative, and the process proceeds to step S11. On the other hand, if it is determined that the execution of autonomous driving control is not scheduled, the determination in step S10 is negative, and the notification process is terminated.
[0061] In step S11, the CPU 11 determines whether or not the driver is permitted to perform the action of not being able to check the surroundings of the vehicle in the scheduled autonomous driving control. If it is determined that the driver is permitted to perform the action of not being able to check the surroundings of the vehicle, the determination in step S11 is affirmative, and the process proceeds to step S12. On the other hand, if it is determined that the driver is not permitted to perform the action of not being able to check the surroundings of the vehicle, the determination in step S11 is negative, and the notification process is terminated.
[0062] In step S12, the CPU 11 checks the position of the mobile terminal 30 based on the function of the determination unit 11A.
[0063] In step S13, the CPU 11 judges whether or not the mobile terminal 30 is inside the vehicle based on the function of the judgment unit 11A. If it is judged that the mobile terminal 30 is not inside the vehicle, the judgment in step S13 is negative, and the process proceeds to step S14. In step S14, the CPU 11 transmits information notifying the vehicle information that the mobile terminal 30 cannot notify the vehicle information to the vehicle-mounted device 25, and notifies the driver P via the vehicle-mounted device 25 (FIG. 4).
[0064] On the other hand, if it is determined in step S13 that the mobile terminal 30 is located inside the vehicle, the determination in step S13 is affirmative, and the process proceeds to step S15.
[0065] In step S15, the CPU 11 starts face authentication of the driver P via the mobile terminal 30 based on the function of the execution unit 11C. Specifically, the camera 38 mounted on the mobile terminal 30 is started, and the image data captured by the camera 38 and face authentication data registered in advance for the driver P are referenced to attempt to detect the face of the driver P from the captured image data (FIG. 5).
[0066] In step S16, the CPU 11 determines whether or not the face of the driver P has been detected from the image data captured by the camera 38. If the face of the driver P has been detected, the determination in step S16 is affirmative, and the process proceeds to step S17. On the other hand, if the face of the driver P has not been detected, the determination in step S16 is negative, and the notification process is terminated.
[0067] In step S17, the CPU 11 transmits execution permission information to the autonomous driving ECU 21 of the vehicle 20 regarding the execution of autonomous driving control in which the driver is permitted to perform an action that does not allow the driver to check the surroundings of the vehicle.
[0068] (Action and Effects) As described above, the control device 10 according to this embodiment judges whether or not the mobile terminal 30 that receives the notification of the vehicle information is inside the vehicle before executing the autonomous driving control that allows the driver to perform an action that does not allow the driver to check the surroundings of the vehicle. Then, if it is judged that the mobile terminal 30 is not inside the vehicle, the control device 10 notifies the driver P via the in-vehicle device 25 that the mobile terminal 30 cannot notify the vehicle information. This makes it possible to prompt the driver P to confirm that the mobile terminal 30 is inside the vehicle.
[0069] For example, the control device 10 can determine that the mobile terminal 30 is inside the vehicle cabin when the vehicle 20 and the mobile terminal 30 are connected so as to be able to communicate with each other. As a result, the mobile terminal 30 is determined to be inside the vehicle cabin when the mobile terminal 30 is in a state in which it can receive notifications of vehicle information, so that the mobile terminal 30 can reliably receive vehicle information, thereby improving safety during autonomous driving.
[0070] Also, for example, a predetermined physical quantity is detected using an on-board sensor 24 mounted on the vehicle 20 and a terminal sensor 40 mounted on the mobile terminal 30. Then, when the detection values of these sensors match, it may be determined that the mobile terminal 30 is inside the vehicle cabin. That is, when the detection values of the on-board sensor 24 and the terminal sensor 40 match, this indicates that the mobile terminal 30 is placed in the same environment as the vehicle 20. Therefore, it is possible to determine whether or not the mobile terminal 30 is inside the vehicle cabin by considering whether the environment around the mobile terminal 30 matches the environment inside the vehicle cabin, so that a highly accurate determination can be made.
[0071] In addition, in this embodiment, when it is determined that the mobile terminal 30 is inside the vehicle, it is possible to execute autonomous driving control that allows the driver to perform actions that do not allow the driver to check the surroundings of the vehicle. This allows the driver to execute autonomous driving control after ensuring a state in which vehicle information can be notified via the mobile terminal 30, thereby improving safety during autonomous driving.
[0072] In addition, in this embodiment, when it is determined that the mobile terminal 30 is in the vehicle interior, the camera 38 mounted on the mobile terminal 30 is activated. Furthermore, the control device 10 refers to image data captured by the camera 38 and face authentication data registered in advance for the driver P, and detects the face of the driver P from the image data. Then, when the face of the driver P is detected from the captured image data, it is determined that the execution of the autonomous driving control has been approved, and the control device 10 enables the execution of the autonomous driving control in which the driver is permitted to act in a manner that does not allow the driver to check the surroundings of the vehicle. This makes it possible to enable the execution of the autonomous driving control based on the fact that the mobile terminal 30 is in the vehicle interior and the fact that the execution of the autonomous driving control has been approved by the face authentication of the driver P. As a result, vehicle information can be appropriately notified to the mobile terminal 30 held by the driver P, and safety during autonomous driving can be improved.
[0073] In the above embodiment, the control device 10 is configured to be mounted on the vehicle 20, but this is not limiting. The control device 10 may be configured as an external device such as a server.
[0074] The notification process may be executed at a predetermined control cycle, or may be executed when the communication conditions between the vehicle 20 and the mobile terminal 30 deteriorate. For example, the notification process may be executed when the ignition switch of the vehicle is turned on or when the autonomous driving system is started, and the location of the mobile terminal may be confirmed in advance to provide a notification. Furthermore, the navigation system 22 may be configured to execute a notification process when a driving route to a destination is set by the navigation system 22 and the driving route includes a section in which autonomous driving is possible, in which the driver is permitted to act without being able to check the vehicle's surroundings.
[0075] In the present invention, the method of determining whether the mobile terminal 30 is in the vehicle cabin is not limited to the method described in the above embodiment. For example, the determination may be made by analyzing an image captured by a camera monitoring the vehicle cabin or a camera capturing an image of the driver. Even if it is confirmed that the mobile terminal is in the vehicle cabin by such a monitoring system for the vehicle cabin or a method of comparing the detection value of the on-board sensor with the detection value of the terminal sensor, the mobile terminal may be determined not to be in the vehicle cabin if the communication state of the mobile terminal is weak, if the power of the mobile terminal is turned off, or if the remaining battery charge of the mobile terminal is less than a threshold value.
[0076] Furthermore, the smart key 50 of the vehicle 20 usually has a built-in battery that can transmit an authentication signal so that the doors of the vehicle 20 can be locked and unlocked from a remote location and the vehicle can confirm that the smart key 50 is inside the vehicle cabin. Using this function, the mobile terminal 30 may detect that the smart key 50 of the vehicle 20 is nearby, notify the server 40 that the smart key 50 has been detected by the mobile terminal 30, and send a signal from the server 30 to the control device 10 via the communication unit 23 to determine that the mobile terminal 30 is inside the vehicle cabin.
[0077] In the above embodiment, the notification process executed by the CPU 11 by reading the software (program) may be executed by various processors other than the CPU. In this case, examples of the processor include a PLD (Programmable Logic Device) such as an FPGA (Field-Programmable Gate Array) whose circuit configuration can be changed after manufacture, and a dedicated electric circuit such as an ASIC (Application Specific Integrated Circuit) which is a processor having a circuit configuration designed exclusively for executing a specific process. The notification process may be executed by one of these various processors, or may be executed by a combination of two or more processors of the same or different types (for example, a plurality of FPGAs, and a combination of a CPU and an FPGA). The hardware structure of these various processors is, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.
[0078] In the above embodiment, the program 14A is stored (installed) in advance in the storage unit 14, but the present invention is not limited to this. The program 14A may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The program 14A may be downloaded from an external device via a network. [Explanation of symbols]
[0079] 10 Control device (vehicle control device) 11A Judgment section 11B Notification Department 11C Executive Department 20 Vehicles 24 Vehicle Sensors 25 Onboard equipment 30 Mobile Devices 38 Camera 40 Terminal Sensor
Claims
1. In a vehicle capable of autonomous driving control in which a driver is permitted to perform an action that does not allow the driver to check the surroundings of the vehicle, a determination unit determines whether or not a mobile terminal that receives a notification of vehicle information is present in a vehicle cabin before the autonomous driving control is executed; a notification unit that notifies the vehicle driver, via an in-vehicle device, that the vehicle information cannot be notified from the mobile terminal when the determination unit determines that the mobile terminal is not located within the vehicle interior; A vehicle control device comprising:
2. the determination unit determines that the mobile terminal is located within the vehicle cabin when the vehicle and the mobile terminal are connected to each other so as to be able to communicate with each other according to a predetermined communication standard; The vehicle control device according to claim 1 .
3. the determination unit determines that the mobile terminal is located inside the vehicle when a detection value of a predetermined physical quantity detected by an in-vehicle sensor mounted on the vehicle matches a detection value of a terminal sensor mounted on the mobile terminal; The vehicle control device according to claim 1 .
4. An execution unit that executes the autonomous driving control, The execution unit enables the execution of the autonomous driving control when the determination unit determines that the mobile terminal is located inside the vehicle. The vehicle control device according to any one of claims 1 to 3.
5. When the determination unit determines that the mobile device is located inside the vehicle, the execution unit activates a camera mounted on the mobile device, refers to image data captured by the camera and face recognition data registered in advance for the driver, and attempts to detect the face of the driver from the image data; When the face of the driver is detected from the image data, it is determined that the execution of the autonomous driving control has been approved, and the execution of the autonomous driving control is enabled. The vehicle control device according to claim 4.
6. In a vehicle capable of autonomous driving control in which a driver is permitted to perform an action that does not allow the driver to check the surroundings of the vehicle, before the autonomous driving control is executed, a determination is made as to whether or not a mobile terminal that receives a notification of vehicle information is present in the vehicle interior; When it is determined that the mobile terminal is not within the vehicle cabin, a notification is given via an in-vehicle device that the vehicle information cannot be notified from the mobile terminal. A control method for a vehicle.
Citation Information
Patent Citations
On-vehicle unit
JP2015097012A
Driver monitoring system
JP2020102252A
Control device
JP2021142936A
Information processing device, program, and information processing method
JP2021189737A
Vehicular device, vehicular system and mislaying notification management program
JP2022057223A
Cited By
Moving body operation system
US12515658B2
