Notification device and notification method

Abstract

The notification device (10) includes: an information acquisition unit (101) that acquires driving operation information indicating the driving operation status of the driver of the vehicle; and a notification unit (102) that, during the period from when the driving authority is transferred from the vehicle to the driver until the driving authority is transferred to the driver, notifies the driving operation requested of the driver, and the notification unit (102) notifies the driving operation that the driver has not completed among the driving operations requested of the driver based on the driving operation information.

Description

Technical Field

[0001] The present invention relates to a notification device and a notification method for notifying a driver of a moving body of an operation to be performed. Background Art

[0002] In a vehicle capable of autonomous driving, the driving authority is sometimes transferred from the vehicle in autonomous driving to the driver. For example, when the vehicle finishes traveling on an exclusive autonomous driving lane, the driving authority of the vehicle is transferred from the vehicle to the driver. However, when the autonomous driving of the vehicle continues for a long time, the reaction speed of the driver may become dull and the driver may not be able to feel the driving operation feeling.

[0003] Therefore, as an existing technique for the purpose of restoring the driving operation feeling of a driver dulled by the autonomous driving of a vehicle, for example, there is a vehicle control device described in Patent Document 1. This vehicle control device is mounted on a vehicle in which, when in autonomous driving, the driving operation unit for driving operation is temporarily separated from the drive device that moves the vehicle. Therefore, the driving operation by the driver using the driving operation unit becomes a so-called virtual driving operation that is not reflected in the movement of the vehicle even if performed.

[0004] The vehicle control device described in Patent Document 1 prompts a driver for an action request during the period from when it is notified that the driving authority is transferred from the vehicle to the driver until the driving authority is transferred to the driver, that is, during the so-called manual driving preparation period, accepts the input of a signal based on the driver's action, and switches the vehicle to manual driving when the difference between the value obtained from the signal based on the driver's action and the reference value corresponding to the action request is within an allowable range.

[0005] Prior Art Documents

[0006] Patent Documents

[0007] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2017-1563 Summary of the Invention

[0008] Problems to be Solved by the Invention

[0009] Generally, even when the vehicle is in autonomous driving, it is possible for the driving authority to be transferred from the vehicle to the driver himself / herself. Therefore, the driver does not necessarily perform only actions unrelated to the driving operation of the vehicle, and mostly performs virtual driving operations. In addition, since the vehicle has various driving operations, it is preferable to perform a plurality of different driving operations without omission rather than performing the same driving operation multiple times during the manual driving preparation period in order to restore the driving operation feeling of the driver.

[0010] However, in the vehicle control device described in Patent Document 1, a driving operation is requested of the driver regardless of whether it is a driving operation already performed by the driver before the preparation period for manual driving. Therefore, the vehicle control device may request the driver to perform a driving operation that the driver has already completed before the preparation period for manual driving. In this case, there is the following problem: The driver needs to perform the same driving operation that has already been completed again during the preparation period for manual driving, and other driving operations cannot be performed during this period.

[0011] The present invention has been completed to solve the above problems, and an object thereof is to obtain a notification device and a notification method capable of preventing a request to perform an operation that the driver has already completed before notifying the transfer of driving authority to the driver.

[0012] Means for Solving the Problems

[0013] The notification device of the present invention includes: an information acquisition unit that acquires operation information indicating the operation status of the driver of the moving body from the time point obtained by looking back from the time point when predetermined information has been acquired to the latest time point, the predetermined information indicating that the driving authority is to be transferred by the moving body; and a notification unit that, during the period from when information indicating the transfer of driving authority from the moving body to the driver is notified until the driving authority is transferred to the driver, notifies the operation requested of the driver. The notification unit, based on the operation information, notifies an operation that has not been completed by the driver before the information indicating the transfer of driving authority from the moving body to the driver among the operations requested of the driver during the period until the driving authority is transferred to the driver, and does not notify an operation that has been completed by the driver before the information indicating the transfer of driving authority from the moving body to the driver. When there is a margin time before the driving authority is transferred from the moving body to the driver, the notification unit notifies an operation that has been completed by the driver before the information indicating the transfer of driving authority from the moving body to the driver and no longer satisfies the implementation completion condition. When there is no margin time before the driving authority is transferred from the moving body to the driver, the notification unit requests the moving body to stop moving. The notification unit adds up the expected implementation completion times for various operations that have not been completed by the driver at the moment when the information acquisition unit acquires the predetermined information related to the transfer of driving authority determined for the moving body moving by autonomous driving, and changes the time point obtained by looking back from the driving transfer time point when the driving authority is transferred from the moving body to the driver by the time interval obtained by adding up the implementation completion times to the notification time point of the information indicating the transfer of driving authority from the moving body to the driver and the operation requested of the driver.

[0014] Advantages of the Invention

[0015] According to the notification device of the present invention, during the period from when the driving authority is transferred from the notification to the driver until the driving authority is transferred to the driver, the operations that the driver has not completed among the operations requested of the driver are notified. Thus, the notification device of the present invention can prevent re-requesting the implementation of operations that the driver has already completed before the driving authority is transferred from the notification to the driver. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a block diagram showing the structure of a vehicle control system having the notification device of Embodiment 1.

[0017] Figure 2 is a diagram showing an example of a notification screen for operations requested of the driver.

[0018] Figure 3 is a flowchart showing the notification method of Embodiment 1.

[0019] Figure 4 Figure 4 A is a screen diagram showing an example of a notification screen for requesting the driver to perform an operation of looking straight ahead at the vehicle, Figure 4 B is a screen diagram showing an example of a notification screen for requesting the driver to perform an operation of holding the steering wheel with both hands, Figure 4 C is a screen diagram showing an example of a notification screen for requesting the driver to perform an operation of placing the feet on the accelerator pedal and the brake pedal, Figure 4 D is a screen diagram showing an example of a notification screen for requesting the driver to perform an operation of looking straight ahead at the vehicle and an operation of holding the steering wheel with both hands.

[0020] Figure 5 is a diagram showing a transformation example (1) of the notification content of the operations in Embodiment 1.

[0021] Figure 6 is a diagram showing a transformation example (2) of the notification content of the operations in Embodiment 1.

[0022] Figure 7 is a block diagram showing the hardware structure for implementing the functions of the notification device of Embodiment 1.

[0023] Figure 8 is a block diagram showing the hardware structure for executing the software that implements the functions of the notification device of Embodiment 1.

[0024] Figure 9 is a block diagram showing the structure of the notification device of Embodiment 2.

[0025] Figure 10 ​​​​​​​​​​​is a flowchart showing the notification method of Embodiment 2.

[0026] Figure 11 is a diagram showing a transition example of the notification content of the operation in Embodiment 2.

[0027] Figure 12 is a block diagram showing the structure of the notification device of Embodiment 3.

[0028] Figure 13 is a flowchart showing the notification method of Embodiment 3.

[0029] Figure 14 is a time change diagram showing an outline of the determination process for the timing of transferring driving authority to the driver in Embodiment 3.

[0030] Figure 15 is a block diagram showing the structure of the notification device of Embodiment 4.

[0031] Figure 16 is a flowchart showing the notification method of Embodiment 4.

[0032] Figure 17 is a time change diagram showing an outline of the determination process for the timing of transferring driving authority to the driver in Embodiment 4. Detailed Embodiments

[0033] Embodiment 1

[0034] Figure 1 is a block diagram showing the structure of the vehicle control system 1 having the notification device 10. The vehicle control system 1 is a system mounted on a vehicle such as an automobile that can perform manual driving dependent on the driver and autonomous driving independent of the driver, and controls the operation of the vehicle. In the vehicle control system 1, the vehicle control device 2 controls the engine 4, transmission 5, brake actuator 6, steering wheel actuator 7, turn signal 8, and headlight 9 of the vehicle by using the sensor information detected by the sensor group 3, and performs various controls for realizing the autonomous driving of the vehicle. Various controls include, for example, lane keeping control, cruise control, and parking control. In addition, the vehicle control device 2 determines the transfer of driving authority from the vehicle to the driver and sets a predetermined location for switching to manual driving. The vehicle control device 2 outputs predetermined information indicating the transfer of driving authority from the vehicle to the driver to the notification device 10 a fixed time before the vehicle actually reaches the predetermined location.

[0035] Figure 1 ​​​​​​​The sensor group 3 shown, for example, includes an accelerator pedal sensor 3a, a brake sensor 3b, a steering wheel sensor 3c, an in-vehicle camera 3d, a seat belt sensor 3e, a seat occupancy sensor 3f, an in-vehicle light sensor 3g, a GPS device 3h, a navigation system 3i, an out-of-vehicle camera 3j, and an object detection sensor 3k. The accelerator pedal sensor 3a is a sensor that detects contact with the accelerator pedal and the amount of pedal operation. The accelerator pedal sensor 3a outputs sensor information indicating the situation where the driver places their foot on the accelerator pedal and the amount of pedal operation as the accelerator opening to the vehicle control device 2.

[0036] The brake sensor 3b is a sensor that detects contact with the brake pedal and the amount of pedal operation. The brake sensor 3b outputs sensor information indicating the situation where the driver places their foot on the brake pedal and the amount of pedal operation to the vehicle control device 2. The steering wheel sensor 3c is a sensor that detects contact with the steering wheel and the amount of steering wheel operation. The steering wheel sensor 3c outputs sensor information indicating the situation where the driver holds the steering wheel and the amount of steering wheel operation to the vehicle control device 2.

[0037] The in-vehicle camera 3d is a camera installed in the front of the vehicle interior and captures the field of view including the driver's seat side inside the vehicle. The in-vehicle camera 3d captures the upper body including the face of the driver sitting in the driver's seat and outputs the captured information to the vehicle control device 2. The seat belt sensor 3e is a sensor that detects the wearing and unfastening of the seat belts of each seat in the vehicle. For example, the seat belt sensor 3e outputs sensor information indicating the wearing and unfastening of the seat belt on the seat where the occupancy of the occupant is detected by the seat occupancy sensor 3f to the vehicle control device 2.

[0038] The seat occupancy sensor 3f is a sensor installed in each seat of the vehicle and detects whether an occupant has taken a seat. For example, the seat occupancy sensor 3f uses a pressure detection sensor to detect the pressing force received by the seat when the occupant has taken a seat. The seat occupancy sensor 3f outputs sensor information indicating the occupancy and departure of the occupant to the vehicle control device 2. The in-vehicle light sensor 3g is a sensor installed in the vehicle interior that detects the lighting and extinguishing of the so-called in-vehicle lights, and outputs sensor information indicating the lighting or extinguishing of the in-vehicle lights to the vehicle control device 2.

[0039] The GPS device 3h is a global positioning system that detects the vehicle position using radio waves transmitted by GPS satellites. The GPS device 3h outputs position information indicating the vehicle position to the vehicle control device 2 and the navigation system 3i.

[0040] The navigation system 3i is a system that calculates a recommended route for guiding a vehicle to a destination based on the position information of the vehicle detected by the GPS device 3h and the map information around the vehicle. The navigation system 3i can also communicate with a server to receive external information such as traffic congestion information and no-entry information of the driving destination of the vehicle from the server, and calculate a recommended route reflecting the received external information. In addition, the navigation system 3i can also be a system that sends the position information and destination information of the vehicle to the server, receives the recommended route calculated by the server, and notifies the driver.

[0041] The outside vehicle camera 3j is a camera that captures the outside of the vehicle. The outside vehicle camera 3j is provided, for example, in the front, rear, left, and right sides of the vehicle, and outputs the captured image information of the front, rear, left, and right sides of the vehicle to the vehicle control device 2. The vehicle control device 2 can perform vehicle control considering the presence of objects such as people or vehicles around the vehicle based on the captured image information captured by the outside vehicle camera 3j.

[0042] The object detection sensor 3k is a sensor that detects objects around the vehicle, such as an ultrasonic sensor, a radar sensor, a millimeter-wave radar sensor, or an infrared laser sensor. The object detection sensor 3k calculates the position of the objects existing around the vehicle and the distance between the vehicle and the objects, and outputs the sensor information indicating the calculated position and distance of the objects to the vehicle control device 2. In addition, the vehicle control device 2 can also calculate the position of the objects and the distance between the vehicle and the objects based on the sensor information of the objects detected by the object detection sensor 3k.

[0043] The engine 4 generates power for driving the vehicle. For example, the engine 4 can be an internal combustion engine that uses gasoline as fuel to generate power for rotating the wheels, or a motor that generates power for rotating the wheels through electricity. The engine 4 can operate according to the instruction of the vehicle control device 2. The transmission 5 transmits the power generated by the engine 4 to the wheels of the vehicle. The transmission 5 changes gears according to the instruction of the vehicle control device 2, and thus can change the torque transmitted to the wheels.

[0044] The brake actuator 6 is an actuator that operates the brake, and the brake decelerates the vehicle. For example, the brake actuator 6 can operate the brake automatically regardless of the driver according to the instruction of the vehicle control device 2. The steering wheel actuator 7 is an actuator that controls the operation of the steering wheel and changes the moving direction of the vehicle. For example, the steering wheel actuator 7 can operate the steering wheel automatically regardless of the driver according to the instruction of the vehicle control device 2 and change the moving direction of the vehicle.

[0045] The direction indicator 8 transmits the traveling direction of the vehicle to the outside of the vehicle by emitting light. For example, the direction indicator 8 blinks according to the instruction of the vehicle control device 2 to notify the traveling direction of the vehicle to the outside. The headlamp 9 is a lighting device provided on the front side of the vehicle and irradiates light forward of the vehicle. For example, the headlamps 9 are provided on the left and right sides in front of the vehicle, and by switching the structure of the light guide portion that guides the irradiated light, it is possible to switch between the high beam that irradiates a long distance and the low beam that irradiates a shorter distance than the high beam.

[0046] The notification device 10 notifies the operations requested of the driver during the period from when the driving authority is transferred from the vehicle to the driver until the driving authority is transferred to the driver. For example, the notification device 10 notifies the operations stored in the operation database 11 that the driver has not completed based on the operation information obtained from the vehicle control device 2. The operation information is information indicating the operation status of the driver of the vehicle. In addition, in the operation information, for example, in addition to the operation status of the driver's operation members for driving the vehicle, it also includes information indicating the status of the occupants in the vehicle and information indicating the status of the interior of the vehicle.

[0047] The operation database 11 is a database that stores various operations requested of the driver for the purpose of restoring the driver's driving operation feeling dulled by the vehicle's autonomous driving. The operations requested of the driver include, for example, driving operations using operation members such as steering wheel operation, accelerator operation, brake operation, lighting of the direction indicator 8, or lighting and extinguishing of the headlamp 9. In addition, the operations requested of the driver include, for example, operations specifying the driver's state such as "look ahead at the vehicle", "hold the steering wheel with both hands", "place feet on the accelerator pedal and brake pedal", and "check the sides and rear of the vehicle". Also, the operations requested of the driver include, for example, operations specifying the status of the vehicle's occupants such as "the occupants of the vehicle wear seat belts" and operations specifying the status of the interior of the vehicle such as "turn off the interior lights".

[0048] The display device 12 is a device that notifies information to the driver of the vehicle visually, for example, a head-up display (hereinafter referred to as HUD) or an instrument display. The HUD is a display that projects and displays information onto a combiner provided on the front windshield of the vehicle or in front of the driver's seat inside the vehicle. The instrument display is a display provided in front of the driver's seat inside the vehicle and displays the vehicle's instruments. In addition, the sound output device 13 is a device that notifies information to the driver of the vehicle audibly, for example, a speaker provided inside the vehicle.

[0049] As Figure 1As shown in the figure, the notification device 10 includes an information acquisition unit 101 and a notification unit 102. The notification unit 102 includes a determination unit 1021 and a notification control unit 1022. The information acquisition unit 101 acquires operation information from the vehicle control device 2. The determination unit 1021 determines whether there is an operation stored in the operation database 11 that the driver has not completed based on the operation information of the driver acquired by the information acquisition unit 101. The notification control unit 1022 notifies the driver of the operation determined by the determination unit 1021 that the driver has not completed in a visual or auditory manner.

[0050] Figure 2 FIG. is an example of a notification screen 12A that requests the driver to perform an operation, and the operation requested to the driver is represented by image information. In Figure 2 this case, the image information 14a represents "look ahead at the vehicle". The image information 14b represents "hold the steering wheel with both hands". The image information 14c represents "place feet on the accelerator pedal and the brake pedal". For example, when the above-mentioned predetermined information is output from the vehicle control device 2 in a vehicle in autonomous driving, before the notification unit 102 notifies the driver to transfer the driving authority of the vehicle, when the operations of "look ahead at the vehicle", "hold the steering wheel with both hands", and "place feet on the accelerator pedal and the brake pedal" are not completed, the image information 14a, 14b, and 14c is output to the display device 12. As Figure 2 shown, the display device 12 displays the notification screen 12A including the image information 14a, 14b, and 14c. Since the notification device 10 only notifies the operations that are not completed, it is possible to prevent requesting the driver to perform the operations that the driver has already completed again.

[0051] In addition, before the notification unit 102 notifies the driver to transfer the driving authority, when the operations of "look ahead at the vehicle", "hold the steering wheel with both hands", and "place feet on the accelerator pedal and the brake pedal" are not completed, the sound information representing these operations is output to the sound output device 13. The sound output device 13 outputs sounds such as "Please look ahead at the vehicle", "Please hold the steering wheel with both hands", and "Please place feet on the accelerator pedal and the brake pedal". Even when the notification device 10 notifies the operations that are not completed in an auditory manner, it is possible to prevent requesting the driver to perform the operations that the driver has already completed again, in the same way as when notifying the operations in a visual manner.

[0052] The details of the notification method of Embodiment 1 are described below.

[0053] Figure 3 FIG. is a flowchart showing the notification method of Embodiment 1, showing the operation of the notification device 10. Figure 3A series of processes, for example, start at the following time points: In a vehicle during autonomous driving, the vehicle control device 2 determines to transfer the driving authority from the vehicle to the driver, and outputs predetermined information indicating that the driving authority is to be transferred from the vehicle to the driver to the notification device 10. However, regardless of the output of the predetermined information, the vehicle control device 2 generates operation information indicating the operation status of the driver based on the sensor information sequentially detected by the sensor group 3 and maintains the time series data of the generated operation information.

[0054] The information acquisition unit 101 acquires the operation information from the vehicle control device 2 (step ST1).

[0055] For example, the information acquisition unit 101 acquires the time series data of the operation information from the time point that is a fixed time back from the time when the predetermined information output from the vehicle control device 2 is acquired to the latest operation information. The operation information acquired by the information acquisition unit 101 is output to the notification unit 102.

[0056] Next, the determination unit 1021 determines, based on the operation information acquired by the information acquisition unit 101, whether there is an operation stored in the operation database 11 that the driver has not completed (step ST2). For example, an implementation completion condition is set for each operation stored in the operation database 11. If the operation status indicated by the operation information satisfies the implementation completion condition of the operation corresponding to the operation information, the determination unit 1021 determines that the driver has completed the implementation of the operation, and if it does not satisfy the implementation completion condition, the determination unit 1021 determines that the driver has not completed the implementation.

[0057] When it is determined by the determination unit 1021 that the implementation of all operations requested for the driver to perform has been completed (step ST2: No), the notification control unit 1022 transfers to the process of step ST8. In addition, when it is determined by the determination unit 1021 that there is an operation requested for the driver to perform that the driver has not completed (step ST2: Yes), the notification control unit 1022 notifies the information indicating the transfer of the driving authority from the vehicle to the driver (hereinafter referred to as the driving authority transfer notification) and the operation that the driver has not completed, and does not notify the operation that the driver has completed (step ST3).

[0058] Figure 4 A is a screen diagram showing an example of the notification screen 12A for requesting the driver to perform the operation of looking straight ahead at the vehicle. Figure 4 B is a screen diagram showing an example of the notification screen 12A for requesting the driver to perform the operation of holding the steering wheel with both hands. Figure 4 C is a screen diagram showing an example of the notification screen 12A for requesting the driver to perform the operation of placing the feet on the accelerator pedal and the brake pedal. Figure 4Figure D is an example of a notification screen 12A that shows a notification to the driver to perform an operation of looking straight ahead at the vehicle and an operation of holding the steering wheel with both hands. In Figure 4 A, Figure 4 B, Figure 4 C, and Figure 4 D, the operations requested of the driver are "the operation of looking straight ahead at the vehicle", "the operation of holding the steering wheel with both hands", and "the operation of placing the feet on the accelerator pedal and the brake pedal".

[0059] For example, when the implementation of "the operation of holding the steering wheel with both hands" among the operations requested of the driver has been completed and the implementation of "the operation of placing the feet on the accelerator pedal and the brake pedal" has been completed, the notification control unit 1022 excludes these operations from the notification targets and determines "the operation of looking straight ahead at the vehicle" as the notification target. The notification control unit 1022 outputs the image information 14a indicating "the operation of looking straight ahead at the vehicle" to the display device 12. As Figure 4 shown in A, the display device 12 displays the notification screen 12A including the image information 14a.

[0060] For example, when the implementation of "the operation of looking straight ahead at the vehicle" among the operations requested of the driver has been completed and the implementation of "the operation of placing the feet on the accelerator pedal and the brake pedal" has been completed, the notification control unit 1022 excludes these operations from the notification targets and determines "the operation of holding the steering wheel with both hands" as the notification target. The notification control unit 1022 outputs the image information 14b indicating "the operation of holding the steering wheel with both hands" to the display device 12. As Figure 4 shown in B, the display device 12 displays the notification screen 12A including the image information 14b.

[0061] For example, when the implementation of "the operation of looking straight ahead at the vehicle" among the operations requested of the driver has been completed and the implementation of "the operation of holding the steering wheel with both hands" has been completed, the notification control unit 1022 excludes these operations from the notification targets and determines "the operation of placing the feet on the accelerator pedal and the brake pedal" as the notification target. The notification control unit 1022 outputs the image information 14c indicating "the operation of placing the feet on the accelerator pedal and the brake pedal" to the display device 12. As Figure 4 shown in C, the display device 12 displays the notification screen 12A including the image information 14c.

[0062] For example, when the execution of the operation of "placing feet on the accelerator pedal and the brake pedal" among the operations requested of the driver has been completed, the notification control unit 1022 excludes this operation from the objects to be notified, and determines the operations of "looking straight ahead at the vehicle" and "holding the steering wheel with both hands" as the objects to be notified. The notification control unit 1022 outputs the image information 14a indicating the operation of "looking straight ahead at the vehicle" and the image information 14b indicating the operation of "holding the steering wheel with both hands" to the display device 12. As Figure 4 shown in D, the display device 12 displays a notification screen 12A including the image information 14a and the image information 14b. In addition, when there are operations among the operations requested of the driver that the driver has not completed, as Figure 4 shown in D, these operations can be notified simultaneously, or can be notified sequentially starting from any one of them.

[0063] The driver only needs to perform the operations indicated by the image information displayed on the display device 12 together with the driving authority transfer notification. Therefore, the notification device 10 can prevent the re-request for the operations that the driver has already completed before the driving authority transfer notification. The notification control unit 1022 can also notify the simulated actions of the vehicle resulting from the execution of the notified operations. For example, when the driver operates the steering wheel, if it is manual driving, the notification control unit 1022 causes the display device 12 to display how the vehicle travels. Thus, the driver can more reliably restore the driving operation feeling.

[0064] During the period when the notification control unit 1022 notifies the operations requested of the driver, the information acquisition unit 101 acquires the operation information from the past to the latest from the vehicle control device 2 (step ST4). The determination unit 1021 determines whether the driver has completed the execution of the notified operations based on the operation information acquired by the information acquisition unit 101 (step ST5). For example, if the operation status indicated by the operation information corresponding to the operation notified to the driver satisfies the execution completion condition of the operation, the determination unit 1021 determines that the driver has completed the execution of the operation, and if the execution completion condition is not satisfied, the determination unit 1021 determines that the driver has not completed the execution.

[0065] When it is determined that the execution of the operation notified to the driver has not been completed (step ST5: No), the notification device 10 repeats the processing starting from step ST4. When it is determined that the execution of the operation notified to the driver has been completed (step ST5: Yes), the notification control unit 1022 does not notify the operations that the driver has completed among the notified operations (step ST6).

[0066] Figure 5 It is a diagram showing a transformation example (1) of the notification content of the operations in Embodiment 1. In Figure 5Among them, the operations requested of the driver are the three operations of "operation of looking straight ahead at the vehicle", "operation of holding the steering wheel with both hands", and "operation of placing the feet on the accelerator pedal and the brake pedal".

[0067] When the driver has not completed any one of the operations of "operation of looking straight ahead at the vehicle", "operation of holding the steering wheel with both hands", and "operation of placing the feet on the accelerator pedal and the brake pedal", the notification control unit 1022 outputs the image information 14a, 14b, and 14c to the display device 12. As Figure 5 shown in the upper part of [], the display device 12 displays a notification screen 12A including the image information 14a, 14b, and 14c.

[0068] Then, when the driver completes the implementation of the "operation of looking straight ahead at the vehicle", the notification control unit 1022 excludes the "operation of looking straight ahead at the vehicle" from the notification targets and stops outputting the image information 14a to the display device 12. Thus, as Figure 5 shown in the middle part of [], the display device 12 displays a notification screen 12A including only the image information 14b and 14c.

[0069] Furthermore, when the driver completes the implementation of the "operation of holding the steering wheel with both hands", the notification control unit 1022 excludes the "operation of holding the steering wheel with both hands" from the notification targets and stops outputting the image information 14b to the display device 12. Thus, as Figure 5 shown in the lower part of [], the display device 12 displays a notification screen 12A including only the image information 14c. In the order of the completed implementations, the operations become non-notification, so the driver can restore the driving operation feeling while recognizing that the operations requested by the notification device 10 have been implemented.

[0070] When the determination unit 1021 determines that the implementation of the operation has been completed by the driver, it determines whether the implementation of all the operations notified to the driver has been completed (step ST7). When there is no operation among the operations requested of the driver that the driver has not completed in step ST2 (step ST2: No), or when it is determined that the implementation of all the operations notified to the driver has been completed (step ST7: Yes), the notification control unit 1022 determines that the driver has completed the preparation for manual driving, and thus requests the vehicle control device 2 to transfer the driving authority to the driver (step ST8). The vehicle control device 2 transfers the driving authority to the driver according to the above request from the notification device 10. Since the preparation for the driver's manual driving has been completed, the vehicle can be driven smoothly.

[0071] When the determination unit 1021 determines that there is an operation whose implementation notified to the driver is not completed (step ST7: No), it determines whether there is a remaining time before the scheduled time to transfer the driving authority to the driver (step ST9). When the determination unit 1021 determines that there is a remaining time before transferring the driving authority to the driver (step ST9: Yes), the information acquisition unit 101 acquires the operation information from the past operation information to the latest operation information from the vehicle control device 2 (step ST10).

[0072] Based on the operation information acquired by the information acquisition unit 101, the determination unit 1021 determines whether there is an operation that does not satisfy the implementation completion condition among the operations notified to the driver in step ST3 (step ST11). That is, the determination unit 1021 determines whether there is an operation that does not satisfy the implementation completion condition before the implementation completion of all the operations notified in step ST3 among the operations that the driver has already completed. When there is no operation that does not satisfy the implementation completion condition (step ST11: No), the notification device 10 returns to the process of step ST1 and repeats the Figure 3 series of processes shown.

[0073] When the determination unit 1021 determines that there is an operation that does not satisfy the implementation completion condition before the implementation completion of all the operations notified to the driver (step ST11: Yes), the notification control unit 1022 transfers to the process of step ST3. Thus, the notification control unit 1022 notifies the operation that does not satisfy the implementation completion condition again.

[0074] Figure 6 It is a diagram showing a change example (2) of the notification content of the operation in the first embodiment. In Figure 6 it, the operations requested for the driver to perform are three operations: "operation of looking straight ahead at the vehicle", "operation of holding the steering wheel with both hands", and "operation of placing the feet on the accelerator pedal and the brake pedal". When the driver has not completed the implementation of any of the "operation of looking straight ahead at the vehicle", "operation of holding the steering wheel with both hands", and "operation of placing the feet on the accelerator pedal and the brake pedal", the notification control unit 1022 outputs the image information 14a, 14b, and 14c to the display device 12. As Figure 6 shown in the upper part of, the display device 12 displays the notification screen 12A including the image information 14a, 14b, and 14c.

[0075] Then, when the driver completes the implementation of the "operation of looking straight ahead at the vehicle" and the "operation of holding the steering wheel with both hands", the notification control unit 1022 excludes the "operation of looking straight ahead at the vehicle" and the "operation of holding the steering wheel with both hands" from the notification targets and stops outputting the image information 14a and the image information 14b to the display device 12. Thus, as Figure 6As shown in the middle section of [Figure 0], the display device 12 displays a notification screen 12A that only contains image information 14c.

[0076] When, before the driver completes the implementation of "the operation of placing the feet on the accelerator pedal and the brake pedal", the condition for the completion of the implementation of "the operation of looking straight ahead at the vehicle" is not met, for example, when the driver turns their eyes away from the front of the vehicle, the notification control unit 1022 regards the "operation of looking straight ahead at the vehicle" as the notification target again and outputs the image information 14a to the display device 12. Thus, as Figure 6 shown in the lower section of [Figure 0], the display device 12 redisplaysthe image information 14c and the image information 14a on the notification screen 12A. By notifying the operation for which the implementation completion condition is not met again, the driver can reliably perform the operation requested by the notification device 10.

[0077] When there is an operation among the operations notified to the driver that the driver has not completed, the driver can determine that the preparation for manual driving is not complete. Therefore, when the determination unit 1021 determines that there is no spare time before the predetermined time for transferring the driving authority to the driver (step ST9: No), the notification control unit 1022 requests the vehicle control device 2 to stop the vehicle, so that the driving authority is not transferred to the driver whose preparation for manual driving is not complete (step ST12).

[0078] When the vehicle control device 2 is requested by the notification device 10 to stop the vehicle, for example, based on the position information of the vehicle detected by the GPS device 3h and the map information around the vehicle, it searches for a place where the vehicle can safely stop. Examples of places where the vehicle can safely stop include parking lots in commercial facilities, gas stations, or public parking lots, but in an emergency, it can also be the road shoulder. When the vehicle control device 2 finds a place where the vehicle can safely stop, it automatically parks the vehicle at the place of the search result.

[0079] The hardware structure for implementing the functions of the notification device 10 is as described below.

[0080] The functions of the information acquisition unit 101 and the notification unit 102 included in the notification device 10 are implemented by a processing circuit. That is, the notification device 10 has a processing circuit that executes the Figure 3 processing of steps ST1 to ST12 shown in [Figure 19]. The processing circuit can be dedicated hardware or a CPU (Central Processing Unit) that executes a program stored in a memory.

[0081] Figure 7 is a block diagram showing the hardware structure for implementing the functions of the notification device 10. In addition, Figure 8 is a block diagram showing the hardware structure for executing the software that implements the functions of the notification device 10. In Figure 7 andFigure 8 In this case, the input interface 1000 is, for example, an interface that relays sensor information output from the vehicle control device 2 to the notification device 10 or information indicating an operation output from the operation database 11 to the notification device 10. The output interface 1001 is an interface that relays information indicating an operation requested of the driver, which is output from the notification device 10, to the display device 12 or the sound output device 13.

[0082] When the processing circuit is Figure 7 the processing circuit 1002 of dedicated hardware shown in the figure, the processing circuit 1002 is, for example, a single circuit, a composite circuit, a programmed processor, a parallel-programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a component combining them. The functions of the information acquisition unit 101 and the notification unit 102 included in the notification device 10 may be implemented by different processing circuits, or these functions may be aggregated and implemented by one processing circuit.

[0083] When the processing circuit is Figure 8 the processor 1003 shown in the figure, the functions of the information acquisition unit 101 and the notification unit 102 included in the notification device 10 are implemented by software, firmware, or a combination of software and firmware. In addition, the software or firmware is described as a program and stored in the memory 1004.

[0084] The processor 1003 reads and executes the program stored in the memory 1004 to implement the functions of the information acquisition unit 101 and the notification unit 102 included in the notification device 10. For example, the notification device 10 has a memory 1004 that stores a program for finally executing the processes of steps ST1 to ST12 in the flowchart when executed by the processor 1003. These programs cause a computer to execute the processes or methods of the information acquisition unit 101 and the notification unit 102. The memory 100 may also be a computer-readable storage medium storing a program for causing a computer to function as the information acquisition unit 101 and the notification unit 102. Figure 3 The memory 100 may also be a computer-readable storage medium storing a program for causing a computer to function as the information acquisition unit 101 and the notification unit 102.

[0085] The memory 1004 is, for example, a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (ReadOnly Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically-EPROM), a magnetic disk, a floppy disk, an optical disk, a compact disk, a mini disk, a DVD, etc.

[0086] Regarding the functions of the information acquisition unit 101 and the notification unit 102 included in the notification device 10, part of them can be implemented by dedicated hardware and part by software or firmware. For example, the function of the information acquisition unit 101 is implemented by the processing circuit 1002 which is dedicated hardware, and regarding the notification unit 102, its function is implemented by the processor 1003 reading and executing the program stored in the memory 1004. In this way, the processing circuit can implement the above functions through hardware, software, firmware, or a combination of them.

[0087] As described above, during the period from when the driving authority is transferred from the notification to the driver until the driving authority is transferred to the driver, the notification device 10 of Embodiment 1 notifies the operations that the driver has not completed among the operations requested by the notification to the driver. Thus, the driver only needs to perform the operations notified by the notification device 10. Therefore, the notification device 10 can prevent re-requesting the operations that the driver has already completed before the driving authority is transferred from the notification to the driver.

[0088] Embodiment 2

[0089] Figure 9 It is a block diagram showing the structure of the notification device 10A of Embodiment 2. The notification device 10A differentially notifies the operations stored in the operation database 11 that the driver has not completed and the operations that the driver has completed based on the operation information obtained from the vehicle control device 2. As Figure 9 shown, the notification device 10A has an information acquisition unit 101 and a notification unit 102A. The notification unit 102A has a determination unit 1021 and a notification control unit 1022A. In Figure 9 it, the same reference numerals are assigned to the same structural elements as Figure 1 and the description thereof is omitted.

[0090] The notification control unit 1022 differentially notifies the driver of the operations that have not been completed and the operations that the driver has completed. For example, the notification control unit 1022A overlays and displays, on the notification screen 12A of the display device 12, the image information indicating the completion of implementation and the image information indicating the operations that the driver has completed. The image information indicating the completion of implementation has, for example, a check mark. In addition, the notification control unit 1022A may also make at least one of the display size or display color of the image information indicating the operations that the driver has completed different from the operations that the driver has not completed.

[0091] In addition to the operations that the driver has not completed, the notification control unit 1022A may also output the voice information indicating the operations that the driver has completed to the voice output device 13. For example, the voice output device 13 outputs a voice such as "The implementation of the operation of placing the feet on the accelerator pedal and the brake pedal has been completed, but the implementation of the operations of looking straight ahead at the vehicle and holding the steering wheel with both hands has not been completed." Even when notifying the operations auditorily, the driver can identify the operations that have been completed and the operations that have not been completed.

[0092] The details of the notification method of Embodiment 2 are described below.

[0093] Figure 10 It is a flowchart showing the notification method of Embodiment 2, showing the operation of the notification device 10A. Figure 10 A series of processes of, for example, start at the following time point: in a vehicle in autonomous driving, the vehicle control device 2 determines to transfer the driving authority to the driver, and outputs the predetermined information indicating the predetermined transfer of the driving authority from the vehicle to the driver to the notification device 10A. However, regardless of the output of the predetermined information, the vehicle control device 2 generates the operation information indicating the operation status of the driver based on the sensor information sequentially detected by the sensor group 3 and holds the time series data of the generated operation information.

[0094] The information acquisition unit 101 acquires the operation information from the vehicle control device 2 (step ST1a). For example, the information acquisition unit 101 acquires the time series data of the operation information from the time point of a fixed time back from the moment when the predetermined information output from the vehicle control device 2 is acquired to the latest operation information. The operation information acquired by the information acquisition unit 101 is output to the notification unit 102A.

[0095] Next, the determination unit 1021 determines whether there is an operation stored in the operation database 11 that the driver has not completed based on the operation information acquired by the information acquisition unit 101 (step ST2a). For example, implementation completion conditions are set for each operation stored in the operation database 11. If the operation status indicated by the operation information satisfies the implementation completion condition of the operation corresponding to the operation information, the determination unit 1021 determines that the driver has completed the implementation of the operation. If the implementation completion condition is not satisfied, the determination unit 1021 determines that the driver has not completed the implementation.

[0096] When the determination unit 1021 determines that the implementation of all operations requested for the driver to perform has been completed (step ST2a: No), the notification control unit 1022A is notified to transfer to the process of step ST6a. When the determination unit 1021 determines that there is an operation stored in the operation database 11 that the driver has not completed (step ST2a: Yes), the notification control unit 1022A notifies information indicating the transfer of driving authority from the vehicle to the driver (hereinafter referred to as driving authority transfer notification), and further notifies the driver of the operations that the driver has not completed and the operations that the driver has completed in a differentiated manner (step ST3a).

[0097] Figure 11 FIG. is a diagram showing a transition example of the content of the operation notification in Embodiment 2. In Figure 11 this case, the operations requested for the driver to perform are three operations: "operation of looking straight ahead at the vehicle", "operation of holding the steering wheel with both hands", and "operation of placing the feet on the accelerator pedal and the brake pedal". When the driver has not completed the implementation of any one of the "operation of looking straight ahead at the vehicle", "operation of holding the steering wheel with both hands", and "operation of placing the feet on the accelerator pedal and the brake pedal", the notification control unit 1022A outputs the image information 14a, 14b, and 14c to the display device 12. As Figure 11 shown in the upper part of, the display device 12 displays a notification screen 12A including the image information 14a, 14b, and 14c.

[0098] Then, when the driver has completed the implementation of the "operation of looking straight ahead at the vehicle" and the "operation of placing the feet on the accelerator pedal and the brake pedal", the notification control unit 1022A outputs a check mark image 14 for overlapping display with the image information 14a indicating the "operation of looking straight ahead at the vehicle" and the image information 14b indicating the "operation of placing the feet on the accelerator pedal and the brake pedal" to the display device 12. Thus, as Figure 11 shown in the middle part of, the display device 12 displays a notification screen 12A obtained by overlapping the check mark image 14 on the image information 14a and the image information 14c.

[0099] Also, when the driver completes the implementation of the "operation of holding the steering wheel with both hands", the notification control unit 1022 outputs the check mark image 14 for overlapping display with the "operation of holding the steering wheel with both hands" to the display device 12. Thus, as shown in the lower part of Figure 11 , the display device 12 displays the notification screen 12A obtained by overlapping the check mark image 14 on the image information 14a, 14b, and 14c. Since check marks are assigned to the operations in the order of completion of implementation, the driver can restore the driving operation feeling while recognizing that the operations requested from the notification device 10A have been implemented.

[0100] During the period when the information acquisition unit 101 is notified by the notification control unit 1022A of the operations requested from the driver, the information acquisition unit 101 acquires the operation information from the past to the latest from the vehicle control device 2 (step ST4a). The determination unit 1021 determines whether all the operations notified to the driver have been completed based on the operation information acquired by the information acquisition unit 101 (step ST5a). The processing of steps ST6a to ST10a after that is the same as the processing of steps ST8 to ST12 shown in Figure 3 , so the description is omitted.

[0101] In addition, the functions of the information acquisition unit 101 and the notification unit 102A provided in the notification device 10A are implemented by a processing circuit. That is, the notification device 10A has a processing circuit for executing the processing of steps ST1a to ST10a shown in Figure 10 . The processing circuit can be the processing circuit 1002 of dedicated hardware shown in Figure 7 , or the processor 1003 that executes the program stored in the memory 1004 shown in Figure 8 .

[0102] As described above, in the notification device 10A of the second embodiment, the notification unit 102A differentially notifies the driver of the operations that have not been completed and the operations that the driver has completed. Thus, the driver only needs to implement the operations that have not been completed among the operations notified by the notification device 10A. Therefore, the notification device 10A can prevent the re-request for implementing the operations that the driver has completed before the driving authority is transferred to the driver.

[0103] Embodiment 3

[0104] Figure 12 is a block diagram showing the structure of the notification device 10B of the third embodiment. The notification device 10B changes the notification timing of the transfer of the driving authority from the vehicle to the driver and the request for the driver to implement the operations according to the number of operations that the driver has not completed. As shown in Figure 12As shown, the notification device 10B includes an information acquisition unit 101 and a notification unit 102B. The notification unit 102B includes a determination unit 1021 and a notification control unit 1022B. In Figure 12 the same structural elements as Figure 1 are denoted by the same reference numerals and their description is omitted.

[0105] The notification control unit 1022B changes the notification timing of transferring the driving authority from the vehicle to the driver and requesting the driver to perform an operation according to the number of operations that the driver has not completed. For example, the notification control unit 1022B causes the display device 12 or the sound output device 13 to notify the driving authority transfer notification and the operations that the driver has not completed at a time corresponding to the number of operations that the driver has not completed among the operations stored in the operation database 11.

[0106] The details of the notification method of Embodiment 3 are described below.

[0107] Figure 13 is a flowchart showing the notification method of Embodiment 3 and shows the operation of the notification device 10B. Figure 13 A series of processes of start, for example, at the following time point: in a vehicle in autonomous driving, the vehicle control device 2 determines to transfer the driving authority from the vehicle to the driver and outputs predetermined information indicating that the driving authority is to be transferred from the vehicle to the driver to the notification device 10B. However, regardless of the output of the predetermined information, the vehicle control device 2 generates operation information indicating the operation status of the driver based on the sensor information sequentially detected by the sensor group 3 and holds the time series data of the generated operation information.

[0108] The information acquisition unit 101 acquires the operation information from the vehicle control device 2 (step ST1b). For example, the information acquisition unit 101 acquires the time series data of the operation information from the time point of looking back a fixed time in the past based on the time point when the predetermined information output from the vehicle control device 2 is acquired to the latest operation information. The operation information acquired by the information acquisition unit 101 is output to the notification unit 102B.

[0109] Next, the determination unit 1021 determines whether there is an operation that the driver has not completed among the operations stored in the operation database 11 based on the operation information acquired by the information acquisition unit 101 (step ST2b). For example, an implementation completion condition is set for each operation stored in the operation database 11. If the operation status indicated by the operation information satisfies the implementation completion condition of the operation corresponding to the operation information, the determination unit 1021 determines that the driver has completed the implementation of the operation, and if the implementation completion condition is not satisfied, the determination unit 1021 determines that the driver has not completed the implementation.

[0110] When the determination unit 1021 determines that the execution of all the operations requested of the driver has been completed (step ST2b: No), the notification control unit 1022B transfers to the process of step ST9b. When the determination unit 1021 determines that there is an operation requested of the driver that the driver has not completed (step ST2b: Yes), the notification control unit 1022B changes the notification timing of the driving authority transfer and the operation requested of the driver according to the number of operations that the driver has not completed (step ST3b).

[0111] Figure 14 FIG. 4 is a time change diagram showing an outline of the process for determining the timing of notifying the driver of the transfer of driving authority in the third embodiment. In Figure 14 FIG. 4, the operations requested of the driver are three operations: "operation of looking straight ahead at the vehicle", "operation of holding the steering wheel with both hands", and "operation of placing the feet on the accelerator pedal and the brake pedal".

[0112] When the information acquisition unit 101 acquires predetermined information indicating the transfer of driving authority from the vehicle control device 2 to the driver, if, as shown by "No" in Figure 14 FIG. 4, any one of the "operation of looking straight ahead at the vehicle", "operation of holding the steering wheel with both hands", and "operation of placing the feet on the accelerator pedal and the brake pedal" has not been completed, it is determined that the number of operations that the driver should perform is large.

[0113] When any one of the "operation of looking straight ahead at the vehicle", "operation of holding the steering wheel with both hands", and "operation of placing the feet on the accelerator pedal and the brake pedal" has not been completed, the notification control unit 1022B determines the time A, which has the longest time interval TA from the time D of the transfer of driving authority from the vehicle to the driver, as the time for notifying the driving authority transfer notice and the above operations.

[0114] In addition, when the information acquisition unit 101 acquires the predetermined information from the vehicle control device 2, if, as shown by "Yes" in Figure 14 FIG. 4, the execution of the "operation of looking straight ahead at the vehicle" has been completed, but the execution of the "operation of holding the steering wheel with both hands" and the "operation of placing the feet on the accelerator pedal and the brake pedal" has not been completed, the notification control unit 1022B determines the time B, which has a time interval TB from the time D of the transfer of driving authority from the vehicle to the driver, as the time for notifying the driving authority transfer notice and the above operations. The time interval TB is a time shorter than the time interval TA.

[0115] Furthermore, when the information acquisition unit 101 acquires the predetermined information from the vehicle control device 2, if, as shown by Figure 14As shown by "already", when the implementation of "the operation of looking ahead at the vehicle" and "the operation of placing the feet on the accelerator pedal and the brake pedal" has been completed, but the implementation of "the operation of holding the steering wheel with both hands" has not been completed, the notification control unit 1022B determines the time C at an interval of TC from the time D when the driving authority is transferred from the vehicle to the driver as the time to notify the driving authority transfer notice and the above operations. The time interval TC is shorter than the time intervals TA and TB. Thus, the driver can calmly perform the notified operations.

[0116] In Figure 13 the processing of steps ST4b to ST13b is the same as the processing of steps ST3 to ST12 shown in Figure 3 and thus the description is omitted.

[0117] In addition, the functions of the information acquisition unit 101 and the notification unit 102B provided in the notification device 10B are implemented by a processing circuit. That is, the notification device 10A has a processing circuit for executing the processing of steps ST1c to ST13c shown in Figure 16 . The processing circuit can be the processing circuit 1002 of dedicated hardware shown in Figure 7 or the processor 1003 that executes the program stored in the memory 1004 shown in Figure 8 .

[0118] As described above, in the notification device 10B of Embodiment 3, the notification unit 102B changes the notification timing of transferring the driving authority from the vehicle to the driver and requesting the driver to perform the operations according to the number of operations that the driver has not completed. For example, when the number of operations that the driver has not completed exceeds the threshold, the notification timing is advanced. Thus, the driver can calmly perform the notified operations, and therefore, the notification device 10B can enable the driver to reliably perform the notified operations.

[0119] Embodiment 4

[0120] Figure 15 is a block diagram showing the structure of the notification device 10C of Embodiment 4. The notification device 10C changes the notification timing of transferring the driving authority from the vehicle to the driver and requesting the driver to perform the operations according to the content of the operations that the driver has not completed. As shown in Figure 15 , the notification device 10C has an information acquisition unit 101 and a notification unit 102C. The notification unit 102C has a determination unit 1021 and a notification control unit 1022C. In Figure 15 the same reference numerals are assigned to the same structural elements as in Figure 1 and the description is omitted.

[0121] The notification control unit 1022C changes the notification timing of the transfer of driving authority from the vehicle to the driver and the operation requested of the driver according to the content of the operation that the driver has not completed. For example, for each operation stored in the operation database 11, an implementation completion time expected for the driver to complete the operation according to its content is set. The notification control unit 1022C causes the display device 12 or the sound output device 13 to notify the driving authority transfer notification and the operation that the driver has not completed at the time corresponding to the total implementation completion time of the operation requested of the driver.

[0122] The details of the notification method of Embodiment 4 are described below.

[0123] Figure 16 It is a flowchart showing the notification method of Embodiment 4 and shows the operation of the notification device 10C. Figure 16 A series of processes such as this start, for example, at the following time point: in a vehicle in autonomous driving, the vehicle control device 2 determines to transfer driving authority from the vehicle to the driver and outputs predetermined information indicating that the driving authority is to be transferred from the vehicle to the driver to the notification device 10C. However, regardless of the output of the predetermined information, the vehicle control device 2 generates operation information indicating the operation status of the driver based on the sensor information sequentially detected by the sensor group 3 and holds the time series data of the generated operation information.

[0124] The information acquisition unit 101 acquires the operation information from the vehicle control device 2 (step ST1c). For example, the information acquisition unit 101 acquires the time series data of the operation information from the time of the operation information at a fixed time back from the time when the predetermined information output from the vehicle control device 2 was acquired to the latest operation information. The operation information acquired by the information acquisition unit 101 is output to the notification unit 102C.

[0125] Next, the determination unit 1021 determines whether there is an operation that the driver has not completed among the operations stored in the operation database 11 based on the operation information acquired by the information acquisition unit 101 (step ST2c). For example, implementation completion conditions are set for each operation stored in the operation database 11. If the operation status indicated by the operation information satisfies the implementation completion condition of the operation corresponding to the operation information, the determination unit 1021 determines that the driver has completed the implementation of the operation, and if the implementation completion condition is not satisfied, the determination unit 1021 determines that the driver has not completed the implementation.

[0126] When it is determined by the determination unit 1021 that all the operations requested to be performed by the driver have been completed (step ST2c: No), the notification control unit 1022C transfers to the process of step ST9c. When it is determined by the determination unit 1021 that there is an operation requested to be performed by the driver that the driver has not completed (step ST2c: Yes), the notification control unit 1022C changes the notification timing of the driving authority transfer and the operation requested to be performed by the driver according to the content of the operation that the driver has not completed (step ST3c).

[0127] Figure 17 It is a time change diagram showing an outline of the determination process of the timing for notifying the driver of the transfer of the driving authority in the fourth embodiment. In Figure 14 the operations requested to be performed by the driver are three operations: "operation of looking straight ahead at the vehicle", "operation of holding the steering wheel with both hands", and "operation of placing the feet on the accelerator pedal and the brake pedal".

[0128] When the information acquisition unit 101 acquires predetermined information indicating the transfer of the driving authority from the vehicle control device 2 to the driver, as Figure 17 shown by "No" in, if any of the operations "operation of looking straight ahead at the vehicle", "operation of holding the steering wheel with both hands", and "operation of placing the feet on the accelerator pedal and the brake pedal" has not been completed, the notification control unit 1022C adds the completion time of the operation corresponding to the content of the uncompleted operation. For example, the completion time of the "operation of looking straight ahead at the vehicle" is 15 seconds, the completion time of the "operation of holding the steering wheel with both hands" is 10 seconds, and the completion time of the "operation of placing the feet on the accelerator pedal and the brake pedal" is 5 seconds.

[0129] The notification control unit 1022B determines that the operation that takes the longest time to complete is the one obtained by adding the completion times of the "operation of looking straight ahead at the vehicle", "operation of holding the steering wheel with both hands", and "operation of placing the feet on the accelerator pedal and the brake pedal" (for example, 30 seconds). Therefore, the moment A1 with the longest time interval TA1 from the moment D of the transfer of the driving authority from the vehicle to the driver is determined as the moment for notifying the transfer of the driving authority and the above operations.

[0130] In addition, when the information acquisition unit 101 acquires the predetermined information from the vehicle control device 2, as Figure 17As shown by "already", when the implementation of "operating the steering wheel with both hands" and "operating by placing the feet on the accelerator pedal and the brake pedal" has been completed, but the implementation of "looking straight ahead at the vehicle" has not been completed, the notification control unit 1022B determines, based on the implementation completion time (e.g., 15 seconds) corresponding to the content of the unimplemented "looking straight ahead at the vehicle" operation, the time B1 with a time interval TB1 from the moment D when the driving authority is transferred from the vehicle to the driver as the moment to notify the driving authority transfer notice and the above operations. The time interval TB is shorter than the time interval TA.

[0131] Similarly, when the predetermined information is obtained by the information acquisition unit 101 from the vehicle control device 2, if the implementation of "looking straight ahead at the vehicle" has been completed, but the implementation of "operating the steering wheel with both hands" and "operating by placing the feet on the accelerator pedal and the brake pedal" has not been completed, the notification control unit 1022B adds the implementation completion time (e.g., 10 seconds) corresponding to the content of the unimplemented "operating the steering wheel with both hands" operation and the implementation completion time (e.g., 5 seconds) corresponding to the content of the "operating by placing the feet on the accelerator pedal and the brake pedal" operation. The notification control unit 1022B determines, based on the added value of the implementation completion times of the unimplemented operations (e.g., 15 seconds), the time B1 with a time interval TB1 from the moment D when the driving authority is transferred from the vehicle to the driver as the moment to notify the driving authority transfer notice and the above operations. Thus, the driver can implement the notified operations calmly.

[0132] In Figure 16 the processing of steps ST4c to ST13c is the same as the processing of steps ST3 to ST12 shown in Figure 3 and thus the description is omitted.

[0133] In addition, the functions of the information acquisition unit 101 and the notification unit 102C provided in the notification device 10C are implemented by a processing circuit. That is, the notification device 10C has a processing circuit for executing the processing of steps ST1b to ST13b shown in Figure 16 The processing circuit can be the processing circuit 1002 of dedicated hardware shown in Figure 7 or the processor 1003 that executes the program stored in the memory 1004 shown in Figure 8

[0134] As described above, in the notification device 10C of the fourth embodiment, the notification unit 102C changes the notification timing of the transfer of the driving authority from the vehicle to the driver and the operation requested of the driver according to the content of the operation that the driver has not completed. For example, when the expected time until the driver completes the operation is long, the notification timing is advanced. As a result, the driver can calmly perform the notified operation, and thus the notification device 10C can enable the driver to reliably perform the notified operation.

[0135] In the first to fourth embodiments, the moving body is described as a vehicle, but any moving body that can perform autonomous driving and sometimes transfers the driving authority to the driver is acceptable. For example, the moving body may be a railway vehicle, a ship, or an airplane.

[0136] In addition, combinations of the respective embodiments or modifications of any structural elements of the respective embodiments can be made, or any structural elements can be omitted in each embodiment.

[0137] Industrial Applicability

[0138] The notification device of the present invention can be used, for example, in a vehicle having an autonomous driving function and performing a switch from autonomous driving to manual driving.

[0139] Reference Numeral Explanation

[0140] 1: Vehicle control system; 2: Vehicle control device; 3: Sensor group; 3a: Throttle sensor; 3b: Brake sensor; 3c: Steering wheel sensor; 3d: In-vehicle camera; 3e: Seat belt sensor; 3f: Occupancy sensor; 3g: In-vehicle light sensor; 3h: GPS device; 3i: Navigation system; 3j: Out-of-vehicle camera; 3k: Object detection sensor; 4: Engine; 5: Transmission; 6: Brake actuator; 7: Steering wheel actuator; 8: Direction indicator; 9: Headlight; 10, 10A, 10B, 10C: Notification device; 11: Operation database; 12: Display device; 12A: Notification screen; 13: Sound output device; 14: Check mark image; 14a, 14b, 14c: Image information; 101: Information acquisition unit; 102, 102A, 102B, 102C: Notification unit; 1000: Input interface; 1001: Output interface; 1002: Processing circuit; 1003: Processor; 1004: Memory; 1021: Determination unit; 1022, 1022A, 1022B, 1022C: Notification control unit.

Claims

1. A notification device, characterized in that, The notification device has: an information acquisition unit that acquires operation information indicating the operation status of the driver of the moving body from a point in time retroactively traced back from the point in time when predetermined information has been acquired to the latest point in time, the predetermined information indicating that the driving authority is to be transferred from the moving body; and a notification unit that, during the period from when information indicating the transfer of the driving authority from the moving body to the driver is notified until the driving authority is transferred to the driver, notifies operations requested of the driver, the notification unit, based on the operation information, notifies operations that are requested of the driver during the period until the driving authority is transferred to the driver and that the driver has not completed before the information indicating the transfer of the driving authority from the moving body to the driver is notified, the notification unit does not notify operations that the driver has completed before the information indicating the transfer of the driving authority from the moving body to the driver is notified, when there is a margin of time before the driving authority is transferred from the moving body to the driver, the notification unit notifies operations that the driver has completed before the information indicating the transfer of the driving authority from the moving body to the driver is notified and that no longer satisfy the implementation completion conditions, when there is no margin of time before the driving authority is transferred from the moving body to the driver, the notification unit requests the moving body to stop moving, the notification unit adds up the estimated completion times of operations that the driver has not completed for various contents envisioned at the moment when the predetermined information related to the transfer of the driving authority to the driver is determined for the moving body moving by autonomous driving and acquired by the information acquisition unit, and changes the point in time retroactively traced back by the time interval obtained by adding up the completion times from the driving transfer time when the driving authority is transferred from the moving body to the driver to the notification time of the information indicating the transfer of the driving authority from the moving body to the driver and the operations requested of the driver.

2. The notification device according to claim 1, characterized in that, The operations requested of the driver include operations specifying the occupancy status of the occupants of the moving body.

3. The notification device according to claim 1, characterized in that, The notification unit does not notify operations that the driver has completed among the notified operations.

4. The notification device according to claim 1, characterized in that, The notification unit requests the moving body to transfer the driving authority to the driver when all the notified operations have been completed.

5. The notification device according to claim 1, characterized in that, The notification unit differentially notifies operations that the driver has not completed and operations that the driver has completed.

6. A notification method, characterized in that, In this notification method, an information acquisition unit acquires operation information indicating the operation status of the driver of the moving body from a point in time retroactively traced back from the point in time when predetermined information has been acquired to the latest point in time, the predetermined information indicating that the driving authority is to be transferred from the moving body; a notification unit, during the period from when information indicating the transfer of the driving authority from the moving body to the driver is notified until the driving authority is transferred to the driver, notifies operations requested of the driver, The notification unit notifies, based on the operation information, the operations that are requested to be performed by the driver during the period until the driving authority is transferred to the driver and that have not been completed by the driver before the information indicating that the driving authority is transferred from the moving body to the driver is notified. The notification unit does not notify the operations that have been completed by the driver before the information indicating that the driving authority is transferred from the moving body to the driver is notified. When there is a margin of time before the driving authority is transferred from the moving body to the driver, the notification unit notifies the operations that have been completed by the driver before the information indicating that the driving authority is transferred from the moving body to the driver and that no longer satisfy the operation completion condition. When there is no margin of time before the driving authority is transferred from the moving body to the driver, the notification unit requests the moving body to stop moving. The notification unit adds up the estimated completion times of the operations of various contents that have not been completed by the driver at the moment when the predetermined information related to the transfer of the driving authority to the driver determined for the moving body moving by autonomous driving is acquired by the information acquisition unit, and changes the notification time of the information indicating that the driving authority is transferred from the moving body to the driver and the operations requested to be performed by the driver to the moment obtained by tracing back from the driving transfer moment when the driving authority is transferred from the moving body to the driver by the time interval obtained by adding up the estimated completion times.

Images