Driving assistance device, driving assistance system and driving assistance method

The driving assistance device addresses non-intuitive manual operations in automatic driving systems by using a human-machine interface for intuitive gesture-based control, improving safety and usability for all drivers.

DE112016003473B4Active Publication Date: 2024-12-05PANASONIC AUTOMOTIVE SYST CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
DE112016003473
Authority / Receiving Office
DE · DE
Patent Type
Patents
Current Assignee / Owner
Priority Date
2015-07-31
Filing Date
2016-07-19
Publication Date
2024-12-05
Estimated Expiration
2036-07-19

AI Technical Summary

Technical Problem

Existing automatic driving systems require drivers to manually operate steering, acceleration, and braking, which can lead to incorrect actions due to non-intuitive switch operations and complex user interfaces, especially for inexperienced or impaired drivers.

Method used

A driving assistance device that uses a human-machine interface (MMI) with integrated display and input units, allowing drivers to intuitively instruct specific driving operations through gesture recognition and intuitive graphical interactions, such as dragging and dropping vehicle icons onto road signs, to control the vehicle's automatic driving functions.

Benefits of technology

Enables drivers to conveniently and safely instruct specific driving operations without physically manipulating steering or pedals, reducing the likelihood of errors and enhancing user experience, particularly for those unfamiliar with or impaired in driving.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

Driving assistance device, which includes the following: an image output unit that outputs image information, comprising a vehicle object representing a vehicle and a sign object representing sign information, to a display unit; an actuation signal input unit, when a receive mode of the driving assistance device is set as an activated receive mode in which an actuation can be received from a user, which receives an actuation signal from the user to move the vehicle object to a position of the sign object in an image displayed on the display unit, or to move the sign object to a position of the vehicle object; and a command output unit that outputs an automatic driving control command, corresponding to the display content in the sign object when the actuation signal is received, to an automatic driving control unit that controls the automatic driving, and wherein the receive mode is changed from the activated receive mode to a deactivated receive mode, in which an actuation by the user cannot be received, when the automatic driving control command has been determined and a first time has elapsed after receipt of the actuation signal from the user.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present invention relates to a technique that supports a driver's driving operation instruction to a vehicle during automatic driving. GENERAL STATE OF THE ART

[0002] DE 10 2011 087 459 A1 relates to a driver assistance system for a motor vehicle, comprising a detection means for detecting the surroundings of the motor vehicle and a user interface operable by the driver of the motor vehicle. The driver assistance system is designed such that the driver can specify one or more objects in the surroundings detected by the detection means via the user interface, whereupon one or more functions of the driver assistance system are executed for the specified object(s).

[0003] DE 10 2013 213 039 A1 relates to an assistance system for supporting the driver in controlling a motor vehicle with sensors for recording measured variables relating to the driving situation, a processing device for determining parameters of a driving situation and control parameters of an assistance control from the recorded measured variables, an output device for displaying the determined parameters of the driving situation, characterized by an input device for recording additional inputs for correcting and / or supplementing the determined parameters of the driving situation and / or the control parameters of the assistance control displayed by the output device.

[0004] A method according to DE 10 2011 121 746 A1 for navigating a vehicle using a graphic projection display comprises monitoring a navigation state graphic representing a navigation intent displayed on the graphic projection display, monitoring a user input indicated on a portion of the graphic projection display, initiating a user-defined navigation command based on the monitored navigation state graphic and the monitored user input, and operating the vehicle according to the user-defined navigation command.

[0005] A method according to DE 10 2011 122 541 A1 for selecting a feature in a view in front of a vehicle comprises monitoring the view in front of the vehicle visible through a graphic projection display, monitoring a user input to the graphic projection display, determining the selected feature based on the monitored view and the monitored user input, determining an aligned graphic representing the selected feature for display on the graphic projection display, and displaying the aligned graphic on the graphic projection display.

[0006] In recent years, developments in automated driving have advanced. In 2013, the National Highway Traffic Safety Administration (NHTSA) defined levels of autonomy for automated driving: no autonomy (Level 0), specific function autonomy (Level 1), complex function autonomy (Level 2), semi-automatic driving (Level 3), and fully automated driving (Level 4). Level 1 is a driving assistance system that automatically performs one of acceleration, deceleration, and steering, and Level 2 is a driving assistance system that automatically performs two or more of acceleration, deceleration, and steering in coordination. In all cases, a driver is required to participate in the driving operation.Level 4 autonomy is a fully automated driving system that automatically performs all acceleration, deceleration, and steering functions, and therefore no driver is involved in driving. Level 3 autonomy is a semi-automatic driving system that automatically performs all acceleration, deceleration, and steering functions, but a driver's input is performed when necessary.

[0007] A form in which a driver does not operate an existing driving operation unit, such as a steering wheel, an accelerator pedal, or the like, but instructs a vehicle for a specific driving operation, such as lane changing, overtaking, continuing to drive, or the like, by issuing a command to a vehicle is considered a form of automatic driving. In this form, a user interface with fewer erroneous operations is required. A technique that extracts a sign from the original image and selectively displays the sign based on a traveling direction or the like is known (see, for example, PTL 1). Citation listPatent literature

[0008] PTL 1: Unexamined Japanese Patent Publication No. JP 2014 - 74 627 A SUMMARY OF THE INVENTION

[0009] To achieve the above-mentioned objects, a driving support device according to one aspect of the present disclosure comprises the features defined in claim 1.

[0010] As another aspect of the present invention, an aspect is also effective in which an expression of the present invention is converted into an apparatus, a system, a method, a program, a recording medium in which a program is recorded, and an automatic vehicle equipped therewith by arbitrarily combining the foregoing components.

[0011] According to the present disclosure, a driver can intuitively and conveniently command a vehicle for a specific driving operation. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram illustrating a configuration of a vehicle according to an embodiment of the present invention. Fig. 2 is a diagram showing an example of a basic sequence of detection unit, automatic driving control device, MMI control device, display unit and input unit in Fig. 1 illustrates. Fig. 3 is a diagram showing an example of a basic flowchart for explaining the processing of the MMI control device in Fig. 1 illustrates. Fig. 4 is a flowchart for explaining the update processing of a reception mode. Fig. Fig. 5 is a flowchart illustrating an example of the determination processing in a case where a gesture operation for instructing a specific operation using a road sign is executed in step S9 of Fig. 3 is entered. Fig. 6 is a flowchart illustrating a processing example of outputting a command corresponding to the content of the road sign by a gesture operation. Fig. 7A and Fig. 7B are diagrams illustrating an example of issuing a path change instruction command or a route change instruction command using a destination sign at an intersection. Fig. 8A and Fig. 8B are diagrams illustrating an example of issuing a path change instruction command or a route change instruction command using a lane indication sign. Fig. 9A and Fig. 9B are diagrams illustrating an example of issuing a setting instruction command for setting an intermediate destination using an exit sign of a toll road. Fig. 10A and Fig. 10B are diagrams illustrating another example of issuing a setting instruction command for setting an intermediate destination using an exit sign of a toll road. Fig. 11A and Fig. 11B are diagrams illustrating an example of issuing a destination setting instruction command for setting a destination using a parking lot sign. Fig. 12A to Fig. 12C are diagrams illustrating another example of issuing a route change instruction command using a destination sign at an intersection. Fig. 13A and Fig. 13B are diagrams illustrating an example of issuing a lane change instruction command and a path change instruction command using a sign on a lane. DESCRIPTION OF EMBODIMENTS

[0012] Before describing an embodiment of the present invention, problems in the prior art will be briefly described.

[0013] In a case where automatic driving is applied to a vehicle, the driver needs some time to confirm a sign and then instruct a path using separate means (a steering, an accelerator pedal, or the like) located at a different location. In a design where each switch is designed for a specific driving operation, such as lane changing, overtaking, continuing to drive, or the like, there is a possibility of incorrect operation if a driver operates the switch without looking at his / her hand while checking a situation around the vehicle. Furthermore, the correspondence between a switch operation and automatic driving control is not intuitive, and selecting an instruction from among many switches or selecting a changed function with reduced switches according to a situation is a complex task.There is a problem in that a wide range of users, including a person who has not previously driven, a person who wishes to continue driving although the person's driving ability has diminished, or the like, cannot use an automatic vehicle without training.

[0014] The present disclosure has been made in consideration of such a situation, and an object thereof is to provide a technique that enables a driver to intuitively and conveniently instruct a vehicle to perform a specific driving operation.

[0015] Embodiments of the present invention will now be described with reference to the drawings. Fig. 1 is a block diagram illustrating a configuration of vehicle 1 according to an embodiment of the present invention, illustrating a configuration related to automatic driving. Vehicle 1 (a self-drive vehicle) with an automatic driving mode includes the driving support device (human-machine interface (MMI) controller) 10, the automatic driving control device (automatic driving control device) 20, the display device 30, the detection unit 40, and the driving operation unit 50.

[0016] The display device 30 includes a display unit 31 and an input unit 32. The display device 30 may be a head unit such as a car navigation system, a display audio, or the like, a portable terminal device such as a smartphone, a tablet, or the like, or a dedicated console terminal device.

[0017] The display unit 31 is a liquid crystal display, an organic electroluminescent (EL) display, or a head-up display (HUD). The input unit 32 is a user interface that receives input from a user. The display unit 31 and the input unit 32 may be an integrated touch panel. For example, a proximity display panel that can detect the approach of a hand on a touch panel or touchpad, or a position of a finger due to a hover operation, or one that receives gesture input at a position away from the display unit by a predetermined distance, may be used. The input unit 32 may include an input device such as a mouse, a tablet pen, a trackball, or the like to support gesture input. A stylus that emits visible light or infrared light may also be used.

[0018] The display unit 31 and the input unit 32 may be physically separated from each other instead of being an integrated touch-sensitive display panel. For example, the input unit 32 includes a sensor such as a camera or the like, and may be a contactless type input device that enables gesture operation input in the air. For example, an operation method is considered that starts the drag with a gesture by pointing a finger at a target and closing a thumb and index finger, and ends the drag with a gesture by separating a thumb and index finger.

[0019] The driving support device 10 and the display device 30 may be connected to each other by wired connection such as a dedicated line, a Controller Area Network (CAN), or the like, or may be connected to each other by wired connection or wireless connection such as USB, Ethernet (registered trademark), Wi-Fi (registered trademark), Bluetooth (registered trademark), or the like.

[0020] The detection unit 40 includes the position information acquisition unit 41, the sensor 42, the speed information acquisition unit 43, and the map information acquisition unit 44. The position information acquisition unit 41 acquires a current position of the vehicle 1 from a global positioning system (GPS) receiver. Sensor 42 is a general term for various sensors for detecting a situation outside the vehicle and a state of the vehicle 1. As a sensor for detecting a situation outside the vehicle, for example, a camera, a millimeter-wave radar, a laser imaging detection and ranging (LIDAR), a temperature sensor, a pressure sensor, a humidity sensor, an illumination sensor, or the like is mounted.As an external situation, a road situation in which the own vehicle is traveling or an environmental situation including weather, and a situation in the vicinity of the own vehicle, such as a traveling position or traveling state of another vehicle traveling in the vicinity of the own vehicle, are considered. Any information outside the vehicle that can be detected by the sensor may be considered. As a sensor for detecting a state of a vehicle 1, for example, an acceleration sensor, a gyro sensor, a geomagnetic sensor, an inclination sensor, or the like is mounted. The speed information acquisition unit 43 acquires the current speed of the vehicle 1 from a vehicle speed sensor. The map information acquisition unit 44 acquires map information in the vicinity of the current position of the vehicle 1 from a map database.The map database may be recorded on a recording medium in the vehicle 1 or may be downloaded from a map server via a network in use.

[0021] The detection unit 40 and the automatic driving control device 20 are connected to each other by wired communication, such as a dedicated line, a USB, an Ethernet (registered trademark), a Controller Area Network (CAN), or the like. A configuration may be adopted in which data acquired and detected by the detection unit 40 is directly output from the detection unit 40 to the driving support device 10.

[0022] The driving operation unit 50 includes the steering 51, the brake pedal 52, the accelerator pedal 53, and the turn signal switch 54. In an automatic driving mode according to the present embodiment, acceleration, deceleration, steering, and the flashing of the turn signal are a target of automatic control by the automatic driving control device 20. In Fig. 1 depicts an operating unit in a case where these controls are performed manually. Information indicating that the driving operating unit 50 is slightly moved manually by a driver can be output to the driving assistance device 10.

[0023] The steering 51 is an actuating unit for steering the vehicle. When the steering 51 is turned by a driver, the vehicle's direction of travel is controlled by a steering actuator. The steering actuator can be controlled electronically by an electronic steering control unit (ECU).

[0024] The brake pedal 52 is an actuating unit for decelerating the vehicle 1. When the brake pedal 52 is pressed by a driver, the vehicle is decelerated via a brake actuator. The brake actuator can be electronically controlled by a brake ECU.

[0025] The accelerator pedal 53 is an operating unit for accelerating the vehicle 1. When the accelerator pedal 53 is depressed by a driver, an engine speed and / or motor speed is controlled via an accelerator pedal actuator. In a pure gasoline car, the engine speed is controlled. In a pure electric car, the engine speed is controlled. In a hybrid car, both the engine speed and the engine speed are controlled. The accelerator pedal actuator can be electronically controlled by at least one of an engine ECU and an engine ECU.

[0026] The turn signal switch 54 is an operating unit for flashing a turn signal to indicate the vehicle's outward path. When the turn signal switch 54 is turned on / off by a driver, the turn signal is turned on / off via a turn signal control device. The turn signal control device includes a driver circuit, such as a relay, that controls the power supply of a turn signal lamp or the like.

[0027] Each of the steering ECU, the brake ECU, the engine ECU, the engine ECU, the turn signal controller, and the automatic driving control device 20 are connected to each other via wired communication such as a CAN, a dedicated line, or the like. Each of the steering ECU, the brake ECU, the engine ECU, the engine ECU, and the turn signal controller sends a status signal indicating the status of each of a steering system, a brake, an engine, an engine, and a turn signal lamp to the automatic driving control device 20.

[0028] In the automatic driving mode, each of the steering ECU, the brake ECU, the engine ECU, and the motor ECU drives the corresponding actuator according to a control signal supplied from the automatic driving control device 20. In the manual driving mode, a configuration in which a command is directly transmitted from each of the steering 51, the brake pedal 52, and the accelerator pedal 53 to the corresponding actuator in a mechanical manner may be adopted, or a configuration in which electronic control intervenes via the corresponding ECU may be adopted. The turn signal control device turns the turn signal lamp on / off according to the control signal supplied from the automatic driving control device 20 or an instruction signal from the turn signal switch 54.

[0029] The automatic driving control device 20 is an automatic driving control device having an automatic driving control function, and includes the control unit 21, the storage unit 22, and the input / output unit 23. The configuration of the control unit 21 can be implemented by cooperation between hardware resources and software resources, or only hardware resources. A processor, a read-only memory (ROM), a random access memory (RAM), and other large-scale integrated (LSI) circuits can be used as the hardware resources, and an operating system, an application, and a program such as firmware or the like can be used as the software resources. The storage unit 22 includes a non-volatile recording medium such as a flash memory or the like.The input-output unit 23 performs various communication controls according to different communication formats.

[0030] The control unit 21 calculates a control value for controlling an automatic control target, such as a traveling direction of the vehicle 1, by applying various parameter values ​​collected by the detection unit 40 and various ECUs to an automatic driving algorithm. The control unit 21 transmits the calculated control value to the ECU or controller of each control target. In the present embodiment, the control value is transmitted to the steering ECU, the brake ECU, the engine ECU, and the turn signal controller. In the case of an electric vehicle or a hybrid car, the control value is transmitted to the engine ECU instead of the engine ECU or in addition to the engine ECU.

[0031] The driving assistance device 10 is a human-machine interface (MMI) control device for performing an interface function between the vehicle 1 and a driver, and includes the determination unit 11, the image generation unit 12, the instruction unit 13, and the input / output unit 14. The determination unit 11, the image generation unit 12, and the instruction unit 13 can be implemented by cooperation between hardware resources and software resources, or only hardware resources. A processor, a ROM, a RAM, and other large-scale integrated (LSI) circuits can be used as the hardware resources, and an operating system, an application, and a program such as firmware or the like can be used as the software resources. The input / output unit 14 performs various communication controls according to various communication formats.The input / output unit 14 includes the image output unit 14a, the operation signal input unit 14b, the command output unit 14c, and the vehicle information input unit 14d. The image output unit 14a outputs an image generated by the image generation unit 12 to the display unit 31. The operation signal input unit 14b receives an operation signal input from the input unit 32 by an operation of a driver, a passenger, or a user outside the vehicle and outputs the operation signal to the determination unit 11. The command output unit 14c outputs the command instructed by the instruction unit 13 to the automatic driving control device 20.The vehicle information input unit 14d receives detection data acquired by the detection unit 40 or vehicle information generated by the automatic driving control device 20 and outputs the detection data or the vehicle information to the image generation unit 12.

[0032] The automatic driving control device 20 and the MMI control device 10 are directly connected to each other via a signal line. A configuration can be used in which the automatic driving control device 20 and the MMI control device 10 are connected to each other via a CAN. A configuration can also be used in which the automatic driving control device 20 and the MMI control device 10 are integrated into a single control device.

[0033] Fig. Fig. 2 is a diagram showing an example of a basic sequence of detection unit 40, automatic driving control device 20, MMI control device 10, display unit 31 and input unit 32 in Fig. 1. The detection unit 40 detects the position information of the own vehicle, the road travel information of the own vehicle, and the surrounding information of the own vehicle, and outputs the detected information to the automatic driving control device 20 (P1). The automatic driving control device 20 outputs the position information of the own vehicle acquired by the detection unit 40, the road travel information of the own vehicle, and the surrounding information of the own vehicle to the MMI control device 10 (P2). The MMI control device 10 generates a schematic diagram including the own vehicle and the surrounding situation of the own vehicle based on the information acquired by the automatic driving control device 20 (P3).The MMI controller 10 outputs the generated schematic diagram to the display device 30, and the schematic diagram is displayed on the display unit 31 (P4).

[0034] A user viewing the schematic diagram displayed on the display unit 31 comes into contact with the input unit 32 (P5). The display unit 31 outputs coordinate data at a position where contact with the MMI controller 10 is detected (P6). The MMI controller 10 determines a type of command based on the coordinate data acquired by the display device 30 (P7). The MMI controller 10 receives additional input until a certain period of time has elapsed (P8 to P12). After determining the command, the MMI controller 10 regenerates a schematic diagram indicating that the command is instructed (P8). The MMI controller 10 outputs the regenerated schematic diagram to the display device 30, and the regenerated schematic diagram is displayed on the display unit 31 (P9).In a case where the command corresponding to the gesture operation by a user's touch is not present, the MMI controller 10 generates a schematic diagram including an error message and outputs the schematic diagram to the display device 30, and the schematic diagram is displayed on the display unit 31.

[0035] When a user viewing the schematic diagram indicating that the command is instructed comes into contact with the input unit 32 (P10), the display unit 31 outputs the coordinate data at a position where contact with the MMI controller 10 is detected (P11). The MMI controller 10 performs additional command processing based on the coordinate data acquired by the display device 30 (P12). In a case where there is no input of a new command in the additional command processing (P12), the MMI controller 10 outputs a command determined in P7 to the automatic driving controller 20 (P13 and P14). In a case where a new command is input in the additional command processing (P12), the MMI controller 10 outputs a new command to the automatic driving controller 20.In a case where the new command being input is a cancellation command, the MMI controller 10 cancels the output of the command. The processing of overwriting and canceling the original command due to a new command can be performed by the automatic driving controller 20. In this case, after the command determination processing in P7 and P12, the MMI controller 10 sends the command to the automatic driving controller 20 and performs the overwriting and cancellation processing according to an internal state of the automatic driving controller 20.

[0036] The detection unit 40 periodically detects the position information of the host vehicle, the road travel information of the host vehicle, and the surrounding information of the host vehicle and outputs the information to the automatic driving controller 20 (P15). Based on the information, the automatic driving controller 20 determines whether the control instructed by the command issued from the MMI controller 10 is executable (P16). If it is determined that the control is executable, the automatic driving controller 20 outputs a control start notification to the MMI controller 10 (P17). Upon receiving the control start notification, the MMI controller 10 again generates a schematic diagram including a message indicating that the control is being executed (P18).The MMI control device 10 outputs the regenerated schematic diagram to the display device 30, and the regenerated schematic diagram is displayed on the display unit 31 (P19). Although not illustrated, the automatic driving control device 20 calculates a control value for controlling the driving operation unit 50, which executes the output command by applying various parameter values ​​collected by the detection unit 40 or various ECUs to an automatic driving algorithm, and transmits the control value to the ECU or controller of each control target. The control value is a specific value for automatically controlling the control target, such as a traveling direction of the vehicle 1 or the like. The driving operation unit 50 is operated based on the specific control value.When a predetermined control value or the detection data acquired by the detection unit 40 is a predetermined value (within a predetermined range) and when the automatic travel control device 20 determines that a condition of the issued command is satisfied, the travel operation unit 50 determines that the execution of the command is completed.

[0037] When a control completion notification is received from the automatic driving control device 20, the MMI control device 10 generates a schematic diagram including a message indicating that control is completed and outputs the generated schematic diagram to the display device 30. During a period during which no operation is received from a user, the MMI control device 10 generates a schematic diagram including a message indicating that no control is received and outputs the generated schematic diagram to the display device 30.

[0038] Fig. 3 is a diagram showing an example of a basic flowchart for explaining the processing of the MMI controller 10 in Fig. 1. The determination unit 11 of the MMI controller 10 checks whether a driving mode is an automatic driving mode or a manual driving mode (S1). In the manual driving mode (N in S2), the process ends. In the case of the automatic driving mode (Y in S2), the processing is performed as follows.

[0039] Sensor information input from the detection unit 40 to the automatic driving controller 20 is updated at any time (S3). The image generation unit 12 of the MMI controller 10 generates a schematic diagram including the own vehicle and the situation around the own vehicle based on the own vehicle position information, the own vehicle road travel information, and the own vehicle surrounding information input by the automatic driving controller 20, and displays the generated schematic diagram on the display unit 31 (S4). The determination unit 11 checks whether or not a reception mode is an enabled reception mode in which an operation from a user can be received or a disabled reception mode in which no operation from a user can be received (S5).In a case where the reception mode is the disabled reception mode (N in S6), the process ends. In a case where the reception mode is the enabled reception mode (Y in S6), the determination unit 11 determines whether or not there is contact by a user with the input unit 32 (S7). In a case where there is no contact by a user with the input unit 32 (N in S8), the processing for determining whether the predetermined time has elapsed (S12) is performed, which will be described below. In a case where there is contact by a user with the input unit 32 (Y in S8), the determination unit 11 determines a control command according to a gesture operation input by a user (S9). The details of the control command determination processing are described below.

[0040] In a case where the control command determined in step S9 is not a cancellation command (N in S10), the image generation unit 12 displays on the display unit 31 that the command is instructed (S11). If a predetermined time has elapsed after the control command was determined (Y in S12), in a case where a control command was determined in step S9 (Y in S13), the disabling of operation reception is displayed on the display unit 31 (S14), the determination unit 11 updates the reception mode from the enabled reception mode to the disabled reception mode (S15), and the instruction unit 13 outputs the determined control command to the automatic driving control device 20 (S16). Until a predetermined time has elapsed (N in S12), the process proceeds to step S3.

[0041] In step S10, if the determined control command is a cancellation command (Y in S10), the cancellation is displayed (S110) and the process is terminated. In step S13, if no control command is determined in step S9 (N in S13), an input error is displayed (S111) and the process is terminated. The automatic driving control device 20 periodically detects the position information of the own vehicle, the road travel information of the own vehicle, and the surrounding information of the own vehicle from the detection unit 40. Since the situation around the own vehicle is constantly changing, after the control command is output to the automatic driving control device 20, there is a case where it is determined that the control command cannot be executed. For example, after a subsequent instruction, there is a case where other cars come between the own vehicle and another vehicle.If it is determined that the control command can be executed by the automatic travel control device 20 (Y in S17), the image generation unit 12 displays on the display unit 31 that control is being performed (S18) and starts counting by activating a timer (S19). If it is determined that the control cannot be executed by the automatic travel control device 20 (N in S17), the image generation unit 12 displays a "control disabled" error on the display unit 31 (S112).

[0042] Fig. 4 is a flowchart for explaining the processing for updating a reception mode. When a count value of the timer reaches a set value (for example, 10 seconds) (Y in S113), the determination unit 11 of the MMI controller 10 updates the reception mode from the disabled reception mode to the enabled reception mode (S114). The count value of the timer can be changed according to the situation surrounding the host vehicle. When a notification indicating the completion of control is received from the automatic driving controller 20, or when it is determined based on the behavior of the vehicle 1 that the control according to the control command is completed, the determination unit 11 can update the reception mode from the disabled reception mode to the enabled reception mode.

[0043] In this embodiment, an example in which a control command for instructing a specific operation is issued using a road sign will be described below. A user inputs a gesture operation for instructing a specific operation into the input unit 32 using a road sign. A specific example of the gesture operation will be described below.

[0044] Fig. 5 is a flowchart illustrating an example of the determination processing in a case where a gesture operation for instructing a specific operation using a road sign in step S9 of Fig. 3 is input. The determination unit 11 of the MMI controller 10 determines whether or not a self-vehicle mark is present at a contact start position (S9a). In a case where no self-vehicle mark is present at the contact start position (N in S9a), the determination unit 11 determines that the gesture operation is a control command of another operation in which the road sign is not used (S9b). In a case where the self-vehicle mark is present at the contact start position (Y in S9a), the generation unit 12 draws an enabled storage area in the schematic diagram and displays the area on the display unit 31 (S9c). At this time, the generation unit 12 generates an enabled storage area for a road sign in the schematic diagram.Information about the activated area for placing the road sign is acquired from a local map information database prepared in advance or a map information database on a network. The coordinates of an object, such as a place name, an arrow, or the like in the sign, can be acquired using image recognition technology based on a real-time image of the road sign captured by a vehicle-mounted camera.

[0045] The determination unit 11 receives a touch event generated in the input unit 32 (S9d) and determines the type of the touch event (S9e). If the type of touch event is motion (motion in S9e), the generation unit 12 draws a predicted trajectory / route candidate of the vehicle 1 in the schematic diagram, and the predicted trajectory / route candidate of the vehicle 1 is displayed on the display unit 31 (S9f).

[0046] In a case where the type of touch event is a contact end (Contact End in S9e), the determination unit 11 determines whether or not a contact end position exists in the road sign area (S9g). In a case where the contact end position does not exist in the road sign area (N in S9g), the determination unit 11 determines that the gesture operation is a control command of another operation in which the road sign is not used (S9h). In a case where the contact end position exists in the road sign area (Y in S9g), the determination unit 11 determines whether the road sign area is a lane sign area or a slow lane sign area (S9i).In a case where the area of ​​the road sign is an area of ​​a lane sign or a slow lane sign (J in S9i), the determination unit 11 determines whether the lane of a lane sign or a slow lane sign is a right-turn only lane or a left-turn only lane (S9j). In a case where the lane is not a right-turn only lane or a left-turn only lane (N in S9j), the determination unit 11 determines that the gesture operation is a lane change instruction command for instructing the lane change to the lane (S9k).In a case where the lane is a right-turn-only lane or a left-turn-only lane (J in S9j), the determination unit 11 determines that the gesture operation is a lane change instruction command for instructing the lane change to the lane and a right-turn instruction or a left-turn instruction (S9k).

[0047] In step S9i, in a case where the filed area is not an area of ​​a lane indication sign or a slow lane sign (N in S9i), the determination unit 11 determines whether or not the filed area is an area where an arrow indicating a direction or a path is displayed (S9m). In a case where the filed area is an area where an arrow indicating a direction or a path is displayed (Y in S9m), the determination unit 11 determines that the gesture operation is a direction change instruction command for instructing the direction change to the direction (S9n). In a case where the filed area is not an area where an arrow indicating a direction or a path is displayed (N in S9m), the determination unit 11 determines whether the filed area is an area where a place name indicating a city or a region is displayed (S9o).In a case where the filed area is an area where a place name indicating a city or region is displayed (Y in S90), the determination unit 11 acquires a traveling direction corresponding to the place name from the map information database (S9p). The determination unit 11 determines that the gesture operation is a traveling direction change instruction command for instructing the traveling direction change to the acquired traveling direction (S9q).

[0048] In step S90, in a case where the filed area is not an area where a place name indicating a city or a region is displayed (N in S9o), the determination unit 11 determines whether or not the filed area is an area where a street name is displayed (S9r). In a case where the filed area is an area where a street name is displayed (Y in S9r), the generation unit 12 generates an image (e.g., a menu or a dialog) or a voice requesting a selection of one of uptown and outtown, and notifies a user of the image or voice (S9s). In a case where uptown is selected by a user (Y in S9t), the determination unit 11 determines that the gesture operation is a direction change instruction command for instructing the direction change to uptown (S9u).In a case where outbound is selected by a user (N in S9t), the determination unit 11 determines that the gesture operation is a direction change instruction command for instructing the direction change to outbound (S9v).

[0049] In step S9r, in a case where the filed area is not an area where a road name is displayed (N in S9r), the determination unit 11 determines whether the filed area is an area where a sign of an entrance, an exit, a parking lot, or a rest area of ​​a toll road is displayed or not (S9w). In a case where the filed area is an area where a sign of an entrance, an exit, a parking lot, or a rest area of ​​a toll road is displayed (Y in S9w), the determination unit 11 determines that the gesture operation is a setting command for setting the entrance, exit, parking lot, or rest area of ​​the toll road as an intermediate destination (S9x).

[0050] In a case where the dropped area is not an area where a sign of an entrance, an exit, a parking lot, or a rest area of ​​a toll road is displayed (N in S9w), the determination unit 11 determines whether or not the dropped area is an area where a sign of a parking lot, a shelter, or an emergency telephone is displayed (S9y). In a case where the dropped area is an area where a sign of a parking lot, a shelter, or an emergency telephone is displayed (J in S9y), the determination unit 11 determines that the gesture operation is a setting instruction command for setting the parking lot, shelter, or shoulder where the emergency telephone is located to a destination (S9z).In step S9y, in a case where the filed area is not a sign of a parking lot, a shelter, or an emergency telephone (N in S9y), the generating unit 12 displays an error message on the display unit 31 (S9zz).

[0051] The following describes a specific example of gesture operation using a road sign. In the following examples, it is assumed that a touch-sensitive display panel is used in which the display unit 31 and the input unit 32 are integrated.

[0052] Fig. 6 is a flowchart illustrating a processing example for issuing a command according to the content of the road sign by a gesture operation. The determination unit 11 of the MMI controller 10 receives a touch event (DOWN) from the touch panel (S50). The touch event (DOWN) is an event representing a contact state change on the touch panel from a non-contact state to a contact state by a finger or a stylus. The determination unit 11 determines whether a coordinate detected by the touch event (DOWN) exists within a display area of ​​the own vehicle icon (S51).In a case where the coordinate detected by the touch event (DOWN) is outside the display range of the own vehicle icon (N in S51), it is determined that the gesture operation is not an operation instruction corresponding to the content of a road sign, and the process is terminated.

[0053] In a case where the coordinate detected by the touch event (DOWN) is present in the display area of ​​the own vehicle icon (Y in S51), the determination unit 11 receives a touch event (MOVE) from the touch panel (S52). The touch event (MOVE) is an event representing a change from a contact state at a certain point on the touch panel to a contact state at another point on the touch panel by a finger or a stylus. Then, the determination unit 11 receives a touch event (UP) from the touch panel (S53). The touch event (UP) is an event representing a contact state change on the touch panel from a contact state to a non-contact state by a finger or a stylus.

[0054] The determination unit 11 determines whether a coordinate detected by the touch event (HIGH) exists within a display area of ​​the road sign (S54). If the coordinate detected by the touch event (HIGH) exists within the display area of ​​the road sign (Y in S54), the instruction unit 13 outputs a command according to the content of the road sign to the automatic driving controller 20 (S55). If the coordinate detected by the touch event (HIGH) exists outside the display area of ​​the road sign (N in S54), it is determined that the gesture operation is not an operation instruction corresponding to the content of the road sign, and the process is terminated.

[0055] Fig. 7A and Fig. 7B are diagrams illustrating an example of issuing a path change instruction command or a route change instruction command using a destination sign at an intersection. In the schematic diagram shown in Fig. As illustrated in FIG. 7A, a vehicle icon, a road, and a destination sign S1 are displayed. As the display forms of the own vehicle, the road, and the sign, various display forms are considered. A real photographed image may be used, and a fine computer graphic image or an animation image may be used. The display of the own vehicle is not limited to an icon, and the own vehicle may be displayed as a simpler mark or character, or may be displayed as a real photographed image. In other words, there is no problem as long as the own vehicle is displayed in any display form as an object on a screen.

[0056] In a case where a driver wishes to change the path of his own vehicle to any path indicated on a destination sign S1 at an intersection, as in Fig. As illustrated in Figure 7A, a driver drags the icon of his own vehicle V1 and places it, as shown in Fig. 7B illustrates that the icon of the own vehicle V1 is projected onto a location named "Machida" N1 of a desired destination in the destination sign S1. Accordingly, a route change instruction command is issued to change the route of the own vehicle to a route of the desired destination. Alternatively, a route change instruction command is issued to change the route of the own vehicle to a route in which the desired destination is set as an intermediate destination or a destination.

[0057] Fig. 8A and Fig. 8B are diagrams illustrating an example of issuing a path change instruction command or a route change instruction command using a lane indication sign. In a case where a driver wants to change the path of the own vehicle to any path indicated on the lane indication signs S1a and S2b, as shown in Fig. 8A, a driver drags the icon of his own vehicle V1, as shown in Fig. 8B, and places the icon of the own vehicle V1 on a desired lane sign "Sagamihara" S2. Accordingly, a route change instruction command is issued to change the route of the own vehicle to a route of the desired destination. Alternatively, a route change instruction command is issued to change the route of the own vehicle to a route in which the desired destination is set as an intermediate destination or a destination.

[0058] Fig. 9A and Fig. 9B are diagrams illustrating an example of issuing a setting instruction command for setting an intermediate destination using an exit sign of a toll road. In a case where a driver wishes to exit a toll road via the nearest exit, as shown in Fig. As illustrated in Figure 9A, a driver drags the icon of his own vehicle V1 and places it as shown in Fig. 9B illustrates the icon of the own vehicle V1 on the "Yokohama Machida" exit sign starting at S3. Accordingly, an intermediate destination setting instruction command is issued to set the exit to an intermediate destination. In a case where the own vehicle's route is changed by setting an intermediate destination, a route change instruction command is issued.

[0059] Fig. 10A and Fig. 10B are diagrams illustrating another example of issuing a setting instruction command for setting an intermediate destination using an exit sign of a toll road. In a case where a driver wishes to exit a toll road via a desired exit, as shown in Fig. 10A, a driver drags the icon of his own vehicle V1 and places it as shown in Fig. 10B illustrates the icon of the own vehicle V1 on a desired exit sign "Tomei Kawasaki" S4c among the multiple exit signs S4a, S4b, and S4c. Accordingly, an intermediate destination setting instruction command for setting the exit to an intermediate destination is issued. In a case where the route of the own vehicle is changed by setting an intermediate destination, a route change instruction command is issued.

[0060] Fig. 11A and Fig. 11B are diagrams illustrating an example of issuing a destination setting instruction command for setting a destination using a parking space sign. In a case where a driver wants to park his or her own vehicle in a parking space, as shown in Fig. 11A, a driver drags the icon of his own vehicle V1 and places it, as shown in Fig. 11B illustrates the icon V1 of the own vehicle on a parking space sign starting at S5. Accordingly, a destination setting instruction command for setting the parking space to a destination is issued. When the destination is set, the automatic driving controller 20 redirects the original destination of the own vehicle to a new destination.

[0061] Fig. 12A to Fig. 12C are diagrams illustrating another example of issuing a route change instruction command using a destination sign at an intersection. In a case where a driver wishes to change the route of the own vehicle to any route indicated on a destination sign S6 at an intersection, as shown in Fig. 12A, a driver drags the icon of the own vehicle V1, as shown in Fig. 12B, the icon of the own vehicle V1 is placed on a road named "Gyoko road" R1 on the destination sign S6. When the icon of the own vehicle V1 is placed on the road name as shown in Fig. As illustrated in FIG. 12C, a destination display window R2 of the upstream direction (toward Machida) and the outstream direction (toward Zama) of the road is displayed. The upstream or outstream direction of the destination display window R2 is touched by a driver. Accordingly, a path change instruction command is issued to change the path of the own vehicle to the path in the touched direction.

[0062] Fig. 13A and Fig. 13B are diagrams illustrating an example of issuing a lane change instruction command and a path change instruction command using a sign on a lane. In a case where a driver wants to turn right with his or her own vehicle, as in Fig. 13A, the driver drags the icon of the own vehicle V1 and places it, as shown in Fig.13B illustrates the presence of the host vehicle icon V1 on a sign M1 of a right-turn lane only on a lane. Accordingly, a lane change instruction command for instructing the lane change to the right-turn lane only and a right-turn instruction command are issued.

[0063] In the above example, during a period during which the own-vehicle icon can be dragged and dropped, the generation unit 12 may keep the original image (own-vehicle icon before movement) in the schematic diagram and delete the original image when the own-vehicle icon is dropped. During the period during which the own-vehicle icon is dragged and dropped, the generation unit 12 may draw a movement trajectory of the own-vehicle icon using a dotted line in the schematic diagram. During the period during which the own-vehicle icon is dragged and dropped, the generation unit 12 may invert the color of the road, and its color may return to the original color of the road when the own-vehicle icon is dropped.

[0064] During the period during which the own vehicle icon is dragged and dropped, if a dropping-disabled area (an oncoming lane or the like) exists or if an operation (lane change or the like) is disabled, the generation unit 12 may change the color of the own vehicle icon (reverse, fade, or the like). When the own vehicle icon is dropped, the own vehicle icon may return to its original position, and an error message such as "operation disabled" may be displayed. Examples of a case where an operation is disabled include the approach of a following vehicle, a lane change prohibited area, exceeding a speed limit, or the like.

[0065] During the period during which the own vehicle icon is dragged and dropped when operation is disabled, the generation unit 12 may change (invert, fade, or the like) a color of a background such as a road or the like. When operation is enabled, the color of the background returns to its original color. During the period during which the own vehicle icon is dragged and dropped, the generation unit 12 changes a color of the dropping-disabled area. During the period during which the own vehicle icon is dragged and dropped, if a dropping-disabled area exists or if operation is disabled, the generation unit 12 performs a notification using an error sound or vibration.

[0066] When the start of an operation is disabled, if the user starts dragging the own vehicle icon, the generation unit 12 may change the color of the own vehicle icon. The drag operation (movement of the own vehicle icon) may be disabled. An error message such as "Operation disabled" may be displayed. When the start of an operation is disabled, if the user starts dragging the own vehicle icon, the generation unit 12 may change the color of the background, such as a road or the like. When an operation is enabled, the background color returns to its original color.When the dragging of the own vehicle icon is started in a state where the dragging operation (movement of the own vehicle icon) is disabled, in a case where the start of an operation is disabled, the generation unit 12 performs the notification using an error sound or vibration.

[0067] During the period during which the own-vehicle icon is put down and the processing according to the control command is completed, the generation unit 12 may display a state of the own-vehicle as a ghost image and display a movement trajectory of the own-vehicle. During the period during which the own-vehicle icon is put down and the own-vehicle completes the processing according to the control command, the generation unit 12 may change a display state of the own-vehicle icon (lighting up, color change, size, position, or the like). During the period during which the own-vehicle icon is put down and the processing according to the control command is completed, the determination unit 11 may queue a next instruction as an additional operation (reserve a control to be performed after the current control is completed).

[0068] As described above, according to the present embodiment, it is possible to transmit the content of various operations to the automatic driving control device 20 by gesture operation by moving an icon displayed on the touch panel. The gesture operation of the icon is a simple operation, thus freeing a driver from a conventional driving operation such as turning the steering wheel 51, pressing the accelerator pedal 53, or the like. For example, it is possible to easily instruct the operation according to the content of the road sign by displaying a schematic diagram including a road sign and moving the icon of the own vehicle to the road sign.A driver can simultaneously confirm the surrounding situation and an operation instruction on the touch-sensitive display panel, thus preventing the driver's line of sight from moving. Accordingly, it is possible to reduce the possibility of incorrect operation and perform safer driving. To issue the control command, an operation for connecting the own vehicle icon and the sign, which is different from the operation, may be used. Although a gesture operation corresponding to a control command has been described as a drag-and-drop operation or the like, a touch-and-touch operation may be used. Preferably, a predetermined gesture operation is used, but an operation that can be customized by a user may be used.Further, in order to recognize the corresponding relationship between a gesture operation and a control command, a comment, an icon, or an arrow may be displayed by the display unit 31, or a guidance display or a voice guidance may be provided by the display unit 31.

[0069] The present invention has been described above based on the embodiments. These embodiments were presented purely as examples, and those skilled in the art will understand that the embodiments can be modified in various forms by combining the corresponding elements or processing methods, and the modification examples are included within the scope of the present invention.

[0070] For example, although an example is assumed in which an MMI controller 10 is implemented by a dedicated LSI, functions of the MMI controller 10 may be implemented using a central processing unit in a portable device such as a smartphone or a tablet used as a display device 30. In this case, a portable device used as a display device 30 and the automatic driving controller 20 are directly connected to each other. Functions of the MMI controller 10 may be implemented by a central processing unit in a head unit such as a car navigation device, a display audio device, or the like. A dedicated LSI on which the MMI controller 10 is mounted may be included in a head unit.

[0071] In the aforementioned embodiment, an example is described in which a command according to the display content in a sign is issued by dragging and dropping the own vehicle icon onto a sign object indicating a road sign. In this regard, a command according to the display content in a sign can also be issued in a similar manner by dragging and dropping the sign object onto the own vehicle icon.

[0072] The embodiments can be specified by the following elements. [Item 1]

[0073] Driving assistance device (10) comprising: an image output unit (14a) that outputs an image including an own vehicle object representing an own vehicle and a sign object representing a sign to a display unit (31); an operation signal input unit (14b) receiving an operation signal from a user to move the own vehicle object to the position of the sign object in the image displayed on the display unit (31) or to move the sign object to the position of the own vehicle object; and a command output unit (14c) that outputs a command according to the display content in the sign object when the operation signal is received to an automatic driving control unit (20) that controls the automatic driving.

[0074] Here, “sign” is a road traffic sign and can be an information sign or a traffic sign.

[0075] In this case, a user can perform an operation according to the content of the sign intuitively and conveniently. [Item 2]

[0076] The driving assistance device (10) according to item 1, wherein the command output unit (14c) outputs a command for setting a location indicated by the display content in the sign object to which the own vehicle object is moved as a destination or an intermediate destination to the automatic driving control unit (20).

[0077] In this case, a user can perform an operation to set a destination or an intermediate destination intuitively and conveniently. [Item 3]

[0078] The driving assistance device (10) according to item 1 or 2, wherein the command output unit (14c) outputs a command for instructing the own vehicle to travel in a traffic lane in the sign object to which the own vehicle object is moved, to the automatic driving control unit (20).

[0079] Here, “driving” includes right turns, left turns, U-turns and lane changes.

[0080] In this case, a user can perform an operation to instruct right turn, left turn, U-turn and lane change intuitively and conveniently. [Item 4]

[0081] The driving assistance device (10) according to item 3, wherein the operation signal input unit (14b) receives an operation signal from the user for moving the own vehicle object in the image displayed on the display unit (31) to a traffic lane or a road name in the sign object, the image output unit (14a) outputs an image representing a selection object for selecting an inbound lane or an outbound lane from the traffic lane to the display unit (31).

[0082] In this case, when changing the path of the own vehicle to another road, a user can intuitively and conveniently instruct the own vehicle to drive in an inbound or outbound lane. [Item 5]

[0083] Driving assistance system (10, 30) comprising: a display device (30) that displays an image; and a driving support device (10) which outputs an image to the display device (30), wherein the driving assistance device (10) comprises: an image output unit (14a) that outputs an image including an own vehicle object representing an own vehicle and a sign object representing a sign to the display unit (30); an operation signal input unit (14b) receiving an operation from a user to move the own vehicle object to the position of the sign object in the image displayed on the display device (30) or to move the sign object to the position of the own vehicle object; and a command output unit (14c) that outputs a command according to the display content in the sign object when the operation is received to an automatic driving control unit (20) that controls the automatic driving.

[0084] In this case, a user can perform an operation according to the content of the sign intuitively and conveniently. [Item 6]

[0085] Driving assistance procedure that includes: a step of outputting an image including an own vehicle object representing an own vehicle and a sign object representing a road sign to a display unit (31); a step of receiving a user's operation to move the own vehicle object to the position of the sign object in the image displayed on the display unit (31) or to move the sign object to the position of the own vehicle object; and a step of outputting a command according to the display content in the sign object when the operation is received to an automatic driving control unit (20) that controls the automatic driving.

[0086] In this case, a user can perform an operation according to the content of the sign intuitively and conveniently. [Item 7]

[0087] Driving assistance program and recording medium in which the driving assistance program is recorded, wherein the driving assistance program causes a computer to perform the following: Processing the output of an image comprising an own vehicle object representing an own vehicle and a sign object representing a sign to a display unit (31); Processing the receipt of an operation by a user to move the own vehicle object to the position of the sign object in the image displayed on the display unit (31) or to move the sign object to the position of the own vehicle object; and Processing the output of a command according to the display content in the sign object when the operation is received to an automatic driving control unit (20) that controls the automatic driving.

[0088] In this case, a user can perform an operation according to the content of the sign intuitively and conveniently. [Item 8]

[0089] Automatic vehicle (1) comprising: an image output unit (14a) that outputs an image including an own vehicle object representing an own vehicle and a sign object representing a sign to a display unit (31); an operation signal input unit (14b) receiving an operation signal from a user to move the own vehicle object to the position of the sign object in the image displayed on the display unit (31) or to move the sign object to the position of the own vehicle object; a command output unit (14c) which outputs a command according to the display content in the sign object when the operation is received; and an automatic driving control unit (20) which executes the output command.

[0090] In this case, a user can perform an operation according to the content of the sign intuitively and conveniently.

[0091] The present invention can be used in a vehicle equipped with an automatic driving mode. REFERENCE SYMBOLS IN THE DRAWINGS 1 vehicle 10 MMI control unit 11 Determination unit 12 generation units 13 Instruction unit 14 Input-output unit 20 automatic driving control device 21 Control unit 22 storage unit 23 Input-output unit 30 Display device 31 Display unit 32 input unit 40 detection unit 41 Position information acquisition unit 42 Sensor 43 Speed ​​information acquisition unit 44 Card information acquisition unit 50 Driving control unit 51 Steering 52 Brake pedal 53 Accelerator pedal 54 indicator light switch

Claims

[1] Driving assistance device comprising: an image output unit that outputs image information including a vehicle object representing a vehicle and a sign object representing sign information to a display unit; an operation signal input unit, when a reception mode of the driving support device is set as an activated reception mode in which an operation from a user can be received, which receives an operation signal from the user to move the vehicle object to a position of the sign object in an image displayed on the display unit or to move the sign object to a position of the vehicle object; and a command output unit that outputs an automatic driving control command corresponding to display content in the sign object when the operation signal is received to an automatic driving control unit that controls the automatic driving, and wherein the reception mode is changed from the activated reception mode to a deactivated reception mode in which a user operation cannot be received when the automatic travel control command has been determined and a first time has elapsed after receiving the user operation signal. [2] The driving support device according to claim 1, wherein the command output unit outputs a command for setting a location indicated by the display content in the sign object to which the vehicle object is moved as a destination or an intermediate destination to the automatic driving control unit. [3] The vehicle support device according to claim 1 or 2, wherein the command output unit outputs a command for instructing the vehicle to enter a traffic lane in the sign object to which the vehicle object is moved to the automatic travel control unit. [4] The driving assistance device according to claim 3, wherein, when the operation signal input unit receives an operation signal of the user for moving the vehicle object in the image displayed on the display unit to a traffic lane or a road name in the sign object, the image output unit outputs image information including a selection object for selecting an inbound or outbound traffic lane to the display unit. [5] Driving assistance system comprising: a display device that displays an image; and a driving support device that outputs image information to the display device, wherein the driving assistance device comprises: an image output unit that outputs image information including a vehicle object representing a vehicle and a sign object representing sign information to the display device; an operation signal input unit that, when a reception mode of the driving support device is set as an enabled reception mode in which an operation from a user can be received, receives an operation signal from the user to move the vehicle object to a position of the sign object in an image displayed on the display device or to move the sign object to a position of the vehicle object; and a command output unit that outputs an automatic driving control command corresponding to display content in the sign object when the operation signal is received to an automatic driving control unit that controls the automatic driving, and wherein the reception mode is changed from the activated reception mode to a deactivated reception mode in which a user operation cannot be received when the automatic travel control command has been determined and a first time has elapsed after receiving the user operation signal. [6] Driving assistance method performed by a driving assistance device, comprising: a step of outputting an image including a vehicle object representing a vehicle and a sign object representing sign information to a display unit; a step of receiving an operation signal from a user to move the vehicle object to a position of the sign object in the image displayed on the display unit, or to move the sign object to a position of the vehicle object when a reception mode of the driving support device is set as an enabled reception mode in which an operation from a user can be received; and a step of outputting an automatic driving control command corresponding to display content in the sign object when the operation signal is received to an automatic driving control unit that controls the automatic driving, wherein the reception mode is changed from the activated reception mode to a deactivated reception mode in which a user operation cannot be received when the automatic travel control command has been determined and a first time has elapsed after receiving the user operation signal. [7] The driving support system according to claim 5, wherein the command output unit outputs a command for setting a location indicated by the display content in the sign object to which the vehicle object is moved as a destination or an intermediate destination to the automatic driving control unit. [8] The vehicle support system according to claim 5, wherein the command output unit outputs a command for instructing the vehicle to enter a traffic lane in the sign object to which the vehicle object is moved to the automatic travel control unit. [9] The driving assistance system according to claim 8, wherein, when the operation signal input unit receives an operation signal of the user for moving the vehicle object in the image displayed on the display unit to a traffic lane or a road name in the sign object, the image output unit outputs image information including a selection object for selecting an inbound or outbound traffic lane to the display unit. [10] The driving support method according to claim 6, wherein outputting the automatic driving control command comprises outputting a command for setting a location indicated by the display content in the sign object to which the vehicle object is moved as a destination or an intermediate destination to the automatic driving control unit. [11] The vehicle support method according to claim 6, wherein outputting the automatic driving control command comprises outputting a command for instructing the vehicle to enter a traffic lane in the sign object to which the vehicle object is moved, to the automatic driving control unit. [12] The driving assistance method according to claim 11, wherein, when a user's operation signal for moving the vehicle object in the image displayed on the display unit to a traffic lane or a road name in the sign object is received, outputting the image comprises outputting image information including a selection object for selecting an inbound or outbound traffic lane to the display unit. [13] Driving assistance device according to claim 1, wherein: the reception mode is changed from the deactivated reception mode to the activated reception mode when (i) a second time has elapsed after the activated receive mode has been changed to the deactivated receive mode, (ii) a message indicating completion of control is received by the automatic driving control unit, or (iii) based on the behavior of the vehicle, it is determined that the control according to the control command is completed. [14] Driving assistance system according to claim 5, wherein: the reception mode is changed from the deactivated reception mode to the activated reception mode when (i) a second time has elapsed after the activated receive mode has been changed to the deactivated receive mode, (ii) a message indicating completion of control is received by the automatic driving control unit, or (iii) based on the behavior of the vehicle, it is determined that the control according to the control command is completed [15] Driving assistance method according to claim 6, wherein: the reception mode is changed from the deactivated reception mode to the activated reception mode when (i) a second time has elapsed after the activated receive mode has been changed to the deactivated receive mode, (ii) a message indicating completion of control is received by the automatic driving control unit, or (iii) based on the behavior of the vehicle, it is determined that the control according to the control command is completed.

Citation Information

Patent Citations

  • Driver assistance system for motor car, has user interface the enables the driver to execute several functions of assistance system as function of specified objects in environment

    DE102011087459A1

  • Method for navigating an autonomous vehicle using a graphical projection display

    DE102011121746A1

  • Virtual cursor for street scene object selection on a full windshield head-up display

    DE102011122541A1

  • Assistance system and assistance procedures to support the control of a motor vehicle

    DE102013213039A1

  • Navigation system for vehicle

    JP2014074627A