Driver assistance device, driver assistance system and driver assistance procedure

Abstract

Driver assistance device (10), comprising: an image output unit which outputs to a display unit (31) an image which contains its own vehicle object, which represents its own vehicle (1), and another vehicle object, which represents another vehicle in front of its own vehicle object; an operating signal input unit that receives an operation from a user to change a positional relationship between its own vehicle object and the other vehicle object in the image displayed on the display unit (31), when a receive mode of the driving assistance device (10) is set to a receive enable mode in which an operation from the user can be received; and a command output unit which issues a command to an automatic driving control unit (21) which controls the automatic driving, instructing the vehicle to overtake the other vehicle when the positional relationship between the own vehicle object and the other vehicle object is changed by the user's operation of the control signal input unit (32) such that the own vehicle object is placed in front of the other vehicle object, whereby the receive mode is switched from receive enable mode to receive block mode in which a user operation cannot be received when the command is determined and an initial time has elapsed since the receipt of the user's control signal.

Description

Field of invention

[0001] The present disclosure relates to a technology that supports a driver's instruction to a vehicle during automated driving. Description of the state of the art

[0002] In recent years, developments in automated driving have progressed significantly. Automation levels for automated driving, established in 2013 by the National Highway Traffic Safety Administration (NHTSA), are divided into: no automation (Level 0), automation of certain functions (Level 1), automation of complex functions (Level 2), semi-automated driving (Level 3), and fully automated driving (Level 4). Level 1 is a driver assistance system that automatically performs one of the following actions: accelerating, decelerating, and steering. Level 2 is a driver assistance system that automatically coordinates two or more of these actions. In all cases, a driver must be involved in driving. Automation Level 4 is a fully automated driving system that automatically performs all operations of accelerating, decelerating, and steering, and therefore a driver is not involved in driving.Automation level 3 is a semi-fully automatic driving system that automatically performs all operations of acceleration, deceleration and steering, but driving operation is carried out by a driver when necessary.

[0003] One form of automated driving is defined as a system in which the driver does not operate any existing driving controls, such as steering, accelerator pedal, or the like, but instead instructs a vehicle to perform a specific driving maneuver, such as changing lanes, overtaking, following another vehicle, or similar actions, by issuing a command. This type of system requires a user interface that minimizes the risk of accidental input. Patent specification 2 discloses an interface for inputting driving maneuvers into a control system for at least semi-autonomous control of a motor vehicle, comprising a touch-sensitive panel with a recording unit for recording a temporal sequence of touches, an analysis unit for categorizing the recorded sequences according to predefined pattern categories, and a command unit for outputting driving commands assigned to the pattern categories to the control system. Patent specification 3 discloses a method for assisting a driver of a motor vehicle during a driving maneuver, in which, in a first step, the environment of the motor vehicle is detected and a two-dimensional representation of the environment of the motor vehicle is displayed, in a second step a desired end position of the vehicle is entered by the driver, wherein the end position is marked in the two-dimensional representation, and in a final step a trajectory to reach the end position is determined and instructions are issued to the driver to follow the trajectory or an automatic driving maneuver is carried out in which the vehicle is maneuvered along the trajectory to the end position. Patent specification 4 discloses an assistance system for supporting the driver in controlling a motor vehicle, comprising sensors for acquiring measurements relating to the driving situation, a processing unit for determining parameters of a driving situation and control parameters of an assistance control system from the acquired measurements, an output unit for displaying the determined parameters of the driving situation, and an input unit for acquiring additional inputs for correcting and / or supplementing the determined parameters of the driving situation and / or the control parameters of the assistance control system displayed by the output unit. This design enables the driver to conveniently correct interpretations and control actions of the assistance system. Patent specification 5 discloses a method for navigating an autonomously driving vehicle using a graphic projection display, the method comprising: monitoring an autonomous driving system; determining a navigation intention of the autonomously driving vehicle based on the monitored autonomous driving system; determining a navigation state graph representing a future navigation maneuver that the autonomously driving vehicle will perform autonomously based on the navigation intention; dynamically aligning a location of the navigation state graph on the graphic projection display according to a driving scene of the autonomously driving vehicle; and displaying the navigation state graph on the graphic projection display; monitoring a user input specified on a section of the graphic projection display corresponding to a location where the navigation state graph is displayed;Operating the navigation state graph based on monitored user input; initiating a user-defined navigation command according to the operated navigation state graph, which changes the future navigation maneuver; determining a navigation command graph that represents the user-defined navigation command; dynamically aligning a location of the navigation command graph on the graphic projection display according to a driving scene of the autonomous vehicle; displaying the navigation command graph on the graphic projection display; and operating the autonomous vehicle according to the user-defined navigation command. Bibliography Patent literature Patent specification 1: Unexamined Japanese patent publication JP H10 - 105 885 A Patent specification 2: German patent application DE 10 2011 078 910 A1 Patent specification 3: German patent application DE 10 2010 030 463 A1 Patent specification 4: German patent application DE 10 2013 213 039 A1 Patent specification 5: German patent application DE 10 2011 121 746 A1 Summary of the invention

[0004] When simply representing road conditions, a driver must, when instructing the vehicle to perform a specific driving maneuver, sequentially operate each dedicated switch, turn a steering wheel, or press an accelerator pedal while observing the road conditions. Furthermore, the driver's line of sight moves, and there is a possibility of accidental operation. Additionally, in a design where each switch is assigned to a specific driving maneuver, such as lane changing, overtaking, following, or the like, the correspondence between a switch operation and the automatic driving control is not intuitive. Moreover, selecting an instruction from among many switches or choosing a modified function with a reduced number of switches, depending on the situation, is a complex task.For this reason, one problem is that a large group of users, including a person who has never driven before, a person who wants to continue driving, even if the person's ability to drive has been impaired, or the like, cannot use an automatically driving vehicle without training.

[0005] The present disclosure was prepared in view of such a situation, and one of its purposes is to create a technique that enables a driver to intuitively and conveniently instruct a vehicle to perform a specific driving operation.

[0006] The problems described above are solved by a driver assistance device according to claim 1, a driver assistance method according to claim 4, and a driver assistance system according to claim 5. Preferred embodiments are defined by the dependent claims.

[0007] The implementation of the present disclosure can also be achieved in a device, a system, a method, a program, a data carrier on which a program is recorded, and a vehicle equipped therewith by any combination of the above components.

[0008] According to the present disclosure, a driver can intuitively and conveniently instruct a vehicle to perform a specific driving maneuver. Brief description of the drawing Fig. Figure 1 is a block diagram representing an arrangement of a vehicle according to an embodiment of the present disclosure. Fig. Figure 2 is a diagram that shows an example of a basic sequence of sensor unit, automatic driving control, MMI control, display unit and input unit in Fig. 1 represents. Fig. Figure 3 is a diagram that shows an example of a basic flowchart to explain how the MMI control processes in Fig. 1 represents. Fig. Figure 4 is a flowchart to explain the update processing of a receive mode. Fig. 5 is a flowchart that illustrates an example of determination processing in a case where a gesture control is used to instruct an overtaking maneuver in step S9 of Fig. 3 is entered. Fig. Figure 6 is a flowchart that represents a first processing example of issuing an overtaking instruction command via a gesture control. Fig. 7 is a diagram that shows an example of gesture control according to the flowchart of Fig. 6 represents. Fig. Figure 8 is a flowchart that shows a second processing example of issuing an overtake instruction command via a gesture control. Fig. 9 is a diagram that shows an example of gesture control according to the flowchart of Fig. 8 represents. Fig. Figure 10 is a flowchart that represents a third processing example of issuing an overtake instruction command via a gesture control. Fig. 11 is a diagram that shows an example of gesture control according to the flowchart of Fig. 10 represents. Fig. Figure 12 is a flowchart that represents a fourth processing example of issuing an overtake instruction command by means of a gesture control. Fig. 13 is a diagram that shows an example of gesture control according to the flowchart of Fig. 12 represents. Fig. Figure 14 is a flowchart that illustrates a fifth processing example of issuing an overtake instruction command by means of a gesture control. Fig. 15 is a diagram that shows an example of gesture control according to the flowchart of Fig. 14 represents. Fig. Figure 16 is a flowchart that represents a sixth processing example of issuing an overtake instruction command by gesture control. Fig. 17 is a diagram that shows an example of gesture control according to the flowchart of Fig. 16 represents. Fig. Figure 18 is a flowchart that illustrates a seventh processing example of issuing a lane change instruction command by gesture control. Fig. 19 is a diagram that shows an example of gesture control according to the flowchart of Fig. 18 represents. Fig. Figure 20 is a flowchart that represents an eighth processing example of issuing an overtake instruction command by gesture control. Fig. 21 is a diagram that shows an example of gesture operation according to the flowchart of Fig. 20 represents. Fig. Figure 22 is a flowchart that represents a ninth processing example of issuing an overtake instruction command by gesture control. Fig. 23 is a diagram that shows an example of gesture control according to the flowchart of Fig. 22 represents. Fig. Figure 24 is a flowchart that illustrates a processing example of issuing a cancel command after issuing an overtake instruction command via gesture control. Fig. 25 is a diagram that shows an example of gesture control according to the flowchart of Fig. 24 represents. Fig. 26 is a diagram that shows another example of gesture control according to the flowchart of Fig. 24 represents. Fig. Figure 27 is a diagram that provides a first example of how to control the user's own vehicle symbol while the user's own vehicle symbol is being placed and the user's own vehicle is completing the overtaking maneuver. Fig. Figure 28 is a diagram that provides a second illustration of how to control a custom vehicle symbol while the custom vehicle symbol is being placed and the custom vehicle is completing the overtaking maneuver. Fig. Figure 29 is a diagram that provides a third illustration of how to control the own vehicle symbol while the own vehicle symbol is being placed and the own vehicle is completing the overtaking maneuver. Fig. Figure 30 is a diagram that provides a fourth illustration of how to control a custom vehicle symbol while the custom vehicle symbol is being placed and the custom vehicle is completing the overtaking maneuver. Detailed description of preferred embodiments

[0009] The following are embodiments of the present disclosure described with reference to the drawing. Fig. Figure 1 is a block diagram representing an arrangement of a vehicle 1 according to an embodiment of the present disclosure and represents an arrangement relating to automatic driving. The vehicle 1 (a separate vehicle) with an automatic driving mode includes a driver assistance device (human-machine interaction device [MMI]) 10, an automatic driving control device (automatic driving controller) 20, a display device 30, a detection unit 40, and a driving control unit 50.

[0010] The display device 30 includes a display unit 31 and an input unit 32. The display device 30 can be a main unit, such as a car navigation system, an audiovisual device, a portable terminal device, such as a smartphone, a tablet or the like, or a separate console data station.

[0011] The display unit 31 is a liquid crystal display, an organic EL display, or a head-up display (HUD). The input unit 32 is a user interface that receives input from a user. The display unit 31 and the input unit 32 can be an integrated touchscreen display. It can be, for example, a proximity touchscreen that can detect the approach of a hand on a touchscreen or touchpad, or the position of a finger due to a hover control; one that receives a gesture entered at a position at a predetermined distance from a display unit. The input unit 32 can include an input device, such as a mouse, stylus, trackball, or the like, to support gesture input. A stylus that emits visible or infrared light can also be used.

[0012] The display unit 31 and the input unit 32 can be physically separate from each other, instead of being an integrated touchscreen display. For example, the input unit 32 contains a sensor, such as a camera or the like, and can be a contactless input device that enables gesture control via air movement. For example, an operating procedure is envisioned that initiates a dragging action with a gesture by pointing at a target with one finger and closing the thumb and index finger, and which ends the dragging action with a gesture by separating the thumb and index finger.

[0013] The driver assistance device 10 and the display device 30 can be connected to each other by wired communication, such as a leased line, a Controller Area Network (CAN) or the like, or can be connected to each other by wired or wireless communication, such as Universal Serial Bus (USB), Ethernet (registered trademark), Wi-Fi (registered trademark), Bluetooth (registered trademark) or the like.

[0014] The acquisition unit 40 contains a position information acquisition unit 41, a sensor 42, a speed information acquisition unit 43, and a map information acquisition unit 44. The position information acquisition unit 41 obtains the current position of the vehicle 1 from a Global Positioning System (GPS) receiver. The term sensor 42 is a general term for various sensors used to detect a situation outside the vehicle 1. Examples of sensors used to detect a situation outside the vehicle include a camera, millimeter-wave radar, laser imaging detection and ranging (LIDAR), a temperature sensor, a pressure sensor, a humidity sensor, an illumination sensor, or similar devices.Situations outside the vehicle include the road conditions on which the vehicle is traveling, environmental conditions including weather, and the position or condition of another vehicle traveling in the vicinity. Any information outside the vehicle that can be detected by the sensor can be considered. For example, an accelerometer, gyroscope, geomagnetic sensor, tilt sensor, or similar sensor can be used to detect the condition of vehicle 1. The speed information acquisition unit 43 obtains the current speed of vehicle 1 from a vehicle speed sensor. The map information acquisition unit 44 obtains map information about the current position of vehicle 1 from a map database.The map database can be stored on a data carrier in vehicle 1 or can be downloaded from a map server via a network when using it.

[0015] The data acquisition unit 40 and the automatic driving control device 20 are connected to each other via wired communication, such as a dedicated line, USB, Ethernet (registered trademark), a Controller Area Network (CAN), or the like. An arrangement can be used in which data acquired and acquired by the data acquisition unit 40 is directly output from the data acquisition unit 40 to the driving assistance device 10.

[0016] The driving unit 50 includes the steering 51, the brake pedal 52, the accelerator pedal 53, and the turn signal switch 54. In an automatic driving mode according to the present embodiment, acceleration, deceleration, steering, and turn signal flashing are controlled automatically by the automatic driving control device 20. Fig. 1 is a control unit shown when these operations are performed manually. Information indicating that the driving control unit 50 is being moved slightly manually by a driver can be output to the driving assistance device 10.

[0017] The steering unit 51 is a control unit for steering the vehicle. When the steering unit 51 is turned by a driver, the vehicle's direction of travel is controlled by a steering actuator. The steering actuator can be electronically controlled by an electronic steering control unit (ECU).

[0018] The brake pedal 52 is a control unit for decelerating the vehicle 1. When the brake pedal 52 is depressed by a driver, the vehicle is decelerated by a brake actuator. The brake actuator can be electronically controlled by a brake ECU.

[0019] The accelerator pedal 53 is a control unit for accelerating the vehicle 1. When the accelerator pedal 53 is depressed by a driver, an internal combustion engine speed and / or an electric motor speed are controlled via an acceleration actuator. In a purely combustion engine vehicle, the internal combustion engine speed is controlled. In a purely electric vehicle, the electric motor speed is controlled. In a hybrid vehicle, both the internal combustion engine speed and the electric motor speed are controlled. The acceleration actuator can be electronically controlled by at least one internal combustion engine ECU and one electric motor ECU.

[0020] The turn signal switch 54 is a control unit for flashing a turn signal to indicate the vehicle's direction of travel. When the turn signal switch 54 is switched on or off by a driver, the turn signal is activated or deactivated by a turn signal control unit. The turn signal control unit contains a control circuit, such as a relay, that controls the power supply to a turn signal indicator or similar device.

[0021] Each of the steering ECU, brake ECU, internal combustion engine ECU, electric motor ECU, and turn signal control unit, as well as the automatic driving control unit 20, are interconnected via wired communication, such as a CAN bus, a dedicated line, or the like. Each of the steering ECU, brake ECU, internal combustion engine ECU, electric motor ECU, and turn signal control unit sends a status signal to the automatic driving control unit 20, indicating the state of each component (steering, brake, internal combustion engine, electric motor, and turn signal).

[0022] In automatic driving mode, each of the steering ECU, brake ECU, internal combustion engine ECU, and electric motor ECU controls the corresponding actuator according to a control signal supplied by the automatic driving control device 20. In manual driving mode, an arrangement can be used in which an instruction is directly transmitted mechanically from each of the steering 51, brake pedal 52, and accelerator pedal 53 to the corresponding actuator, or an arrangement can be used in which electronic control intervenes via the corresponding ECU. The turn signal control switches the turn signal on / off according to the control signal supplied by the automatic driving control device 20 or according to an instruction signal from the turn signal switch 54.

[0023] The automatic driving control device 20 is an automatic driving controller with an automatic driving control function and comprises a control unit 21, a memory unit 22, and an input / output unit 23. The arrangement of the control unit 21 can be implemented through an interaction of hardware and software resources or solely hardware resources. Hardware resources can include a processor, read-only memory (ROM), random access memory (RAM), and other low-level integrated (LSI) circuits, while software resources can include an operating system, an application, and a program such as firmware or the like. The memory unit 22 contains a non-volatile storage medium, such as flash memory or the like. The input / output unit 23 performs various communication controls according to different communication formats.

[0024] The control unit 21 calculates a control value for controlling an automatic control target, such as the direction of travel of the vehicle 1, by applying various parameter values ​​collected by the sensing unit 40 and various ECUs to an automatic driving algorithm. The control unit 21 transmits the calculated control value to the ECU or control of each control target. In the present embodiment, the control value is transmitted to the steering ECU, the brake ECU, the internal combustion engine ECU, and the turn signal control. In the case of an electric vehicle or a hybrid vehicle, the control value is transmitted to the electric motor ECU instead of, or in addition to, the internal combustion engine ECU.

[0025] The driver assistance device 10 is a human-machine interaction (MMI) controller for performing an interface function between the vehicle 1 and a driver and comprises a destination unit 11, a generation unit 12, an instruction unit 13, and an input / output unit 14. Destination unit 11, generation unit 12, and instruction unit 13 can be implemented through an interaction of hardware and software resources or solely by hardware resources. Hardware resources can include a processor, ROM, RAM, and other LSI circuits, while software resources can include an operating system, an application, and a program such as firmware or the like. The input / output unit (I / O unit) 14 performs various communication controls according to different communication formats.The input / output unit 14 comprises an image output unit 14a, an operating signal input unit 14b, a command output unit 14c, and a vehicle information input unit 14d. The image output unit 14a outputs an image generated by the generation unit 12 to the display unit 31. The operating signal input unit 14b receives an operating signal input from the input unit 32 by a driver, passenger, or user outside the vehicle and outputs the operating signal to the destination unit 11. The command output unit 14c outputs the command instructed by the instruction unit 13 to the automatic driving control 20. The vehicle information input unit 14d receives acquisition data obtained by the acquisition unit 40 or vehicle information generated by the automatic driving control 20 and outputs the acquisition data or the vehicle information to the generation unit 12.

[0026] The automatic driving control unit 20 and the MMI control unit 10 are directly connected to each other via a signal line. An arrangement in which the automatic driving control unit 20 and the MMI control unit 10 are connected to each other via a CAN bus can be used. An arrangement in which the automatic driving control unit 20 and the MMI control unit 10 are combined in a single control unit can also be used.

[0027] Fig. Figure 2 is a diagram showing an example of a basic sequence of sensor unit 40, automatic drive control 20, MMI control 10, display unit 31 and input unit 32 in Fig. Figure 1 represents the process. The detection unit 40 acquires position information for the vehicle, road information for the vehicle, and environmental information for the vehicle, including any other vehicle traveling in the vicinity, and outputs this information to the automatic driving control unit 20 (P1). The automatic driving control unit 20 outputs the position information, road information, and environmental information obtained from the detection unit 40 to the MMI control unit 10 (P2). The MMI control unit 10 generates a schematic diagram, based on the information obtained from the automatic driving control unit 20, showing the vehicle, any other vehicle, and the vehicle's surroundings (P3).The MMI control unit 10 outputs the schematic diagram to the display device 30, and the schematic diagram is displayed on the display unit 31 (P4).

[0028] A user viewing the schematic diagram displayed on the display unit 31 makes contact with the input unit 32 (P5). The display unit 31 outputs coordinate data to the MMI control unit 10 at the position where the contact is detected (P6). Based on the coordinate data obtained from the display unit 30, the MMI control unit 10 determines a type of command (P7). The MMI control unit 10 receives additional input until a certain period of time has elapsed (P8 to P12). After determining the command, the MMI control unit 10 rebuilds a schematic diagram indicating that the command is being issued (P8). The MMI control unit 10 outputs the rebuilt schematic diagram to the display unit 30, and the rebuilt schematic diagram is displayed on the display unit 31 (P9).In a case where the command corresponding to gesture control by touching a user does not exist, the MMI control 10 generates a schematic diagram containing an error message and outputs the schematic diagram to the display device 30, and the schematic diagram is displayed on the display unit 31.

[0029] When a user viewing the schematic diagram indicating that the command is being issued comes into contact with input unit 32 (P10), display unit 31 outputs the coordinate data at the position where the contact was detected to MMI control 10 (P11). MMI control 10 then performs additional command processing based on the coordinate data obtained from display unit 30 (P12). If no new command is entered during this additional command processing (P12), MMI control 10 outputs a command specified in P7 to automatic drive control 20 (P13 and P14). If a new command is entered during this additional command processing (P12), MMI control 10 outputs a new command to automatic drive control 20. In a case where the new command entered is a cancel command, the MMI control 10 cancels the output of a command.The processing of overwriting and canceling the original command due to a new command can be performed by the automatic driving control 20. In this case, after the command determination processing in P7 and P12, the MMI control 10 sends the command to the automatic driving control 20 and performs the overwriting and canceling processing according to an internal state of the automatic driving control 20.

[0030] The detection unit 40 periodically acquires the vehicle's position information, road information, and surroundings and transmits this information to the automatic driving control unit 20 (P15). Based on this information, the automatic driving control unit 20 determines whether a control action instructed by the MMI control unit 10 can be executed (P16). If the control action is deemed executable, the automatic driving control unit 20 sends a control start message to the MMI control unit 10 (P17). Upon receiving the control start message, the MMI control unit 10 rebuilds a schematic diagram containing a message indicating that the control action is being carried out (P18).The MMI control unit 10 outputs the newly constructed schematic diagram to the display device 30, and the newly constructed schematic diagram is displayed on the display unit 31 (P19). Although not shown, the automatic driving control unit 20 calculates a specific control value to control the driving unit 50, which executes the output command by applying various parameter values ​​collected by the sensing unit 40 or different ECUs to an automatic driving algorithm, and transmits the control value to the ECU or the control of each control target, where the control value is a target of the automatic control, such as a direction of travel of the vehicle 1. The driving unit 50 operates based on the specific control value.If a predefined control value or the data obtained by the acquisition unit 40 is a predefined value (within a predefined range), and if the automatic driving control 20 determines that a condition of the issued command is met, the driving control unit 50 determines that the execution of the command is complete.

[0031] When a control completion message is received from the automatic driving control 20, the MMI control 10 generates a schematic diagram containing a message indicating that the control process is complete and outputs the generated schematic diagram to the display device 30. During a period in which no user input is received, the MMI control 10 generates a schematic diagram containing a message indicating that no user input is received and outputs the generated schematic diagram to the display device 30.

[0032] Fig. Figure 3 is a diagram that shows an example of a basic flowchart to explain the process of MMI control 10 in Fig. 1 represents the determination unit 11 of the MMI control 10, which checks whether a driving mode is automatic or manual (S1). The process ends in manual mode (N in S2). In the case of automatic mode (J in S2), the processing is carried out as follows.

[0033] Sensor information input by the sensing unit 40 to the automatic driving control 20 is continuously updated (S3). The generation unit 12 of the MMI control 10 generates a schematic diagram containing the vehicle itself, another vehicle, and the vehicle's surroundings based on the vehicle's position information, road information, and surroundings, including any other vehicle in the vicinity, input by the automatic driving control 20. The generated schematic diagram is then displayed on the display unit 31 (S4). The determination unit 11 checks whether a receive mode is an enable mode, in which a user can receive input, or an block mode, in which a user cannot receive input (S5).In a case where the receive mode is receive-block mode (N in S6), the process ends. In a case where the receive mode is receive-enable mode (J in S6), the destination unit 11 determines whether or not there is user contact at the input unit 32 (S7). In a case where there is no user contact at the input unit 32 (N in S8), a determination process, described below, is performed to determine whether a predetermined time has elapsed (S12). In a case where there is user contact at the input unit 32 (J in S8), the destination unit 11 determines a control command according to a gesture input from the user (S9). The details of the determination process are described below.

[0034] In a case where the control command determined in step S9 is not a cancel command (N in S10), the generation unit 12 indicates on the display unit 31 that the command is being issued (S11). If a predetermined time has elapsed after the control command has been determined (J in S12), and if step S9 indicates that a control command exists (J in S13), an operating receive lock is displayed on the display unit 31 (S14), the determination unit 11 updates the receive mode from receive enable mode to receive lock mode (S15), and the instruction unit 13 outputs the determined control command to the automatic drive controller 20 (S16). Until a predetermined time has elapsed (N in S12), the process proceeds to step S3.

[0035] In step S10, if the specified control command is a cancel command (J in S10), "Cancel" is displayed (S110), and the process ends. In step S13, if no control command is specified in step S9, an input error is displayed (S111), and the process ends. The automatic driving control 20 periodically acquires the position information, the road information, and the environmental information for the vehicle from the acquisition unit 40. Because the environmental information for the vehicle is constantly changing, there is a case after the control command for the automatic driving control 20 has been issued where it is determined that the control command cannot be executed. For example, after a subsequent instruction, there is a case where other vehicles interfere with the vehicle's path.In a case where it is determined that the control command can be executed by the automatic drive control 20 (J in S17), the generating unit 12 indicates on the display unit 31 that the control is being carried out (S18) and starts a count by activating a timer (S19). In a case where it is determined that the control by the automatic drive control 20 cannot be executed (N in S17), the generating unit 12 indicates a control lock error on the display unit 31 (S112).

[0036] Fig. Figure 4 is a flowchart illustrating the update processing of a receive mode. When a timer count reaches a setpoint (for example, 10 seconds) (J in S113), the destination unit 11 of the MMI control 10 updates the receive mode from receive lock mode to receive enable mode (S114). The timer count can be changed according to the vehicle's environmental conditions. If a message indicating the completion of control is received from the automatic driving control 20, or if, based on the vehicle's behavior 1, it is determined that control is complete according to the control command, the destination unit 11 can update the receive mode from receive lock mode to receive enable mode.

[0037] In this embodiment, an example of a control command is described below, in which a control command to instruct an overtaking maneuver is issued. A user enters a gesture command to instruct the overtaking maneuver into the input unit 32. A specific example of the gesture command is described further below.

[0038] Fig. 5 is a flowchart that illustrates an example of determination processing in a case where a gesture control is used to instruct an overtaking maneuver in step S9 of Fig. 3 is entered. The determination unit 11 of the MMI control 10 determines whether a custom vehicle symbol is present at a contact start position or not (S9a). If the custom vehicle symbol is not present at the contact start position (N in S9a), the determination unit 11 determines that the gesture operation is a control command for an instruction other than an overtaking instruction (S9b). If the custom vehicle symbol is present at the contact start position (J in S9a), the generation unit 12 draws a storage release area in the schematic diagram and displays the area on the display unit 31 (S9c). A specific example of the storage release area is described below.

[0039] The determination unit 11 receives a contact event (S9d) generated in the input unit 32 and determines the type of contact event (S9e). In a case where the type of contact event is movement (movement in S9e), the generation unit 12 draws a predicted trajectory / path candidate of vehicle 1 in the schematic diagram, and the predicted trajectory / path candidate of vehicle 1 is displayed on the display unit 31 (S9f).

[0040] In a case where the type of contact event is a contact end (contact end in S9e), the determination unit 11 determines whether a contact end position is located in front of another vehicle symbol or not (S9g). In a case where the contact end position is not located in front of another vehicle symbol (N in S9g), the determination unit 11 determines that the gesture operation is a control command of an instruction other than an overtaking instruction (S9h). In a case where the contact end position is located in front of another vehicle symbol (J in S9g), the determination unit 11 determines whether the other vehicle symbol is positioned in the same lane as the lane in which the user's own vehicle symbol is positioned (S9i).In a case where the other vehicle symbol is not positioned in the same lane as the lane in which the vehicle symbol is positioned (N in S9i), the determination unit 11 determines whether the direction of travel of the vehicle is the same as the direction of travel of the other vehicle (S9j). In a case where the direction of travel of the vehicle is the same as the direction of travel of the other vehicle (J in S9j), the determination unit 11 determines that the gesture control is a lane change instruction command to instruct the lane change to the lane in which the other vehicle symbol is positioned (S9k). In a case where the direction of travel of the vehicle is not the same as the direction of travel of the other vehicle (N in S9j), the generation unit 12 displays an error message on the display unit 31 (S91).

[0041] In step S9i, if the other vehicle symbol is positioned in the same lane as the lane in which the user's own vehicle symbol is positioned (J in S9i), the determination unit 11 determines whether the path of the contact point passed through the left or right side of the other vehicle symbol (S9m). If the path of the contact point passed through the left side of the other vehicle symbol (left in S9m), the determination unit 11 determines whether at least one other vehicle symbol is present between the user's own vehicle symbol and the other vehicle symbol (S9n). If no other vehicle symbol is present between the user's own vehicle symbol and the other vehicle symbol (N in S9n), the determination unit 11 determines that the gesture control is an overtaking instruction command to instruct the user's own vehicle to overtake another vehicle on the left (S9o).In a case where at least one other vehicle symbol is present between the vehicle's own symbol and the other vehicle symbol (J in S9n), the determination unit 11 determines that the gesture operation is an overtaking instruction command to instruct the vehicle's own vehicle to overtake a multitude of other vehicles from the left (S9p).

[0042] In step S9m, if the path of the contact point passed through the right side of the other vehicle symbol (right in S9m), the determining unit 11 determines whether at least one other vehicle symbol is present between the vehicle's own symbol and the other vehicle symbol (S9q). If no other vehicle symbol is present between the vehicle's own symbol and the other vehicle symbol (N in S9q), the determining unit 11 determines that the gesture control is an overtaking instruction command to instruct the vehicle's own vehicle to overtake another vehicle on the right (S9r).In a case where at least one other vehicle symbol is present between the own vehicle symbol and the other vehicle symbol (J in S9q), the determination unit 11 determines that the gesture operation is an overtaking instruction command to instruct the own vehicle to overtake a multitude of other vehicles from the right-hand side (S9s).

[0043] Below is a specific example of gesture control used during overtaking. The following examples assume the use of a touchscreen display in which display unit 31 and input unit 32 are integrated.

[0044] Fig. Figure 6 is a flowchart illustrating a first processing example of issuing an overtaking instruction command via gesture control. The determination unit 11 of the MMI control 10 receives a touch event "DOWN" from the touchscreen (S30). The touch event "DOWN" represents a change in the contact state on the touchscreen from a non-contact state to a contact state via a finger or stylus. The determination unit 11 determines whether a coordinate detected by the touch event "DOWN" is located within the display area of ​​its own vehicle icon (S31). If the coordinate detected by the touch event "DOWN" is outside the display area of ​​its own vehicle icon (N in S31), it is determined that the gesture control is not an overtaking instruction, and the process ends.

[0045] In a case where the coordinate detected by the touch event "DOWN" is located within the display area of ​​the vehicle's own symbol (J in S31), the determining unit 11 receives a touch event "MOVING" from the touch screen (S32). The touch event "MOVING" is an event that represents a change from a contact state at a specific point on the touch screen to a contact state at another point on the touch screen by a finger or stylus. Then, the determining unit 11 receives a touch event "UPWARDS" from the touch screen (S33). The touch event "UPWARDS" is an event that represents a change in the contact state on the touch screen from a contact state to a non-contact state by a finger or stylus.

[0046] The determination unit 11 determines whether a coordinate detected by the "UPWARDS" touch event is located in an area in front of another vehicle symbol or not (S34). If the coordinate detected by the "UPWARDS" touch event is located in the area in front of another vehicle symbol (J in S34), the instruction unit 13 issues an overtaking instruction command to the automatic driving control 20, instructing the vehicle to overtake the other vehicle corresponding to the other vehicle symbol (S35). If the coordinate detected by the "UPWARDS" touch event is not located in front of another vehicle symbol (N in S34), it is determined that the gesture operation is not an overtaking instruction, and the process ends.

[0047] Fig. 7 is a diagram that shows an example of gesture control according to the flowchart of Fig. 6 represents. In the section in (a) of Fig. Figure 7 of the schematic diagram shows a vehicle symbol (V1) and another vehicle symbol (V2) on the same lane. Various representations are considered for the vehicle, the other vehicle, and the surrounding environment, including the road. A true photograph can be used, as well as a detailed computer graphic or animation. The representation of the vehicle is not limited to a symbol; it can be a simple marker or sign, or a true photograph. In other words, there is no problem as long as the vehicle is displayed on a screen as an object in any form. The same applies to the representation of another vehicle.

[0048] In a case where a driver wants their own vehicle to overtake another vehicle, as in (a) of Fig. As shown in (7), a driver pulls their own vehicle symbol V1 and places it as shown in (b) and (c). Fig. Figure 7 shows the vehicle's own symbol V1 in the lane in front of the other vehicle symbol V2. Accordingly, an overtaking instruction is issued.

[0049] Fig. Figure 8 is a flowchart illustrating a second processing example of issuing an overtaking instruction command via a gesture control. This second processing example involves dragging and dropping a different vehicle icon. The MMI control unit 10's determination unit 11 receives a "DOWN" touch event from the touchscreen (S30). Determination unit 11 then determines whether a coordinate detected by the "DOWN" touch event is within the display area of ​​another vehicle icon (a vehicle ahead) or not (S36). If the coordinate detected by the "DOWN" touch event is outside the display area of ​​another vehicle icon (a vehicle ahead) (N in S36), the gesture control is determined not to be an overtaking instruction, and the process terminates.

[0050] In a case where the coordinate detected by the touch event "DOWN" is located within the display area of ​​another vehicle symbol (a vehicle ahead) (J in S36), the determination unit 11 receives a touch event "MOVING" from the touch screen (S32). Then, the determination unit 11 receives a touch event "UPWARDS" from the touch screen (S33). The determination unit 11 determines whether a coordinate detected by the touch event "UPWARDS" is located in an area behind its own vehicle symbol or not (S37). In a case where the coordinate detected by the contact event “UPWARDS” is located in the area behind the vehicle’s own symbol (J in S37), the instruction unit 13 issues an overtaking instruction command to the automatic driving control 20 to instruct the vehicle’s own vehicle to overtake the other vehicle corresponding to the other vehicle symbol (of the vehicle ahead) (S35).In a case where the coordinate detected by the touch event “UPWARDS” is not behind the vehicle’s own symbol (N in S37), it is determined that the gesture control is not an overtaking instruction and the process ends.

[0051] Fig. 9 is a diagram that shows an example of gesture control according to the flowchart of Fig. 8 represents. In a case where a driver wants their own vehicle to overtake another vehicle, as in (a) of Fig. As shown in (9), a driver pulls the other vehicle symbol V2 and places it as shown in (b) and (c) of Fig. 9 is shown, the other vehicle symbol V2 is shown in the lane behind the own vehicle symbol V1. Accordingly, an overtaking instruction is issued.

[0052] Fig. Figure 10 is a flowchart illustrating a third processing example of issuing an overtaking instruction command via gesture control. This third processing example involves swapping (rotating) a user's own vehicle icon and another vehicle icon using a gesture. The MMI control unit 10's determination unit 11 receives a two-point touch event "DOWN" from the touchscreen (S38). Determination unit 11 then determines whether a point's coordinate detected by the "DOWN" touch event lies within the display area of ​​the user's own vehicle icon (S39). If the point's coordinate detected by the "DOWN" touch event lies outside the display area of ​​the user's own vehicle icon (N in S39), the gesture is determined not to be an overtaking instruction, and the process terminates.

[0053] In a case where the coordinate of a point detected by the "DOWN" touch event is within the display area of ​​the vehicle's own icon (J in S39), the determination unit 11 determines whether a coordinate of the other point detected by the "DOWN" touch event is within the display area of ​​the other vehicle icon (the vehicle ahead) or not (S310). In a case where the coordinate of the other point detected by the "DOWN" touch event is outside the display area of ​​the other vehicle icon (the vehicle ahead) (N in S310), it is determined that the gesture operation is not an overtaking instruction, and the process ends.

[0054] In a case where the coordinate of the other point detected by the "DOWN" touch event is located within the display area of ​​the other vehicle symbol (the vehicle ahead) (J in S310), the determination unit 11 receives a "MOVING" touch event from the touch screen (S311). Then, the determination unit 11 receives a two-point "UPWARDS" touch event from the touch screen (S312). The determination unit 11 determines whether a coordinate detected by the "UPWARDS" touch event on its own vehicle symbol is located in front of a coordinate detected by the "UPWARDS" touch event on the other vehicle symbol (S313).In a case where the vehicle's own symbol is in front of the other vehicle symbol (J in S313), instruction unit 13 issues an overtaking instruction command to the automatic driving control 20 to instruct the vehicle's own vehicle to overtake the other vehicle corresponding to the other vehicle symbol (the vehicle ahead) (S314). In a case where the vehicle's own symbol is not in front of the other vehicle symbol (N in S313), it is determined that the gesture control is not an overtaking instruction, and the process ends.

[0055] Fig. 11 is a diagram that shows an example of gesture control according to the flowchart of Fig. 10 represents. In a case where a driver wants their own vehicle to overtake another vehicle, as in (a) of Fig. As shown in Figure 11, a driver touches their own vehicle symbol V1 and the other vehicle symbol V2 with two fingers and swaps them, as in (b) and (c) of Fig. Figure 11 shows the sequence of the vehicle's own symbol V1 and the other vehicle symbol V2, and places the vehicle's own symbol V1 and the other vehicle symbol V2. Accordingly, an overtaking instruction is issued.

[0056] Fig. Figure 12 is a flowchart illustrating a fourth processing example of issuing an overtaking instruction command via a gesture control. This fourth processing example involves touching a user's own vehicle icon and then changing the scale of the schematic diagram. The determination unit 11 of the MMI control 10 receives a touch event "DOWN" from the touchscreen (S30). The determination unit 11 determines whether a coordinate detected by the touch event "DOWN" is within a display area of ​​the user's own vehicle icon (S31). If the coordinate detected by the touch event "DOWN" is outside the display area of ​​the user's own vehicle icon (N in S31), it is determined that the gesture control is not an overtaking instruction, and the process terminates.

[0057] In a case where the coordinate detected by the "DOWN" touch event is within the display area of ​​the vehicle's own icon (J in S31), the determination unit 11 determines whether another vehicle icon (of a preceding vehicle) is included in the schematic diagram displayed on the touchscreen (S315). In a case where no other vehicle icon (of a preceding vehicle) is included in the schematic diagram on the screen (N in S315), the generation unit 12 changes (reduces) the scale of the schematic diagram so that another vehicle icon (of a preceding vehicle) is included in the schematic diagram (S316). In a case where another vehicle icon (of a preceding vehicle) is included in the schematic diagram on the screen (J in S315), the operation of step S316 is skipped.The determining unit 11 receives a touch event “MOVE” from the touch screen (S32). The processes from step S33 to step S35 of . Fig. 12 are the same as those in the flowchart of Fig. 6.

[0058] Fig. 13 is a diagram that shows an example of gesture control according to the flowchart of Fig. 12 represents. If, as in (a) of Fig. 13 shows a driver touching their own vehicle symbol V1, in a case where no other vehicle symbol is present in the schematic diagram shown on the screen, as in (b) of Fig. As shown in Figure 13, the driver reduces the scale of the schematic diagram until another vehicle symbol V2 is included in the schematic diagram. As shown in (c) and (d) of Fig. As shown in Figure 13, a driver places their own vehicle symbol V1 in front of the other vehicle symbol V2, which is now displayed. Accordingly, an overtaking instruction is issued. Therefore, it is possible to easily find another vehicle and issue an overtaking instruction more quickly.

[0059] Fig. Figure 14 is a flowchart that depicts a fifth processing example of issuing an overtaking instruction command via a gesture control. This fifth processing example involves a single vehicle overtaking a large number of other vehicles. The operations from step S30 to step S33 are the same as those in the flowchart of [reference missing]. Fig. 6.

[0060] The determination unit 11 determines whether the coordinate detected by the "UPWARDS" touch event is located in an area in front of a multitude of other vehicle symbols or not (S317). If the coordinate detected by the "UPWARDS" touch event is located in the area in front of the multitude of other vehicle symbols (J in S317), the instruction unit 13 issues a sequential overtaking instruction command to the automatic driving control 20 (S318) to instruct the vehicle to overtake a multitude of other vehicles sequentially, corresponding to the multitude of other vehicle symbols. If the coordinate detected by the "UPWARDS" touch event is not located in front of the multitude of other vehicle symbols (N in S317), it is determined that the gesture operation is not a sequential overtaking instruction command, and the process ends.

[0061] Fig. 15 is a diagram that shows an example of gesture control according to the flowchart of Fig. 14 represents. In a case where a driver wants their vehicle to overtake two other vehicles ahead at once, as in (a) of Fig. As shown in 15, the driver draws their own vehicle symbol V1 and places it as shown in (b) and (c) of Fig. Figure 15 shows the vehicle's own symbol V1 positioned in front of two other vehicle symbols V2a and V2b. Accordingly, a sequential overtaking instruction is issued.

[0062] Fig. Figure 16 is a flowchart that represents a sixth processing example of issuing an overtaking instruction command via a gesture control. This sixth processing example describes an overtaking sequence when overtaking another vehicle. The operations from step S30 to step S34 are the same as those in the flowchart of [reference missing]. Fig. 6.

[0063] The determination unit 11 determines whether an intersection point between the path of the contact event "MOVE" and a horizontal line passing through the other vehicle symbol lies on the right or left side of the coordinate of the other vehicle symbol (S319). In a case where the intersection point lies on the right side of the coordinate of the other vehicle symbol (right in S319), the instruction unit 13 issues an overtaking instruction command to the automatic driving control 20, instructing the own vehicle to overtake the other vehicle corresponding to the other vehicle symbol from the right (S320).In a case where the intersection point is on the left side of the coordinate of the other vehicle symbol (left in S319), the instruction unit 13 issues an overtaking instruction command to the automatic driving control 20 to instruct its own vehicle to overtake the other vehicle corresponding to the other vehicle symbol from the left (S321).

[0064] Fig. 17 is a diagram that shows an example of gesture control according to the flowchart of Fig. 16 represents. If a driver wants their vehicle to overtake a vehicle ahead on the right, as in (a) of Fig. As shown in (b) 17, the driver draws their own vehicle symbol V1, as shown in (b) of Fig. As shown in (c) of 17, the own vehicle symbol V1 moves so that it goes around the right side of the other vehicle symbol V2, and, as shown in (c) of Fig. As shown in Figure 17, the vehicle's own symbol V1 is positioned in front of the other vehicle symbol V2. Accordingly, an overtaking instruction is issued, instructing the vehicle to overtake from the right.

[0065] On the other hand, if a driver wishes their vehicle to overtake a vehicle ahead on the left, as in (d) of Fig. As shown in 17, the driver draws their own vehicle symbol V1, as in (e) of Fig. As shown in (f) of 17, the own vehicle symbol V1 moves so that it passes around the left side of the other vehicle symbol V2, and, as shown in (f) of Fig. As shown in Figure 17, the vehicle's own symbol V1 is positioned in front of the other vehicle symbol V2. Accordingly, an overtaking instruction is issued, signaling the vehicle to overtake from the left.

[0066] Instead of a user instructing a specific control procedure for overtaking, a user might, for example, simply instruct the overtaking maneuver by moving their own vehicle icon to a position in front of the icon of a preceding (other) vehicle. A specific control procedure associated with a command can be modified by a vehicle according to the context. For example, in general Japanese traffic regulations, although it is permitted to overtake another vehicle from a right-hand lane, if no other vehicle is present in a left-hand lane, the user's own vehicle may overtake another vehicle from a left-hand lane. Furthermore, the overtaking maneuver can be performed in a manner that suits the driver's personal preferences.In the special control procedure associated with a command, for example, the determination unit 11 of the MMI control 10 can determine the special control procedure with respect to a path, an execution time, a speed, and the like, and send a command containing the special control procedure to the automatic driving control 20, or the automatic driving control 20 can determine the special control procedure after receiving the command. Furthermore, the special control procedure can be selected by a driver by displaying a selection of special control procedures, determined by the MMI control 10 or the automatic driving control 20, on the display unit 31, or by visually / tactilely communicating the special control procedure to a driver using a notification unit containing a loudspeaker or a vibration device, which is not shown.

[0067] Fig. Figure 18 is a flowchart illustrating a seventh processing example of issuing a lane change instruction via gesture control. This seventh processing example demonstrates displaying the path of the vehicle's own icon in the schematic diagram. The MMI control unit 10's determination unit 11 receives a "DOWN" touch event from the touchscreen (S30). Determination unit 11 then determines whether a coordinate detected by the "DOWN" touch event is within the display area of ​​the vehicle's own icon (S31). If the coordinate detected by the "DOWN" touch event is outside the display area of ​​the vehicle's own icon (N in S31), the system determines that the gesture is not an overtaking instruction, and the process terminates.

[0068] In a case where the coordinate detected by the touch event "DOWNWARDS" is located within the display area of ​​the vehicle's own symbol (J in S31), the determination unit 11 receives a touch event "MOVING" / "UPWARDS" from the touch screen (S322). The determination unit 11 determines the type of touch event (S323). In a case where the type of touch event is "MOVING" (MOVING in S323), the determination unit 11 determines whether a coordinate detected by the touch event "MOVING" is located in an area in front of another (preceding) vehicle symbol or not (S324).In a case where the coordinate detected by the "MOVE" touch event is located in the area in front of another (ahead) vehicle symbol (J in S324), generation unit 12 generates a predicted path for the own vehicle when the overtaking maneuver is performed and displays the predicted path on the touch screen (S325). In a case where the coordinate detected by the "MOVE" touch event is not located in the area in front of another (ahead) vehicle symbol (N in S324), the operation of step S325 is skipped. The process then proceeds to step S322. Since the present flowchart focuses on displaying the predicted path, the subsequent processing is omitted in a case where the touch event type is "UP" (UP in S323).

[0069] Fig. 19 is a diagram that shows an example of gesture control according to the flowchart of Fig. 18 represents. In a case where a driver wants their own vehicle to overtake another vehicle, as in (a) of Fig. 19 shows that when a driver pulls their own vehicle symbol V1 and, as in (b) of Fig. 19 shows the own vehicle symbol V1 being placed in a position in front of the other vehicle symbol V2, and a predicted path T1 is shown that the own vehicle will take when overtaking another vehicle.

[0070] Fig. Figure 20 is a flowchart that depicts an eighth processing example of issuing an overtake instruction command via a gesture control. The eighth processing example is an example of requesting a confirmation control. The operations from step S30 to step S34 are the same as those in the flowchart of [reference missing]. Fig. 6.

[0071] In a case where the coordinate detected by the "UPWARDS" touch event is located in the area in front of another (ahead) vehicle symbol (J in S34), the instruction unit 13, when the destination unit 11 receives an input of a confirmation gesture (J in S326), issues an overtaking instruction command to the automatic driving control 20 (S35), instructing its own vehicle to overtake the other vehicle corresponding to the other vehicle symbol. During a period in which no confirmation gesture is entered (N in S326), the output of the command is deferred. This further prevents erroneous operations.

[0072] Fig. 21 is a diagram that shows an example of gesture operation according to the flowchart of Fig. 20 represents. In a case where a driver wants their vehicle to overtake a vehicle in front, as in (a) of Fig. As shown in (b) 21, the driver pulls their own vehicle symbol V1 and moves it as shown in (b) Fig. As shown in (c) of 21, the own vehicle symbol V1 is moved to a position in front of the other vehicle symbol V2. When the own vehicle symbol V1 is moved, a confirmation button C1 is displayed. As in (c) of Fig. As shown in Figure 21, after the vehicle symbol V1 is placed in the position in front of the other vehicle symbol V2, an overtaking instruction command is issued when the confirmation button C1 is pressed.

[0073] Fig. Figure 22 is a flowchart that represents a ninth processing example of issuing an overtaking instruction command via a gesture control. This ninth processing example illustrates a case where a representation is added during control. "During control" means during a period in which an overtaking instruction command is issued and then the overtaking of the vehicle is completed. The operations from step S30 to step S35 are the same as those in the flowchart of [reference missing]. Fig. 6.

[0074] After the instruction unit 13 has issued an overtaking instruction command to the automatic driving control 20 (S35), the generation unit 12 generates its own vehicle symbol at the coordinate before the movement of its own vehicle and the coordinate after the movement of its own vehicle, generates a predicted path of its own vehicle and displays the predicted path of its own vehicle on the touchscreen (S327).

[0075] Fig. 23 is a diagram that shows an example of gesture control according to the flowchart of Fig. 22 represents. In a case where a driver wants their own vehicle to overtake a vehicle in front, as in (a) and (d) of Fig. As shown in 23, the driver pulls their own vehicle symbol V1 and places it as shown in (b) and (e) of Fig. As shown in figure 23, the vehicle's own symbol V1 is positioned in front of the other vehicle symbol V2. Accordingly, an overtaking instruction is issued. During the execution of the overtaking instruction, as shown in (c) and (f) of Fig. As shown in (f) of 23, the vehicle's own symbol V1a is shown at the original position before the overtaking maneuver began, and the vehicle's own symbol V1b is shown at the target position after the overtaking maneuver is completed. Fig. Figure 23 may show a predicted path T1 of the vehicle.

[0076] Fig. Figure 24 is a flowchart illustrating a processing example of issuing a cancel command following an overtaking instruction via gesture control. After an overtaking instruction command is issued to the automatic driving control unit 20, the destination unit 11 of the MMI control unit 10 receives a touch event “DOWN” from the touchscreen (S328). Then, the destination unit 11 of the MMI control unit 10 receives a touch event “MOVE” from the touchscreen (S329). Finally, the destination unit 11 of the MMI control unit 10 receives a touch event “UP” from the touchscreen (S330). The destination unit 11 determines whether or not there is an intersection between the movement path of the touch event and the overtaking path (S331).In a case where there is an intersection (J in S331), instruction unit 13 issues a command to cancel the previously issued overtaking instruction command to the automatic driving control 20 (S332). In a case where there is no intersection (N in S331), no cancellation command is issued.

[0077] Fig. 25 is a diagram that shows an example of gesture control according to the flowchart of Fig. 24 represents. In (a) of Fig. Figure 25 shows an example in which, if a wipe S1 of the predicted path T1 is performed in such a way that it crosses the predicted path T1, a cancel command is issued to rescind the overtaking instruction. In (b) of Fig. Figure 25 shows an example in which, if a wipe S2 of the own vehicle symbol V1b is performed after the movement in such a way that it crosses the own vehicle symbol V1b, a cancel command is issued to cancel the overtaking instruction command. In (c) of Fig. Figure 25 shows an example in which, if a flick of the own vehicle symbol V1b is performed after the movement, starting at the own vehicle symbol V1b, a cancel command is issued to cancel the overtaking instruction command. In (d) of Fig. Figure 25 shows an example in which, if a flick F2 of the predicted path T1 is performed, starting from the predicted path T1, a cancel command is issued to cancel the overtaking instruction command.

[0078] Fig. 26 is a diagram that shows another example of gesture control according to the flowchart of Fig. 24 represents. If, as in (a) of Fig. 26 shows a driver dragging their own vehicle symbol V1b after the movement, and as in (b) of Fig. As shown in 26, if the vehicle's own symbol V1c is placed on the position of the vehicle's own symbol before the movement, a cancellation command is issued to cancel the overtaking instruction command.

[0079] During the period in which the user's vehicle symbol is being dragged and dropped, the generation unit 12 can retain the original image (the user's vehicle symbol before the movement) in the schematic diagram and delete the original image when the user's vehicle symbol is dropped. During the period in which the user's vehicle symbol is being dragged and dropped, the generation unit 12 can draw a path of the dragged user's vehicle symbol using a dashed line in the schematic diagram. During the period in which the user's vehicle symbol is being dragged and dropped, the generation unit 12 can invert the color of the road and change the road color back to its original color when the user's vehicle symbol is dropped.

[0080] If, during the period in which the own vehicle icon is dragged and dropped, there is a no-drop zone (an oncoming lane or similar), or if an operation (overtaking or similar) is blocked, the generating unit 12 can change the color of the own vehicle icon (invert, thin it, or similar). When the own vehicle icon is dropped, it may return to its original position, and an error message, such as "Operation Blocked," may be displayed. Examples of situations where an operation is blocked include an approaching vehicle, a no-overtaking zone, exceeding a speed limit, or similar.

[0081] If an operation is locked while the user's vehicle icon is being dragged and dropped, the generation unit 12 may change (invert, thin, or otherwise alter) the color of a background, such as a road or similar. When an operation is released, the background color reverts to its original color. During the same period, the user's vehicle icon may also change (invert, thin, or otherwise alter) the color of the drop lock area. If a drop lock area is active or an operation is locked while the user's vehicle icon is being dragged and dropped, the generation unit 12 may issue a notification using an error tone or vibration.

[0082] When dragging the user's vehicle icon begins, if the start of an operation is locked, the generation unit 12 may change the color of the user's vehicle icon. The dragging operation (the movement of the user's vehicle icon) may be locked. An error message, such as "Operation locked," may be displayed. When dragging the user's vehicle icon begins, if the start of an operation is locked, the generation unit 12 may change the background color, such as that of a road or similar. When an operation is released, the background color reverts to its original color. When dragging the user's vehicle icon begins, if the start of an operation is locked, the generation unit 12 will issue a notification using an error tone or vibration in a state where the dragging operation (the movement of the user's vehicle icon) is locked.

[0083] Fig. Figure 27 is a diagram that provides a first example of how to control the own vehicle icon while the own vehicle icon is being lowered and the own vehicle is completing the overtaking maneuver. In a case where a driver wants the own vehicle to overtake a vehicle ahead, as in (a) of Fig. As shown in (b) 27, the driver pulls their own vehicle symbol V1 and places it as shown in (b) Fig. 27 shows the own vehicle symbol V1 in a position in front of the other vehicle symbol V2. During a period in which the own vehicle symbol V1 is placed in front of the other vehicle symbol V2 and the own vehicle completes the overtaking maneuver, as in (c) of Fig. As shown in Figure 27, the state (current position) of the vehicle is represented as a ghost symbol V1g. A moving track T1 is also shown. When the overtaking maneuver is complete, as in (d) of Fig. When 27 is displayed, the ghost symbol V1g and the track T1 of the own vehicle are deleted.

[0084] Fig. Figure 28 is a diagram that provides a second display example of how an own vehicle icon is controlled while the own vehicle icon is being dropped and the own vehicle is completing the overtaking maneuver. In a case where a driver wants the own vehicle to overtake a vehicle ahead, as in (a) of Fig. As shown in (b) 28, the driver pulls their own vehicle symbol V1 and places it as shown in (b) Fig. 28 shows the own vehicle symbol V1 in a position in front of the other vehicle symbol V2. During the period in which the own vehicle symbol V1 is placed in front of the other vehicle symbol V2 and the own vehicle completes the overtaking maneuver, as in (c) of Fig. As shown in (28), the display state of the own vehicle V1 at the destination of the movement is changed. In (c) of Fig. Figure 28 shows an example where the vehicle's own symbol V1 is flashing, but a color change, a size change, a position change, or the like can be performed. When the overtaking maneuver is completed, as in (d) of Fig. As shown in 28, the display state of the own vehicle V1 is changed back to its original display state.

[0085] Fig. Figure 29 is a diagram that presents a third display example of how an own vehicle icon is controlled while the own vehicle icon is being dropped and the own vehicle is completing the overtaking maneuver. In a case where a driver wants the own vehicle to overtake a vehicle ahead, as in (a) of Fig. As shown in (b) 29, the driver pulls their own vehicle symbol V1 and places it as shown in (b) Fig. Figure 29 shows the own vehicle symbol V1 in a position in front of the other vehicle symbol V2. During the period in which the own vehicle symbol V1 is placed in front of the other vehicle symbol V2 and the own vehicle completes the overtaking maneuver, a next instruction is queued as an additional operation. (A control to be carried out after the completion of a current control is reserved.) In (c) and (d) of Fig. Figure 29 shows an example in which a lane change is additionally instructed during an overtaking maneuver. In this case, although the overtaking maneuver is as follows: Lane change to an overtaking lane → Overtake another vehicle → Lane change to the original lane → Lane change to an overtaking lane, the last two maneuvers are omitted, and thus the overtaking maneuver is as follows: Lane change to an overtaking lane → Overtake another vehicle.

[0086] Fig. Figure 30 is a diagram that presents a fourth display example of how an own vehicle icon is controlled while the own vehicle icon is being dropped and the own vehicle is completing the overtaking maneuver. In a case where a driver wants the own vehicle to overtake a vehicle ahead, as in (a) of Fig. As shown in (b) 30, the driver pulls their own vehicle symbol V1 and places it as shown in (b) Fig. As shown in (30), the own vehicle symbol V1 is positioned in front of the other vehicle symbol V2. During the period in which the own vehicle symbol V1 is placed in front of the other vehicle symbol V2 and the own vehicle completes the overtaking maneuver, as shown in (c) and (d) of Fig.As shown in Figure 30, if the vehicle's own symbol V1 is dragged and placed back in its original position, the following processing occurs. If the vehicle's own symbol V1 is placed back in its original position, a command is sent instructing a lane change to the original lane if a lane change to an overtaking lane is initiated or completed. If the vehicle's own symbol V1 is placed back in its original position, an error message is displayed if the vehicle is overtaking another vehicle, if the vehicle has overtaken another vehicle, or if the vehicle is in the process of changing lanes back to the original lane (e.g., "Since a control operation is currently in progress, the operation cannot be carried out." or "Please wait a moment").“or the like.” While dragging or dropping a vehicle symbol, or during a period in which a vehicle symbol is dropped and the vehicle completes the overtaking maneuver, a predicted path is received from the automatic driving control 20, and the predicted path can be drawn on the display unit 31. In a case where overtaking is temporarily blocked because the vehicle is waiting for the traffic light to change, or in a no-overtaking zone, the generation unit 12 displays a message such as “Attempting” until the overtaking control is established and can draw a predicted path at a time when processing becomes possible.While dragging a vehicle icon, or while a vehicle icon is being placed and the vehicle is completing the overtaking maneuver, the generating unit 12 can display an estimated time required or an estimated time remaining until the vehicle icon is placed and the vehicle reaches the destination position. In a case where the overtaking control is temporarily locked, the continuation of the control can be preset until the control is released or deferred and stopped. Because in some cases the dragging is started accidentally, or a cancellation operation is required after the dragging, a cancellation drop area can be provided. If a driver places their vehicle icon in a cancellation drop area after dragging it, the issuance of a command can be canceled.After overtaking control, in a case where another vehicle is in front of one's own vehicle, a way to control a repeat overtaking maneuver may be provided.

[0087] As described above, according to the present embodiment, it is possible to transfer the content of various processes to the automatic driving control 20 by moving a symbol displayed on the touchscreen using a gesture. The gesture control of the symbol is a simple process, thus relieving the driver of prior art driving controls such as turning the steering wheel 51, pressing the accelerator pedal 53, or the like. For example, it is easy to instruct an overtaking maneuver by displaying a schematic diagram containing a lane, a vehicle symbol, and another vehicle symbol, and by changing the positional relationship between the vehicle symbol and the other vehicle symbol. A driver can simultaneously confirm an environmental situation and issue a process instruction on the touchscreen, and thus the driver's line of sight remains unchanged.Accordingly, it is possible to reduce the possibility of accidental operation and achieve safer driving. To issue a different command for instructing the driver to pass or overtake than in the procedure described above, a process for changing the relative positions of the driver's own vehicle icon and the other vehicle can be used. Although a gesture control corresponding to a control command is described as a drag-and-drop operation or the like, a touch-and-touch operation can be used. Preferably, a predefined gesture or operation is used, but an operation that can be customized by a driver can be used. Furthermore, to indicate the corresponding relationship between a gesture control and a control command, a comment, symbol, or arrow can be displayed by the display unit, or a guide indicator or voice guidance can be provided by the display unit.

[0088] As above, the present disclosure is described on the basis of embodiments. These embodiments are presented only as examples, and it should be understood by those skilled in the art that the embodiments can be modified in various ways by combining the respective elements or processing operations, and that the examples of modifications are included within the scope of the present disclosure.

[0089] For example, although an example is assumed in which the MMI control 10 is implemented by an associated LSI, functions of the MMI control 10 can be implemented using a central processing unit (CPU) in a portable device, such as a smartphone or a tablet used as a display device 30. In this case, a portable device used as a display device 30 and an automatic driving control 20 are directly connected. Functions of the MMI control 10 can be implemented by a CPU in a main unit, such as a car navigation system, an audiovisual device, a portable terminal device, or the like.

[0090] An associated LSI, on which the MMI control unit 10 is mounted, can be contained in a main unit.

[0091] The embodiments can be characterized by the following points. [Point 1]

[0092] A driver assistance device (10) comprises: an image output unit (14a) that outputs to a display unit (31) an image containing a vehicle object representing the vehicle itself and another vehicle object representing another vehicle in front of the vehicle itself; an operating signal input unit (14b) that receives an operation from a user to change the positional relationship between the vehicle itself and the other vehicle object in the image displayed on the display unit (31); and a command output unit (14c) that issues a command to an automatic driving control unit (20) that controls the automatic driving, instructing the vehicle itself to overtake the other vehicle when the positional relationship between the vehicle itself and the other vehicle object is changed such that the vehicle itself is placed in front of the other vehicle object.

[0093] In this case, a user can intuitively and conveniently perform an operation to instruct an overtaking maneuver. [Point 2]

[0094] In the driver assistance device (10) of point 1, when the control signal input unit (14b) receives a user operation as described below, the command output unit (14c) issues a command to the automatic driving control unit (20) to instruct the own vehicle to overtake the other vehicle. The user operation is an operation to move the own vehicle object in the image displayed on the display unit (31) to a position in front of the other vehicle object, an operation to move the other vehicle object to a position behind the own vehicle object, or an operation to swap the own vehicle object and the other vehicle object.

[0095] In this case, a user can intuitively and conveniently perform an operation to instruct an overtaking maneuver. [Point 3]

[0096] In the driver assistance device (10) of point 1, the image output unit (14a) outputs an image containing the vehicle's own object and a multitude of other vehicle objects representing a multitude of other vehicles in front of the vehicle's own object. When the control signal input unit (14b) receives an operation from the user to change the positional relationships between the vehicle's own object and the multitude of other vehicle objects in the image displayed on the display unit (31), such that the vehicle's own object is placed in front of the multitude of other vehicle objects, the command output unit (14c) issues a command to the automatic driving control unit (20) to instruct the vehicle's own vehicle to overtake the multitude of other vehicles.

[0097] In this case, it is possible to intuitively and conveniently perform a process to instruct your own vehicle to overtake the multitude of other vehicle objects all at once. [Point 4]

[0098] A driver assistance system (10, 30) according to point 4 includes: a display device (30) that displays an image; and a driver assistance device (10) that outputs an image to the display device (30).The driving assistance device (10) includes: an image output unit (14a) that outputs to the display device (30) an image containing a vehicle object representing the vehicle itself and another vehicle object representing another vehicle in front of the vehicle itself; an operating signal input unit (14b) that receives an operation from a user to change the positional relationship between the vehicle itself and the other vehicle object in the image displayed on the display device (30); and a command output unit (14c) that issues a command to an automatic driving control unit (20) that controls the automatic driving, instructing the vehicle itself to overtake the other vehicle when the positional relationship between the vehicle itself and the other vehicle object is changed such that the vehicle itself is placed in front of the other vehicle object.

[0099] In this case, a user can intuitively and conveniently perform an operation to instruct an overtaking maneuver. [Point 5]

[0100] A driver assistance procedure according to point 5 includes: a step of outputting to a display unit (31) an image containing the own vehicle object, representing the own vehicle, and another vehicle object, representing another vehicle in front of the own vehicle object; a step of receiving an operation from a user to change the positional relationship between the own vehicle object and the other vehicle object in the image displayed on the display unit (31); and a step of outputting to an automatic driving control unit (20), which controls the automatic driving, a command to instruct the own vehicle to overtake the other vehicle when the positional relationship between the own vehicle object and the other vehicle object is changed such that the own vehicle object is placed in front of the other vehicle object.

[0101] In this case, a user can intuitively and conveniently perform an operation to instruct an overtaking maneuver. [Point 6]

[0102] Driving assistance program according to point 6, which causes a computer to execute: processing to output an image to a display unit (31) containing a vehicle object representing the vehicle itself and another vehicle object representing another vehicle in front of the vehicle itself; processing to receive an operation from a user to change the positional relationship between the vehicle itself and the other vehicle object in the image displayed on the display unit (31); and processing to output to an automatic driving control unit (20) that controls automatic driving, a command to instruct the vehicle itself to overtake the other vehicle when the positional relationship between the vehicle itself and the other vehicle object is changed such that the vehicle itself is placed in front of the other vehicle object.

[0103] In this case, a user can intuitively and conveniently perform an operation to instruct an overtaking maneuver. [Point 7]

[0104] A vehicle (1) for automated driving according to point 7 comprises: an image output unit (14a) which outputs to a display unit (31) an image which contains a vehicle object representing the vehicle itself and another vehicle object representing another vehicle in front of the vehicle itself; an operating signal input unit (14b) which receives an operation from a user to change the positional relationship between the vehicle itself and the other vehicle object in the image displayed on the display unit (31); and a command output unit (14c) which issues a command to instruct the vehicle itself to overtake the other vehicle when the positional relationship between the vehicle itself and the other vehicle object is changed such that the vehicle itself is placed in front of the other vehicle object; and an automatic driving control unit (20) which executes the issued command.

[0105] In this case, a user can intuitively and conveniently perform an operation to instruct an overtaking maneuver.

[0106] The present disclosure can be used in a vehicle equipped with an automatic driving mode. 1 vehicle 10 Driver assistance device (MMI control) 11 Unit of determination 12 production units 13 Instruction Unit 14 Input / Output Unit 20 Automatic driving control device (automatic driving control) 21 Control unit 22 storage units 23 Input / Output Unit 30 Display device 31 Display unit 32 Input unit 40 recording units 41 Position Information Acquisition Unit 42 Sensor 43 Speed ​​Information Acquisition Unit 44 Map Information Acquisition Unit 50 driving unit 51 Steering 52 Brake pedal 53 Accelerator pedal 54 Turn signal switches

Claims

Driving assistance device (10), comprising: an image output unit which outputs to a display unit (31) an image which contains a vehicle object representing a vehicle (1) and another vehicle object representing another vehicle in front of the vehicle object; an operating signal input unit which receives an operation from a user to change a positional relationship between the vehicle object and the other vehicle object in the image displayed on the display unit (31) when a receive mode of the driving assistance device (10) is set to a receive enable mode in which an operation from the user can be received;and a command output unit that issues a command to an automatic driving control unit (21) that controls the automatic driving, instructing the vehicle to overtake the other vehicle when the positional relationship between the own vehicle object and the other vehicle object is changed by the user's operation of the control signal input unit (32) such that the own vehicle object is placed in front of the other vehicle object, wherein the receive mode is switched from receive enable mode to receive block mode in which a user operation cannot be received when the command is determined and an initial time has elapsed since the receipt of the user's control signal. Driving assistance device (10) according to claim 1, wherein when the operating signal input unit (32) receives an operation from the user to move their own vehicle object in the image displayed on the display unit (31) to a position in front of the other vehicle object, an operation from the user to move the other vehicle object to a position behind their own vehicle object, or an operation from the user to exchange their own vehicle object and the other vehicle object, the command output unit issues a command to the automatic driving control unit (21) to instruct the vehicle to overtake the other vehicle. Driving assistance device (10) according to claim 1, wherein the image output unit (14) outputs an image containing the vehicle's own object and a plurality of other vehicle objects representing a plurality of other vehicles in front of the vehicle's own object, and when the operating signal input unit (32) receives an operation from the user to change a positional relationship between the vehicle's own object and the plurality of other vehicle objects in the image displayed on the display unit (31), the command output unit issues a command to the automatic driving control unit (21) to instruct the vehicle's own vehicle to overtake the plurality of other vehicles. Driving assistance method comprising: a step of outputting to a display unit (31) an image containing a vehicle object representing a vehicle and another vehicle object representing a vehicle in front of the vehicle object; a step of receiving an operation from a user to change a positional relationship between the vehicle object and the other vehicle object in the image displayed on the display unit (31), when a receive mode of the driving assistance device (10) is set to a receive enable mode in which an operation by the user can be received;and a step of issuing to an automatic driving control unit (21) that controls automatic driving, a command to instruct the own vehicle to overtake the other vehicle when the positional relationship between the own vehicle object and the other vehicle object is changed by the user's operation such that the vehicle object is placed in front of the other vehicle object, whereby the receive mode is changed from the receive enable mode to a receive block mode in which an actuation from the user cannot be received when the command is determined and an initial time has elapsed after the user's operating signal has been received.; Driving assistance system comprising: a display device (30) that displays an image; and a driving assistance device (10) that outputs the image to the display device (30), wherein the driving assistance device (10) includes: an image output unit that outputs to the display device (30) an image containing its own vehicle object, representing its own vehicle, and another vehicle object, representing another vehicle in front of its own vehicle object; an operating signal input unit (14) that receives an operation from a user to change a positional relationship between the vehicle object and the other vehicle object in the image displayed on the display device (30), when a receive mode of the driving assistance device (10) is set to a receive enable mode in which an operation by the user can be received;and a command output unit (14) which issues a command to an automatic driving control unit (20) which controls the automatic driving, instructing the own vehicle to overtake the other vehicle when the positional relationship between the vehicle object and the other vehicle object is changed by the user's operation of the control signal input unit (14) such that the vehicle object is placed in front of the other vehicle object, whereby the receive mode is switched from receive enable mode to receive block mode in which an operation by the user cannot be received when the command is determined and an initial time has elapsed since the receipt of the user's control signal. Driving assistance device according to claim 1, wherein the receive mode is changed from the receive lock mode to the receive enable mode when (i) a second time has elapsed from the time at which the receive mode is changed from the receive enable mode to the receive lock mode, (ii) a notification indicating the completion of the control according to the command is received by the automatic driving control unit (21), or (iii) it is determined from the behavior of the vehicle that the control according to the command is completed. Driving assistance method according to claim 4, wherein the receive mode is changed from the receive lock mode to the receive enable mode when (i) a second time has elapsed from the time at which the receive mode is changed from the receive enable mode to the receive lock mode, (ii) a notification indicating the completion of the control according to the command is received by the automatic driving control unit (21), or (iii) it is determined from the behavior of the vehicle that the control according to the command is completed. Driving assistance system according to claim 5, wherein the receive mode is changed from the receive lock mode to the receive enable mode when (i) a second time has elapsed from the time at which the receive mode is changed from the receive enable mode to the receive lock mode, (ii) a notification indicating the completion of the control according to the command is received by the automatic driving control unit (21), or (iii) it is determined that the control according to the command has been completed based on the behavior of the vehicle.

Images