Driving assistance devices

Abstract

To provide a technique that enables a driver to learn or understand a driving-support function.SOLUTION: A driving support device comprises: an operating part that receives operation by a driver concerning driving support of a vehicle; a driving control part that executes driving support on the basis of the operation by the driver; a memorizing part that memorizes an explanation of the driving support corresponding to the operating part; a notifying part that provides the explanation of the driving support to the driver by any one method of an augmented reality, a mixed reality and a virtual reality; and a control part connected to the operating control part, the notifying part and the memorizing part. The control part executes a running mode in which the vehicle is made to run while the driving control part is executing the driving support on the basis of operation by the driver and an explanation mode in which the notifying part is operated to provide the explanation of the driving support memorized in the memorizing part to the driver, without reflecting the operation by the driver on running of the vehicle, while switching between the modes.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a driving assistance device. [Background technology]

[0002] Patent Document 1 discloses a computer-based system that assists users in virtually experiencing the effects of assistance functions installed in a vehicle. This system includes a server owned by a business entity that manufactures or sells the vehicle, and a client terminal used by users who are potential customers of the vehicle. The client terminal communicates with the server and displays images that a driver sitting in the driver's seat of the vehicle will see while driving. The server accepts mouse operations on the client terminal and displays images on the client terminal showing the assistance functions installed in the vehicle in operation. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2002-230587 Summary of the Invention [Problem to be solved by the invention]

[0004] In the system described in Patent Document 1, the user cannot have a simulated experience using a real vehicle. As a result, the user may not be able to fully understand the assistance functions. The present disclosure provides a driving assistance device that allows the driver to learn or understand the driving assistance functions. [Means for solving the problem]

[0005] A driving assistance device according to one embodiment of the present disclosure includes an operation unit, a driving control unit, a memory unit, an alarm unit, and a control unit. The operation unit accepts driver operations related to vehicle driving assistance. The driving control unit performs driving assistance based on the driver's operation. The memory unit stores an explanation of the driving assistance corresponding to the operation unit. The alarm unit provides the driver with an explanation of the driving assistance using any one of augmented reality, mixed reality, and virtual reality. The control unit is connected to the driving control unit, the alarm unit, and the memory unit. The control unit switches between a driving mode in which the driving control unit drives the vehicle while performing driving assistance based on the driver's operation, and an explanation mode in which the alarm unit operates to provide the driver with an explanation of the driving assistance stored in the memory unit without reflecting the driver's operation in the driving of the vehicle.

[0006] This driving assistance device switches between a driving mode and an explanation mode. In the driving mode, the vehicle drives while the driving control unit performs driving assistance based on the driver's operation. In the explanation mode, the driver's operation is not reflected in the vehicle's driving, and an explanation of the driving assistance is provided to the driver using one of augmented reality, mixed reality, and virtual reality. In this way, in the explanation mode, even if the driver operates the operating units of the actual vehicle, this is not reflected in the driving, so that the driving assistance can be explained while the driver operates the operating units of the actual vehicle. This allows the driving assistance device to allow the driver to learn or understand the driving assistance function.

[0007] In one embodiment, the notification unit may provide the driver with an explanation of the driving assistance in augmented reality or mixed reality. In this case, the driving assistance device may superimpose an explanatory image or three-dimensional object corresponding to the operation unit that the driver is looking at, thereby enabling the driver to more easily learn or understand the driving assistance function.

[0008] In one embodiment, the explanation of the driving assistance corresponding to the operation unit may include at least one of an operation method and an operation timing of the operation unit. In this case, the driving assistance device can allow the driver to learn or understand at least one of the operation method and the operation timing of the operation unit.

[0009] In one embodiment, the control unit may operate the notification unit to superimpose an operation guide for the operation unit on an image of the operation unit, or to superimpose an operation guide for the operation unit in the field of view of the driver who sees the operation unit. By presenting the operation guide to the driver, the driving assistance device can allow the driver to further learn or understand how to operate the operation unit and the operation timing.

[0010] In one embodiment, the control unit may operate the notification unit to suggest driving assistance that operates in the explanation mode to the driver. The driving assistance device can allow the driver to learn or understand driving assistance functions that the driver is not familiar with. [Effects of the Invention]

[0011] According to the present disclosure, it is possible to allow a driver to learn or understand driving assistance functions. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a vehicle equipped with a driving assistance device according to an embodiment. [Figure 2] FIG. 2 is a flowchart showing the operation of the driving assistance device. [Figure 3] FIG. 3 is a flowchart showing another operation of the driving assistance device. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0014] [Vehicle configuration] Fig. 1 is a block diagram showing an example of the configuration of a vehicle equipped with a driving assistance device according to an embodiment. As shown in Fig. 1, the driving assistance device 1 is mounted on a vehicle 2, as an example. The vehicle 2 is a vehicle to which driving assistance is provided. The driving assistance may be a state in which driving operations, including braking, accelerating, and steering, are partially automated, a state in which all driving tasks are automated on the condition that preset driving conditions are met, or a state in which all driving tasks are automated without setting any conditions.

[0015] Examples of driving assistance include lane keeping functions, vehicle distance control functions, starting functions, stopping functions, lane changing functions, right and left turning functions, collision prevention functions, and parking functions.

[0016] The vehicle 2 includes a driving ECU 22 for executing driving assistance. The ECU (Electronic Control Unit) is an electronic control unit having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a CAN (Controller Area Network) communication circuit, etc. The driving assistance is realized by the driving ECU 22 operating actuators 23 of the vehicle 2 based on at least the detection results of the sensor group 21.

[0017] The sensor group 21 includes, for example, sensors that detect the surrounding environment of the vehicle 2 and sensors that detect the traveling state of the vehicle 2. The sensors that detect the surrounding environment of the vehicle 2 include, for example, at least one of a camera and a radar sensor. The sensors that detect the traveling state of the vehicle 2 include, for example, a GPS receiver, a vehicle speed sensor, an acceleration sensor, and a yaw rate sensor.

[0018] The actuators 23 are devices used for controlling the running of the vehicle 2. The actuators 23 include at least a drive actuator, a brake actuator, and a steering actuator.

[0019] The driving ECU 22 executes driving assistance based on an operation by the driver. The driver's operation is not particularly limited as long as it is related to the driving assistance function. For example, the driver's operation may be a start operation or a stop operation for starting and stopping the driving assistance function, or may be an operation related to the driving of the vehicle 2, such as a brake operation, an accelerator operation, or a steering operation during driving assistance. The driving ECU 22 may simply execute driving assistance based on the start operation, or may reflect the driver's brake operation, accelerator operation, steering operation, or the like on the driving of the vehicle 2 during driving assistance.

[0020] [Configuration of driving assistance device] In addition to the driving ECU 22 described above, the driving assistance device 1 includes an operation unit 3, a storage unit 4, a notification ECU 5 (an example of a control unit), and a notification unit 6, for example.

[0021] The operation unit 3 receives driver operations related to driving assistance of the vehicle 2. The operation unit 3 includes, for example, operation buttons related to driving assistance, a brake pedal, an accelerator pedal, a steering wheel, and the like.

[0022] The memory unit 4 is a storage device that stores a description of the driving assistance corresponding to the operation unit 3. The description of the driving assistance corresponding to the operation unit 3 includes at least one of the operation method and operation timing of the operation unit 3. The description of the driving assistance may be information for performing the driving assistance function, such as the operation method and operation timing of a button for starting driving assistance, the operation method and operation timing of a button for ending driving assistance, and the operation method and operation timing of a button for changing a set speed, or information about settings related to the driving assistance. The description of the driving assistance may be content that explains, for example, how the driver's operation during driving assistance is reflected in the traveling of the vehicle 2. The description of the driving assistance is prepared corresponding to the operation unit 3. For example, if the operation unit 3 is an operation button, the description of the driving assistance will be a description of the operation button. Alternatively, if the operation unit 3 is a brake pedal, the description of the driving assistance will be a description of the brake pedal.

[0023] The notification unit 6 is a device that notifies the driver of information. The notification unit 6 provides the driver with an explanation of driving assistance using, for example, any one of augmented reality, mixed reality, and virtual reality. The notification unit 6 provides the driver with information using, for example, augmented reality. Augmented reality is a technology that overlays digital information on the real world. More specifically, the notification unit 6 acquires an image (including video) from a camera that is the same as the driver's line of sight, synthesizes information or content with the image, and provides the driver with an image in which the information is overlaid on the real world. The notification unit 6 may superimpose operation guidance for the operation unit 3 on the image of the operation unit 3. The operation guidance is information that instructs the driver on how to move the operation unit 3. By providing operation guidance, understanding of the operation unit 3 is improved.

[0024] Alternatively, the notification unit 6 may be provided with a transparent display, and may display information or content on the display to the driver viewing the operation unit 3 through the display, superimposing the information on the real world. The notification unit 6 may superimpose operation guidance for the operation unit 3 in the field of view of the driver viewing the operation unit 3. Providing operation guidance improves understanding of the operation unit 3.

[0025] The notification unit 6 may provide information to the driver using, for example, mixed reality. Mixed reality is a technology that overlays digital information on the real world, similar to augmented reality, and is a technology that can provide the driver with more realistic information than augmented reality, such as three-dimensional objects. Mixed reality may overlay information on an image, similar to augmented reality, or may display information on a transparent display that the driver looks through. In this case, the notification unit 6 is an MR (Mixed Reality) device that can provide augmented reality.

[0026] The notification unit 6 may provide information to the driver in, for example, virtual reality. Virtual reality is a technology that provides a virtual space and allows a simulated experience of the real world. When providing information in virtual reality, the notification unit 6 is a VR device (Virtual Reality device). Various simulations are possible in virtual reality. The driver may operate the operation unit 3 while in virtual reality.

[0027] The notification unit 6 may be a head-mounted display that can be worn on the driver's head. This allows the driver to sit in the driver's seat even in a small vehicle interior and experience switch operation and vehicle behavior. In this way, the reality of the information provided by any one of augmented reality, mixed reality, and virtual reality is improved.

[0028] The notification ECU 5 is connected to the operation ECU 22, the operation unit 3, the storage unit 4, and the notification unit 6. The notification ECU 5 is an electronic control unit having a CPU, a ROM, a RAM, a CAN communication circuit, etc. The notification ECU 5 may be an ECU formed by integrating multiple ECUs.

[0029] The notification ECU 5 switches between a driving mode and an explanation mode. For example, the notification ECU 5 switches between the driving mode and the explanation mode based on an instruction from the driver. The driving mode is a mode in which the vehicle 2 is driven while the driving ECU 22 executes driving assistance based on the driver's operation. In this case, the notification ECU 5 outputs all of the driver's instructions received by the operation unit 3 to the driving ECU 22. This enables the driving ECU 22 to acquire the driving assistance start operation, setting operation, steering operation during driving assistance, etc., and reflect them in the driving of the vehicle 2.

[0030] The explanation mode is a mode in which the driver's operation is not reflected in the traveling of the vehicle 2, and the notification unit 6 is operated to provide the driver with an explanation of the driving assistance stored in the memory unit 4. In other words, when the explanation mode is selected, the operation buttons related to the driving assistance are operable but disabled, and the driver's instructions received by the operation unit 3 are not output to the driving ECU 22. Alternatively, the driver's instructions received by the operation unit 3 may be output to the driving ECU 22, but the actuator 23 may be prohibited from operating for driving assistance using by-wire technology.

[0031] Then, the notification ECU 5 provides the driver with an explanation of the driving assistance stored in the storage unit 4. When the driver recognizes the operation unit 3 by actually looking at the operation unit 3, the notification ECU 5 provides the driver with an explanation of the driving assistance corresponding to the operation unit 3 by using augmented reality or mixed reality. When the driver recognizes the operation unit 3 by looking at an image of the operation button, the notification ECU 5 provides the driver with an explanation of the driving assistance corresponding to the operation unit 3 by using augmented reality or mixed reality. When the driver recognizes the operation button by looking at the operation button created in a virtual space, the notification ECU 5 provides the driver with an explanation of the driving assistance corresponding to the operation unit 3 in virtual reality.

[0032] The notification ECU 5 may use by-wire technology to apply a reaction force to the operation unit 3 to improve reality. Furthermore, the notification ECU 5 may use by-wire technology to output signals from the accelerator, brake, steering wheel, shift, etc. to the notification unit 6, without adding any sensors or output devices, and provide feedback to the driver by creating an image and generating a reaction force. This allows the driver to experience driving assistance while operating the operation unit 3 in the real world or a virtual space that is close to reality.

[0033] The notification ECU 5 may operate the notification unit 6 to suggest to the driver a driving assistance function that operates in explanation mode. For example, the notification ECU 5 stores a history (usage record) of driving assistance functions that operated in explanation mode. As an example, the notification ECU 5 stores the history of driving assistance functions that operated in explanation mode in the memory unit 4. The notification ECU 5 refers to the memory unit 4 and determines a driving assistance function that has been executed less frequently from the driving assistance history as the driving assistance function to recommend. This allows the driver to learn or understand even driving assistance functions that the driver is not familiar with. Alternatively, the notification ECU 5 may compile a questionnaire about the driver's understanding in explanation mode and determine a driving assistance function that the driver does not understand as the driving assistance function to recommend. This makes it possible to efficiently compensate for the driver's lack of understanding.

[0034] [Operation of driving assistance device] 2 is a flowchart showing the operation of the driving assistance device 1. The flowchart shown in FIG. 2 starts when the driving assistance device 1 receives a start instruction operation.

[0035] 2, first, the notification ECU 5 of the driving assistance device 1 determines whether or not an instruction for the explanation mode has been received (step S10). If the notification ECU 5 determines that an instruction for the explanation mode has been received via the operation unit 3 or the like (step S10: YES), the notification ECU 5 operates in the explanation mode (step S12). In step S12, the notification ECU 5 does not reflect the driver's operation in the traveling of the vehicle 2, and operates the notification unit 6 to provide the driver with an explanation of the driving assistance stored in the memory unit 4.

[0036] If it is determined that the instruction for the explanation mode has not been received (step S10: NO), the notification ECU 5 operates in the traveling mode (step S14). In step S14, the notification ECU 5 outputs all of the driver's instructions received by the operation unit 3 to the driving ECU 22, causing the driving ECU 22 to perform the driving assistance function.

[0037] When step S12 or step S14 is completed, the flowchart shown in Fig. 2 is completed. In this manner, the driving mode and the explanation mode are switched and executed.

[0038] Next, another operation of the driving assistance device will be described. Fig. 3 is a flowchart showing another operation of the driving assistance device. The flowchart shown in Fig. 3 starts when the driving assistance device 1 receives a start instruction operation.

[0039] 3, first, the notification ECU 5 of the driving assistance device 1 refers to the storage unit 4 and determines the driving assistance to be recommended based on the usage history of the driving assistance (step S20). Next, the notification ECU 5 proposes the recommended driving assistance determined in step S20 to the driver (step S22). For example, the notification ECU 5 proposes the recommended driving assistance to the driver via the notification unit 6.

[0040] Next, the notification ECU 5 determines whether the driver has accepted the suggestion (step S24). If the notification ECU 5 determines that the driver has accepted the suggestion (step S24: YES), the notification ECU 5 operates in explanation mode (step S26). In step S26, the notification ECU 5 does not reflect the driver's operation in the traveling of the vehicle 2, and operates the notification unit 6 to provide the driver with an explanation of the recommended driving assistance.

[0041] When step S26 is completed, or when it is determined that the driver does not accept the proposal (step S24: NO), the flowchart shown in Fig. 3 ends. In this way, the driving assistance explained in the explanation mode is proposed.

[0042] [Summary of the embodiment] The driving assistance device 1 switches between a driving mode and an explanation mode. In the driving mode, the vehicle 2 drives while the driving ECU 22 performs driving assistance based on the driver's operation. In the explanation mode, the driver's operation is not reflected in the driving of the vehicle 2, and an explanation of the driving assistance is provided to the driver using one of augmented reality, mixed reality, and virtual reality. In this way, in the explanation mode, even if the driver operates the operation unit 3 of the actual vehicle 2, it is not reflected in the driving. Therefore, the driving assistance can be explained while the driver operates the operation unit 3 of the actual vehicle 2. This allows the driving assistance device 1 to help the driver learn or understand the driving assistance functions. A video explanation from a third-person perspective may differ from the image of the actual driving behavior. Furthermore, a generic video common to all vehicle models may have a different switch arrangement from that of the actual vehicle purchased, making operation difficult. The driving assistance device 1 allows the driver to understand and experience the functions while operating the actual vehicle.

[0043] Although exemplary embodiments have been described above, various omissions, substitutions, and modifications may be made without being limited to the above exemplary embodiments. For example, the driving assistance for the vehicle 2 does not exclude fully automated driving, and the vehicle 2 may perform a simulation of an emergency avoidance scenario in which the driver attempts to avoid a vehicle emergency during fully automated driving.

[0044] The storage unit 4 may be a cache (temporary storage medium) or a communication buffer used when exchanging information via communication. In other words, the driving assistance device 1 does not need to store some or all of the explanation of the driving assistance on the vehicle 2 side, and the explanation of the driving assistance may be provided to the driving assistance device 1 via communication as needed. Alternatively, the storage unit 4 may be integrated with the storage unit of the notification ECU 5. The explanation of the driving assistance stored in the storage unit 4 may be updated wirelessly or via a wired connection.

[0045] This disclosure also includes the following provisions: [Article 1] an operation unit that receives driver operations related to vehicle driving assistance; a driving control unit that executes the driving assistance based on an operation of the driver; a storage unit that stores an explanation of the driving assistance corresponding to the operation unit; a notification unit that provides an explanation of the driving assistance to the driver using any one of augmented reality, mixed reality, and virtual reality; a control unit connected to the operation control unit, the notification unit, and the storage unit; Equipped with The control unit a driving mode in which the vehicle is driven while the driving control unit executes the driving assistance based on the operation of the driver, and an explanation mode in which the notification unit is operated so as to provide the driver with an explanation of the driving assistance stored in the storage unit without reflecting the operation of the driver in the driving of the vehicle; Driving assistance device. [Clause 2] The driving assistance device according to clause 1, wherein the notification unit provides an explanation of the driving assistance to the driver in augmented reality or mixed reality. [Article 3] The driving assistance device according to clause 1 or 2, wherein the explanation of the driving assistance corresponding to the operation unit includes at least one of an operation method and an operation timing of the operation unit. [Article 4] The driving assistance device according to any one of clauses 1 to 3, wherein the control unit operates the notification unit to superimpose operation guidance for the operation unit on an image of the operation unit, or to superimpose operation guidance for the operation unit in the field of view of the driver who visually recognizes the operation unit. [Article 5] The driving assistance device according to any one of clauses 1 to 4, wherein the control unit operates the notification unit to suggest to the driver driving assistance that operates in the explanation mode. [Explanation of symbols]

[0046] 1... driving assistance device, 2... vehicle, 3... operation unit, 4... memory unit, 5... notification ECU (an example of a control unit), 6... notification unit, 22... driving ECU

Claims

1. an operation unit that receives driver operations related to vehicle driving assistance; a driving control unit that executes the driving assistance based on an operation of the driver; a storage unit that stores an explanation of the driving assistance corresponding to the operation unit; a notification unit that provides the driver with an explanation of the driving assistance in a space seen from the driver's viewpoint using any one of augmented reality, mixed reality, and virtual reality; a control unit connected to the operation control unit, the notification unit, and the storage unit; Equipped with The control unit a driving mode in which the vehicle is driven while the driving control unit executes the driving assistance based on the operation of the driver, and an explanation mode in which the notification unit is operated so as to provide the driver with an explanation of the driving assistance stored in the storage unit without reflecting the operation of the driver in the driving of the vehicle; Driving assistance device.

2. The driving assistance device according to claim 1 , wherein the notification unit provides an explanation of the driving assistance to the driver in augmented reality or mixed reality.

3. The driving assistance device according to claim 1 , wherein the explanation of the driving assistance corresponding to the operation unit includes at least one of an operation method and an operation timing of the operation unit.

4. 3. The driving assistance device according to claim 1, wherein the control unit operates the notification unit to superimpose an operation guide for the operation unit on an image of the operation unit, or to superimpose an operation guide for the operation unit in the field of view of the driver who visually recognizes the operation unit.

5. The driving assistance device according to claim 1 , wherein the control unit operates the notification unit to suggest to the driver the driving assistance that operates in the explanation mode.

Images