Driving assistance devices

Abstract

To provide driving assistance with improved safety for vehicle driving assistance devices. [Solution] A driving assistance device (10) can perform an approach warning to warn a driver of a traveling vehicle (90) that the vehicle is approaching an obstacle, and can perform avoidance control to drive the vehicle (90) so as to avoid contact between the vehicle and the obstacle. The driving assistance device (10) includes a visual recognition determination unit (21) that determines whether the driver is in a reduced visual recognition state, in which the driver's visual recognition is reduced, an auditory recognition determination unit (22) that determines whether the driver is in a reduced auditory recognition state, in which the driver's auditory recognition is reduced, and an early warning control unit (23) that performs early warning processing. When the driver is in at least one of the reduced visual recognition state and the reduced auditory recognition state, the early warning processing performs a warning at a timing earlier than that for an approach warning, using at least one of a display device (51), an audio device (52), and a vibration device (53), depending on the driver's state.

Description

[Technical Field]

[0001] The present invention relates to a driving assistance device for a vehicle. [Background technology]

[0002] Patent Document 1 discloses a device configured to automatically operate the brake device of the vehicle to slow down or stop the vehicle when it is determined that the condition of a vehicle occupant is abnormal based on sounds inside the vehicle, such as the voices emitted by the occupant. Patent Document 1 also describes a configuration in which, when determining that the condition of a vehicle occupant is abnormal, the determination is made based on an image of the occupant. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-66299 Summary of the Invention [Problem to be solved by the invention]

[0004] The driver's awareness may be impaired by disturbances in the environment outside the vehicle, such as backlighting, noise, etc. If the driver's awareness is impaired, this may lead to actions that are at high risk of delayed judgment, misrecognition, and incorrect operation. The device disclosed in Patent Document 1 makes a determination by observing the passengers inside the vehicle, and therefore may not be able to determine whether the driver's awareness is impaired due to influences from the environment outside the vehicle.

[0005] Furthermore, if the vehicle occupants are already in an abnormal state, as determined by the device disclosed in Patent Document 1, the distance between the vehicle and the obstacle may already be short, and there may not be enough time before contact occurs.

[0006] From the viewpoint of further improving safety, it is not sufficient to simply define the state determined by the device disclosed in Patent Document 1 as an abnormal state. [Means for solving the problem]

[0007] A driving assistance device for solving the above problem is a driving assistance device that, when a traveling vehicle approaches an obstacle, issues an approach notification to notify a driver of the vehicle that the vehicle is approaching the obstacle, and performs avoidance control to drive the vehicle so as to avoid contact between the vehicle and the obstacle, wherein the vehicle has an external light detection device that detects light rays outside the vehicle, a sound collection device that detects sound outside the vehicle or inside the vehicle cabin, a display device as a first notification device that communicates warning information to the driver by acting on the driver's vision, a sound device as a second notification device that communicates warning information by acting on the driver's hearing, and a third notification device that communicates warning information to the driver by acting on a sense other than the driver's vision and hearing, The driving assistance device includes a visual recognition determination unit that determines whether the driver is in a visual recognition reduced state, in which the driver's visual recognition is reduced, based on light rays detected by the external light detection device; an auditory recognition determination unit that determines whether the driver is in an auditory recognition reduced state, in which the driver's auditory recognition is reduced, based on sounds detected by the sound collection device; and an early warning control unit that, when the driver is in at least one of the visual recognition reduced state and the auditory recognition reduced state, performs early warning processing to alert the driver at an earlier timing than the approach warning is performed by controlling at least one of the display device, the acoustic device, and the third warning device depending on the driver's state. [Effects of the Invention]

[0008] It is possible to provide driving assistance with improved safety. [Brief explanation of the drawings]

[0009] [Figure 1]FIG. 1 is a block diagram showing an embodiment of a driving assistance device and a vehicle equipped with the driving assistance device. [Figure 2] FIG. 2 is a flowchart showing the flow of processing executed by the driving assistance device of FIG. [Figure 3] FIG. 3 is a flowchart showing the flow of the early warning process executed by the driving assistance device of FIG. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of a driving assistance device will be described with reference to the drawings. FIG. 1 shows a vehicle 90 equipped with a driving assistance device 10 . The vehicle 90 is equipped with a traveling system. An example of a traveling system includes a drive unit 81, a braking unit 82, and a steering unit 83. The drive unit 81 is a power source for the vehicle 90. The braking unit 82 can apply braking force to the vehicle 90. The steering unit 83 can steer the vehicle 90. More specifically, the steering unit 83 deflects the orientation of the front-to-rear direction of the wheels to the left or right relative to the front-to-rear direction of the vehicle 90. The traveling system can be operated by the driver of the vehicle 90.

[0011] The driving assistance device 10 can automatically drive the vehicle 90 by controlling the driving system of the vehicle 90. As an example, the driving assistance device 10 can perform avoidance control to drive the vehicle 90 so as to avoid contact between the vehicle 90 and an obstacle. Details of the avoidance control will be described later.

[0012] When the traveling vehicle 90 approaches an obstacle, the driving assistance device 10 can perform an approach notification to notify the driver of the vehicle 90 that the vehicle 90 is approaching an obstacle. The approach notification will be described in detail later.

[0013] <Alarm device> The vehicle 90 is equipped with various notification devices. Examples of the notification devices equipped in the vehicle 90 include the following devices.

[0014] The vehicle 90 is equipped with a display device 51 as a first notification device that conveys warning information to the driver by acting on the driver's vision. The display device 51 can display, for example, characters, symbols, images, etc. Specific examples of the display device 51 include a display, an indicator light, etc. The display device 51 may be, for example, a display of a car navigation system or the like installed in the vehicle 90, or may be a separate device that functions as a notification device. The display device 51 may include an operation unit that can be operated by the driver in addition to a display unit having a display function. Examples of the operation unit include a touch display, a switch, etc.

[0015] The vehicle 90 is equipped with an acoustic device 52 as a second notification device that communicates warning information by affecting the driver's hearing. The acoustic device 52 can output, for example, a beep, a voice, etc. Specific examples of the acoustic device 52 include a speaker, an electronic buzzer, etc. The acoustic device 52 may be, for example, a speaker for playing music that is mounted on the vehicle 90, or may be a separate device that functions as a notification device.

[0016] The vehicle 90 is provided with a third notification device that conveys caution information to the driver by acting on a sense other than the driver's sight and hearing. An example of the third notification device is a notification device that conveys caution information to the driver by acting on the driver's sense of touch.

[0017] The vehicle 90 is equipped with, for example, a vibration device 53 as a third notification device, which notifies the driver by generating vibrations to be transmitted to the driver. The vibration device 53 can transmit vibrations to the driver, for example, using an electric motor. The vibration device 53 is attached, for example, to an operation device of the vehicle 90 that can be operated by the driver. Examples of the operation device of the vehicle 90 include the steering wheel of the vehicle 90, the accelerator pedal of the vehicle 90, and the brake pedal of the vehicle 90. The vibration device 53 may be attached to, for example, a seat on which the driver sits, an armrest, a footrest, or the like, as long as it is in contact with the driver of the vehicle 90. The vehicle 90 may be equipped with one vibration device 53 or two or more vibration devices 53. When the vehicle 90 is equipped with multiple vibration devices 53, the vibration devices 53 may be attached to two or more of the above-mentioned mounting locations.

[0018] <Detection device> The vehicle 90 is equipped with various detection devices. Examples of the detection devices equipped in the vehicle 90 include the following devices.

[0019] The vehicle 90 is equipped with an external light detection device 31. The external light detection device 31 is a device that detects light rays outside the vehicle. The external light detection device 31 can detect light rays from ahead of the vehicle 90. Examples of the external light detection device 31 include a front camera, an omnidirectional camera, and a brightness sensor. When a camera such as a front camera or an omnidirectional camera is used as the external light detection device 31, the intensity of light rays outside the vehicle can be detected by calculating the brightness of an image obtained by the camera. The external light detection device 31 may be attached to the outside of the vehicle 90 or may be attached inside the vehicle 90, as long as it can detect light rays from ahead of the vehicle 90.

[0020] The vehicle 90 may be equipped with an outdoor sound collection device 32. The outdoor sound collection device 32 can detect sounds outside the vehicle 90. The outdoor sound collection device 32 is attached to, for example, the outside of the vehicle 90. Examples of the outdoor sound collection device 32 include a microphone and a noise sensor.

[0021] The vehicle 90 may be equipped with an interior camera 41. The interior camera 41 can capture an image of the driver. The vehicle 90 may be equipped with an interior sound collection device 42. The interior sound collection device 42 can detect sounds inside the interior of the vehicle 90. An example of the interior sound collection device 42 is a microphone installed inside the interior of the vehicle 90.

[0022] The vehicle 90 may be equipped with a periphery monitoring device 71. An example of the periphery monitoring device 71 is a camera. An example of the periphery monitoring device 71 is a device such as a LiDAR or a millimeter wave radar. The periphery monitoring device 71 may be configured by one of the above devices, or may be configured by a combination of two or more devices.

[0023] When the vehicle 90 is equipped with a camera as the above-mentioned external light detection device 31, the camera can also be used as the periphery monitoring device 71. The periphery monitoring device 71 can acquire the size of structures and the like located around the vehicle 90. The periphery monitoring device 71 can also acquire the relative distance between the vehicle 90 and structures and the like located around the vehicle 90.

[0024] <Driving assistance devices> As shown in FIG. 1, the driving assistance device 10 includes a processing circuit 11. One example of the processing circuit 11 is an electronic control device. In this case, the driving assistance device 10 includes a CPU 12, a first memory 13, and a second memory 14. The first memory 13 stores a control program executed by the CPU 12. The second memory 14 stores the results of calculations by the CPU 12. The CPU 12 executes the control program in the first memory 13, which enables the processing circuit 11 to control the vehicle 90.

[0025] The processing circuit 11 functions as a plurality of functional units as a result of the CPU 12 executing the control program stored in the first memory 13. As shown in FIG. 1 , the plurality of functional units include a visual recognition determination unit 21, an auditory recognition determination unit 22, an early warning control unit 23, and a driving assistance unit 29.

[0026] <Driving Assistance Department> The driving assistance unit 29 can perform avoidance control to cause the vehicle 90 to travel automatically. The avoidance control is for controlling the traveling system of the vehicle 90 so as to avoid a collision between the vehicle 90 and an obstacle. The driving assistance unit 29 can, for example, decelerate the vehicle 90 so as to avoid contact with the obstacle. The driving assistance unit 29 can, for example, stop the vehicle 90 before contact with the obstacle. The driving assistance unit 29 can also steer the vehicle 90 so as to avoid the obstacle, for example.

[0027] The driving assistance unit 29 can start avoidance control based on a time to collision (TTC), which is the time until the vehicle 90 collides with an obstacle. The time to collision can be calculated, for example, based on the distance from the vehicle 90 to the obstacle and the relative speed of the vehicle 90 with respect to the obstacle. The driving assistance unit 29 starts avoidance control, for example, when the time to collision is shorter than an avoidance start threshold. The avoidance start threshold can be set to a value calculated in advance through experiments or the like.

[0028] The driving assistance unit 29 can issue an approach notification to notify the driver of the vehicle 90 of the approach of the vehicle 90 to an obstacle. The approach notification can be issued, for example, by controlling at least one of the above-mentioned notification devices provided in the vehicle 90. The driving assistance unit 29 issues the approach notification, for example, when the time to collision is shorter than an approach threshold. The approach threshold can be set to a value calculated in advance by experiments or the like. The approach threshold can be set to a value greater than the avoidance start threshold, for example.

[0029] <Visual Recognition Determination Unit and Auditory Recognition Determination Unit> The visual recognition determination unit 21 can determine whether or not the driver is in a state of reduced visual recognition, which is a state in which the driver's visual recognition is reduced. The visual recognition determination unit 21 can determine whether or not the driver is in a state of reduced visual recognition based on light rays detected by the external light detection device 31.

[0030] The auditory perception determination unit 22 can determine whether the driver is in a state of reduced auditory perception, which is a state in which the driver's auditory perception is reduced. The auditory perception determination unit 22 can determine whether the driver is in a state of reduced auditory perception based on the sound detected by the exterior sound collection device 32.

[0031] As used herein, the term "state of cognitive impairment" refers to a state in which cognitive impairment is temporarily impaired due to an external factor, including a state in which it is difficult to recognize the surroundings, a state in which the surroundings cannot be recognized, and a state that may result in these conditions.

[0032] For example, the state of impaired visual perception refers to a state in which the driver's eyes are dazzled by backlighting. Backlighting is a situation in which strong light rays are incident on the eyes due to a light source such as the sun in the field of view. Examples of light sources include the sun, lights from other vehicles, streetlights, etc.

[0033] For example, the state of reduced hearing awareness refers to a state in which noise makes it difficult to hear the sound of other vehicles traveling, etc. The noise includes noise outside the vehicle and noise inside the vehicle. Disturbances such as backlighting and noise are examples of external factors. It may take time for a driver who has become aware of the situation to recover from the impaired awareness. In other words, the impaired awareness may continue even after the external factors are resolved.

[0034] For example, the visual recognition determination unit 21 can determine that the driver is in a state of reduced visual recognition when the luminance based on detection by the external light detection device 31 remains higher than the external light threshold for an external light determination time or longer. The external light threshold and the external light determination time can be set to values ​​calculated in advance through experiments, etc. The visual recognition determination unit 21 acquires the luminance based on detection by the external light detection device 31 at each predetermined control cycle and repeatedly determines whether the driver is in a state of reduced visual recognition.

[0035] For example, the auditory awareness determination unit 22 can determine that the driver is in a state of reduced auditory awareness when the sound level (unit: dB) based on detection by the outdoor sound collection device 32 remains higher than the noise threshold for a noise determination time or longer. The noise threshold and the noise determination time can be set to values ​​calculated in advance through experiments, etc. The auditory awareness determination unit 22 acquires the sound level based on detection by the outdoor sound collection device 32 at each predetermined control cycle and repeatedly determines whether the driver is in a state of reduced auditory awareness.

[0036] <Early alarm control unit> When the driver is in at least one of a state of reduced visual recognition and a state of reduced auditory recognition, the early warning control unit 23 can perform early warning processing to notify the driver at an earlier timing than when an approach warning is performed.

[0037] The early warning control unit 23 can perform early warning processing by controlling at least one of the display device 51, the sound device 52, and the vibration device 53 according to the driver's state of impaired awareness.

[0038] The warning information transmitted using the display device 51 in the early warning process preferably includes information that warns the driver that their awareness may be impaired, and text information is more preferable. The warning information transmitted using the display device 51 may include information that warns the driver to pay attention to the road ahead. The warning information transmitted using the display device 51 may also be information that can distinguish between a state of impaired visual awareness and a state of impaired auditory awareness.

[0039] As an example of the early notification process, a mode in which the early notification control unit 23 performs a first notification process and a second notification process that starts after the first notification process will be described below. The early notification control unit 23 performs the second notification process when a second notification condition is met after the first notification process is performed. An example of the second notification condition is a condition that is met when the elapsed time from the start of the first notification process is equal to or greater than a transition determination time. The transition determination time is set to a value calculated in advance by an experiment or the like.

[0040] In the first notification process, the early notification control unit 23 performs notification by controlling, for example, two devices out of the display device 51, the sound device 52, and the vibration device 53. It is preferable that the early notification control unit 23 selects the display device 51 as one of the two notification devices used in the first notification process. In this case, when the driver is in a state of reduced awareness, the display device 51 is selected regardless of the type of reduced awareness state. As a result, when the driver is in a state of reduced awareness, notification is performed by controlling the display device 51 in both the first notification process and the second notification process, regardless of the type of reduced awareness state.

[0041] The early notification control unit 23 can select the notification device to be used in the first notification process depending on the driver's awareness deterioration state. For example, if the driver's visual perception is impaired, the early warning control unit 23 can select a warning device that acts on a sense other than the visual sense. In this case, the early warning control unit 23 can select the audio device 52 as one of the two warning devices used in the first warning process.

[0042] For example, if the driver's hearing is impaired, the early warning control unit 23 selects a notification device that acts on a sense other than hearing. In this case, the early warning control unit 23 selects the vibration device 53 as one of the two notification devices used in the first notification process.

[0043] In the second notification process, the early notification control unit 23 selects a notification device that was not selected in the first notification process. For example, in the second notification process, the early notification control unit 23 performs notification by controlling all of the display device 51, the sound device 52, and the vibration device 53. In other words, in the second notification process, the early notification control unit 23 selects and notifies using a notification device that was not selected in the first notification process, in addition to the notification device selected in the first notification process.

[0044] Table 1 is a correspondence table illustrating the correspondence between the driver's state and the notification means when the driving assistance device 10 selects the notification means in the early notification process.

[0045] [Table 1] In the example shown in Table 1, when the driver is in a state of reduced visual awareness but not in a state of reduced auditory awareness, the early notification control unit 23 performs the first notification process by controlling the display device 51 and the sound device 52. When the driver is in a state of reduced auditory awareness, the early notification control unit 23 performs the first notification process by controlling the display device 51 and the vibration device 53.

[0046] In the example shown in Table 1, when the early warning control unit 23 issues a warning using the vibration device 53 as the third warning device in the early warning processing, in addition to issuing a warning by controlling the vibration device 53, it also issues a warning by controlling the display device 51.

[0047] In the first notification process and the second notification process, the duration of notification by each notification device may be the same or different. In the first notification process, the duration of notification by each notification device is set, for example, to be equal to or shorter than the transition determination time. Furthermore, notifications by each notification device may start simultaneously, or one notification may be started and then another notification may be started while the first notification is being performed.

[0048] For a common notification device used in the first notification process and the second notification process, the early notification control unit 23 may continue notification by the notification device from the first notification process to the second notification process. For example, the notification by the display device 51 performed in the second notification process may be a notification that has been performed continuously since the start of the first notification process.

[0049] An example of a specific aspect of the early notification process is shown below. For example, in a state of impaired visual recognition, the early warning control unit 23 performs a first warning process by issuing a warning using the display device 51 and the audio device 52. The early warning control unit 23 simultaneously starts issuing a warning using the display device 51 and the audio device 52. The early warning control unit 23 ends the warning using the audio device 52 after a predetermined time has elapsed. On the other hand, the early warning control unit 23 continues issuing a warning using the display device 51. Thereafter, when a second warning condition is met, the early warning control unit 23 performs a second warning process. The warning using the display device 51 in the second warning process is continued from the first warning process. The early warning control unit 23 performs a second warning process by issuing a warning using the audio device 52 and the vibration device 53. The early warning control unit 23 simultaneously starts issuing a warning using the audio device 52 and the vibration device 53. The early notification control unit 23 ends the notification by the display device 51, the notification by the sound device 52, and the notification by the vibration device 53 after a predetermined time has elapsed.

[0050] The early warning control unit 23 may terminate the early warning process if an interruption condition is met during the execution of the early warning process. For example, the early warning control unit 23 may determine that the interruption condition is met if an implementation condition for approach warning is met. The early warning control unit 23 may also determine that the interruption condition is met if an implementation condition for avoidance control is met. The early warning control unit 23 may also determine that the interruption condition is met if the driver responds to the early warning process. For example, the early warning control unit 23 may determine that the interruption condition is met if the driver indicates his or her intention to accept the warning information displayed on the display device 51 by operating the operation unit. The manner of the driver's response is not particularly limited, and may be, for example, a response by voice input.

[0051] When performing the early warning process, the early warning control unit 23 can also advance the timing of implementing the avoidance control. Specifically, the early warning control unit 23 corrects the avoidance start threshold for the avoidance control to a larger value, so that the avoidance control is implemented from a point when the time to collision is longer. As a result, the timing of implementing the avoidance control can be advanced. The early warning control unit 23 may be configured to advance the timing of implementing the avoidance control when the driver is in both the visual recognition reduced state and the auditory recognition reduced state, compared to when the driver is in either the visual recognition reduced state or the auditory recognition reduced state. The early warning control unit 23 can initialize the timing of implementing the avoidance control when the driver is neither in the visual recognition reduced state nor the auditory recognition reduced state. For example, the early warning control unit 23 can discard the corrected value of the avoidance start threshold, so that the avoidance control is implemented based on the avoidance start threshold before correction.

[0052] <Determination process> The flow of processing executed by the processing circuit 11 when determining whether or not the recognition is impaired will be described with reference to Fig. 2. The flow of processing shown in Fig. 2 is the determination processing. For example, when the conditions for performing approach notification are not satisfied, the processing circuit 11 repeatedly executes the determination processing at predetermined control cycles.

[0053] In step S101, the processing circuit 11 determines whether the driver is in a state of reduced awareness. Specifically, the processing circuit 11 first acquires a determination result of whether the driver is in a state of reduced visual awareness and a determination result of whether the driver is in a state of reduced auditory awareness. The determination results of whether the driver is in a state of reduced visual awareness and a determination result of whether the driver is in a state of reduced auditory awareness are obtained by the processing circuit 11 functioning as the visual awareness determination unit 21 and the auditory awareness determination unit 22. Here, the processing circuit 11 determines that the driver is in a state of reduced awareness when the driver is in at least one of the states of reduced visual awareness and reduced auditory awareness based on the acquired determination results.

[0054] If the processing circuit 11 determines that the driver is in a state of reduced awareness (S101: YES), the processing circuit 11 proceeds to step S102. On the other hand, if the processing circuit 11 determines that the driver is not in a state of reduced awareness (S101: NO), the processing circuit 11 proceeds to step S104.

[0055] In step S102, the processing circuit 11 advances the timing of implementing avoidance control by functioning as the early warning control unit 23. Subsequently, the processing circuit 11 shifts the processing to step S103.

[0056] In step S103, the processing circuitry 11 performs early notification processing by functioning as the early notification control unit 23. When the early notification processing ends, the processing circuitry 11 temporarily ends the determination processing.

[0057] In step S104, the processing circuit 11 initializes the timing of implementing avoidance control by functioning as the early warning control unit 23. Specifically, if the avoidance start threshold has been corrected, the corrected value is discarded, so that implementation of avoidance control is controlled based on the avoidance start threshold before correction. Once the timing of implementing avoidance control has been initialized, the processing circuit 11 temporarily ends the determination process. If the avoidance start threshold has not been corrected, the processing circuit 11 temporarily ends the determination process as is.

[0058] <Early notification processing> The flow of the early notification process will be described with reference to Fig. 3. The flow of the process shown in Fig. 3 is the early notification process. The processing circuit 11 performs the early notification process in step S103.

[0059] In step S201, the processing circuit 11 selects a notification means according to the driver's awareness deterioration state acquired in step S101. After that, the processing circuit 11 shifts the process to step S202.

[0060] In step S202, processing circuitry 11 performs the first notification process by controlling the notification device selected in step S201. After that, processing circuitry 11 proceeds to step S203.

[0061] In step S203, the processing circuit 11 determines whether or not the interruption condition is met. If the interruption condition is met (S203: YES), the processing circuit 11 ends the early notification process.

[0062] On the other hand, if the interruption condition is not met (S203: NO), the processing circuit 11 shifts the processing to step S204. In step S204, the processing circuit 11 determines whether the second notification condition is met. If the second notification condition is not met (S204: NO), the processing circuit 11 shifts the process back to step S203.

[0063] On the other hand, if the second notification condition is met (S204: YES), the processing circuit 11 proceeds to step S205. In step S205, the processing circuit 11 performs the second notification process by controlling the notification device selected in step S201. Thereafter, the processing circuit 11 ends the early notification process.

[0064] <Actions and Effects of This Embodiment> According to the driving assistance device 10, the early warning process is performed when the driver's awareness is reduced due to an external disturbance. Therefore, when the driver is in a state of reduced awareness, the warning is performed regardless of the time to collision related to the approach warning, even if the time to collision is longer than the approach threshold. In other words, the early warning process is performed earlier than the approach warning.

[0065] Since the driving assistance device 10 can warn the driver of potential risks in advance based on the occurrence of a disturbance, for example, the driving assistance device 10 can warn the driver of potential risks before the effects of the disturbance are felt by the driver. Therefore, the driving assistance device 10 can warn the driver at an earlier timing than when it detects that the driver is already in an abnormal state. In this way, the driving assistance device 10 can warn the driver from a time when the risk is relatively low, thereby contributing to safer driving.

[0066] The driving assistance device 10 selects a notification device to be used in the first notification process of the early notification process depending on the state of the driver in the state of reduced awareness. For example, if the driver is in the state of reduced visual awareness, the sound device 52 is selected. For example, if the driver is in the state of reduced auditory awareness, the vibration device 53 is selected. This makes it possible to implement a notification that acts on a sense that is less susceptible to the disturbance that is causing the state of reduced awareness. This makes it easier for the driver to notice that a notification is being implemented.

[0067] In the second notification process that is performed after the first notification process, a notification device different from the notification device used in the first notification process is selected. By using notification devices that stimulate different senses in this way, it is possible to make the notification easier to notice.

[0068] By using the notification by the display device 51 in combination with notifications by other notification devices, the intention of the notification can be more easily conveyed to the driver. For example, if only a beep is sounded, the driver may not understand why the beep is sounding. Also, for example, if only vibration is generated, the driver may not understand why the vibration is being generated. In contrast, if the notification by the display device 51 is performed simultaneously with notifications by other notification devices, the driver can more easily understand why the notification is being performed.

[0069] <Example of change> This embodiment can be modified as follows: This embodiment and the following modifications can be combined and implemented within the scope of technical compatibility.

[0070] In the above embodiment, the first notification process selects the display device 51 and the sound device 52 when the driver is in a state of reduced visual awareness. Alternatively, the first notification process may select the display device 51 and the vibration device 53 as the third notification device. With the above configuration, the vibration device 53 is selected as the notification device that acts on something other than vision, so even with this configuration, a notification that is easy for a driver in a state of reduced visual awareness to notice can be made.

[0071] The notification performed in the second notification process of the early notification process may be the same as or different from the notification performed in the first notification process. For example, the second notification process may be configured to be more attention-grabbing than the first notification process. For example, the characters displayed on the display device 51 may be made larger or the color of the characters may be changed to make them more noticeable. For example, the sound output from the audio device 52 may be made louder or the pitch of the sound may be changed to make the sound easier to hear. For example, the vibration of the vibration device 53 may be made stronger or the frequency of the vibration may be changed to make the sound easier to convey to the driver.

[0072] In the above embodiment, a configuration in which all notification devices are used to perform notification in the second notification process has been described. However, the second notification process is not limited to this configuration and may be configured to perform notification by controlling two or more notification devices, including one notification device among the display device 51, the audio device 52, and the vibration device 53 that was not selected in the first notification process. Specifically, the second notification process may be configured to switch from the notification device selected in the first notification process to another notification device. For example, in the first notification process, two devices among the display device 51, the audio device 52, and the vibration device 53 are selected. In the subsequent second notification process, one of the two devices selected in the first notification process and the device not selected in the first notification process are selected. Even in this case, it is preferable to perform notification by the display device 51 in both the first notification process and the second notification process, as in the above embodiment. As an example, if in the first notification process, notification is made by the display device 51 and the sound device 52, in the second notification process, notification is not made by the sound device 52, but notification is made by the display device 51 and the vibration device 53.

[0073] [Table 2] Table 2 is a correspondence table illustrating the correspondence between the notification means selected in the configuration embodying the above modified example and the driver's state. In the above configuration, as in the above embodiment, the first notification process can implement a notification that acts on a sense that is less susceptible to the influence of external disturbances that are the cause of the impaired awareness state. Furthermore, in the second notification process performed after the first notification process, a notification device that acts on a sense different from the notification device used in the first notification process can be used, making it easier for the driver to notice the notification.

[0074] In the above embodiment, a configuration in which multiple notification devices are used to perform the first notification process and the second notification process in the early notification process is illustrated. Alternatively, the first notification process and the second notification process may be configured to perform notification using any one of the notification devices.

[0075] In the above embodiment, an example has been given of a configuration in which, if it is determined in the processing of step S101 that the vehicle is not in a state of impaired awareness, the timing of implementing the avoidance control is initialized as the processing of step S104. Initializing the timing of implementing the avoidance control when it is determined that the vehicle is not in a state of impaired awareness is not a necessary configuration. The timing of initializing the timing of implementing the avoidance control can be changed. For example, the timing of implementing the avoidance control may be initialized when the vehicle 90 stops. Note that, if the processing of step S104 is omitted, the processing circuit 11 temporarily terminates the determination processing shown in FIG. 2 when a negative determination is made in the processing of step S101.

[0076] Monitoring the driver's state may be combined as a condition for determining whether the driver is in a state of impaired visual awareness. For example, a driver who sees backlighting may take action to remove the light source from their field of vision. Therefore, by monitoring the movement of the driver's head, arms, eyelids, etc., the accuracy of determining whether the driver is in a state of impaired visual awareness can be improved.

[0077] The interruption condition for the early warning process may be determined to be satisfied when the time to collision becomes longer than a specified time as a result of the driver operating the vehicle 90. The specified time is a value greater than the avoidance start threshold and the approach threshold.

[0078] The sound level determined by the auditory recognition determination unit 22 may be related to a sound that can be heard by the driver of the vehicle 90. Therefore, the vehicle 90 may be equipped with a sound collection device that detects sound outside the vehicle or inside the vehicle cabin. For example, the vehicle 90 may be equipped with only one of the exterior sound collection device 32 and the interior sound collection device 42, or may be equipped with both the exterior sound collection device 32 and the interior sound collection device 42. Noise outside the vehicle 90 may be detected by the interior sound collection device 42 attached to the interior of the vehicle 90. Furthermore, noise outside the vehicle 90 may be detected based on the difference between the sound detected by the exterior sound collection device 32 and the sound detected by the interior sound collection device 42.

[0079] In the above embodiment, the vibration device 53 is exemplified as transmitting vibrations generated by an electric motor to the driver. However, the notification device that notifies the driver by transmitting vibrations is not limited to this configuration. For example, in a vehicle 90 that includes an internal combustion engine as the drive unit 81, vibrations may be generated by increasing the rotation speed of the internal combustion engine. For example, vibrations may be generated by controlling the driving system of the vehicle 90 so as to induce a pitching motion of the vehicle 90.

[0080] In the above embodiment, the vibration device 53 is exemplified as the third notification device. The third notification device may be any notification device that conveys warning information to the driver by acting on the driver's senses other than their sight and hearing, and may be, for example, a notification device that can provide the driver with an electrical stimulus as a tactile stimulus. Such a notification device may include, for example, an electrode unit that provides the driver with the electrical stimulus.

[0081] The driving assistance device 10 is not limited to a processing circuit that includes a CPU and memories such as ROM and RAM and executes software processing. For example, the driving assistance device 10 may include a dedicated hardware circuit that executes at least some of the various processes executed in the above-described embodiment. An example of a dedicated hardware circuit is an ASIC. ASIC is an abbreviation for "Application Specific Integrated Circuit." In other words, the driving assistance device 10 may have any of the following configurations (a) to (c):

[0082] (a) A processing circuit that includes a processing device that executes all of the above processes according to a program, and a program storage device such as a ROM that stores the program. (b) A processing circuit comprising a processing device and a program storage device that execute part of the above processing according to a program, and a dedicated hardware circuit that executes the remaining processing.

[0083] (c) A processing circuit having dedicated hardware circuitry to perform all of the above processes. Here, there may be a plurality of software execution devices each having a processing device and a program storage device, and a plurality of dedicated hardware circuits.

[0084] <Other technical ideas> The technical ideas that can be understood from the above-described embodiment and modified examples will be described. [Appendix 1] The early notification process includes a first notification process and a second notification process that starts after the first notification process, In the first notification process, notification is performed by controlling two devices among the display device, the audio device, and the third notification device; In the second notification process, notification is performed by controlling all of the display device, the audio device, and the third notification device.

[0085] [Appendix 2] The early notification process includes a first notification process and a second notification process that starts after the first notification process, In the first notification process, notification is performed by controlling two devices among the display device, the audio device, and the third notification device; In the second notification process, one of the two devices selected in the first notification process; The notification is performed by controlling one of the display device, the sound device, and the third notification device that has not been selected in the first notification process.

[0086] The expression "at least one" used in this specification means "one or more" of the desired options. As an example, the expression "at least one" used in this specification means "only one option" or "both of two options" if the number of options is two. As another example, the expression "at least one" used in this specification means "only one option" or "any combination of two or more options" if the number of options is three or more. [Explanation of symbols]

[0087] 10...Driving assistance device 21...Visual recognition judgment unit 22...Auditory recognition determination unit 23...Early warning control unit 29... Driving Assistance Department 31...External light detection device 32...Outdoor sound collection device 51...Display device 52...Sound equipment 53...Vibration device 90...Vehicle

Claims

1. A driving assistance device that, when a traveling vehicle approaches an obstacle, performs an approach notification to notify a driver of the vehicle that the vehicle is approaching the obstacle, and an avoidance control to cause the vehicle to travel so as to avoid contact between the vehicle and the obstacle, The vehicle has an external light detection device that detects light rays outside the vehicle, a sound collection device that detects sounds outside the vehicle or inside the vehicle cabin, a display device as a first alert device that communicates warning information to the driver by acting on the driver's vision, an acoustic device as a second alert device that communicates warning information to the driver by acting on the driver's hearing, and a third alert device that communicates warning information to the driver by acting on a sense other than the driver's vision and hearing, The driving assistance device a visual recognition determination unit that determines whether the driver is in a visual recognition reduced state, which is a state in which the driver's visual recognition is reduced, based on light rays detected by the external light detection device; an auditory perception determination unit that determines whether the driver is in a state of impaired auditory perception, which is a state in which the driver's auditory perception is impaired, based on the sound detected by the sound collection device; and an early notification control unit that, when the driver is in at least one of the visual recognition reduced state and the auditory recognition reduced state, controls at least one of the display device, the acoustic device, and the third notification device in accordance with the driver's state, thereby performing early notification processing to notify the driver at an earlier timing than when the approach notification is performed. Driving assistance device.

2. When the third notification device is used to notify in the early notification process, In addition to the notification by controlling the third notification device, the notification is also performed by controlling the display device. The driving assistance device according to claim 1 .

3. The third notification device is a vibration device that notifies the driver by generating vibrations that are transmitted to the driver. The driving assistance device according to claim 1 or 2.

4. The early notification process includes a first notification process and a second notification process that starts after the first notification process, In the first notification process, notification is performed by controlling two devices among the display device, the audio device, and the third notification device; In the second notification process, notification is performed by controlling two or more devices including one device not selected in the first notification process among the display device, the sound device, and the third notification device. The driving assistance device according to claim 1 or 2.

5. If the driver is in the reduced visual awareness state but not the reduced auditory awareness state, As the first notification process, notification is performed by controlling the display device and the audio device. The driving assistance device according to claim 4.

6. If the driver is in the hearing recognition impaired state, As the first notification process, notification is performed by controlling the display device and the third notification device. The driving assistance device according to claim 4.

Images