Driving assistance device and driving assistance method

The driving assistance device addresses individual field of view differences by using gaze and obstacle detection to provide tailored warnings, improving safety through accurate obstacle detection and notification.

JP7865241B2Active Publication Date: 2026-05-26TOYOTA JIDOSHA KK
View PDF 5 Cites 0 Cited by

Patent Information

Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
TOYOTA JIDOSHA KK
Filing Date
2023-02-15
Publication Date
2026-05-26

AI Technical Summary

Technical Problem

Existing systems fail to accurately account for individual differences in human field of view, leading to reduced detection accuracy and potential misjudgment of a driver's ability to recognize obstacles due to visual impairments.

Method used

A driving assistance device that includes a gaze detection unit, obstacle detection unit, vehicle control unit, learning unit, and notification unit to estimate the driver's field of view, learn from vehicle control execution, and provide tailored warnings based on individual visual recognition capabilities.

Benefits of technology

Accurately estimates the driver's field of view, providing appropriate driving assistance by considering individual differences, thereby enhancing safety through precise obstacle detection and notification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007865241000001
    Figure 0007865241000001
  • Figure 0007865241000002
    Figure 0007865241000002
  • Figure 0007865241000003
    Figure 0007865241000003
Patent Text Reader

Abstract

To enable provision of appropriate driving assistance to a driver by accurately estimating a visible range of the driver in consideration of individual differences in the field of view.SOLUTION: A driving assistance device is provided, comprising: a line-of-sight detection unit 14 for detecting a line-of-sight direction of a driver of a vehicle 1; an obstacle detection unit 15 for detecting obstacles near the vehicle based on information on the surroundings of the vehicle; a vehicle control unit 16 configured to provide predetermined vehicle control to avoid collision between the vehicle and an obstacle; a learning unit 18 for learning information on the field of view of the driver on the basis of the state of provision of the vehicle control; a visible range estimation unit 17 configured to estimate a visible range of the driver on the basis of the line-of-sight direction and the field-of-view information; and a notification unit 19 configured to notify the driver with an alarm when an obstacle lies outside the visible range.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a driving support device and a driving support method.

Background Art

[0002] Conventionally, it is known to notify a driver of a warning when the driver of a vehicle does not recognize an obstacle around the vehicle. In the vehicle control system described in Patent Document 1, it is described that in order to determine whether a warning to the driver is necessary, the driver's field of view is calculated based on the driver's viewpoint movement speed and the fixation period of the driver's viewpoint.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, there are individual differences in the human field of view. It is also conceivable that the driver's field of view is narrowed due to a visual field disorder such as a narrow visual field or a visual field defect. Therefore, with the method described in Patent Document 1, the detection accuracy of the driver's visual recognition range is reduced, and there is a risk of misjudging that the driver can visually recognize an obstacle when the driver cannot visually recognize the obstacle.

[0005] Therefore, in view of the above problems, an object of the present invention is to provide appropriate driving support to a driver by accurately estimating the driver's visual recognition range in consideration of individual differences in the visual field.

Means for Solving the Problems

[0006] The gist of this disclosure is as follows:

[0007] (1) A driving assistance device comprising: a gaze detection unit for detecting the direction of a vehicle occupant's gaze; an obstacle detection unit for detecting obstacles around the vehicle based on information about the vehicle's surroundings; a vehicle control unit for executing predetermined vehicle control to avoid a collision between the vehicle and the obstacle; a learning unit for learning the occupant's field of view information based on the execution status of the vehicle control; a field of view estimation unit for estimating the occupant's field of view based on the gaze direction and the field of view information; and a notification unit for notifying the occupant of a warning when the obstacle is outside the field of view.

[0008] (2) The driving assistance device according to (1) above, wherein the learning unit detects a missing area in the occupant's field of view based on the execution status of the vehicle control, and learns the field of view information to exclude the missing area in the specific direction from the occupant's field of view when the frequency at which a missing area in a specific direction is detected is greater than or equal to a predetermined value.

[0009] (3) The driving assistance device according to (1) above, wherein the learning unit detects the area of ​​the occupant's field of view loss based on the execution status of the vehicle control, and learns the field of view information to exclude the area of ​​the loss from the occupant's field of view only if the occupant acknowledges the loss of the field of view loss.

[0010] (4) The vehicle control unit advances the start timing of the vehicle control when the obstacle is outside the visibility range compared to when the obstacle is inside the visibility range, according to any one of (1) to (3) above.

[0011] (5) A driving assistance method performed by a computer, comprising: detecting the direction of a vehicle occupant's line of sight; detecting obstacles around the vehicle based on information about the vehicle's surroundings; performing predetermined vehicle control to avoid a collision between the vehicle and the obstacle; learning the occupant's field of view information based on the status of the vehicle control; estimating the occupant's field of view based on the direction of sight and the field of view information; and notifying the occupant of a warning when the obstacle is outside the field of view. [Effects of the Invention]

[0012] According to the present invention, by accurately estimating the driver's field of view while taking into account individual differences in field of vision, appropriate driving assistance is provided to the driver. [Brief explanation of the drawing]

[0013] [Figure 1] This diagram schematically shows a part of the configuration of a vehicle equipped with a driver assistance device according to the first embodiment of the present invention. [Figure 2] This is a functional block diagram of the ECU processor. [Figure 3] This flowchart shows the control routine executed by the ECU's processor in the first embodiment. [Figure 4] This figure shows an example of the extent of visual field loss in a driver. [Figure 5] In the second embodiment, this is a flowchart showing the control routine executed by the ECU's processor. [Modes for carrying out the invention]

[0014] Embodiments of the present invention will be described in detail below with reference to the drawings. In the following description, similar components will be given the same reference numerals.

[0015] <First Embodiment> Hereinafter, a first embodiment of the present invention will be described with reference to FIGS. 1 to 4. FIG. 1 is a diagram schematically showing a part of the configuration of a vehicle 1 provided with a driving support device according to the first embodiment of the present invention.

[0016] As shown in FIG. 1, the vehicle 1 includes a driver monitor camera 2, a surrounding information detection device 3, a vehicle behavior detection device 4, an actuator 5, a human machine interface (HMI) 6, and an electronic control unit (ECU) 10. The driver monitor camera 2, the surrounding information detection device 3, the vehicle behavior detection device 4, the actuator 5, and the HMI 6 are electrically connected to the ECU 10 via an in-vehicle network or the like compliant with a standard such as CAN (Controller Area Network).

[0017] The driver monitor camera 2 photographs the face of the driver of the vehicle 1 and generates a face image representing the face of the driver. The output of the driver monitor camera 2, that is, the face image generated by the driver monitor camera 2 is transmitted to the ECU 10.

[0018] The surrounding information detection device 3 acquires data (images, point cloud data, etc.) around the vehicle 1 and detects the surrounding information of the vehicle 1. For example, the surrounding information detection device 3 includes a camera (monocular camera or stereo camera), a millimeter wave radar, a lidar (LIDAR: Laser Imaging Detection And Ranging), or an ultrasonic sensor (sonar), or any combination thereof. The output of the surrounding information detection device 3, that is, the surrounding information of the vehicle 1 detected by the surrounding information detection device 3 is transmitted to the ECU 10.

[0019] The vehicle behavior detection device 4 detects the behavior information of the vehicle 1. The vehicle behavior detection device 4 includes, for example, a vehicle speed sensor that detects the speed of the vehicle 1, a brake pedal stroke sensor that detects the depression force of the brake pedal provided in the vehicle 1, an accelerator position sensor that detects the depression amount (accelerator opening) of the accelerator pedal provided in the vehicle 1, and the like. The output of the vehicle behavior detection device 4, that is, the behavior information of the vehicle 1 detected by the vehicle behavior detection device 4 is transmitted to the ECU 10.

[0020] The actuator 5 operates the vehicle 1. For example, the actuator 5 includes a driving device (for example, at least one of an internal combustion engine and an electric motor) for accelerating the vehicle 1, a brake actuator for braking (decelerating) the vehicle 1, a steering actuator for steering the vehicle 1, and the like. The ECU 10 controls the actuator 5 to control the behavior of the vehicle 1.

[0021] The HMI 6 performs information transmission between the vehicle 1 and the occupants (for example, the driver) of the vehicle 1. The HMI 6 has an output unit (for example, a display, a speaker, a light source, a vibration unit, etc.) that provides information to the occupants of the vehicle 1 and an input unit (for example, a touch panel, an operation button, an operation switch, a microphone, etc.) through which information is input by the occupants of the vehicle 1. The output of the ECU 10 is notified to the occupants of the vehicle 1 via the HMI 6, and the input from the occupants of the vehicle 1 is transmitted to the ECU 10 via the HMI 6. The HMI 6 is an example of an input device, an output device, or an input / output device. Note that a mobile terminal (for example, a smartphone, a tablet terminal, etc.) of the occupants of the vehicle 1 may be communicably connected to the ECU 10 by wire or wirelessly and function as the HMI 6.

[0022] The ECU 10 executes various controls of the vehicle 1. As shown in FIG. 1, the ECU 10 includes a communication interface 11, a memory 12, and a processor 13. The communication interface 11 and the memory 12 are connected to the processor 13 via signal lines. In this embodiment, one ECU 10 is provided, but a plurality of ECUs may be provided for each function.

[0023] The communication interface 11 has an interface circuit for connecting the ECU 10 to the in-vehicle network. The ECU 10 is connected to other in-vehicle equipment via the communication interface 11.

[0024] Memory 12 includes, for example, volatile semiconductor memory and non-volatile semiconductor memory. Memory 12 stores programs, data, etc., used when various processes are executed by the processor 13.

[0025] The processor 13 has one or more CPUs (Central Processing Units) and their peripheral circuits. The processor 13 may also have additional arithmetic circuits such as a logic unit, a numerical unit, or a graphics processing unit.

[0026] In this embodiment, the ECU 10 functions as a driver assistance device that supports the driver in operating the vehicle 1. Figure 2 is a functional block diagram of the processor 13 of the ECU 10. The processor 13 includes a gaze detection unit 14, an obstacle detection unit 15, a vehicle control unit 16, a visibility range estimation unit 17, a learning unit 18, and a notification unit 19. These are functional modules that are realized by the execution of a computer program stored in the memory 12 of the ECU 10 by the processor 13 of the ECU 10. Note that each of these functional modules may be realized by a dedicated arithmetic circuit provided in the processor 13.

[0027] The gaze detection unit 14 detects the direction of the driver's gaze in the vehicle 1. The driver's gaze direction is calculated as 0 degrees when the driver is facing the front of the vehicle 1, as a positive value when the driver is facing to the right of the front of the vehicle 1, and as a negative value when the driver is facing to the left of the front of the vehicle 1.

[0028] For example, the gaze detection unit 14 detects the driver's gaze direction by the following method. First, the gaze detection unit 14 identifies the face region from the face image generated by the driver monitor camera 2 and detects facial components by extracting feature points of facial components such as the eyes, nose, and mouth. Next, the gaze detection unit 14 detects the position of the Purkinje image (corneal reflection image) and the position of the pupil center, and detects the driver's gaze direction based on the positional relationship between the Purkinje image and the pupil center.

[0029] The gaze detection unit 14 may also detect the driver's face orientation as the driver's gaze direction. In this case, for example, the gaze detection unit 14 detects the driver's face orientation by matching the face image generated by the driver monitor camera 2 with multiple face shape data sets that represent different driver face orientations. The gaze detection unit 14 detects the face orientation of the face shape data set that yields the highest match rate as the angle of the driver's face orientation. Multiple face shape data sets are pre-stored in the memory 12 of the ECU 10 or another storage device. The multiple face shape data sets may be general human face data, or they may be acquired for each driver.

[0030] Furthermore, the gaze detection unit 14 may detect the driver's gaze direction by other methods. For example, the gaze detection unit 14 may detect the driver's gaze direction using a classifier that has been pre-trained to output the driver's gaze direction from facial image data. Examples of such classifiers include machine learning models such as neural networks, support vector machines, and random forests.

[0031] The obstacle detection unit 15 detects obstacles around the vehicle 1 based on the surrounding information of the vehicle 1 detected by the surrounding information detection device 3. Obstacles include, for example, surrounding vehicles, pedestrians, bicycles, and fallen objects. For example, the obstacle detection unit 15 detects obstacles using a classifier that has been pre-trained to output the presence or absence of obstacles and the direction of obstacles from the surrounding information of the vehicle 1. Examples of such classifiers include machine learning models such as neural networks, support vector machines, and random forests.

[0032] The vehicle control unit 16 performs predetermined vehicle control to avoid a collision between the vehicle 1 and an obstacle. Predetermined vehicle control is, for example, vehicle control performed when the Pre-crash Safety (PCS) driver assistance function is activated in the vehicle 1. In PCS, as the Time To Collision (TTC) with respect to the obstacle decreases, vehicle control such as warning, braking control and steering control is performed in stages.

[0033] The visibility range estimation unit 17 estimates the visibility range of the driver of vehicle 1. Specifically, the visibility range estimation unit 17 estimates the driver's visibility range based on the driver's gaze direction detected by the gaze detection unit 14 and the driver's field of view information.

[0034] The driver's field of view information includes the driver's field of view in the horizontal direction. However, there are individual differences in human field of view. Furthermore, a driver's field of view may be narrowed due to visual impairments such as constriction or loss of vision. Therefore, if the driver's field of view is set to a uniform value, there is a risk of misjudging that the driver has seen an obstacle when the driver has not. To address this, in this embodiment, the driver's field of view information is learned by the learning unit 18. This allows for accurate estimation of the driver's field of view range, taking into account each driver's field of view, and ultimately providing the driver with appropriate driving assistance.

[0035] In this embodiment, the learning unit 18 learns the driver's field of view information based on the execution status of vehicle control by the vehicle control unit 16. The initial value of the driver's field of view is set to a range between -45 degrees to -20 degrees and +20 degrees to +45 degrees, for example, when the driver's line of sight direction is set to 0 degrees. As a specific example, if the driver's line of sight direction is +20 degrees and the value of the driver's field of view is in the range between -40 degrees and +40 degrees, the visibility range estimation unit 17 calculates the driver's visibility range as being in the range between -20 degrees and +60 degrees.

[0036] When a driver can see an obstacle, vehicle control measures to avoid a collision between vehicle 1 and the obstacle are generally not performed. Therefore, if such vehicle control measures are performed for an obstacle that is within the driver's line of sight, it is highly likely that the driver did not see the obstacle within their line of sight.

[0037] Therefore, in response to the execution of a predetermined vehicle control for an obstacle located within the driver's field of view, the learning unit 18 learns the driver's field of view information. For example, the learning unit 18 detects a missing area in the driver's field of view based on the execution status of a predetermined vehicle control and learns the field of view information to exclude the missing area from the driver's field of view. The field of view information updated by learning is stored, for example, in the memory 12 of the ECU 10 or in another storage device. Once the driver's field of view information is learned, the field of view estimation unit 17 then estimates the driver's field of view based on the learned field of view information.

[0038] The notification unit 19 determines whether the obstacle detected by the obstacle detection unit 15 is located outside the visibility range estimated by the visibility range estimation unit 17. When the obstacle is located inside the visibility range, it is assumed that the driver is aware of the obstacle; when the obstacle is located outside the visibility range, it is assumed that the driver is not aware of the obstacle. Therefore, the notification unit 19 notifies the driver of a warning when the obstacle is located outside the visibility range. This makes it easier for the driver to recognize obstacles and assists the driver in driving the vehicle 1. Furthermore, in this embodiment, since the driver's visibility range is estimated based on learned field of view information, the driver can be notified of effective warnings that are tailored to the driver's field of view characteristics.

[0039] The following describes the control process flow described above with reference to Figure 3. Figure 3 is a flowchart of the control routine executed by the processor 13 of the ECU 10 in the first embodiment. This control routine is executed repeatedly at predetermined execution intervals.

[0040] First, in step S101, the obstacle detection unit 15 of the processor 13 determines whether or not there are obstacles around the vehicle 1. If it is determined that there are no obstacles, this control routine terminates. On the other hand, if it is determined that there are obstacles, this control routine proceeds to step S102.

[0041] In step S102, the gaze detection unit 14 of the processor 13 detects the driver's gaze direction based on the driver's face image generated by the driver monitor camera 2. Then, in step S103, the viewing range estimation unit 17 estimates the driver's viewing range based on the driver's gaze direction and field of view information.

[0042] Next, in step S104, the notification unit 19 of the processor 13 determines whether the obstacle detected by the obstacle detection unit 15 is located within the driver's line of sight estimated in step S103. For example, the notification unit 19 makes this determination by comparing the direction and line of sight of the obstacle expressed in the same coordinate system (for example, a vehicle coordinate system based on vehicle 1).

[0043] If it is determined in step S104 that an obstacle is outside the line of sight, the control routine proceeds to step S105. In step S105, the notification unit 19 notifies the driver of vehicle 1 of a warning via HMI 6. For example, the notification unit 19 notifies an auditory warning, such as a buzzer sound or voice, via the speaker of HMI 6. Alternatively, the notification unit 19 may notify a visual warning by displaying information about the obstacle via the display of HMI 6. The notification unit 19 may also notify a tactile warning via a vibration unit of HMI 6 located on the steering wheel or accelerator pedal of vehicle 1. After step S105, the control routine terminates.

[0044] On the other hand, if it is determined in step S104 that the obstacle is within the line of sight, the control routine proceeds to step S106. In step S106, the learning unit 18 of the processor 13 determines whether a predetermined vehicle control to avoid a collision between the vehicle 1 and the obstacle has been executed by the vehicle control unit 16 of the processor 13. The predetermined vehicle control is, for example, a vehicle control executed when the driver assistance function of the PCS is activated, and is at least one of warning, braking control, and steering control.

[0045] If it is determined in step S106 that the predetermined vehicle control has not been performed, this control routine terminates. On the other hand, if it is determined in step S106 that the predetermined vehicle control has been performed, this control routine proceeds to step S107.

[0046] In step S107, the learning unit 18 detects the missing area of ​​the driver's field of view. Specifically, the learning unit 18 detects the area where an obstacle exists in relation to the driver's field of view as the missing area of ​​the driver's field of view. Next, in step S108, the learning unit 18 learns the driver's field of view information so as to exclude the missing area from the driver's field of view. As a result, in subsequent control routines, the processing in step S103 is executed using the learned field of view information. After step S108, this control routine terminates.

[0047] Figure 4 shows an example of a driver's field of view defect. In the example in Figure 4, the driver is facing forward, and the driver's line of sight is 0 degrees. Because a predetermined vehicle control is performed for a pedestrian within the driver's line of sight, the area where the pedestrian is located (in this example, the area between +5 degrees and +25 degrees) is excluded from the driver's field of view as a defect. As a result, the driver's field of view, with the driver's line of sight angle set to 0 degrees, changes from the area between -40 degrees and +40 degrees to the area between -40 degrees and +5 degrees and the area between +25 degrees and +40 degrees.

[0048] <Second Embodiment> The driver assistance system according to the second embodiment is basically the same as the driver assistance system according to the first embodiment in terms of configuration and control, except for the points described below. For this reason, the second embodiment of the present invention will be described below, focusing on the parts that differ from the first embodiment.

[0049] As described above, the learning unit 18 learns the driver's field of view information in response to the execution of a predetermined vehicle control for an obstacle located within the driver's field of view. However, it is possible that the vehicle control to avoid the obstacle was executed not due to a problem with the driver's field of view characteristics, but rather due to a decrease in the driver's attention. On the other hand, if a range in a particular direction is frequently detected as a missing area in the driver's field of view, it is highly likely that there is a problem with the driver's field of view characteristics.

[0050] Therefore, in the second embodiment, the learning unit 18 detects the missing area of ​​the driver's field of view based on the execution status of a predetermined vehicle control, and learns the field of view information to exclude the missing area in that specific direction from the driver's field of view when the frequency of detection of a missing area in a specific direction is greater than or equal to a predetermined value. This improves the accuracy of learning the field of view information and allows for a more accurate estimation of the driver's field of view.

[0051] Figure 5 is a flowchart showing the control routine executed by the processor 13 of the ECU 10 in the second embodiment. This control routine is executed repeatedly at predetermined execution intervals.

[0052] Steps S201 to S207 are performed in the same manner as steps S101 to S107 in Figure 3. After step S207, in step S208, the learning unit 18 determines whether the frequency at which a missing range in a specific direction is detected is greater than or equal to a predetermined value. The predetermined value is set to a frequency of 2 times per trip (the period from when the ignition switch of vehicle 1 is turned on until it is turned off).

[0053] If it is determined in step S208 that the frequency of detecting a missing area in a specific direction is less than a predetermined value, this control routine terminates. On the other hand, if it is determined in step S208 that the frequency of detecting a missing area in a specific direction is equal to or greater than a predetermined value, this control routine proceeds to step S209.

[0054] In step S209, the learning unit 18 learns the driver's field of view information to exclude a specific direction of missing field from the driver's field of view. For example, if the missing field in a specific direction is between +10 degrees and +30 degrees, the driver's field of view, with the driver's line of sight angle set to 0 degrees, is changed from a range between -40 degrees and +40 degrees to a range between -40 degrees and +10 degrees and a range between +30 degrees and +40 degrees. After step S209, this control routine terminates.

[0055] Furthermore, drivers are likely to be aware of their own visual field characteristics. For this reason, the learning unit 18 may learn visual field information to detect the area of ​​visual field defects in the driver based on the execution status of predetermined vehicle control, and to exclude the area of ​​defects from the driver's visual field only if the driver acknowledges the visual field defects. This also improves the accuracy of learning visual field information and allows for a more accurate estimation of the driver's visible range. In this case, in step S208, the learning unit 18 determines whether the driver has acknowledged the visual field defects. For example, the learning unit 18 requests acknowledgment from the driver via the HMI 6 and makes this determination based on the driver's input to the HMI 6.

[0056] <Other Embodiments> Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes can be made within the scope of the claims. For example, the predetermined vehicle control performed by the vehicle control unit 16 to avoid a collision between the vehicle 1 and an obstacle may be acceleration suppression control when the vehicle 1 is starting or driving at low speed.

[0057] Furthermore, when an obstacle is located outside the driver's line of sight, the vehicle control unit 16 may initiate predetermined vehicle control to avoid a collision between the vehicle 1 and the obstacle earlier than when the obstacle is located inside the line of sight. This further enhances safety for obstacles that are unlikely to be visible to the driver. In this case, for example, the vehicle control unit 16 initiates vehicle control in the PCS earlier by increasing the collision margin time for the obstacle when the PCS begins to issue a warning.

[0058] Furthermore, in response to a predetermined vehicle control being performed on an obstacle located within the driver's field of view, the learning unit 18 may learn field of view information to narrow the driver's field of view outside of it. As a specific example in this case, the learning unit 18 changes the range of the driver's field of view from a range between -40 degrees and +40 degrees to a range between -35 degrees and +35 degrees.

[0059] Furthermore, Vehicle 1 may be a vehicle capable of performing Level 1 or Level 2 autonomous driving. Alternatively, Vehicle 1 may be a vehicle capable of performing Level 3 autonomous driving in a pre-defined Operational Design Domain (ODD) in which the driver is not required to monitor the surroundings. In this case, the control routine shown in Figure 3 or Figure 5 is executed when Level 3 autonomous driving is not being performed in Vehicle 1. Note that the autonomous driving levels in this specification are based on the definitions in SAE (Society of Automotive Engineers) J3016.

[0060] Furthermore, the computer program that enables the computer to implement the functions of each part of the processor 13 of the ECU 10 may be provided in the form of a recording medium that can be read by the computer. The recording medium that can be read by the computer may be, for example, a magnetic recording medium, an optical recording medium, or a semiconductor memory. [Explanation of Symbols]

[0061] 1 vehicle 10. Electronic Control Unit (ECU) 13 processors 14 Eye-line detection unit 15 Obstacle detection unit 16. Vehicle Control Unit 17. Visibility Range Estimation Unit 18. Learning Department 19 Notification Department

Claims

1. A gaze detection unit that detects the direction of the vehicle occupant's gaze, An obstacle detection unit that detects obstacles around the vehicle based on information about the vehicle's surroundings, A vehicle control unit that performs predetermined vehicle control to avoid a collision between the vehicle and the obstacle, A learning unit that learns the occupant's field of view information based on the execution status of the vehicle control, A viewing range estimation unit that estimates the occupant's viewing range based on the line of sight direction and the field of view information, A notification unit that alerts the occupant when the aforementioned obstacle is located outside the visual range. Equipped with, The vehicle control unit is a driving assistance device that, when the obstacle is located outside the visibility range, starts the vehicle control earlier than when the obstacle is located inside the visibility range.

2. The driving assistance device according to claim 1, wherein the learning unit detects a missing area in the occupant's field of view based on the execution status of the vehicle control, and learns the field of view information to exclude the missing area in a specific direction from the occupant's field of view when the frequency at which a missing area in a specific direction is detected is greater than or equal to a predetermined value.

3. The driving assistance device according to claim 1, wherein the learning unit detects the area of ​​the occupant's field of view loss based on the execution status of the vehicle control, and learns the field of view information to exclude the area of ​​the loss from the occupant's field of view only if the occupant acknowledges the loss of the field of view loss.

4. A driving assistance method performed by a computer, Detecting the direction of the vehicle occupants' gaze, Based on the surrounding information of the vehicle, the system detects obstacles in the vicinity of the vehicle. To perform predetermined vehicle control to avoid a collision between the vehicle and the obstacle, Based on the execution status of the vehicle control, the occupant's field of view information is learned, Estimating the occupant's field of view based on the line of sight direction and the field of view information, To notify the occupant of a warning when the obstacle is outside the line of sight, When the obstacle is located outside the visibility range, the vehicle control start timing is made earlier compared to when the obstacle is located inside the visibility range. Driving assistance methods, including those mentioned above.