Driving support device

Abstract

To reduce troublesome feeling imparted to a driver by excluding a risk object recognized by the driver and the risk object that is supposed to be recognized from a notification object.SOLUTION: According to a driving support device, risk detection processing and notification processing comprise the steps of detecting a risk object based on information about a surrounding environment of a vehicle, and giving notice of presence of the detected risk object, respectively, that are performed. The notification processing comprises a step of specifying a visible risk that is present in a visual-line retention area in timing when a driver turns look to the visual-line retention area to exclude the visible risk from a notification object, based on visual line movement pattern information obtained by learning a visual line movement pattern including the visual-line retention area and a visual-line transition area according to a time-sequential movement pattern of a driver's visual line in a predetermined traffic scene, when the vehicle is located in the predetermined traffic scene.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present disclosure relates to a driving assistance device. [Background technology]

[0002] There is a known technology that uses a sensor to detect objects around a vehicle and notify the driver of the detected objects. However, it is thought that the driver will feel annoyed if the notification is given even though the driver is aware of the object. For this reason, a technology has been proposed that excludes objects that meet certain conditions from being notified.

[0003] For example, Patent Document 1 discloses a notification control device that includes an alarm device that detects objects present around the vehicle and alerts the driver of the presence of the detected objects, in which a control unit sequentially detects the line of sight of the vehicle driver, sets a notification range by excluding the recognition direction range determined based on the detected line of sight from a standard notification range determined based on the object detection range of a peripheral object detection device, and causes the alarm device to alert the driver of the presence of an object based on the fact that an object detected by the peripheral object detection device is present within the notification range.

[0004] Patent Document 2 also discloses a recognition device that refers to the detection results of a line-of-sight detection device, recognizes a range that extends from the viewpoint of the driver of a moving body in the line of sight direction and widens at a predetermined angle as it gets farther away from the viewpoint, recognizes targets that exist in the environment around the moving body captured in the image data based on image data of the surroundings of the moving body captured by a visual sensor arranged on the moving body, sets the area of ​​the recognized target in a predetermined shape, sets multiple judgment points in the target area, determines whether the driver recognizes the target based on the degree of overlap between the multiple judgment points and the field of view, causes an alarm device to notify the presence of targets that it determines the driver does not recognize in a first manner, causes the alarm device to notify the presence of targets that it determines the driver recognizes partially in a second manner, and when the driver recognizes the target whose presence it has not recognized, causes the alarm device to terminate the notification of the target. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2019-117668 [Patent Document 2] Japanese Patent Publication No. 2020-20167 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the technologies described in Patent Documents 1 and 2 are essentially technologies that exclude objects that are within the line of sight (recognition direction range or field of view range) from objects to be notified. Therefore, if an object is not present within the range where the driver is looking at a certain time, the object will be notified even if the driver recognizes the object through subsequent driver actions, which leads to annoyance. Furthermore, even if the driver is not looking directly at an object, the driver may recognize the presence of an object that comes into the field of view as the driver's line of sight changes, which leads to annoyance if the driver is not notified of the object. Furthermore, even if the driver is not looking directly at an object, the driver may recognize the presence of an object if information suggesting the presence of an object, such as headlight illumination, comes into the field of view, which leads to annoyance if the driver is not notified of the object.

[0007] The present disclosure has been made in consideration of the above-mentioned problems, and the purpose of the present disclosure is to provide a driving assistance device that can reduce the inconvenience to the driver by excluding risk objects that the driver is aware of and that the driver should be aware of from the objects to be notified based on the driver's gaze movement pattern information. [Means for solving the problem]

[0008] In order to solve the above problem, according to one aspect of the present disclosure, a driving assistance device that alerts a driver to the presence of a moving object around a vehicle is provided, comprising one or more processors and one or more memories communicatively connected to the one or more processors, wherein the one or more processors execute a risk detection process that detects a risk object based on information about the environment surrounding the vehicle, and a notification process that notifies the driver of the presence of the detected risk object, and in the notification process, when the vehicle is placed in a predetermined traffic scene, the driving assistance device identifies a visibility risk present in the gaze dwell area at the time the driver directs their gaze to the gaze dwell area based on gaze movement pattern information that has learned a time-series movement pattern of the driver's gaze in the predetermined traffic scene, the gaze movement pattern including a gaze dwell area where the gaze is directed for a predetermined time or more, and a gaze transition area where the gaze is directed for a time less than the predetermined time, and excludes the visibility risk from the targets for notification. [Effects of the Invention]

[0009] As described above, according to the present disclosure, risk objects that the driver is aware of and that the driver should be aware of can be excluded from the notification targets based on the gaze movement pattern information of the vehicle driver, thereby reducing the inconvenience to the driver. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 10 is an explanatory diagram showing a reference example in which risk objects recognized by the driver are excluded from objects to be notified. [Figure 2] FIG. 10 is an explanatory diagram showing a reference example in which risk objects recognized by the driver are excluded from objects to be notified. [Figure 3] FIG. 10 is an explanatory diagram showing a reference example in which risk objects recognized by the driver are excluded from objects to be notified. [Figure 4] FIG. 10 is an explanatory diagram showing a reference example in which risk objects recognized by the driver are excluded from objects to be notified. [Figure 5] FIG. 10 is an explanatory diagram showing a reference example in which risk objects recognized by the driver are excluded from objects to be notified. [Figure 6] 1 is a schematic diagram illustrating a configuration example of a vehicle equipped with a driving assistance device according to an embodiment of the present disclosure. [Figure 7] FIG. 2 is a block diagram showing a configuration example of a driving assistance device according to the embodiment; [Figure 8] 10 is an explanatory diagram showing an example of information on time changes in the direction of the driver's line of sight (line of sight movement pattern information); FIG. [Figure 9] 4 is a flowchart showing a routine of a driving assistance process performed by the driving assistance device according to the embodiment. [Figure 10] 4 is a flowchart showing a routine of a driving assistance process performed by the driving assistance device according to the embodiment. [Figure 11] 10A and 10B are explanatory diagrams showing an example in which a notification process is applied by the driving assistance device according to the embodiment. [Figure 12] 10 is an explanatory diagram showing another example in which the notification process by the driving assistance device according to the embodiment is applied. FIG. [Figure 13] 10 is an explanatory diagram showing another example in which the notification process by the driving assistance device according to the embodiment is applied. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0012] 1. Background of the Disclosure First, the background of the present disclosure will be described.

[0013] In recent years, vehicles have been equipped with surrounding environment sensors to detect obstacles around the vehicle (hereinafter referred to as "risk objects"). One of the technologies that uses surrounding environment sensors to assist the driver of a vehicle is a technology that alerts the driver to the presence of risk objects. "Risk objects" include not only moving objects such as pedestrians, bicycles, and other vehicles, but also stationary objects that the vehicle may collide with.

[0014] In such driving assistance technology, if a notification is issued even though the driver is aware of a risk object, the driver may feel annoyed. In response to this, it is thought that the annoyance can be reduced to a certain extent by excluding risk objects that the driving assistance device determines the driver has recognized from the notification objects.

[0015] Figures 1 to 5 are explanatory diagrams showing an example in which a driving assistance device excludes risk objects recognized by a driver from those to be notified. Figures 1 to 5 show a scene in which a vehicle 100 turns left from a public road 111, and show in chronological order the events from when the vehicle 100 stops to wait for pedestrians and cyclists to pass until it starts moving forward. Figures 1 to 5 also show in chronological order the areas Rr0 to Rr4 to which the driver's gaze is directed from when the vehicle 100 stops until it starts moving forward.

[0016] FIG. 1 shows the state of the vehicle 100 at time T=t when the vehicle 100 stops. FIG. 2 shows the state of a first period from time T=t when the vehicle 100 stops to time T=t+3, three seconds later. FIG. 3 shows the state of a second period from time T=t+3 to time T=t+5, two seconds later. FIG. 4 shows the state of a third period from time T=t+5 to time T=t+6, one second later. FIG. 5 shows the state of a fourth period from time T=t+6 to time T=t+6, 0.5 seconds later, when the vehicle 100 starts moving.

[0017] As shown in FIG. 1, there is a crosswalk 115 on roadway 113 ahead of the left turn, and a first pedestrian 103a and a second pedestrian 103b are moving from the front right of vehicle 100 toward crosswalk 115. Also, a first bicycle 105a, a second bicycle 105b, and a third bicycle 105c are moving from the rear left of vehicle 100 toward crosswalk 115, and a fourth bicycle 105d is present on the left side of vehicle 100, moving toward roadway 113. First pedestrian 103a, second pedestrian 103b, first bicycle 105a, second bicycle 105b, third bicycle 105c, and fourth bicycle 105d are each traveling at different speeds. The first pedestrian 103a, the second pedestrian 103b, the first bicycle 105a, the second bicycle 105b, the third bicycle 105c and the fourth bicycle 105d are all detected by ambient environment sensors mounted on the vehicle 100.

[0018] At time T=t, the driver is looking in the direction of the crosswalk 115, which is the traveling direction of the vehicle 100. At this time, the driving assistance device notifies the driver of the presence of all first pedestrian 103a, second pedestrian 103b, first bicycle 105a, second bicycle 105b, third bicycle 105c, and fourth bicycle 105d whose depths are not within the predetermined distance area Rr0 within the range of the driver's gaze (see FIG. 1).

[0019] During the following first period, the driver of vehicle 100 visually checks the area to the left rear of vehicle 100. At this time, the driving assistance device excludes second bicycle 105b and third bicycle 105c, which are within a predetermined distance in the area where the driver is looking, from the areas to be notified, and notifies the driver of the presence of the other bicycles, first pedestrian 103a, second pedestrian 103b, first bicycle 105a, and fourth bicycle 105d (see FIG. 2). In the example shown in FIG. 2, the driving assistance device assumes that the driver will not recognize areas that the driver's line of sight simply passes through during the transition from area Rr0 to area Rr1 as risk objects.

[0020] During the subsequent second period, the driver of the vehicle 100 visually checks the area to the right front of the vehicle 100. At this time, the driving assistance device further excludes a first pedestrian 103a who enters area Rr2, the depth of which is within a predetermined distance within the range of the driver's line of sight, from the targets to be notified. The driving assistance device notifies of the presence of a second pedestrian 103b, a first bicycle 105a, and a fourth bicycle 105d other than the first pedestrian 103a and the second and third bicycles 105b and 105c that were already excluded (see FIG. 3). In the example shown in FIG. 3, the driving assistance device assumes that the driver will not recognize areas that the driver's line of sight simply passes through during the transition from area Rr1 to area Rr2 as risk targets.

[0021] During the following third period, the driver of vehicle 100 visually checks the area directly ahead of vehicle 100. At this time, the driving assistance device excludes second bicycle 105b and third bicycle 105c from the notification targets because they are within region Rr3, the depth of which is within a predetermined distance within the driver's line of sight. However, because second bicycle 105b and third bicycle 105c have already been excluded, the driving assistance device continues to notify of the presence of second pedestrian 103b, first bicycle 105a, and fourth bicycle 105d (see FIG. 4).

[0022] During the following fourth period, the driver of the vehicle 100 visually checks the direction of travel of the vehicle 100. At this time, the driving assistance device excludes a fourth bicycle 105d that has entered region Rr4 within a predetermined distance of the driver's line of sight from the targets of notification. The driving assistance device notifies the driver of the presence of the second pedestrian 103b and the first bicycle 105a other than the fourth bicycle 105d and the first pedestrian 103a, second bicycle 105b, third bicycle 105c, and fourth bicycle 105d that have already been excluded (see FIG. 5). In the example shown in FIG. 5, the driving assistance device assumes that the driver will not recognize areas that the driver's line of sight simply passes through during the transition from region Rr3 to region Rr4 as risk targets.

[0023] 1 to 5, after stopping vehicle 100, the driver checks the area behind vehicle 100 to the left for three seconds, the area in front of vehicle 100 to the right for two seconds, and the area directly ahead for one second, before starting vehicle 100. In the above example, as the driver moves his or her line of sight, the driver eventually sees all but second pedestrian 103b, first bicycle 105a, and fourth bicycle 105d, but a warning is issued to first pedestrian 103a, second bicycle 105b, and third bicycle 105c.

[0024] It is believed that the driver's line of sight (line of sight movement pattern) is patterned for each driver depending on the traffic scene. Therefore, it is believed that the annoyance felt by the driver due to the notification can be reduced by excluding pedestrians and bicycles that the driver is predicted to recognize based on the driver's line of sight movement pattern information from the notification targets.

[0025] Furthermore, even if the first bicycle 105a is not within the driver's visibility range, for example, if the first bicycle 105a has its lights on at night, the driver is likely to recognize the illumination of the lights as information suggesting the presence of the first bicycle 105a while looking in the direction of the first bicycle 105a. Nevertheless, in the above example, a warning is issued to the first bicycle 105a.

[0026] Even in such cases, it is believed that the annoyance felt by the driver due to the notification can be further reduced by excluding pedestrians and bicycles that are predicted to cause the driver to recognize information suggesting the presence of a risk object based on the driver's gaze movement pattern information from the targets of the notification. The driving assistance device according to the present disclosure is configured as a device that can further reduce the annoyance felt by the driver by excluding risk objects that the driver is aware of and that the driver should recognize from the targets of the notification based on the vehicle driver's gaze movement pattern information.

[0027] In addition, the second pedestrian 103b and the fourth bicycle 105d are in the driver's field of view at least while the driver is moving his or her line of sight, and it can be considered that the driver will recognize them if the amount of change in position (movement speed) of the moving object is large. In the above example, even in such a case, the second pedestrian 103b and the fourth bicycle 105d will be notified.

[0028] In such cases, it is believed that the annoyance felt by the driver from the notification target can be further reduced by excluding pedestrians and bicycles whose state change amount was large at the time the driver shifted his or her gaze from the notification target. The driving assistance device according to the present disclosure may exclude risk targets that are predicted to be recognized by the driver from the notification target, in which case it is possible to further reduce the annoyance felt by the driver.

[0029] <2. Overall configuration of the vehicle> Next, an example of the overall configuration of a vehicle equipped with a driving assistance device according to an embodiment of the present disclosure will be described.

[0030] FIG. 6 is a schematic diagram showing an example of the configuration of the vehicle 1. As shown in FIG. The vehicle 1 is equipped with a front camera 31a, a rear camera 31b, a left rear camera 31c, a right rear camera 31d, a LiDAR (Light Detection And Ranging) 31e, a radar sensor 31f, a vehicle status sensor 33, a position sensor 35, an interior camera 37, and a notification device 43.

[0031] The front photographing camera 31a, the rear photographing camera 31b, the left rear photographing camera 31c, the right rear photographing camera 31d, the LiDAR (Light Detection And Ranging) 31e, and the radar sensor 31f constitute an ambient environment sensor 31 for acquiring information about the ambient environment of the vehicle 1. The front photographing camera 31a, the rear photographing camera 31b, the left rear photographing camera 31c, and the right rear photographing camera 31d each include an imaging element such as a CCD (Charged Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and transmit the generated image data to the driving assistance device 50. The front photographing camera 31a is a pair of left and right stereo cameras that captures images of the area in front of the vehicle 1 and generates image data. In the vehicle 1 shown in FIG. 6, the front photographing camera 31a is configured as a stereo camera including a pair of left and right cameras, but the front photographing camera 31a may also be a monocular camera.

[0032] The LiDAR 31e detects an object present in front of the vehicle 1 and measures the position and speed of the object. The radar sensor 31f is, for example, a millimeter wave radar, and is provided at each of the four corners of the vehicle 1. Note that the surrounding environment sensor 31 may include one or more sensors other than the sensors shown in the figure.

[0033] The vehicle state sensor 33 is composed of at least one sensor that detects the operating state and behavior of the vehicle 1. The vehicle state sensor 33 includes, for example, a steering angle sensor, an accelerator position sensor, a brake stroke sensor, a brake pressure sensor, or an engine rotation speed sensor. The vehicle state sensor 33 also includes, for example, at least one of a vehicle speed sensor, an acceleration sensor, and an angular velocity sensor. The vehicle state sensor 33 transmits a sensor signal indicating the detected information to the driving assistance device 50.

[0034] The position sensor 35 receives satellite signals transmitted from satellites of a Global Navigation Satellite System (GNSS), such as a Global Positioning System (GPS). The satellite signals include position information of the vehicle 1, such as latitude and longitude. The position sensor 35 transmits a sensor signal indicating the acquired position information to the driving assistance device 50.

[0035] The interior camera 37 captures images of the interior of the vehicle and generates image data. The interior camera 37 is equipped with an imaging element such as a CCD or CMOS, and transmits the generated image data to the driving assistance device 50. The interior camera 37 is installed in a position where it can capture an image of at least the face of the driver.

[0036] The notification device 43 is driven by the driving assistance device 50 and notifies the driver of various information by means of image display, audio output, etc. The notification device 43 includes, for example, a display device provided in the instrument panel and a speaker provided in the vehicle 1. The display device may be a display device that displays information from a navigation system. The notification device 43 may also include a HUD (head-up display) that displays information on the front window superimposed on the scenery around the vehicle 1. The speaker may be a speaker of the audio system of the vehicle 1 or a speaker dedicated to the driving assistance device 50.

[0037] <3. Driving assistance devices> Next, the driving assistance device 50 according to this embodiment will be described in detail.

[0038] (3-1. Configuration example) The driving assistance device 50 functions as a device that assists in driving a vehicle by having one or more processors, such as CPUs (Central Processing Units), execute a computer program. The computer program is a computer program that causes the processor to execute the operations, described below, that should be performed by the driving assistance device 50. The computer program executed by the processor may be recorded on a recording medium that functions as a storage unit (memory) 53 provided in the driving assistance device 50, or may be recorded on a recording medium built into the driving assistance device 50 or any recording medium that can be externally attached to the driving assistance device 50.

[0039] Recording media for recording computer programs include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs (Compact Disc Read Only Memory), DVDs (Digital Versatile Discs), and Blu-ray (registered trademark), magneto-optical media such as floptical disks, memory elements such as RAMs and ROMs, flash memories such as USB (Universal Serial Bus) memories and SSDs (Solid State Drives), and other media capable of storing programs.

[0040] FIG. 7 is a block diagram showing an example of the configuration of the driving assistance device 50 according to this embodiment. The driving assistance device 50 is connected to a surrounding environment sensor 31, a vehicle state sensor 33, a position sensor 35, and an in-vehicle camera 37 via a dedicated line or communication means such as a CAN (Controller Area Network) or a LIN (Local Inter Net). In addition, a notification device 43 is connected to the driving assistance device 50.

[0041] The driving assistance device 50 is not limited to an electronic control device mounted on the vehicle 1, but may be a terminal device such as a smartphone or a wearable device.

[0042] The driving assistance device 50 includes a processing unit 51, a storage unit 53, and a gaze movement pattern storage unit 55. The processing unit 51 is configured to include one or more processors such as a CPU and various peripheral components. Part or all of the processing unit 51 may be configured with updatable firmware or the like, or may be a program module or the like that is executed by commands from the CPU or the like.

[0043] (Storage part) The storage unit 53 is configured with one or more storage elements such as RAM or ROM connected to the processing unit 51 so as to be able to communicate with it. However, the type and number of storage units 53 are not particularly limited. The storage unit 53 stores information such as computer programs executed by the processing unit 51, various parameters used in arithmetic processing, detection data, and arithmetic results. A part of the storage unit 53 is used as a work area for the processing unit 51.

[0044] (Gaze movement pattern memory unit) The gaze movement pattern storage unit 55 is configured by a storage element such as RAM or ROM communicably connected to the processing unit 51, or a storage medium such as an HDD, CD, DVD, SSD, USB flash, or storage device. The gaze movement pattern storage unit 55 stores a series of time-series information on the driver's visual behavior (gaze movement pattern information) acquired when the vehicle 1 passes through a predetermined traffic scene. The gaze movement pattern information indicates a time-series change in the direction of the driver's gaze in the predetermined traffic scene.

[0045] The traffic scene is set in advance as a traffic scene that requires careful driver behavior, for example. For example, the traffic scene is a scene in which the vehicle 1 crosses a pedestrian crossing, and is further classified as a scene in which the vehicle 1 is going straight, turning left, or turning right. The gaze movement pattern information is linked to the type of traffic scene and stored together with identification information set for each driver.

[0046] The gaze movement pattern information also includes information on gaze dwell areas where the gaze dwell time is equal to or longer than a predetermined time, and gaze transition areas where the gaze dwell time is shorter than the predetermined time, depending on the duration of gaze direction (gaze dwell time). Measurement of the gaze dwell time is determined, for example, based on whether the angular velocity of the gaze movement is less than 90 degrees per second. For example, an area where the gaze is directed at an angular velocity of less than 90 degrees per second for one second or longer is set as a gaze dwell area, and an area where the gaze has moved at an angular velocity of 90 degrees per second or greater is set as a gaze transition area.

[0047] The depth (radius) distance of the line-of-sight stagnation area and line-of-sight transition area may be arbitrarily set as a distance at which the driver can recognize the risk object, for example, set to 70 m. The depth distance may be variable depending on the driver's visual acuity, or may be variable depending on visibility conditions caused by the surrounding brightness or weather. For example, the driver's visual acuity is input in advance into the driving assistance device 50 by the driver. In addition, the surrounding brightness and weather are obtained from the detection results of a camera or illuminance meter, or by communication with a telematics service.

[0048] FIG. 8 is a conceptual diagram showing gaze movement pattern information of a driver in the traffic scene illustrated in FIGS. 1 to 5. In the example traffic scene illustrated in FIGS. 1 to 5, the driver stops vehicle 100, then directs his gaze to the rear left for three seconds, then to the front right for two seconds, then to the front forward for another second, and then directs his gaze in the direction of travel before starting vehicle 100. Therefore, the gaze movement pattern information includes a first gaze dwell area Rr1, a second gaze dwell area Rr2, and a third gaze dwell area Rr3, which change over time, in this order. The first gaze dwell area Rr1 is the area where the driver's gaze was directed during a first period from time T=t to time T=t+3 when vehicle 100 was stopped. The second gaze dwell area Rr2 is the area where the driver's gaze was directed during a second period from time T=t+3 to time T=t+5. The third gaze staying area Rr3 is the area where the gaze was directed during a third period from time T=t+5 to time T=t+6. Furthermore, the area where the gaze was directed from the time vehicle 100 was stopped until it was started moving is not determined to be a gaze staying area, and is set as a gaze transition area Rt1.

[0049] The gaze movement pattern information learned for each driver may be information on the average change in gaze direction over time in each traffic scene, information on the most frequent gaze movement pattern, or information on the most recent gaze movement pattern.

[0050] (3-2. Functional configuration of the processing unit) Next, the functional configuration of the processing unit 51 of the driving assistance device 50 will be described. The processing unit 51 includes an acquisition unit 61, a surrounding environment detection unit 63, a traffic scene detection unit 65, a risk detection processing unit 67, a gaze detection unit 69, a notification processing unit 71, and a gaze movement pattern learning unit 73. Each of these units is a function realized by execution of a computer program by one or more processors such as a CPU. However, part of the acquisition unit 61, the surrounding environment detection unit 63, the traffic scene detection unit 65, the risk detection processing unit 67, the gaze detection unit 69, the notification processing unit 71, and the gaze movement pattern learning unit 73 may be configured using analog circuits.

[0051] (Acquisition Department) The acquisition unit 61 acquires information or messages transmitted from the surrounding environment sensor 31 , the vehicle state sensor 33 , the position sensor 35 and the in-vehicle camera 37 .

[0052] (Ambient environment detection section) The surrounding environment detection unit 63 detects the surrounding environment of the vehicle 1. Based on image data or detection information transmitted from the surrounding environment sensor 31, the surrounding environment detection unit 63 detects obstacles such as moving and stationary objects present around the vehicle 1, as well as the surrounding environment such as crosswalks and boundary lines on the road. The surrounding environment detection unit 63 may further identify the position of the vehicle 1 on map data based on the position information of the vehicle 1 transmitted from the position sensor 35, and acquire information about the roads around the vehicle 1. In this case, the road information includes information such as the type of road, the number and width of lanes, crosswalks, intersections, or corners.

[0053] (Traffic Scene Detection Unit) The traffic scene detection unit 65 detects that the traffic scene in which the vehicle 1 is traveling is a predetermined traffic scene, based on the information about the surrounding environment detected by the surrounding environment detection unit 63. The predetermined traffic scenes are set in advance, such as a scene of passing beside a parked vehicle at the edge of a road, a scene of merging at a junction of two driving lanes, a scene of passing beside a blind spot such as a building, a scene of passing through an intersection without traffic lights, a scene of passing through a road with a pedestrian crossing without traffic lights, etc., and the traffic scene detection unit 65 determines whether the surrounding environment of the vehicle 1 corresponds to any of the traffic scenes.

[0054] (Risk detection processing unit) The risk detection processing unit 67 executes processing to detect risk objects based on information about the surrounding environment detected by the surrounding environment detection unit 63. Risk objects are objects with which the vehicle 1 may collide. The risk detection processing unit 67 detects objects with a possibility of collision as risk objects based on the positions, movement direction, and movement speed of moving bodies and stationary objects detected by the surrounding environment detection unit 63, and the position, movement direction, and movement speed of the vehicle 1. For example, the risk detection processing unit 67 assumes the traveling trajectory and movement speed of the vehicle 1 in a predetermined traffic scene, and detects objects with a possibility of approaching or colliding with the vehicle 1 at a certain time as risk objects.

[0055] (Gaze detection unit) The gaze detection unit 69 detects the direction of the gaze of the driver of the vehicle 1. For example, the gaze detection unit 69 detects the direction of the gaze of the driver based on image data of the driver's face captured by the in-vehicle camera 37.

[0056] (Notification processing unit) The notification processing unit 71 executes processing to notify the presence of a risk object detected by the risk detection processing unit 67. The notification processing unit 71 has a configuration to exclude risk objects that the driver may be able to recognize based on the line-of-sight movement pattern information from the notification targets, thereby reducing the annoyance caused to the driver by the notification.

[0057] For example, when the vehicle 1 is placed in a specified traffic scene, the notification processing unit 71 identifies a visibility risk present in the gaze stagnation area at the time the driver directs his or her gaze toward the gaze stagnation area based on the gaze movement pattern information stored in the gaze movement pattern memory unit 55, and excludes the visibility risk from the notification targets.

[0058] In addition, in this embodiment, the notification processing unit 71 identifies visible risks that are present in the gaze transition area at the time the driver's gaze passes through the gaze transition area, based on gaze movement pattern information, and whose predetermined state change amount is greater than a predetermined standard, and further excludes the visible risks from the notification targets.

[0059] In addition, in this embodiment, the notification processing unit 71 identifies suggested risks that are risk targets that are not present within the gaze dwell area at the time the driver directs their gaze toward the gaze dwell area, but for which information suggesting the presence of the risk target is present within the gaze dwell area, and excludes the suggested risks from the targets to be notified.

[0060] (Gaze movement pattern learning unit) The gaze movement pattern learning unit 73 learns gaze movement pattern information based on changes over time in the direction of the driver's gaze as the vehicle 1 passes through a specified traffic scene, and stores the information in the gaze movement pattern memory unit 55.

[0061] (3-3. Operation of driving assistance device) Next, an example of the processing operation by the processing unit 51 of the driving support device 50 according to this embodiment will be described with reference to a flowchart.

[0062] 9 and 10 show flowcharts of the driving assistance process performed by the processing unit 51. FIG. First, the processing unit 51 activates (turns on) the driving assistance function (step S11). The processing unit 51 may activate the driving assistance function based on an input operation by the driver, or may activate the driving assistance function together with activation of the system of the vehicle 1.

[0063] Next, the processing unit 51 identifies the driver of the vehicle 1 (step S13). For example, the processing unit 51 extracts feature points of the driver's face based on image data transmitted from the in-vehicle camera 37 and identifies the driver. The processing unit 51 assigns identification information to each identified driver and records the assigned identification information. The identification of the driver is not limited to the example performed based on image data. For example, the processing unit 51 may identify the driver based on the driver's name, nickname, etc. input by the driver himself via a touch panel or microphone.

[0064] Next, the gaze detection unit 69 of the processing unit 51 starts a process of detecting the direction of the driver's gaze based on the image data output from the in-vehicle camera 37 (step S15). The process of detecting the direction of the driver's gaze is repeatedly executed at a predetermined sampling period. The process of detecting the driver's gaze may be executed using a known technique, and therefore a detailed description thereof will be omitted.

[0065] Next, the surrounding environment detection unit 63 of the processing unit 51 starts processing to detect the surrounding environment of the vehicle 1 based on the information output from the surrounding environment sensor 31 (step S17). For example, the surrounding environment detection unit 63 detects the surrounding environment of the vehicle 1 based on the sensor signal or image data output from the surrounding environment sensor 31. The surrounding environment detection unit 63 extracts feature points from the image data and recognizes objects by matching the feature point patterns with pre-prepared reference data. The surrounding environment detection unit 63 recognizes various objects, including moving objects such as people, bicycles, motorcycles, and four-wheeled automobiles, as well as man-made or natural stationary objects, and lines painted on roads such as white lines and pedestrian crossings. The surrounding environment detection unit 63 also calculates the position and speed of the recognized objects, as well as the distance to the objects.

[0066] The surrounding environment detection unit 63 may also identify the position of the vehicle 1 on the map data based on the position information of the vehicle 1 transmitted from the position sensor 35, and acquire information on the road on which the vehicle 1 is traveling and information on surrounding buildings. The process of detecting the surrounding environment is repeatedly executed at a predetermined sampling period.

[0067] Next, the traffic scene detection unit 65 of the processing unit 51 determines whether the vehicle 1 passes through a predetermined traffic scene based on the detected information about the surrounding environment (step S19). For example, the traffic scene detection unit 65 determines whether the surrounding environment of the vehicle 1 corresponds to one of the predetermined traffic scenes based on information such as the type and position of one or more obstacles around the vehicle 1, the shape of the road, boundary lines and crosswalks, and the presence or absence of traffic lights.

[0068] If the traffic scene detection unit 65 does not determine that the vehicle 1 will pass through the predetermined traffic scene (S19 / No), the process proceeds to step S33. On the other hand, if the traffic scene detection unit 65 determines that the vehicle 1 will pass through the predetermined traffic scene (S19 / Yes), the notification processing unit 71 of the processing unit 51 reads out the current driver's gaze movement pattern information associated with the traffic scene that will be passed through from the gaze movement pattern information stored in the gaze movement pattern storage unit 55 (step S21).

[0069] If there is no gaze movement pattern information of the current driver associated with the traffic scene to be passed through, the notification processor 71 proceeds to step S33 without executing the process of excluding the traffic scene from the notification target. Alternatively, if there is no gaze movement pattern information of the current driver associated with the traffic scene to be passed through, the notification processor 71 may read out preset average gaze movement pattern information.

[0070] Next, the risk detection processing unit 67 executes a process of detecting a risk object based on the information on the surrounding environment detected by the surrounding environment detection unit 63 (step S23). For example, the risk detection processing unit 67 assumes the travel trajectory and movement speed of the vehicle 1 according to the traffic scene that the vehicle is about to pass through, and detects the object as a risk object at a certain time based on the travel trajectory and movement speed of the vehicle 1 and the position, movement speed and movement direction of each object.

[0071] Next, the notification processing unit 71 identifies a visibility risk, which is a risk object that may exist in the gaze stasis area of ​​the gaze movement pattern information at the timing when the driver directs his / her gaze to the gaze stasis area, from among the detected risk objects, and excludes the visibility risk from the notification targets (step S25). For example, the notification processing unit 71 identifies a visibility risk that exists in the gaze stasis area at the timing when the driver directs his / her gaze to the gaze stasis area until the vehicle 1 passes through the traffic scene, based on the position, movement speed, and movement direction of the risk object. Then, the notification processing unit 71 sets the identified visibility risk as a risk object to be excluded from the notification targets.

[0072] This visibility risk is a risk that the driver is estimated to recognize based on the gaze movement pattern information before the driver actually directs his or her gaze to the gaze stagnation area. Therefore, the visibility risk is excluded from the notification targets at an early stage when the vehicle 1 passes through a predetermined traffic scene.

[0073] Next, the notification processing unit 71 identifies visible risks, which are risk objects that exist in the line-of-sight transition area of ​​the line-of-sight movement pattern at the timing when the driver's line of sight passes through the line-of-sight transition area, from among the detected risk objects, and excludes the visible risks from the notification targets (step S27). For example, the notification processing unit 71 extracts risk objects that exist in the line-of-sight transition area at the timing when the driver's line of sight passes through the line-of-sight transition area before the vehicle 1 passes through the traffic scene, based on the position, movement speed, and movement direction of the risk objects. Furthermore, the notification processing unit 71 identifies visible risks, among the extracted risk objects, whose predetermined state change amount is greater than a predetermined standard. Then, the notification processing unit 71 sets the identified visible risks as risk objects to be excluded from the notification targets.

[0074] Even if the angular velocity of the line of sight movement in the line of sight transition area is greater than or equal to a predetermined time, it is believed that the driver will recognize the object if the change in the object is significant. Therefore, for example, the notification processor 71 identifies, as a visible risk, a risk object whose position change per unit time at the time the driver passes through the line of sight transition area exceeds a predetermined reference distance. Alternatively, the notification processor 71 may identify a bicycle or pedestrian as a visible risk when the lights of a bicycle that were off at the time the driver passes through the line of sight transition area are turned on (change in illuminance) or when a pedestrian suddenly moves part or all of their body (change in movement). The predetermined state change amount standard for determining the magnitude of the change in the risk object may be set to any appropriate value.

[0075] This visible risk includes risk objects that the driver is expected to recognize based on the gaze movement pattern information before the driver's gaze actually passes through the gaze transition area. For example, if the speed of the risk object is high and the amount of position change per unit time within the gaze transition area exceeds a reference distance, the driver is likely to notice the risk object. In this case, the visible risk is excluded from the notification targets at an early stage when the vehicle 1 passes through a predetermined traffic scene.

[0076] Visible risks also include risk targets that are estimated to be recognized by the driver when the driver's line of sight actually passes through the line of sight transition area. For example, if the lights of a bicycle that were off when the driver's line of sight passes through the line of sight transition area turn on (change in illuminance), or if a pedestrian suddenly moves part or all of their body (change in movement), the driver is likely to notice the bicycle or pedestrian. In this case, visible risks are initially subject to notification, but are excluded from the notification targets when the driver's line of sight passes through the line of sight transition area.

[0077] Next, the notification processing unit 71 identifies a suggested risk that is a risk object that is not present in the gaze dwell area at the time the driver directs his or her gaze to the gaze dwell area, but that is information that suggests the presence of the risk object within the gaze dwell area, and excludes the suggested risk from the targets to be notified (step S29). Even if the risk object itself is not present in the gaze dwell area, if information that suggests the presence of a risk object near the gaze dwell area exists within the gaze dwell area, it is considered that the driver will recognize the risk object.

[0078] For example, if the light from a bicycle outside the gaze dwelling area is illuminating the gaze dwelling area at the timing when the driver directs their gaze at the gaze dwelling area, or if the shadow of a moving object such as a bicycle or pedestrian outside the gaze dwelling area is present in the gaze dwelling area, the notification processing unit 71 will identify the moving object generating the light or shadow as a suggestive risk. The presence of the light or shadow can be detected based on image data from the camera.

[0079] This suggested risk includes a risk that is estimated to cause the driver to recognize information suggesting the presence of a risk object based on gaze movement pattern information before the driver's gaze is actually directed toward the gaze stasis area. For example, if a bicycle with its lights on is located outside the gaze stasis area, or if the shadow of a moving object such as a bicycle or pedestrian is generated outside the gaze stasis area, and the light or shadow is predicted to enter the gaze stasis area at the time the driver's gaze is directed toward the gaze stasis area based on the bicycle or pedestrian's position and movement speed, the driver is likely to notice the presence of the risk object. In this case, the suggested risk is excluded from the notification targets at an early stage when the vehicle 1 passes through a specified traffic scene.

[0080] Furthermore, the suggested risk includes a risk that is estimated to be recognized by the driver when the driver's gaze is actually directed toward the gaze dwell area. For example, if the light of a bicycle or the shadow of a bicycle or pedestrian enters the gaze dwell area at the time the driver's gaze is directed toward the gaze dwell area, the driver is likely to notice the bicycle or pedestrian. In this case, the suggested risk is a target for notification at an early stage, but is excluded from the target for notification when the driver's gaze is directed toward the gaze dwell area.

[0081] Next, the notification processing unit 71 determines whether or not a behavior change risk, which is a risk target that has undergone a predetermined behavior change at a timing when the driver is not looking at the risk target, has been detected among the risk targets excluded from the notification targets (step S31). For example, the notification processing unit 71 determines whether or not a risk target that has been excluded from the notification targets in any of steps S25 to S29 above has suddenly accelerated at a timing when the driver is not looking at the risk target.

[0082] In steps S25 to S29, the notification processing unit 71 excludes the corresponding risk target from the notification targets on the assumption that the driver has visually confirmed the position, movement speed, and movement direction of each risk target at that time. Nevertheless, if the behavior of the risk target changes thereafter, there is a risk that the vehicle 1 will collide with the risk target due to the risk target's movement that the driver does not expect. Therefore, the notification processing unit 71 detects risk targets that have undergone a predetermined behavioral change when the driver is not looking at them, among the risk targets excluded from the notification targets, in order to include the risk of a behavior change risk that has undergone a predetermined behavioral change when the driver is not looking at them as a notification target again.

[0083] When the notification processing unit 71 determines that a behavior change risk in which a predetermined behavior change occurred at a timing when the driver was not looking at the risk target excluded from the notification targets has not been detected (S31 / No), the notification processing unit 71 proceeds to step S33. On the other hand, when the notification processing unit 71 determines that a behavior change risk in which a predetermined behavior change occurred at a timing when the driver was not looking at the risk target excluded from the notification targets has been detected (S31 / Yes), the notification processing unit 71 restores the risk target (behavior change risk) to the notification targets.

[0084] In addition, after the notification processing unit 71 restores the behavior change risk to the notification target, if the risk target again falls under either a visible risk, a visible risk, or a suggested risk, the notification processing unit 71 will again exclude the risk from the notification target.

[0085] Next, the notification processing unit 71 determines whether or not there are any risk objects that are to be notified (step S35). The notification processing unit 71 determines whether or not there are any detected risk objects other than the risk objects excluded from the risk objects to be notified.

[0086] If the notification processing unit 71 does not determine that a risk object to be notified exists (S35 / No), the process proceeds to step S39 without issuing a notification. On the other hand, if the notification processing unit 71 determines that a risk object to be notified exists (S35 / Yes), the notification processing unit 71 notifies the driver of the presence of the risk object (step S37). For example, the notification processing unit 71 notifies the driver of the type, location, and direction of movement of the risk object by voice. Furthermore, instead of or in addition to the voice notification, the notification processing unit 71 may temporarily or continuously display an icon or text indicating the risk object on the image data of the camera. This allows the driver to recognize and pay attention to the risk object when passing through a traffic scene.

[0087] Next, the notification processing unit 71 determines whether the vehicle 1 has passed through a predetermined traffic scene (step S39). If the notification processing unit 71 does not determine that the vehicle 1 has passed through the predetermined traffic scene (S39 / No), the process returns to step S23 and repeats the processing of each step described above. On the other hand, if the notification processing unit 71 determines that the vehicle 1 has passed through the predetermined traffic scene (S39 / Yes), the gaze movement pattern learning unit 73 associates the data of the driver's gaze direction acquired while passing through this traffic scene with information about the traffic scene, and records the data together with the driver's identification information in the gaze movement pattern storage unit 55 (step S41).

[0088] At this time, gaze movement pattern learning unit 73 sets, for example, an area where the gaze is directed for one second or more at an angular velocity of less than 90 degrees / second as a gaze stagnant area, and sets an area where the gaze has moved at an angular velocity of 90 degrees / second or more as a gaze transition area, and records information on the gaze stagnant area and the gaze transition area together in gaze movement pattern storage unit 55. In this way, data on the driver's gaze movement pattern is accumulated in gaze movement pattern storage unit 55 together with information on the traffic scene.

[0089] Next, the processing unit 51 determines whether the driving support function has been turned off (step S43). If the processing unit 51 does not determine that the driving support function has been turned off (S43 / No), the processing returns to step S19 and repeats the processing of each step described above. On the other hand, if the processing unit 51 determines that the driving support function has been turned off (S43 / Yes), the processing ends.

[0090] <4. Application Examples> Next, a specific example of the notification process when the driving support device 50 according to this embodiment is applied will be described.

[0091] Fig. 11 is an explanatory diagram showing an example in which the notification process by the driving assistance device 50 of this embodiment is applied to the scene in which the vehicle 100 turns left from the public road 111 shown in Figs. 1 to 5. Fig. 11 shows the line-of-sight movement pattern information shown in Fig. 8.

[0092] 1 to 5, the second bicycle 105b and the third bicycle 105c are designated as targets for notification at time T=t when the vehicle 100 stops, and are removed from the targets for notification when they enter the first gaze stay area Rr1 during the first period from time T=t to time T=t+3. In contrast, in the example of this embodiment shown in Fig. 11, the second bicycle 105b and the third bicycle 105c are predicted to enter the first gaze stay area Rr1 during the first period from time T=t to time T=t+3 based on the gaze movement pattern information, and are removed from the targets for notification at time T=t when the vehicle 100 stops.

[0093] 1 to 5, the first pedestrian 103a is set as a notification target from time T=t when the vehicle 100 stops to time T=t+3, and is removed from the notification target when the first pedestrian enters the gaze stagnant region Rr2 during the second period from time T=t+3 to time T=t+5. In contrast, in the example of this embodiment shown in Fig. 11, the first pedestrian 103a is predicted to enter the gaze stagnant region Rr2 during the second period from time T=t+3 to time T=t+5 based on the gaze movement pattern information, and is removed from the notification target at time T=t when the vehicle 100 stops.

[0094] This allows risk objects that are predicted to be seen by the driver based on the driver's line of sight movement pattern information to be excluded from the notification targets from an early stage, thereby reducing the annoyance felt by the driver by the notification.

[0095] 12 illustrates an example in which the position change amount (movement speed) of the second pedestrian 103b is equal to or greater than a predetermined reference distance (reference speed) in the application example illustrated in FIG. 11. In this case, when the driver's gaze passes through the gaze transition area Rt1 while moving from the first gaze stagnant area Rr1 to the second gaze stagnant area Rr2, the driver may visually recognize the second pedestrian 103b moving through the gaze transition area Rt1 at a movement speed equal to or greater than the predetermined reference speed. Therefore, the notification processor 71 excludes the second pedestrian 103b from the notification targets when the driver's gaze moves to the second gaze stagnant area Rr2.

[0096] This allows risk objects that are present in the line of sight transition area Rt1 but that are presumed to have been seen by the driver to be excluded from the notification targets, thereby reducing the annoyance felt by the driver by the notification.

[0097] 13 illustrates an example of the application example shown in FIG. 11 in which, at the timing when the driver directs his or her gaze toward the first gaze retention area Rr1, information (illumination of a headlight) 106 indicating the presence of a first bicycle 105a is present within the first gaze retention area Rr1. In this case, while the driver's gaze is directed toward the first gaze retention area Rr1, the driver may notice the illumination of the headlight 106 and recognize the presence of the first bicycle 105a. For this reason, when the illumination of the headlight 106 enters the first gaze retention area Rr1 while the driver's gaze is directed toward the first gaze retention area Rr1, the notification processing unit 71 excludes the first bicycle 105a from the notification targets.

[0098] This means that risk objects that are presumed to have been seen by the driver, even if they are not in the gaze dwell area, are excluded from the notification targets, thereby reducing the annoyance felt by the driver by the notification.

[0099] As described above, the driving assistance device 50 according to this embodiment is configured to, when the vehicle 1 is placed in a predetermined traffic scene, identify a visual risk present in the gaze stagnation area at the timing when the driver directs his or her gaze to the gaze stagnation area based on gaze movement pattern information learned from the time-series movement pattern of the driver's gaze in the predetermined traffic scene, the pattern including a gaze stagnation area where the gaze is directed for a predetermined time or more and a gaze transition area where the gaze is directed for a time less than the predetermined time, and exclude the visual risk from the target of notification. This reduces the annoyance felt by the driver when a notification is given for a risk target that the driver has already viewed.

[0100] Furthermore, the driving assistance device 50 according to this embodiment identifies visible risks that are present in the gaze transition area at the time the driver's gaze passes through the gaze transition area and have a predetermined state change amount greater than a predetermined standard, based on the gaze movement pattern information, and excludes the visible risks from the targets to be notified. This eliminates risk targets that are not present in the gaze stagnation area but are estimated to be seen by the driver, further reducing the annoyance felt by the driver when a notification is issued for a risk target that the driver has already seen.

[0101] Furthermore, the driving assistance device 50 according to this embodiment identifies a suggested risk that is a risk object that is not present in the gaze dwell area at the time the driver directs his or her gaze toward the gaze dwell area but that contains information suggesting the presence of the risk object, and excludes the suggested risk from the targets to be notified. This eliminates risk objects that are not present in the gaze dwell area but that are estimated to be seen by the driver from the targets to be notified, further reducing the annoyance felt by the driver when a notification is issued for a risk object that the driver has already seen.

[0102] Furthermore, when a behavior change risk is detected among risk targets excluded from the notification targets, in which a predetermined behavior change occurs when the driver is not looking at the target, the driving support device 50 according to this embodiment restores the behavior change risk to the notification targets. As a result, when the behavior of the risk target changes while the driver is not aware of it, the presence of the risk target can be notified to the driver, and the driver can be urged to pay attention.

[0103] Although the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art to which the present disclosure pertains can conceive of various modifications or alterations within the scope of the technical ideas set forth in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

[0104] For example, in the example shown in FIG. 11 above, the driving assistance device 50 may further identify a suggested risk, based on the gaze movement pattern information, that is a risk object that is not present in the gaze stagnation area or the gaze transition area, but for which information suggesting the presence of the risk object and that a predetermined state change amount of the risk object is greater than a predetermined standard is present in the gaze transition area, and exclude the suggested risk from the list of objects to be notified. That is, in the example shown in FIG. 13 , if information suggesting the presence of a risk object is present in the gaze stagnation area, the risk object is excluded from the list of objects to be notified. However, if information suggesting the presence of a risk object is present in the gaze transition area, the risk object may also be excluded from the list of objects to be notified. This allows risk objects that do not fall under any of the above-mentioned visible risks, visible risks, and suggested risks to be excluded from the list of objects to be notified, thereby further reducing the annoyance felt by the driver when a notification is given for a risk object that the driver has already seen.

[0105] In addition, in the above embodiment, the process of assisting the driving of the vehicle is performed by the driving assistance device 50 mounted on the vehicle 1, but some or all of the functions may be provided by an external management server.

[0106] In addition, the technology of the present disclosure can also be realized as a vehicle equipped with the driving assistance device described in the above embodiment, a driving assistance processing method using the driving assistance device, a computer program that causes a computer to function as the above driving assistance system, and a non-transitory tangible recording medium on which the computer program is recorded. [Explanation of symbols]

[0107] 1: Vehicle 31: Ambient environment sensor 43: Notification device 50: Driving assistance device 51: Processing section 53: Storage section 55: Eye movement pattern memory unit 61: Acquisition part 63: Surrounding environment detection unit 65: Traffic scene detection unit 67: Risk detection processing section 69: Gaze detection unit 71: Notification processing unit 73: Eye movement pattern learning unit 100: Vehicle 103a: First Pedestrian 103b: Second Pedestrian 105a: First Bicycle 105b: Second Bicycle 105c: The third bike 105d: The fourth bicycle 106: Lighting 111: Public road 113: Roadway 115: Crosswalk

Claims

1. A driving assistance device that notifies a driver of the presence of a moving object around a vehicle, one or more processors; and one or more memories communicatively coupled to the one or more processors; the one or more processors: a risk detection process for detecting a risk object based on information about the surrounding environment of the vehicle; a notification process for notifying the presence of the detected risk object; In the notification process, A driving assistance device that, when the vehicle is placed in a predetermined traffic scene, identifies a visibility risk present in the gaze stagnation area at the time the driver directs his or her gaze toward the gaze stagnation area based on gaze movement pattern information that has learned a time-series movement pattern of the driver's gaze in the predetermined traffic scene, the gaze movement pattern including a gaze stagnation area where the gaze is directed for a predetermined period of time or more and a gaze transition area where the gaze is directed for a period of time less than the predetermined period of time, and excludes the visibility risk from the list of targets for notification.

2. the one or more processors:

2. The driving assistance device of claim 1, further comprising: a step of: identifying, based on the gaze movement pattern information, a visible risk that is present in the gaze transition area at the time the driver's gaze passes through the gaze transition area and whose predetermined state change amount is greater than a predetermined standard, and excluding the visible risk from the notification targets.

3. the one or more processors: The driving assistance device of claim 1, further comprising: a driver assistance system that, in the notification process, identifies a suggested risk that is not present within the gaze dwell area at the time the driver directs his or her gaze toward the gaze dwell area, and that indicates the presence of the risk object, and excludes the suggested risk from the notification targets.

4. the one or more processors:

2. The driving assistance device according to claim 1, further comprising: a driver assistance system that, in the notification process, identifies a suggested risk based on the gaze movement pattern information, the risk object that is not present within the gaze stagnation area or the gaze transition area, and information that suggests the presence of the risk object and that a predetermined amount of change in the risk object's state is greater than a predetermined standard, and excludes the suggested risk from the notification targets.

5. the one or more processors:

2. The driving assistance device of claim 1, wherein, in the notification process, if a behavioral change risk is detected among the risk objects excluded from the notification targets, in which a predetermined behavioral change occurs at a time when the driver is not looking, the behavioral change risk is restored to the notification targets.

Images