Notification control device, notification control method and computer program

The notification control device addresses the issue of bothersome warnings by detecting and projecting targets on a virtual plane aligned with the driver's view, adjusting notification criteria based on target count and type, and optimizing recognition times to prevent distractions.

JP2024162140APending Publication Date: 2024-11-21TOYOTA JIDOSHA KK +1
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
JP2023077399
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-09
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

Conventional notification systems issue bothersome warnings when drivers gaze at multiple external targets, as they evenly distribute their gaze without focusing on one, leading to unnecessary alerts.

Method used

A notification control device that detects targets and monitors the driver's gaze, sets a virtual plane for target projection based on the driver's field of view, and adjusts notification criteria based on the number and type of targets, including recognition determination times and weighting coefficients.

Benefits of technology

Prevents unnecessary warnings by ensuring drivers are not alerted when their gaze is distributed among multiple targets, reducing annoyance and potential distraction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024162140000001_ABST
    Figure 2024162140000001_ABST
Patent Text Reader

Abstract

To prevent a driver from being inconvenienced by a notification about an object requiring attention.SOLUTION: A notice control device 4 which performs a notification to a driver of a vehicle 100: identifies an object which exists around the vehicle 100 and requires attention based on surrounding data representing a surrounding state of the vehicle 100; sets a field of view of the driver based on driver data representing a state of the driver; places virtual planes on which the object is projected, if the number of objects is a predetermined value or less, on a location of the object opposite to the vehicle; performs a notification to the driver about existence of the object requiring attention if the field of view of the driver does not overlap with the location on which the object is projected on the virtual plane for a predetermined period since the object is identified; and does not perform the notification to the driver about the existence of the object if the number of objects is equal to or larger than the predetermined value.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a notification control device, a notification control method, and a computer program. [Background technology]

[0002] Patent Document 1 discloses a conventional warning device capable of issuing an alert depending on the driver's gaze state. The warning device is configured to calculate the gaze point at which the driver is gazing based on gaze information indicating the direction of the driver's gaze and eye position information indicating the position of the driver's eyeball on a three-dimensional coordinate system, determine whether the driver is gazing at the target based on the target to be gazed at and the gaze point, and issue an alert if it is determined that the driver is not gazing at the target. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-185218 A Summary of the Invention [Problem to be solved by the invention]

[0004] When there are a plurality of targets to be watched outside the vehicle, it is expected that the driver will not fix his / her gaze on one of the targets, but will turn his / her gaze evenly to each target to grasp the position of each target, etc. Therefore, in the above-mentioned conventional warning device, when there are a plurality of targets to be watched outside the vehicle, even if the driver can grasp each target, the driver may not turn his / her gaze to one of the targets, and a notification (warning) regarding the target to be watched may be given, and the driver may feel that the notification is annoying.

[0005] The present invention has been made in light of such problems, and has an object to prevent the driver from feeling annoyed by notifications regarding targets that should be watched. [Means for solving the problem]

[0006] The gist of the present disclosure is as follows.

[0007] (1) A notification control device for notifying a driver of a vehicle, the notification control device detecting targets that should be watched around the vehicle based on surrounding data that represents the situation around the vehicle, setting the driver's field of view based on driver data that represents the driver's state, and when the number of targets is a predetermined number or less, setting a virtual plane onto which the targets are projected to face the vehicle at the positions of the targets, and if the driver's field of view does not overlap with the projection location of the target on the virtual plane within a predetermined period after the detection of the target, notifying the driver that a target that should be watched is present, and when the number of targets is greater than the predetermined number, notifying the driver.

[0008] (2) A notification control device for notifying a driver of a vehicle, the notification control device detecting targets that should be watched around the vehicle based on surrounding data that represents the situation around the vehicle, setting the driver's field of view based on driver data that represents the driver's state, setting a virtual plane onto which the target is projected at the target's position so that it faces the vehicle, and determining that the driver recognizes the target if the driver's field of view overlaps with the projection location of the target on the virtual plane for a predetermined recognition determination time or more within a predetermined period after the target is detected, and if there are any targets among the detected targets that the driver does not recognize, notifying the driver that there are targets that should be watched, and configured to shorten the recognition determination time when the number of detected targets is large compared to when the number is small.

[0009] (3) A notification control device as described in (2) above, in which when the number of targets is a predetermined number or less, the recognition judgment time is a second judgment time, and when the number of targets is greater than the predetermined number, the recognition judgment time is a first judgment time that is shorter than the second judgment time.

[0010] (4) A notification control device described in any one of (1) to (3) above, configured to perform a correction on the number of detected targets using a weighting coefficient set for each type of target.

[0011] (5) A notification control method by a notification control device that notifies a driver of a vehicle, comprising: detecting targets that should be watched around the vehicle based on surrounding data that represents the situation around the vehicle; setting the driver's field of view based on driver data that represents the driver's state; when the number of targets is a predetermined number or less, setting a virtual plane onto which the targets are projected to face the vehicle at the positions of the targets; if the driver's field of view does not overlap with the projection location of the targets on the virtual plane within a predetermined period after the detection of the targets, notifying the driver that a target that should be watched exists; and when the number of targets is greater than the predetermined number, notifying the driver.

[0012] (6) A notification control method by a notification control device that notifies a driver of a vehicle, the notification control method comprising the steps of: detecting targets that should be watched around the vehicle based on surrounding data that represents the situation around the vehicle; setting the driver's field of view based on driver data that represents the driver's state; setting a virtual plane onto which the targets are projected so as to face the vehicle at the positions of the targets; determining that the driver recognizes the target if the driver's field of view overlaps with the projection location of the target on the virtual plane for a predetermined recognition determination time or more within a predetermined period after the target is detected; and if there are any targets among the detected targets that the driver does not recognize, notifying the driver that there are targets that should be watched; and shortening the recognition determination time when the number of detected targets is large compared to when the number is small.

[0013] (7) A computer program for a notification control device that notifies a driver of a vehicle, the computer program causing the notification control device to detect targets that should be watched around the vehicle based on surrounding data that represents the situation around the vehicle, set the driver's field of view based on driver data that represents the driver's state, and when the number of targets is a predetermined number or less, set a virtual plane onto which the targets are projected so that it faces the vehicle at the target positions, and if the driver's field of view does not overlap with the projection location of the targets on the virtual plane within a predetermined period after the target is detected, notify the driver that there is a target that should be watched, and when the number of targets is greater than the predetermined number, notify the driver.

[0014] (8) A computer program for a notification control device that notifies a driver of a vehicle, the computer program causing the notification control device to detect targets that should be watched around the vehicle based on surrounding data that represents the situation around the vehicle, set the driver's field of view based on driver data that represents the driver's state, set a virtual plane onto which the targets are projected at the positions of the targets so that it faces the vehicle, and if the driver's field of view overlaps with the projection location of the target on the virtual plane for a predetermined recognition determination time or more within a predetermined period after the target is detected, determine that the driver recognizes the target, and if there are any targets among the detected targets that the driver does not recognize, notify the driver that there are targets that should be watched, and shorten the recognition determination time when the number of detected targets is large compared to when the number is small. Effect of the Invention

[0015] According to these aspects of the present invention, when there are a large number of targets to watch outside the vehicle, it is possible to prevent unnecessary warnings from being issued to the driver, thereby preventing the driver from feeling annoyed by notifications about the targets. [Brief description of the drawings]

[0016] [Figure 1] 1 is a schematic system configuration diagram of a vehicle according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating a hardware configuration of a control device. [Diagram 3] 5 is a flowchart illustrating details of a notification process according to the first embodiment of the present invention. [Figure 4] FIG. 2 is a diagram illustrating a driver's field of view. [Figure 5A] 1 is a diagram illustrating an example of a method for determining whether or not a driver can recognize a gaze target; [Figure 5B] 1 is a diagram illustrating an example of a method for determining whether or not a driver can recognize a gaze target; [Figure 6] 10 is a flowchart illustrating details of a notification process according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, like components are designated by like reference numerals.

[0018] (First embodiment) FIG. 1 is a system schematic diagram of a vehicle 100 according to a first embodiment of the present invention.

[0019] The vehicle 100 includes a surrounding sensor 1, a driver sensor 2, an output device 3, and a control device 4. The surrounding sensor 1, the driver sensor 2, the output device 3, and the control device 4 are communicatively connected via an in-vehicle network 5 that complies with a standard such as a controller area network. The vehicle 100 may be a vehicle with an automatic driving function or a driving assistance function, or may be a manually driven vehicle.

[0020] The surrounding sensor 1 is a sensor for generating surrounding data that indicates the situation around the vehicle 100. In this embodiment, the surrounding sensor 1 includes one or more external cameras for capturing images of the surroundings of the vehicle 100. The external cameras capture images of the surroundings of the vehicle 100 at a predetermined frame rate (for example, 10 [Hz] to 40 [Hz]) and generate surrounding images showing the surroundings of the vehicle 100. Each time the external cameras generate a surrounding image, they transmit the generated surrounding image to the control device 4 as surrounding data.

[0021] Instead of or in addition to the external camera, a distance measuring sensor that measures the distance to targets such as other vehicles, pedestrians, and bicycles present around the vehicle 100 may be provided as the surrounding sensor 1. Examples of distance measuring sensors include a LiDAR (Light Detection And Ranging) that irradiates radar light and measures distance based on the reflected light, and a millimeter wave radar sensor that irradiates radio waves and measures distance based on the reflected waves.

[0022] The driver sensor 2 is a sensor for generating driver data representing the state of the driver. In this embodiment, the driver sensor 2 includes a driver monitor camera for capturing an image of the driver's appearance including the driver's face. The driver monitor camera captures the driver's appearance at a predetermined frame rate (for example, 10 [Hz] to 40 [Hz]) and generates an appearance image showing the driver's appearance. Every time the driver monitor camera generates an appearance image of the driver, it transmits the generated appearance image to the control device 4 as driver data.

[0023] The output device 3 is a device for notifying the driver of the vehicle 100 through the driver's bodily senses (for example, vision, hearing, touch, etc.). In this embodiment, the output device 3 includes a display (for example, a meter display, a center display, or a head-up display) arranged at a position visible to the driver, and a speaker. The display displays information (for example, text information or image information) according to a display signal output from the control device 4. The speaker outputs sound according to an audio signal output from the control device 4.

[0024] The control device 4 is an ECU (Electronic Control Unit) including a communication unit 41, a storage unit 42, and a processing unit 43 (see FIG. 2), and is configured at least to identify a target outside the vehicle to be watched (gaze target target) and to notify the driver of the target via the output device 3 according to the driver's recognition state of the target. In addition to the above-mentioned surrounding data and driver data, various data such as vehicle position data acquired by a positioning sensor and vehicle speed data acquired by a vehicle speed sensor can be input to the control device 4 as necessary.

[0025] FIG. 2 is a diagram showing a hardware configuration of the control device 4. As shown in FIG.

[0026] The control device 4 includes a communication unit 41, a storage unit 42, and a processing unit 43.

[0027] The communication unit 41 includes an interface circuit for connecting the control device 4 to the in-vehicle network 5. The communication unit 41 supplies data received from the outside (peripheral data, driver data, etc.) to the processing unit 43. The communication unit 41 also transmits the display signal and audio signal output from the processing unit 43 to the output device 3.

[0028] The storage unit 42 has a storage medium such as a hard disk drive (HDD), a solid disk drive (SSD), or a semiconductor memory, and stores various computer programs and data used for processing by the processing unit 43.

[0029] The processing unit 43 has one or more CPUs (Central Processing Units) and their peripheral circuits, and is, for example, a processor, which executes various computer programs stored in the storage unit 42. Hereinafter, an example of a notification process, which is one of the processes executed by the processing unit 43 and, in turn, the control device 4, will be described with reference to FIG.

[0030] FIG. 3 is a flowchart illustrating details of the notification process according to this embodiment.

[0031] In step S1, the control device 4 detects targets existing around the vehicle 100 based on the surrounding data received from the surrounding sensor 1, specifically, the surrounding image received from the external camera.

[0032] In this embodiment, the control device 4 detects the area in which the target is shown in the surrounding image and the type of the target shown in the area by sequentially inputting the surrounding images received from the external camera to the classifier. The classifier can be, for example, a convolutional neural network (CNN) having a plurality of convolution layers connected in series from the input side to the output side. The control device 4 estimates the distance from the external camera to the target using the standard size of the target stored in the storage unit 42 for each type of target and the size of the target detected in the surrounding image, and calculates the position and speed of the target by tracking the target detected in the surrounding image in a time series. Note that the method of detecting the target is not limited to such a method, and it is possible to detect the target using various known methods.

[0033] In step S2, the control device 4 identifies an object to be watched (hereinafter referred to as a "gaze target object") among objects present around the vehicle 100. The gaze target object may be, for example, another vehicle approaching the vehicle 100, a pedestrian, a bicycle, or the like, that is, an object that may collide with the vehicle 100. However, the present invention is not limited to this, and a predetermined object such as a pedestrian or a bicycle may be identified as the gaze target object, or an object whose distance from the vehicle 100 in a direction perpendicular to the traveling direction of the vehicle 100 (lateral direction) is smaller than a predetermined threshold value may be identified as the gaze target object.

[0034] In step S3, the control device 4 sets the field of view of the driver based on the driver data received from the driver sensor 2, specifically, on the external image of the driver received from the driver monitor camera.

[0035] In this embodiment, the control device 4 detects the position of the pupil center (i.e., the driver's viewpoint position) which is the moving point and the position of the Purkinje image (corneal reflection image) which is the reference point by sequentially performing image processing on the face image received from the driver monitor camera 21, and detects the driver's line of sight direction based on the positional relationship between the Purkinje image and the pupil center (the position of the moving point relative to the reference point).Then, as shown in Fig. 4, the control device 4 sets a visual cone having a predetermined apex angle (angle of view) with the driver's viewpoint position as the apex and the line of sight vector extending from the viewpoint position toward the line of sight direction as the central axis, as the driver's visual field.

[0036] The method of setting the driver's field of view is not limited to the above method, and may be set by various known methods. In this embodiment, the shape of the field of view is a cone as shown in Fig. 4, but the shape of the field of view is not limited to a cone, and may be other cones such as an elliptical cone or a polygonal cone. The viewing cone shown in Fig. 4 is a so-called straight cone whose central axis passes through the center of gravity of the bottom, but may be a so-called oblique cone whose central axis does not pass through the center of gravity of the bottom.

[0037] In step S4, the control device 4 determines whether or not the number of gazed-after objects (hereinafter referred to as the "gaze-target object number") Nobj is larger than a preset number Nth.

[0038] As described above, when there are a plurality of gaze target objects outside the vehicle 100, as the number of gaze target objects Nobj increases, it is assumed that the driver will not fix his / her gaze on one of the plurality of gaze target objects, but will turn his / her gaze evenly to each target object to grasp the position, etc. of each target object. Therefore, if a notification (alarm) regarding the gaze target object is given to the driver just because the driver's gaze is not directed to one of the plurality of gaze target objects, the notification will be given even though the driver is in the middle of turning his / her gaze to each gaze target object to grasp them, or even though the driver has grasped each gaze target object, and the driver may feel that the notification is bothersome.

[0039] Therefore, in this embodiment, it is determined whether the number of gazed-on objects Nobj is greater than a predetermined number Nth, and if the number of gazed-on objects Nobj is greater than the predetermined number Nth, the process is terminated without notifying the driver of the gazed-on object regardless of whether the gazed-on object is in the driver's field of view. If the number of gazed-on objects Nobj is equal to or less than the predetermined number Nth, the process proceeds to step S5, and it is determined whether the gazed-on object is in the driver's field of view in order to determine the necessity of the notification of the gazed-on object.

[0040] This makes it possible to suppress notification of the gaze target object even when the driver is in the process of directing his / her gaze to each gaze target object and grasping them, or even when the driver is able to grasp each gaze target object, when there are many gaze target objects outside the vehicle 100. Therefore, when there are many gaze target objects outside the vehicle, it is possible to suppress unnecessary warnings being issued to the driver.

[0041] In this embodiment, the predetermined number Nth is set to 6, but may be another value. In this embodiment, the number of gazed object targets Nobj is simply the number of targets. For example, when four pedestrians and one bicycle are detected as gazed object targets, the number of gazed object targets Nobj is 5. However, this is not limited to this, and the number of gazed object targets Nobj may be calculated after weighting according to the type of target, for example, the number of targets per pedestrian may be set to 1 and the number of targets per bicycle may be set to 2. That is, the number of targets may be corrected by a weighting coefficient set for each type of target. In this example, the weighting coefficient for pedestrians is 1 and the weighting coefficient for bicycles is 2. Therefore, when four pedestrians and one bicycle are detected as gazed object targets, the number of gazed object targets Nobj is 6 (=4 people × weighting coefficient 1 + 1 bicycle × weighting coefficient 2).

[0042] In step S5, the control device 4 judges whether the driver can recognize the gaze target object. If the driver can recognize each gaze target object, the control device 4 ends this processing without notifying the driver about the gaze target object. On the other hand, if the driver cannot recognize any gaze target object, the control device 4 proceeds to processing in step S6.

[0043] In this embodiment, the control device 4 sets the virtual plane on which the object to be gazed upon is projected so as to be perpendicular to the vehicle travel direction, for example as shown in FIG. 5A, and if the driver's field of view overlaps with the projection location of the object to be gazed upon on the virtual plane for a predetermined recognition judgment time Trec or more within a predetermined period after the object to be gazed upon is detected, it is determined that the driver has recognized the object to be gazed upon.

[0044] On the other hand, as shown in FIG. 5B, if the driver's field of view on the virtual plane does not overlap with the projection location of the gazed-up object for a predetermined period of time after the detection of the gazed-up object for a predetermined recognition judgment time Trec or more, the control device 4 determines that the driver is unable to recognize the gazed-up object.

[0045] 5A and 5B, the virtual plane is set at a position that is the shortest distance from the driver's viewpoint to the gaze target, but is not limited thereto, and may be set at a position that includes the gaze target. Also, the virtual plane is set for each gaze target.

[0046] In step S6, the control device 4 notifies the driver about the gaze target object via the output device 3. The notification method is not particularly limited, and for example, the driver may be notified by making a voice announcement that the gaze target object cannot be recognized, by issuing a warning sound, by making full use of image information or text information, or by a combination of these.

[0047] According to the present embodiment described above, the control device 4 (notification control device) for notifying the driver of the vehicle 100 detects objects to be gazed upon that exist around the vehicle 100 based on surrounding data that represents the situation around the vehicle 100, sets the driver's field of view based on driver data that represents the driver's state, and when the number of objects to be gazed upon Nobj is equal to or less than a predetermined number Nth, sets a virtual plane onto which the object to be gazed upon is projected at the position of the object to be gazed upon so as to face the vehicle 100, and if the driver's field of view does not overlap with the projection location of the object to be gazed upon on the virtual plane for a predetermined recognition judgment time or more within a predetermined period after the detection of the object to be gazed upon, notifies the driver that an object to be gazed upon exists, and is configured not to notify the driver when the number of objects to be gazed upon Nobj is greater than the predetermined number Nth.

[0048] As a result, when there are many gaze-target objects outside the vehicle 100, the driver is not notified of the gaze-target objects, so that it is possible to prevent notifications of the gaze-target objects from being issued to the driver, even when the driver is in the process of directing his / her gaze to each gaze-target object and grasping them, or even when the driver has grasped each gaze-target object. Therefore, when there are many gaze-target objects outside the vehicle 100, it is possible to prevent unnecessary warnings from being issued to the driver.

[0049] Second embodiment Next, a second embodiment of the present invention will be described. This embodiment differs from the first embodiment in that the recognition determination time Trec is changed according to the number of gazed-up objects Nobj, thereby suppressing unnecessary issuance of an alarm when a plurality of gazed-up objects are present outside the vehicle 100. The following mainly describes the differences.

[0050] As described above, when there are a plurality of gaze target objects outside the vehicle 100, if the number of gaze target objects Nobj increases, the driver is expected to grasp the position of each target by directing his / her gaze evenly to each target without fixing his / her gaze on one of the plurality of gaze target objects. In other words, if the number of gaze target objects Nobj increases, the time to visually recognize each target is expected to become relatively shorter.

[0051] Therefore, if the number of gazed-up objects Nobj is large, unless the recognition determination time Trec is shortened accordingly, it may be determined that the gazed-up object is not recognized even though the gazed-up object is visually recognized because the visual recognition time (the time during which the driver's field of vision overlaps the projection location of the gazed-up object) is short. As a result, even if the gazed-up object is recognized, a notification regarding the gazed-up object is made, and the driver may feel that the notification is annoying.

[0052] Therefore, in this embodiment, the recognition determination time Trec is changed according to the number of gaze target objects Nobj. Specifically, when the number of gaze target objects Nobj is large, the recognition determination time Trec is set shorter than when the number of gaze target objects Nobj is small. Hereinafter, an example of the notification process according to this embodiment will be described with reference to FIG. 6.

[0053] Fig. 6 is a flowchart for explaining the details of the notification process according to this embodiment. In Fig. 6, the contents of the process from step S1 to S6 are the same as those in the first embodiment, so the explanation will be omitted here.

[0054] In step S21, the control device 4 sets the recognition determination time Trec to a predetermined first determination time T1.

[0055] In step S22, the control device 4 sets the recognition determination time Trec to a predetermined second determination time T2.

[0056] The first judgment time T1 is set to a value smaller than the second judgment time T2. As a result, when the number Nobj of gazed object targets is large, the recognition judgment time Trec can be shortened, so that it is possible to suppress a case where a gazed object target is visually recognized but is not judged to be recognized.

[0057] It should be noted that the method of changing the recognition determination time Trec is not limited to the above method. For example, the recognition determination time Trec may be decreased gradually or in stages as the number of gaze target objects Nobj increases.

[0058] According to the present embodiment described above, the control device 4 (notification control device) for notifying the driver of the vehicle 100 identifies a gaze target object present around the vehicle 100 based on surrounding data representing the situation around the vehicle 100, sets the driver's field of view based on driver data representing the driver's state, sets a virtual plane onto which the gaze target object is projected at the position of the gaze target object so as to face the vehicle 100, and if the driver's field of view overlaps with the projection location of the gaze target object on the virtual plane for a predetermined recognition determination time Trec or more within a predetermined period after the gaze target object is detected, it determines that the driver recognizes the gaze target object. If there is an object among the gaze target objects that the driver does not recognize, a notification is given to the driver to inform him / her that there is an object to be gazed at, and is configured to shorten the recognition determination time Trec when the gaze target object number Nobj is large compared to when it is small.

[0059] In this embodiment, the control device 4 is configured to set the recognition judgment time Trec to a second judgment time T2 when the number of gazed-up object targets Nobj is equal to or less than a predetermined number Nth, and to set the recognition judgment time Trec to a first judgment time T1 which is shorter than the second judgment time T2 when the number of gazed-up object targets Nobj is greater than the predetermined number Nth.

[0060] As a result, when there are many gaze target objects outside the vehicle, the recognition determination time Trec can be shortened, so that it is possible to suppress the driver from determining that he / she is unable to recognize a gaze target object even though the gaze target object is visually recognized. Therefore, it is possible to suppress the driver from being notified about the gaze target object even though each gaze target object is grasped. In other words, when there are many gaze target objects outside the vehicle, it is possible to suppress the driver from being given an unnecessary warning.

[0061] Although the embodiments of the present invention have been described above, the above-mentioned embodiments merely show some of the application examples of the present invention, and it is not intended that the technical scope of the present invention be limited to the specific configurations of the above-mentioned embodiments.

[0062] For example, in the above embodiment, the computer program (notification process) executed in the control device 4 may be provided in a form recorded on a computer-readable portable recording medium, such as a semiconductor memory, a magnetic recording medium, or an optical recording medium. [Explanation of symbols]

[0063] 3 Output Device 4 Control Device (Notification Control Device) 100 vehicles

Claims

1. A notification control device for notifying a driver of a vehicle, Detecting a target to be watched around the vehicle based on surrounding data representing a situation around the vehicle; setting a field of view of the driver based on driver data representing a state of the driver; When the number of the targets is equal to or less than a predetermined number, a virtual plane onto which the targets are projected is set at the positions of the targets so as to face the vehicle, and if the driver's field of view does not overlap the projection location of the target on the virtual plane for a predetermined recognition determination time or more within a predetermined period after the detection of the target, a notification is given to the driver that a target to be watched is present; When the number of the targets is greater than the predetermined number, the notification is not performed for the driver. Notification control device.

2. A notification control device for notifying a driver of a vehicle, Detecting a target to be watched around the vehicle based on surrounding data representing a situation around the vehicle; setting a field of view of the driver based on driver data representing a state of the driver; a virtual plane on which the target is projected is set at the position of the target so as to face the vehicle, and if the driver's field of view overlaps with the projection location of the target on the virtual plane for a predetermined recognition determination time or more during a predetermined period after the target is detected, it is determined that the driver recognizes the target; If there is a target that the driver does not recognize among the detected targets, a notification is given to the driver to inform the driver that there is a target that the driver should look at; When the number of detected targets is large, the recognition determination time is shortened in comparison with when the number of detected targets is small. Notification control device.

3. When the number of the targets is equal to or less than a predetermined number, the recognition determination time is set as a second determination time; When the number of the targets is greater than the predetermined number, the recognition determination time is set to a first determination time that is shorter than the second determination time. The notification control device according to claim 2 .

4. The number of detected targets is corrected using a weighting coefficient set for each type of target. The notification control device according to claim 1 or 2.

5. A notification control method by a notification control device that notifies a driver of a vehicle, Detecting a target to be watched around the vehicle based on surrounding data representing a situation around the vehicle; setting a field of view of the driver based on driver data representing a state of the driver; When the number of the targets is equal to or less than a predetermined number, a virtual plane onto which the targets are projected is set at the positions of the targets so as to face the vehicle, and if the driver's field of view does not overlap the projection location of the target on the virtual plane for a predetermined recognition determination time or more within a predetermined period after the detection of the target, a notification is given to the driver that a target to be watched is present; When the number of the targets is greater than the predetermined number, the notification is not given to the driver. Notification control methods.

6. A notification control method by a notification control device that notifies a driver of a vehicle, Detecting a target to be watched around the vehicle based on surrounding data representing a situation around the vehicle; setting a field of view of the driver based on driver data representing a state of the driver; a virtual plane on which the target is projected is set at the position of the target so as to face the vehicle, and if the driver's field of view overlaps with the projection location of the target on the virtual plane for a predetermined recognition determination time or more during a predetermined period after the target is detected, it is determined that the driver recognizes the target; If there is a target that the driver does not recognize among the detected targets, a notification is given to the driver to inform the driver that there is a target that the driver should look at; When the number of detected targets is large, the recognition determination time is made shorter than when the number of detected targets is small. Notification control methods.

7. A computer program for a notification control device that notifies a driver of a vehicle, Detecting a target to be watched around the vehicle based on surrounding data representing a situation around the vehicle; setting a field of view of the driver based on driver data representing a state of the driver; When the number of the targets is equal to or less than a predetermined number, a virtual plane onto which the targets are projected is set at the positions of the targets so as to face the vehicle, and if the driver's field of view does not overlap the projection location of the target on the virtual plane for a predetermined recognition determination time or more within a predetermined period after the detection of the target, a notification is given to the driver that a target to be watched is present; When the number of the targets is greater than the predetermined number, the notification is not given to the driver. A computer program for causing the notification control device to execute the above.

8. A computer program for a notification control device that notifies a driver of a vehicle, Detecting a target to be watched around the vehicle based on surrounding data representing a situation around the vehicle; setting a field of view of the driver based on driver data representing a state of the driver; a virtual plane on which the target is projected is set at the position of the target so as to face the vehicle, and if the driver's field of view overlaps with the projection location of the target on the virtual plane for a predetermined recognition determination time or more during a predetermined period after the target is detected, it is determined that the driver recognizes the target; If there is a target that the driver does not recognize among the detected targets, a notification is given to the driver to inform the driver that there is a target that the driver should look at; When the number of detected targets is large, the recognition determination time is made shorter than when the number of detected targets is small. A computer program for causing the notification control device to execute the above.

Citation Information

Patent Citations

  • Object detection apparatus

    JP2007279970A

  • Vehicle surroundings monitor device

    JP2014006776A

  • Vehicle periphery monitoring equipment

    JP2014056295A

  • Collision avoidance device

    JP2016170728A

  • Alarm device

    JP2019185218A