Driving assistance device, driving assistance method, and program

Abstract

To provide a driving assistance device and the like, which are capable of issuing a warning at an appropriate time.SOLUTION: A driving assistance device 100 comprises: a visual line detection unit 101 for detecting a visual line of a first user driving a first vehicle; a target detection unit 102 for detecting a target around the first vehicle; a visual recognition determination unit 103 that determines whether the first user has visually recognized the target based on the detected visual line and the detected target; a position detection unit 104 for detecting a first position of the first vehicle; an acquisition unit 106 for acquiring a second position of a second vehicle when the target was previously and visually recognized by a second user; and an attention calling unit 107 that issues an attention calling to the first user to visually recognize the target at a timing determined based on a result of comparing the first position and the second position in the case that the visual recognition determination unit determines that the first user is not visually recognizing the target.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a driving assistance device, a driving assistance method, and a program. [Background technology]

[0002] Patent Document 1 discloses a driving support device that compares the driver's gaze distribution with a gaze distribution in a database and determines whether the driver is directing his / her gaze to a target around the vehicle that should be seen from the comparison result. This driving support device calls attention according to the driver's driving skill level based on the driving skill level calculated from the gaze distribution result. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-71537 A Summary of the Invention [Problem to be solved by the invention]

[0004] The above-mentioned conventional technology does not take into consideration the timing of issuing a warning to the driver, and therefore, even if a warning is issued to the driver, the vehicle may have passed the target that should be seen, and the driver may miss the target that should be seen around the vehicle.

[0005] The present disclosure provides a driving assistance device etc. that can issue a warning at an appropriate time. [Means for solving the problem]

[0006] A driving assistance device according to one embodiment of the present disclosure includes a gaze detection unit that detects the gaze of a first user who is driving a first vehicle, a target detection unit that detects targets around the first vehicle, a visibility determination unit that determines whether the first user has viewed the target based on the detected gaze and the detected target, a position detection unit that detects a first position of the first vehicle, an acquisition unit that acquires a second position of the second vehicle when the target was previously viewed by a second user, and if the visibility determination unit determines that the first user has not viewed the target, The line of sight used in the determination by the visual recognition determination unit and the target are detected at the same timing as the target is detected. and an attention calling unit that issues an attention calling to the first user to visually recognize the target at a timing determined based on a result of comparing the first position and the second position.

[0007] Furthermore, a driving assistance method according to one aspect of the present disclosure includes detecting a line of sight of a first user driving a first vehicle, detecting a target object around the first vehicle, determining whether the first user has viewed the target object based on the detected line of sight and the detected target object, detecting a first position of the first vehicle, acquiring a second position of the second vehicle when the target object was previously viewed by a second user, and if it is determined that the first user has not viewed the target object, The line of sight used in the determination and the target were detected at the same time that either one of the target was detected. An attention call is issued to prompt the first user to visually recognize the target at a timing determined based on a result of comparing the first position and the second position.

[0008] Moreover, a program according to an aspect of the present disclosure is a program for causing a computer to execute a driving assistance method according to an aspect of the present disclosure.

[0009] These comprehensive or specific aspects may be realized by a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized by any combination of an apparatus, a system, a method, an integrated circuit, a computer program, and a recording medium. Effect of the Invention

[0010] According to a driving assistance device or the like according to one aspect of the present disclosure, it is possible to issue a warning at an appropriate time. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing a configuration of a driving assistance system including a driving assistance device according to the first embodiment. [Diagram 2] FIG. 2 is a diagram for explaining a specific example of a target. [Diagram 3] FIG. 3 is a diagram for explaining a specific example of a target. [Figure 4] FIG. 4 is a diagram for explaining a specific example of a target. [Diagram 5] FIG. 5 is a flowchart illustrating an example of a process for storing visual confirmation information of the driving support device. [Figure 6] FIG. 6 is a flowchart showing an example of an attention calling process performed by the driving support device. [Figure 7] FIG. 7 is a diagram for explaining a specific example of the attention calling process in the first embodiment. [Figure 8] FIG. 8 is a block diagram showing a configuration of a driving assistance system including a driving assistance device according to the second embodiment. [Figure 9] FIG. 9 is a diagram for explaining a specific example of the attention calling process in the second embodiment. [Figure 10] FIG. 10 is a flowchart showing an example of the attention calling setting process according to the second embodiment. [Figure 11] FIG. 11 is a flowchart showing an example of the attention calling process according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] A driving assistance device according to a first aspect of the present disclosure includes a gaze detection unit that detects the gaze of a first user driving a first vehicle, a target detection unit that detects targets around the first vehicle, a visibility determination unit that determines whether the first user has viewed the target based on the detected gaze and the detected target, a position detection unit that detects a first position of the first vehicle, an acquisition unit that acquires a second position of the second vehicle when the target was previously viewed by a second user, and a warning unit that, when the visibility determination unit determines that the first user has not viewed the target, issues a warning to encourage the first user to view the target at a timing determined based on a result of comparing the first position and the second position.

[0013] According to this, a warning is issued to prompt the first user to view the target at a timing determined based on a result of comparing a first position of the first vehicle when it is determined that the first user has not viewed the target with a second position of the second vehicle when the second user previously viewed the target. In other words, a warning can be issued at a timing when the first user can view the target based on the position of the first vehicle, thereby reducing the first user from overlooking the target.

[0014] A driving assistance device according to a second aspect of the present disclosure is a driving assistance device according to the first aspect, wherein when the visibility determination unit determines that the first user is not viewing the target, the attention warning unit executes the attention warning at a timing when the detected first position is the same as the second position.

[0015] According to this, since the first user is alerted at least at the timing when the second user has previously visually recognized the target, it is possible to reduce the first user overlooking the target.

[0016] A driving assistance device according to a third aspect of the present disclosure is the driving assistance device according to the first aspect, The acquisition unit further acquires a third position of the first vehicle when the target was previously viewed by the first user, and when the visibility determination unit determines that the first user did not view the target and the third position is located at a position closer to the target than the second position, the warning unit executes the warning at a timing when the first position is at a fourth position farther from the target than the second position.

[0017] According to this, since the first user is alerted at a timing earlier than the timing at which at least the second user previously viewed the target, it is possible to reduce the first user overlooking the target.

[0018] A driving assistance device according to a fourth aspect of the present disclosure is a driving assistance device according to any one of the first to third aspects, further comprising: a memory unit that stores, for each target, attention warning setting information indicating a setting as to whether or not to execute the attention warning; and an update unit that updates the attention warning setting information of a target previously viewed by the first user at the third position, which is located farther from the target than the second position, to a setting indicating that the attention warning will not be executed, wherein the attention warning unit executes the attention warning for a first target for which the attention warning setting information stored in the memory unit is set to execute the attention warning, and does not execute the attention warning for a second target for which the attention warning is set not to be executed.

[0019] According to this, since it can be determined that a target that the first user has previously viewed earlier than the second user is a target that the first user is unlikely to overlook, the attention-call setting information is updated so as not to execute an attention-call. Therefore, the processing load can be reduced by skipping the attention-call for targets that the first user is likely to view, and the first user's feeling of complication can be reduced by calling attention to targets that are unlikely to overlook.

[0020] A driving assistance device according to a fifth aspect of the present disclosure is a driving assistance device according to any one of the first to third aspects, and further includes a memory unit that stores attention warning setting information indicating a setting for whether or not to execute the attention warning for each type of target, a reception unit that accepts changes to the setting, and an update unit that updates the attention warning setting information in accordance with the change accepted by the reception unit, wherein the attention warning unit executes the attention warning for a third target of a first type for which the attention warning is set to be executed in the attention warning setting information stored in the memory unit, and does not execute the attention warning for a fourth target of a second type for which the attention warning is set not to be executed.

[0021] According to this, in response to the accepted change, for example, the warning setting information is updated so as not to execute a warning, so that a necessary and sufficient warning can be given to the first user. In other words, the processing load can be reduced by skipping warnings that are unnecessary for the first user.

[0022] A driving assistance device according to a sixth aspect of the present disclosure is a driving assistance device according to any one of the first to fifth aspects, and further includes a memory unit that stores visibility information in which the judgment result by the visibility judgment unit is associated with the line of sight on which the judgment result is based and the position of the first vehicle when the target was detected, and the acquisition unit acquires the second position based on past visibility information stored in the memory unit.

[0023] Therefore, the visual confirmation results by the first user can be accumulated, and the accumulated results can be used to determine the timing to issue a warning. Therefore, since the latest visual confirmation results can be accumulated, the timing to issue a warning can be determined with high accuracy.

[0024] A driving assistance device according to a seventh aspect of the present disclosure is a driving assistance device according to any one of the first to sixth aspects, further comprising a speed detection unit that detects a first speed of the first vehicle, and the acquisition unit further acquires a second speed of the second vehicle when the target was previously viewed by the second user, and the timing is further determined based on a result of comparing the first speed and the second speed.

[0025] According to this, the timing for issuing a warning is determined according to the speed of each vehicle, so that the timing can be determined with high accuracy.

[0026] A driving assistance device according to an eighth aspect of the present disclosure is a driving assistance device according to any one of the first to seventh aspects, further comprising an environment detection unit that detects first environmental information indicating the state of the environment outside the first vehicle, and the acquisition unit further acquires second environmental information indicating the state of the environment outside the second vehicle when the target was previously viewed by the second user, and the timing is further determined based on a result of comparing the first environmental information and the second environmental information.

[0027] According to this, the timing for issuing a warning is determined according to the environment outside each vehicle, so that the timing can be determined with high accuracy.

[0028] A driving assistance device according to a ninth aspect of the present disclosure is a driving assistance device according to any one of the first to eighth aspects, wherein the second user has better driving skills than the first user.

[0029] According to this, the timing for issuing a warning is determined based on the second position of the second vehicle when previously viewed by a second user who has better driving skills than the first user, so that the timing can be determined more appropriately.

[0030] A driving assistance method according to a tenth aspect of the present disclosure detects the line of sight of a first user driving a first vehicle, detects targets around the first vehicle, determines whether the first user has viewed the target based on the detected line of sight and the detected target, detects a first position of the first vehicle, obtains a second position of the second vehicle when the target was previously viewed by a second user, and if it is determined that the first user has not viewed the target, issues a warning to encourage the first user to view the target at a timing determined based on a result of comparing the first position and the second position.

[0031] According to this, a warning is issued to prompt the first user to view the target at a timing determined based on a result of comparing a first position of the first vehicle when it is determined that the first user has not viewed the target with a second position of the second vehicle when the second user previously viewed the target. In other words, a warning can be issued at a timing when the first user can view the target based on the position of the first vehicle, thereby reducing the first user from overlooking the target.

[0032] A driving assistance according to an eleventh aspect of the present disclosure is a program for causing a computer to execute the driving assistance method according to the tenth aspect.

[0033] In addition, these comprehensive or specific aspects may be realized by a system, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or may be realized by any combination of an apparatus, a system, a method, an integrated circuit, a computer program, or a non-transitory recording medium.

[0034] Hereinafter, the embodiment will be specifically described with reference to the drawings.

[0035] In addition, the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, arrangement and connection forms of components, steps, order of steps, etc. shown in the following embodiments are examples and are not intended to limit the present disclosure. In addition, among the components in the following embodiments, components that are not described in an independent claim showing the highest concept are described as optional components. In addition, each figure is a schematic diagram and is not necessarily illustrated strictly. Therefore, for example, the scale, etc., does not necessarily match in each figure. In addition, the same reference numerals are given to the same components in each figure.

[0036] (Embodiment 1) [composition] FIG. 1 is a block diagram showing a configuration of a driving assistance system including a driving assistance device according to the first embodiment.

[0037] The driving assistance system 1 is mounted on a vehicle and assists a first user (the driver of the vehicle 10) in driving the vehicle. The driving assistance system 1 includes a driving assistance device 100, a storage device 120, a gaze detection device 210, an object detection device 220, a position detection device 230, a route setting device 240, a display device 310, and an audio device 320. The first user is a user who is driving the vehicle 10 and is a user whose gaze is to be detected.

[0038] First, the line of sight detection device 210, the object detection device 220, the position detection device 230, and the route setting device 240 will be described.

[0039] The gaze detection device 210 detects the gaze of the first user and outputs gaze information, which is the detection result, to the driving support device 100. For example, the gaze detection device 210 may detect the gaze of the first user based on the direction of the eyes of the first user detected from an image obtained from a camera arranged in front of the driver's seat that captures an image including the eyes of the first user. For example, the gaze detection device 210 sequentially detects the gaze of the first user at a predetermined sampling period and sequentially outputs the gaze information. The gaze detection device 210 is realized, for example, by a camera and a computer that executes a process of detecting the gaze of the first user from an image obtained by the camera.

[0040] The object detection device 220 detects objects around the vehicle and outputs the object detection result to the driving support device 100. The objects around the vehicle 10 include, for example, other vehicles, pedestrians, and other obstacles that are traveling around the vehicle 10. The object detection device 220 detects objects within a predetermined distance range from the vehicle 10. For example, the object detection device 220 may detect an object by photographing the surroundings of the vehicle 10 with a camera and recognizing the object in the image obtained by photographing using a machine learning model for detecting the object. The object detection device 220 may be, for example, a device that detects the distance to an object by irradiating a laser light such as LiDAR (Light Detection And Ranging) and measuring the time it takes for the laser light irradiated to the object to be reflected by the object and returned. In addition, the object detection device 220 detects the direction from the vehicle 10 to the detected object. In other words, the object detection device 220 detects the object around the vehicle 10 and the direction in which the object is located relative to the vehicle 10. The object detection device 220, for example, sequentially detects objects and their directions around the vehicle at a predetermined sampling period, and sequentially outputs the object detection results. The object detection device 220 may be a combination of a device that recognizes objects from a camera image and a device that detects objects from the detection results of LiDAR. In other words, the object detection device 220 is realized by a device including a camera and a computer that executes a process of recognizing objects from an image obtained by the camera, a device including a LiDAR and a computer that analyzes the detection results of the LiDAR and executes a process of detecting objects, or a device that combines these devices.

[0041] The position detection device 230 detects the position (traveling position) of the vehicle 10 and outputs the traveling position information, which is the detection result, to the driving assistance device 100. The traveling position information is, for example, position information indicating the current position of the vehicle 10 obtained by receiving a signal of a satellite positioning system such as a GPS (Global Positioning System) (not shown). The position detection device 230 may detect the traveling position of the vehicle 10 by comparing the detection result of the LiDAR with a detection result that is acquired in advance according to the traveling position of the vehicle 10 and stored in a storage device such as a memory. For example, the position detection device 230 sequentially detects the position of the vehicle 10 at a predetermined sampling period and sequentially outputs the detection result. The position detection device 230 is realized, for example, by a receiving device that receives a signal of the satellite positioning system and a computer that executes a process of calculating the position information from the signal.

[0042] The route setting device 240 is a device that sets a planned driving route from the driving position of the vehicle 10 detected by the position detection device 230 to a destination set by the first user. The route setting device 240 is realized by, for example, a car navigation system.

[0043] The predetermined sampling periods set in the gaze detection device 210, the object detection device 220, and the position detection device 230 may be the same as or different from each other. The gaze detection device 210, the object detection device 220, and the position detection device 230 only need to repeatedly perform detection, and the detection timings of these devices do not need to be synchronized with each other.

[0044] Next, the configuration of the driving assistance device 100 will be described.

[0045] The driving assistance device 100 includes a gaze detection unit 101 , a target detection unit 102 , a visibility determination unit 103 , a position detection unit 104 , a generation unit 105 , an acquisition unit 106 , and an attention calling unit 107 .

[0046] The gaze detection unit 101 detects the gaze of a first user who is driving the vehicle 10 (first vehicle) based on gaze information obtained by the gaze detection device 210. The gaze detection unit 101 obtains, for example, gaze information obtained at a predetermined sampling period from the gaze detection device 210, and sequentially detects the gaze of the first user at the predetermined sampling period.

[0047] The target detection unit 102 detects targets around the vehicle 10 based on the object detection results obtained by the object detection device 220. The target detection unit 102 detects targets around the vehicle 10 by extracting predetermined targets from the obtained object detection results. The targets are visual objects that the driver should visually recognize when driving the vehicle in order to drive the vehicle safely. The targets are road signs, road markings, traffic lights, road reflectors, etc. The targets may also include a direction in which a blind spot of a building exists at an intersection in a residential area. The target detection unit 102 detects the target and detects a direction associated with the target. That is, the target detection unit 102 detects the direction of the target in which the target is located relative to the vehicle 10 together with the target. For example, the target detection unit 102 obtains object detection results obtained at a predetermined sampling period from the object detection device 220, and sequentially detects targets at the predetermined sampling period.

[0048] 2 to 4 are diagrams for explaining specific examples of targets.

[0049] 2 shows landmarks at an intersection 403. As shown in FIG 2, the landmarks include a stop line 401, a lane 402, an intersection 403, a pedestrian signal light 404, a crosswalk 405, a vehicle signal light 406, and the like.

[0050] 3 shows landmarks at a T-junction 414. As shown in FIG 3, the landmarks include a road marking 411, a stop line 412, a road sign 413, a T-junction 414, a road sign 415, and the like.

[0051] 4 shows landmarks at a T-junction 423. As shown in FIG 4, the landmarks include a road sign 421, a stop line 422, a T-junction 423, and the like.

[0052] The visual recognition determination unit 103 determines whether the first user has visually recognized the target based on the visual line detected by the visual line detection unit 101 and the target detected by the target detection unit 102. Specifically, the visual recognition determination unit 103 compares the visual line direction with the target direction, and determines that the first user has visually recognized the target when the difference between the visual line direction and the target direction is less than a predetermined error, and determines that the first user has not visually recognized the target when the difference is equal to or greater than the predetermined error. The visual line and the target used for the determination are detection results detected at the same timing. The detection results detected at the same timing are, for example, detection results detected at two times with a time difference less than the longer of a predetermined sampling period for detecting the visual line and a predetermined sampling period for detecting the target. In other words, if the difference between the timing at which the visual line is detected and the timing at which the target is detected is less than the longer of a predetermined sampling period for detecting the visual line and a predetermined sampling period for detecting the target, the timing at which the two are detected may be considered to be the same. The visibility determination unit 103 outputs the determination result to the generation unit 105 .

[0053] The position detection unit 104 detects the position of the vehicle 10 (first position) based on the traveling position information obtained by the position detection device 230.

[0054] The generating unit 105 generates visual information including the determination result by the visual identification determination unit 103, a first position detected at the same timing as the timing when either the line of sight or the target used in the determination by the visual identification determination unit 103 was detected, and user identification information for identifying a user whose visual line of sight was detected. The user identification information may be specified by inputting the user's name when the user drives the vehicle 10, or may be specified by performing face recognition on an image including the user's face photographed by the visual line detection device 210, or may be specified by wireless communication with a terminal or circuit carried by the user. The visual identification information may further include route information indicating a route along which the vehicle 10 is traveling. The generating unit 105 outputs the visual identification information to the storage device 120.

[0055] The acquisition unit 106 reads out from the storage device 120 a plurality of past visual recognition information stored in the storage device 120 for each user, and acquires a second position of the vehicle 10 when the target detected by the target detection unit 102 was visually recognized by the second user in the past. Here, the second user is a user of the vehicle 10 driven by the first user, and is a user different from the first user. Note that the second user may be a user driving another vehicle. The second user is a user that serves as a criterion for the judgment of the attention calling unit 107 described later. The second user may be a user designated by the first user, or may be a user determined according to the level of the driving skill of the first user. The second user is, for example, a user with a higher driving skill than the first user.

[0056] When the visual recognition determination unit 103 determines that the first user has not visually recognized the target detected by the target detection unit 102, the attention calling unit 107 issues a warning to prompt the first user to visually recognize the target at a timing determined based on a result of comparing a first position, which is the current position of the vehicle 10, with a second position of the vehicle 10 when the target detected by the target detection unit 102 was previously visually recognized by the second user. Specifically, when the visual recognition determination unit 103 determines that the first user has not visually recognized the target detected by the target detection unit 102, the attention calling unit 107 issues a warning at a timing when the first position successively detected by the position detection unit 104 reaches within a section from the second position to the target. In other words, the attention calling unit 107 issues a warning at a timing when it is determined that the traveling vehicle 10 has reached the second position (or passed the second position). In the attention-calling, display information for causing the display device 310 to display characters or an image including a message urging the driver to visually recognize the target may be output to the display device 310. In addition, in the attention-calling, audio information for causing the audio device 320 to output an audio message urging the driver to visually recognize the target may be output to the audio device 320.

[0057] The storage device 120 stores the visual information output by the generation unit 105. The storage device 120 may store display information or audio information used for alerting. The storage device 120 is an example of a storage unit.

[0058] The display device 310 is a device that displays an image under the control of the driving support device 100, and is, for example, a display device such as a liquid crystal display, an organic EL display, or a HUD (Head Up Display). The display device 310 displays characters or images including a message based on display information output from the driving support device 100.

[0059] The audio device 320 is an audio device that outputs sound under the control of the driving assistance device 100. The audio device 320 may output sound by outputting an audio signal reproduced in the driving assistance device 100. The audio device 320 outputs the audio of a message based on the audio information output from the driving assistance device 100. The audio device 320 is realized by, for example, an amplifier and a speaker.

[0060] [Operation] Next, there will be described an operation of the driving support device 100. Fig. 5 is a flowchart showing an example of a process for storing visual confirmation information of the driving support device.

[0061] The driving assistance device 100 acquires route information indicating a planned driving route set by the route setting device 240 (S11).

[0062] Next, the driving assistance device 100 detects when the vehicle 10 starts to travel (S12). For example, the driving assistance device 100 may determine that the vehicle 10 has started to travel when the detected position of the vehicle 10 has transitioned from a state in which the position is not changing to a state in which the position is changing. Alternatively, the driving assistance device 100 may detect the traveling speed of the vehicle 10 and determine that the vehicle 10 has started to travel when the traveling speed becomes greater than 0.

[0063] Next, the driving assistance device 100 detects the line of sight of the first user who is driving the vehicle 10 (first vehicle) based on the line of sight information obtained by the line of sight detection device 210 (S13).

[0064] Next, the driving assistance device 100 detects targets around the vehicle 10 based on the object detection result obtained by the object detection device 220 (S14).

[0065] It should be noted that step S13 and step S14 may be executed in parallel, with step S13 not necessarily being executed before step S14, and step S14 may be executed before step S13.

[0066] Next, the driving assistance device 100 determines whether or not the first user has visually recognized the target based on the detected line of sight and the detected target (S15).

[0067] Next, the driving assistance device 100 detects the position (first position) of the vehicle 10 based on the traveling position information obtained by the position detection device 230 (S16).

[0068] Next, the driving assistance device 100 acquires auxiliary information (S17). The auxiliary information is, for example, information indicating the speed of the vehicle 10, the state of the environment outside the vehicle 10, etc. The state of the environment outside the vehicle 10 includes the date and time, the weather, etc.

[0069] Next, the driving assistance device 100 generates visual confirmation information including the determination result by the visual confirmation determination unit 103, a first position detected at the same timing as the timing when either the line of sight or the target used in the determination by the visual confirmation determination unit 103 was detected, and user identification information for identifying the user whose line of sight was detected (S18). The visual confirmation information may further include route information indicating a travel route on which the vehicle 10 is traveling, or may include auxiliary information.

[0070] Next, the driving assistance device 100 executes an attention calling process (S19). The attention calling process will be described in detail later with reference to FIG.

[0071] Next, the driving support device 100 determines whether the vehicle 10 has finished traveling (S20). If the driving support device 100 determines that the vehicle 10 has finished traveling (Yes in S20), it executes step S21, and if the driving support device 100 determines that the vehicle 10 has not finished traveling (No in S20), it returns to step S13. In step S21, the driving support device 100 stores the visual recognition information generated from the start of traveling to the end of traveling in the storage device 120 (S21).

[0072] FIG. 6 is a flowchart showing an example of the attention calling process (S19) performed by the driving support device.

[0073] The driving assistance device 100 identifies the route on which the vehicle 10 is traveling (S31). The driving assistance device 100 identifies the route on which the vehicle 10 is traveling based on a time-series change in the position of the vehicle 10 up to the present (the timing at which step S31 is executed). For example, the route on which the vehicle 10 is traveling may be represented by information including the current position of the vehicle 10 and the traveling direction of the vehicle 10.

[0074] Next, the driving assistance device 100 acquires from the storage device 120 visibility information including route information indicating the same route as the identified route of the vehicle 10 (S32).

[0075] Next, the driving assistance device 100 detects the position (first position) of the vehicle 10 based on the traveling position information obtained by the position detection device 230 (S33).

[0076] Next, the driving assistance device 100 acquires the second visual information, which is the visual information of the second user, from the storage device 120 (S34).

[0077] Next, the driving assistance device 100 acquires the first visual information, which is the visual information of the first user, from the storage device 120 (S35). Note that in step S35, the first visual information is acquired from the storage device 120, but this is not limiting, and the driving assistance device 100 may acquire information generated in step S18.

[0078] Next, the driving assistance device 100 compares the first visual information with the second visual information (S36).

[0079] Next, the driving support device 100 judges whether or not a first distance from a first position obtained from the first visual information to the detected target is equal to or less than a second distance from a second position obtained from the second visual information to the target (S37). That is, the driving support device 100 judges whether or not the first position is closer to the detected target than the second position, or whether or not the first position is the second position. In other words, the driving support device 100 judges whether or not the timing is when the first position has arrived within the section from the second position to the target (the vehicle 10 has arrived at the second position). When the driving support device 100 judges that the first distance is equal to or less than the second distance (Yes in S37), it executes step S38, and when the driving support device 100 judges that the first distance is longer than the second distance (No in S37), it ends the attention calling process in step S19. Note that, when it is judged as Yes in step S37, it means that it took the first user a longer time to visually recognize the target than the second user.

[0080] In step S38, the driving assistance device 100 issues a warning (S38).

[0081] Fig. 7 is a diagram for explaining a specific example of the attention calling process in the embodiment 1. Fig. 7 is a diagram showing the position of the vehicle 10 when the user of the vehicle 10 visually recognizes a road sign 431 as a target while the vehicle 10 is traveling.

[0082] Position P1 is the position of the vehicle when a first user visually recognizes road sign 431 when driving vehicle 10. Position P2 is the position of vehicle 10 (or another vehicle) when a second user visually recognizes road sign 431 when driving vehicle 10 (or another vehicle).

[0083] In this case, even if the vehicle 10 driven by the first user reaches position P2, the first user does not visually recognize the road sign 431, so a warning is issued at the time the vehicle 10 reaches position P2.

[0084] [Effects, etc.] The driving support device 100 according to the present embodiment includes a gaze detection unit 101, a target detection unit 102, a visual recognition determination unit 103, a position detection unit 104, an acquisition unit 106, and an attention calling unit 107. The gaze detection unit 101 detects the gaze of a first user who is driving the vehicle 10. The target detection unit 102 detects targets around the vehicle 10. The visual recognition determination unit 103 determines whether the first user has visually recognized the target based on the detected gaze and the detected target. The position detection unit 104 detects a first position of the vehicle 10. The acquisition unit 106 acquires a second position of the second vehicle when the target was previously visually recognized by the second user. When the visual recognition determination unit 103 determines that the first user has not visually recognized the target, the attention calling unit 107 executes an attention calling to prompt the first user to visually recognize the target at a timing determined based on a result of comparing the first position and the second position.

[0085] According to this, a warning is issued to prompt the first user to view the target at a timing determined based on a result of comparing a first position of the vehicle 10 when it is determined that the first user has not visually recognized the target with a second position of the second vehicle when the second user previously visually recognized the target. In other words, a warning can be issued at a timing when the first user can visually recognize the target based on the position of the vehicle 10, thereby reducing the first user overlooking the target.

[0086] In addition, in the driving assistance device 100 according to this embodiment, when the visibility determination unit 103 determines that the first user does not view the detected target, the attention warning unit 107 issues an attention warning at a timing when the detected first position is the same as the second position.

[0087] According to this, since the first user is alerted at least at the timing when the second user has previously visually recognized the target, it is possible to reduce the first user overlooking the target.

[0088] Moreover, the driving assistance device 100 according to the present embodiment further includes a storage device 120 that stores visual recognition information in which the determination result by the visual recognition determination unit 103 is associated with the line of sight on which the determination result is based and the position of the vehicle 10 when the target is detected. The acquisition unit 106 acquires the second position based on the past visual recognition information stored in the storage device 120.

[0089] Therefore, the visual confirmation results by the first user can be accumulated, and the accumulated results can be used to determine the timing to issue a warning. Therefore, since the latest visual confirmation results can be accumulated, the timing to issue a warning can be determined with high accuracy.

[0090] In addition, in the driving assistance device 100 according to the present embodiment, the second user has a higher driving skill than the first user. This allows the timing for issuing a warning to be determined based on the second position of the second vehicle previously viewed by the second user who has a higher driving skill than the first user, and therefore the timing can be determined more appropriately.

[0091] [Modification of the first embodiment] (1) The driving support device 100 according to the above embodiment may detect the vehicle speed of the vehicle 10, and store the detected vehicle speed in the storage device 120 while including it in the visual recognition information. In this case, the attention calling unit 107 of the driving support device 100 may adjust the timing of issuing an attention call in consideration of the vehicle speed. That is, the attention calling unit 107 may compare the detected first vehicle speed of the vehicle 10 with the second speed of the vehicle 10 (or the vehicle) when the target was previously visually recognized by the second user, and determine the timing of issuing an attention call based on the result of comparing the first speed and the second speed.

[0092] Specifically, when the first vehicle speed is faster than the second vehicle speed, the attention warning may be issued when the vehicle 10 passes a position before reaching the second position (i.e., a position where the distance from the target object is farther than the second position). Conversely, when the first vehicle speed is slower than the second vehicle speed, the attention warning may be issued when the vehicle passes a position after passing the second position (i.e., a position where the distance from the target object is closer than the second position). In this way, the timing for issuing the attention warning is determined according to the speed of each vehicle, so that the timing can be determined with high accuracy.

[0093] (2) The driving support device 100 according to the above embodiment may detect first environmental information indicating the state of the environment outside the vehicle 10, and store the detected environmental information in the storage device 120 by including it in the visual recognition information. In this case, the attention calling unit 107 of the driving support device 100 may adjust the timing of issuing an attention call in consideration of the state of the environment outside the vehicle 10. That is, the attention calling unit 107 may determine the timing of issuing an attention call based on a result of comparing the first environmental information with second environmental information indicating the state of the environment outside the vehicle 10 (or another vehicle) when the target was previously visually recognized by the second user.

[0094] Specifically, by narrowing down the second visual information acquired from the storage device 120 to the second visual information including the second environmental state that is the same as the first environmental information, it is possible to determine the timing to issue a warning based on the second visual information having the same environmental condition outside the vehicle 10. In this way, the timing to issue a warning is determined according to the environment outside each vehicle, so that the timing can be determined with high accuracy.

[0095] (Embodiment 2) [composition] FIG. 8 is a block diagram showing a configuration of a driving assistance system including a driving assistance device according to the second embodiment.

[0096] A driving assistance system 1A according to the embodiment 2 has a different configuration of a driving assistance device 100A compared to the driving assistance system 1 according to the embodiment 1. Therefore, the different configuration of the driving assistance device 100A will be mainly described.

[0097] The driving support device 100A is different from the driving support device 100 of the first embodiment in that it further includes a receiving unit 108, an updating unit 109, and a setting unit 110. The processes executed by the acquiring unit 106 and the attention calling unit 107 are different. Also, the information stored in the storage device 120 is different.

[0098] The storage device 120 stores, for each target, attention-call setting information indicating whether or not to execute an attention-call. The attention-call setting information may be determined for each type of target.

[0099] The acquisition unit 106 further acquires a third position of the vehicle 10 when the target was previously viewed by the first user.

[0100] When the visual recognition determination unit 103 determines that the first user does not visually recognize the target, and the third position is located at a position closer to the target than the second position, the attention calling unit 107 issues an attention call at the timing when the first position reaches within the section from the fourth position, which is farther from the target than the second position, to the target. That is, in this case, the attention calling unit 107 issues an attention call at the timing when it is determined that the traveling vehicle 10 has reached the fourth position (or passed the fourth position).

[0101] Fig. 9 is a diagram for explaining a specific example of the attention calling process in the embodiment 2. Fig. 9 is a diagram showing the position of the vehicle 10 when the user of the vehicle 10 visually recognizes a road sign 431 as a target while the vehicle 10 is traveling.

[0102] Position P12 is the position of the vehicle 10 (or another vehicle) when the second user visually recognized the road sign 431 when driving the vehicle 10 (or another vehicle) in the past. Position P13 is the position of the vehicle 10 when the first user visually recognized the road sign 431 when driving the vehicle 10 in the past. Position P14 is set to a position farther from the road sign 431 than position P12, and is a position indicating the timing to issue an attention call when the vehicle 10 driven by the first user passes through. In other words, when the first user visually recognized the road sign 431 at a timing later than the second user in the past, the attention call unit 107 issues an attention call at a timing when the first user is located at a fourth position P14 before the second position P12 when the first user next drives the vehicle 10 on the same route.

[0103] The accepting unit 108 accepts an input indicating a change in the settings from the first user.

[0104] The update unit 109 updates the attention call setting information of the target previously viewed by the first user at a third position that is located farther from the target than the second position to a setting indicating that attention will not be issued. In other words, the attention call setting information may be set so as not to issue an attention call to the specific target if the timing at which the first user previously viewed the specific target is earlier than the timing at which the second user previously viewed the specific target. Furthermore, the update unit 109 may update the attention call setting information in response to the change accepted by the acceptance unit 108.

[0105] The attention warning unit 107 may issue a warning to a first target for which the warning setting information stored in the memory device 120 is set to issue a warning, and may not issue a warning to a second target for which the warning setting information is set not to issue a warning.

[0106] [Operation] FIG. 10 is a flowchart showing an example of the attention calling setting process according to the second embodiment.

[0107] The attention calling setting process is the same as the attention calling process of the first embodiment from step S31 to step S37, so different processes will be mainly described.

[0108] If the driving assistance device 100A determines that the first distance is longer than the second distance (Yes in S37), it executes step S38a, and if the first distance is equal to or shorter than the second distance (No in S37), it executes step S38b.

[0109] In step S38a, the driving assistance device 100A adds the target object to the attention list in order to update the setting indicating that attention is to be issued to the target object that is the processing target of the attention calling process (S38a). The attention calling list is information indicating attention calling setting information set for each of a plurality of targets.

[0110] In step S38b, the driving assistance device 100A removes the target object that is the processing target of the attention drawing process from the attention drawing list in order to update the setting to indicate that attention drawing will not be executed for the target object (S38a).

[0111] FIG. 11 is a flowchart showing an example of the attention calling process according to the second embodiment.

[0112] The driving assistance device 100A acquires the attention-call target list from the storage device 120 (S41).

[0113] Next, the driving assistance device 100 detects the position (first position) of the vehicle 10 based on the traveling position information obtained by the position detection device 230 (S42).

[0114] Next, the driving assistance device 100 acquires the second visual information, which is the visual information of the second user, from the storage device 120 (S43).

[0115] Next, the driving assistance device 100A generates a fourth position by adding a margin to the second position obtained based on the second visual information (S44).

[0116] Next, the driving assistance device 100 acquires the first visual information, which is the visual information of the first user, from the storage device 120 (S45). Note that in step S35, the first visual information is acquired from the storage device 120, but this is not limiting, and the driving assistance device 100 may acquire information generated in step S18.

[0117] Next, the driving assistance device 100 compares the first position with the fourth position (S46).

[0118] Next, the driving support device 100 determines whether or not the first distance from the first position to the detected target is equal to or less than the fourth distance from the fourth position to the target (S47). That is, the driving support device 100 determines whether or not the first position is closer to the detected target than the fourth position, or whether or not the first position is the fourth position. In other words, the driving support device 100 determines whether or not the timing is such that the first position has arrived within the section from the fourth position to the target (the vehicle 10 has arrived at the fourth position). If the first distance is equal to or less than the fourth distance (Yes in S47), the driving support device 100 executes step S48, and if the first distance is not equal to or less than the fourth distance (No in S47), the driving support device 100 ends the attention calling process of step S19.

[0119] In step S48, the driving assistance device 100 issues a warning (S48).

[0120] [Effects, etc.] In the driving support device 100A according to the present embodiment, the acquisition unit 106 further acquires a third position of the first vehicle when the target was previously viewed by the first user. When the viewing determination unit 103 determines that the first user has not viewed the target and the third position is located at a position closer to the target than the second position, the attention calling unit 107 executes an attention calling at a timing when the first position is a fourth position farther from the target than the second position.

[0121] According to this, since the first user is alerted at a timing earlier than the timing at which at least the second user previously viewed the target, it is possible to reduce the first user overlooking the target.

[0122] The driving support device 100A according to the present embodiment further includes a storage device 120 and an update unit 109. The storage device 120 stores, for each target, attention call setting information indicating whether or not to execute an attention call. The update unit 109 updates the attention call setting information of a target previously viewed by the first user at a third position located farther from the target than the second position to a setting indicating that an attention call is not to be executed. The attention call unit 107 issues an attention call to a first target for which an attention call is set to be executed in the attention call setting information stored in the storage device 120, and does not issue an attention call to a second target for which an attention call is set not to be executed.

[0123] According to this, since it can be determined that a target that the first user has previously viewed earlier than the second user is a target that the first user is unlikely to overlook, the attention-call setting information is updated so as not to execute an attention-call. Therefore, the processing load can be reduced by skipping the attention-call for targets that the first user is likely to view, and the first user's feeling of complication can be reduced by calling attention to targets that are unlikely to overlook.

[0124] Moreover, the driving assistance device 100A according to the present embodiment further includes a storage device 120, a reception unit 108, and an update unit 109. The storage device 120 stores, for each type of target, attention call setting information indicating whether or not to execute an attention call. The reception unit 108 receives a change in the setting. The update unit 109 updates the attention call setting information in accordance with the change received by the reception unit 108. The attention call unit 107 issues an attention call to a third target of a first type for which an attention call is set to be executed in the attention call setting information stored in the storage device 120, and does not issue an attention call to a fourth target of a second type for which an attention call is set not to be executed.

[0125] According to this, in response to the accepted change, for example, the warning setting information is updated so as not to execute a warning, so that a necessary and sufficient warning can be given to the first user. In other words, the processing load can be reduced by skipping warnings that are unnecessary for the first user.

[0126] (Other embodiments) In the above embodiment, the storage device 120 included in the driving assistance system 1, 1A stores the visual information, but the present invention is not limited to this. An external storage device connected to the driving assistance system 1, 1A may acquire the visual information from the driving assistance system 1, 1A and store it. In this case, the driving assistance device 100, 100A may acquire the visual information for determining the timing of issuing an attention call from the external storage device. The external storage device is a device (a storage device on a network) communicably connected to the driving assistance system 1, 1A via a network, and is a device provided outside the vehicle 10. The external storage device is, for example, a cloud server.

[0127] Although the display control device and the like according to one or more aspects have been described based on the above embodiment, the present disclosure is not limited to the above embodiment. As long as it does not deviate from the gist of the present disclosure, various modifications conceived by a person skilled in the art to the above embodiment may also be included within the scope of the present disclosure.

[0128] Also, for example, in the above embodiment, each component of the processing unit of each device included in the driving assistance system 1, 1A may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU (Central Processing Unit) or a processor reading and executing a software program recorded on a non-transitory recording medium such as a hard disk or a semiconductor memory.

[0129] The following cases are also included in this disclosure:

[0130] (1) The at least one device is specifically a computer system consisting of a microprocessor, a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk unit, a display unit, a keyboard, a mouse, etc. A computer program is stored in the RAM or the hard disk unit. The at least one device achieves its function by the microprocessor operating in accordance with the computer program. Here, the computer program is composed of a combination of multiple instruction codes that indicate commands for a computer to achieve a specified function.

[0131] (2) Some or all of the components constituting at least one of the above devices may be composed of one system LSI (Large Scale Integration). The system LSI is an ultra-multifunctional LSI manufactured by integrating multiple components on one chip, and specifically, is a computer system composed of a microprocessor, ROM, RAM, etc. A computer program is stored in the RAM. The system LSI achieves its functions by the microprocessor operating in accordance with the computer program.

[0132] (3) Some or all of the components constituting at least one of the above devices may be composed of an IC card or a standalone module that is detachable from the device. The IC card or module is a computer system composed of a microprocessor, ROM, RAM, etc. The IC card or module may include the above-mentioned ultra-multifunction LSI. The IC card or module achieves its functions by the microprocessor operating according to a computer program. This IC card or module may be tamper-resistant.

[0133] (4) The present disclosure may be embodied as any of the methods described above, a computer program for implementing these methods on a computer, or a digital signal comprising a computer program.

[0134] The present disclosure may also be a computer program or a digital signal recorded on a computer-readable non-transitory recording medium, such as a flexible disk, a hard disk, a CD (Compact Disc)-ROM, a DVD, a DVD-ROM, a DVD-RAM, a BD (Blu-ray (registered trademark) Disc), a semiconductor memory, etc. Also, the present disclosure may be a digital signal recorded on such a recording medium.

[0135] The present disclosure may also involve transmitting a computer program or a digital signal via a telecommunications line, a wireless or wired communication line, a network such as the Internet, data broadcasting, or the like.

[0136] Furthermore, the program or digital signal may be implemented by another independent computer system by recording it on a recording medium and transferring it, or by transferring the program or digital signal via a network or the like. [Industrial Applicability]

[0137] The present disclosure is useful as a driving assistance device or the like that can issue a warning at an appropriate time. [Explanation of symbols]

[0138] 1. Driving assistance system 10 Vehicles 100 Driving support device 101 Line of sight detection unit 102 Target detection unit 103 Visibility Judgment Section 104 Position detection unit 105 Generation part 106 Acquisition Department 107 Alert section 108 Reception 109 Update section 120 Storage device 210 Line of Sight Detection Device 220 Object detection device 230 Position detection device 240 Routing Device 310 Display device 320 Sound equipment 401, 412, 422 stop line 402 lanes 403 Intersection 404 Pedestrian Signals 405 Crosswalk 406 Vehicle Signals 411 Road markings 412 Stop line 413, 415, 421 road signs 414, 423 T-junction

Claims

1. a gaze detection unit that detects a gaze of a first user who is driving a first vehicle; A target detection unit that detects targets around the first vehicle; a visibility determination unit that determines whether or not the first user has visually recognized the target based on the detected line of sight and the detected target; a position detection unit that detects a first position of the first vehicle; an acquisition unit that acquires a second position of the second vehicle when the target was previously viewed by a second user; and an attention calling unit that, when it is determined by the visibility determination unit that the first user does not visually recognize the target, issues an attention calling to the first user to visually recognize the target at a timing determined based on a result of comparing the line of sight used in the determination by the visibility determination unit and the first position and the second position detected at the same timing as a timing at which any of the targets was detected. Driving assistance device.

2. The attention calling unit executes the attention calling at a timing when the detected first position reaches within a section from the second position to the target when the visibility determining unit determines that the first user does not visually recognize the target. The driving assistance device according to claim 1 .

3. The acquisition unit further acquires a third position of the first vehicle when the target was previously viewed by the first user; When the visibility determination unit determines that the first user does not visually recognize the target and the third position is located at a position closer to the target than the second position, the attention calling unit executes the attention at a timing when the first position reaches within a section from a fourth position, which is farther from the target than the second position, to the target. The driving assistance device according to claim 1 .

4. moreover, A storage unit that stores, for each target, attention calling setting information indicating whether or not the attention calling is to be executed; an update unit that updates attention-call setting information of a target previously viewed by the first user at the third position that is farther from the target than the second position to a setting indicating that the attention will not be executed; The attention calling unit issues the attention to a first target for which the attention calling is set to be executed in the attention calling setting information stored in the storage unit, and does not issue the attention calling to a second target for which the attention calling is set not to be executed. The driving assistance device according to claim 3.

5. moreover, A storage unit that stores, for each type of target, attention call setting information indicating whether or not the attention call is to be executed; A reception unit that receives changes to the settings; an update unit that updates the alert setting information in response to the change accepted by the acceptance unit, The attention calling unit issues the attention to a third target of a first type for which the attention calling is set to be executed in the attention calling setting information stored in the storage unit, and does not issue the attention calling to a fourth target of a second type for which the attention calling is set not to be executed. A driving assistance device according to any one of claims 1 to 3.

6. moreover, a storage unit that stores visibility information in which a determination result by the visibility determination unit is associated with the line of sight on which the determination result is based and the position of the first vehicle when the target is detected, The acquisition unit acquires the second position based on past visual recognition information stored in the storage unit. A driving assistance device according to any one of claims 1 to 3.

7. moreover, a speed detection unit that detects a first speed of the first vehicle; The acquisition unit further acquires a second speed of the second vehicle when the target was previously viewed by the second user; When the first speed is faster than the second speed, the attention calling unit issues the attention at a timing when the first vehicle passes a position before reaching the second position, and when the first speed is slower than the second speed, the attention calling unit issues the attention at a timing when the first vehicle passes a position after passing the second position. A driving assistance device according to any one of claims 1 to 3.

8. moreover, an environment detection unit that detects first environment information indicating a state of an environment outside the first vehicle; The acquisition unit further acquires second environmental information indicating a state of an environment outside the second vehicle when the target was previously visually recognized by the second user; The attention calling unit determines the timing based on the second position acquired in the second environmental information that is in the same state as the first environmental information. A driving assistance device according to any one of claims 1 to 3.

9. The second user has a higher driving skill than the first user. A driving assistance device according to any one of claims 1 to 3.

10. Detecting a line of sight of a first user who is driving a first vehicle; Detecting targets around the first vehicle; determining whether the first user has visually recognized the target based on the detected line of sight and the detected target; Detecting a first location of the first vehicle; acquiring a second position of the second vehicle when the target was previously viewed by a second user; When it is determined that the first user does not visually recognize the target, a warning is issued to prompt the first user to visually recognize the target at a timing determined based on a result of comparing the line of sight used for the determination and the first position and the second position detected at the same timing as a timing when any of the targets was detected. Driving assistance methods.

11. A program for causing a computer to execute the driving support method according to claim 10.

Images