Eye-tracking device, eye-tracking method, and program

Abstract

The driver (U) will be able to understand the situation of their vehicle in relation to contact with object (6). [Solution] The eye-tracking device (7) is an eye-tracking device (7) that guides the gaze of the driver (U) of a vehicle (1), and comprises a first detection unit (101) that detects an object (6) located in front of the vehicle (1), and a display control unit (104) that, when the first detection unit (101) detects the object (6), displays an eye-tracking display (VI) on the windshield (3) of the vehicle (1) to guide the gaze of the driver (U) toward the object (U), wherein the display control unit (104) displays the eye-tracking display (VI) so as to move from the display start position (P2) of the eye-tracking display (VI) toward the object (6) by changing the speed of movement based on the risk level, which is the degree of possibility that the vehicle (1) will come into contact with the object (6).

Description

Technical Field

[0001] The present invention relates to a gaze guidance device, a gaze guidance method, and a program.

Background Art

[0002] In recent years, in order to further improve traffic safety and contribute to the development of a sustainable transportation system, research and development on the visibility of vehicle drivers has been carried out. Conventionally, as a technology related to the visibility of vehicle drivers, a technology for guiding the gaze of vehicle drivers is known. For example, Patent Document 1 discloses a gaze guidance device that guides a driver's gaze in the direction of a target by moving a visual stimulus displayed on a windshield. Further, Patent Document 1 discloses that the moving speed of the visual stimulus is increased as the angle formed by the target and the gaze increases.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, since Patent Document 1 is configured to move the visual stimulus to the target in the same required time regardless of the distance between the driver's viewpoint and the target, it is difficult for the driver to grasp from the visual stimulus the degree of the risk of contact with the target. Therefore, with the configuration of Patent Document 1, it is difficult for the driver to grasp the situation of the host vehicle regarding contact with the target. Therefore, the present invention has been made in view of the above circumstances, and an object thereof is to enable a driver to grasp the situation of the host vehicle regarding contact with an object.

Means for Solving the Problems

[0005] One aspect of the present invention is a gaze guidance device for guiding the gaze of a vehicle driver, comprising: a first detection unit for detecting an object located in front of the vehicle; and a display control unit that, when the first detection unit detects the object, displays a gaze guidance display on the windshield of the vehicle to guide the driver's gaze toward the object, wherein the display control unit displays the gaze guidance display so as to move from the display start position toward the object by changing the movement speed based on the risk level, which is the degree of possibility that the vehicle will come into contact with the object. [Effects of the Invention]

[0006] According to one aspect of the present invention, the driver can understand the situation of their vehicle in relation to contact with an object. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 shows the layout of the vehicle's interior. [Figure 2] Figure 2 shows the configuration of the vehicle. [Figure 3] Figure 3 is a diagram illustrating the processing of the display control unit. [Figure 4] Figure 4 is a diagram illustrating the determination of the second movement end position. [Figure 5] Figure 5 shows an example of the movement of eye-tracking indicators. [Figure 6] Figure 6 shows an example of the movement of eye-tracking indicators. [Figure 7] Figure 7 shows an example of the movement of eye-tracking indicators. [Figure 8] Figure 8 is a diagram illustrating the return of the eye-tracking indicator. [Figure 9] Figure 9 is a flowchart showing the operation of the eye-tracking device. [Modes for carrying out the invention]

[0008] [1. Vehicle Configuration] The embodiments will be described below with reference to the drawings. Figures 1 and 3-8 illustrate the X, Y, and Z axes. The X, Y, and Z axes are orthogonal to each other. The Z axis indicates the vertical direction. The X and Y axes are parallel to the horizontal direction when vehicle 1 is in motion. The X axis indicates the left-right direction as the vehicle width direction. The Y axis indicates the front-back direction. The positive direction of the X axis is to the right. The positive direction of the Y axis is forward. The positive direction of the Z axis is upward.

[0009] Figure 1 shows the interior layout of vehicle 1. In this embodiment, Vehicle 1 is exemplified by a four-wheeled automobile.

[0010] The interior of vehicle 1 is equipped with a steering wheel 2 for operating vehicle 1, a windshield 3 that separates the interior from the exterior, and an instrument panel 4. The steering wheel 2 is positioned on the instrument panel 4 opposite the driver U seated in the driver's seat.

[0011] The instrument panel 4 is equipped with a Head-Up Display (HUD) 5. The HUD 5 projects light onto the windshield 3 to display a gaze guidance display VI that guides the driver's (see Figure 4) gaze to an object 6 (for example, see Figure 4). The gaze guidance display VI is a virtual image. The object 6 represents an object that is desirable for the driver's (see Figure 4) attention. By displaying the gaze guidance display VI on the windshield 3, the driver (see Figure 4) seated in the driver's seat can see the gaze guidance display VI along with the scenery in front of the vehicle 1 through the windshield 3.

[0012] Note that Figure 1 shows a circular shape as an example of the shape of the eye-tracking indicator VI. However, the shape of the eye-tracking indicator VI shown in Figure 1 is merely an example, and it could also be rectangular or star-shaped, for example.

[0013] In FIG. 1, a displayable area A1 where a line-of-sight guidance display VI can be displayed is illustrated as an area closed by a dotted line L1. The size of the displayable area A1 corresponds to the size of an enlarged mirror (concave mirror) provided in the HUD 5. Note that the size of the displayable area A1 with respect to the windshield 3 is not limited to the size shown in FIG. 1. Also, in FIG. 1, a shape in which the longitudinal direction of the displayable area A1 is the vehicle width direction is illustrated, but the shape of the line-of-sight guidance display VI is not limited to the shape shown in FIG. 1. The displayable area A1 corresponds to the "first area".

[0014] <于 Also, in FIG. 1, a non-displayable area A2 where the line-of-sight guidance display VI is not displayed is illustrated. In FIG. 1, the non-displayable area A2 is an area on the windshield 3 excluding the displayable area A1. The non-displayable area A2 corresponds to the "second area".

[0015] FIG. 2 is a diagram showing the configuration of the vehicle 1. The vehicle 1 includes a line-of-sight guidance device 7. The line-of-sight guidance device 7 includes a processor 100 such as a CPU (Central Processing Unit) or MPU (Micro-processing unit), and a memory 110.

[0016] The processor 100 controls each part of the line-of-sight guidance device 7 by reading and executing a control program 111 stored in the memory 110. The processor 100 functions as an object detection unit 101, a line-of-sight detection unit 102, a head detection unit 103, a risk degree calculation unit 104, and a display control unit 105 by executing the control program 111 stored in the memory 110. The object detection unit 101 corresponds to the "first detection unit". The line-of-sight detection unit 102 corresponds to the "second detection unit".

[0017] Memory 110 is a storage device that stores programs executed by the processor 100 and data processed by the processor 100. Memory 110 stores control programs 111 executed by the processor 100 and various other data. Memory 110 has a non-volatile storage area. Memory 110 also has a volatile storage area and constitutes the work area of ​​the processor 100. Memory 110 is composed of, for example, ROM (Read Only Memory) and RAM (Random Access Memory). Control program 111 corresponds to "program".

[0018] The eye-tracking device 7 is connected to a HUD 5, a front camera 8, a driver monitoring camera 9, a position detection device 10, and a vehicle speed sensor 11. However, the devices connected to the eye-tracking device 7 are not limited to these; other devices such as a vehicle-to-vehicle communication device, a GNSS (Global Navigation Satellite System) unit, and a rear camera may also be connected.

[0019] The front camera 8 is installed at a predetermined position on the vehicle 1 and is a camera that photographs the area in front of the vehicle 1. The front camera 8 takes pictures at predetermined intervals when the ignition of the vehicle 1 is on or when the accessory power of the vehicle 1 is on. Each time the front camera 8 takes a picture, it outputs the image data of the captured image SG (see Figure 3) obtained from the picture to the eye-tracking device 7.

[0020] The driver monitoring camera 9 is installed in a predetermined position inside the vehicle 1 and is a camera that photographs the driver U seated in the driver's seat. The shooting range of the driver monitoring camera 9 includes at least the head HD (see Figure 4) of the driver U seated in the driver's seat. The driver monitoring camera 9 takes pictures at predetermined intervals when the ignition of the vehicle 1 is on or when the accessory power of the vehicle 1 is on. Each time the driver monitoring camera 9 takes a picture, it outputs the image data of the captured image to the eye guidance device 7.

[0021] The position detection device 10 is a device capable of detecting the position of objects present around the vehicle 1. The position detection device 10 consists of at least one of the following: a sonar, radar, lidar, etc., capable of measuring the distance between the vehicle 1 and an object, and a stereo camera capable of measuring the distance between the vehicle 1 and an object using parallax.

[0022] The vehicle speed sensor 11 is a sensor that detects the speed of vehicle 1. The vehicle speed sensor 11 detects the speed of vehicle 1 at predetermined intervals and outputs a signal corresponding to the detected speed of vehicle 1 to the eye guidance device 7 each time it detects a speed.

[0023] As described above, the processor 100 of the gaze guidance device 7 functions as an object detection unit 101, a gaze detection unit 102, a head detection unit 103, a risk level calculation unit 104, and a display control unit 105.

[0024] [1-1. Object Detection Unit] The object detection unit 101 detects an object 6 located in front of the vehicle 1. Based on the image data of the captured image SG received from the front camera 8, the object detection unit 101 detects the object 6 captured in the captured image SG obtained by the front camera 8. As described above, the object 6 refers to an object that should be directed at the driver U. Specific examples of the object 6 include pedestrians, other vehicles, and fixed objects. Examples of fixed objects include road signs and road traffic guidance markers. The object detection unit 101 detects the object 6 captured in the captured image SG by performing pattern matching and color-based image processing on the captured image SG. The data necessary for detecting the object 6 (for example, shape data and color data) is stored in the memory 110 for each type of object 6 to be detected.

[0025] Furthermore, the object detection unit 101 detects the position of the detected object 6. More specifically, the object detection unit 101 detects the relative position of the object 6 with respect to the vehicle 1 when the vehicle 1 is viewed from above. The object detection unit 101 detects the relative position of the detected object 6 based on at least one of the detection results of the position detection device 10 and the captured image SG of the front camera 8. If the detected object 6 is another vehicle and the vehicle-to-vehicle communication device and GNSS unit are connected to the eye-tracking device 7, the object detection unit 101 may detect the relative position of the detected object 6 based on the position of the other vehicle received by the vehicle-to-vehicle communication device and the position of vehicle 1 received by the GNSS unit.

[0026] In addition to the front camera 8, vehicle-to-vehicle communication device, and GNSS unit, the object detection unit 101 may also use V2X (vehicle-to-infrastructure or pedestrian-to-pedestrian communication, etc.) or determination in a virtual environment via a server to detect objects 6 in the captured image SG and to detect the relative position of objects 6.

[0027] When the object detection unit 101 detects an object 6, it outputs data indicating the relative position of the detected object 6 with respect to the vehicle 1 to the risk level calculation unit 104. Furthermore, when the object detection unit 101 detects an object 6, it outputs data indicating the relative position of the detected object 6 with respect to the vehicle 1, and data indicating the position of the detected object 6 in the captured image, to the display control unit 105.

[0028] [1-2. Eye-tracking unit] The gaze detection unit 102 detects the direction of the driver U's gaze. The gaze detection unit 102 detects the direction of the driver U's gaze based on the image data of the captured image received from the driver monitoring camera 9. The gaze detection unit 102 detects the driver U's eyes from the captured image obtained from the driver monitoring camera 9 using pattern matching, color, etc., and detects the direction the detected eyes are facing as the direction of the gaze. The data necessary for eye detection (data on eye shape and color) is stored in the memory 110.

[0029] When the gaze detection unit 102 detects the direction of the driver U's gaze, it outputs data indicating the detected direction of the gaze to the display control unit 105.

[0030] [1-3. Head detection unit] The head detection unit 103 detects the head HD of driver U seated in the driver's seat. The head detection unit 103 detects the head HD of driver U based on image data of the captured image received from the driver monitoring camera 9. The head detection unit 103 detects the head HD from the captured image obtained by the driver monitoring camera 9 using pattern matching, color, etc. Next, the head detection unit 103 detects the position of the head HD in the captured image. Then, based on the size of the head HD in the captured image and the position of the head HD in the captured image, the head detection unit 103 detects the position of the head HD in the vehicle 1 when viewed from above. The size of the head HD in the captured image and the position of the head HD in the captured image, and the position of the head HD in the vehicle 1 are determined by prior tests and simulations and stored as data in the memory 110.

[0031] [1-4. Risk Calculation Section] The risk calculation unit 104 calculates the risk level, which is the degree to which vehicle 1 is likely to come into contact with object 6.

[0032] For example, the risk level calculation unit 104 calculates the distance between the vehicle 1 and the object 6 based on the relative position data received from the object detection unit 101. Next, the risk level calculation unit 104 calculates a risk level from among the multi-stage evaluation values, which corresponds to the calculated distance between the vehicles. In this embodiment, the risk level is one of 11 evaluation values ​​from "0" to "10", and the larger the number of stages (the number in parentheses), the higher the risk level. For example, a risk level of "0" means there is no risk. Also, for example, "10" means the highest risk level. In this calculation method, the risk level calculation unit 104 calculates a risk level with a larger number of stages the shorter the calculated distance between the vehicles. Note that "0" is an example of a predetermined value that can be considered as having no possibility of contact with the object.

[0033] Furthermore, for example, the risk level calculation unit 104 calculates the relative speed between the vehicle 1 and the object 6 based on the multiple relative position data received from the object detection unit 101 and the detection result of the vehicle speed sensor 11. The risk level calculation unit 104 also calculates the distance between the vehicle 1 and the object 6 based on the relative position data received from the object detection unit 101. Next, the risk level calculation unit 104 calculates the time required for the vehicle 1 to make contact with the object 6 based on the calculated relative speed and distance. Then, the risk level calculation unit 104 calculates the risk level as an evaluation value corresponding to the calculated time required, within an 11-level evaluation scale. In this calculation method, the risk level calculation unit 104 calculates a higher risk level the shorter the calculated time required.

[0034] Furthermore, for example, if the eye-tracking device 7 is connected to a vehicle-to-vehicle communication device and a GNSS unit, and the object 6 detected by the object detection unit 101 is another vehicle, the risk level calculation unit 104 may calculate the risk level as follows. That is, the risk level calculation unit 104 calculates the relative speed between vehicle 1 and object 6, and the distance between vehicle 1 and object 6, based on the speed of vehicle 1 detected by the vehicle speed sensor 11, the position of vehicle 1 received by the GNSS unit, and the position and speed of the other vehicle received by the vehicle-to-vehicle communication device. Next, the risk level calculation unit 104 calculates the time required until vehicle 1 makes contact with object 6 based on the calculated relative speed and distance. Then, the risk level calculation unit 104 calculates the risk level as an evaluation value corresponding to the calculated time required, within a 10-level evaluation scale. In this risk level calculation, the risk level calculation unit 104 calculates a higher risk level the shorter the calculated time required.

[0035] It should be noted that the risk level calculation method described above is merely an example and is not limited to the method described above.

[0036] When the risk level calculation unit 104 calculates the risk level, it outputs data indicating the calculated risk level to the display control unit 105.

[0037] [1-5. Display Control Unit] The display control unit 105 controls the operation of the HUD5 to display the eye-tracking display VI on the windshield 3. The display control unit 105 displays the eye-tracking display VI on the windshield 3 and moves the eye-tracking display VI displayed on the windshield 3 by performing the following processing.

[0038] The processing of the display control unit 105 will be explained with reference to Figure 3. Figure 3 is a diagram illustrating the processing of the display control unit 105.

[0039] The display control unit 105 detects the position of the driver U's gaze on the windshield 3 (hereinafter referred to as "gaze position P1" with the designation "P1") based on the direction of the gaze indicated by the data received from the gaze detection unit 102. For example, if the memory 110 stores data that associates the direction of the driver U's gaze with the driver U's gaze position P1 on the windshield 3, the display control unit 105 refers to this data to detect the driver U's gaze position P1 on the windshield 3.

[0040] The display control unit 105 determines whether the detected driver U's gaze position P1 is within the displayable area A1. The memory 110 stores data indicating the position of the displayable area A1 on the windshield 3. The display control unit 105 determines whether the detected driver U's gaze position P1 is within the displayable area A1 by referring to this data stored in the memory 110.

[0041] If the display control unit 105 determines that the detected driver U's gaze position P1 is within the displayable area A1, it determines a position shifted by a predetermined distance L2 from the driver U's gaze position P1 as the display start position P2 of the gaze guidance display VI, as shown in Figure 3. The display control unit 105 then expands the coordinate system that defines the shape, size, and up / down / left / right directions of the displayable area A1 into the memory 110, and determines the display start position P2 by referring to the expanded coordinate system.

[0042] The predetermined distance L2 is preferably a distance such that the display start position P2 is located within the range that includes the central field of view centered on the line of sight P1. For example, the predetermined distance L2 is a distance such that the display start position P2 is located at a position where the vertical and horizontal fields of view are 5 degrees from the driver U's line of sight. Note that the position where the vertical and horizontal fields of view are 5 degrees from the driver U's line of sight is within the effective field of view centered on the line of sight P1. Figure 3 illustrates a configuration in which the display start position P2 is determined to be below and to the left of the line of sight P1, but the position of the display start position P2 may be, for example, below and to the right of the line of sight P1, above and to the right of the line of sight P1, or above and to the left of the line of sight P1.

[0043] The display control unit 105 determines the display start position P2 and then determines the movement end position P3 of the eye-tracking display VI. The display control unit 105 expands the coordinate system that defines the shape, size, and up / down / left / right directions of the displayable area A1 into the memory 110 and determines the movement end position P3 by referring to the expanded coordinate system.

[0044] The display control unit 105 determines the movement end position P3 in the vertical direction of the displayable area A1 (hereinafter referred to as "first movement end position P3-1" with the designation "P3-1"). The vertical direction of the displayable area A1 corresponds to the vertical direction of the windshield 3 and the short-side direction of the displayable area A1. Furthermore, in determining the movement end position P3, the display control unit 105 determines the movement end position P3 in the left-right direction of the displayable area A1 (hereinafter referred to as "second movement end position P3-2" with the designation "P3-2"). The left-right direction of the displayable area A1 corresponds to the left-right direction of the windshield 3 and the longitudinal direction of the displayable area A1.

[0045] First, let's explain how to determine the end position P3-1 of the first movement. The display control unit 105 obtains the position of the object 6 in the vertical direction of the captured image SG from the data output by the object detection unit 101. The vertical direction of the captured image SG corresponds to the vertical direction of the scene captured in the captured image SG. Next, the display control unit 105 converts the obtained position of the object 6 into a vertical position on the windshield 3. The relationship between the position of the object 6 in the vertical direction of the captured image SG and the vertical position on the windshield 3 is determined by prior simulations, etc., and is stored as data in the memory 110.

[0046] If the converted position is above the upper edge JT of the displayable area A1 on the windshield 3, the display control unit 105 determines the position of the upper edge JT of the displayable area A1 to be the first movement end position P3-1. Furthermore, if the converted position is below the lower end KT of the displayable area A1 on the windshield 3, the display control unit 105 determines the position of the lower end KT of the displayable area A1 to be the first movement end position P3-1. Furthermore, if the converted position is located between the upper edge JT and the lower edge KT of the displayable area A1 on the windshield 3, the display control unit 105 determines the converted position to be the first movement end position P3-1. Figure 3 illustrates the case where the converted position is located between the upper edge JT and the lower edge KT of the displayable area A1 on the windshield 3.

[0047] Next, we will explain how to determine the second movement end position P3-2. The display control unit 105 determines the second movement end position P3-2 based on the relative position of the detected object 6 and the position of the detected head HD.

[0048] Figure 4 is a diagram illustrating the determination of the second movement end position P3-2. In Figure 4, object 6 is exemplified as a pedestrian located in front of vehicle 1.

[0049] The display control unit 105 detects the position in the left-right direction of the vehicle 1 where the line connecting the position of the driver U's head HD and the position of the object 6 intersects with the windshield 3 in a top view of the vehicle 1. Based on the relative position of the object 6 detected by the object detection unit 101 and the position of the head HD detected by the head detection unit 103, the display control unit 105 detects the position in the left-right direction where the line connecting the position of the head HD and the position of the object 6 intersects with the windshield 3.

[0050] In Figure 4, line L3 is the line connecting the position of the head HD and the position of the object 6 in a top view of vehicle 1. In Figure 4, the display control unit 105 detects the position P4 in the left-right direction where line L3 intersects with the windshield 3 in a top view of vehicle 1.

[0051] Returning to the explanation of the second movement end position P3-2 with reference to Figure 3, when the display control unit 105 detects a position in the intersecting left-right direction, if the detected position is to the left of the left edge ST of the displayable area A1 on the windshield 3, it determines the position of the left edge ST of the displayable area A1 as the second movement end position P3-2. Furthermore, when the display control unit 105 detects a position in the intersecting left-right direction, if the detected position is to the right of the right edge UT of the displayable area A1 on the windshield 3, it determines the position of the right edge UT of the displayable area A1 to be the second movement end position P3-2. Furthermore, when the display control unit 105 detects a position in the intersecting left-right direction, if the detected position is between the left edge ST and the right edge UT of the displayable area A1 on the windshield 3, it determines the detected position to be the second movement end position P3-2. Figure 3 illustrates the case where the position in the intersecting left-right direction is between the rightmost UT and leftmost ST of the displayable area A1.

[0052] When the display control unit 105 determines the first movement end position P3-1 and the second movement end position P3-2, it determines the position defined by the first movement end position P3-1 and the second movement end position P3-2 as the movement end position P3.

[0053] As will be revealed later, the display control unit 105 moves the eye-tracking display VI from the determined display start position P2 to the determined movement end position P3. By determining the movement end position P3 as described above, the display control unit 105 can move the eye-tracking display VI to a position that overlaps with the object 6 as seen by the driver U, if the object 6 is in the displayable area A1 as seen by the driver U. Also, by determining the movement end position P3 as described above, the display control unit 105 can move the eye-tracking display VI to the dotted line L1, which is the boundary between the displayable area A1 and the non-displayable area A2, if the object 6 is not in the displayable area A1 as seen by the driver U.

[0054] Now, once the display control unit 105 has determined the display start position P2 and the movement end position P3, it determines, based on the data input from the risk level calculation unit 104, whether the risk level calculated by the risk level calculation unit 104 is greater than the first threshold, between the first threshold and a second threshold that is smaller than the first threshold, or less than the second threshold and not "0". The second threshold is a threshold value smaller than the first threshold. For example, the first threshold is "7" and the second threshold is "3".

[0055] [1-4-1. If it is greater than the first threshold] If the display control unit 105 determines that the risk level is greater than the first threshold, it determines the mode of change in the movement speed when moving the eye-tracking display VI from the display start position P2 to the movement end position P3 to be the first mode.

[0056] The first mode is one in which the movement decelerates as it approaches the object 6. In other words, the first mode is one in which the movement decelerates as it approaches the end position P3. The degree of deceleration may be linear or nonlinear.

[0057] When the display control unit 105 determines the mode of change in the movement speed of the eye guidance display VI to be the first mode, it moves the eye guidance display VI linearly from the display start position P2 to the movement end position P3 on the windshield 3 with the movement speed change of the first mode.

[0058] Figure 5 shows an example of the movement of the eye-tracking indicator VI. Figure 5 shows the case where the eye-tracking indicator VI is moved with the change in movement speed of the first embodiment. Also, Figure 5 illustrates the case where, as viewed from the driver U, the object 6 is in the displayable area A1. Therefore, in Figure 5, the end position P3 of the movement is not on the dotted line L1 which is the boundary between the displayable area A1 and the non-displayable area A2.

[0059] In Figure 5, each of the black circles represents a gaze guidance display VI. In Figure 5, the multiple black circles indicate the position of the gaze guidance display VI per unit time during movement from the display start position P2 to the movement end position P3. As shown in Figure 5, in the change of movement speed of the first embodiment, the movement speed of the gaze guidance display VI decelerates from the speed at the start of movement to the movement end position P3 during movement from the display start position P2 to the movement end position P3.

[0060] [1-4-2. Between the first and second thresholds] If the display control unit 105 determines that the risk level is between the first threshold and the second threshold, it determines the mode of change in the movement speed when moving the eye-tracking display VI from the display start position P2 to the movement end position P3 to be the second mode.

[0061] The second mode is one in which the change in movement speed is the same, that is, a constant velocity mode. In other words, the second mode is one in which the movement speed does not change until the end position P3. Note that the movement speed in the second mode is slower than the initial velocity in the movement in the first mode.

[0062] When the display control unit 105 determines the mode of change in the movement speed of the eye guidance display VI to be the second mode, it moves the eye guidance display VI linearly from the display start position P2 to the movement end position P3 on the windshield 3 with the change in movement speed of the second mode.

[0063] Figure 6 shows an example of the movement of the eye-tracking indicator VI. Figure 6 shows the case where the eye-tracking indicator VI is moved with the change in movement speed of the second embodiment. Also, Figure 6 illustrates the case where, from the perspective of the driver U, the object 6 is not in the displayable area A1. Therefore, in Figure 6, the end position P3 of the movement lies on the dotted line L1, which is the boundary between the displayable area A1 and the non-displayable area A2.

[0064] In Figure 6, each of the black circles represents a gaze guidance display VI. In Figure 6, the multiple black circles indicate the position of the gaze guidance display VI per unit time during movement from the display start position P2 to the movement end position P3. As shown in Figure 6, in the second embodiment of movement speed change, the movement speed of the gaze guidance display VI is constant during movement from the display start position P2 to the movement end position P3.

[0065] [1-4-3. If it is less than the second threshold] If the display control unit 105 determines that the risk level is less than the second threshold and not "0", it determines the mode of change in the movement speed when moving the eye-tracking display VI from the display start position P2 to the movement end position P3 to be the third mode.

[0066] The third mode is one in which the movement accelerates as it approaches object 6. In other words, the third mode is one in which the movement accelerates as it approaches the end position P3. The degree of acceleration may be linear or nonlinear.

[0067] When the display control unit 105 determines the mode of change in the movement speed of the eye guidance display VI to be the third mode, it moves the eye guidance display VI linearly from the display start position P2 to the movement end position P3 on the windshield 3 with the movement speed change of the third mode.

[0068] Figure 7 shows an example of the movement of the eye-tracking indicator VI. Figure 7 shows the case where the eye-tracking indicator VI is moved with the change in movement speed of the first embodiment. Also, Figure 7 illustrates the case where, as viewed from the driver U, the object 6 is not in the displayable area A1. Therefore, in Figure 7, the end position P3 of the movement is on the dotted line L1, which is the boundary between the displayable area A1 and the non-displayable area A2.

[0069] In Figure 7, each of the black circles represents a gaze guidance display VI. In Figure 7, the multiple black circles indicate the position of the gaze guidance display VI per unit time during movement from the display start position P2 to the movement end position P3. As shown in Figure 7, in the third embodiment of movement speed change, the movement speed of the gaze guidance display VI accelerates from the speed at the start of movement to the movement end position P3 during movement from the display start position P2 to the movement end position P3.

[0070] [1-4-4. When the risk level becomes "0"] If the risk level calculated by the risk level calculation unit 104 becomes "0" between the start of movement of the eye-tracking display VI and the end of movement position P3, the display control unit 105 returns the eye-tracking display VI to the display start position P2. More specifically, the display control unit 105 returns the eye-tracking display VI to the display start position P2 from the position at the moment the risk level becomes "0".

[0071] The display control unit 105 returns the eye-tracking display VI to the display start position P2 so that the movement speed corresponding to the distance between the display start position P2 and the eye-tracking display VI is the same as the movement speed from the display start position P2. For example, when moving from the display start position P2, if the distance between the display start position P2 and the eye-tracking display VI is "D1", then the movement speed of the eye-tracking display VI is "V1". Also, when moving from the display start position P2, if the distance between the display start position P2 and the eye-tracking display VI is "D2", then the movement speed of the eye-tracking display VI is "V2". Here, D1<D2であり、V1> Let's assume it's V2. In other words, this example shows the case where the eye-tracking display VI moves from the display start position P2 due to a change in the movement speed in the first embodiment. In this example, when returning the eye-tracking display VI to the display start position P2, the display control unit 105 displays the eye-tracking display VI such that the movement speed of the eye-tracking display VI becomes "V1" when the distance between the display start position P2 and the eye-tracking display VI is "D1". Also, the display control unit 105 displays the eye-tracking display VI such that the movement speed of the eye-tracking display VI becomes "V2" when the distance between the display start position P2 and the eye-tracking display VI is "D2".

[0072] Figure 8 is a diagram illustrating the return of the eye-tracking indicator VI. Figure 8 illustrates a case where the risk level becomes "0" when the eye-tracking display VI is moved from the display start position P2 to the movement end position P3 shown in Figure 5, due to a change in the movement speed in the first embodiment.

[0073] In Figure 8, each of the black circles represents a gaze guidance indicator VI. In Figure 8, the multiple black circles indicate the position of the gaze guidance indicator VI per unit time during movement from the end position P3 to the start position P2.

[0074] The upper part of Figure 8 shows the movement of the eye-tracking display VI from the display start position P2. The lower part of Figure 8 shows the movement of the eye-tracking display VI back to the display start position P2. When moving from the display start position P2 to the movement end position P3, if the distance from the display start position P2 is "D3", then the movement speed of the eye-tracking display VI is "V3". In this case, when returning to the display start position P2, if the distance from the display start position P2 is "D3", then the movement speed of the eye-tracking display VI will be "V3". Furthermore, when moving from the display start position P2 to the movement end position P3, assume that the movement speed of the eye-tracking display VI is "V4" when the distance from the display start position P2 is "D4". Note that D3<D4であり、V3> It is V4. In this case, when returning to the display start position P2, if the distance from the display start position P2 is "D4", the movement speed of the eye-tracking display VI will be "V4". Furthermore, when moving from the display start position P2 to the movement end position P3, assume that the movement speed of the eye-tracking display VI is "V5" when the distance from the display start position P2 is "D5". Note D4<D5であり、V4> It is V5. In this case, when returning to the display start position P2, if the distance from the display start position P2 is "D5", the movement speed of the eye-tracking display VI will be "V5".

[0075] [2. Operation of the eye-tracking device] Next, the operation of the gaze guidance device 7 according to this embodiment will be described. Figure 9 is a flowchart showing the operation of the eye-tracking device 7.

[0076] The object detection unit 101 detects an object 6 located in front of the vehicle 1 (step S1). Step S1 corresponds to "Step 1".

[0077] The risk level calculation unit 104 determines whether or not the object 6 was detected in step S1 (step S2). The determination in step S2 is made based on whether or not the risk level calculation unit 104 receives data from the object detection unit 101.

[0078] If the risk calculation unit 104 determines that object 6 is not detected (step S2: NO), the processor 100 returns to step S1 and performs the process of step S1 again.

[0079] On the other hand, if the risk level calculation unit 104 determines that the object 6 has been detected (step S2: YES), it calculates the risk level related to the object 6 detected by the object detection unit 101 (step S3).

[0080] Next, the display control unit 105 determines whether to set the mode of change in the movement speed of the eye-tracking display VI to the first mode, the second mode, or the third mode (step S4). Step S4 is performed based on the risk level calculated in step S3.

[0081] If the display control unit 105 determines that the mode of change in movement speed should be the first mode (step S4: first mode), it displays the eye-tracking display VI and starts moving the eye-tracking display VI with the change in movement speed of the first mode (step S5). Step S5 corresponds to "Step 2".

[0082] Returning to the explanation of step S4, if the display control unit 105 determines that the mode of change in movement speed should be the second mode (step S4: second mode), it displays the eye-tracking display VI and starts moving the eye-tracking display VI with the change in movement speed of the second mode (step S6). Step S6 corresponds to "Step 2".

[0083] Returning to the explanation of step S4, if the display control unit 105 determines that the mode of change in movement speed should be the third mode (step S4: third mode), it displays the eye-tracking display VI and starts moving the eye-tracking display VI with the change in movement speed of the third mode (step S7). Step S7 corresponds to "Step 2".

[0084] When one of steps S5, S6, or S7 is performed and the eye-tracking display VI is moved to the end position P3, the risk level calculation unit 104 calculates the risk level (step S8).

[0085] Next, the display control unit 105 determines whether the risk level calculated in step S8 is "0" or not (step S9).

[0086] If the risk level is determined to be non-zero (step S9: NO), the display control unit 105 determines whether or not the eye-tracking display VI has been moved to the end position P3 (step S10).

[0087] If the display control unit 105 determines that the eye-tracking display VI has not been moved to the end position P3 (step S10: NO), the processor 100 returns to step S8 and executes step S8 again.

[0088] On the other hand, if the display control unit 105 determines that the eye-tracking display VI has been moved to the end position P3 (step S10: YES), it hides the eye-tracking display VI (step S11).

[0089] Returning to the explanation of step S9, if the display control unit 105 determines that the risk level is "0" (step S9: YES), it returns the eye-tracking display VI to the display start position P3 (step S12).

[0090] Next, the display control unit 105 returns the eye-tracking display VI to the display start position P3, and then hides the eye-tracking display VI (step S11).

[0091] [3. Other Embodiments] The embodiments described above are merely examples and can be modified and applied as needed.

[0092] In the embodiment described above, a four-wheeled automobile vehicle 1 was used as an example of a "vehicle," but the number of wheels is not limited to four as long as the "vehicle" has a windshield 3.

[0093] In the embodiment described above, the risk level is calculated based on an 11-level evaluation score. In other embodiments, the number of levels of the calculated risk level is not limited to 11 levels; it may be fewer than 11 levels or more than 11 levels.

[0094] In the embodiment described above, "0" was given as an example of a predetermined value for the risk level that can be considered to have no possibility of contact with the object 6. However, any value smaller than the second threshold could be, for example, "1" or "2".

[0095] In the embodiment described above, the eye-tracking display VI is displayed on the windshield 3 by the HUD 5. However, the means for displaying the eye-tracking display VI can be any means for displaying a virtual image on the windshield 3, and is not limited to the HUD 5. For example, it could be a display means such as an LED (Light Emitting Diode).

[0096] The processor 100 may consist of multiple processors or a single processor. The processor 100 may also be hardware programmed to implement the functions described above. In this case, the processor 100 may consist of, for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0097] Furthermore, the configuration of each part of Vehicle 1 shown in Figure 2 is merely an example, and the specific implementation is not particularly limited. In other words, it is not necessarily required that hardware corresponding to each part be implemented individually; it is certainly possible to configure a system where a single processor executes a program to realize the functions of each part. Also, in the embodiments described above, some of the functions realized by software may be implemented as hardware, or conversely, some of the functions realized by hardware may be implemented as software.

[0098] Furthermore, the operation steps shown in Figure 9 are divided according to the main processing content, and the present invention is not limited by the way the processing units are divided or by their names. Depending on the processing content, it may be further divided into more steps. Alternatively, it may be divided so that one step unit includes even more processing. Also, the order of the steps may be rearranged as appropriate, as long as it does not impede the spirit of the present invention.

[0099] Furthermore, when the eye-tracking method using the eye-tracking device 7 described above is implemented using the processor 100, the program to be executed by the processor 100 can be configured as a recording medium or a transmission medium for transmitting this program. In other words, the control program 111 can also be implemented by recording it on a portable information recording medium. Examples of information recording media include magnetic recording media such as hard disks, optical recording media such as CDs, and semiconductor storage devices such as USB (Universal Serial Bus) memory and SSDs (Solid State Drives), but other recording media can also be used.

[0100] [4. Configurations supported by the above embodiments] The above embodiment supports the following configuration.

[0101] (Composition 1) A gaze guidance device for guiding the gaze of a vehicle driver, comprising: a first detection unit for detecting an object located in front of the vehicle; and a display control unit for displaying a gaze guidance display on the vehicle's windshield when the first detection unit detects the object, the display control unit for displaying the gaze guidance display such that it moves from the display start position toward the object, based on a change in movement speed determined by a risk level, which is the degree to which the vehicle is likely to come into contact with the object. With the eye-tracking device of Configuration 1, the speed at which the eye-tracking display moves can be varied according to the degree of risk, allowing the driver to understand the likelihood of the vehicle contacting an object from the change in the speed of the eye-tracking display. Therefore, the driver can understand the situation of their vehicle regarding contact with an object. Furthermore, because the driver can understand the situation of their vehicle regarding contact with an object at the moment the eye-tracking display is moving, the driver can quickly understand the situation of their vehicle regarding contact with an object.

[0102] (Configuration 2) The display control unit moves the eye-tracking display to a position that overlaps with the object if the object is in the first region where the eye-tracking display can be displayed, and moves the eye-tracking display to the boundary between the first region and the second region where the eye-tracking display cannot be displayed if the object is not in the first region. According to the eye-tracking device of Configuration 2, if there is an object within the area where the eye-tracking display can be displayed, the eye-tracking display is moved to a position that overlaps with the object, thereby increasing the likelihood of guiding the driver's gaze to the object. Humans have the characteristic that even if the movement of the eye-tracking display stops midway, their gaze will continue to move towards the point of stopping in the direction of the eye-tracking display's movement. Therefore, according to the eye-tracking device of Configuration 2, if there is no object within the area where the eye-tracking display can be displayed, the eye-tracking display is moved to the edge of the area in the direction toward the object, thereby increasing the likelihood of guiding the driver's gaze to the object even when there is no object within the area where the eye-tracking display can be displayed. Thus, according to the eye-tracking device of Configuration 2, the likelihood of guiding the driver's gaze to the object can be increased.

[0103] (Composition 3) The eye-tracking device according to configuration 1 or 2, wherein the display control unit determines, based on the risk level, the mode of change in the movement speed of the eye-tracking display to be one of the following: a first mode in which the movement speed is reduced as it approaches the object, a second mode in which the movement speed does not change, or a third mode in which the movement speed is accelerated as it approaches the object. According to the eye-tracking device of configuration 3, for example, the driver can understand the likelihood of the vehicle contacting an object in three stages: high, medium, and low. Therefore, the driver does not need to keep track of the degree of risk associated with various changes in the speed of the eye-tracking display. Thus, the driver can easily and quickly understand the situation of their vehicle regarding contact with an object.

[0104] (Composition 4) The gaze guidance device according to configuration 3, wherein the display control unit determines the first mode when the risk level is greater than a first threshold, determines the second mode when the risk level is between the first threshold and a second threshold less than the first threshold, and determines the third mode when the risk level is less than the second threshold. Humans have a tendency to focus their gaze on objects that are moving significantly. Therefore, with the eye-tracking device of configuration 4, if the risk level is greater than the first threshold, the driver can quickly notice the eye-tracking indicator and quickly guide their gaze. Furthermore, with the eye-tracking device of configuration 4, if the risk level is less than the second threshold, the eye-tracking indicator moves significantly as the driver approaches the object, thus increasing the certainty of guiding the driver's gaze to the object.

[0105] (Composition 5) The eye-tracking device according to any one of configurations 1 to 4, wherein the display control unit returns the eye-tracking display to the display start position after the movement of the eye-tracking display, if the risk level falls below a predetermined value that can be considered to be no possibility of contact with the object. According to the eye-tracking device of configuration 5, the driver can easily understand that the possibility of the vehicle contacting the object has been eliminated when the moving eye-tracking display returns to its starting position.

[0106] (Composition 6) The eye-tracking device according to configuration 5, wherein the display control unit returns the eye-tracking display to the display start position such that the movement speed corresponding to the distance between the display start position and the eye-tracking display is the same as the movement speed from the display start position. According to the eye-tracking device of configuration 6, by returning the eye-tracking display with a change in movement speed that is the same as the movement from the display start position, the driver can more easily understand that there is no longer a possibility of contact with the object that the eye-tracking display was guiding the driver to.

[0107] (Composition 7) A gaze guidance device according to any one of configurations 1 to 6, comprising a second detection unit for detecting the position of the driver's line of sight on the windshield, wherein the display start position is a position shifted by a predetermined distance from the position of the driver's line of sight detected by the second detection unit. Humans have a tendency to notice displays that appear outside their line of sight. Therefore, with the eye-tracking device of configuration 7, the driver can quickly notice the eye-tracking display and quickly guide the driver's gaze.

[0108] (Composition 8) A method for guiding the gaze of a vehicle driver, comprising: a first step of detecting an object located in front of the vehicle; and a second step of, if the object is detected in the first step, displaying a gaze guidance indicator on the windshield of the vehicle to guide the driver's gaze toward the object, wherein the second step involves displaying the gaze guidance indicator so that it moves from a starting position toward the object by a change in movement speed based on a risk level, which is the degree to which the vehicle is likely to come into contact with the object. The gaze guidance method of configuration 8 produces the same effect as the gaze guidance device of configuration 1.

[0109] (Composition 9) A processor for a gaze guidance device that guides the gaze of a vehicle driver comprises a first detection unit that detects an object located in front of the vehicle, and a display control unit that, when the first detection unit detects the object, displays a gaze guidance display on the vehicle's windshield to guide the driver's gaze toward the object, and the display control unit is programmed to display the gaze guidance display so that it moves from the display start position toward the object, based on a change in movement speed that is determined by the risk level, which is the degree to which the vehicle is likely to come into contact with the object. According to the program of Configuration 9, it produces the same effect as the eye-tracking device of Configuration 1. [Explanation of Symbols]

[0110] 1...Vehicle, 2...Steering wheel, 3...Windshield, 4...Instrument panel, 5...HUD, 6...Object, 7...Eye-tracking device, 8...Front camera, 9...Driver monitoring camera, 10...Position detection device, 11...Vehicle speed sensor, 100...Processor, 101...Object detection unit (first detection unit), 102...Eye-tracking detection unit (second detection unit), 103...Head detection unit, 104...Risk level calculation unit, 110...Memory, 111...Control program Ram (program), HD...head, JT...top, KT...bottom, L1...dotted line, L3...line, P1...gaze position, P2...display start position, P3...movement end position, P3-1...first movement end position, P3-2...second movement end position, P4...position, S1...step (first step), S2~S4...step, S5~S7...step (second step), S8~S12...step, SG...captured image, ST...left edge, U...driver, UT...right edge, VI...gaze guidance display.

Claims

1. A gaze guidance device that guides the driver's gaze, A first detection unit for detecting an object located in front of the vehicle, The system includes a display control unit that, when the first detection unit detects the object, displays a gaze guidance display on the vehicle's windshield to guide the driver's gaze toward the object, The display control unit, The visual guidance display is displayed such that it moves from the starting position of the visual guidance display toward the object, based on a change in the movement speed that corresponds to the degree of risk, which is the degree to which the vehicle is likely to come into contact with the object. Eye guidance device.

2. The display control unit, If the object is located in the first region where the eye-tracking display can be displayed, move the eye-tracking display to a position that overlaps with the object. If there is no object in the first region, the eye-tracking display is moved to the boundary between the first region and the second region where the eye-tracking display cannot be displayed. The gaze guidance device according to claim 1.

3. The display control unit, Based on the aforementioned risk level, the mode of change in the movement speed of the eye-tracking display is determined to be one of the following: a first mode in which it decelerates as it approaches the object, a second mode in which the movement speed does not change, or a third mode in which it accelerates as it approaches the object. The gaze guidance device according to claim 1 or 2.

4. The display control unit, If the risk level is greater than the first threshold, the first embodiment is determined. If the risk level is between the first threshold and a second threshold smaller than the first threshold, the second embodiment is determined. If the risk level is less than the second threshold, the third embodiment is determined. The gaze guidance device according to claim 3.

5. The display control unit, If, during the period from the start to the end of the movement of the eye-tracking indicator, the risk level falls below a predetermined value at which it can be considered that there is no possibility of contact with the object, the eye-tracking indicator is returned to the starting position. The gaze guidance device according to claim 1 or 2.

6. The display control unit, The eye-tracking display is returned to the display start position such that the movement speed corresponding to the distance between the display start position and the eye-tracking display is the same as the movement speed from the display start position. The gaze guidance device according to claim 5.

7. The system includes a second detection unit that detects the position of the driver's line of sight on the windshield, The display start position is a position shifted by a predetermined distance from the driver's line of sight position detected by the second detection unit. The gaze guidance device according to claim 1 or 2.

8. A method for guiding the gaze of a vehicle driver, The first step is to detect an object located in front of the vehicle, If the object is detected in the first step, the second step includes displaying a visual guidance indicator on the vehicle's windshield to guide the driver's gaze to the object, The second step described above is: The visual guidance display is displayed such that it moves from the starting position of the visual guidance display toward the object, based on a change in the movement speed that corresponds to the degree of risk, which is the degree to which the vehicle is likely to come into contact with the object. Eye guidance method.

9. The processor for the eye-tracking device that guides the driver's gaze is A first detection unit for detecting an object located in front of the vehicle, When the first detection unit detects the object, it functions as a display control unit that displays a gaze guidance display on the vehicle's windshield to guide the driver's gaze towards the object. The display control unit, The visual guidance display is displayed such that it moves from the starting position of the visual guidance display toward the object, based on a change in the movement speed that corresponds to the degree of risk, which is the degree to which the vehicle is likely to come into contact with the object. program.

Images