Eye-tracking device, eye-tracking method, and program

Abstract

The goal is to make it easy for the driver (U) to understand which object (6) they want to direct their gaze towards. [Solution] The eye-tracking device (7) is an eye-tracking device (7) that guides the gaze of the driver (U) of a vehicle (1), and comprises a detection unit (101) that detects an object (6) located in front of the vehicle (1), and a display control unit (104) that, when the detection unit (101) detects the object (6), displays an eye-tracking display (VI) on the windshield (3) of the vehicle (1) to guide the gaze of the driver (U) to the object (6), wherein the display control unit (104) displays the eye-tracking display (VI) so as to move from the display start position (P2) toward the object (6) with a change in movement speed according to the type of object (6).

Description

Technical Field

[0004] , ,

[0005] , , , , , ,

[0001] The present invention relates to a line-of-sight guidance device, a line-of-sight guidance method, and a program.

Background Art

[0002] In recent years, in order to further improve traffic safety and contribute to the development of a sustainable transportation system, research and development on the visibility of vehicle drivers has been carried out. Conventionally, as a technology related to the visibility of vehicle drivers, a technology for guiding the line of sight of vehicle drivers is known. For example, Patent Document 1 discloses a line-of-sight guidance device that guides the driver's line of sight in the direction of a target by moving a visual stimulus displayed on the windshield. Further, Patent Document 1 discloses that the moving speed of the visual stimulus is increased as the angle formed by the target and the line of sight increases.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

[0006] According to one aspect of the present invention, the driver can easily understand which object they want to direct their gaze towards. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 shows the layout of the vehicle's interior. [Figure 2] Figure 2 shows the configuration of the vehicle. [Figure 3] Figure 3 is a diagram illustrating the processing of the display control unit. [Figure 4] Figure 4 is a diagram illustrating the determination of the second movement end position. [Figure 5] Figure 5 is a diagram illustrating the change in travel speed in the first embodiment. [Figure 6] Figure 6 shows an example of the movement of eye-tracking indicators. [Figure 7] Figure 7 is a diagram illustrating the change in travel speed in the second embodiment. [Figure 8] Figure 8 shows an example of the movement of eye-tracking indicators. [Figure 9] Figure 9 is a diagram illustrating the change in travel speed in the third embodiment. [Figure 10] Figure 10 shows an example of the movement of eye-tracking indicators. [Figure 11] Figure 11 shows an example of the movement of eye-tracking indicators. [Figure 12]Figure 12 is a flowchart showing the operation of the eye-tracking device. [Modes for carrying out the invention]

[0008] [1. Vehicle Configuration] The embodiments will be described below with reference to the drawings. Figures 1, 3, 4, 6, 8, 10, and 11 illustrate the X, Y, and Z axes. The X, Y, and Z axes are orthogonal to each other. The Z axis indicates the vertical direction. The X and Y axes are parallel to the horizontal direction when vehicle 1 is in motion. The X axis indicates the left-right direction as the vehicle width direction. The Y axis indicates the front-rear direction. The positive direction of the X axis is to the right. The positive direction of the Y axis is forward. The positive direction of the Z axis is upward.

[0009] Figure 1 shows the interior layout of vehicle 1. In this embodiment, Vehicle 1 is exemplified by a four-wheeled automobile.

[0010] The interior of vehicle 1 is equipped with a steering wheel 2 for operating vehicle 1, a windshield 3 that separates the interior from the exterior, and an instrument panel 4. The steering wheel 2 is positioned on the instrument panel 4 opposite the driver U seated in the driver's seat.

[0011] The instrument panel 4 is equipped with a Head-Up Display (HUD) 5. The HUD 5 projects light onto the windshield 3 to display a gaze guidance display VI that guides the driver U's (see Figure 4) gaze to an object 6 (see, for example, Figure 4). The gaze guidance display VI is a virtual image. The object 6 represents an object that is desirable for the driver U to focus on. By displaying the gaze guidance display VI on the windshield 3 with the HUD 5, the driver U, seated in the driver's seat, can see the gaze guidance display VI along with the scenery in front of the vehicle 1 through the windshield 3.

[0012] In FIG. 1, a circular shape is illustrated as the shape of the line-of-sight guidance display VI. However, the shape of the line-of-sight guidance display VI shown in FIG. 1 is merely an example, and it may be, for example, rectangular or star-shaped.

[0013] In FIG. 1, a displayable area A1 where the line-of-sight guidance display VI can be displayed is illustrated by a dotted line. The size of the displayable area A1 corresponds to the size of the magnifying mirror (concave mirror) provided in the HUD 5. Note that the size of the displayable area A1 with respect to the windshield 3 is not limited to the size shown in FIG. 1. Also, in FIG. 1, a shape where the longitudinal direction of the displayable area A1 is in the vehicle width direction is illustrated, but the shape of the line-of-sight guidance display VI is not limited to the shape shown in FIG. 1.

[0014] FIG. 2 is a diagram showing the configuration of the vehicle 1. The vehicle 1 includes a line-of-sight guidance device 7. The line-of-sight guidance device 7 includes a processor 100 such as a CPU (Central Processing Unit) or MPU (Micro-processing unit), and a memory 110.

[0015] The processor 100 controls each part of the line-of-sight guidance device 7 by reading and executing a control program 111 stored in the memory 110. The processor 100 functions as an object detection unit 101, a line-of-sight detection unit 102, a head detection unit 103, and a display control unit 104 by executing the control program 111 stored in the memory 110. The object detection unit 101 is an example of a "detection unit".

[0016] The memory 110 is a storage device that stores programs executed by the processor 100 and data processed by the processor 100. The memory 110 stores a control program 111 executed by the processor 100 and various other data. The memory 110 has a non-volatile storage area. Also, the memory 110 includes a volatile storage area that constitutes the work area of the processor 100. The memory 110 is constituted by, for example, a ROM (Read Only Memory) and a RAM (Random Access Memory). Control program 111 corresponds to "program".

[0017] The eye-tracking device 7 is connected to a HUD 5, a front camera 8, a driver monitoring camera 9, and a position detection device 10. However, the devices connected to the eye-tracking device 7 are not limited to these; other devices such as a vehicle-to-vehicle communication device, a GNSS (Global Navigation Satellite System) unit, a rear camera, and a vehicle speed sensor may also be connected.

[0018] The front camera 8 is installed at a predetermined position on the vehicle 1 and is a camera that photographs the area in front of the vehicle 1. The front camera 8 takes pictures at predetermined intervals when the ignition of the vehicle 1 is on or when the accessory power of the vehicle 1 is on. Each time the front camera 8 takes a picture, it outputs the image data of the captured image SG (see Figure 3) obtained from the picture to the eye-tracking device 7.

[0019] The driver monitoring camera 9 is installed in a predetermined position inside the vehicle 1 and is a camera that photographs the driver U seated in the driver's seat. The shooting range of the driver monitoring camera 9 includes at least the head HD (see Figure 4) of the driver U seated in the driver's seat. The driver monitoring camera 9 takes pictures at predetermined intervals when the ignition of the vehicle 1 is on or when the accessory power of the vehicle 1 is on. Each time the driver monitoring camera 9 takes a picture, it outputs the image data of the captured image to the eye guidance device 7.

[0020] The position detection device 10 is a device capable of detecting the position of objects present around the vehicle 1. The position detection device 10 consists of at least one of the following: a sonar, radar, lidar, etc., capable of measuring the distance between the vehicle 1 and an object, and a stereo camera capable of measuring the distance between the vehicle 1 and an object using parallax.

[0021] As described above, the processor 100 of the gaze guidance device 7 functions as an object detection unit 101, a gaze detection unit 102, a head detection unit 103, and a display control unit 104.

[0022] [1-1. Object Detection Unit] The object detection unit 101 detects an object 6 located in front of the vehicle 1. Based on the image data of the captured image SG received from the front camera 8, the object detection unit 101 detects the object 6 captured in the captured image SG obtained by the front camera 8. As described above, the object 6 refers to an object that should be directed at the driver U. In this embodiment, the object 6 includes pedestrians, passenger cars, motorcycles, bicycles, trailers, and fixed objects. Pedestrians include children. Examples of fixed objects include road signs and road traffic guidance markers. The object detection unit 101 detects the object 6 captured in the captured image SG by performing pattern matching and color-based image processing on the captured image SG. The data necessary for detecting the object 6 (for example, shape data and color data) is stored in the memory 110 for each type of object: pedestrians, passenger cars, motorcycles, bicycles, trailers, and fixed objects.

[0023] Furthermore, the object detection unit 101 detects the position of the detected object 6. More specifically, the object detection unit 101 detects the relative position of the object 6 with respect to the vehicle 1 when the vehicle 1 is viewed from above. The object detection unit 101 detects the relative position of the detected object 6 based on at least one of the detection results of the position detection device 10 and the captured image SG of the front camera 8. If the detected object 6 is another vehicle and the vehicle-to-vehicle communication device and GNSS unit are connected to the eye-tracking device 7, the object detection unit 101 may detect the relative position of the detected object 6 based on the position of the other vehicle received by the vehicle-to-vehicle communication device and the position of vehicle 1 received by the GNSS unit.

[0024] In addition to the front camera 8, vehicle-to-vehicle communication device, and GNSS unit, the object detection unit 101 may also use V2X (vehicle-to-infrastructure or pedestrian-to-pedestrian communication, etc.) or determination in a virtual environment via a server to detect objects 6 in the captured image SG and to detect the relative position of objects 6.

[0025] When the object detection unit 101 detects an object 6, it outputs data indicating the type of object 6 detected, data indicating the relative position of the detected object 6 with respect to the vehicle 1, and data indicating the position of the detected object 6 in the captured image to the display control unit 104.

[0026] [1-2. Eye-tracking unit] The gaze detection unit 102 detects the direction of the driver U's gaze. The gaze detection unit 102 detects the direction of the driver U's gaze based on the image data of the captured image received from the driver monitoring camera 9. The gaze detection unit 102 detects the driver U's eyes from the captured image obtained from the driver monitoring camera 9 using pattern matching, color, etc., and detects the direction the detected eyes are facing as the direction of the gaze. The data necessary for eye detection (data on eye shape and color) is stored in the memory 110.

[0027] When the gaze detection unit 102 detects the direction of the driver U's gaze, it outputs data indicating the detected direction of the gaze to the display control unit 104.

[0028] [1-3. Head detection unit] The head detection unit 103 detects the head HD of driver U seated in the driver's seat. The head detection unit 103 detects the head HD of driver U based on image data of the captured image received from the driver monitoring camera 9. The head detection unit 103 detects the head HD from the captured image obtained by the driver monitoring camera 9 using pattern matching, color, etc. Next, the head detection unit 103 detects the position of the head HD in the captured image. Then, based on the size of the head HD in the captured image and the position of the head HD in the captured image, the head detection unit 103 detects the position of the head HD in the vehicle 1 when viewed from above. The size of the head HD in the captured image and the position of the head HD in the captured image, and the position of the head HD in the vehicle 1 are determined by prior tests and simulations and stored as data in the memory 110.

[0029] [1-4. Display Control Unit] The display control unit 104 controls the operation of the HUD5 to display the eye-tracking display VI on the windshield 3. The display control unit 104 displays the eye-tracking display VI on the windshield 3 and moves the eye-tracking display VI displayed on the windshield 3 by performing the following processes.

[0030] The processing of the display control unit 104 will be explained with reference to Figure 3. Figure 3 is a diagram illustrating the processing of the display control unit 104.

[0031] The display control unit 104 detects the position of the driver U's gaze on the windshield 3 (hereinafter referred to as "gaze position P1" with the designation "P1") based on the direction of the gaze indicated by the data received from the gaze detection unit 102. For example, if the memory 110 stores data that associates the direction of the driver U's gaze with the driver U's gaze position P1 on the windshield 3, the display control unit 104 refers to this data to detect the driver U's gaze position P1 on the windshield 3.

[0032] The display control unit 104 determines whether the detected driver U's gaze position P1 is within the displayable area A1. The memory 110 stores data indicating the position of the displayable area A1 on the windshield 3. The display control unit 104 determines whether the detected driver U's gaze position P1 is within the displayable area A1 by referring to this data stored in the memory 110.

[0033] If the display control unit 104 determines that the detected driver U's gaze position P1 is within the displayable area A1, it determines a position shifted by a predetermined distance L1 from the driver U's gaze position P1 as the display start position P2 of the gaze guidance display VI, as shown in Figure 3. The display control unit 104 then expands the coordinate system that defines the shape, size, and up / down / left / right directions of the displayable area A1 into the memory 110, and determines the display start position P2 by referring to the expanded coordinate system.

[0034] The predetermined distance L1 is preferably a distance such that the display start position P2 is located within the range that includes the central field of view centered on the line of sight P1. For example, the predetermined distance L1 is a distance such that the display start position P2 is located at a position where the vertical and horizontal fields of view are 5 degrees from the driver U's line of sight. Note that the position where the vertical and horizontal fields of view are 5 degrees from the driver U's line of sight is within the effective field of view centered on the line of sight P1. Figure 3 illustrates a configuration in which the display start position P2 is determined to be to the lower left of the line of sight P1, but the position of the display start position P2 may be, for example, to the lower right, upper right, or upper left of the line of sight P1.

[0035] The display control unit 104 determines the display start position P2 and then determines the movement end position P3 of the eye-tracking display VI. The display control unit 104 expands the coordinate system that defines the shape, size, and up / down / left / right directions of the displayable area A1 into the memory 110 and determines the movement end position P3 by referring to the expanded coordinate system.

[0036] The display control unit 104 determines the movement end position P3 in the vertical direction of the displayable area A1 (hereinafter referred to as "first movement end position P3-1" with the designation "P3-1"). The vertical direction of the displayable area A1 corresponds to the vertical direction of the windshield 3 and the short-side direction of the displayable area A1. Furthermore, in determining the movement end position P3, the display control unit 104 determines the movement end position P3 in the left-right direction of the displayable area A1 (hereinafter referred to as "second movement end position P3-2" with the designation "P3-2"). The left-right direction of the displayable area A1 corresponds to the left-right direction of the windshield 3 and the longitudinal direction of the displayable area A1.

[0037] First, let's explain how to determine the end position P3-1 of the first movement. The display control unit 104 obtains the position of the object 6 in the vertical direction of the captured image SG from the data output by the object detection unit 101. The vertical direction of the captured image SG corresponds to the vertical direction of the scene captured in the captured image SG. Next, the display control unit 104 converts the obtained position of the object 6 into a position in the vertical direction of the displayable area A1, and determines the converted position as the first movement end position P3-1. The relationship between the position of the object 6 in the vertical direction of the captured image SG and the position in the vertical direction of the displayable area A1 is determined by prior simulations, etc., and is stored as data in the memory 110.

[0038] Next, we will explain how to determine the second movement end position P3-2. The display control unit 104 determines the second movement end position P3-2 based on the relative position of the detected object 6 and the position of the detected head HD.

[0039] Figure 4 is a diagram illustrating the determination of the second movement end position P3-2. In Figure 4, object 6 is exemplified as a pedestrian located in front of vehicle 1.

[0040] The display control unit 104 detects the position in the left-right direction of the vehicle 1 where the line connecting the position of the driver U's head HD and the position of the object 6 intersects with the windshield 3 in a top view of the vehicle 1. Based on the relative position of the object 6 detected by the object detection unit 101 and the position of the driver U's head HD detected by the head detection unit 103, the display control unit 104 detects the position in the left-right direction where the line connecting the position of the driver U's head HD and the position of the object 6 intersects with the windshield 3.

[0041] In Figure 4, line L2 is the line connecting the position of the driver U's head HD and the position of the pedestrian in a top view of vehicle 1. In Figure 4, the display control unit 104 detects the position P4 in the left-right direction where line L2 intersects with the windshield 3 in a top view of vehicle 1.

[0042] Next, when the display control unit 104 detects a position in the intersecting left-right direction, if the detected position is within the displayable area A1, it determines the detected position as the second movement end position P3-2.

[0043] Returning to the explanation of the display control unit 104 with reference to Figure 3, once the display control unit 104 has determined the first movement end position P3-1 and the second movement end position P3-2, it determines the position defined by the first movement end position P3-1 and the second movement end position P3-2 as the movement end position P3.

[0044] The display control unit 104 determines the display start position P2 and the movement end position P3, and then calculates the distance between the display start position P2 and the movement end position P3 on the windshield 3. The display control unit 104 finds the distance between the display start position P2 and the movement end position P3 in a straight line in the coordinate system expanded in the memory 110, and converts the calculated distance into a distance on the windshield 3 to calculate the distance between the display start position P2 and the movement end position P3 on the windshield 3.

[0045] The display control unit 104 determines the distance between the display start position P2 and the movement end position P3 on the windshield 3, and then determines whether the determined distance is greater than or equal to a predetermined distance. Examples of predetermined distances include 10 cm (centimeters) and 20 cm.

[0046] [1-4-1. Distance exceeding the specified distance] If the display control unit 104 determines that the distance between the display start position P2 and the movement end position P3 on the windshield 3 is greater than or equal to a predetermined distance, it determines the manner in which the movement speed changes when moving the eye guidance display VI from the display start position P2 to the movement end position P3, according to the type of object 6 detected.

[0047] If the detected object 6 is a pedestrian, the display control unit 104 determines the mode of change in the movement speed of the eye-tracking display VI to the first mode. The first aspect is 1 / f fluctuation.

[0048] The change in movement speed in the first embodiment is shown, for example, in Figure 5. Figure 5 is a diagram illustrating the change in travel speed in the first embodiment. In Figure 5, the vertical axis represents the distance on the windshield 3, indicating the distance along a straight line passing from the display start position P2 to the movement end position P3. The horizontal axis in Figure 5 represents the movement speed of the eye-tracking display VI on the windshield 3.

[0049] Figure 5 shows graph GF1, which illustrates an example of the change in movement speed with respect to the distance traveled from the display start position P3. As shown in graph GF1, the change in movement speed in the first embodiment is a change in which acceleration and deceleration are randomly repeated during movement from the display start position P2 to the movement end position P3.

[0050] When the display control unit 104 determines the mode of change in the movement speed of the eye-tracking display VI to be the first mode, it moves the eye-tracking display VI linearly from the display start position P2 to the movement end position P3 with the movement speed change of the first mode.

[0051] Figure 6 shows an example of the movement of the eye-tracking indicator VI. Figure 6 shows the case where the eye-tracking indicator VI is moved with the change in movement speed described in the first embodiment.

[0052] In Figure 6, each black circle represents a gaze guidance display VI. In Figure 6, multiple black circles indicate the position of the gaze guidance display VI per unit time during movement from the display start position P2 to the movement end position P3. As shown in Figure 6, in the movement speed change of the first embodiment, the gaze guidance display VI randomly accelerates and decelerates multiple times during movement from the display start position P2 to the movement end position P3.

[0053] Pedestrians are more likely to make unpredictable changes in direction compared to vehicles and stationary objects. Therefore, pedestrians are objects that drivers U should pay more attention to than vehicles and stationary objects. Therefore, as described above, if the detected object 6 is a pedestrian, the display control unit 104 moves the gaze guidance display VI by changing the movement speed in the first mode.

[0054] 1 / f fluctuation is said to be one type of biological movement. Humans are said to be more sensitive to biological movements, and the greater the degree of randomness in the movement, the easier it is to notice. 1 / f fluctuation is a biological movement, and it is more random than the second type of movement described later. Therefore, if the detected object 6 is a pedestrian, the eye-tracking device 7 moves the eye-tracking display VI by changing the movement speed in the first mode. As a result, the driver U can easily understand that the eye-tracking display VI is intended to guide the driver's gaze to the pedestrian. Furthermore, by changing the movement speed of the eye-tracking display VI in the first mode, the driver U can quickly notice the eye-tracking display VI, and thus quickly understand that the eye-tracking display VI is intended to guide the driver's gaze to the pedestrian.

[0055] If the detected object 6 is a passenger car, motorcycle, or bicycle (hereinafter referred to as "passenger car, etc." as appropriate), the display control unit 104 determines the mode of change in the movement speed of the eye guidance display VI to the second mode. The second embodiment is the embodiment with minimum jerk. More specifically, the second embodiment is the embodiment in which the change in acceleration is small at three timings: when movement starts, when switching from acceleration to deceleration, and when movement ends. More specifically, the second embodiment is the embodiment in which the change in acceleration at the above three timings is smaller than that of the third embodiment described later. In this embodiment, the meaning of minimum jerk does not include zero change in acceleration.

[0056] The change in movement speed in the second embodiment is shown, for example, in Figure 7. Figure 7 is a diagram illustrating the change in travel speed in the second embodiment. In Figure 7, the vertical and horizontal axes are the same as those in Figure 5.

[0057] Figure 7 shows graph GF2, which illustrates an example of the change in movement speed with respect to the distance traveled from the display start position P2. As shown in graph GF2, the change in movement speed in the second mode is a change that involves one acceleration or deceleration from the start to the end of movement, and the change in acceleration when starting the movement, when switching from acceleration to deceleration, and when ending the movement is smaller than in the third mode. In other words, the change in movement speed in the second mode is a change that involves one acceleration or deceleration from the start to the end of movement, and the change in movement speed when starting the movement, when switching from acceleration to deceleration, and when ending the movement is smoother than in the third mode.

[0058] When the display control unit 104 determines the mode of change in the movement speed of the eye-tracking display VI to be the second mode, it moves the eye-tracking display VI in a straight line from the display start position P2 to the movement end position P3 according to the change in movement speed of the second mode.

[0059] Figure 8 shows an example of the movement of the eye-tracking indicator VI. Figure 8 shows the case where the eye-tracking indicator VI is moved with the change in movement speed described in the second embodiment.

[0060] In Figure 8, each of the black circles represents a gaze guidance display VI. In Figure 8, the multiple black circles indicate the position of the gaze guidance display VI per unit time during movement from the display start position P2 to the movement end position P3. As shown in Figure 8, in the second mode of movement speed change, the gaze guidance display VI gradually accelerates to start moving from the display start position P2 to the movement end position P3, and gradually decelerates to end moving.

[0061] While passenger cars and similar vehicles may change direction from side to side, they are less likely to make sudden changes in the forward and backward directions compared to pedestrians. Therefore, compared to pedestrians, passenger cars and similar vehicles are objects that drivers U need to pay less attention to. On the other hand, compared to trailers and stationary objects, which are less likely to make unpredictable changes in direction, passenger cars and similar vehicles are objects that drivers U need to pay more attention to. Therefore, as described above, if the detected object 6 is a passenger car or the like, the display control unit 104 moves the eye-tracking display VI by changing the movement speed in the second mode.

[0062] Velocity changes with minimum jerk are said to be one type of biological movement, and are less random than the first type. Therefore, if the detected object 6 is a passenger car or the like, the eye-tracking device 7 moves the eye-tracking display VI by changing the movement speed in a second mode, which is different from the first mode. As a result, the driver U can easily understand that the eye-tracking display VI wants to guide the driver's gaze to the object 6, which is a passenger car or the like. In addition, by changing the movement speed of the eye-tracking display VI to the second mode, the driver U can quickly notice the eye-tracking display VI, and thus quickly understand that the eye-tracking display VI wants to guide the driver's gaze to a pedestrian.

[0063] If the detected object 6 is a trailer or a fixed object, the display control unit 104 determines the mode of change in the movement speed of the eye-tracking display VI to be the third mode. The third embodiment is one in which the vehicle accelerates linearly and then decelerates linearly.

[0064] The change in movement speed in the third embodiment is shown, for example, in Figure 9. Figure 9 is a diagram illustrating the change in travel speed in the second embodiment. In Figure 9, the vertical and horizontal axes are the same as those shown in Figure 5.

[0065] Figure 9 shows graph GF3, which illustrates an example of the change in movement speed with respect to the distance traveled from the display start position P2. As shown in graph GF3, the change in movement speed in the third mode is a change in which acceleration and deceleration occur linearly once each from the start to the end of movement, and the change in acceleration when starting movement, when switching from acceleration to deceleration, and when ending movement is greater than in the second mode.

[0066] When the display control unit 104 determines the mode of change in the movement speed of the eye-tracking display VI to be the third mode, it moves the eye-tracking display VI in a straight line from the display start position P2 to the movement end position P3 according to the change in movement speed of the third mode.

[0067] Figure 10 shows an example of the movement of the eye-tracking indicator VI. Figure 10 shows the case where the eye-tracking indicator VI is moved with the change in movement speed described in the second embodiment.

[0068] In Figure 10, each black circle represents a gaze guidance display VI. In Figure 10, multiple black circles indicate the position of the gaze guidance display VI per unit time during movement from the display start position P2 to the movement end position P3. As shown in Figure 10, in the third mode of movement speed change, the gaze guidance display VI starts moving while accelerating linearly from the display start position P2 to the movement end position P3, and ends moving while decelerating linearly.

[0069] Trailers and stationary objects are less likely to make unpredictable changes in direction compared to pedestrians, passenger cars, motorcycles, and bicycles. In particular, the possibility of stationary objects making unpredictable changes in direction is extremely close to zero. Therefore, compared to pedestrians, passenger cars, motorcycles, and bicycles, trailers and stationary objects are objects that drivers U do not need to pay close attention to.

[0070] Therefore, as described above, when the detected object 6 is a trailer or a fixed object, the display control unit 104 moves the eye-tracking display VI by changing the movement speed in a third mode, which is neither the first nor the second mode. As a result, the driver U can easily understand that the eye-tracking display VI is intended to guide the driver's gaze to the trailer or fixed object. Furthermore, since the change in movement speed in the third mode is not a biological movement, it is less likely to be noticed by the eye-tracking display VI compared to the first and second modes. In other words, the change in movement speed in the third mode is less likely to distract the driver U compared to the first and second modes. Thus, when guiding the driver's gaze to an object 6 that does not require the driver U to pay attention, it is possible to suppress the driver U from being distracted by the eye-tracking display VI.

[0071] [1-4-2. Less than the specified distance] If the display control unit 104 determines that the distance between the display start position P2 and the movement end position P3 on the windshield 3 is less than a predetermined distance, it moves the eye-tracking display VI from the display start position P2 to the movement end position P3 at a constant speed.

[0072] Figure 11 shows an example of the movement of the eye-tracking indicator VI. Figure 11 shows the case where the eye-tracking indicator VI is moved at a constant speed.

[0073] In Figure 11, each black circle represents a gaze guidance display VI. In Figure 11, multiple black circles indicate the position of the gaze guidance display VI per unit time during movement from the display start position P2 to the movement end position P3. As shown in Figure 11, if it is determined that the distance between the display start position P2 and the movement end position P3 on the windshield 3 is less than a predetermined distance, the gaze guidance display VI moves at a constant speed from the display start position P2 to the movement end position P3.

[0074] The shorter the distance between the display start position P2 and the movement end position P3, the more difficult it becomes for the driver U to understand the nature of the change in the movement speed of the eye guidance display VI, even if it is changed. Therefore, when the distance between the display start position P2 and the movement end position P3 is less than a predetermined distance, the eye guidance device 7 sets the movement speed to a constant speed regardless of the type of object 6. In other words, when the distance between the display start position P2 and the movement end position P3 is less than a predetermined distance, the eye guidance device 7 does not change the movement speed of the eye guidance display VI according to the type of object 6. This prevents confusion for the driver U when it is difficult to understand the nature of the change in the movement speed of the eye guidance display VI.

[0075] [2. Operation of the eye-tracking device] Next, the operation of the gaze guidance device 7 according to this embodiment will be described. Figure 12 is a flowchart showing the operation of the eye-tracking device 7.

[0076] The object detection unit 101 detects an object 6 located in front of the vehicle 1 (step S1). Step S1 corresponds to "Step 1".

[0077] The display control unit 104 determines whether or not the object 6 was detected in step S1 (step S2). The determination in step S2 is made by whether or not the display control unit 104 receives data from the object detection unit 101.

[0078] If the display control unit 104 determines that object 6 has not been detected (step S2: NO), the processor 100 returns to step S1 and performs the process of step S1 again.

[0079] On the other hand, if the display control unit 104 determines that the object 6 has been detected (step S2: YES), it determines whether the distance between the display start position P2 and the display end position P3 is greater than or equal to a predetermined distance (step S3).

[0080] If the display control unit 104 determines that the distance between the display start position P2 and the display end position P3 is greater than or equal to a predetermined distance (step S3: YES), it determines whether the detected object 6 is a pedestrian, a passenger car or the like, a trailer or a fixed object (step S4).

[0081] If the display control unit 104 determines that the detected object 6 is a pedestrian (step S4: pedestrian), it displays the eye-tracking display VI and moves the eye-tracking display VI with the change in movement speed as in the first mode (step S5). Step S5 corresponds to "Step 2".

[0082] Returning to the explanation of step S4, if the display control unit 104 determines that the detected object 6 is a passenger car or the like (step S4: passenger car or the like), it displays the eye-tracking display VI and moves the eye-tracking display VI with the change in movement speed of the second mode (step S6). Step S6 corresponds to "Step 2".

[0083] Returning to the explanation of step S4, if the display control unit 104 determines that the detected object 6 is a trailer or a fixed object (step S4: trailer or fixed object), it displays the eye-tracking display VI and moves the eye-tracking display VI with the change in movement speed of the third mode (step S7). Step S7 corresponds to "Step 2".

[0084] Returning to the explanation of step S3, if the display control unit 104 determines that the distance between the display start position P2 and the display end position P3 is less than a predetermined distance (step S3: NO), it displays the eye-tracking display VI and moves the eye-tracking display VI at a constant speed (step S8). Step S8 corresponds to "Step 2".

[0085] [3. Other Embodiments] The embodiments described above are merely examples and can be modified and applied as needed.

[0086] In the embodiment described above, a four-wheeled automobile vehicle 1 was used as an example of a "vehicle," but the number of wheels is not limited to four as long as the "vehicle" has a windshield 3.

[0087] In the embodiments described above, the "first mode" is exemplified as a 1 / f fluctuation, the "second mode" is exemplified as a mode with minimal jerk, and the "third mode" is exemplified as a mode with linear acceleration and linear deceleration. However, the "first mode" is not limited to 1 / f fluctuation and may be any mode other than the "second mode" and the "third mode". However, it is preferable that the "first mode" is a mode in which a person is more likely to notice the eye guidance display VI than the "second mode" and the "third mode". Furthermore, the "second mode" is not limited to a mode with minimal jerk and may be any mode other than the "first mode" and the "third mode". However, it is preferable that the "second mode" is a mode in which a person is at least more likely to notice the eye guidance display VI than the "third mode". Furthermore, the "third mode" is not limited to a mode with linear acceleration and linear deceleration and may be any mode other than the "first mode" and the "second mode". However, if the "third aspect" is a form of biological movement, then it is preferable that the "third aspect" is a form of movement that does not involve biological movement.

[0088] In the embodiment described above, the eye-tracking display VI is displayed on the windshield 3 by the HUD 5. However, the means for displaying the eye-tracking display VI can be any means for displaying a virtual image on the windshield 3, and is not limited to the HUD 5. For example, it could be a display means such as an LED (Light Emitting Diode).

[0089] The processor 100 may consist of multiple processors or a single processor. The processor 100 may also be hardware programmed to implement the functions described above. In this case, the processor 100 may consist of, for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0090] Furthermore, the configuration of each part of Vehicle 1 shown in Figure 2 is merely an example, and the specific implementation is not particularly limited. In other words, it is not necessarily required that hardware corresponding to each part be implemented individually; it is certainly possible to configure a system where a single processor executes a program to realize the functions of each part. Also, in the embodiments described above, some of the functions realized by software may be implemented as hardware, or conversely, some of the functions realized by hardware may be implemented as software.

[0091] Furthermore, the operation steps shown in Figure 12 are divided according to the main processing content, and the present invention is not limited by the way the processing units are divided or their names. Depending on the processing content, it may be further divided into more steps. Alternatively, it may be divided so that one step unit includes even more processing. Also, the order of the steps may be changed as appropriate, as long as it does not impede the spirit of the present invention.

[0092] Furthermore, when the eye-tracking method using the eye-tracking device 7 described above is implemented using the processor 100, the program to be executed by the processor 100 can be configured as a recording medium or a transmission medium for transmitting this program. In other words, the control program 111 can also be implemented by recording it on a portable information recording medium. Examples of information recording media include magnetic recording media such as hard disks, optical recording media such as CDs, and semiconductor storage devices such as USB (Universal Serial Bus) memory and SSDs (Solid State Drives), but other recording media can also be used.

[0093] [4. Configurations supported by the above embodiments] The above embodiment supports the following configuration:

[0094] (Composition 1) A gaze guidance device for guiding the gaze of a vehicle driver, comprising: a detection unit for detecting an object located in front of the vehicle; and a display control unit that, when the detection unit detects the object, displays a gaze guidance display on the windshield of the vehicle to guide the driver's gaze toward the object, wherein the display control unit displays the gaze guidance display so as to move from the display start position toward the object with a change in movement speed according to the type of object. With the eye-tracking device of Configuration 1, the movement speed of the eye-tracking display can be varied depending on the type of object. Therefore, the driver can understand which object they want to guide their gaze to, not only from the movement result of the eye-tracking display but also from the change in the movement speed of the eye-tracking display. Thus, the driver can easily understand which object they want to guide their gaze to. Furthermore, because the driver can understand which object they want to guide their gaze to from the change in the movement speed of the eye-tracking display, the driver can quickly understand which object they want to guide their gaze to.

[0095] (Configuration 2) The eye-tracking device according to configuration 1, wherein the display control unit displays the eye-tracking display in accordance with the change in movement speed of the first embodiment when the type of object is a pedestrian. According to the eye-tracking device of configuration 2, the eye-tracking display can be moved in response to changes in movement speed corresponding to the object being a pedestrian. Therefore, the driver can easily understand that they want to guide the driver's gaze to a pedestrian.

[0096] (Composition 3) The first aspect is the gaze guidance device according to configuration 2, wherein the fluctuation is 1 / f. According to the eye-tracking device of configuration 3, the eye-tracking display can be moved by changing the speed of movement in a manner that is easily noticed by people. Therefore, drivers can easily understand that they want to guide the driver's gaze to pedestrians, and the eye-tracking display can increase the likelihood that the driver will turn their gaze towards pedestrians.

[0097] (Composition 4) The eye-tracking device according to any one of Technology 1 to Technology 3, wherein the display control unit displays the eye-tracking display in accordance with the change in movement speed of the second embodiment when the type of object is a passenger car, a motorcycle, or a bicycle. According to the eye-tracking device of configuration 4, the eye-tracking display can be moved in accordance with changes in travel speed corresponding to whether the target object is a passenger car, motorcycle, or bicycle. Therefore, the driver can easily understand that they want to guide their gaze to an object that is either a passenger car, motorcycle, or bicycle.

[0098] (Composition 5) The second embodiment is the gaze guidance device according to Technical Reference 4, which is an embodiment with minimal jerk. According to the eye-tracking device of configuration 5, the eye-tracking display can be moved by changing the speed of movement in a manner that is easily noticed by people. Therefore, drivers can easily understand that they want to guide their gaze to an object that is either a passenger car, a motorcycle, or a bicycle, and the eye-tracking display can increase the likelihood that the driver will direct their gaze towards the object.

[0099] (Composition 6) The eye-tracking device according to any one of the technologies 1 to 5, wherein the display control unit displays the eye-tracking display in accordance with the change in the movement speed of the third embodiment when the type of object is a trailer or a fixed object. According to the eye-tracking device of configuration 6, the eye-tracking display can be moved in accordance with changes in the travel speed corresponding to whether the target object is a trailer or a fixed object. Therefore, the driver can easily understand that they want to guide their gaze to the trailer or fixed object.

[0100] (Composition 7) A third aspect is the gaze guidance device according to Technical Reference 6, wherein the device accelerates linearly and then decelerates linearly. According to the eye-tracking device of configuration 7, by displaying the eye-tracking indicator in a manner that does not mimic biological movement, the driver is less likely to be distracted by the indicator. Therefore, when guiding the driver's gaze to an object that does not require the driver to pay close attention, it is possible to suppress the driver from being distracted by the eye-tracking indicator.

[0101] (Composition 8) The eye-tracking device according to any one of configurations 1 to 7, wherein the display control unit changes the movement speed of the eye-tracking display when the distance to be moved of the eye-tracking display is greater than or equal to a predetermined distance. According to the eye-tracking device of configuration 8, the movement speed of the eye-tracking display is changed when changes in movement speed are easily perceived, so that the driver can accurately and easily understand which object they want to guide their gaze to.

[0102] (Composition 9) The eye-tracking device according to configuration 8, wherein the display control unit moves the eye-tracking display at a constant speed when the distance to be moved the eye-tracking display is less than a predetermined distance. The eye-tracking device of configuration 9 can prevent confusion for the driver in cases where it is difficult to understand the nature of the change in the moving speed of the eye-tracking display.

[0103] (Composition 10) A method for guiding the gaze of a vehicle driver, comprising: a first step of detecting an object located in front of the vehicle; and a second step of, if the object is detected in the first step, displaying a gaze guidance indicator on the windshield of the vehicle to guide the driver's gaze toward the object, wherein the second step involves displaying the gaze guidance indicator so that it moves from its display start position toward the object with a change in movement speed according to the type of object. The gaze guidance method of configuration 10 produces the same effect as the gaze guidance device of configuration 1.

[0104] (Composition 11) A processor for a gaze guidance device that guides the gaze of a vehicle driver comprises a detection unit that detects an object located in front of the vehicle, and a display control unit that, when the detection unit detects the object, displays a gaze guidance display on the vehicle's windshield to guide the driver's gaze toward the object, and the display control unit is programmed to display the gaze guidance display so that it moves from the display start position toward the object with a change in movement speed according to the type of object. According to the program of configuration 11, it produces the same effect as the eye-tracking device of configuration 1. [Explanation of symbols]

[0105] 1...Vehicle, 2...Steering wheel, 3...Windshield, 4...Instrument panel, 5...HUD, 6...Object, 7...Eye guidance device, 8...Front camera, 9...Driver monitoring camera, 10...Position detection device, 100...Processor, 101...Object detection unit, 102...Eye detection unit, 103...Head detection unit, 104...Display control unit, 110...Memory, 111...Control program (program), GF1, GF2, GF3...Graph, HD...Head, L1...Determined distance, L2...Line, P1...Eye position, P2...Display start position, P3...Movement end position, P3-1...First movement end position, P3-2...Second movement end position, P4...Position, S1...Step (first step), S2~S4...Step, S5~S8...Step (second step), SG...Captured image, U...Driver, VI...Eye guidance display.

Claims

1. A gaze guidance device that guides the driver's gaze, A detection unit for detecting an object located in front of the vehicle, The vehicle comprises a display control unit that, when the detection unit detects the object, displays a gaze guidance display on the vehicle's windshield to guide the driver's gaze toward the object, The display control unit, The eye-tracking display is displayed so that it moves from the starting position of the eye-tracking display towards the object, with a change in movement speed according to the type of object. Eye guidance device.

2. The display control unit, If the type of object is a pedestrian, the visual guidance display is displayed in accordance with the change in movement speed in the first embodiment. The gaze guidance device according to claim 1.

3. The first aspect is 1 / f fluctuation, The gaze guidance device according to claim 2.

4. The display control unit, If the type of object is a passenger car, a motorcycle, or a bicycle, the eye-tracking indicator is displayed in accordance with the change in the movement speed of the second embodiment. The gaze guidance device according to claim 1.

5. The second embodiment described above is the embodiment with the minimum jerk. The gaze guidance device according to claim 4.

6. The display control unit, If the type of object is a trailer or a fixed object, the visual guidance display is displayed in accordance with the change in the movement speed of the third embodiment. The gaze guidance device according to claim 1.

7. The third aspect is an aspect in which the vehicle accelerates linearly and then decelerates linearly. The gaze guidance device according to claim 6.

8. The display control unit, If the distance to which the eye-tracking indicator is moved is greater than or equal to a predetermined distance, the movement speed of the eye-tracking indicator is changed. The gaze guidance device according to claim 1.

9. The display control unit, If the distance to which the eye-tracking indicator is moved is less than a predetermined distance, the eye-tracking indicator is moved at a constant speed. The gaze guidance device according to claim 8.

10. A method for guiding the gaze of a vehicle driver, The first step is to detect an object located in front of the vehicle. If the object is detected in the first step, the second step includes displaying a visual guidance indicator on the vehicle's windshield to guide the driver's gaze towards the object, The second step described above is: The eye-tracking display is displayed so that it moves from the starting position of the eye-tracking display towards the object, with a change in movement speed according to the type of object. Eye guidance method.

11. The processor for the eye-tracking device that guides the driver's gaze is A detection unit for detecting an object located in front of the vehicle, When the detection unit detects the object, the display control unit functions as a display control unit that displays a gaze guidance display on the vehicle's windshield to guide the driver's gaze towards the object. The display control unit, The eye-tracking display is displayed so that it moves from the starting position of the eye-tracking display towards the object, with a change in movement speed according to the type of object. program.

Images