Line-of-sight guiding device, line-of-sight guiding method, and computer program product

Abstract

A gaze guiding device, a gaze guiding method, and a computer program product are provided. A line-of-sight guide device (7) for guiding the line of sight of a driver (U) of a vehicle (1) is provided with: a detection unit (101) for detecting an object (6) present in front of the vehicle (1); and a display control unit (104) that, when the detection unit (101) detects the object (6), displays a line-of-sight guidance display (VI) for guiding the line of sight of the driver (U) to the object (6) on a windshield (3) of the vehicle (1). The display control unit (104) displays the visual line guide display (VI) such that the visual line guide display moves from a display start position (P2) of the visual line guide display (VI) in the direction of the object (6) in accordance with a change in a movement speed corresponding to the type of the object (6).

Description

Technical Field

[0001] This invention relates to a line-of-sight guidance device, a line-of-sight guidance method, and a computer program product. Background Technology

[0002] In recent years, research related to driver visual recognition has progressed in order to further improve traffic safety and contribute to the development of sustainable transportation systems. Furthermore, prior to this, techniques for guiding the driver's gaze have been known as technologies related to driver visual recognition. For example, Patent Document 1 discloses a gaze guidance device that guides the driver's gaze toward an object by moving a visual stimulus displayed on the windshield. Additionally, Patent Document 1 discloses that the larger the angle between the object and the line of sight, the greater the speed at which the visual stimulus moves.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2017-187955 Summary of the Invention

[0006] The problem that the invention aims to solve

[0007] Patent Document 1 describes a structure that moves a visual stimulus to an object in the same amount of time, regardless of the distance between the driver's viewpoint and the object. Therefore, in the structure of Patent Document 1, there is a possibility that the driver can only determine the object being guided by the movement of the visual stimulus, and may not be able to easily determine what object the visual stimulus intends to guide their gaze to.

[0008] Therefore, the present invention was made in view of the above circumstances, and its object is to enable the driver to easily determine which object to guide his gaze to.

[0009] Methods for solving problems

[0010] One aspect of the present invention is a gaze guidance device that guides the driver's gaze toward a vehicle. The gaze guidance device comprises: a detection unit that detects an object present in front of the vehicle; and a display control unit that, when the detection unit detects the object, displays a gaze guidance display on the windshield of the vehicle that guides the driver's gaze toward the object. The display control unit displays the gaze guidance display such that it moves from a display start position toward the object according to a change in movement speed corresponding to the type of object.

[0011] Invention Effects

[0012] According to one aspect of the invention, the driver can easily determine which object to direct their gaze to. Attached Figure Description

[0013] Figure 1 It is a diagram showing the structure of the vehicle's interior.

[0014] Figure 2 It is a diagram showing the structure of a vehicle.

[0015] Figure 3 This is a diagram used to illustrate the processing of the display control unit.

[0016] Figure 4 This is a diagram used to illustrate the determination of the end position of the second movement.

[0017] Figure 5 This is a graph used to illustrate the change in movement speed in the first mode.

[0018] Figure 6 This is an example of a viewpoint guiding the movement of the display.

[0019] Figure 7 This is a graph used to illustrate the change in movement speed in the second method.

[0020] Figure 8 This is an example of a viewpoint guiding the movement of the display.

[0021] Figure 9 It is a chart used to illustrate the changes in the movement speed of a third party.

[0022] Figure 10 This is an example of a viewpoint guiding the movement of the display.

[0023] Figure 11 This is an example of a viewpoint guiding the movement of the display.

[0024] Figure 12 This is a flowchart illustrating the operation of the line-of-sight guidance device.

[0025] Explanation of reference numerals in the attached figures

[0026] 1…Vehicle; 2…Steering wheel; 3…Windshield; 4…Instrument panel; 5…HUD; 6…Object; 7…Gaze guidance device; 8…Front camera; 9…Driver monitoring camera; 10…Position detection device; 100…Processor; 101…Object detection unit; 102…Gaze detection unit; 103…Head detection unit; 104…Display control unit; 110…Memory; 111…Control program (program); GF1, GF2, GF3…Graph; HD…Head; L1…Specified distance; L2…Line; P1…Gaze position; P2…Display start position; P3…Movement end position; P3-1…First movement end position; P3-2…Second movement end position; P4…Position; S1…Step (first step); S2~S4…Step; S5~S8…Step (second step); SG…Image capture; U…Driver; VI…Gaze guidance display. Detailed Implementation

[0027] [1. Vehicle Structure]

[0028] The embodiments will now be described with reference to the accompanying drawings.

[0029] exist Figure 1 , Figure 3 , Figure 4 , Figure 6 , Figure 8 , Figure 10 as well as Figure 11 The diagram illustrates the X, Y, and Z axes. The X, Y, and Z axes are orthogonal to each other. The Z axis represents the vertical direction. The X and Y axes are parallel to the horizontal direction when vehicle 1 is in motion. The X axis represents the left-right direction as the vehicle's width. The Y axis represents the front-back direction. The positive direction of the X axis represents the right. The positive direction of the Y axis represents the front. The positive direction of the Z axis represents the top.

[0030] Figure 1 This is a diagram showing the structure inside vehicle 1.

[0031] As an example of vehicle 1 in this embodiment, a four-wheeled automobile is shown.

[0032] The vehicle 1 is equipped with a steering wheel 2 for operating the vehicle 1, a windshield 3 that separates the outside of the vehicle from the interior, and an instrument panel 4. The steering wheel 2 is located in the instrument panel 4 opposite to the driver U who is seated in the driver's seat.

[0033] A HUD (Head-Up Display) 5 is located on the dashboard 4. The HUD 5 displays information to the driver (U) by projecting light onto the windshield 3. Figure 4 The line of sight is directed toward the object 6 (e.g., reference). Figure 4The eye guidance display VI is a virtual image. Additionally, object 6 represents the object that the driver U is expected to look at. The HUD5 displays the eye guidance display VI on the windshield 3, allowing the driver U, seated in the driver's seat, to simultaneously observe the scene in front of the vehicle 1 and the eye guidance display VI through the windshield 3.

[0034] In addition, Figure 1 In the image, the shape of the visual indicator (VI) is exemplified as a circle. However, Figure 1 The shape of the visual guide VI shown is just one example; it could be a rectangle or a star.

[0035] exist Figure 1 The dashed line diagram illustrates the displayable area A1, which displays the line-of-sight guidance display VI. The size of the displayable area A1 corresponds to the size of the magnifying reflector (concave mirror) on the HUD5. Furthermore, the size of the displayable area A1 on the windshield 3 is not limited to a specific size. Figure 1 The size shown. Additionally, in Figure 1 The example illustrates a shape where the long side of the displayable area A1 is aligned with the vehicle width direction, but the shape of the displayable area A1 is not limited to this. Figure 1 The shape shown.

[0036] Figure 2 This is a diagram showing the structure of vehicle 1.

[0037] Vehicle 1 is equipped with a line-of-sight guidance device 7. The line-of-sight guidance device 7 is equipped with a processor 100 such as a CPU (Central Processing Unit) and an MPU (Micro-processing Unit) and a memory 110.

[0038] The processor 100 controls each part of the gaze guidance device 7 by reading and executing the control program 111 stored in the memory 110. The processor 100 functions as the object detection unit 101, gaze detection unit 102, head detection unit 103, and display control unit 104 by executing the control program 111 stored in the memory 110.

[0039] Object Inspection Department 101 is an example of an "inspection department".

[0040] Memory 110 is a storage device that stores programs executed by processor 100 and data processed by processor 100. Memory 110 stores control program 111 executed by processor 100 and various other data. Memory 110 has non-volatile storage areas. In addition, memory 110 has volatile storage areas, which constitute the working area of ​​processor 100. Memory 110 is composed, for example, of ROM (Read Only Memory) or RAM (Random Access Memory).

[0041] Control program 111 is equivalent to "program".

[0042] The line-of-sight guidance device 7 is connected to the HUD 5, the front camera 8, the driver monitoring camera 9, and the position detection device 10. Furthermore, the devices connected to the line-of-sight guidance device 7 are not limited to these; they can also be connected to other devices such as vehicle-to-vehicle communication devices, GNSS (Global Navigation Satellite System) units, rear cameras, and vehicle speed sensors.

[0043] The front camera 8 is a camera positioned at a predetermined location on the vehicle 1 to capture images of the front of the vehicle 1. The front camera 8 captures images at predetermined intervals, including when the vehicle 1 is ignited and when the vehicle 1's auxiliary power is on. Each time the front camera 8 captures an image SG (refer to...) Figure 3 The image data is output to the line-of-sight guidance device 7.

[0044] The driver monitoring camera 9 is a camera installed in a designated location inside the vehicle 1 to capture images of the driver U seated in the driver's seat. The camera's field of view includes at least the head (HD) of the driver U seated in the driver's seat. Figure 4 The driver monitoring camera 9 takes pictures at predetermined intervals when the vehicle 1 is ignited and when the vehicle 1's auxiliary power is on. Each time it takes a picture, the driver monitoring camera 9 outputs the image data of the captured image to the line-of-sight guidance device 7.

[0045] The position detection device 10 is a device capable of detecting the position of objects existing around the vehicle 1. The position detection device 10 is composed of at least one of the following: sonar, radar, lidar, which can determine the distance between the vehicle 1 and the object, and a stereo camera that can determine the distance between the vehicle 1 and the object using parallax.

[0046] As described above, the processor 100 of the gaze guidance device 7 functions as the object detection unit 101, the gaze detection unit 102, the head detection unit 103, and the display control unit 104.

[0047] [1-1. Object Inspection Department]

[0048] The object detection unit 101 detects an object 6 present in front of the vehicle 1. The object detection unit 101 detects the object 6 reflected in the captured image SG obtained by the front camera 8 based on image data received from the front camera 8. As mentioned above, the object 6 refers to an object that the driver U should be looking at. In this embodiment, the object 6 can be any of a pedestrian, a passenger vehicle, an autonomous two-wheeled vehicle, a bicycle, a trailer, or a fixed object. Pedestrians also include children. Examples of fixed objects include road signs and road sight guidance signs. The object detection unit 101 detects the object 6 reflected in the captured image SG by performing pattern matching and color-based image processing on the captured image SG. Furthermore, the data required for detecting the object 6 (e.g., shape data and color data) is stored in the memory 110 for each pedestrian, passenger vehicle, autonomous two-wheeled vehicle, bicycle, trailer, and fixed object.

[0049] Furthermore, the object detection unit 101 detects the position of the detected object 6. More specifically, the object detection unit 101 detects the relative position of the detected object 6 with respect to the vehicle 1 when viewed from above. The object detection unit 101 detects the relative position of the detected object 6 based on at least one of the detection result of the position detection device 10 and the image SG captured by the front camera 8. In addition, if the detected object 6 is another vehicle and the line-of-sight guidance device 7 is connected to a vehicle-to-vehicle communication device and a GNSS unit, the object detection unit 101 can also detect the relative position of the detected object 6 based on the position of the other vehicle received by the vehicle-to-vehicle communication device and the position of the vehicle 1 received by the GNSS unit.

[0050] In addition to the front camera 8, the vehicle-to-vehicle communication device, and the GNSS unit, the object detection unit 101 can also use V2X-based (road-to-vehicle, vehicle-to-vehicle, etc.) communication and judgment performed by the server in a virtual environment to detect the object 6 reflected in the captured image SG and to detect the relative position of the object 6.

[0051] When the object detection unit 101 detects an object 6, it outputs data indicating the type of the detected object 6, data indicating the relative position of the detected object 6 with respect to the vehicle 1, and data indicating the position of the detected object 6 in the captured image to the display control unit 104.

[0052] [1-2. Line of sight detection unit]

[0053] The gaze detection unit 102 detects the direction of the driver U's gaze. The gaze detection unit 102 detects the direction of the driver U's gaze based on image data from images captured by the driver monitoring camera 9. The gaze detection unit 102 detects the driver U's eyes from the images captured by the driver monitoring camera 9 using pattern matching, color analysis, etc., and determines the direction in which the detected eyes are looking as the direction of the gaze. Furthermore, the data required for eye detection (data on eye shape and color) is stored in the memory 110.

[0054] When the gaze detection unit 102 detects the direction of the driver U's gaze, it outputs data indicating the direction of the detected gaze to the display control unit 104.

[0055] [1-3. Head Detection Section]

[0056] The head detection unit 103 detects the head (HD) of the driver U seated in the driver's seat. The head detection unit 103 detects the head (HD) of the driver U based on image data from images captured by the driver's monitoring camera 9. The head detection unit 103 detects the head (HD) from the images captured by the driver's monitoring camera 9 using pattern matching, color analysis, etc. Next, the head detection unit 103 detects the position of the head (HD) in the captured image. Furthermore, based on the size of the head (HD) reflected in the captured image and its position in the captured image, the head detection unit 103 detects the position of the head (HD) in the vehicle 1 when viewed from above. Moreover, the position of the head (HD) in the vehicle 1 corresponding to its size and position in the captured image is determined through prior testing and simulation, and stored as data in the memory 110.

[0057] [1-4. Display Control Unit]

[0058] The display control unit 104 controls the operation of the HUD5 to display the line-of-sight display VI on the windshield 3. The display control unit 104 displays the line-of-sight display VI on the windshield 3 by performing the following processes, and also moves the line-of-sight display VI displayed on the windshield 3.

[0059] Reference Figure 3 The processing of the display control unit 104 will be explained.

[0060] Figure 3 This diagram illustrates the processing of the display control unit 104.

[0061] The display control unit 104 detects the position of the driver U's gaze on the windshield 3 (hereinafter referred to as "gaze position P1" by the reference numeral "P1") based on the direction of the gaze shown in the data received from the gaze detection unit 102. For example, if the memory 110 stores data that corresponds the direction of the driver U's gaze to the driver U's gaze position P1 on the windshield 3, the display control unit 104 refers to this data to detect the driver U's gaze position P1 on the windshield 3.

[0062] The display control unit 104 determines whether the detected driver U's line of sight P1 is within the displayable area A1. Data indicating the position of the displayable area A1 on the windshield 3 is stored in the memory 110. The display control unit 104 determines whether the detected driver U's line of sight P1 is within the displayable area A1 by referring to this data stored in the memory 110.

[0063] If the display control unit 104 determines that the detected driver U's line of sight P1 is within the displayable area A1, then... Figure 3 As shown, the position offset by a predetermined distance L1 from the driver U's line of sight position P1 is determined as the display start position P2 of the line-of-sight guidance display VI. In addition, the display control unit 104 loads a coordinate system defining the shape, size, and vertical and horizontal directions of the displayable area A1 into the memory 110, and determines the display start position P2 by referring to the loaded coordinate system.

[0064] The specified distance L1 is preferably a distance that places the display start position P2 within the range of the central field of vision centered on the line of sight position P1. For example, the specified distance L1 is a distance that places the display start position P2 at a position 5 degrees from the driver's line of sight U in both the vertical and horizontal fields of vision. Furthermore, the position 5 degrees from the driver's line of sight U in both the vertical and horizontal fields of vision is a position within the effective field of vision centered on the line of sight position P1. Figure 3 The example shows a structure where the display start position P2 is determined to be the lower left of the viewing position P1. However, the position of the display start position P2 can be, for example, the lower right of the viewing position P1, the upper right of the viewing position P1, or the upper left of the viewing position P1.

[0065] When the display control unit 104 determines the display start position P2, it also determines the end position P3 of the line-guided display VI. Furthermore, the display control unit 104 loads a coordinate system defining the shape, size, and vertical / horizontal directions of the displayable area A1 into the memory 110, and determines the end position P3 by referring to the loaded coordinate system.

[0066] In determining the end position P3 of the movement, the display control unit 104 determines the end position P3 of the movement in the vertical direction of the displayable area A1 (hereinafter, it is referred to as "first end position P3-1" by the reference numeral "P3-1"). Furthermore, the vertical direction of the displayable area A1 corresponds to the vertical direction of the windshield 3 and the direction of the short side of the displayable area A1.

[0067] Furthermore, in determining the end position P3 of the movement, the display control unit 104 determines the end position P3 of the movement in the left-right direction of the displayable area A1 (hereinafter, it is referred to as "second end position P3-2" by the reference numeral "P3-2"). In addition, the left-right direction of the displayable area A1 corresponds to the left-right direction of the windshield 3 and the long side direction of the displayable area A1.

[0068] First, the determination of the first movement end position P3-1 will be explained.

[0069] The display control unit 104 obtains the position of the object 6 in the vertical direction of the captured image SG from the data output by the object detection unit 101. The vertical direction of the captured image SG corresponds to the vertical direction of the scene reflected in the captured image SG. Next, the display control unit 104 converts the obtained position of the object 6 into the vertical direction of the displayable area A1, and determines the converted position as the first movement end position P3-1. Furthermore, the relationship between the position of the object 6 in the vertical direction of the captured image SG and its position in the vertical direction of the displayable area A1 is determined through prior simulation, etc., and stored as data in the memory 110.

[0070] Next, the determination of the second movement end position P3-2 will be explained.

[0071] The display control unit 104 determines the second movement end position P3-2 based on the relative position of the detected object 6 and the position of the detected head HD.

[0072] Figure 4 This is a diagram used to illustrate the determination of the final position P3-2 of the second movement.

[0073] exist Figure 4 In the example, object 6 represents a pedestrian present in front of vehicle 1.

[0074] When viewed from above, the display control unit 104 detects the position of the intersection of the line connecting the position of the driver U's head HD and the position of the object 6 with the windshield 3 in the left-right direction of the vehicle 1. Based on the relative position of the object 6 detected by the object detection unit 101 and the position of the driver U's head HD detected by the head detection unit 103, the display control unit 104 detects the position of the intersection of the line connecting the position of the driver U's head HD and the position of the object 6 with the windshield 3 in the left-right direction.

[0075] exist Figure 4 In the diagram, line L2 is the line connecting the position of the driver U's head HD to the position of the pedestrian when viewed from above vehicle 1. Figure 4 In this case, when looking down at vehicle 1, the position P4 of the intersection of detection line L2 of display control unit 104 and windshield 3 in the left-right direction is displayed.

[0076] Next, when the display control unit 104 detects the position of the intersection in the left-right direction, if the detected position is within the displayable area A1, it determines the detected position as the second movement end position P3-2.

[0077] Return to reference Figure 3 The display control unit 104, when determining the first movement end position P3-1 and the second movement end position P3-2, determines the position defined by the first movement end position P3-1 and the second movement end position P3-2 as the movement end position P3.

[0078] When determining the display start position P2 and the movement end position P3, the display control unit 104 calculates the distance between the display start position P2 and the movement end position P3 on the windshield 3. The display control unit 104 calculates the straight-line distance between the display start position P2 and the movement end position P3 in the coordinate system unfolded in the memory 110, converts the calculated distance into a distance on the windshield 3, and thereby calculates the distance between the display start position P2 and the movement end position P3 on the windshield 3.

[0079] When calculating the distance between the start position P2 and the end position P3 of the display on the windshield 3, the display control unit 104 determines whether the calculated distance is above a specified distance. Examples of specified distances include 10cm and 20cm.

[0080] [1-4-1. Above the specified distance]

[0081] When the display control unit 104 determines that the distance between the display start position P2 and the movement end position P3 on the windshield 3 is more than a predetermined distance, it determines the method of changing the movement speed of the line-guided display VI from the display start position P2 to the movement end position P3 based on the type of object 6 detected.

[0082] When the detected object 6 is a pedestrian, the display control unit 104 determines the first mode as the way to guide the movement speed of the display VI by the change of the line of sight.

[0083] The first method is 1 / f fluctuation.

[0084] The change in movement speed in the first mode is shown for example. Figure 5 The change in moving speed is shown.

[0085] Figure 5 This is a graph used to illustrate the change in movement speed in the first mode.

[0086] exist Figure 5 In the center, the vertical axis shows the movement speed of the line-of-sight guidance display VI on the windshield 3. Additionally, in... Figure 5 In the middle, the horizontal axis represents the straight-line distance from the starting position P2 to the ending position P3 after the movement, which is the distance on the windshield 3.

[0087] exist Figure 5 The diagram shows graph GF1, which illustrates an example of the change in movement speed relative to the distance moved from the starting position P2. As shown in graph GF1, the change in movement speed in the first scenario refers to the random and repeated acceleration and deceleration during the movement from the starting position P2 to the ending position P3.

[0088] When the display control unit 104 determines the mode of changing the movement speed of the eye-guided display VI as a first mode, it causes the eye-guided display VI to move linearly from the display start position P2 to the movement end position P3 according to the movement speed change of the first mode.

[0089] Figure 6 This is an example of a diagram illustrating the movement of a visually guided VI.

[0090] exist Figure 6 The image shows the scenario where the eye-guided display VI moves according to the changing speed of the first method.

[0091] exist Figure 6 In the center, each black circle represents a visual guide sign (VI). Figure 6 In the diagram, multiple black circles indicate the position of the line-of-sight guidance display VI per unit time during the movement from the starting position P2 to the ending position P3. (Example)Figure 6 As shown, in the first mode of movement speed change, during the movement from the display start position P2 to the movement end position P3, the eye-guided display VI randomly and repeatedly accelerates and decelerates multiple times.

[0092] Compared to vehicles and fixed objects, pedestrians are more likely to make unpredictable directional changes. Therefore, pedestrians can be considered objects that require more attention from the driver than vehicles and fixed objects.

[0093] Therefore, as described above, when the detected object 6 is a pedestrian, the display control unit 104 guides the viewpoint display VI to move according to the change in the moving speed of the first method.

[0094] 1 / f fluctuations are considered one type of biological motion. Generally speaking, people easily notice biological motions, and the greater the randomness of the motion, the easier it is to notice. 1 / f fluctuations are biological motions, and their randomness is greater than the second type described later.

[0095] Therefore, when the detected object 6 is a pedestrian, the gaze guidance device 7 moves the gaze guidance display VI according to the change in its movement speed in the first mode. Thus, the driver U can easily grasp that the gaze guidance display VI is guiding the driver's gaze towards the pedestrian. Furthermore, by setting the change in the movement speed of the gaze guidance display VI to the first mode, the driver U will quickly notice the gaze guidance display VI, and therefore can quickly grasp that the gaze guidance display VI is guiding the driver's gaze towards the pedestrian.

[0096] When the detected object 6 is any one of a passenger car, an automatic two-wheeled vehicle, or a bicycle (hereinafter, appropriately referred to as "passenger car, etc."), the display control unit 104 determines the second mode to guide the change in the movement speed of the display VI by the line of sight.

[0097] The second method is the method with minimal jerk. More specifically, the second method is the method where the changes in acceleration are small at the three points of origin: when starting to move, when switching from acceleration to deceleration, and when ending to move. More specifically, the second method is the method where the changes in acceleration at the above three points of origin are smaller than the third method described later. Furthermore, in this embodiment, minimal jerk does not include the case where the change in acceleration is zero.

[0098] The change in movement speed in the second method is shown for example. Figure 7 The change in moving speed is shown.

[0099] Figure 7 This is a graph used to illustrate the change in movement speed in the second method.

[0100] exist Figure 7 In the middle, the vertical axis and the horizontal axis are...Figure 5 The vertical axis and the horizontal axis are the same.

[0101] exist Figure 7 The diagram shows graph GF2, which illustrates an example of the change in movement speed relative to the distance traveled from the starting position P2. As graph GF2 shows, the change in movement speed in the second method refers to a change of acceleration or deceleration from the start to the end of the movement. This means that the changes in acceleration at the start of the movement, when switching from acceleration to deceleration, and at the end of the movement are smaller than those in the third method. In other words, the change in movement speed in the second method refers to a change of acceleration or deceleration from the start to the end of the movement, and this change in movement speed at the start, when switching from acceleration to deceleration, and at the end of the movement is smoother than that in the third method.

[0102] When the display control unit 104 determines the method of changing the movement speed of the eye-guided display VI as the second method, it causes the eye-guided display VI to move linearly from the display start position P2 to the movement end position P3 according to the movement speed change of the second method.

[0103] Figure 8 This is an example of a diagram illustrating the movement of a visually guided VI.

[0104] exist Figure 8 The image shows the scenario where the eye-guided display VI moves according to the change in the moving speed of the second method.

[0105] exist Figure 8 In the center, each black circle represents a visual guide sign (VI). Figure 8 In the diagram, multiple black circles indicate the position of the line-of-sight guidance display VI per unit time during the movement from the starting position P2 to the ending position P3. (Example) Figure 8 As shown, in the second mode of movement speed change, during the movement from the display start position P2 to the movement end position P3, the eye guidance display VI gradually accelerates and begins to move, and then the eye guidance display VI gradually decelerates and ends the movement.

[0106] While passenger vehicles can change direction left and right, they are less likely to make sudden changes in direction forward and backward compared to pedestrians. Therefore, passenger vehicles are arguably less necessary objects for the driver U to focus on compared to pedestrians. On the other hand, passenger vehicles are arguably more necessary objects for the driver U to focus on compared to trailers and fixed objects, which are less likely to make unpredictable changes in direction.

[0107] Therefore, as described above, when the detected object 6 is any of the passenger car or the like, the display control unit 104 guides the viewpoint to move the display VI according to the change in the movement speed in the second manner.

[0108] The velocity change with minimal acceleration is known as one of the biological motions, and its randomness is less than that of the first type.

[0109] Therefore, when the detected object 6 is any one of the passenger vehicles, the gaze guidance device 7 moves the gaze guidance display VI according to a second method of changing its movement speed, different from the first method. Thus, the driver U can easily grasp that the gaze guidance display VI is intended to guide their gaze to the object 6, which is a passenger vehicle, etc. Furthermore, by setting the change in the movement speed of the gaze guidance display VI to the second method, the driver U will quickly notice the gaze guidance display VI, and therefore can quickly grasp that the gaze guidance display VI is intended to guide their gaze to the passenger vehicle, etc.

[0110] When the detected object 6 is a trailer or a fixed object, the display control unit 104 determines the method of guiding the movement speed of the display VI to a third mode.

[0111] The third method is to perform linear acceleration followed by linear deceleration.

[0112] The change in movement speed in the third mode is shown for example. Figure 9 The change in moving speed is shown.

[0113] Figure 9 It is a chart used to illustrate the changes in the movement speed of a third party.

[0114] exist Figure 9 In the middle, the vertical axis and the horizontal axis are... Figure 5 The vertical axis and horizontal axis shown are the same.

[0115] exist Figure 9 The diagram shows graph GF3, which illustrates an example of the change in movement speed relative to the distance moved from the starting position P2. As graph GF3 shows, the third-mode movement speed change refers to a linear acceleration and deceleration cycle from the start to the end of the movement, where the changes in acceleration at the start of the movement, when switching from acceleration to deceleration, and at the end of the movement are greater than those in the second-mode.

[0116] When the display control unit 104 determines the method of changing the movement speed of the eye-guided display VI as the third method, it causes the eye-guided display VI to move linearly from the display start position P2 to the movement end position P3 according to the third method movement speed change.

[0117] Figure 10 This is an example of a diagram illustrating the movement of a visually guided VI.

[0118] exist Figure 10 The image shows the scenario where the eye-guided display VI moves according to a third-party movement speed variation.

[0119] exist Figure 10 In the center, each black circle represents a visual guide sign (VI). Figure 10 In the diagram, multiple black circles indicate the position of the line-of-sight guidance display VI per unit time during the movement from the starting position P2 to the ending position P3. (Example) Figure 10 As shown, during the change in the third-party movement speed, the eye-guided display VI starts moving while linearly accelerating from the display start position P2 to the movement end position P3, and ends moving while linearly decelerating.

[0120] Trailers or fixed objects are less likely to change direction unpredictably compared to pedestrians, passenger cars, autonomous two-wheelers, and bicycles. In particular, the probability of a fixed object changing direction unpredictably is infinitesimally close to zero. Therefore, trailers or fixed objects are arguably less necessary for the driver U to keep an eye on compared to pedestrians, passenger cars, autonomous two-wheelers, and bicycles.

[0121] Therefore, as described above, when the detected object 6 is a trailer or a fixed object, the display control unit 104 moves the gaze guidance display VI according to a third-party speed change that is neither the first nor the second method. Thus, the driver U can easily grasp that the gaze guidance display VI is guiding their gaze towards the trailer or fixed object. The third-party speed change is not a biological action, and therefore, compared to the first and second methods, it is not as easy for the driver U to notice the gaze guidance display VI. In other words, compared to the first and second methods, the third-party speed change makes it difficult for the driver U to notice the gaze guidance display VI. Therefore, in guiding the gaze towards an object 6 that is of low necessity for the driver U to look at, the driver U's attention to the gaze guidance display VI can be suppressed.

[0122] [1-4-2. Less than the specified distance]

[0123] If the display control unit 104 determines that the distance between the display start position P2 and the movement end position P3 on the windshield 3 is less than a predetermined distance, it causes the line-of-sight guidance display VI to move from the display start position P2 to the movement end position P3 at a constant speed.

[0124] Figure 11 This is an example of a diagram illustrating the movement of a visually guided VI.

[0125] exist Figure 11 The image shows the case where the eye-guided display VI moves at a constant speed.

[0126] exist Figure 11 In the center, each black circle represents a visual guide sign (VI). Figure 11 In the diagram, multiple black circles indicate the position of the line-of-sight guidance display VI per unit time during the movement from the starting position P2 to the ending position P3. (Example) Figure 11 As shown, when the distance between the display start position P2 and the movement end position P3 on the windshield 3 is determined to be less than a specified distance, the line-of-sight guidance display VI moves from the display start position P2 to the movement end position P3 at a constant speed.

[0127] The shorter the distance between the starting position P2 and the ending position P3, the more difficult it is for the driver U to grasp the manner in which the movement speed of the gaze guidance display VI changes. Therefore, when the distance between the starting position P2 and the ending position P3 is less than a predetermined distance, the gaze guidance device 7 sets the movement speed to a constant speed regardless of the type of object 6. That is, when the distance between the starting position P2 and the ending position P3 is less than a predetermined distance, the gaze guidance device 7 does not change the movement speed of the gaze guidance display VI in a way that corresponds to the type of object 6. As a result, it is possible to suppress confusion and distress for the driver U when it is difficult to grasp the manner in which the movement speed of the gaze guidance display VI changes.

[0128] [Action of the gaze guidance device]

[0129] Next, the operation of the line-of-sight guidance device 7 in this embodiment will be explained.

[0130] Figure 12 This is a flowchart illustrating the operation of the line-of-sight guidance device 7.

[0131] The object detection unit 101 detects the object 6 present in front of the vehicle 1 (step S1).

[0132] Step S1 is equivalent to "the first step".

[0133] The display control unit 104 determines whether the object 6 was detected in step S1 (step S2). The determination in step S2 is based on whether the display control unit 104 received data from the object detection unit 101.

[0134] If the display control unit 104 determines that no object 6 has been detected (step S2: No), the processor 100 returns the processing to step S1 and performs the processing of step S1 again.

[0135] On the other hand, when the display control unit 104 determines that an object 6 has been detected (step S2: Yes), it determines whether the distance between the display start position P2 and the movement end position P3 is greater than or equal to a predetermined distance (step S3).

[0136] If the display control unit 104 determines that the distance between the display start position P2 and the movement end position P3 is more than a predetermined distance (step S3: YES), then it determines whether the detected object 6 is a pedestrian, a passenger vehicle, a trailer, or a fixed object (step S4).

[0137] When the display control unit 104 determines that the detected object 6 is a pedestrian (step S4: pedestrian), it displays the gaze guidance display VI and moves the gaze guidance display VI according to the change in the movement speed of the first method (step S5).

[0138] Step S5 is equivalent to "the second step".

[0139] Returning to the explanation of step S4, if the display control unit 104 determines that the detected object 6 is any one of the passenger car, etc. (step S4: passenger car, etc.), it displays the line-of-sight guidance display VI, and moves the line-of-sight guidance display VI according to the change in the movement speed of the second method (step S6).

[0140] Step S6 is equivalent to "the second step".

[0141] Returning to the explanation of step S4, if the display control unit 104 determines that the detected object 6 is a trailer or a fixed object (step S4: trailer or fixed object), it displays the line-of-sight display VI and moves the line-of-sight display VI according to the change in the third-party movement speed (step S7).

[0142] Step S7 is equivalent to "the second step".

[0143] Returning to the explanation of step S3, if the display control unit 104 determines that the distance between the display start position P2 and the movement end position P3 is less than a predetermined distance (step S3: No), it displays the line-of-sight guidance display VI and moves the line-of-sight guidance display VI at a constant speed (step S8).

[0144] Step S8 is equivalent to "the second step".

[0145] [Other Implementation Methods]

[0146] The above-described implementation method is merely one approach and can be arbitrarily modified and applied.

[0147] In the above embodiments, vehicle 1, which is a four-wheeled automobile, is exemplified as a "vehicle," but as long as it is a "vehicle" with a windshield 3, the number of wheels is not limited to four.

[0148] In the above embodiments, the "first mode" is illustrated as a 1 / f fluctuation, the "second mode" as a mode with minimal acceleration, and the "third mode" as a mode of linear acceleration and deceleration. However, the "first mode" is not limited to a 1 / f fluctuation; any mode other than the "second mode" and the "third mode" is acceptable. However, it is preferable that the "first mode" is one that makes the eye-guided display VI more noticeable compared to the "second mode" and the "third mode." Furthermore, the "second mode" is not limited to a mode with minimal acceleration; any mode other than the "first mode" and the "third mode" is acceptable. However, it is preferable that the "second mode" is one that makes the eye-guided display VI more noticeable compared to at least the "third mode." Furthermore, the "third mode" is not limited to a mode of linear acceleration and deceleration; any mode other than the "first mode" and the "second mode" is acceptable. However, if the "first mode" and the "second mode" are biological motion modes, then the "third mode" is preferably not a biological motion mode.

[0149] In the above embodiment, the structure of displaying the line-of-sight display VI on the windshield 3 is achieved through HUD5. However, the unit that displays the line-of-sight display VI can be any unit that displays a virtual image on the windshield 3, and is not limited to HUD5. For example, it can also be a display unit such as LED (Light Emitting Diode).

[0150] The processor 100 can be composed of multiple processors or a single processor. The processor 100 can also be hardware programmed to implement the aforementioned functionalities. In this case, the processor 100 may be, for example, composed of an ASIC (Application Specific Integrated Circuit) or a FPGA (Field Programmable Gate Array).

[0151] in addition, Figure 2 The structure of each part of the vehicle 1 shown is an example, and the specific installation method is not particularly limited. That is, it is not necessarily necessary to install hardware corresponding to each part separately; of course, it is also possible to configure the vehicle so that the functions of each part are implemented by a processor executing a program. In addition, in the above-described embodiment, a part of the function implemented by software can be implemented as hardware, or a part of the function implemented by hardware can be implemented by software.

[0152] in addition, Figure 12 The steps shown are divided according to the main processing content, and this invention is not limited to the method or name of the processing unit division. The action can also be divided into more step units based on the processing content. Alternatively, it can be divided so that one step unit contains more processing steps. Furthermore, the order of these steps can be appropriately changed without affecting the spirit of this invention.

[0153] Furthermore, when implementing the gaze guidance method based on the gaze guidance device 7 using the processor 100, the program executed by the processor 100 can also be configured as a recording medium or a transmission medium for transmitting the program. That is, the control program 111 can also be implemented by recording the control program 111 on a removable information recording medium. Examples of information recording media include magnetic recording media such as hard disks, optical recording media such as CDs, USB (Universal Serial Bus) memory, SSD (Solid State Drive) and other semiconductor storage devices, but other recording media can also be used.

[0154] [4. Structures supported by the above embodiments]

[0155] The above implementation supports the following structures.

[0156] (Structure 1)

[0157] A gaze guidance device for guiding the driver's gaze in a vehicle, comprising: a detection unit for detecting an object present in front of the vehicle; and a display control unit for displaying a gaze guidance display on the windshield of the vehicle that guides the driver's gaze toward the object when the detection unit detects the object, wherein the display control unit displays the gaze guidance display such that the gaze guidance display moves from a display start position toward the object at a speed corresponding to the type of object.

[0158] According to the gaze guidance device of Structure 1, the movement speed of the gaze guidance display can vary depending on the type of object. Therefore, the driver can not only grasp the movement result of the gaze guidance display, but also determine which object to guide their gaze based on the changes in the movement speed of the gaze guidance display. Thus, the driver can easily determine which object to guide their gaze to. Furthermore, the driver can determine which object to guide their gaze to based on the changes in the movement speed of the gaze guidance display, allowing the driver to quickly determine which object to guide their gaze to.

[0159] (Structure 2)

[0160] According to the line-of-sight guidance device of structure 1, when the object is a pedestrian, the display control unit displays the line-of-sight guidance display according to the change in the movement speed in a first manner.

[0161] According to the line-of-sight guidance device in structure 2, the line-of-sight guidance display can be moved according to the change in movement speed corresponding to the situation where the object is a pedestrian. Therefore, the driver can easily grasp the direction to guide the line of sight towards the pedestrian.

[0162] (Structure 3)

[0163] According to the line-of-sight guidance device of structure 2, the first mode is 1 / f fluctuation.

[0164] According to the gaze guidance device in structure 3, the gaze guidance display can be moved in a manner that is easily noticed by the driver. Therefore, the driver can easily grasp the situation where the driver's gaze needs to be directed towards the pedestrian, and the gaze guidance display can increase the likelihood of the driver directing their gaze towards the pedestrian.

[0165] (Structure 4)

[0166] According to any one of structures 1 to 3, in the case that the object is any one of a passenger car, an automatic two-wheeled vehicle, and a bicycle, the display control unit displays the line-of-sight display according to the change in the movement speed in the second manner.

[0167] According to the line-of-sight guidance device in structure 4, the line-of-sight guidance display can be moved according to the change in movement speed corresponding to any of the objects being a passenger car, an autonomous two-wheeler, or a bicycle. Therefore, the driver can easily determine when to guide the line of sight to any of the objects being a passenger car, an autonomous two-wheeler, or a bicycle.

[0168] (Structure 5)

[0169] According to the line-of-sight guidance device of structure 4, the second method is the method with the minimum acceleration.

[0170] According to the gaze guidance device in structure 5, the gaze guidance display can be moved in a manner that is easily noticed by the driver. Therefore, the driver can easily grasp the situation of guiding their gaze to any object, such as a passenger car, an autonomous two-wheeler, or a bicycle, and the gaze guidance display can increase the likelihood of the driver directing their gaze toward the object.

[0171] (Structure 6)

[0172] According to the line-of-sight guidance devices of structures 1 to 5, when the object is a trailer or a fixed object, the display control unit displays the line-of-sight guidance display according to the change in the third-party movement speed.

[0173] According to the line-of-sight guidance device in structure 6, the line-of-sight guidance display can be moved according to the change in movement speed corresponding to whether the object is a trailer or a fixed object. Therefore, the driver can easily determine whether to guide the line of sight to the trailer or the fixed object.

[0174] (Structure 7)

[0175] According to the line-of-sight guidance device of structure 6, the third method is to perform linear deceleration after linear acceleration.

[0176] According to the gaze guidance device of structure 7, by displaying the gaze guidance display in a manner that is not biologically driven, the driver is less likely to pay attention to the gaze guidance display. Therefore, it is possible to suppress the driver's attention to the gaze guidance display when guiding the driver towards objects that are of low necessity to be looked at.

[0177] (Structure 8)

[0178] According to any one of structures 1 to 7, the display control unit changes the movement speed of the gaze guidance display when the distance the gaze guidance display moves is greater than or equal to a predetermined distance.

[0179] According to the line-of-sight guidance device of structure 8, since the line-of-sight guidance display changes speed when the speed of movement is easily controlled, the driver can accurately and easily determine which object to guide the line of sight to.

[0180] (Structure 9)

[0181] According to the line-of-sight guidance device of structure 8, the display control unit moves the line-of-sight guidance display at a constant speed when the distance the line-of-sight guidance display moves is less than a predetermined distance.

[0182] According to the eye guidance device of structure 9, it is possible to suppress the confusion and distress to the driver when it is difficult to grasp the way the eye guidance display changes its speed.

[0183] (Structure 10)

[0184] A gaze guidance method for guiding the driver's gaze in a vehicle, comprising: a first step of detecting an object present in front of the vehicle; and a second step of, if the object is detected in the first step, displaying a gaze guidance display on the windshield of the vehicle to guide the driver's gaze toward the object, wherein the gaze guidance display is displayed such that it moves from a display start position toward the object according to a change in movement speed corresponding to the type of the object.

[0185] The line-of-sight guidance method of structure 10 will achieve the same effect as the line-of-sight guidance device of structure 1.

[0186] (Structure 11)

[0187] A computer program product wherein a processor of a line-of-sight guidance device for guiding the driver's line of sight in a vehicle functions as a detection unit and a display control unit. The detection unit detects an object present in front of the vehicle, and the display control unit, upon detection of the object, displays a line-of-sight guidance display on the windshield of the vehicle that guides the driver's line of sight toward the object. The display control unit displays the line-of-sight guidance display such that it moves from its initial display position toward the object at a speed corresponding to the type of object.

[0188] The computer program product of structure 11 will achieve the same effect as the line-of-sight guidance device of structure 1.

Claims

1. A sight line guiding device that guides a sight line of a driver of a vehicle, wherein the sight line guiding device comprises: a detection section that detects an object present in front of the vehicle; and a display control section that displays, on a windshield of the vehicle, a sight line guiding display that guides the sight line of the driver toward the object, in a case where the object is detected by the detection section, the display control section displays the sight line guiding display in such a manner that the sight line guiding display moves from a display start position of the sight line guiding display toward the object at a variation of a moving speed corresponding to a kind of the object.

2. The sight line guiding device according to claim 1, wherein in a case where the kind of the object is a pedestrian, the display control section displays the sight line guiding display at a variation of a moving speed of a first manner.

3. The sight line guiding device according to claim 2, wherein the first manner is a 1 / f fluctuation.

4. The sight line guiding device according to claim 1, wherein in a case where the kind of the object is any of a passenger car, an automatic two-wheeled vehicle, and a bicycle, the display control section displays the sight line guiding display at a variation of a moving speed of a second manner.

5. The sight line guiding device according to claim 4, wherein the second manner is a manner in which jerk is minimum.

6. The sight line guiding device according to claim 1, wherein in a case where the kind of the object is a trailer or a stationary object, the display control section displays the sight line guiding display at a variation of a moving speed of a third manner.

7. The sight line guiding device according to claim 6, wherein the third manner is a manner in which linear deceleration is performed after linear acceleration is performed.

8. The sight line guiding device according to claim 1, wherein the display control section varies the moving speed of the sight line guiding display in a case where a distance of movement of the sight line guiding display is equal to or greater than a prescribed distance.

9. The sight line guiding device according to claim 8, wherein the display control section moves the sight line guiding display at a constant speed in a case where the distance of movement of the sight line guiding display is less than the prescribed distance.

10. A sight line guiding method that guides a sight line of a driver of a vehicle, wherein the sight line guiding method comprises: a first step of detecting an object present in front of the vehicle; and a second step of displaying, on a windshield of the vehicle, a sight line guiding display that guides the sight line of the driver toward the object, in a case where the object is detected in the first step, in the second step, the sight line guiding display is displayed in such a manner that the sight line guiding display moves from a display start position of the sight line guiding display toward the object at a variation of a moving speed corresponding to a kind of the object.

11. A computer program product that comprises a program that causes a processor of a sight line guiding device that guides a sight line of a driver of a vehicle to function as: ​ ​ a detection unit that detects an object present in front of the vehicle; and a display control unit that displays, when the detection unit detects the object, a line-of-sight guide display that guides a line of sight of the driver toward the object on a windshield of the vehicle, the display control unit displays the line-of-sight guide display in such a manner that the line-of-sight guide display moves from a display start position of the line-of-sight guide display toward the object at a change in moving speed corresponding to a kind of the object.

Images