Head-up display system

Abstract

An AR-HUD system (1) is provided with a display unit (42), a line-of-sight detection unit, and a control unit. An object detection sensor (10) detects a plurality of objects (OJ) included in a scenery ahead of a vehicle. The line-of-sight detection unit detects a line-of-sight (En) of a driver of the vehicle. A distance output device (30) determines (measures) an actual distance (D1) from an eye point (EP) of a driver for a specific object (OJ), which is located in front of a sight line (En) of the driver detected by a sight line detection unit and coincides with a focal point of the driver, among a plurality of objects (OJ) detected by an object detection sensor (10). The control unit controls the display unit (42) so that the virtual image (S) and the specific object (OJ) are displayed superimposed on each other, and performs ghosting prevention processing in which a virtual image display distance (D2) from the eye point (EP) of the driver to the virtual image (S) matches the actual distance (D1).

Description

Technical Field

[0001] The present invention relates to a head-up display system. Background Art

[0002] Conventional head-up display systems, such as those described in Patent Document 1, illuminate display light representing a display image toward a transmissive reflective portion of a vehicle, allowing an observer to visually perceive a virtual image of the display image. This head-up display includes a display that displays the display image; the display is controlled so that, for at least a certain period of time until the vehicle starts driving, a highlight image is displayed superimposed on an object in the scenery ahead of the vehicle. Thus, by demonstrating the superimposed display of the highlight image before the vehicle begins driving, the head-up display can help the observer recognize the existence and significance of the function of superimposing the highlight image on the object.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2019-038451 Summary of the Invention

[0006] Technical problem that the invention aims to solve

[0007] However, when the head-up display described in Patent Document 1 displays a highlight image superimposed on an object, for example, if the object is focused on, the highlight image may be seen as a double image due to crowding.

[0008] Therefore, the present invention has been made in view of the above circumstances, and an object of the present invention is to provide a head-up display system capable of appropriately performing overlapping display.

[0009] Technical means to solve the problem

[0010] To solve the above-mentioned problems and achieve the purpose, a head-up display system according to the present invention is characterized by comprising: a display unit for irradiating display light including a display image toward a transmissive reflective member provided on a vehicle to display a virtual image; an object detection unit for detecting an object included in scenery in front of the vehicle; a line of sight detection unit for detecting the line of sight of a driver of the vehicle; a distance measurement unit for measuring, for a specific object among the objects detected by the object detection unit that is located in front of the driver's line of sight detected by the line of sight detection unit and is in focus of the driver, an actual distance between the specific object and an eye point of the driver; and a control unit for controlling the display unit to display the virtual image superimposed on the specific object, and controlling the display unit to perform anti-ghosting processing to make the virtual image display distance from the driver's eye point to the virtual image consistent with the actual distance measured by the distance measurement unit.

[0011] Effects of the Invention

[0012] The head-up display system according to the present invention can prevent a virtual image from appearing double due to crowding when a driver focuses on a specific object. As a result, the head-up display system can appropriately perform overlapping display. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a schematic diagram showing a configuration example of an AR-HUD system according to an embodiment.

[0014] Figure 2 This is a block diagram showing a configuration example of an AR-HUD system according to an embodiment.

[0015] Figure 3 It is a schematic diagram showing a configuration example of a display according to an embodiment.

[0016] Figure 4 It is a schematic diagram showing a configuration example of a display according to an embodiment.

[0017] Figure 5 This is a diagram showing the relationship between the parallax between the left-eye image and the right-eye image and the virtual image display position (Part 1) according to the embodiment.

[0018] Figure 6 This is a diagram showing the relationship between the parallax between the left-eye image and the right-eye image and the virtual image display position according to the embodiment (Part 2).

[0019] Figure 7 This is a diagram showing the relationship between the parallax between the left-eye image and the right-eye image and the virtual image display position according to the embodiment (Part 3).

[0020] Figure 8 This is a flowchart showing an example of the operation of the AR-HUD system involved in the embodiment.

[0021] Explanation of symbols

[0022] 1AR-HUD system (head-up display system)

[0023] 10 Object detection sensor (object detection unit, distance measurement unit)

[0024] 21 Eyesight Detection Unit

[0025] 30 distance output device (distance measuring unit)

[0026] 42 Display unit

[0027] 43 Control Department

[0028] D1 Actual distance

[0029] D2 virtual image display distance

[0030] En Sight

[0031] EP Eye Spot

[0032] P Display image

[0033] LE Left Eye

[0034] RE right eye

[0035] LP Left eye image

[0036] RP Right Eye Image

[0037] OJ object (object)

[0038] S virtual image

[0039] W windshield (reflective parts) DETAILED DESCRIPTION

[0040] The modes (embodiments) for implementing the present invention will be described in detail with reference to the accompanying drawings. The present invention is not limited to the contents described in the following embodiments. In addition, the constituent elements described below include elements that can be easily thought of by those skilled in the art, and substantially the same elements. Moreover, the structures described below can be appropriately combined. In addition, various omissions, replacements or changes in the structure can be made within the scope of the present invention.

[0041] [Implementation Method]

[0042] An AR-HUD system 1 according to an embodiment will be described with reference to the accompanying drawings. The AR-HUD system 1 is an example of a head-up display system, installed in a vehicle. It irradiates a windshield W, a transmissive reflective member, with display light containing a display image P. This causes a virtual image S, reflected from the windshield W toward the eye point EP, to be displayed superimposed on an object OJ. The eye point EP is either a pre-conceived position of the driver's eyes or the actual position of the driver's eyes. For example, if the eye point EP is the actual position of the driver's eyes, the driver's eye position is detected by the driver monitor 20, described later.

[0043] like Figure 1 As shown, the AR-HUD system 1 includes an object detection sensor 10, a driver monitor 20, a distance output device 30, and an AR-HUD device 40. The object detection sensor 10, the driver monitor 20, the distance output device 30, and the AR-HUD device 40 are connected to each other so as to be able to communicate with each other.

[0044] The object detection sensor 10 is installed in the vehicle and detects objects OJ contained in the scenery ahead of the vehicle. Objects OJ are, for example, people, other vehicles, signs, and other objects that the driver needs to recognize. The object detection sensor 10 is configured, for example, to include a stereo camera that captures the scenery ahead of the vehicle. The object detection sensor 10 then analyzes the captured images using known image processing methods such as pattern matching to detect multiple objects OJ, such as people, other vehicles, and signs. Furthermore, the object detection sensor 10 measures the actual distance D1 from the driver's eye point EP to each detected object OJ. Specifically, the object detection sensor 10 measures the actual distance D1, which is the distance from the driver's eye point EP to the object OJ. The object detection sensor 10 outputs object detection information indicating the position (XY coordinates) of the detected object OJ, the size of the object OJ, and the actual distance D1 from the driver's eye point EP to the object OJ, to the distance output device 30.

[0045] Next, the driver monitor 20 will be described. The driver monitor 20 is installed in the vehicle and monitors the driver, and is configured to include a line of sight detection unit 21. The line of sight detection unit 21 detects the driver's line of sight En. The line of sight detection unit 21 is configured, for example, with the camera lens facing the driver. The line of sight detection unit 21 detects the driver's line of sight En by a well-known line of sight detection method. The line of sight detection unit 21 detects the driver's line of sight En based on, for example, the pupil position of the eyeball in the driver's facial image. In this case, the line of sight detection unit 21 compares a predetermined eye image with the driver's facial image and detects the driver's pupil position from the driver's facial image. The line of sight detection unit 21 detects the driver's line of sight En based on the detected pupil position of the driver. The line of sight detection unit 21 outputs line of sight information (XY coordinates) representing the detected line of sight En to the distance output device 30.

[0046] Next, the distance output device 30 will be described. The distance output device 30 outputs the actual distance D1 from the eye point EP to the object OJ. The distance output device 30 outputs the actual distance D1 from the eye point EP to the object OJ based on the object detection information output by the object detection sensor 10 and the driver's line of sight information output by the line of sight detection unit 21 of the driver monitor 20. The distance output device 30 detects a specific object OJ from among multiple objects OJ, for example, based on line of sight information (XY coordinates) representing the driver's line of sight En. Specifically, the distance output device 30 detects a specific object OJ that is located in front of the driver's line of sight En and coincides with the driver's focus from among the multiple objects OJ. That is, the distance output device 30 detects the specific object OJ that the driver is actually looking at from among the multiple objects OJ. In other words, the distance output device 30 detects a specific object OJ whose position (XY coordinates) coincides with the position (XY coordinates) of the driver's line of sight En. Then, the distance output device 30 outputs object information indicating the position (XY coordinates) of the specific object OJ and the actual distance D1 from the eye point EP to the specific object OJ to the AR-HUD device 40 .

[0047] Next, the AR-HUD device 40 will be described. The AR-HUD device 40 irradiates display light including a display image P toward the windshield W, causing a virtual image S reflected from the windshield W toward the eye point EP to be displayed superimposed on the object OJ. The AR-HUD device 40 includes a reflecting unit 41, a display unit 42, and a control unit 43. The reflecting unit 41, the display unit 42, and the control unit 43 are interconnected and communicative.

[0048] The reflecting unit 41 reflects the display light emitted from the display unit 42 toward the windshield W. The reflecting unit 41 includes a first intermediate reflector 411, a second intermediate reflector 412, and a final reflector 413. The first intermediate reflector 411 totally reflects the display light emitted from the display unit 42 toward the second intermediate reflector 412. The second intermediate reflector 412 totally reflects the display light emitted from the display unit 42 and reflected by the first intermediate reflector 411 toward the final reflector 413. The final reflector 413 totally reflects the display light emitted from the display unit 42 and reflected by the first intermediate reflector 411 and the second intermediate reflector 412 toward the windshield W.

[0049] The display unit 42 emits display light including the display image P, and emits the display light toward the windshield W via the reflecting unit 41. The display unit 42 is configured to include a display 421. For example, Figure 4 As shown, the display 421 is composed of a liquid crystal panel 421a and a cylindrical lens 421b. Figure 3 As shown, the liquid crystal panel 421a is constructed to include: a pixel column La for displaying a left-eye image LP and a pixel column Ra for displaying a right-eye image RP. The pixel column La and the pixel column Ra are alternately arranged for each pixel. The liquid crystal panel 421a is set on the back of the cylindrical lens 421b, and emits the display image P after synthesizing the left-eye image LP and the right-eye image RP to the cylindrical lens 421b. Here, the display image P is a three-dimensional image composed of the left-eye image LP for the driver's left eye LE to visually recognize and the right-eye image RP for the driver's right eye RE to visually recognize. As shown in FIG. Figure 4 As shown, the cylindrical lens 421b is a lens with countless tiny, elongated semi-cylindrical convex lenses formed on its surface. The cylindrical lens 421b reflects the display image P from the liquid crystal panel 421a, refracting the display light of the left-eye image LP within the display image P toward the driver's left eye LE, and refracting the display light of the right-eye image RP toward the driver's right eye RE. The display light containing the left-eye image LP emitted from the cylindrical lens 421b passes through the reflective portion 41 and the windshield W and enters the driver's left eye LE. The display light containing the right-eye image RP emitted from the cylindrical lens 421b passes through the reflective portion 41 and the windshield W and enters the driver's right eye RE. With the display light containing the left-eye image LP entering the left eye LE and the display light containing the right-eye image RP entering the right eye RE, the driver visually recognizes a three-dimensional virtual image S superimposed on the specific object OJ.

[0050] The control unit 43 controls the display unit 42 so that the virtual image S is displayed superimposed on the specific object OJ. At this time, based on the object information output from the distance output device 30, the control unit 43 performs anti-ghosting processing to align the virtual image display distance D2 from the eye point EP to the virtual image S with the actual distance D1 from the eye point EP to the specific object OJ. Typically, when performing this anti-ghosting processing, the control unit 43 aligns the virtual image display distance D2 from the eye point EP to the virtual image S with the actual distance D1 from the eye point EP to the specific object OJ. However, as long as ghosting caused by crowding can be prevented, the control unit 43 may not align the virtual image display distance D2 from the eye point EP to the virtual image S with the actual distance D1 from the eye point EP to the specific object OJ, or may slightly differ from the actual distance D1 to the virtual image display distance D2. The virtual image display distance D2 is the distance from the eye point EP to the display position of the virtual image S. In other words, the virtual image display distance D2 is the straight line distance connecting the eye point EP and the display position of the virtual image S. In other words, the virtual image display distance D2 is the distance from the eye point EP to the imaging position where the virtual image S is formed.

[0051] The control unit 43 controls the display unit 42 to perform anti-ghosting processing to make the virtual image display distance D2 consistent with the actual distance D1 by changing the parallax between the left-eye image LP and the right-eye image RP. In the anti-ghosting processing, the control unit 43 adjusts the inclination of the reflecting surface of each reflector of the reflecting unit 41, and changes the optical path of the display light including the left-eye image LP and the optical path of the display light including the right-eye image RP emitted from the cylindrical lens 421b, thereby changing the parallax between the left-eye image LP and the right-eye image RP. The control unit 43, for example, adjusts the inclination of the reflecting surface of each reflector of the reflecting unit 41, and changes the optical path of the display light including the left-eye image LP and the right-eye image RP emitted from the cylindrical lens 421b, thereby changing the parallax between the left-eye image LP and the right-eye image RP. Figure 5 As shown, by relatively increasing the parallax between the left-eye image LP and the right-eye image RP, the virtual image display distance D2 can be relatively lengthened. When the actual distance D1 from the eye point EP to the specific object OJ is relatively long, the control unit 43 increases the parallax between the left-eye image LP and the right-eye image RP so that the virtual image display distance D2 matches the actual distance D1, and displays the virtual image S superimposed on the specific object OJ.

[0052] In addition, if Figure 6 As shown, the control unit 43 can relatively shorten the virtual image display distance D2 by relatively reducing the parallax between the left-eye image LP and the right-eye image RP. When the actual distance D1 from the eye point EP to the specific object OJ is relatively short, the control unit 43 relatively reduces the parallax between the left-eye image LP and the right-eye image RP to make the virtual image display distance D2 consistent with the actual distance D1, thereby causing the virtual image S to be displayed superimposed on the specific object OJ.

[0053] In addition, if Figure 7As shown, the control unit 43 can further reduce the parallax between the left-eye image LP and the right-eye image RP, thereby further shortening the virtual image display distance D2. When the actual distance D1 from the eye point EP to the specific object OJ is relatively shorter, the control unit 43 further reduces the parallax between the left-eye image LP and the right-eye image RP, aligning the virtual image display distance D2 with the actual distance D1, and displays the virtual image S superimposed on the specific object OJ.

[0054] Next, refer to Figure 8 The following flowchart illustrates an operation example of the AR-HUD system 1. The AR-HUD system 1 detects the driver's line of sight En (step S1). The AR-HUD system 1 detects the driver's line of sight En based on the pupil position of the driver's eyeball in the driver's facial image, for example, using the line of sight detection unit 21.

[0055] Next, the AR-HUD system 1 determines whether the detected line of sight En of the driver is within the display range (step S2). The AR-HUD system 1 determines whether the driver's line of sight En is included in the display range of the virtual image S that is predetermined and displayed, for example, through the driver monitor 20. When the driver's line of sight En is within the display range (step S2; yes), the AR-HUD system 1 detects multiple objects OJ contained in the scenery in front of the vehicle (step S3). The AR-HUD system 1, for example, uses the object detection sensor 10 to perform image analysis on the captured image using well-known image processing methods such as pattern matching methods to detect multiple objects OJ such as people, other vehicles, and signs. Then, the object detection sensor 10 measures the actual distance D1 from the driver's eye point EP for the detected objects OJ such as people, other vehicles, and signs.

[0056] Next, the AR-HUD system 1 determines whether the driver's line of sight En coincides with the object OJ (step S4). The AR-HUD system 1 detects the specific object OJ from among the multiple objects OJ based on the line of sight information (XY coordinates) representing the driver's line of sight En, using, for example, the distance output device 30. If the driver's line of sight En coincides with the object OJ (step S4: Yes), the AR-HUD system 1 outputs the actual distance D1 from the driver's eye point EP to the corresponding specific object OJ (step S5).

[0057] Next, the AR-HUD system 1 matches the virtual image display distance D2 from the eye point EP to the virtual image S with the actual distance D1 from the eye point EP to the specific object OJ, and displays the virtual image S superimposed on the specific object OJ. The AR-HUD system 1 controls the display unit 42, for example, through the control unit 43, to change the parallax between the left-eye image LP and the right-eye image RP, thereby matching the virtual image display distance D2 with the actual distance D1, and displays the virtual image S superimposed on the specific object OJ (step S6).

[0058] Furthermore, in step S2 above, if the driver's line of sight En is detected to be outside the display range (step S2; No), the AR-HUD system 1 ends the process. In step S4 above, if the driver's line of sight En does not coincide with the object OJ (step S4; No), the AR-HUD system 1 ends the process.

[0059] As described above, the AR-HUD system 1 according to the embodiment includes a display unit 42, a line of sight detection unit 21, and a control unit 43. The display unit 42 irradiates display light including a display image P toward a translucent windshield W provided on the vehicle to display a virtual image S. The object detection sensor 10 detects multiple objects OJ included in the scenery in front of the vehicle. The line of sight detection unit 21 detects the line of sight En of the driver of the vehicle. The distance output device 30 determines (measures) the actual distance D1 from the driver's eye point EP for a specific object OJ, among the multiple objects OJ detected by the object detection sensor 10, that is located in front of the driver's line of sight En detected by the line of sight detection unit 21 and is in focus with the driver. The control unit 43 controls the display unit 42 to perform anti-ghosting processing so that the virtual image S is displayed superimposed on the specific object OJ and the virtual image display distance D2 from the driver's eye point EP to the virtual image S is consistent with the actual distance D1.

[0060] With this configuration, the AR-HUD system 1 can prevent the virtual image S from appearing double due to crowding when the driver focuses on the specific object OJ. In particular, when displaying the virtual image S representing emergency information, the AR-HUD system 1 can ensure that the driver clearly visually recognizes the virtual image S. In this way, the AR-HUD system 1 can appropriately display the virtual image S in an overlapping manner.

[0061] In the AR-HUD system 1 described above, the displayed image P is a three-dimensional image composed of a left-eye image LP, perceived by the driver's left eye LE, and a right-eye image RP, perceived by the driver's right eye RE. The control unit 43 controls the display unit 42 to perform anti-ghosting processing to align the virtual image display distance D2 with the actual distance D1 by changing the parallax between the left-eye image LP and the right-eye image RP. This configuration prevents the three-dimensional virtual image S from appearing double due to crowding when the driver focuses on a specific object OJ. As a result, the AR-HUD system 1 can appropriately display the three-dimensional virtual image S in a superimposed manner.

[0062] [Variation]

[0063] Next, a modification of the embodiment will be described. In addition, in the modification, the same reference numerals are used for the structural elements equivalent to those in the embodiment, and their detailed description is omitted. In the AR-HUD system 1, an example in which the display image P is a three-dimensional image is described, but this is not limited to this. For example, it can also be a two-dimensional image that is visually recognized by the driver's left eye LE and right eye RE. In this case, the control unit 43 controls the display unit 42 to change the optical path length of the display light, thereby performing anti-ghosting processing to make the virtual image display distance D2 consistent with the actual distance D1. For example, the display unit 42 has an optical path length adjustment unit that is configured to include a plurality of reflective mirrors. The optical path length adjustment unit adjusts (selects) the reflective mirror that reflects the display light based on the actual distance D1, thereby changing the optical path length of the display light to the windshield W, so that the virtual image display distance D2 of the virtual image S composed of the two-dimensional image is consistent with the actual distance D1. As a result, the AR-HUD system 1 involved in the modification can prevent the two-dimensional virtual image S from appearing as double due to crowding when the driver focuses on the specific object OJ. As a result, the AR-HUD system 1 according to the modification example can appropriately perform superimposed display of the two-dimensional virtual image S.

[0064] While the object detection sensor 10 has been described as including a stereo camera, the present invention is not limited thereto. Any sensor capable of detecting the object OJ and measuring the actual distance D1 may be used. For example, the object detection sensor 10 may include a known sensor such as a monocular camera, an infrared camera, a laser radar, a millimeter-wave radar, or an ultrasonic sensor.

Claims

1. A head-up display system, characterized in that: have: A display unit that irradiates a display light including a display image toward a reflective member that is provided in the vehicle and has transmissivity, thereby displaying a virtual image; an object detection unit configured to detect an object included in a scene in front of the vehicle; a sight line detection unit configured to detect a sight line of a driver of the vehicle; a distance measuring unit for measuring an actual distance between a specific object detected by the object detecting unit and an eye point of the driver, for a specific object that is located in front of the line of sight of the driver detected by the line of sight detecting unit and coincides with the focus of the driver; as well as a control unit, the control unit controlling the display unit to display the virtual image superimposed on the specific object, The control unit controls the display unit to execute anti-ghosting processing to make a virtual image display distance from the eye point of the driver to the virtual image coincide with the actual distance measured by the distance measuring unit.

2. The head-up display system according to claim 1, wherein: The display image is a three-dimensional image composed of a left-eye image to be visually recognized by the left eye of the driver and a right-eye image to be visually recognized by the right eye of the driver. The control unit controls the display unit to change the parallax between the left-eye image and the right-eye image to thereby execute the anti-ghosting process for making the virtual image display distance coincide with the actual distance.

3. The head-up display system according to claim 1, wherein: The display image is a two-dimensional image that is visually recognized by the left eye and the right eye of the driver. The control unit controls the display unit to change the optical path length of the display light to perform the anti-ghosting process so that the virtual image display distance coincides with the actual distance.

Images