Head-up display system
The head-up display system addresses double image issues by adjusting virtual image display distance and parallax to match the driver's focus, ensuring clear superimposed display without convergence artifacts.
Patent Information
- Application Number
- JP2022169695
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-10-24
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2042-10-24
AI Technical Summary
Existing head-up displays can cause a double image due to convergence when an emphasized image is displayed superimposed on a target object, leading to visual recognition issues.
A head-up display system that includes an object detection sensor, a line-of-sight detection unit, a distance measurement unit, and a control unit to adjust the virtual image display distance and parallax to match the actual distance to the driver's focus, preventing double images by controlling the display unit to superimpose the virtual image appropriately.
Prevents the virtual image from appearing as a double image due to convergence, ensuring clear and appropriate superimposed display of the virtual image, particularly for emergency information.
Smart Images

Figure 0007593981000001 
Figure 0007593981000002 
Figure 0007593981000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a head-up display system. [Background technology]
[0002] Conventionally, as a head-up display system, for example, Patent Document 1 describes a head-up display that irradiates a display light showing a display image on a transmissive reflector on a vehicle to allow a viewer to view a virtual image of the display image. This head-up display includes a display device that displays a display image, and controls the display device to display an emphasis image superimposed on an object included in the scenery in front of the vehicle for at least a certain period of time until the vehicle is started and starts traveling. In this way, the head-up display can make the viewer aware of the existence and significance of the function of superimposing an emphasis image on an object by demonstrating the superimposed display of the emphasis image before the vehicle starts traveling. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2019-038451 A Summary of the Invention [Problem to be solved by the invention]
[0004] Incidentally, in the head-up display described in the above-mentioned Patent Document 1, for example, when an emphasized image is displayed superimposed on an object, there is a risk that the emphasized image may be perceived as a double image due to convergence when focusing on the object.
[0005] Therefore, the present invention has been made in consideration of the above, and has an object to provide a head-up display system that can properly perform superimposed display. [Means for solving the problem]
[0006] In order to solve the above-mentioned problems and achieve the object, the head-up display system of the present invention comprises a display unit that displays a virtual image by irradiating display light including a display image toward a transparent reflective member provided in a vehicle, an object detection unit that detects objects included in the scenery in front of the vehicle, a gaze detection unit that detects the gaze of the driver of the vehicle, a distance measurement unit that measures the actual distance from the driver's eye point to a specific object that is located in the line of sight of the driver detected by the gaze detection unit and on which the driver's focus is coincident among the objects detected by the object detection unit, and a control unit that controls the display unit and displays the virtual image superimposed on the specific object, wherein the control unit controls the display unit and performs double image prevention processing to adjust the virtual image display distance from the driver's eye point to the virtual image to the actual distance measured by the distance measurement unit. Effect of the Invention
[0007] The head-up display system according to the present invention can prevent a virtual image from appearing as a double image due to convergence when a driver focuses on a specific object. As a result, the head-up display system can perform a superimposed display appropriately. [Brief description of the drawings]
[0008] [Figure 1] FIG. 1 is a schematic diagram illustrating a configuration example of an AR-HUD system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram illustrating an example of the configuration of the AR-HUD system according to the embodiment. [Diagram 3] FIG. 3 is a schematic diagram illustrating a configuration example of a display according to the embodiment. [Figure 4] FIG. 4 is a schematic diagram showing a configuration example of a display according to the embodiment. [Diagram 5] FIG. 5 is a diagram showing a relationship (part 1) between the parallax of a left eye image and a right eye image and a virtual image display position according to the embodiment. [Figure 6]FIG. 6 is a diagram showing a relationship (part 2) between the parallax of the left eye image and the right eye image and the virtual image display position according to the embodiment. [Figure 7] FIG. 7 is a diagram showing a relationship (part 3) between the parallax of the left eye image and the right eye image and the virtual image display position according to the embodiment. [Figure 8] FIG. 8 is a flowchart illustrating an operation example of the AR-HUD system according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] The form (embodiment) for carrying out the present invention will be described in detail with reference to the drawings. The present invention is not limited to the contents described in the following embodiment. Furthermore, the components described below include those that a person skilled in the art can easily imagine and those that are substantially the same. Furthermore, the configurations described below can be appropriately combined. Furthermore, various omissions, substitutions, or modifications of the configurations can be made without departing from the gist of the present invention.
[0010] [Embodiment] An AR-HUD system 1 according to an embodiment will be described with reference to the drawings. The AR-HUD system 1 is an example of a head-up display system, which is provided in a vehicle, irradiates display light including a display image P toward a windshield W as a reflective member having transparency, and displays a virtual image S reflected by the windshield W toward an eye point EP by superimposing it on an object OJ. Here, the eye point EP is a position assumed in advance as the position of the driver's eyes, or the actual position of the driver's eyes. When the eye point EP is the actual position of the driver's eyes, for example, the position of the driver's eyes is detected by a driver monitor 20 described later.
[0011] 1, the AR-HUD system 1 includes an object detection sensor 10, a driver monitor 20, a distance output device 30, and an AR-HUD device 40. The object detection sensor 10, the driver monitor 20, the distance output device 30, and the AR-HUD device 40 are connected to each other so as to be able to communicate with each other.
[0012] The object detection sensor 10 is provided in a vehicle and detects an object OJ included in a scene in front of the vehicle. Here, the object OJ is, for example, a person, another vehicle, a sign, or other object that the driver is to recognize. The object detection sensor 10 is configured to include, for example, a stereo camera, and captures an image of a scene in front of the vehicle. The object detection sensor 10 then performs image analysis on the captured image using a known image processing method such as a pattern matching method, and detects a plurality of objects OJ, such as a person, another vehicle, and a sign. Furthermore, the object detection sensor 10 measures an actual distance D1 from the driver's eye point EP to the detected object OJ, such as a person, another vehicle, or a sign. That is, the object detection sensor 10 measures an actual distance D1, which is a distance from the driver's eye point EP to the object OJ. The object detection sensor 10 outputs object detection information representing the position (XY coordinates) of the detected object OJ, the size of the object OJ, and the actual distance D1 from the driver's eye point EP to the object OJ, to the distance output device 30.
[0013] Next, the driver monitor 20 will be described. The driver monitor 20 is provided in a vehicle and monitors the driver, and includes a gaze detection unit 21. The gaze detection unit 21 detects the driver's gaze En. The gaze detection unit 21 is disposed, for example, with a camera lens facing the driver. The gaze detection unit 21 detects the driver's gaze En by a well-known gaze detection method. The gaze detection unit 21 detects the driver's gaze En based on, for example, the position of the pupil of the eyeball in the face image of the driver. In this case, the gaze detection unit 21 compares a predetermined eye image with the face image of the driver, and detects the position of the driver's pupil from the face image of the driver. The gaze detection unit 21 detects the driver's gaze En from the detected position of the driver's pupil. The gaze detection unit 21 outputs gaze information (XY coordinates) representing the detected gaze En to the distance output device 30.
[0014] Next, the distance output device 30 will be described. The distance output device 30 outputs the actual distance D1 from the eye point EP to the object OJ. The distance output device 30 outputs the actual distance D1 from the eye point EP to the object OJ based on the object detection information output from the object detection sensor 10 and the driver's gaze information output from the gaze detection unit 21 of the driver monitor 20. The distance output device 30 detects a specific object OJ from among the multiple objects OJ based on, for example, gaze information (XY coordinates) representing the driver's gaze En. Specifically, the distance output device 30 detects a specific object OJ that is located ahead of the driver's gaze En and that the focus of the driver coincides with, among the multiple objects OJ. That is, the distance output device 30 detects a specific object OJ that is actually visually recognized by the driver, among the multiple objects OJ. In other words, the distance output device 30 detects a specific object OJ at a position (XY coordinates) that coincides with the position (XY coordinates) of the driver's gaze En. Then, the distance output device 30 outputs to the AR-HUD device 40 object information indicating the position (XY coordinates) of the specific object OJ and the actual distance D1 from the eyepoint EP to the specific object OJ.
[0015] Next, the AR-HUD device 40 will be described. The AR-HUD device 40 irradiates display light including a display image P toward the windshield W, and displays a virtual image S reflected by the windshield W toward the eye point EP by superimposing it on an object OJ. The AR-HUD device 40 includes a reflector 41, a display 42, and a controller 43. The reflector 41, the display 42, and the controller 43 are connected to each other so as to be able to communicate with each other.
[0016] The reflecting unit 41 reflects the display light emitted from the display unit 42 toward the windshield W. The reflecting unit 41 includes a first intermediate mirror 411, a second intermediate mirror 412, and a final mirror 413. The first intermediate mirror 411 totally reflects the display light emitted from the display unit 42 toward the second intermediate mirror 412. The second intermediate mirror 412 totally reflects the display light emitted from the display unit 42 and reflected by the first intermediate mirror 411 toward the final mirror 413. The final mirror 413 totally reflects the display light emitted from the display unit 42 and reflected by the first intermediate mirror 411 and the second intermediate mirror 412 toward the windshield W.
[0017] The display unit 42 emits display light including a display image P, and emits the display light to the windshield W via the reflector 41. The display unit 42 includes a display 421. The display 421 includes, for example, a liquid crystal panel 421a and a lenticular lens 421b as shown in FIG. 4. The liquid crystal panel 421a includes a pixel row La for displaying a left-eye image LP and a pixel row Ra for displaying a right-eye image RP as shown in FIG. 3. The pixel rows La and Ra are arranged alternately for each pixel. The liquid crystal panel 421a is provided on the back surface of the lenticular lens 421b, and emits a display image P, which is a combination of the left-eye image LP and the right-eye image RP, to the lenticular lens 421b. Here, the display image P is a three-dimensional image composed of a left-eye image LP to be viewed by the driver's left eye LE and a right-eye image RP to be viewed by the driver's right eye RE. As shown in Fig. 4, the lenticular lens 421b is a lens having a surface on which a countless number of fine, elongated, semi-cylindrical convex lenses are formed. The lenticular lens 421b refracts and emits the display light of the left-eye image LP in the display image P toward the driver's left eye LE, and refracts and emits the display light of the right-eye image RP toward the driver's right eye RE. The display light including the left-eye image LP emitted from the lenticular lens 421b is incident on the driver's left eye LE via the reflector 41 and the windshield W. The display light including the right-eye image RP emitted from the lenticular lens 421b is incident on the driver's right eye RE via the reflector 41 and the windshield W. The driver views a three-dimensional virtual image S superimposed on a specific object OJ as display light including an image LP for the left eye is incident on the left eye LE and display light including an image RP for the right eye is incident on the right eye RE.
[0018] The control unit 43 controls the display unit 42 to display the virtual image S superimposed on the specific object OJ. At this time, the control unit 43 executes a double image prevention process to match the virtual image display distance D2 from the eyepoint EP to the virtual image S to the real distance D1 from the eyepoint EP to the specific object OJ based on the object information output from the distance output device 30. Typically, when executing this double image prevention process, the control unit 43 matches the virtual image display distance D2 from the eyepoint EP to the virtual image S to the real distance D1 from the eyepoint EP to the specific object OJ. Note that, as long as the control unit 43 can prevent double images due to convergence, it is not necessary to match the virtual image display distance D2 from the eyepoint EP to the virtual image S to the real distance D1 from the eyepoint EP to the specific object OJ, and the real distance D1 and the virtual image display distance D2 may be slightly different. The above-mentioned virtual image display distance D2 is the distance from the eyepoint EP to the display position where the virtual image S is displayed. That is, the virtual image display distance D2 is the straight-line distance connecting the eye point EP and the position where the virtual image S is displayed. In other words, the virtual image display distance D2 is the distance from the eye point EP to the imaging position where the virtual image S is formed.
[0019] The control unit 43 controls the display unit 42 to change the parallax between the left eye image LP and the right eye image RP, thereby executing a double image prevention process to adjust the virtual image display distance D2 to the actual distance D1. In the double image prevention process, the control unit 43 changes the parallax between the left eye image LP and the right eye image RP by, for example, adjusting the inclination of the reflecting surface of each mirror of the reflecting unit 41 and changing the optical path of the display light including the left eye image LP and the display light including the right eye image RP emitted from the lenticular lens 421b. For example, as shown in FIG. 5, the control unit 43 can relatively increase the virtual image display distance D2 by relatively increasing the parallax between the left eye image LP and the right eye image RP. When the actual distance D1 from the eye point EP to the specific object OJ is relatively long, the control unit 43 relatively increases the parallax between the left eye image LP and the right eye image RP to match the virtual image display distance D2 to the actual distance D1, and displays the virtual image S superimposed on the specific object OJ.
[0020] Furthermore, the control unit 43 can relatively shorten the virtual image display distance D2 by relatively reducing the parallax between the left eye image LP and the right eye image RP as shown in Fig. 6. When the actual distance D1 from the eye point EP to the specific object OJ is relatively short, the control unit 43 relatively reduces the parallax between the left eye image LP and the right eye image RP to match the virtual image display distance D2 to the actual distance D1, and displays the virtual image S superimposed on the specific object OJ.
[0021] 7, the control unit 43 can relatively further shorten the virtual image display distance D2 by relatively further reducing the parallax between the left eye image LP and the right eye image RP. When the real distance D1 from the eye point EP to the specific object OJ is relatively shorter, the control unit 43 relatively further reduces the parallax between the left eye image LP and the right eye image RP to match the virtual image display distance D2 to the real distance D1, and displays the virtual image S superimposed on the specific object OJ.
[0022] Next, an example of the operation of the AR-HUD system 1 will be described with reference to the flowchart in Fig. 8. The AR-HUD system 1 detects the line of sight En of the driver (step S1). For example, the AR-HUD system 1 detects the line of sight En of the driver by the line of sight detection unit 21 based on the positions of the pupils of the eyes of the driver's face image.
[0023] Next, the AR-HUD system 1 judges whether the detected line of sight En of the driver is within the display range (step S2). For example, the AR-HUD system 1 judges whether the line of sight En of the driver is included within a display range in which a virtual image S is displayed, which is determined in advance by the driver monitor 20. If the line of sight En of the driver is within the display range (step S2; Yes), the AR-HUD system 1 detects a plurality of objects OJ included in the scenery ahead of the vehicle (step S3). For example, the AR-HUD system 1 performs image analysis of the captured image by the object detection sensor 10 using a known image processing method such as a pattern matching method, and detects a plurality of objects OJ such as people, other vehicles, and signs. Then, the object detection sensor 10 measures the actual distance D1 from the driver's eye point EP to the detected object OJ such as a person, other vehicle, or sign.
[0024] Next, the AR-HUD system 1 determines whether the driver's line of sight En matches the object OJ (step S4). The AR-HUD system 1 detects a specific object OJ from among the multiple objects OJ based on line of sight information (XY coordinates) representing the driver's line of sight En, for example, by the distance output device 30. If the driver's line of sight En matches the object OJ (step S4; Yes), the AR-HUD system 1 outputs an actual distance D1 from the driver's eye point EP to the matching specific object OJ (step S5).
[0025] Next, the AR-HUD system 1 adjusts the virtual image display distance D2 from the eye point EP to the virtual image S to the actual distance D1 from the eye point EP to the specific object OJ, and displays the virtual image S superimposed on the specific object OJ. (Step S6) In the AR-HUD system 1, for example, the control unit 43 controls the display unit 42 to change the parallax between the left eye image LP and the right eye image RP, thereby adjusting the virtual image display distance D2 to the actual distance D1 and displaying the virtual image S superimposed on the specific object OJ.
[0026] In addition, in the above-mentioned step S2, if the detected line of sight En of the driver is not within the display range (step S2; No), the AR-HUD system 1 ends the process. In the above-mentioned step S4, if the line of sight En of the driver does not match the object OJ (step S4; No), the AR-HUD system 1 ends the process.
[0027] As described above, the AR-HUD system 1 according to the embodiment includes the display unit 42, the line-of-sight detection unit 21, and the control unit 43. The display unit 42 irradiates the display light including the display image P toward the windshield W provided in the vehicle and having transparency to display the virtual image S. The object detection sensor 10 detects a plurality of objects OJ included in the scenery ahead of the vehicle. The line-of-sight detection unit 21 detects the line of sight En of the driver of the vehicle. The distance output device 30 specifies (measures) an actual distance D1 from the driver's eye point EP to a specific object OJ that is located ahead of the line of sight En of the driver detected by the line-of-sight detection unit 21 and on which the focus of the driver coincides, among the plurality of objects OJ detected by the object detection sensor 10. The control unit 43 controls the display unit 42 to display the virtual image S superimposed on the specific object OJ, and executes a double image prevention process to adjust a virtual image display distance D2 from the driver's eye point EP to the virtual image S to the actual distance D1.
[0028] With this configuration, the AR-HUD system 1 can prevent the virtual image S from appearing as a double image due to convergence when the driver focuses on a specific object OJ. In particular, when displaying a virtual image S representing emergency information, the AR-HUD system 1 can allow the driver to clearly view the virtual image S. In this way, the AR-HUD system 1 can properly perform the superimposed display of the virtual image S.
[0029] In the above AR-HUD system 1, the display image P is a three-dimensional image composed of a left-eye image LP to be viewed by the driver's left eye LE and a right-eye image RP to be viewed by the driver's right eye RE. The control unit 43 controls the display unit 42 to change the parallax between the left-eye image LP and the right-eye image RP, thereby executing a double-image prevention process to match the virtual image display distance D2 to the actual distance D1. With this configuration, the AR-HUD system 1 can prevent the three-dimensional virtual image S from appearing as a double image due to convergence when the driver focuses on a specific object OJ. As a result, the AR-HUD system 1 can properly perform the superimposed display of the three-dimensional virtual image S.
[0030] [Modifications] Next, a modified example of the embodiment will be described. In the modified example, the same reference numerals are given to components equivalent to those in the embodiment, and detailed description thereof will be omitted. In the AR-HUD system 1, the display image P is described as a three-dimensional image, but the present invention is not limited thereto, and may be a two-dimensional image to be viewed by the driver's left eye LE and right eye RE. In this case, the control unit 43 controls the display unit 42 to change the optical path length of the display light, thereby executing a double image prevention process to adjust the virtual image display distance D2 to the actual distance D1. For example, the display unit 42 includes an optical path length adjustment unit including a plurality of folding mirrors. The optical path length adjustment unit adjusts (selects) the folding mirror that reflects the display light based on the actual distance D1, thereby changing the optical path length of the display light to the windshield W, and adjusts the virtual image display distance D2, which displays the virtual image S consisting of a two-dimensional image, to the actual distance D1. As a result, the AR-HUD system 1 according to the modified example can prevent the two-dimensional virtual image S from appearing as a double image due to convergence when the driver focuses on a specific object OJ. As a result, the AR-HUD system 1 according to the modified example can appropriately perform superimposed display of the two-dimensional virtual image S.
[0031] Although the object detection sensor 10 has been described as including a stereo camera, the present invention is not limited thereto and may be any sensor capable of detecting the object OJ and measuring the actual distance D1. The object detection sensor 10 may be configured to include any known sensor, such as a monocular camera, an infrared camera, a laser radar, a millimeter wave radar, or an ultrasonic sensor. [Explanation of symbols]
[0032] 1 AR-HUD system (head-up display system) 10 Object detection sensor (object detection part, distance measurement part) 21 Gaze detection unit 30 Distance output device (distance measurement unit) 42 Display section 43 Control Unit D1 Actual distance D2 Virtual image display distance En gaze EP Eyepoint P Display image LE left eye RE right eye LP left eye image RP right eye image OJ Object S Virtual Image W Windshield (reflective material)
Claims
1. a display unit that is provided in the vehicle and that projects a display light including a display image toward a reflective member having transparency to display a virtual image; an object detection unit that detects an object included in a scene ahead of the vehicle; A gaze detection unit that detects the gaze of a driver of the vehicle; a distance measurement unit that measures an actual distance from an eye point of the driver to a specific object detected by the line of sight detection unit, the specific object being located in front of the line of sight of the driver and on which the driver's focus coincides, among the objects detected by the object detection unit; and A control unit that controls the display unit to superimpose the virtual image on the specific object, The control unit controls the display unit to perform double image prevention processing to prevent double images due to convergence by matching a virtual image display distance from the driver's eye point to the virtual image with the actual distance measured by the distance measurement unit.
2. the display image is a three-dimensional image including a left-eye image to be viewed by the left eye of the driver and a right-eye image to be viewed by the right eye of the driver, 2. The head-up display system according to claim 1, wherein the control unit controls the display unit to change the parallax between the image for the left eye and the image for the right eye, thereby executing the double image prevention process to adjust the virtual image display distance to the actual distance.
3. The display image is a two-dimensional image to be viewed by the left eye and the right eye of the driver, The head-up display system according to claim 1 , wherein the control unit executes the double image prevention process by controlling the display unit to change an optical path length of the display light so as to match the virtual image display distance to the actual distance.
Citation Information
Patent Citations
Three-dimensional (3D) navigation
JP2015069656A
On-vehicle display device
JP2017226292A
Head-up display
JP2019038451A
Display control device, head-up display apparatus and display control method of image
JP2022083609A
Projection-type display device and projection control method
WO2017043108A1
