Virtual image display device

Abstract

This virtual image display device comprises a control unit. When the number of low-gradation pixels having a gradation value of less than or equal to 5, which is the threshold value, among all pixels of a portion corresponding to a certain illumination segment in a display unit is less than 90% of the number of all the pixels, the control unit turns on a light-emitting element corresponding to an image included in the illumination segment, with the illumination segment being an image-corresponding illumination segment. Furthermore, when the number of the low-gradation pixels is at least 90% of the number of all the pixels, the control unit turns off the light-emitting element in the illumination segment, with the illumination segment being an image non-corresponding illumination segment.

Description

Virtual image display device

[0001] The present disclosure relates to a virtual image display device.

[0002] A conventional virtual image display device is described in, for example, Patent Document 1.

[0003] Japanese Patent Application Laid-Open No. 2019-002931

[0004] A virtual image display device according to the present disclosure is a virtual image display device that displays a virtual image superimposed on an external scene on a reflective member by irradiating the reflective member with light from an image displayed on a display unit of a display panel, the virtual image display device comprising: a backlight, in which light-emitting elements are provided in each of a plurality of illumination segments, and that causes the light-emitting elements of an illumination segment selected from the plurality of illumination segments and corresponding to the image to emit light from the emitted light-emitting elements toward the display panel; and a control unit that performs local dimming control of the backlight to turn on the light-emitting elements in the image-corresponding illumination segment corresponding to the image and turn off the light-emitting elements in the image-non-corresponding illumination segment not corresponding to the image, when the number of low-gradation pixels having gradation values ​​equal to or less than a predetermined threshold among all the pixels in a portion of the display unit corresponding to a certain illumination segment is less than a predetermined ratio of the number of all the pixels, the control unit determines that the illumination segment is the image-corresponding illumination segment and turns on the light-emitting elements included in the illumination segment that correspond to the image; When the number of low-gradation pixels among all the pixels is equal to or greater than the predetermined ratio to the number of all the pixels, the illumination segment is determined to be the image-non-corresponding illumination segment, and the light-emitting elements in the illumination segment are turned off.

[0005] 4B is a diagram showing an example of an input image histogram of an image-non-corresponding illumination segment; FIG. 4C is a diagram showing an example of an input image histogram of an image-corresponding illumination segment; FIG. 4D is a diagram showing an example of an input image histogram of an image-corresponding illumination segment; FIG. 4E is a diagram showing an example of a virtual image displayed on a display panel; FIG. 4F is a diagram showing an example of an input image histogram of an image-corresponding illumination segment; FIG. 4G is a diagram showing an example of a virtual image displayed by a virtual image display device according to another embodiment of the present disclosure; FIG. 4H is a diagram showing an example of a dummy image added to the virtual image shown in FIG. 4A; FIG. 4H is a diagram showing a change in luminance with respect to the gradation value of the virtual image; FIG. 4I is a diagram showing an example of a side view of an optical path visually recognized by a user when the virtual image display device is used in a HUD (Head Up Display) device; FIG. 4J is a diagram showing the line of sight from a viewing position and the position of the dummy image; FIG. 4J is a diagram showing an example of a virtual image showing that the dummy image is not visibly displayed even when illuminated with light from a backlight; FIG. 4J is a diagram showing an example of a virtual image showing an example of a dummy image; FIG. 4J is a diagram showing an example of a virtual image displayed by a virtual image display device according to an embodiment of the present disclosure;

[0006] Conventionally, a virtual image display device is configured to display a virtual image visible to a user (e.g., a driver) by reflecting display light of an image displayed on a display panel (e.g., a liquid crystal display panel) located on the light source side by a reflective member (e.g., an automobile windshield). This virtual image display device includes a transmissive display panel having a display screen for displaying an image, a backlight having a plurality of lighting segments arranged to partition the light source surface and illuminating the display panel from behind, and a backlight control unit that individually switches the lighting state and extinguisher state of each lighting segment to perform local dimming control of the backlight.

[0007] As seen by the user, a virtual image of the light source surface of the backlight, i.e., a light source surface virtual image, is formed behind the virtual image. When a virtual line is imagined passing through the virtual image and the assumed viewpoint of the user viewing the virtual image, the backlight control unit turns on the lighting segments in the light source surface virtual image that correspond to the intersection positions of the virtual line. The backlight control unit also turns off the lighting segments other than the lighting segments that are turned on. In other words, it performs local dimming control. This results in a virtual image with good visibility.

[0008] In the prior art described in Patent Document 1, for example, when a virtual image is viewed diagonally to the left or right, lighting segments on an extension of a virtual line from the assumed viewpoint are turned on to prevent a decrease in brightness of the virtual image perceived by the user in the eyebox. That is, lighting segments on an extension of the virtual line (also referred to as peripheral lighting segments) other than the lighting segment directly below the virtual image (also referred to as direct lighting segments) are turned on. That is, lighting segments that are not directly below the displayed portion of the virtual image but correspond to the intersection of the virtual line on the light source surface virtual image are turned on. In this case, lighting both the direct lighting segment and the peripheral lighting segment for the virtual image increases the power consumption of the backlight. Furthermore, when the virtual image is viewed from the front, if the peripheral lighting segment is turned on, the area around the virtual image becomes excessively bright, making the edges of the virtual image appear blurred. Therefore, there is a need for a virtual image with even better visibility.

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The drawings used in the following description are schematic. The dimensional ratios and the like in the drawings do not necessarily correspond to the actual ones.

[0010] FIG. 1 is a cross-sectional view of a main part of a virtual image display device according to an embodiment of the present disclosure. As shown in FIG. 1 , the virtual image display device 1 of this embodiment includes a display device 2, an optical system 3, and a control unit 4. The virtual image display device 1 allows a user (also referred to as a viewer) 5 to view a virtual image. The virtual image display device 1 is also referred to as a head-up display (HUD). In the following description, the head-up display may be abbreviated as HUD. The virtual image display device 1 may be mounted on a mobile object such as a vehicle, a ship, or an aircraft. When the virtual image display device 1 is mounted on a mobile object, the user 5 may be a driver, an operator, or a passenger of the mobile object. The virtual image display device 1 can display a virtual image by reflecting display light of the image using the optical system 3.

[0011] The virtual image display device 1 includes a display panel 7 having a display surface (also referred to as a display unit) 7a that displays an image, a backlight 6 that divides a light source surface 62 on which a plurality of light-emitting elements 61 are provided and has a plurality of lighting segments 63 (63a, 63b (shown in FIG. 4B)) that can be switched between a lit state and an extinguished state, and emits light from the light-emitting elements 61 of the image-corresponding lighting segments 63a that are in a lit state to the display panel 7, and a control unit 4 that controls the backlight 6 by local dimming to turn on the image-corresponding lighting segments 63a located directly below the image display and turn off the image-non-corresponding lighting segments 63b that are not located directly below the image display.

[0012] In local dimming control, typically, multiple light-emitting elements 61, such as light-emitting diodes, arranged on the light source surface of the backlight 6 are individually controlled according to the image. This allows the light-emitting elements 61 to be relatively bright in relatively bright areas of the display unit 7a (e.g., image areas) and relatively dark in relatively dark areas of the display unit 7a (e.g., non-image areas), thereby increasing the contrast ratio between the image and non-image areas. Furthermore, local dimming control can prevent the occurrence of a postcard phenomenon, in which the light source surface of the backlight 6 is faintly reflected as a background image (also called a postcard image) of a virtual image. The virtual image is, for example, 20Q shown in FIG. 1 , and the postcard image is, for example, 10Q shown in FIG. 1 .

[0013] The virtual image display device 1 of this embodiment includes a control unit 4 that performs local dimming control on the backlight 6 to turn on light-emitting elements 61 in an image-corresponding illumination segment 63a that corresponds to an image on the display panel 7 and turn off light-emitting elements 61 in an image-non-corresponding illumination segment 63b that does not correspond to the image. When the number of low-gradation pixels, whose gradation values ​​are equal to or less than a predetermined threshold, among all pixels in a portion of the display unit 7a that corresponds to a certain illumination segment 63, is less than a predetermined ratio of the total number of pixels, the control unit 4 determines that the illumination segment 63 is an image-corresponding illumination segment 63a and turns on the light-emitting elements 61 included in the illumination segment 63 that correspond to the image. Furthermore, when the number of low-gradation pixels among all the pixels is equal to or greater than a predetermined ratio of the total number of pixels, the control unit 4 determines that the illumination segment 63 is an image-non-corresponding illumination segment 63b and turns off the light-emitting elements 61 in the illumination segment 63.

[0014] The virtual image display device 1 of this embodiment has the following advantages due to its configuration. When the number of low-level pixels having a gradation value equal to or less than a predetermined threshold among all pixels in a portion of the display unit 7a of the display panel 7 corresponding to a certain illumination segment 63 is equal to or greater than a predetermined percentage of the total number of pixels, the illumination segment 63 is determined to be a non-image-corresponding illumination segment 63b, and the light-emitting elements in the illumination segment 63 are turned off. Therefore, even if the gradation value of the portion of the display unit 7a corresponding to the non-image-corresponding illumination segment 63b (also referred to as a non-image portion or non-virtual image portion) is not 0, the non-virtual image portion can be set to the lowest luminance (e.g., black display). As a result, the contrast ratio between the virtual image portion and the non-virtual image portion can be increased, improving the visibility of the virtual image. Furthermore, even if the gradation value of the non-virtual image portion is not 0, the light-emitting elements in the non-image-corresponding illumination segment 63b are turned off, thereby reducing the power consumption of the backlight.

[0015] The image-corresponding illumination segment 63a corresponding to the image on the display panel 7 may be an illumination segment 63 located directly below the image. The image-non-corresponding illumination segment 63b not corresponding to the image on the display panel 7 may be an illumination segment 63 not located directly below the image.

[0016] Furthermore, as shown in FIG. 4B , when dummy images D2 and D4 with gradation values ​​exceeding a predetermined threshold and below the upper limit gradation value that is not visible from the front are added to areas adjacent to the virtual image (e.g., "ABCDEF" in FIG. 4B ), when the virtual image is viewed from an oblique direction other than the front, the brightness of the edges of the virtual image adjacent to the dummy images D2 and D4 (e.g., "A" and "F" in FIG. 4B ) that are visible to the user 5 can be reduced. For example, when the virtual image is viewed obliquely from the left, the brightness of the right edge of the virtual image (e.g., "F" in FIG. 4B ) is reduced and easily visible. However, when the dummy image D4 is added to the area adjacent to the right edge of the virtual image, the reduced brightness of the right edge of the virtual image can be reduced. Note that the same effect as described above can be achieved with the virtual image "THE NEW HEAD UP DISPLAY" shown in FIG. 4B by adding dummy images D1 and D3.

[0017] The virtual image display device 1 may include a detector for detecting the position of the user's 5 eye, which is viewing the virtual image. When the user's 5 eye deviates from a position directly facing the virtual image, a dummy image is added to a portion adjacent to the virtual image in the direction opposite to the direction of deviation. In this case, the following effects are achieved. As shown in FIG. 7 , when the user's 5 eye 5e is at a position 5er diagonally to the right of the virtual image F1, if the leftmost illumination segment 63L corresponding to the dummy image D1 is turned off, the virtual image "A" and the virtual-image non-display portion F11 corresponding to the illumination segment 63L enter the field of view of the eye 5e at position 5er. This reduces the brightness of the left end of the virtual image "A," making it easier to view. A similar problem occurs when the user's 5 eye 5e is at a position 5el diagonally to the left of the virtual image F1, if the rightmost illumination segment 63R corresponding to the dummy image D2 is turned off. The virtual image display device 1 of this embodiment can solve this problem. The detector for detecting the position of the user's 5 eye may be an imaging device, such as a camera equipped with a CCD imaging element.

[0018] The dummy images D1 to D4 may flash. In this case, it becomes easier to adjust the gradation values ​​of the dummy images D1 to D4. That is, if the flashing frequency is set to 20 Hz or higher, or even 60 Hz or higher, the user 5's eyes will find it difficult to perceive changes in the gradation values ​​of the dummy images D1 to D4. For example, if the virtual image display device 1 is mounted on a moving object such as an automobile and the brightness of the external environment changes suddenly, such as when entering or exiting a tunnel, the gradation values ​​of the dummy images D1 to D4 may be adjusted. In this case, if the gradation values ​​of the dummy images D1 to D4 are changed while flashing at a frequency of 20 Hz or higher, the user 5's eyes will find it difficult to perceive changes in the gradation values ​​of the dummy images D1 to D4.

[0019] The blinking frequency of the dummy images D1 to D4 may be higher than the frame frequency of the virtual image. In this case, the rate of change in the gradation values ​​of the dummy images D1 to D4 becomes faster than the rate of change in the brightness, etc., of the virtual image, making it more difficult for the eyes of the user 5 to perceive the change in the gradation values ​​of the dummy images D1 to D4. The blinking frequency of the dummy images D1 to D4 may be more than 1 time but not more than about 10 times the frame frequency of the virtual image, but is not limited to this range.

[0020] For the dummy images D1 to D4, when the maximum gradation value is 255, the upper limit gradation value that is not visible from the front may exceed the predetermined threshold of 5, and may be, for example, about 6 to 68, about 9 to 68, or about 10 to 68. As shown in FIG. 9, the upper limit gradation value that is not visible from the front may be adjusted depending on the external environment. However, it is not limited to these ranges. Note that "to" means "through," and the same applies hereinafter.

[0021] The number of pixels in each of the dummy images D1-D4 may be equal to or less than the number of all pixels corresponding to the image-corresponding illumination segment 63a. For example, if the total number of pixels is 3,712 (58 vertically and 64 horizontally), the number of pixels in each of the dummy images D1-D4 may be 3,240 (54 vertically and 60 horizontally). In this case, (number of pixels in the dummy images) / (number of all pixels) is approximately 87%. If the number of pixels in the dummy images is less than the total number of pixels, the power consumption of the backlight 6 can be further reduced while preventing the brightness of the edges of the virtual image from being reduced and becoming visible. The ratio (number of pixels in the dummy images) / (number of all pixels) may be approximately 50% to 90%, but is not limited to this range.

[0022] The control unit 4 controls the lighting / extinguishing of the image for each of the multiple divided images corresponding to the multiple illumination segments 63. That is, if the number of low-gradation pixels, whose gradation values ​​are equal to or less than a predetermined threshold (for example, 5 or less when the maximum gradation value is 255), of each divided image corresponding to each illumination segment 63 is a predetermined percentage (for example, 90% or more) of the total number of pixels in the divided image, the control unit 4 determines that the illumination segment 63 is an image-non-corresponding illumination segment 63b, and turns off the light-emitting elements 61 of the illumination segment 63. Conversely, if the number of low-gradation pixels is less than a predetermined percentage of the total number of pixels in the divided image, the control unit 4 determines that the illumination segment 63 is an image-corresponding illumination segment 63a, and turns on the light-emitting elements 61 of the illumination segment 63.

[0023] The threshold value of the gradation value of low gradation pixels can be set to various values ​​depending on the external environment, as shown in the table in FIG. 9. For example, as will be described later, it can be set within a range of 0 to 68. The predetermined ratio of the number of low gradation pixels to the total number of pixels in the divided image is not limited to 90%, and can be changed depending on the image resolution. If the resolution is relatively low, it may be around 70%, and if the resolution is relatively high, it may be around 95%. Therefore, it can be set within a range of approximately 70% to 95%.

[0024] The display device 2 displays an image to be projected into the field of view of the user 5. The display device 2 may be configured as a transmissive display device. The transmissive display device may be, for example, a liquid crystal display (LCD) including a backlight 6 and a display panel 7. The backlight 6 and the display panel 7 are arranged in this order from the side farthest from the user 5 along the optical path of image light of the image displayed to the user 5.

[0025] The backlight 6 may be a direct type backlight. The backlight 6 may be a fluoride red phosphor (K ２ SiF ６ : Mn ４＋ In this case, the KSF type backlight 6 can prevent a decrease in the color purity of the virtual image. ４+ This is because KSF (Korean Fluorescent Fiber) is a type of line-emitting phosphor with a wide color gamut and a narrow wavelength spectrum width of the three primary colors RGB (highly sharp wavelength spectrum). The direct-type KSF-type backlight 6 includes a plurality of light-emitting elements 61 having KSF and an emission surface 6a capable of emitting light from the plurality of light-emitting elements 61. The backlight 6 may include a base 62 and a diffuser 65. The base 62 may have a main surface (also referred to as a light source surface) 62a facing the display panel 7, and the plurality of light-emitting elements 61 may be two-dimensionally arranged on the main surface 62a. The diffuser 65 may be located in front of the base 62 (on the left side in FIG. 1 ) and cover the plurality of light-emitting elements 61. The diffuser 65 may diffuse light emitted from the plurality of light-emitting elements 61, thereby allowing the backlight 6 to irradiate the display panel 7 with light in a planar manner. The light exit surface 6 a may be the surface of the diffuser plate 65 on the display panel 7 side, or may be the main surface 62 a of the base 62 .

[0026] Each of the light-emitting elements 61 includes at least one light source. The light source may be a point light source such as a light-emitting diode (LED). The LED includes a mini-LED and a micro-LED.

[0027] The display panel 7 has a display surface 7 a. The display surface 7 a is a surface that displays images based on an image display signal, and can output images to the optical system 3 side by light emitted from the emission surface 6 a of the backlight 6.

[0028] The display panel 7 may be a known liquid crystal panel, such as an IPS (In-Plane Switching) liquid crystal panel, an FFS (Fringe Field Switching) liquid crystal panel, a VA (Vertical Alignment) liquid crystal panel, or an ECB (Electrically Controlled Birefringence) liquid crystal panel. The display panel 7 includes two glass substrates (an array substrate and a color filter substrate) arranged to sandwich a liquid crystal layer, and the color filter substrate has a color filter CF on its surface facing the liquid crystal layer.

[0029] The control unit 4 controls the backlight 6. The control unit 4 switches each of the plurality of light-emitting elements 61 between an emitting state and a non-emitting state. The control unit 4 may control the luminance of the light-emitting elements 61 that are in the emitting state based on image data of an image displayed on the display unit 7a. The control unit 4 may control the luminance of the light-emitting elements 61 that are in the emitting state based on the background luminance in the field of view of the user 5.

[0030] The control unit 4 may control not only the backlight 6 but also each component of the virtual image display device 1. The control unit 4 may be configured as, for example, a processor. The control unit 4 may include one or more processors. The processor may include a general-purpose processor that loads a specific program to execute a specific function, and a dedicated processor specialized for a specific process. The dedicated processor may include an application-specific integrated circuit (ASIC). The processor may include a programmable logic device (PLD). The PLD may include an FPGA (field-programmable gate array). The control unit 4 may be either a system-on-a-chip (SoC) or a system-in-a-package (SiP) in which one or more processors work together. The control unit 4 may include a memory unit that stores various information or programs for operating each component of the virtual image display device 1. The memory unit may be configured, for example, as a semiconductor memory. The memory unit may function as a work memory for the control unit 4.

[0031] The optical system 3 is located in the optical path of image light emitted from the display device 2 and reaching the eyes of the user 5. The optical system 3 projects the image light emitted from the display device 2 onto the eyes of the user 5, forming a virtual image in the eyes of the user 5. The optical system 3 may enlarge or reduce the image displayed on the display device 2 to form an image in the eyes of the user 5. The optical system 3 includes a first optical member (also referred to as a first reflecting member) 3a and a second optical member (also referred to as a second reflecting member) 3b. The second optical member 3b is a reflecting member (e.g., a windshield of a moving body such as an automobile) having a transmissive / reflective function. The light of the image displayed on the display unit 7a of the display panel 7 is irradiated onto the reflecting member, thereby displaying a virtual image superimposed on the external scene viewed by the viewer 5 through the reflecting member.

[0032] The first reflecting member 3a reflects the image on the exit surface 6a of the backlight 6 and the image on the display surface 7a of the display panel 7 toward the second reflecting member 3b. The second reflecting member 3b reflects the image on the exit surface 6a and the image on the display surface 7a reflected by the first reflecting member 3a toward the eyes of the user 5. The number of optical members constituting the optical system 3 is not limited to two, but may be three or more, or may be four or more. The optical members may include reflective members including convex or concave mirrors. The optical members may include reflective members including free-form surface mirrors. The optical members may include refractive members including convex or concave lenses. The convex lenses include biconvex lenses, plano-convex lenses, and convex meniscus lenses. The concave lenses include biconcave lenses, plano-concave lenses, and concave meniscus lenses. The optical members are not limited to reflective or refractive members, but may include various other optical members such as hologram elements.

[0033] As shown in FIG. 1 , the second reflecting member 3b, which is a reflecting member, may have a transflective concave surface. The second reflecting member 3b may have a concave surface that reflects the image light (also referred to as first image light) from the output surface 6a and the image light (also referred to as second image light) from the display surface 7a, which are reflected by the first reflecting member 3a, toward the eyes of the user 5. In this case, the first image light and the second image light converge toward the user 5, thereby suppressing a decrease in the brightness and resolution of the virtual image 10Q generated by the first image light and suppressing a decrease in the brightness and resolution of the virtual image 20Q generated by the second image light. The concave surface of the second reflecting member 3b may be a spherical surface, an elliptical surface, a hyperbolic surface, a free-form surface, or the like. Furthermore, since the windshield of a moving object such as an automobile has a concave surface on the side facing the user 5, the second reflecting member 3b may be a windshield.

[0034] FIG. 2 is a diagram showing an example of an image displayed on the display panel 7. The backlight 6 is locally dimmed by the control unit 4. Each lighting segment 63 of the backlight 6 is equipped with a plurality of light-emitting elements 61. The display panel 7 displays an image on the display surface 7a using light from each lighting segment 63 of the backlight 6 that is locally dimmed by the control unit 4. The image on the display surface 7a is projected toward the user 5 by the optical system 3 and is visually recognized by the user 5 as a virtual image. In FIG. 2, as an example, a navigation image is displayed, which includes a road display 64a for the current road, a cross road display 64b indicating that there is an intersecting road on the left side of the road ahead in the direction of travel of the vehicle, a speed limit 64c, a driving speed 64d, an engine RPM 64e, and a lane number 64f indicating that the road is a two-lane road. An approach warning display 64g indicating that the approach warning function is activated is also displayed to the right of the road display 64a.

[0035] The backlight 6 lights up the light-emitting elements 61 of the lighting segments 63 located directly below each image, such as the road marking 64a, cross road marking 64b, speed limit 64c, driving speed 64d, rotation speed 64e, number of lanes 64f, and approach warning marking 64g. Light is irradiated onto the display panel 7 from the lighting segments 63 (image-corresponding lighting segments 63a), while lighting segments 63 not located directly below the image (image-non-corresponding lighting segments 63b) are turned off. As a result, the image on the display panel 7 is displayed on the display surface 7a and is presented to the user 5 as a virtual image by the optical system 3.

[0036] FIG. 3A is a graph showing an example of an input image histogram for the image-non-corresponding illumination segment 63b, and FIG. 3B is a graph showing an example of an input image histogram for the image-corresponding illumination segment 63a. In the case of FIG. 3A , when the maximum number of gradations is 255 and the control unit 4 sets the gradation threshold to 5, in a portion of the display panel 7 corresponding to one illumination segment 63, the number of pixels having a gradation threshold of 5 or less accounts for 90% or more of the total number of pixels in the portion corresponding to one illumination segment 63. In this case, the control unit 4 regards the image as background even if an image is present, and regards the illumination segment 63 as the image-non-corresponding illumination segment 63b, thereby turning off the light-emitting element 61 in the illumination segment 63. Note that the number of light-emitting elements 61 in the illumination segment 63 may be one or more. In this embodiment, the number of light-emitting elements 61 in the illumination segment 63 is one, and the same applies hereinafter.

[0037] 3B , in the portion of the display panel 7 corresponding to one illumination segment 63, the number of pixels having a gradation threshold of 5 or less is less than 90% of the total number of pixels. The control unit 4 regards the illumination segment 63 as an image-corresponding illumination segment 63a, and turns on the light-emitting element 61 in the illumination segment 63.

[0038] Therefore, the virtual image display device 1 lights up the light-emitting elements 61 in the image-corresponding illumination segments 63a, and turns off the light-emitting elements 61 in the image-non-corresponding illumination segments 63b if the image corresponds to the image-non-corresponding illumination segments 63b, even if the image includes pixels with a gradation greater than 0 and less than 5. This increases the contrast ratio between the virtual image and non-virtual image areas, improving the visibility of the virtual image. It also reduces the power consumption of the backlight.

[0039] Furthermore, the threshold value of the gradation value is not limited to 5, and may be changed depending on the brightness of the external environment (background brightness). FIG. 9 is a table showing an example of changing the range of the threshold value depending on the background brightness a. At night, the background brightness a is 5 (cd / m 2 ), and when the HUD display color (color of the virtual image) is white, the HUD display luminance (virtual image luminance) b is 350 (cd / m 2). The background contrast (a+b) / a for a white virtual image is 71. In this case, the background contrast of the non-virtual image portion only needs to be about 1 so that the non-virtual image portion is not visible compared to the background. When the background contrast of the non-virtual image portion is about 1, the maximum gradation value of the non-virtual image portion is 9. Therefore, the threshold at night can be set to 9 or in a range greater than 0 and equal to or less than 9. Similarly, when the external environment is twilight, the threshold can be set to 17 or in a range greater than 0 and equal to or less than 17. When the external environment is a snow wall or direct light (sunlight), the threshold can be set to 68 or in a range greater than 0 and equal to or less than 68.

[0040] FIG. 4A is a front view showing an example of a virtual image produced by a virtual image display device according to another embodiment of the present disclosure, and FIG. 4B is a front view showing dummy images D1 to D4 added to the virtual image shown in FIG. 4A . Note that parts corresponding to those in the above-described embodiment are designated by the same reference numerals, and redundant description will be omitted. In the virtual image display device 1 of this embodiment, the control unit 4 adds dummy images D2 and D4, whose gradation values ​​exceed the threshold value 5 and are equal to or lower than the upper limit gradation value (e.g., gradation value 6) that is not visible from the front, to portions adjacent to the virtual image (e.g., "ABCDEF" in FIG. 4B ). In this case, when the virtual image is viewed from an oblique direction other than the front, a decrease in brightness of the edges of the virtual image adjacent to the dummy images D2 and D4 (e.g., "A" and "F" in FIG. 4B ) that are visible to the user 5 can be suppressed. For example, when a virtual image is viewed diagonally from the left, the brightness of the right edge of the virtual image (e.g., "F" in FIG. 4B) tends to decrease and be perceived as low. However, adding a dummy image D4 to a portion adjacent to the right edge of the virtual image can prevent the brightness of the right edge of the virtual image from decreasing and being perceived as low. The same effect can be obtained for the virtual image "THE NEW HEAD UP DISPLAY" shown in FIG. 4B by adding dummy images D1 and D3.

[0041] The dummy images D1 to D4 are generated by turning off the light-emitting elements 61 in the image-non-corresponding illumination segments 63b, and black image data may be input to the portions of the divided images on the display panel that correspond to the image-non-corresponding illumination segments 63b. The black image data may be synthesized by image processing by the control unit 4. In this case, the light-emitting elements 61 in the image-non-corresponding illumination segments 63b may be slightly turned on, and black image data may be input to the portions of the divided images on the display panel that correspond to the image-non-corresponding illumination segments 63b.

[0042] In addition to black image data, RGB image data can also be used as long as the largest gradation value is within the invisible range. For example, when the external environment shown in Figure 9 is night, gradation values ​​of 0 to 9 are in the invisible range, so image data with (R, G, B) = (0 to 9, 0 to 9, 0 to 9) can be used. When the external environment is twilight, gradation values ​​of 0 to 17 are in the invisible range, so image data with (R, G, B) = (0 to 17, 0 to 17, 0 to 17) can be used. When the external environment is snowy or directly lit, gradation values ​​of 0 to 68 are in the invisible range, so image data with (R, G, B) = (0 to 68, 0 to 68, 0 to 68) can be used.

[0043] FIG. 5 is a graph showing the change in luminance with respect to the gradation value of the virtual image, and FIG. 6 is a side view schematically showing the optical path visually recognized by the user 5 when the virtual image display device 1 is used in a HUD device. When the virtual image display device 1 is used in a HUD device, as shown in Table 1 below, a vehicle equipped with the HUD device may be traveling at night or during the day. In Table 1, the background is the external scenery ahead in the direction of travel visually recognized by the user 5. In the background, the surface of an asphalt-paved road at night is designated as "asphalt," and a state in which the road surface is covered with snow due to snowfall is designated as "snowy road." During the day, an asphalt-paved road surface on a cloudy day is designated as "cloudy-asphalt," an asphalt-paved road surface on a sunny day is designated as "sunny-asphalt," and a snow-covered road surface on a sunny day is designated as "sunny-snowy road." "Background luminance a" is the luminance of the background visually recognized by the user 5. "HUD display luminance" is the luminance of a displayed virtual image (for example, the virtual image shown in FIGS. 4A and 4B). "HUD display luminance b" is the luminance of a virtual image with a gradation value of 6, for example. "Background contrast" is the contrast of the outside scene visually recognized by the user 5 (background contrast γ), and the background contrast γ is a value calculated by (background luminance a+HUD display luminance b) / (background luminance a). In the virtual image display device 1, the value of the background contrast γ is "1" regardless of whether it is day or night, which indicates excellent contrast.

[0044]

[0045] FIG. 7 is a diagram showing the line of sight and the positions of dummy images D1 and D2 from the viewing position of the user 5. An eye tracking system or an eye box 8 worn on the head lights up the light-emitting elements 61 of the lighting segments 63 directly below the HUD display pattern F1, which is a virtual image. Furthermore, dummy images D1 and D2 are added to the eye box 8 at positions adjacent to both ends of the HUD display pattern F1 in the longitudinal direction (left-right direction). Although the light-emitting elements 61 of the lighting segments 63 directly below the dummy images D1 and D2 are turned on, these dummy images D1 and D2 are not viewed by the user 5 from the front. This prevents a decrease in brightness at the edges of the virtual image (the left or right edges of the virtual image in FIG. 7 ) when the virtual image is viewed from an oblique direction (the diagonal left or right direction in FIG. 7 ).

[0046] When the eye 5e of the user 5 is at a position 5er diagonally to the right of the virtual image F1, if the leftmost illumination segment 63L corresponding to the dummy image D1 is turned off, the eye 5e at the position 5er will see the virtual image "A" as well as the virtual image non-display portion F11 corresponding to the illumination segment 63L, reducing the brightness of the left end of the virtual image "A" and making it easier to see. A similar problem occurs when the eye 5e of the user 5 is at a position 5el diagonally to the left of the virtual image F1 if the rightmost illumination segment 63R corresponding to the dummy image D2 is turned off. The virtual image display device 1 of this embodiment can solve this problem.

[0047] FIG. 8A is a front view of a virtual image showing that the dummy image is not visibly displayed even when light from the backlight 6 is irradiated. FIG. 8B is a front view of a virtual image schematically showing a dummy image. The pixel size of the display surface 7a of the display panel 7 is, for example, 1280 x 640, and the image is divided into 22 x 10 divisions. Therefore, the pixel size of one division is 58 x 64 (= 3712). The threshold may be set to 5, for example, when the maximum number of gradations is 255. The image pattern of the dummy images D1 to D4 may have a gradation value of 6 (maximum gradation value 255) and a pixel size of 54 x 60. The dummy images D1 to D4 may be, for example, rectangular, positioned adjacent to the virtual image "A" on the top, bottom, left, and right sides, as shown on the display surface 7a in FIG. 8B.

[0048] The virtual image display device 1 according to the present disclosure achieves the following effects. When the number of low-gradation pixels, whose gradation values ​​are equal to or less than a predetermined threshold, among all pixels in a portion corresponding to a certain illumination segment 63 on the display unit 7a of the display panel 7, is equal to or greater than a predetermined percentage of the total number of pixels, the illumination segment 63 is determined to be a non-image-corresponding illumination segment 63b, and the light-emitting elements in the illumination segment 63 are turned off. Therefore, even if the gradation value of the portion of the display unit 7a corresponding to the non-image-corresponding illumination segment 63b is not 0, the non-virtual image portion can be set to the lowest brightness (e.g., black display). As a result, the contrast ratio between the virtual image portion and the non-virtual image portion can be increased, improving the visibility of the virtual image. Furthermore, even if the gradation value of the non-virtual image portion is not 0, the light-emitting elements in the non-image-corresponding illumination segment 63b are turned off, thereby reducing the power consumption of the backlight.

[0049] According to the virtual image display device disclosed herein, when the number of low-gradation pixels, whose gradation values ​​are equal to or less than a predetermined threshold, among all pixels in a portion of a display panel corresponding to a certain illumination segment is equal to or greater than a predetermined percentage of the total number of pixels, the illumination segment is determined to be a non-image-corresponding illumination segment, and the light-emitting elements in the illumination segment are turned off. Therefore, even if the gradation value of the portion of the display section corresponding to the non-image illumination segment (also referred to as a non-image portion or non-virtual image portion) is not 0, the non-virtual image portion can be displayed at the lowest brightness (e.g., black display). As a result, the contrast ratio between the virtual image portion and the non-virtual image portion can be increased, improving the visibility of the virtual image. Furthermore, backlight power consumption can be reduced.

[0050] Furthermore, when a dummy image having a gradation value that exceeds a predetermined threshold and is equal to or less than the upper limit gradation value that is not visible from the front is added to a portion adjacent to the virtual image, it is possible to prevent a decrease in brightness at the edge of the virtual image adjacent to the dummy image, which is visible to the user when the virtual image is viewed from an oblique direction other than the front. For example, when a virtual image is viewed obliquely from the left, the brightness at the right edge of the virtual image is easily reduced and easily visible, but when a dummy image is added to a portion adjacent to the right edge of the virtual image, it is possible to prevent the brightness at the right edge of the virtual image from being reduced and easily visible.

[0051] The present disclosure can be implemented in the following configurations (1) to (14).

[0052] (1) A virtual image display device that displays a virtual image superimposed on an external scene on a reflective member by irradiating the reflective member with light of an image displayed on a display unit of a display panel, the virtual image display device comprising: a backlight in which light-emitting elements are provided in each of a plurality of illumination segments, and that causes the light-emitting elements of a selected illumination segment from the plurality of illumination segments that corresponds to the image to emit light from the emitted light-emitting elements toward the display panel; and a control unit that performs local dimming control of the backlight to turn on the light-emitting elements in the image-corresponding illumination segment that corresponds to the image and turn off the light-emitting elements in the image-non-corresponding illumination segment that does not correspond to the image, when the number of low-level pixels whose gradation value is equal to or less than a predetermined threshold among all the pixels in a portion of the display unit that corresponds to a certain illumination segment is less than a predetermined ratio of the number of all the pixels, the control unit determines that the illumination segment is the image-corresponding illumination segment and turns on the light-emitting elements included in the illumination segment that correspond to the image, A virtual image display device that, when the number of low-tone pixels among all pixels is equal to or greater than the predetermined ratio of the number of all pixels, determines that the lighting segment is an image-non-corresponding lighting segment and turns off the light-emitting element in the lighting segment.

[0053] (2) The control unit adds the dummy image, whose gradation value exceeds the predetermined threshold and is equal to or less than an upper limit gradation value that is not visible from the front, to an area adjacent to the virtual image by lighting the non-image-corresponding lighting segment adjacent to the image-corresponding lighting segment.

[0054] (3) The virtual image display device according to (2) above, wherein the number of pixels of the dummy image is equal to or less than the number of pixels of all the pixels corresponding to the image-corresponding illumination segment.

[0055] (4) A virtual image display device as described in (2) or (3) above, which includes a detector for detecting the position of the user's eye viewing the virtual image, and when the user's eye shifts from a position directly facing the virtual image, adds the dummy image to a location adjacent to the virtual image in the direction opposite to the shifted direction.

[0056] (5) The virtual image display device according to any one of (2) to (4) above, wherein the dummy image blinks.

[0057] (6) The virtual image display device according to (5), wherein the blinking frequency of the dummy image is higher than the frame frequency of the virtual image.

[0058] (7) The virtual image display device according to any one of (1) to (6) above, wherein the predetermined threshold value is set in the range of more than 0 and not more than 68 when the maximum gradation value is 255.

[0059] (8) The virtual image display device according to (7), wherein the predetermined threshold is changed depending on background luminance, which is the luminance of the external environment.

[0060] (9) The virtual image display device according to (8), wherein the predetermined threshold is set in the range of more than 0 and not more than 9 when the external environment is night.

[0061] (10) The virtual image display device according to any one of (1) to (9) above, wherein the predetermined threshold value is set to 5 when the maximum gradation value is 255.

[0062] (11) The virtual image display device according to any one of (1) to (10) above, wherein the predetermined ratio is set to 70% to 90% of the number of all pixels.

[0063] (12) The virtual image display device according to (11) above, wherein the predetermined ratio is set to 90% of the number of all pixels.

[0064] (13) The virtual image display device according to any one of (1) to (12) above, wherein the backlight includes a plurality of light-emitting elements having a fluoride red phosphor.

[0065] (14) The virtual image display device according to any one of (1) to (13), wherein the reflecting member has a transmissive-reflective concave surface.

[0066] Although the embodiments of the present disclosure have been described in detail above, the present disclosure is not limited to the above-described embodiments, and various modifications and improvements are possible within the scope of the gist of the present disclosure. It goes without saying that all or part of the components constituting each of the above-described embodiments can be combined as appropriate within the scope of not contradicting each other.

[0067] REFERENCE SIGNS LIST 1 Virtual image display device 2 Display unit 3 Optical system 3a First optical member (first reflecting member) 3b Second optical member (second reflecting member) 4 Control unit 5 User 6 Backlight 6a Emission surface 7 Display panel 7a Display surface 8 Eye box 61 Light-emitting element 62 Light source surface 62a Main surface 63 Illumination segment 63a Image-corresponding segment 63b Image-non-corresponding segment 64 Base 64a Road display 64b Cross road display 64c Speed ​​limit 64d Traveling speed 64e RPM 64f Number of lanes 64g Approach warning display 65 Diffuser D; D1, D2, D3, D4 Dummy image

Claims

1. A virtual image display device that displays a virtual image superimposed on an external scene on a reflective member by irradiating the reflective member with light of an image displayed on a display unit of a display panel, the virtual image display device comprising: a backlight, in which a plurality of illumination segments are each provided with a light-emitting element, and which causes the light-emitting elements of a selected illumination segment from the plurality of illumination segments that corresponds to the image to emit light from the emitted light-emitting elements toward the display panel; and a control unit that performs local dimming control of the backlight to turn on the light-emitting elements in the image-corresponding illumination segment that corresponds to the image and to turn off the light-emitting elements in the image-non-corresponding illumination segment that does not correspond to the image, wherein when the number of low-level pixels having a gradation value equal to or less than a predetermined threshold value among all the pixels in a portion of the display unit that corresponds to a certain illumination segment is less than a predetermined ratio to the number of all the pixels, the control unit determines that the illumination segment is the image-corresponding illumination segment and turns on the light-emitting elements included in the illumination segment that correspond to the image; A virtual image display device that, when the number of low-tone pixels among all pixels is equal to or greater than the predetermined ratio to the number of all pixels, determines that the lighting segment is an image non-corresponding lighting segment and turns off the light-emitting element in the lighting segment.

2. The virtual image display device according to claim 1, wherein the control unit adds a dummy image, the gradation value of which exceeds the predetermined threshold and is equal to or lower than an upper limit gradation value that is not visible from the front, to an area adjacent to the virtual image by lighting the non-image-corresponding lighting segment adjacent to the image-corresponding lighting segment.

3. The virtual image display device according to claim 2, wherein the number of pixels of the dummy image is equal to or less than the number of pixels of all the pixels corresponding to the image-corresponding illumination segment.

4. A virtual image display device as described in claim 2 or 3, comprising a detector for detecting the position of the eye of a user viewing the virtual image, and when the eye of the user deviates from a position directly facing the virtual image, the dummy image is added to a portion adjacent to the virtual image in the direction opposite to the direction of deviation.

5. The virtual image display device according to any one of claims 2 to 4, wherein the dummy image blinks.

6. The virtual image display device according to claim 5, wherein the blinking frequency of the dummy image is higher than the frame frequency of the virtual image.

7. The virtual image display device according to any one of claims 1 to 6, wherein the predetermined threshold value is set in the range exceeding 0 and not exceeding 68 when the maximum gradation value is 255.

8. The virtual image display device according to claim 7, wherein the predetermined threshold is changed according to background luminance, which is the luminance of the external environment.

9. The virtual image display device according to claim 8, wherein the predetermined threshold is set in the range of more than 0 and not more than 9 when the external environment is night.

10. A virtual image display device according to any one of claims 1 to 9, wherein the predetermined threshold value is set to 5 when the maximum gradation value is 255.

11. The virtual image display device according to any one of claims 1 to 10, wherein the predetermined ratio is set to 70% to 90% of the number of all pixels.

12. The virtual image display device according to claim 11, wherein the predetermined ratio is set to 90% of the total number of pixels.

13. The virtual image display device according to any one of claims 1 to 12, wherein the backlight comprises a plurality of light-emitting elements having a fluoride red phosphor.

14. The virtual image display device according to any one of claims 1 to 13, wherein the reflective member has a transmissive-reflective concave surface.

Images