Floating image display device

Abstract

A floating image display device according to the present disclosure comprises a display unit, an optical system, and a detection unit. The optical system forms an image of light of a display image displayed on the display unit as a floating image of a real image. The detection unit is positioned in the vicinity of the display unit. The detection unit detects, via the optical system, a reflection image of an object that is illuminated by the light of the display image and that overlaps a virtual imaging surface of the floating image in the optical axis direction of the floating image. When the value of the modulation transfer function of the display image is M1 and the value of the modulation transfer function of the reflection image is M2, the floating image display device has a ratio M2 / M1 of 0.3 or more.

Description

Floating Image Display

[0001] The present disclosure relates to a floating image display device.

[0002] 2. Description of the Related Art Conventionally, a non-contact input device is known, for example, as described in Japanese Patent Application Laid-Open No. 2003-222999.

[0003] Patent No. 5856357

[0004] The floating image display device of the present disclosure comprises: a display unit; an optical system that focuses the light of a display image displayed on the display unit as a floating image of a real image; and a detection unit located near the display unit that detects, via the optical system, a reflected image of an object that is illuminated by the light and overlaps with a virtual imaging plane of the floating image in the optical axis direction of the floating image, wherein when the value of the modulation transfer function of the display image is M1 and the value of the modulation transfer function of the reflected image is M2, the ratio M2 / M1 is 0.3 or more.

[0005] Furthermore, the floating image display device of the present disclosure comprises: a display unit; an optical system that focuses the light of a display image displayed on the display unit as a floating image of a real image; and a detection unit located near the display unit that detects, via the optical system, a reflected image of an object that is illuminated by the light and overlaps with a virtual imaging plane of the floating image in the optical axis direction of the floating image, wherein when the area obtained by integrating the modulation transfer function of the display image on the spatial frequency axis is S1 and the area obtained by integrating the modulation transfer function of the reflected image on the spatial frequency axis is S2, the ratio S2 / S1 is 0.3 or more.

[0006] Furthermore, the floating image display device of the present disclosure comprises: a display unit; an optical system that forms the light of a display image displayed on the display unit as a floating image of a real image; and a detection unit located near the display unit, which detects, via the optical system, a reflected image of an object that is illuminated by the light and overlaps with a virtual imaging plane of the floating image in the optical axis direction of the floating image, and wherein when the spatial frequency is λ, the value of the modulation transfer function of the display image is M1, and the value of the modulation transfer function of the reflected image is M2 (where M1 and M2 are values ​​normalized to a maximum value of 1), M1 is 0.9 to 1 when λ is greater than 0 / mm and 3 / mm or less, and M2 is 0.047λ when λ is greater than 0 / mm and 3 / mm or less. 3 -0.218λ 2 −0.020λ+1<M2≦1.

[0007] 9A and 9B are graphs showing the relationship between a first modulation transfer function and a second modulation transfer function and spatial frequency; FIG. 10B is a front view showing a test pattern for measuring the first modulation transfer function; FIG. 11C is a diagram showing an overall image of a captured image of the test pattern of FIG. 7; FIG. 12A is a diagram showing a captured image used to measure the first modulation transfer function; FIG. 13 is a graph showing a line spread function obtained from the captured image of FIG. 9; FIG. 14A is a diagram showing a captured image of a reflected image; FIG. 15A is a diagram showing the contour of an object extracted from the captured image of FIG. 11; FIG. 16A is a diagram showing a cross-sectional view of a floating image display device showing different positions of an object relative to a virtual imaging plane of a floating image; 8 is a diagram showing the test pattern of FIG. 7 divided into a plurality of regions, and the captured image of FIG. 8 is a diagram showing the test pattern of FIG. 7 divided into a plurality of regions.

[0008] Patent Document 1 discloses a non-contact input device that forms a floating image in the air, detects a user's gesture with respect to the floating image, and executes a predetermined function based on the detected gesture.

[0009] There is a demand for a device that can form a floating image in the air and accurately detect a user's gestures and authentication information (e.g., fingerprints, palm prints, etc.). Conventional non-contact input devices sometimes cannot accurately detect a user's gestures and authentication information.

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The drawings referred to below are schematic. Dimensional ratios and the like in the drawings do not necessarily correspond to the actual ones. In this specification, for convenience, a Cartesian coordinate system XYZ is defined in some of the drawings. In Figures 3B, 3C, 4A, and 4B, the visible light source, infrared light source, detection element, and infrared light-emitting element are shown with hatching to facilitate illustration. Furthermore, in Figures 7 to 9 and 14A, low-brightness areas in the images are shown with a brightness higher than the actual brightness.

[0011] As shown in FIG. 1 , a floating image display device 1 according to an embodiment of the present disclosure includes a display unit 2, an optical system 3, and a detection unit 4.

[0012] The display unit 2 has a display surface 2a, and displays a display image D formed on the display surface 2a as a real floating image (also referred to as an aerial image) F. That is, the display unit 2 emits light (also referred to as image light) L1 formed as the floating image F from the display surface 2a.

[0013] The floating image display device 1 of this embodiment includes a display unit 2, an optical system 3 that focuses the light of a display image D displayed on the display unit 2 as a floating image F, which is a real image, and a detection unit 4 located near the display unit 2, that detects, via the optical system 3, a reflected image R of an object that is illuminated by the light of the display image D and overlaps with a virtual imaging plane Fp of the floating image F in the optical axis direction of the floating image F. The detection unit 4 is configured such that, when the value of the modulation transfer function of the display image D is M1 and the value of the modulation transfer function of the reflected image R is M2, the ratio M2 / M1 is 0.3 or more. Alternatively, when the area obtained by integrating the modulation transfer function of the display image D on the spatial frequency axis is S1 and the area obtained by integrating the modulation transfer function of the reflected image R on the spatial frequency axis is S2, the ratio S2 / S1 is 0.3 or more. These configurations provide the following effects. Since the reflected image R of the object that overlaps with the virtual imaging plane Fp of the floating image F in the optical axis direction of the floating image F can be clearly detected with high resolution, it becomes possible to accurately detect the gesture and authentication information of the user 5. Furthermore, since the reflected image R of the object is detected via the optical system 3, it is possible to detect the reflected image R of the object as seen from the front. As a result, if the object is the fingers 5f of the user 5, it is possible to accurately detect the gesture and type of gesture of the fingers 5f of the user 5.

[0014] The display unit 2 may be a transmissive display unit. As shown in FIG. 3A , the transmissive display unit may include a liquid crystal panel 21 and a backlight 22. The liquid crystal panel 21 may be a known liquid crystal panel. The known liquid crystal panel may be, for example, an IPS (In-Plane Switching) type, an FFS (Fringe Field Switching) type, a VA (Vertical Alignment) type, an ECB (Electrically Controlled Birefringence) type, or the like.

[0015] As shown in FIGS. 3B and 3C , the backlight 22 may be a direct-type backlight including a substrate 23 having a first surface 23 a and a plurality of visible light sources 24 arranged in a matrix on the first surface 23 a. The substrate 23 may be made of, for example, a glass material, a resin material, a ceramic material, a semiconductor material, or the like. The backlight 22 is positioned such that the first surface 23 a faces the rear surface of the liquid crystal panel 21. The visible light source 24 is configured to emit white light. The visible light source 24 may include a white light-emitting element that emits white light, or may include a red light-emitting element that emits red light, a green light-emitting element that emits green light, and a blue light-emitting element that emits blue light. The white light-emitting element, red light-emitting element, green light-emitting element, and blue light-emitting element of the visible light source 24 may be, for example, a light-emitting diode (LED), an organic light-emitting diode (OLED), or the like. The LEDs may be mini-LEDs or micro-LEDs. The backlight 22 may include a lens array, a diffusion plate, and the like for uniformly illuminating the liquid crystal panel 21 .

[0016] 3B and 3C, the backlight 22 may include a plurality of infrared light sources 25 arranged in a matrix on the first surface 23a. The infrared light sources 25 may emit infrared light with a wavelength of, for example, about 0.8 to 2.5 μm. The infrared light sources 25 may be, for example, infrared LEDs, infrared OLEDs, etc. The infrared light sources 25 may impart a warming sensation to the fingers of the user 5, for example, when the fingers approach or touch the floating image F. Furthermore, the infrared light sources 25 may also be a detection light source of a detection unit that detects the fingers of the user 5, for example, when the fingers approach or touch the floating image F.

[0017] The backlight 22 may have more visible light sources 24 than infrared light sources 25, as shown in Fig. 3B, or may have the same number of visible light sources 24 and infrared light sources 25, as shown in Fig. 3C. In Fig. 3C, reference numeral 41 denotes a detection element. Hereinafter, the visible light sources 24 and the infrared light sources 25 may be collectively referred to as light sources 24, 25.

[0018] The transmissive display unit does not need to include the liquid crystal panel 21. The transmissive display unit may be configured to include a MEMS (Micro Electro Mechanical Systems) shutter type display panel and a backlight.

[0019] The display unit 2 is not limited to a transmissive display unit and may be a self-luminous display unit. As shown in FIGS. 4A and 4B , the self-luminous display unit may include a substrate 26 having a second surface 26a and a plurality of pixels 27 arranged in a matrix on the second surface 26a. The substrate 26 may be made of, for example, a glass material, a resin material, a ceramic material, a semiconductor material, or the like. The pixels 27 may include a red light-emitting element 27R that emits red light, a green light-emitting element 27G that emits green light, and a blue light-emitting element 27B that emits blue light. The red light-emitting element 27R, the green light-emitting element 27G, and the blue light-emitting element 27B may be, for example, an LED, an OLED, or the like. The LED may be a micro-LED. The red light-emitting element 27R, the green light-emitting element 27G, and the blue light-emitting element 27B may be arranged adjacent to each other in the planar direction of the second surface 26a, as shown in FIG. 4A . The red light emitting element 27R, the green light emitting element 27G, and the blue light emitting element 27B may be stacked in order in a direction perpendicular to the second surface 26a to form a stacked light emitting element. In this case, as shown in FIG. 4B , two or more stacked light emitting elements can be arranged in each pixel 27, thereby increasing the pixel density of the display unit 2.

[0020] As shown in FIG. 4B , the pixel 27 may further include an infrared light-emitting element 27I. The infrared light-emitting element 27I may emit infrared light with a wavelength of, for example, about 0.8 to 2.5 μm. The infrared light-emitting element 27I may be, for example, an infrared LED, an infrared OLED, or the like. The infrared light-emitting element 27I may impart a warming sensation to the fingers of the user 5, for example, when the fingers of the user 5 approach or touch the floating image F. Furthermore, the infrared light-emitting element 27I may be a detection light source of a detection unit that detects the fingers of the user 5, for example, when the fingers of the user 5 approach or touch the floating image F.

[0021] The optical system 3 focuses the image light L1 emitted from the display unit 2 on a virtual imaging plane (hereinafter also referred to as the virtual imaging plane) Fp (shown in FIGS. 1 and 2 ), allowing the user 5 to view the image light L1 as a real floating image F. The eyes of the user 5 and the virtual imaging plane Fp may be separated by a predetermined design distance (hereinafter also referred to as the appropriate viewing distance) d in the traveling direction of the image light L1 emitted from the optical system 3. The appropriate viewing distance d is a distance at which the user 5 can appropriately view the floating image F. The appropriate viewing distance d may be, for example, approximately 300 mm to 700 mm, or may be approximately 500 mm. The image light L1 may be composed of visible light, or may be composed of visible light and infrared light.

[0022] The optical system 3 may be composed of reflective members such as a concave mirror, a convex mirror, etc. As shown in Figure 1, the optical system 3 may be composed of a first concave mirror 3a, a convex mirror 3b, and a second concave mirror 3c.

[0023] The first concave mirror 3a reflects the image light L1 emitted from the display unit 2 in a direction different from the direction toward the display unit 2. The first concave mirror 3a may be a free-form concave mirror, or may be a spherical concave mirror or an aspherical concave mirror.

[0024] The convex mirror 3b reflects the image light L1 reflected by the first concave mirror 3a in a direction different from the direction toward the first concave mirror 3a. The convex mirror 3b may be a free-form convex mirror, a spherical convex mirror, or an aspherical convex mirror.

[0025] The second concave mirror 3c reflects the image light L1 reflected by the convex mirror 3b in a direction different from the direction toward the convex mirror 3b, forming a floating image F. The second concave mirror 3c may be a free-form concave mirror, or may be a spherical concave mirror or an aspherical concave mirror.

[0026] The first concave mirror 3a and the second concave mirror 3c may be free-form concave mirrors, and the convex mirror 3b may be a free-form convex mirror. In this case, the distortion of the floating image F can be made extremely small. For example, the distortion of the floating image F can be made 5% or less. In particular, the corners or four corners of the rectangular display area (virtual image plane Fp) of the floating image F may have the largest distortion, and in that case, by reducing the distortion of the corners or four corners of the display area to 5% or less, the distortion of the entire display area can be kept to 5% or less.

[0027] The optical system 3 is not limited to the configuration shown in Fig. 1. The optical system 3 may be configured with a first concave mirror 3d and a second concave mirror 3e as shown in Fig. 2.

[0028] The first concave mirror 3d reflects the image light L1 emitted from the display unit 2 in a direction different from the direction toward the display unit 2. The first concave mirror 3d may be a free-form concave mirror, or may be a spherical concave mirror or an aspherical concave mirror.

[0029] The second concave mirror 3e reflects the image light L1 reflected by the first concave mirror 3d in a direction different from the direction toward the first concave mirror 3d, and forms a floating image F. The second concave mirror 3e may be a free-form concave mirror, or may be a spherical concave mirror or an aspherical concave mirror.

[0030] The first concave mirror 3a and the second concave mirror 3c may be free-form concave mirrors. In this case, the distortion of the floating image F can be made extremely small. For example, the distortion of the floating image F can be made 5% or less. In particular, the corners or four corners of the rectangular display area (virtual image plane Fp) of the floating image F may have the largest distortion, and in this case, by reducing the distortion of the corners or four corners of the display area to 5% or less, the distortion of the entire display area can be kept to 5% or less.

[0031] In the floating image display device 1, the optical system 3 includes only reflective members, and does not include semi-transmissive and semi-reflective members such as half mirrors, beam splitters, or wire grid polarizers, and therefore, it is possible to reduce degradation of the resolution of the floating image F due to refraction, deflection, etc. of the image light L1, and to increase the utilization rate of the image light L1.

[0032] When the object O in contact with the floating image F is illuminated by the image light L1, the detection unit 4 detects, via the optical system 3, a reflected image R (shown in FIGS. 1 and 2) of the object O that overlaps with the virtual image plane Fp of the floating image F in the optical axis direction Fa (shown in FIGS. 1 and 2). In other words, when the virtual image plane Fp of the floating image F is viewed from the front from the optical axis direction Fa of the floating image F, the detection unit 4 detects, via the optical system 3, the reflected image R of the object O that overlaps with the virtual image plane Fp. The detection unit 4 can clearly capture the reflected image R by detecting the light L2 of the reflected image R. Since the reflected image R of the object O is detected via the optical system 3, it can detect the reflected image R when the object O is viewed from the front. As a result, when the object O is a finger 5f of the user 5, the gesture and the type of gesture of the finger 5f of the user 5 can be accurately detected. For example, types of gestures include drag, swipe, tap, double tap, multi-tap, flick, pinch out, pinch in, rotation, etc. Gestures may also include eye movement, eye blinking, mouth movement, etc.

[0033] The light L2 of the reflected image R is also referred to as reflected light. In the following, the object O is assumed to be the hand of the user 5, particularly the fingers 5f, but the object O may also be the head, face, eye, etc. of the user 5. The reflected image R of the object O can be detected when the object O is in direct contact with the floating image F, and can also be detected when the object O is not in direct contact with the floating image F. Therefore, when we say that the reflected image R is detected when the object O is in contact with the floating image F, it means both that the reflected image R is detected when the object O is located near the floating image F, and that the reflected image R is detected when the object O is located within the virtual image plane Fp.

[0034] When detecting a reflected image R when the object O is located within the virtual imaging plane Fp, the reflected image R may be detected when a part of the object O is located within the virtual imaging plane Fp. For example, when the object O is a finger of the user 5, the reflected image R of the finger may be detected when a part of the finger is located within the virtual imaging plane Fp.

[0035] When detecting a reflected image R when the object O is located near the floating image F, the object O may be within the range of the focal depth of the image light L1, including the virtual image plane Fp. In this case, it becomes easier for the detection unit 4 to clearly capture the reflected image R. The object O may be within a range of about 1 to 2 times the focal depth of the image light L1, including the virtual image plane Fp, or may be within a range of about 1 to 1.5 times the focal depth of the image light L1. In this case, it becomes easier for the detection unit 4 to almost clearly capture the reflected image R. Note that "to" means "through," and the same applies hereinafter.

[0036] The detection unit 4 is located near the display unit 2. The detection unit 4 may be located so that the optical path length of the reflected light L2 from the detection unit 4 to the floating image F via the optical system 3 (first concave mirrors 3a, 3d) matches or approximately matches the optical path length of the image light L1 from the display unit 2 to the floating image F via the optical system 3 (first concave mirrors 3a, 3d). In this case, the detection unit 4 is located substantially at a confocal position with respect to the display unit 2. Then, the reflected image R of the object O in contact with the floating image F is formed at high resolution on the detection unit 4, so that the reflected image R can be captured at high resolution. In other words, the detection unit 4 may be located substantially at the confocal position of the optical system 3. In this case, when the object O is in contact with the floating image F, it is possible to accurately detect the position, shape (external shape and surface shape), etc. of the object O. In addition, the fact that the detection unit 4 is located at the confocal position of the optical system 3 means that the optical path from the display unit 2 through the optical system 3 to the floating image F and the optical path from the detection unit 4 through the optical system 3 to the floating image F have the same optical path length, regardless of whether they are the same or not.

[0037] The detection unit 4 is configured to include, for example, a plurality of detection elements 41. The detection elements 41 may include photodiodes. The detection elements 41 may include a photodiode for detecting red light, a photodiode for detecting green light, and a photodiode for detecting blue light. When the display unit 2 includes the infrared light source 25 or the infrared light-emitting element 27I, the detection elements 41 may include photodiodes for detecting infrared light.

[0038] The detection unit 4 may be integrated with the display unit 2. When the display unit 2 is a transmissive display unit, the detection unit 4 may be integrated with the backlight 22. The plurality of detection elements 41 may be arranged in a matrix on the first surface 23a of the substrate 23. When the display unit 2 is a self-luminous display unit, the detection unit 4 may be arranged in a matrix on the second surface 26a of the substrate 26. When the detection unit 4 and the display unit 2 are integrated, a reflected image R of the object O is formed on the detection unit 4 with high resolution, and therefore the detection unit 4 can capture the reflected image R with high resolution.

[0039] The detection element 41 may include a lens located above the photodiode (in the positive direction of the Z axis in FIGS. 3B, 3C, 4A, and 4B). The lens may be configured to collimate the reflected light L2 incident on the photodiode. The lens may be, for example, a plano-concave lens arranged with its flat surface facing the photodiode. The detection element 41 may also include a light-shielding member. In this case, it is possible to prevent light emitted from the light source 24, 25 or the pixel 27 from directly entering the photodiode. The light-shielding member may be, for example, a frame arranged to surround the photodiode when viewed in a direction perpendicular to the first surface 23 a or the second surface 26 a.

[0040] The detection unit 4 does not have to be integrated with the display unit 2. As shown in FIG. 5 , the floating image display device 1 may include a detection unit 4 configured separately from the display unit 2 and an optical member 6. The optical member 6 is located in the optical path of the image light L1 and the reflected light L2 between the display unit 2 and the optical system 3 (first concave mirrors 3 a, 3 d). The optical member 6 transmits a portion of the image light L1 that is emitted from the display unit 2 and travels toward the first concave mirrors 3 a, 3 d. The optical member 6 also reflects a portion of the reflected light L2 that is reflected by the first concave mirrors 3 a, 3 d and travels toward the display unit 2, toward the detection unit 4. The detection unit 4 detects the portion of the reflected light L2 reflected by the optical member 6. The optical member 6 may be configured as a semi-transmissive / semi-reflective member such as a half mirror, a beam splitter, or a wire grid polarizer. By configuring the detection unit 4 separately from the display unit 2, it is possible to prevent the image light L1 emitted from the light source 24, 25 or the pixel 27 from directly entering the photodiode, thereby reducing the mixing of the image light L1 with the reflected light L2.

[0041] The detection unit 4 can also function as a control unit of the floating image display device 1. The detection unit 4 is connected to each component of the floating image display device 1 and can control each component. The detection unit 4 can perform image processing on the captured image of the reflected image R. The image processing includes, for example, filter processing, color space conversion, measurement of the modulation transfer function, calculation of the area of ​​the modulation transfer function, etc. The image processing further includes detecting the movement and surface shape of the object O based on the captured image of the reflected image R.

[0042] The detection unit 4 may include, for example, a processor. The detection unit 4 may include one or more processors. The processor may include at least one of a general-purpose processor configured to load a specific program and execute a specific function, and a dedicated processor specialized for a specific process. The dedicated processor may include an ASIC (Application Specific Integrated Circuit). The processor may include a PLD (Programmable Logic Device). The PLD may include an FPGA (Field-Programmable Gate Array). The detection unit 4 may include at least one of a SoC (System-on-a-Chip) and a SiP (System-in-a-Package) in which one or more processors are configured to work together. The detection unit 4 may include a memory unit that stores various information or programs for operating each component of the floating image display device 1. The memory unit may be configured, for example, with a semiconductor memory. The memory unit may function as a work memory for the detection unit 4.

[0043] The detection unit 4 detects the movement, surface shape, etc. of the object O based on the captured image of the reflected image R. The detection unit 4 can detect the movement of the object O and the surface shape of the object O based on the modulation transfer function M1 of the displayed image D and the modulation transfer function M2 of the reflected image R. The modulation transfer function M1 of the displayed image D is also referred to as a first modulation transfer function, and the modulation transfer function M2 of the reflected image R is also referred to as a second modulation transfer function.

[0044] The modulation transfer function (MTF) is generally a performance index for evaluating the resolution of an optical system. The modulation transfer function can be used as a resolution index for comparing the high and low resolution of images formed by an optical system. The modulation transfer function is a function of spatial frequency (1 / mm). The modulation transfer function may be maximum when the spatial frequency is 0 / mm. The modulation transfer function may be normalized so that the maximum value is 1. In the following description, the modulation transfer function is also referred to as the MTF.

[0045] The MTFM1 of the display image D can be measured, for example, based on an image captured by an imaging device such as a camera of the display surface 2a displaying the display image D. The distance between the display surface 2a and the imaging device may be set to the optimum viewing distance d (see FIGS. 1 and 2).

[0046] The image light L1 of the display image D displayed on the display surface 2a is focused as a floating image F on a virtual image plane Fp by the optical system 3. The MTF of the floating image F can be measured, for example, based on an image of the floating image F captured by an imaging device such as a camera.

[0047] The MTF value of the floating image F may be equal to or less than the MTFM1 value of the display image D at any value of the spatial frequency. That is, the floating image F is based on light (image light L1) that has passed through the optical system 3 once. The larger the ratio of the MTF value of the floating image F to the value of the first modulation transfer function M1 of the display image D, the more the degradation in resolution of the floating image F compared to the display image D is suppressed, and it can be said that the resolution of the floating image F formed on the virtual image plane Fp is higher.

[0048] The reflected light L2 of the reflected image R of the object O in contact with the floating image F is focused on the detection unit 4 by the optical system 3. The second MTFM2 of the reflected image R can be measured, for example, based on a captured image of the reflected image R captured by the detection unit 4. The value of the second MTFM2 may be equal to or less than the value of the first MTFM1 at any value of spatial frequency. This is because the reflected image R is based on light (image light L1, reflected light L2) that has passed through the optical system 3 twice. The larger the ratio of the value of the second MTFM2 to the value of the first MTFM1, the more suppressed is the degradation in resolution of the captured image of the reflected image R compared to the displayed image D, and the higher the resolution of the captured image of the reflected image R. Furthermore, the larger the ratio of the value of the second MTFM2 to the value of the first MTFM1, the closer the object O is to the floating image F (virtual image plane).

[0049] FIG. 6 is a graph illustrating the relationship between the first MTFM1 (solid line) of the displayed image D and the second MTFM2 (dashed line) of the reflected image R and spatial frequency. For comparison, FIG. 6 also shows the relationship between the MTF of the reflected image in the aerial image display device of Conventional Example 1 (dashed line) and the MTF of the reflected image in the aerial image display device of Conventional Example 2 (dashed line), and spatial frequency. The aerial image display device of Conventional Example 1 is an aerial image display device that uses an aerial image forming element (product name "ASKA3D" manufactured by Asukanet Co., Ltd.) that is a lattice-shaped reflecting element. The aerial image display device of Conventional Example 2 is an aerial image display device that uses an aerial image forming element including a beam splitter and a retroreflector. As shown in FIG. 6, the value of the second MTFM2 is greater than the MTF values ​​of Conventional Examples 1 and 2 at all spatial frequencies in the range of 0 / mm to 3 / mm. In other words, the floating image display device 1 can capture a reflected image R with higher resolution than the aerial image display devices of Conventional Examples 1 and 2.

[0050] As shown in FIG. 6, in the floating image display device 1 of another embodiment, when the spatial frequency is λ, the value of the modulation transfer function of the displayed image D is M1, and the value of the modulation transfer function of the reflected image R is M2 (where M1 and M2 are values ​​normalized to 1 as the maximum value), M1 is 0.9 or more and 1 or less when λ is greater than 0 / mm and 3 / mm or less, and M2 is 0.047λ when λ is greater than 0 / mm and 3 / mm or less. 3 -0.218λ 2 -0.020λ+1<M2≦1. 3 -0.218λ 2The polynomial (cubic equation) expressed by -0.020λ+1 represents the curve of Conventional Example 1 in FIG. 6. This curve is a cubic curve that passes through the points where (λ, M) (where M is the modulation transfer function) are (0, 1), (0.5, 0.96), (1, 0.8), (2, 0.47), and (3, 0.24). The M2 value in the floating image display device 1 of this embodiment is higher than that of Conventional Example 1. This configuration allows the reflected image R of an object that overlaps with the virtual imaging plane Fp of the floating image F in the optical axis direction Fa of the floating image F to be clearly detected with high resolution, thereby enabling the gesture and authentication information of the user 5 to be detected with high accuracy. Furthermore, since the reflected image R of the object is detected via the optical system 3, the reflected image R of the object viewed from the front can be detected. As a result, when the object is the fingers 5f of the user 5, the gesture and the type of gesture of the fingers 5f of the user 5 can be accurately detected.

[0051] Furthermore, M2 is 0.058λ when λ is greater than 0 / mm and 3 / mm or less. 3 -0.273λ 2 +0.064λ+1≦M2≦1. 3 -0.273λ 2 The polynomial (cubic expression) expressed by +0.064λ+1 is a curve that follows the curve of Conventional Example 1 in FIG. 6 and represents a curve with a higher M value than the curve of Conventional Example 1. This curve is a cubic curve that passes through points where (λ, M) are (0, 1), (0.5, 0.97), (1, 0.85), (2, 0.5), and (3, 0.3). M2 in the floating image display device 1 of this embodiment is configured to have a higher value than Conventional Example 1. This configuration improves the above-mentioned effects.

[0052] In the floating image display device 1 of this embodiment, λ may be 0 / mm to 3 / mm and M2 may be 0.3 or greater, and particularly λ may be 3 / mm and M2 may be 0.3 or greater. In this case, when an edge extraction image Q' of object O as shown in FIG. 12 is generated, it becomes easy to generate a clear edge (contour) extraction image Q' of object O. Conversely, when λ is 0 / mm to 3 / mm and less than 0.3, the edge extraction image Q' of object O may become unclear or may disappear. For example, as shown in FIG. 12, if object O is a finger and the tip of the finger becomes unclear or disappears, this portion corresponds to a portion where M2 is less than 0.3. Furthermore, the floating image display device 1 of this embodiment also achieves the same effect as described above when configured such that, when λ is 0 / mm to 3 / mm, the M1 value of the display image D is 0.9 or greater and the M2 value of the reflected image R is 0.3 or greater.

[0053] In the floating image display device 1 of this embodiment, M2 may be configured such that λ is 1 / mm or more and less than 1.5 / mm and 0.85 or more, λ is 1.5 / mm or more and less than 2 / mm and 0.65 or more, λ is 2 / mm or more and less than 2.5 / mm and 0.5 or more, or λ is 2.5 / mm or more and 3 / mm and 0.35 or more. In this case, the same effect as above is improved.

[0054] In the floating image display device 1 of this embodiment, the detection unit 4 may be configured to detect the movement of the object O based on the reflected image R. The object O may be a stick, pen, etc. held by the user 5, a glove, etc. covering the hand of the user 5, or the head, mouth, eyes, etc. of the user 5. The object O may also be a flying object such as a pedestrian, luggage (for example, a passenger's luggage at an airport), or an insect.

[0055] The object O may be the fingers 5 f of the user 5, and the detection unit 4 may be configured to detect a gesture of the fingers 5 f based on the reflected image R. Gestures include dragging, swiping, tapping, double tapping, multi-tapping, flicking, pinching out, pinching in, rotation, etc. Gestures may also include eye movement, blinking of the eyelids, movement of the mouth, etc.

[0056] The detection unit 4 may be configured to detect the surface shape of the object O based on the reflected image R. The surface shape may be the unevenness of the surface of the object O, or a shadow pattern caused by the unevenness. The surface shape may be a grayscale pattern formed on the surface of the object O, for example, a code such as a QR code (registered trademark) or a barcode held by the user so as to come into contact with the floating image F. The surface shape of the object O may be a fingerprint of the finger 5f, a palm print (a linear pattern on the palm, also called palm lines), a vascular pattern such as veins on the back of the hand, the shape of the nails, the shape of the eyelids (single eyelid, double eyelid, etc.), the iris of the eye, the distance between the eyes, the shape of the hair, the shape of the nose, the shape of the ears, etc.

[0057] The object O may be a finger of the user 5, and the detection unit may be configured to detect a fingerprint of the finger when the M2 value of the reflected image R is 0.63 (70% of 0.9) or greater. In this case, the object O (finger 5f) is located within the virtual image plane Fp of the floating image F, i.e., at position "II" shown in FIG. 13 . Because the reflected image R of the finger 5f is formed on the detection unit 4 with high resolution, the fingerprint of the finger 5f can be detected based on the captured image Q of the reflected image R. The detection unit 4 may detect a fingerprint, palm print, etc. of the finger when the M2 value of the reflected image R is 0.72 (80% of 0.9) or greater. In this case, the detection unit 4 can more accurately authenticate the user 5 based on the detected fingerprint, palm print, etc. The authentication of the user 5 may be performed using a portion of the fingerprint, etc. of the finger 5f of the user 5. For example, the user 5 may be authenticated using the center of the fingerprint, the upper half, the lower half, the left half, the right half, etc. Alternatively, for example, authentication may be performed by combining a part of the fingerprint with at least one of the linear pattern on the palm, the vascular pattern such as veins on the back of the hand, the shape of the nails, the iris, etc.

[0058] The method for measuring the first MTFM1 and the second MTFM2 will be described.

[0059] (Measurement of First MTFM1) A method for measuring the first MTFM1 will be described. The first MTFM1 may be measured during the manufacturing process of the floating image display device 1.

[0060] A test pattern 7 for measuring the first MTFM1 is displayed on the display surface 2a of the display unit 2. The test pattern 7 may be a repeated pattern of a first band-shaped image 7a and a second band-shaped image 7b, as shown in FIG. 7 . The test pattern 7 may include three or more first band-shaped images 7a. While FIG. 7 shows an example in which the first band-shaped image 7a is a white image and the second band-shaped image 7b is a black image, this is not limiting. The first band-shaped image 7a and the second band-shaped image 7b may differ in at least one of brightness and color.

[0061] Next, an imaging device such as a camera is used to capture an image D of the display surface 2a displaying the test pattern 7. The imaging device is placed at a position separated from the display surface 2a by an appropriate viewing distance d (see FIGS. 1 and 2) in the optical axis direction of the display surface 2a. The imaging device captures the display image D in the optical axis direction of the display surface 2a. The aperture value of the imaging device may be set to 3 or less. The focal length of the imaging device may be set to the appropriate viewing distance d.

[0062] When the first band-shaped image 7a and the second band-shaped image 7b of the test pattern 7 are elongated images extending in a predetermined direction, the imaging device may be tilted by approximately 3 to 5 degrees around a rotation axis parallel to the imaging direction so that the up-down direction of the imaging device (for example, the direction perpendicular to the top surface of the housing of the imaging device) is not parallel to the predetermined direction. In this case, the number of samples taken when measuring the first MTFM1 can be increased, enabling the first MTFM1 to be measured with high accuracy.

[0063] FIG. 8 shows an example of an image captured when the imaging device captures the entire test pattern 7. The imaging device does not need to capture the entire test pattern 7. The imaging device only needs to capture at least one portion (imaging portion) of the test pattern 7. The at least one imaging portion may be a portion corresponding to imaging portion F4 shown in FIG. 8, or may be any of imaging portions F1 to F7. Imaging portion F4 is located near the center of the test pattern 7 when viewed along the imaging direction of the imaging device. Imaging portions F3 to F5 may be aligned in a predetermined direction when viewed along the imaging direction of the imaging device. Imaging portions F1, F2, F6, and F7 may be located at the four corners of the test pattern 7, respectively. The following describes a case where the imaging device captures imaging portion F4.

[0064] The imaging device captures an image of an imaging portion F4 of the test pattern 7 and outputs image data of the captured image P4. Note that the term "captured image P4" refers to an image captured of the imaging portion F4, and the same applies to the captured images P1-P3 and P5-P7. The test pattern may be configured such that each of the captured images P1-P7 includes at least one first band-shaped image 7a. The test pattern may be configured such that each of the captured images P1-P7 includes one first band-shaped image 7a. In this case, processing of the image data of each captured image P1-P7 is simplified, and the first MTFM1 can be calculated with high accuracy. Each of the captured images P1-P7 may have approximately 50 to 100 vertical pixels and approximately 50 to 100 horizontal pixels.

[0065] By scanning the luminance values ​​of each pixel arranged in the horizontal direction (left-right direction in FIG. 9 ) of the captured image P4 and performing a binning process, a luminance distribution waveform LSF as shown in FIG. 10 can be calculated. The luminance distribution waveform LSF is a pulse-like waveform that represents changes in luminance depending on the position in the horizontal direction. In FIG. 10 , the horizontal axis represents the position in the horizontal direction, and the vertical axis represents luminance. The luminance distribution waveform is also referred to as a line spread function (LSF). When the captured image P4 includes an image of one first band-shaped image 7a (see FIG. 9 ), the line spread function LSF of the captured image P4 may have a substantially Gaussian shape, as shown in FIG. 10 .

[0066] The first MTFM1 can be calculated by Fourier transforming the line spread function LSF of the captured image P4 according to the following equation (1).

[0067]

[0068] In equation (1), LSF(x) represents the line spread function as a function of position x in the captured image P4, MTF(ν) represents the MTF as a function of spatial frequency ν, and C is a constant for normalizing MTF(0) to “1”.

[0069] (Measurement of the Second MTFM2) The measurement of the second MTFM2 will be described. The second MTFM2 can be measured based on the captured image of the reflected image R captured by the detection unit 4 when the floating image display device 1 is used by the user 5.

[0070] The detection unit 4 captures the reflected image R and obtains a captured image Q as shown in FIG. 11 . The detection unit 4 applies a differential filter to the image data of the captured image Q to generate an edge-extracted image Q′ as shown in FIG. 12 . The edge-extracted image Q′ is an image in which the edges (contours) of the object O are extracted from the captured image Q. The differential filter may be a first-order differential filter (gradient filter). The differential filter may be, for example, a Roberts filter, a Prewitt filter, a Sobel filter, or the like. The detection unit 4 extracts from the edge-extracted image Q′ an area (edge ​​area) consisting of a low-brightness image (black image) and a single strip of a high-brightness image (white image), similar to the captured image P4 shown in FIG. 9 . The detection unit 4 scans the brightness values ​​of each pixel aligned horizontally in the edge area, performs binning processing, and measures the brightness distribution waveform LSF of the captured image Q. The detector 4 can measure the second MTFM2 by Fourier transforming the line spread function LSF of the captured image Q as shown in equation (1).

[0071] When the detection unit 4 extracts an area (edge ​​area) consisting of a low-brightness image (black image) and a single strip-shaped high-brightness image (white image) from the edge-extracted image Q', the high-brightness image does not have to be linear, but may be curved. The curved high-brightness image may be approximated to a curve (also called an edge curve) expressed by a quadratic to quintic equation. For example, the quadratic equation is y = a0 + a1x + a2x 2 (y is the position of the pixel on the edge curve, x is the coordinate in the row direction (horizontal direction), and a0, a1, and a2 are constants.) For example, a quintic equation is expressed as y = a0 + a1x + a2x 2 +a3x 3 +a4x 4 +a5x 5 (a0, a1, ..., a5 are constants) Binning processing may be performed based on these edge curves to measure the luminance distribution waveform LSF of the captured image Q.

[0072] In the floating image display device 1 of this embodiment, when the value of the first MTFM1 of the display image D at a predetermined spatial frequency (1 / mm) is M1 and the value of the MTFM2 of the reflected image R is M2, the ratio M2 / M1 is 0.3 or more, and when an edge extraction image Q' of the object O as shown in FIG. 12 is generated, it is easy to generate a clear edge (contour) extraction image Q' of the object O. Conversely, when the ratio M2 / M1 is less than 0.3, the edge extraction image Q' of the object O may become unclear or may disappear. For example, as shown in FIG. 12, when the object O is a finger and the tip of the finger becomes unclear or disappears, such a portion corresponds to a portion where the ratio M2 / M1 is less than 0.3. Furthermore, the floating image display device 1 of this embodiment also achieves the same effect as above when the value of the first MTF of the display image D is 0.9 or more and the value of the second MTF of the reflected image R is 0.3 or more when the spatial frequency is 0 / mm or more and 3 / mm or less.

[0073] If the captured image Q of the reflection image R is a color image, the captured image Q may be grayscaled, and the contour of the object O may be extracted from the grayscaled captured image Q. A known method may be used for grayscaling. Grayscaling may involve converting the red (R), green (G), and blue (B) color components of each pixel in the color image into the Y component of the YCbCr color space. Grayscaling may also involve converting the R, G, and B color components of each pixel in the color image into a hue (H). In this case, the influence of external light on the grayscaled captured image Q can be reduced. Furthermore, if the object O is the fingers 5f of the user 5, the color (skin color) of the fingers 5f can be emphasized, making it easier to extract the edges of the fingers 5f. The object O may be a stick, pen, or the like held by the user 5, a glove or the like covering the hand of the user 5, or the head, mouth, eyes, or the like of the user 5. The object O may also be a pedestrian, luggage (e.g., a passenger's luggage at an airport), or a flying object such as an insect.

[0074] In the floating image display device 1, when the value of the first MTFM1 of the display image D at a predetermined spatial frequency (1 / mm) is M1 and the value of the MTFM2 of the reflected image R is M2, the ratio M2 / M1 is 0.3 or more. In the floating image display device 1, the optical system 3 includes only reflective members such as concave mirrors and convex mirrors, so that degradation of the resolution of the reflected image R due to the installation of optical elements utilizing separation, retroreflection, polarization, diffraction, refraction, etc. of the reflected light L2 can be suppressed, and the ratio M2 / M1 can be made 0.3 or more. Furthermore, if the ratio M2 / M1 is 0.3 or more, the movement, surface shape, etc. of the object O in contact with the floating image F can be accurately detected based on the captured image Q of the reflected image R. As a result, the gesture and authentication information of the user 5 can be accurately detected.

[0075] For the purpose of detecting the movement, surface shape, etc. of the object O in contact with the floating image F with higher accuracy, the ratio M2 / M1 may be 0.4 or more, 0.7 or more, 0.8 or more, or 0.85 or more.

[0076] The detection unit 4 may change the operation of the floating image display device 1 in response to the captured image Q of the reflected image R based on the ratio M2 / M1 at a predetermined spatial frequency fs (1 / mm). The spatial frequency fs may be a spatial frequency selected from the range of 0 / mm or more and 3 / mm or less. The range of spatial frequencies of 0 / mm or more and 3 / mm or less is a region with a high MTF value, and is suitable for comparing multiple MTF values. By changing the operation in response to the captured image Q of the reflected image R based on the ratio M2 / M1 at the spatial frequency fs, it becomes possible to accurately detect the gesture and authentication information of the user 5.

[0077] The detection unit 4 may detect the movement of the object O based on the captured image Q of the reflected image R when the ratio M2 / M1 at the spatial frequency fs is 0.3 or greater. When the ratio M2 / M1 at the spatial frequency fs is 0.3 or greater, the movement of the object O can be detected with high accuracy. The detection unit 4 may detect the movement of the object O based on the captured image Q of the reflected image R when the ratio M2 / M1 at the spatial frequency fs is 0.4 or greater. In this case, the movement of the object O can be detected with higher accuracy.

[0078] The detection unit 4 may capture the reflected image R at a predetermined frame rate. The predetermined frame rate may be, for example, 30 to 120 fps (frames per second). The detection unit 4 may detect at least one feature point of the object O from each captured image Q and track the at least one feature point to detect the movement of the object O. The at least one feature point may be the tip of a finger, the base of a finger, or the like. The detection unit 4 may use a technique such as pattern matching to detect the at least one feature point. The detection unit 4 may detect the movement of the object O by calculating the difference between two temporally consecutive captured images Q.

[0079] The detection unit 4 may change the display image D according to the movement of the detected object O. The detection unit 4 may change the display image D according to the movement of the detected object O so that the floating image F moves, rotates, enlarges, or reduces. When the object O is the fingers 5 f of the user 5, the detection unit 4 may detect a gesture of the fingers 5 f based on the captured image Q of the reflected image R. The detection unit 4 may determine whether the gesture of the fingers 5 f of the user 5 is a swipe, pinch out, pinch in, or the like based on the captured image Q of the reflected image R. The detection unit 4 may change the display image D based on the determined gesture of the fingers 5 f so that the floating image F moves, rotates, enlarges, or reduces.

[0080] The detection unit 4 may detect the surface shape of the object O based on the captured image Q of the reflected image R when the ratio M2 / M1 at the spatial frequency fs is 0.3 or greater. The surface shape may be the unevenness of the surface of the object O, or a shadow pattern caused by the unevenness. The surface shape may also be a grayscale pattern formed on the surface of the object O, for example, a code such as a QR code (registered trademark) or a barcode that is held by the user so as to come into contact with the floating image F. The detection unit 4 may detect the surface shape of the object O based on the captured image Q of the reflected image R when the ratio M2 / M1 at the spatial frequency fs is 0.4 or greater. In this case, the surface shape of the object O can be detected with greater accuracy.

[0081] The detection unit 4 may detect the fingerprint, palm print, etc. of the user 5 when the ratio M2 / M1 at the spatial frequency fs is 0.7 or greater. The detection unit 4 may authenticate the user 5 based on the detected fingerprint, palm print, etc. When the ratio M2 / M1 at the spatial frequency fs is 0.7 or greater, the object O (finger 5f) is located within the virtual imaging plane Fp of the floating image F, i.e., at the position "II" shown in FIG. 13. Because the reflected image R of the finger 5f is formed on the detection unit 4 with high resolution, the fingerprint of the finger 5f can be detected based on the captured image Q of the reflected image R. The floating image display device 1 may be configured to allow the authenticated user 5 to view the floating image F including the user 5's personal information (e.g., age, address, nationality, race, job description, health information, asset information, etc.). The floating image display device 1 may be configured to emit a voice to confirm the name of the authenticated user 5, a display such as an arrow to guide the operation, a bright spot, a flashing part, a voice, or the like.

[0082] Furthermore, the detection unit 4 may detect a fingerprint, palm print, etc. of the user 5 when the ratio M2 / M1 at the spatial frequency fs is 0.8 or greater. In this case, the detection unit 4 can more accurately authenticate the user 5 based on the detected fingerprint, palm print, etc. Furthermore, the detection unit 4 may detect a fingerprint, palm print, etc. of the user 5 when the ratio M2 / M1 at the spatial frequency fs is 0.85 or greater. In this case, the detection unit 4 can more accurately authenticate the user 5 based on the detected fingerprint, palm print, etc.

[0083] In order to improve the accuracy of authentication, the detection unit 4 may perform a similarity determination using artificial intelligence (AI) program software. The similarity determination may be, for example, determining to what extent the fingerprint of the user's finger matches data of the fingerprint of the user's finger that has been registered in advance.

[0084] When the ratio M2 / M1 at the spatial frequency fs is less than 0.3, the floating image display device 1 does not need to perform operations such as detecting the movement of the reflected image R relative to the captured image Q or detecting the surface shape. When the ratio M2 / M1 at the spatial frequency fs is less than 0.3, the object O (finger 5f) is located away from the virtual imaging plane Fp of the floating image F in the traveling direction of the image light Lp, that is, at the position "I" or "III" shown in Fig. 13. When the ratio M2 / M1 at the spatial frequency fs is less than 0.3, by not performing operations relative to the captured image Q of the reflected image R, the risk of the floating image display device 1 malfunctioning can be reduced.

[0085] Next, a floating image display device according to another embodiment of the present disclosure will be described. The floating image display device 1A of this embodiment is different from the floating image display device 1 of the above embodiment in the operation of the detection unit 4, but has the same configuration as the floating image display device 1 of the above embodiment, so detailed description of the same configuration will be omitted.

[0086] 1 and 2, the floating image display device 1A of this embodiment includes a display unit 2, an optical system 3, and a detection unit 4. The display unit 2, the optical system 3, and the detection unit 4 have the same configurations as the display unit 2, the optical system 3, and the detection unit 4 of the floating image display device 1, respectively. The detection unit 4 may be integrated with the display unit 2 as shown in FIGS. 1 and 2, or may not be integrated with the display unit 2 as shown in FIG. 5.

[0087] The floating image display device 1A can detect the movement of the object O and the surface shape of the object O based on the area obtained by integrating the first MTFM1 with respect to spatial frequency and the area obtained by integrating the second MTFM2 with respect to spatial frequency. The area obtained by integrating the MTF with respect to spatial frequency is also called the MTF area. The MTF area S can be calculated by integrating the MTF on the spatial frequency axis according to the following equation (2):

[0088]

[0089] In equation (2), the integral interval is set to 0 / mm to ∞ / mm, but the integral interval may be set to approximately 0 / mm to 15 / mm. When the spatial frequency exceeds approximately 15 / mm, the MTF value is close to 0 in most cases, so approximately 15 / mm may be set as the substantial upper limit of the integral interval. The integral interval may also be set to approximately 0 / mm to 12 / mm. When the spatial frequency exceeds approximately 12 / mm, the MTF value is almost always 0.1 or less, so approximately 12 / mm may be set as the substantial upper limit of the integral interval. The integral interval may also be set to approximately 0 / mm to 3 / mm. In this case, the integral interval becomes a high interval where the MTF value is close to 1.0, and can be suitably used for comparing multiple MTFs.

[0090] In the floating image display device 1A, when the MTF area of ​​the first MTF model 1 is S1 and the MTF area of ​​the second MTF model 2 is S2, the ratio S2 / S1 is 0.3 or greater. Because the optical system 3 of the floating image display device 1A includes only reflective elements such as concave mirrors and convex mirrors, degradation of the resolution of the reflected image R due to the installation of optical elements utilizing separation, retroreflection, polarization, diffraction, refraction, etc. of the reflected light L2 can be suppressed, and the ratio S2 / S1 can be set to 0.3 or greater. Furthermore, if the ratio S2 / S1 is 0.3 or greater, the movement, surface shape, etc. of the object O in contact with the floating image F can be accurately detected based on the captured image Q of the reflected image R. As a result, the gestures and authentication information of the user 5 can be accurately detected. Furthermore, because the MTF area is not easily affected by external light, the floating image display device 1A can accurately detect the gestures and authentication information of the user 5 even when used in an environment with a high amount of external light.

[0091] For the purpose of detecting the movement, surface shape, etc. of the object O in contact with the floating image F more accurately based on the captured image Q of the reflected image R, the ratio S2 / S1 may be 0.4 or more, 0.7 or more, 0.8 or more, or 0.85 or more.

[0092] The detection unit 4 may change the operation of the floating image display device 1 in response to the captured image Q of the reflected image R based on the ratio S2 / S1. By changing the operation according to the captured image Q of the reflected image R based on the ratio S2 / S1, it becomes possible to accurately detect the gesture and authentication information of the user 5.

[0093] The detection unit 4 may detect the movement of the object O based on the captured image Q of the reflected image R when the ratio S2 / S1 is 0.3 or more. When the ratio S2 / S1 is 0.3 or more, the movement of the object O can be detected with high accuracy. When the ratio S2 / S1 is 0.4 or more, the detection unit 4 may detect the movement of the object O based on the captured image Q of the reflected image R. In this case, the movement of the object O can be detected with higher accuracy.

[0094] The detection unit 4 may capture the reflected image R at a predetermined frame rate. The predetermined frame rate may be, for example, 30 to 120 fps. The detection unit 4 may detect at least one feature point of the object O from each captured image Q and track the at least one feature point to detect the movement of the object O. The at least one feature point may be the tip of a finger, the base of a finger, or the like. The detection unit 4 may use a technique such as pattern matching to detect the at least one feature point. The detection unit 4 may detect the movement of the object O by calculating the difference between two temporally consecutive captured images Q.

[0095] The detection unit 4 may change the display image D according to the movement of the detected object O. The detection unit 4 may change the display image D according to the movement of the detected object O so that the floating image F moves, rotates, enlarges, or reduces. When the object O is the fingers 5 f of the user 5, the detection unit 4 may detect a gesture of the fingers 5 f based on the captured image Q of the reflected image R. The detection unit 4 may determine whether the gesture of the fingers 5 f of the user 5 is a swipe, pinch out, pinch in, or the like based on the captured image Q of the reflected image R. The detection unit 4 may change the display image D based on the determined gesture of the fingers 5 f so that the floating image F moves, rotates, enlarges, or reduces.

[0096] When the ratio S2 / S1 is 0.3 or greater, the detection unit 4 may detect the surface shape of the object O based on the captured image Q of the reflected image R. The surface shape may be the unevenness of the surface of the object O, or a shadow pattern caused by the unevenness. The surface shape may also be a grayscale pattern formed on the surface of the object O, for example, a code such as a QR code (registered trademark) or a barcode that is held by the user so as to come into contact with the floating image F. When the ratio S2 / S1 is 0.4 or greater, the detection unit 4 may detect the surface shape of the object O based on the captured image Q of the reflected image R. In this case, the surface shape of the object O can be detected with greater accuracy.

[0097] The detection unit 4 may detect the fingerprint, palm print, etc. of the user 5 when the ratio S2 / S1 is 0.7 or greater. The detection unit 4 may authenticate the user 5 based on the detected fingerprint, palm print, etc. When the ratio S2 / S1 is 0.7 or greater, the object O (finger 5f) is located within the virtual imaging plane Fp of the floating image F, i.e., at the position "II" shown in FIG. 13. Because the reflected image R of the finger 5f is formed on the detection unit 4 with high resolution, the fingerprint of the finger 5f can be detected based on the captured image Q of the reflected image R. The floating image display device 1 may be configured to allow the authenticated user 5 to visually recognize a floating image F including the user 5's personal information (e.g., age, address, nationality, race, job description, health information, asset information, etc.). The floating image display device 1 may be configured to emit a voice confirming the authenticated user 5's name, a display such as an arrow guiding the operation, a bright spot, a flashing display, a voice, etc.

[0098] In order to improve the accuracy of authentication, the detection unit 4 may perform a similarity determination using AI program software. The similarity determination may be, for example, determining to what extent the fingerprint of the user's finger 5f matches data of the user's finger fingerprint that has been registered in advance.

[0099] In addition to fingerprints of the fingers 5f, authentication of the user 5 may be performed using palm prints (linear patterns on the palm, also called palm lines), vascular patterns such as veins on the back of the hand, nail shapes, irises, eyelid shapes (single eyelid, double eyelid, etc.), the distance between the eyes, hair shapes, nose shapes, ear shapes, etc. Authentication of the user 5 may also be performed using a combination of fingerprints of the fingers 5f, linear patterns on the palm, vascular patterns such as veins on the back of the hand, nail shapes, irises, eyelid shapes, the distance between the eyes, hair shapes, nose shapes, ear shapes, etc.

[0100] Authentication of user 5 may also be performed using a part of the fingerprint or the like of a finger 5f of user 5. For example, authentication of user 5 may be performed using the center of the fingerprint, the upper half, the lower half, the left half, the right half, etc. Authentication may also be performed by combining a part of the fingerprint with at least one of the linear pattern on the palm, the vascular pattern such as veins on the back of the hand, the shape of the nails, the iris, etc.

[0101] When the ratio S2 / S1 is less than 0.3, the floating image display device 1 does not need to perform operations such as detecting the movement of the reflected image R relative to the captured image Q or detecting the surface shape. When the ratio S2 / S1 is less than 0.3, the object O (finger 5f) is located away from the virtual imaging plane Fp of the floating image F in the traveling direction of the image light Lp, that is, at the position "I" or "III" shown in Fig. 13. When the ratio S2 / S1 is less than 0.3, by not performing operations relative to the captured image Q of the reflected image R, the risk of the floating image display device 1 malfunctioning can be reduced.

[0102] The floating image display device 1, 1A may be configured such that, when the ratio M2 / M1 (ratio S2 / S1) becomes equal to or greater than a predetermined threshold T1, the detection unit 4 notifies the user that the ratio M2 / M1 (ratio S2 / S1) is equal to or greater than the threshold T1. The threshold T1 may be, for example, 0.7. In this case, the user 5 can be notified that the finger 5f of the user 5 is substantially located within the virtual imaging plane Fp and the detection unit 4 can detect the fingerprint of the finger 5f, thereby improving the convenience of the user 5. The floating image display device 1 may display, as a notification function, characters such as "Authentication Start," "Authentication START," or "Authentication OK," as well as bright spots, blinking areas, etc. on the display unit 2. The floating image display device 1 may also have, as a notification function, a sound generator 8 that generates sound. The floating image display device 1 may also have, as a notification function, an area used as an authentication unit (also referred to as an authentication area) within the floating image F. The authentication area may have a specific shape, such as a circle, an oval, a rectangle, or the shape of a finger. The user 5 may place his / her finger in contact with the authentication area. When the ratio M2 / M1 (ratio S2 / S1) becomes equal to or greater than the threshold T1, the detection unit 4 may notify the user 5 that the floating image display device 1, 1A can detect the fingerprint of the finger 5f by generating a sound from the sound generator.

[0103] The detection unit 4 may divide the captured image of the test pattern 7 into a plurality of regions A1 to A9 in advance, as shown in FIG. 14A , measure the first MTFM1 for the plurality of regions A1 to A9, and store the results. The detection unit 4 may divide the captured image Q of the reflection image R into a plurality of regions B1 to B9, each corresponding to the plurality of regions A1 to A9, as shown in FIG. 14B , and measure the second MTFM2 for each of the plurality of regions B1 to B9. The detection unit 4 may calculate a ratio M2 / M1 of the value M2 of the second MTFM2 for region Bk (k=j) to the value M1 of the first MTFM1 for region Aj (j=1, ..., 9), and count the number of ratios M2 / M1 that are equal to or greater than a predetermined threshold T2 among the nine ratios M2 / M1. The value M1 of the first MTFM1 and the value M2 of the second modulation transfer function M2 are values ​​at a spatial frequency fs selected from a range of 0 / mm to 3 / mm. The threshold T2 may be 0.3 or 0.7.

[0104] For example, when the threshold value T2 is set to 0.3, if the number of ratios M2 / M1 that are equal to or greater than the threshold value T2 is two, it can be determined that there are two fingers 5f near or in contact with the floating image F. When the fingers 5f are near the floating image F, it can be determined that the fingers 5f are within a range of about 1 to 2 times the focal depth of the image light L1, or within a range of about 1 to 1.5 times the focal depth. Furthermore, it can be determined that the gesture of the user 5 with respect to the floating image F is a pinch out or a pinch in.

[0105] Furthermore, if the number of ratios M2 / M1 that are equal to or greater than the threshold T2 is one, it can be determined that one finger 5f is in contact with the floating image F. Furthermore, it can be determined that the gesture of the user 5 on the floating image F is a swipe or a tap. In the above, the captured image of the test pattern 7 and the captured image Q of the reflected image R are each divided into nine regions, but the number of divided regions may be set appropriately. The above determination can also be made using the ratio S2 / S1 instead of the ratio M2 / M1.

[0106] 1 and 2, the floating image display device 1, 1A may include a housing 10. The display unit 2, optical system 3, and detection unit 4 of the floating image display device 1, 1A may be arranged inside the housing 10. The floating image display device 1, 1A may include a circuit board, wiring, cables, heat dissipation members (e.g., heat sinks, etc.), holding members for holding the optical system 3, adjustment members for adjusting the position of the optical system 3, etc., arranged inside the housing 10. The housing 10 may be made of, for example, a resin material, a metal material, a ceramic material, etc.

[0107] The housing 10 has an image light emitting unit 10a. The image light emitting unit 10a is located at a portion of the housing 10 facing the user 5. The image light L1 is emitted from inside the housing 10 to outside the housing 10 via the image light emitting unit 10a. The reflected light L2 is incident on the housing 10 from outside the housing 10 via the image light emitting unit 10a. The image light emitting unit 10a may include a light-shielding film or light-shielding glass to make it difficult for the user 5 to view the display unit 2, the optical system 3, the detection unit 4, etc. located inside the housing 10. In this case, the visibility of the floating image F can be improved.

[0108] The present disclosure can be implemented in the following aspects (1) to (17).

[0109] (1) A floating image display device comprising: a display unit; an optical system that focuses the light of a display image displayed on the display unit as a floating image of a real image; and a detection unit located near the display unit that detects, via the optical system, a reflected image of an object that is illuminated by the light and overlaps with a virtual imaging plane of the floating image in the optical axis direction of the floating image, wherein when the value of the modulation transfer function of the display image is M1 and the value of the modulation transfer function of the reflected image is M2, the ratio M2 / M1 is 0.3 or more.

[0110] (2) The floating image display device according to (1) above, wherein M1 and M2 are values ​​at a spatial frequency greater than 0 / mm and equal to or less than 3 / mm.

[0111] (3) The floating image display device according to (1) or (2), wherein the detection unit detects the movement of the object based on the reflected image.

[0112] (4) The floating image display device according to any one of (1) to (3) above, wherein the object is a user's finger, and the detection unit detects a gesture of the finger based on the reflected image.

[0113] (5) The floating image display device according to any one of (1) to (4) above, wherein the detection unit detects the surface shape of the object based on the reflected image.

[0114] (6) The floating image display device according to (5) above, wherein the object is a user's finger, and the detection unit detects a fingerprint of the finger when the ratio M2 / M1 is 0.7 or more.

[0115] (7) A floating image display device comprising: a display unit; an optical system that focuses the light of a display image displayed on the display unit as a floating image of a real image; and a detection unit located near the display unit that detects, via the optical system, a reflected image of an object that is illuminated by the light and overlaps with a virtual imaging plane of the floating image in the optical axis direction of the floating image, wherein when S1 is an area obtained by integrating a modulation transfer function of the display image on a spatial frequency axis and S2 is an area obtained by integrating a modulation transfer function of the reflected image on a spatial frequency axis, the ratio S2 / S1 is 0.3 or more.

[0116] (8) The floating image display device according to (7), wherein the detection unit detects the movement of the object based on the reflected image.

[0117] (9) The floating image display device according to (7) or (8), wherein the object is a user's finger, and the detection unit detects a gesture of the finger based on the reflected image.

[0118] (10) The floating image display device according to any one of (7) to (9) above, wherein the detection unit detects the surface shape of the object based on the reflected image.

[0119] (11) The floating image display device according to (10), wherein the object is a user's finger, and the detection unit detects a fingerprint of the finger when the ratio S2 / S1 is 0.7 or more.

[0120] (12) A display unit, an optical system that forms a floating image of a real image using light from a display image displayed on the display unit, and a detection unit located near the display unit that detects, via the optical system, a reflected image of an object that is illuminated by the light and overlaps with a virtual imaging plane of the floating image in the optical axis direction of the floating image, wherein when the spatial frequency is λ, the value of the modulation transfer function of the display image is M1, and the value of the modulation transfer function of the reflected image is M2 (where M1 and M2 are values ​​normalized to a maximum value of 1), M1 is 0.9 to 1 when λ is greater than 0 / mm and 3 / mm or less, and M2 is 0.047λ when λ is greater than 0 / mm and 3 / mm or less. 3 -0.218λ 2A floating image display device, wherein −0.020λ+1<M2≦1.

[0121] (13) The floating image display device according to (12), wherein M2 is such that λ is 1 / mm or more and less than 1.5 / mm and 0.85 or more, λ is 1.5 / mm or more and less than 2 / mm and 0.65 or more, λ is 2 / mm or more and less than 2.5 / mm and 0.5 or more, and λ is 2.5 / mm or more and 3 / mm and 0.35 or more.

[0122] (14) The floating image display device according to (12) or (13), wherein the detection unit detects the movement of the object based on the reflected image.

[0123] (15) The floating image display device according to (14), wherein the object is a user's finger, and the detection unit detects a gesture of the finger based on the reflected image.

[0124] (16) The floating image display device according to any one of (12) to (15) above, wherein the detection unit detects the surface shape of the object based on the reflected image.

[0125] (17) The floating image display device according to (16), wherein the object is a user's finger, and the detection unit detects a fingerprint of the finger when the value of M2 is 0.63 or more.

[0126] According to the floating image display device of the present disclosure, it is possible to accurately detect the gestures and authentication information of the user.

[0127] Although the embodiments of the present disclosure have been described in detail above, the present disclosure is not limited to the above embodiments. Various modifications, improvements, etc. are possible within the scope of the gist of the present disclosure. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the invention of the present disclosure. For example, functions contained in each component, etc., can be rearranged so as not to cause logical contradictions, and multiple components, etc., can be combined into one or divided. In other words, it should be noted that those skilled in the art can easily make various modifications or alterations based on the present disclosure. It should also be noted that these modifications, alterations, or alterations are included within the scope of the present disclosure.

[0128] DESCRIPTION OF SYMBOLS 1, 1A Floating image display device 2 Display unit 2a Display surface 21 Liquid crystal panel 22 Backlight 23 Substrate 23a First surface 24 Visible light source 25 Infrared light source 26 Substrate 26a Second surface 27 Pixel 27R Red light emitting element 27G Green light emitting element 27B Blue light emitting element 27I Infrared light emitting element 3 Optical system 3a First concave mirror 3b Convex mirror 3c Second concave mirror 3d First concave mirror 3e Second concave mirror 4 Detection unit 41 Detection element 5 User 5f Finger 6 Optical member 7 Test pattern 7a First strip-shaped image 7b Second strip-shaped image 8 Sound generator 10 Housing 10a Image light emitting unit

Claims

1. A floating image display device comprising: a display unit; an optical system that forms the light of a display image displayed on the display unit into a real floating image; and a detection unit located near the display unit, the detection unit detecting, via the optical system, a reflected image of an object that is illuminated by the light and overlaps a virtual imaging plane of the floating image in the optical axis direction of the floating image, wherein when a value of a modulation transfer function of the display image is M1 and a value of a modulation transfer function of the reflected image is M2, a ratio M2 / M1 is 0.3 or more.

2. The floating image display device according to claim 1, wherein M1 and M2 are values at a spatial frequency greater than 0 / mm and equal to or less than 3 / mm.

3. The floating image display device according to claim 1 or 2, wherein the detection unit detects movement of the object based on the reflected image.

4. The floating image display device according to any one of claims 1 to 3, wherein the object is a user's finger, and the detection unit detects a gesture of the finger based on the reflected image.

5. The floating image display device according to any one of claims 1 to 4, wherein the detection unit detects a surface shape of the object based on the reflected image.

6. The floating image display device according to claim 5, wherein the object is a user's finger, and the detection unit detects a fingerprint of the finger when the ratio M2 / M1 is 0.7 or more.

7. A floating image display device comprising: a display unit; an optical system that forms the light of a display image displayed on the display unit into a real floating image; and a detection unit located near the display unit, the detection unit detecting, via the optical system, a reflected image of an object that is illuminated by the light and overlaps a virtual imaging plane of the floating image in the optical axis direction of the floating image, wherein when an area obtained by integrating the modulation transfer function of the display image on a spatial frequency axis is S1 and an area obtained by integrating the modulation transfer function of the reflected image on a spatial frequency axis is S2, a ratio S2 / S1 is 0.3 or more.

8. The floating image display device according to claim 7, wherein the detection unit detects movement of the object based on the reflected image.

9. The floating image display device according to claim 8, wherein the object is a user's finger, and the detection unit detects a gesture of the finger based on the reflected image.

10. The floating image display device according to any one of claims 7 to 9, wherein the detection unit detects a surface shape of the object based on the reflected image.

11. The object is a user's finger, and the detection unit detects the fingerprint of the finger when the ratio S2 / S1 is 0.7 or more. The floating image display device according to claim 10.

12. A display unit, an optical system that forms an image of the light of the display image displayed on the display unit as a floating image of a real image, and a detection unit located near the display unit, which detects, via the optical system, a reflected image of an object illuminated by the light and overlapping the virtual imaging plane of the floating image in the optical axis direction of the floating image. When the spatial frequency is λ, the value of the modulation transfer function of the display image is M1, and the value of the modulation transfer function of the reflected image is M2 (where M1 and M2 are values normalized with a maximum value of 1), M1 is 0.9 or more and 1 or less when λ is greater than 0 / mm and 3 / mm or less, and M2 is 0.047λ 3 -0.218λ 2 -0.020λ + 1 < M2 ≤ 1, a floating image display device.

13. M2 is such that when λ is 1 / mm or more and less than 1.5 / mm, it is 0.85 or more; when λ is 1.5 / mm or more and less than 2 / mm, it is 0.65 or more; when λ is 2 / mm or more and less than 2.5 / mm, it is 0.5 or more; and when λ is 2.5 / mm or more and 3 / mm or less, it is 0.35 or more. The floating image display device according to claim 12.

14. The detection unit detects the movement of the object based on the reflected image. The floating image display device according to claim 12 or 13.

15. The object is a user's finger, and the detection unit detects the gesture of the finger based on the reflected image. The floating image display device according to claim 14.

16. The detection unit detects the surface shape of the object based on the reflected image. The floating image display device according to any one of claims 12 to 15.

17. The object is a user's finger, and the detection unit detects the fingerprint of the finger when the value of M2 is 0.63 or more. The floating image display device according to claim 16.

Images